Failed to start PBS dataservice-lock seems to be held


#1

I am getting failed to acquire lock problem.
Someone in the forum had a similar problem and they had corrected it.
The suggested solutions are:- 1.To clean /var folder if it is full(My /var folder is only 74% full)
2.Few lines be added in pbspro/sbin/pbs_dataservice to check the ownership of /var/run/postgresql.
if it is different from the database user defined in /var/spool/pbspro/server_priv/db_user, the ownership should be changed accordingly.(I don’t have postgresql directory created in /var/run)
Here is what I am getting-
[root@host1 datastore]# service pbs restart
Restarting PBS
Stopping PBS
PBS mom - was pid: 117648
PBS sched - was pid: 117662
PBS comm - was pid: 117614
Waiting for shutdown to complete
Starting PBS
PBS comm
/opt/pbs/sbin/pbs_comm ready (pid=119554),Threads:4
PBS mom
PBS sched
Connecting to PBS dataservice…Failed to start PBS dataservice:[Failed to acquire lock on /var/spool/pbs/datastore/pbs_dblock. Lock seems to be held by pid: 119947 running on host: host1 cannot start]
.Failed to start PBS dataservice:[Failed to acquire lock on /var/spool/pbs/datastore/pbs_dblock. Lock seems to be held by pid: 119947 running on host: host1 - cannot start]
.Failed to start PBS dataservice:[Failed to acquire lock on /var/spool/pbs/datastore/pbs_dblock. Lock seems to be held by pid: 119947 running on host: host1- cannot start]
.Failed to start PBS dataservice:[Failed to acquire lock on /var/spool/pbs/datastore/pbs_dblock. Lock seems to be held by pid: 119947 running on host: host1 - cannot start]
continuing in background.
PBS server
Please help me.


Failed to start PBS dataservice
#2

Looks like another instance of the database may be running. What is PID 119947?

The following command will show you processes related to PBS…
ps -ef | grep -i pbs

Please make sure all PBS Pro processes are stopped before you try to start the service.


#3

I stopped all pbs processes and then restarted.Everytime some other process with different pid seems to hold the lock.I have the same problem
[root@host1 kbharadw]# sudo /etc/init.d/pbs stop
Stopping PBS
PBS mom - was pid: 1072123
PBS sched - was pid: 1072137
PBS comm - was pid: 1072085
Waiting for shutdown to complete
[root@host1 datastore]# ps -ef | grep -i pbs
root 1073924 3148393 0 10:13 pts/1 00:00:00 grep --color=auto -i pbs
[root@host1 datastore]# sudo /etc/init.d/pbs start
Starting PBS
PBS comm
/opt/pbs/sbin/pbs_comm ready (pid=1074020), Threads:4
PBS mom
PBS sched
Connecting to PBS dataservice…Failed to start PBS dataservice:[Failed to acquire lock on /var/spool/pbs/datastore/pbs_dblock. Lock seems to be held by pid: 1074411 running on host: host1- cannot start]
.Failed to start PBS dataservice:[Failed to acquire lock on /var/spool/pbs/datastore/pbs_dblock. Lock seems to be held by pid: 1074411 running on host: host1 - cannot start]
.Failed to start PBS dataservice:[Failed to acquire lock on /var/spool/pbs/datastore/pbs_dblock. Lock seems to be held by pid: 1074411 running on host: host1 - cannot start]
.Failed to start PBS dataservice:[Failed to acquire lock on /var/spool/pbs/datastore/pbs_dblock. Lock seems to be held by pid: 1074411 running on host: host1 - cannot start]
continuing in background.
PBS server
[root@host1 datastore]#


#4

Please run the following command:

sudo /opt/pbs/sbin/pbs_ds_monitor check

Run it once with PBS running and once with PBS stopped. Please capture the return code (echo $?) and any output produced.


#5

did you try rm /tmp/.s.postgresql.15007*.

if theses file exists, you need to delete it manually as well as pbs_dblock file in $PBS_HOME/databases

Also make sure the permission of /tmp is 1777
hope it helps