Update job log live in particular dir


#1

Hello!

As I can understand, after you submit the job, PBS mom daemon create 2 files (output and error) and upload it into work dir after a job is finished.

So my question is: can I set up the PBS in that way when PBS mom daemon will create the file in the selected directory after job submitted and will update it on the spot?


#2

Please refer the below section from https://www.pbsworks.com/pdfs/PBSAdminGuide18.2.pdf
3.3.6 Writing Files Directly to Final Destination
Also, man qsub has information.


#3

PBS ver 14.1.2
I think you misunderstand me. Need to create the file in users directory (NOT IN MoM’s HOST DIR) immediately after job START and write all information to file meanwhile job is running.

And there is no 3.3.6 in Admin Guide))))


#4

This is avaialble in PBS Pro 18.x.x

Sorry, It is in the PBS Pro User guide


Section: 3.3.6 Writing Files Directly to Final Destination

Yes, it will write in the final destination mentioned in the $PBS_HOME/mom_priv/config against the $usecp attribute.


#5

Is it possible to select the path in qsub when you submit the job?


#6

No, we cannot specify the path in the qsub

Marking @nithinj


#7

Interesting. You told me it is impossible, but changelog of ver 18 sad:

New features include:

Who is right?


#8

Please check the PBS Pro 18.2 User Guide

We cannot specify a path in the qsub command line to write stderr/err files to their final destination.

You can use the below to write stderr/out
qsub -o :/path/to/stdout -e :/path/to/stderr (but this will be copied only after job is finished ). which was not your requirement

To directly write to a destination, you need to use the $usecp directive in the mom_priv/config file, which means you are not allowed to use qsub -k :/path/to/stdout


#9

You can chose the path using -o and -e option while submitting a job. But the output and error path should have been mapped using $usecp in mom config file unless the path is part of the mom.