Nonconsumable resource on host


#1

Hi,

I want to ask if there is a support for nonconsumable resources on host with the float and long type.

I need a resource like this:

spec type=float flag=h

I added the resource on the node:

set node torque1 resources_available.spec = 30.6

When I try to run two same jobs (qsub -I -l select=spec=30) within one scheduler cycle, only the first one will start. The second one will start later (in next sched. cycle).

01/23/2017 15:42:24;0040;pbs_sched;Job;10445.torque1.ics.muni.cz;Insufficient amount of resource: spec (R: 30 A: 0.6 T: 30.6)

I can see that the nonconsumable resource is considered as a consumable one, why? See src/scheduler/resource.c:conv_rsc_type()

Thx,
Vasek


#2

Vasek,

I tried to reproduce what you are seeing and I am unable to.

[root@pbs-server centos]# qstat --version
pbs_version = 14.1.0

I create a resource in qmgr

create resource bob type=float,flag=h

I then add the resource bob to the resources line in sched_config

resources: ncpus,mem,host,vnode,vmem,bob

I then restart the pbs server and run some jobs.

Last login: Mon Jan 23 17:22:44 UTC 2017 on pts/1
[centos@pbs-server ~]$ qsub -lselect=1:ncpus=1:bob=30 – /bin/sleep 120
247.pbs-server
[centos@pbs-server ~]$ qsub -lselect=1:ncpus=1:bob=30 – /bin/sleep 120
248.pbs-server
[centos@pbs-server ~]$ qsub -lselect=1:ncpus=1:bob=30 – /bin/sleep 120
249.pbs-server
[centos@pbs-server ~]$ qstat
Job id Name User Time Use S Queue


247.pbs-server STDIN centos 00:00:00 R workq
248.pbs-server STDIN centos 00:00:00 R workq
249.pbs-server STDIN centos 00:00:00 R workq

I believe that this is what your are expecting. Am I missing something?

Jon


#3

Thank you for your reply @jon

Yes, that is what I expect. But I think that each of your job is started within different scheduler cycle…

You need to try to run the jobs within the same scheduler cycle. Do not do that interactively.

Please, try to reproduce this:

  1. stop the pbs_sched (only)
  2. submit two or more jobs requesting bob=30
  3. start the pbs_sched
  4. only first job will start within the first cycle, next job will start later

Vasek


#4

Vasek,

I am now able to reproduce this following your steps. This is not what I would expect. Please file a ticket with the steps to reproduce the issue.

Jon


#5

@vchlum
It is documented in PBS Professional reference guide that only boolean, duration, string and string-array can be used as non-consumable resources.
Please refer to Chapter 5 section 5.2 in the guide on this url- http://www.pbsworks.com/pdfs/PBSReferenceGuide13.0.pdf


#6

Is their a reason you need a float or an int verses just using a string? I believe you will get the expected behavior using a string.


#7

I want to add SPECfp2006 to each node and I want to be able to request the minimal spec. This means that the node with spec=30.6 is suitable for jobs with select=spec=10 or spec=30 or for spec=30.6.
So there is no way how to do that now and it is not a bug?

String doesn’t work. If I use string i get: Insufficient amount of resource: spec (30 != 30.6).


#8

Not sure if this will help me with my issue? Float will still not work?


#9

@vchlum Yes, I think with the proposed design to use conditional operators this might work for you.
With that change in place you can make the resource spec a string and submit jobs with “spec <= 30.6”, scheduler will select all nodes that have their spec specified as 30.6 or less.


#10

Vasek,
What you are seeing is some legacy functionality. Until we added the ability to query the server’s resource types, the scheduler had to derive the type based on the value. Any numeric value was marked as consumable. This means within a cycle, the scheduler will treat such a resource as consumable. You didn’t make the value consumable in the server (flag=hn). This means the next cycle, the scheduler queries the world and doesn’t see any resources_assigned values and starts afresh.

Fixing this isn’t as easy as just querying the type from the server. We make resources consumable in two ways. If it is a static resource, it requires flag=hn. If it is a server_dyn_res resource, it requires flag=h.

This issue can be resolved easy enough when server periodic hooks can modify resources. At that point a hook can serve as the method to update a server_dyn_res resource and all consumable resources can be marked as flag=hn. Nonconsumable resources can be marked as flag=h and be made to work as you desire.

Bhroam


#11

Thank you for answers. I’m curious to PP-507.