Proposing method for PTL packaging


#1

EDD link: https://pbspro.atlassian.net/wiki/spaces/PD/pages/672006148/PTL+package+design

A standard package for PTL will be useful for all PTL users. As such, I am proposing two approaches for PTL packaging.
The first approach is .whl files as a pip enabled package.
The second one is .rpm packages which is currently being used by PBS to package itself.

Some notable points for each packaging mechanism.

Whl packaging Rpm packaging
* Whl will handle dependencies through requirements.txt file * Rpms will easily be able to handle all types of dependencies
* It is platform dependent * Also platform dependent
* Existing rpm infrastructure present in PBS packaging
* Easier to implement than whl on our current infrastructure
* It will be easier to use by current users as they already use PBS rpms

As it suggests both methods are almost equivalent, except that we already have infrastructue for rpm packages. Contemplating on various pros on said packages I am propsing to go with rpm packaging over whl packaging. Building up on the existing directory structure we will create an additional rpm package for PTL, which will be compatible with the PBSpro package.

Please suggest any moderation/alternate approach if needed.


#2

Thanks for proposing this packaging solution.

My 2¢*

If we are focusing only on Rhel based Linux, its totally fine to go with RPM based solution.

But if we have any plan to provide common solution for Debain, Rhel/SLES and Windows platforms. I suggest to go with any one of Python’s de facto packaging solutions like 1) Wheel , 2) PyPI.


#3

I would prefer an RPM package for PTL. Thanks.


#4

+1 for RPM package for PTL.


#5

@bremanandjk for non RPM based platform we can easily convert RPM to specific package like deb on Debain like how we are doing for PBS RPMs.


#6

@hirenvadalia, ya if we have platform independent code, we can use alien for that purpose.
But for Windows ?


#7

@bremanandjk PTL is not supported on windows yet! :smile:


#8

@hirenvadalia ya that’s true…
My forward thinking got that suggestion! :sunglasses:
As I said above, Only if we have any plan to support windows. we should do for a common solution.


#9

I’ll add another +1 for RPM.


#10

@bremanandjk I totally understand your forward thinking and I am also with you that if possible then definitely we should use common solution for all supported platforms.

And off course we have plan to add support for Windows in PTL.

But here we are taking about *nix vs Windows which is totally separate world that’s the reason we have separate packaging even in PBS itself (RPM/DEB for *nix and MSI for Windows)

Also here are few extra points which says it’s better not to use whl:

  • Whl adds extra dependency of installing pip in system which user/admin has to take care of it, while for RPM/DEB you don’t need any extra packages as its native package manager
  • If you want to keep whl platform independent then you have to generate something calling “Universal WHL” (aka noarch.whl) and universal whl is only possible and good (as far as I know) if you don’t have any C extensions in PTL. Which we don’t have as of now but we never know, in future we might add (for example like C extensions for PBS IFL API for PTL API Mode)
  • About Windows, we still don’t know what we will need on Windows to port PTL on it, there are lots of gray area which we need to fill it up first before we will able run PTL on windows (for example ~2 years back I tried to put PTL on Windows which has PowerShell extensions, C extensions, you can look at that code https://github.com/hirenvadalia/pbspro/tree/windows-ptl) so my point here is why to worry about part which we don’t know yet!

And if we use RPM/DEB right now then in future (Once we have Windows PTL) we can re-evaluate things and move to whl if it wins at that time, it’s a software development world where everything moves based on current and known upcoming future situations. :smile:


#11

I have added EDD link in post for rpm packaging design for PTL.


#12

I suggest you name the package pbspro-ptl-VERSION.x86_64.rpm
If the package does not contain any compiled binaries, then you should change x86_64.rpm to noarch. Automake might actually do that for you.


#13

I like naming convention suggested by Mike.


#14

+1 for this naming convention.


#15

I have changed the nomenclature in the EDD accordingly.


#16

@sandisamp: thanks for the EDD, Can we have specific default location of ptl like /opt/ptl ? Since we are providing ptl as separate rpm.

Let me know your thoughts.


#17

Something to keep in mind… when PBS Pro is built for distribution with SUSE/RHEL we will not be allowed to place anything under /opt. We currently package as an ISV so /opt is the right place to put the files. We’ll have to support both scenarios. Perhaps we could add a new parameter to configure that would enable building the pbspro-ptl package. I suggest it be disabled by default. It could be enabled with something like…
./configure --enable-ptl


#18

+1 for /opt/ptl assuming that PTL RPM will allow user to change installation directory like how PBS RPM does by setting PBS_EXEC environment variable.

+1 for ./configure --enable-ptl


#19

Made recommended chnages to the EDD.:+1:


#20

Looks good. Perhaps you want to mention some dependencies the package will have. Will PBS Pro itself be a prerequisite? These will appear as “Requires” lines in the spec file. Other than that, the design looks fine.