Regarding one of the latest changes (adding a default for the --map= option): I suggest putting the default map file at the same level as the snapshot directory (not inside the snapshot directory). I feel including the map file inside the snapshot will lead to the unintended inclusion of unobfuscated data in otherwise cleansed snapshots, as the reasonable behavior of simply making a tar file of the whole snapshot directory will include the unobfuscated map file, rendering the obfuscation moot.
That would be a pretty useful facility. I think pbs_config already has the ability to ingest a pbs_diag output to recreate the environment, so once pbs_snapshot replaces pbs_diag, I think we’ll modify pbs_config as well to support recreating environments using snapshots. I’ve filed a ticket for it here: https://pbspro.atlassian.net/browse/PP-787
I wish I would have commented on this earlier, but I have a slight organizational change to propose. It seems to me that the contents of the top level queue/ and resource/ directories are directly applicable to server/, and so their contents could be moved into that directory and clean up the top level a bit.
I have the same thought about moving reservation/ to scheduler/, but one could also argue that it belongs in server/ which means it is probably correct for it to be separate.
Thanks Scott. I like the idea of moving queue/ and resource/ to server/ to tidy up the top level.
I personally think that reservations should be kept separate, it wouldn’t be obvious to me to look for them under a scheduler/ directory.
I wanted to clarify something about --obfuscate flag, right now ACLs are listed under the items to remove, not obfuscate. I was thinking maybe we should just obfuscate them, not sure why we wanna delete them off. What do you think?
From the troubleshooting aspect of using this tool I don’t think it is a huge deal either way. ACL configurations are self contained, and if the problem at hand is related to ACLs (or manager/operator privileges) it is usually pretty obvious and can be discussed directly. Put another way, ACL configuration is unlikely to be the problematic needle if the pbs_snapshot haystack.
On the other hand, since we are already obfuscating euser and egroup and putting them in the map (which we absolutely need to do), is it really that much more effort to go through the ACLs as well and obfuscate them under the same mapping? Same with group_list, managers, and operators?
Moved ‘hook/’ to mom_priv/hooks/ to again be consistent with PBS_HOME
I don’t think this is correct. The output in the commands listed contains all hooks, server and mom, I think either the top level hooks/ directory should remain with qmgr_ph_default.out and qmgr_lpbshook.out in it, or those two files could be moved to server/ (because mom hooks are configured via the server). I prefer the top level hooks/ directory.
As for the actual hooks directories in server_priv and mom_priv, for mom_priv why not just follow how server_priv is currently defined in the EDD, since as it is written already hooks/ will be copied in?[quote=“agrawalravi90, post:74, topic:520”]
Added ps -leaf output.
I think it is fine as it is. Adding another layer just means more typing to get at the same information without adding to the overall organization. Others may of course disagree, and if so please speak up.
One more thought is that currently we have qmgr_lsched.out at the top level rather than in a scheduler/ directory. That makes some amount of sense because currently that’s the only relevant command output. Once PP-748 goes in, will we need more, like “list fs_group”, “list policy”, “list time_window” (that may be a question for @jon and @arungrover , and probably added to the WIP PP-748 EDD as far as what gets listed or not listed when doing “list sched”)? I would suggest creating a scheduler/ directory now and putting qmgr_lsched.out in it and we can add to it later as needed.
Thanks Ravi, inching towards the finish line here! What about this:
As the EDD currently stands there will be no hooks/ directory containing the actual contents of the directory from mom_priv/, and while that SHOULD not be needed since in theory that all matches what is in qmgr_ph_default.out, I think it is worth collecting for troubleshooting purposes.