How can a PBSPro Server installed externally get recognized by a head node?
An externally installed PBS Pro server is not visible to Bright Cluster Manager, by default.
But this can be made possible, with a few tricks, as described below:
On the Bright head node:
1. IF BCM version 6.0 is used, then update Advanced Config Flags in /cm/local/apps/cmd/etc/cmd.conf
a. AlwaysAssumePBSProRunning=1
b. If PBS Pro is not installed in a Bright standard location (/cm/shared/apps/pbspro), then
also set the following additional advanced config flags:PBSProRoot=/path/to/pbspro/exec/directoryPBSProVar=/path/to/pbspro/spool/directory
The flags must be added as a comma-separated list in the 'AdvancedConfig' section as follows (create the section if it does not exist already):AdvancedConfig = {"AlwaysAssumePBSProRunning=1", "PBSProRoot=/path/to/pbspro/exec/directory", "PBSProVar=/path/to/pbspro/spool/directory"}
If a BCM version greater than 6.0 is used, then advanced configuration flags should not be set. Instead, the following similar parameters are configured in the pbsproserver role on a head node: prefix, spool and externalserver. For example:
[root@bright61 ~]# cmsh[bright61]% device roles master[bright61->device[bright61]->roles]% use pbsproserver [bright61->device[bright61]->roles[pbsproserver]]% set prefix /path/to/pbspro/exec/directory[bright61->device*[bright61*]->roles*[pbsproserver*]]% set spool /path/to/pbspro/spool/directory[bright61->device*[bright61*]->roles*[pbsproserver*]]% set externalserver yes[bright61->device*[bright61*]->roles*[pbsproserver*]]% commit[bright61->device[bright61]->roles[pbsproserver]]%
PS:
- The above two flags (PBSProRoot and PBSProVar in case of BCM version 6.0) are not required, if no additional PBS Pro has been installed (i.e. the default PBS Pro installed by Bright will be used).
- It is recommended that the versions of PBS Pro on the Bright head node and the external PBS Pro server node are the same.
- /path/to/pbspro/exec/directory and /path/to/pbspro/spool/directory should be accessable on the head node. For instance, the directories can be mounted by NFS.
2. If Bright Cluster Manager should not update any PBS Pro configuration,
then the directive FreezeChangesToPBSProConfig should be set to TrueFreezeChangesToPBSProConfig = True
3. Edit /etc/pbs.conf
PBS_SERVER=pbsservernode.domain.name/ipaddress
4. Restart cmdaemon
/etc/init.d/cmd restart
5. Assign pbsproserver role to the Bright head node (example below):[root@bright60 ~]# cmsh[bright60]% device roles master[bright60->device[bright60]->roles]% assign pbsproserver[bright60->device*[bright60*]->roles*[pbsproserver*]]% commit
6. Assign pbsproclient role to the node category (example below):[root@bright60 ~]# cmsh[bright60]% cetegory roles default[bright60->category[default]->roles]% assign pbsproclient[bright60->category*[default*]->roles*[pbsproclient*]]% commit
On the external PBS Pro server node
- Make sure that there is no firewall blocking tcp connection requests from the Bright head node.
(If it does, then PBS Pro calls made from the Bright head node will fail).
And that's it.
Verify that the PBS Pro commands work from the Bright head node (qmgr, qstat, etc). If it does, then
the PBS Pro server, PBS Pro moms, jobs, jobqueues should be visible to the Bright Cluster Manager.
They can be viewed with cmsh or cmgui.