middleware:Globus/extension/mpi
From Dgiref
(Redirected from middleware:Globus/420/server/configure/extension)
Globus Toolkit package provided by the D-Grid reference installation
From a user's perspective the change allows to use extensions to load the mpi module before the job starts as shown below:
<extensions> <preamble> module load mpi </preamble> </extensions> </job>
 In the latest globus 4.2 reference package the changes provided below are already made. |
Globus Toolkit package do not provided by the D-Grid reference installation
- Administrative changes
It is necessary to change the file $GLOBUS_LOCATION/lib/perl/Globus/GRAM/JobManager/pbs.pm. The output of diff pbs.pm.orig pbs.pm is shown below containing all the necessary changes including the ones proposed by Benjamin Henne (the changes below the diff entry 280c311 are from Benjamin and not related to this extension).
patch pbs.pm -The patch program will wait for your input. Paste the diff output below and when finished press enter and then ctrl-D (strg-D auf Deutsch) TWICE (ZWEIMAL).
30c30 < $mpirun = 'no'; --- > $mpirun = '/home/software/mpi/mpiexec_wrapper'; 37,39c37,39 < $softenv_dir = ''; < $soft_msc = "$softenv_dir/bin/soft-msc"; < $softenv_load = "$softenv_dir/etc/softenv-load.sh"; --- > #$softenv_dir = ''; > #$soft_msc = "$softenv_dir/bin/soft-msc"; > #$softenv_load = "$softenv_dir/etc/softenv-load.sh"; 41a42,62 > sub new > { > > my $cln = shift; > my $class = ref($cln) || $cln; > > my $self = $class->SUPER::new(@_); > bless $self, $class; > > $self->log("INFO: exec new from sge::new\n"); > > my $description = $self->{JobDescription}; > use Globus::GRAM::ExtensionsHandler; > new Globus::GRAM::ExtensionsHandler($class, $self, $description); > > # use Data::Dumper; > # $self->log("INFO: job description\n"); > # $self->log(Dumper($description)); > # if (defined($description->march)){$self->log("INFO: march available\n");} > return $self; > } 268,275c289,296 < if ($softenv_dir ne '') < { < $self->setup_softenv( < $self->job_dir() . '/pbs_softenv_job_script', < $soft_msc, < $softenv_load, < *JOB); < } --- > #if ($softenv_dir ne '') > #{ > # $self->setup_softenv( > # $self->job_dir() . '/pbs_softenv_job_script', > # $soft_msc, > # $softenv_load, > # *JOB); > #} 276a298,307 > > # > # If preamble defined, insert it > # > if( $description->preamble() ) { > $self->log("INFO: Preamble:\n"); > $self->log( $description->preamble() ); > print JOB "\n# Preamble\n"; > print JOB $description->preamble() . "\n\n"; > } 280c311 < $library_vars{LD_LIBRARY_PATH} = 0; --- > #$library_vars{LD_LIBRARY_PATH} = 0; 444c475 < $machinefilearg = ' -machinefile $PBS_NODEFILE'; --- > # $machinefilearg = ' -machinefile $PBS_NODEFILE'; 469c500 < print JOB ' -machinefile $PBS_NODEFILE'; --- > # print JOB ' -machinefile $PBS_NODEFILE'; 481,490c512,525 < hosts=\`cat \$PBS_NODEFILE\`; < counter=0 < while test \$counter -lt $count; do < for host in \$hosts; do < if test \$counter -lt $count; then < $remote_shell \$host "/bin/sh $cmd_script_name; echo \\\$? > $exit_prefix.\$counter" < $stdin & < counter=\`expr \$counter + 1\` < else < break < fi --- > if [ $count -eq 1 ]; then > /bin/sh $cmd_script_name; echo \$? > $exit_prefix.0 < $stdin > else > hosts=\`cat \$PBS_NODEFILE\`; > counter=0 > while test \$counter -lt $count; do > for host in \$hosts; do > if test \$counter -lt $count; then > $remote_shell \$host "/bin/sh $cmd_script_name; echo \\\$? > $exit_prefix.\$counter" < $stdin & > counter=\`expr \$counter + 1\` > else > break > fi > done 492,493c527,528 < done < wait --- > wait > fi
The file $GLOBUS_LOCATION/lib/perl/Globus/GRAM/ExtensionsHandler.pm should also be changed! The diff is provided below.
15a16,17 > # LRZ: added $jm arg to get access to the JM log file > my $jm = shift; 26a29,30 > # LRZ: set log file handle > $self->{log} = $jm->{log} if ( $jm->{log} );
Finally the file <code>$GLOBUS_LOCATION/lib/perl/Globus/GRAM/JobManager.pm</code> should be changed as follows. 93c93,97 < if ($description->xml_extensions()) --- > #LRZ# if ($description->xml_extensions()) > #LRZ#{ > #LRZ# new Globus::GRAM::ExtensionsHandler($class, $description); > #LRZ#} > if( ! defined($description->logfile()) ) 95c99,103 < new Globus::GRAM::ExtensionsHandler($class, $description); --- > my $WHOAMI = `whoami`; > chomp $WHOAMI; > > my $logfile = "/tmp/$WHOAMI-jobmanager.log"; > $description->add('logfile', $logfile); 413a422 > # chdir("/"); is missing from 430c439 < chdir("/"); --- > #chdir("/");
