(r1) JobWrapperGeneration < EgeeJra1It

Tags: , view all tags
---+ How the Job Wrapper is Generated

The job wrapper is a shell script that wraps the user's job execution on the Worker Node.

In the LCG RB the full script generation was hard-coded in the RB itself.

In the gLite WMS, instead, the body of the shell script comes from a template file distributed with the WMS packages and installed in =${GLITE_LOCATION}/etc/templates/template.sh=. For each job the template is augmented by the WMS with a few instructions in order to generate the final job wrapper. These instructions set a number of variables that are used inside the template. The value of these variables depends on the specific job.

The template has the following structure

<verbatim>
#!/bin/sh
<empty line>
<body>
</verbatim>

The WMS in correspondence of the empty line adds a header consisting in a number of instructions of the form:

<verbatim>
<variable>=<job-specific value>
</verbatim>

For example, the template contains the following excerpt of code:

<verbatim>
if [ ${__job_type} -eq 0 ]; then # normal
  cmd_line="${__job} ${__arguments}"
elif [ ${__job_type} -eq 1 -o ${__job_type} -eq 2 ]; then # MPI LSF, PBS
  cmd_line="mpirun -np ${__nodes} -machinefile ${HOSTFILE} ${__job} ${__arguments}"
elif [ ${__job_type} -eq 3 ]; then # interactive
  cmd_line="./glite-wms-job-agent $BYPASS_SHADOW_HOST $BYPASS_SHADOW_PORT '${__job} ${__arguments}'"
fi
</verbatim>

where a number of reserved variables are used: =__job_type=, =__job=, =__arguments=, =__nodes=. The setting of these variables is included by the WMS just after the line

<verbatim>
#!/bin/sh
</verbatim>

at the beginning of the script, so that they are well defined during the script execution.

---++ Why the change?

We changed the approach on how to generate the job wrapper script because the complete generation from code is very inflexible: any change would require a change to the code, with all the obvious consequences (rebuilding other components depending on this one, testing, certification, etc.). With the current approach, instead, changing the way the job wrapper is generated most of the times requires only a change in the template file. For example, in order to fix bug [[https://savannah.cern.ch/bugs/?29604][#29604]] ("WMS job wrapper must not reset umask"), waiting for an official patch, a WMS sysadmin could simply remove the following line

<verbatim>
umask 022
</verbatim>

from the template file.

-- Main.FrancescoGiacomini - 15 Nov 2007
Topic revision: r1 - 2007-11-15 - FrancescoGiacomini
Account
- Log In
~~Edit~~
~~Attach~~