Difference: JdlGuide (7 vs. 8)

Revision 82011-04-29 - MassimoSgaravatto

Line: 1 to 1
 
META TOPICPARENT name="UserGuide"

CREAM JDL guide

Line: 48 to 48
 This a string representing the type of the job described by the JDL, e.g.:
Changed:
<
<
JobType = �Normal�;
>
>
JobType = "Normal";
 

At least for the time being the only possible value is: Normal.

Line: 62 to 62
 This is a string representing the executable/command name. The user can specify an executable that lies already on the remote CE and in this case the absolute path, possibly including environment variables referring to this file should be specified, e.g.:
Changed:
<
<
Executable = �/usr/local/java/j2sdk1.4.0_01/bin/java�;
>
>
Executable = "usr/local/java/j2sdk1.4.0_01/bin/java";
 

Or:

Changed:
<
<
Executable = �$JAVA_HOME/bin/java�;
>
>
Executable = "$JAVA_HOME/bin/java";
 

The other possibility is to provide either an executable located on a remote gridFTP server accessible by the user (HTTPS servers are also supported but this requires to have the GridSite htcp client command installed on the WN). In both cases the executable file will be staged from the original location to the Computing Element WN. In both cases only the file name has to be specified as executable. The URI of the executable should be then listed in the InputSandbox attribute expression to make it be transferred. E.g. respectively:

Changed:
<
<
Executable = �cms_sim.exe�; InputSandbox = {�file:///home/edguser/sim/cms_sim.exe�, ��� };
>
>
Executable = "cms_sim.exe"; InputSandbox = {"file:///home/edguser/sim/cms_sim.exe", ...};
 

Or:

Changed:
<
<
Executable = �cms_sim.exe�; InputSandbox = {�gsiftp://neo.datamat.it:5678/tmp/cms_sim.exe�, ��� };
>
>
Executable = "cms_sim.exe"; InputSandbox = {"gsiftp://neo.datamat.it:5678/tmp/cms_sim.exe", ...};
 

Also descriptions as follows:

Changed:
<
<
Executable = �cms_sim.exe�; InputSandbox = {�/home/edguser/sim/cms_sim.exe�, ��� };
>
>
Executable = "cms_sim.exe"; InputSandbox = {"/home/edguser/sim/cms_sim.exe", ... };
 

are accepted and interpreted as in the first case, i.e. the executable file is available on the local file system.

Line: 105 to 105
 This is a string containing all the job command line arguments. E.g. an executable sum that has to be started as:
Changed:
<
<
$ sum N1 N2 �out result.out
>
>
$ sum N1 N2 -out result.out
 

is described by:

Changed:
<
<
Executable = �sum�; Arguments = �N1 N2 �out result.out�;
>
>
Executable = "sum"; Arguments = "N1 N2 -out result.out";
 

If you want to specify a quoted string inside the Arguments then you have to escape quotes with the \ character. E.g. when describing a job like:

Changed:
<
<
$ grep �i �my name� *.txt
>
>
$ grep -i "my name" *.txt
 

you will have to specify:

Changed:
<
<
Executable = �/bin/grep�; Arguments = �-i \�my name\� *.txt�;
>
>
Executable = "/bin/grep"; Arguments = "-i \"my name\" *.txt";
 

Analogously, if the job takes as argument a string containing a special character (e.g. the job is the tail command issued on a file whose name contains the ampersand character, say file1&file2), since on the shell line you would have to write:

Changed:
<
<
$ tail �f file1\&file2
>
>
$ tail -f file1\&file2
 

in the JDL you’ll have to write:

Changed:
<
<
Executable = �/usr/bin/tail�; Arguments = �-f file1\\\&file2�;
>
>
Executable = "/usr/bin/tail"; Arguments = "-f file1\\\&file2";
 

i.e. a \ for each special character. In general, special characters such as:

Line: 146 to 146
  are only allowed if specified inside a quoted string or preceded by triple \. The character ` cannot be specified in the JDL.
Added:
>
>
Some other guidelines for the management of special characters:

  • If the arguments string contains one or more spaces, the string must be specified in single quotes
  • If the arguments string contains single quotes, then the string must be specified in double quotes (which must be escaped)
  • If the arguments string contains spaces and single quotes, then the string must be specified in double quotes (which must be escaped)

Some examples:

Arguments = "\"Problematic character is: '\" " ;

Arguments = "-o '/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=romanov/CN=427293/CN=Vladimir Romanovskiy' \"domani verro' \"" ;

Arguments = "-o \"'option'\"";
 Mandatory: No

Default: No

Line: 161 to 177
 It can be an absolute path possibly including environment variables (wild cards are instead not allowed), i.e. it is already available on the CE, e.g.
Changed:
<
<
StdInput = �/var/tpm/jobInput�;
>
>
StdInput = "/var/tpm/jobInput";
 

or just a file name, e.g.

Changed:
<
<
StdInput = �myjobInput�;
>
>
StdInput = "myjobInput";
 

and this means that file needs to be made available on the WN where the job is run. Therefore the standard input file has to be added to the InputSandbox file list so that it will be downloaded on the WN. The same rules described for the Executable attribute apply to StdInput.

Line: 181 to 197
 This is a string representing the file name where the standard output of the job is saved. The user can specify either a file name or a relative path (with respect to the job working directory on the WN), e.g.:
Changed:
<
<
StdOutput = �myjobOutput�;
>
>
StdOutput = "myjobOutput";
 
Changed:
<
<
StdOutput = �event1/myjobOutput�;
>
>
StdOutput = "event1/myjobOutput";
 

Wild cards are not allowed. The value specified for StdError can be the same as the one for StdOutput: this means that the two standard streams of the job are saved in the same file. The user can choose to have this file staged automatically on a GridFTP server specifying a URI for that file in the OutputSandbox attribute expression. E.g.:

Changed:
<
<
StdOutput = �myjobOutput�;
>
>
StdOutput = "myjobOutput";
 OutputSandbox = {
Changed:
<
<
�gsiftp://fox.infn.it:5678/home/gftp/myjobOutput�, �
>
>
"gsiftp://fox.infn.it:5678/home/gftp/myjobOutput", ...
 };
Line: 207 to 223
 This is a string representing the file name where the standard error of the job is saved. The user can specify either a file name or a relative path (with respect to the job working directory on the WN), e.g.:
Changed:
<
<
StdError = �myjobError�;
>
>
StdError = "myjobError";
 
Changed:
<
<
StdError = �event1/myjobError�;
>
>
StdError = "event1/myjobError";
 

Wild cards are not allowed. The value specified for StdError can be the same as the one for StdOutput: this means that the two standard streams of the job are saved in the same file. The user can choose to have this file staged automatically on a GridFTP server specifying a URI for that file in the OutputSandboxDestURI attribute expression The same rules as for the StdOutput apply to StdError.

Line: 268 to 284
 
InputSandbox = {
Changed:
<
<
>
>
...
  "data/event1.txt",
Changed:
<
<
>
>
...
  }; InputSandboxBaseURI = "gsiftp://matrix.datamat.it:5432/tmp";
Line: 295 to 311
 
InputSandbox = {
Changed:
<
<
>
>
...
  "data/event2.txt",
Changed:
<
<
>
>
...
  }; InputSandboxBaseURI = "gsiftp://lxsgaravatto.pd.infn.it:47320/tmp?DN=/C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto";
Line: 469 to 485
 This is a list of string representing environment settings that have to be performed on the execution machine and are needed by the job to run properly. The JobWrapper on the Worker Node performs these settings just before the job is started. Each item of the list is an equality “VAR_NAME=VAR_VALUE”. E.g.:
Changed:
<
<
Environment = {�JOB_LOG_FILE=/tmp/myjob.log�, �ORACLE_SID=edg_rdbms_1�, �JAVABIN=/usr/local/java�};
>
>
Environment = {"JOB_LOG_FILE=/tmp/myjob.log", "ORACLE_SID=edg_rdbms_1", "JAVABIN=/usr/local/java"};
 

Mandatory: No

Line: 521 to 537
 The PerusalListFileURI attribute is a string representing the URI of the file on a gridFTP server, containing the list of files (one for each line), that must be saved and uploaded on the location specified by the attribute PerusalFilesDestURI at regular time intervals. E.g.
Changed:
<
<
PerusalListFileURI = �gsiftp://ghemon.cnaf.infn.it/peek/files2peek�;
>
>
PerusalListFileURI = "gsiftp://ghemon.cnaf.infn.it/peek/files2peek";
 

Mandatory: No (unless PerusalFileEnable is true)

Line: 533 to 549
 This is a string representing the Local Resource Management System (LRMS), that is the batch system type (e.g. LSF, PBS, etc.) of the target CREAM CE. Here below follows an example for this attribute:
Changed:
<
<
BatchSystem = �cms�;
>
>
BatchSystem = "cms";
 

Mandatory: Yes (unless the CREAM CLI, since it retrieves this attribute from the CEID and automatically fills this attribute)

Line: 545 to 561
 This is a string representing the queue name in the Local Resource Management System (LRMS) where the job has to be submitted on the target CREAM CE. Here below follows an example for this attribute:
Changed:
<
<
QueueName = �long�;
>
>
QueueName = "long";
 

Mandatory: Yes (unless the CREAM CLI is used, since it retrieves this attribute from the CEID and automatically fills this attribute)

Line: 599 to 615
 This is an example of CERequirements expression:
Changed:
<
<
CERequirements = �other.GlueCEPolicyMaxCPUTime >= 100 && other.GlueHostMainMemoryRAMSize > 2000�;
>
>
CERequirements = "other.GlueCEPolicyMaxCPUTime >= 100 && other.GlueHostMainMemoryRAMSize > 2000";
 

The CERequirements attribute can be used when jobs are submitted directly to a CREAM based CE, while for jobs submitted to a CREAM based CE via the WMS, the CERequirements expression is automatically filled by the WMS considering the user’s Requirements expression, and the value of the CeForwardParameters attribute in the WMS configuration file.

 
This site is powered by the TWiki collaboration platformCopyright © 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback