Difference: UserGuide (7 vs. 8)

Revision 82011-05-02 - MassimoSgaravatto

Line: 1 to 1
 
META TOPICPARENT name="UserDocumentation"
Line: 161 to 161
 The command glite-ce-delegate-proxy is the command to be used to explicitly delegate the user credentials to a CREAM CE.

The following shows an example of job submission, performed explicitly delegating credentials.

Changed:
<
<
So first of all the credentials are delegated to a CREAM based CE (whose endpoint is specified with the option –endpoint (-e):
>
>
So first of all the credentials are delegated to a CREAM based CE (whose endpoint is specified with the option --endpoint (-e):
 
> glite-ce-delegate-proxy -e cream-ce-01.pd.infn.it mydelid
Line: 169 to 169
 delegated to endpoint [https://cream-ce-01.pd.infn.it:8443//ce-cream/services/CREAMDelegation]
Changed:
<
<
The identifier of the delegation is then specified with the –delegationId (-D) option in the job submit operation:
>
>
The identifier of the delegation is then specified with the --delegationId (-D) option in the job submit operation:
 
> glite-ce-job-submit -D mydelid -r cream-ce-01.pd.infn.it:8443/cream-lsf-grid02 myjob1.jdl
Changed:
<
<
The option -r (–resource) has been used to specify the identifier of the CREAM CE where the job has to be submitted to.
>
>
The option -r (--resource) has been used to specify the identifier of the CREAM CE where the job has to be submitted to.
  myjob1.jdl is the JDL file describing the job to be submitted.
Line: 235 to 235
 

0.1 Retrieving output of jobs

Changed:
<
<
User can choose to save the output sandbox (OSB) files on a remote server, or save them in the CREAM CE node. In the latter case these files can then be retrieved using the =glite-ce-job-output command.
>
>
User can choose to save the output sandbox (OSB) files on a remote server, or save them in the CREAM CE node. In the latter case these files can then be retrieved using the glite-ce-job-output command.
  For example the following command retrieves the output sandbox files of the specified job from the relevant CREAM CE node:
Line: 311 to 311
 
> glite-ce-proxy-renew -e cream-ce-01.pd.infn.it:8443 mydelid
Changed:
<
<
</verbat
>
>
  renews the proxy of all the jobs having mydelid as delegation id.

It must be stressed that for jobs submitted to CREAM based CEs via the Workload Management System (WMS), proxy renewal is automatically dealt by the middleware.

Added:
>
>

0.1 Handling job identifiers

Handling the job identifiers directly quickly becomes tedious. To avoid this, you can make the glite-ce-job-submit and glite-ce-job-list commands append the job Id(s) to a named file using the --output (-o) option. On the other side, the CREAM client commands which take job identifier(s) as argument accept also the =--input (=-i) option which allows the job identifier(s) to be read from a file.

The following shows an example:

> glite-ce-job-submit -a -r cream-ce-01.pd.infn.it:8443/cream-lsf-grid02 -o idfile myjob.jdl
https://cream-ce-01.pd.infn.it:8443/CREAM116jbs5b9

The returned job id got also inserted in the specified file (idfile), which can be specified with the --input (-i) option e.g. with the glite-ce-job-status command:

> glite-ce-job-status -i idfile
****** JobID=[https://cream-ce-01.pd.infn.it:8443/CREAM116jbs5b9]
Status=[REALLY-RUNNING]

0.2 Restricting job submissions

In order to prevent that a CREAM CE gets overloaded, the CREAM CE administrator can set a specific policy to disable new job submissions when certain conditions are met.

If submissions are disabled because of that, if newer job submissions are attempted, users will get an error message such as:

> glite-ce-job-submit -a -r cream-38.pd.infn.it:8443/cream-pbs-creamtest1 oo.jdl
MethodName=[jobRegister] ErrorCode=[0] Description=[The CREAM service cannot accept jobs at the moment]
FaultCause=[Threshold for Load Average(1 min): 30 => Detected value for Load Average(1 min): 31.13]
Timestamp=[Sat 29 Jan 2011 11:55:18]

In order to avoid degrading the performance of the system, the specified policy is not evaluated for each job submission, but instead it is evaluated and imposed from time to time (so it might happen that for a short time job submissions are allowed even if the specified threshold has been reached).

CREAM “super-users” can also disable newer job submissions via the command glite-ce-disable-submission. Submissions can then be re-enabled by a CREAM “super-user” via the command glite-ce-enable-submission.

To check if job submissions on a specific CREAM CE are allowed, the command glite-ce-allowed-submission can be used.

E.g.:

> glite-ce-disable-submission grid006.pd.infn.it:8443
Operation for disabling new submissions succeeded
>
> glite-ce-allowed-submission grid006.pd.infn.it:8443
Job Submission to this CREAM CE is disabled
>
> glite-ce-enable-submission grid006.pd.infn.it:8443
Operation for enabling new submissions succeeded
>
> glite-ce-allowed-submission grid006.pd.infn.it:8443
Job Submission to this CREAM CE is enabled

It must be stressed that if job submissions to a specific CREAM CE are disabled, all other operations (job status, job cancellations, etc.) can still be performed.

0.3 Getting information about the CREAM service

It is possible to get information about the CREAM service (interface and service version, status, etc) using the glite-ce-service-info command, e.g.:

> glite-ce-service-info cream-13.pd.infn.it:8443
Interface Version = [2.1]
Service Version = [1.12]
Description = [CREAM 2]
Started at = [Tue Nov 10 14:42:12 2009]
Submission enabled = [YES]
Status = [RUNNING]
Service Property = [SUBMISSION_THRESHOLD_MESSAGE]->
[Threshold for Load Average
(1 min): 10 => Detected value for Load Average(1 min): 0.03
Threshold for Load Average(5 min): 10 => Detected value for Load Average(5 min): 0.03
Threshold for Load Average(15 min): 10 => Detected value for Load Average(15 min): 0.00
Threshold for Memory Usage: 95 => Detected value for Memory Usage: 57.41%
Threshold for Swap Usage: 95 => Detected value for Swap Usage: 2.02%
Threshold for Free FD: 500 => Detected value for Free FD: 204500
Threshold for tomcat FD: 800 => Detected value for Tomcat FD: 107
Threshold for FTP Connection: 30 => Detected value for FTP Connection: 1
Threshold for Number of active jobs: -1 => Detected value for Number of active jobs: 0
Threshold for Number of pending commands: -1 => Detected value for Number of pending commands: 0

A CREAM CE is usually coupled with a CEMon service, which can be queried to get information about the CE and/or can notify clients with specific CE events. The command glite-ce-get-cemon-url can be used to get the end-point of this CEMon service, e.g.:

> glite-ce-get-cemon-url grid005.pd.infn.it:8443
https://grid005.pd.infn.it:8443/ce-monitor/services/CEMonitor
 

1 Man pages for CREAM Command Line Interface

1.1 glite-ce-job-submit

1.2 glite-ce-job-cancel

Changed:
<
<

1 Forward of requirements to the batch system

>
>

1 Use specific functionality of the CREAM CE

1.1 Forward of requirements to the batch system

  The CREAM CE allows to forward, via tha BLAH component, requirements to the batch system.
Line: 428 to 524
 x_Min='a'
Added:
>
>

1 CREAM job states

Here below is provided a brief description of the meaning of each possible state a CREAM job can enter:

  • REGISTERED: the job has been registered but it has not been started yet.

  • PENDING the job has been started, but it has still to be submitted to the LRMS abstraction layer module (i.e. BLAH).

  • IDLE: the job is idle in the Local Resource Management System (LRMS).

  • RUNNING: the job wrapper, which “encompasses” the user job, is running in the LRMS.

  • REALLY-RUNNING: the actual user job (the one specified as Executable in the job JDL) is running in the LRMS.

  • HELD: the job is held (suspended) in the LRMS.

  • CANCELLED: the job has been cancelled.

  • DONE-OK: the job has successfully been executed.

  • DONE-FAILED: the job has been executed, but some errors occurred.

  • ABORTED: errors occurred during the “management” of the job, e.g. the submission to the LRMS abstraction layer software (BLAH) failed.

  • UNKNOWN: the job is an unknown status.

The following figure shows the possible job states transitions.

 

-- MassimoSgaravatto - 2011-04-07

 
This site is powered by the TWiki collaboration platformCopyright © 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback