Tags:
,
view all tags
<noautolink> The CREAM user's guide is available at: [[https://edms.cern.ch/document/595770][https://edms.cern.ch/document/595770]] --- ---+!! CREAM User's Guide %TOC% ---# CREAM Command Line Interface Guide This section briefly explains the sequence of operations to be performed by a user to submit and then manage jobs on CREAM based CEs, referring to the C++ Command Line Interface (CLI). ---## Before starting: get your user proxy Before using any of the CREAM client commands, it is necessary to have a valid proxy credential available on the client machine. You can create it using the =voms-proxy-init= command. If you already have a valid proxy available on your machine just make the =X509_USER_PROXY= environment variable point to it. In order to get a proxy certificate issued by VOMS, you should have in the directory =/etc/vomses= the proper VOMS file containing a line as follows: <verbatim> "EGEE" "kuiken.nikhef.nl" "15001" "/O=dutchgrid/O=hosts/OU=nikhef.nl/CN=kuiken.nikhef.nl" "EGEE" "22" </verbatim> or the corresponding line for your VO. You also need to install the VO related .lsc files in the =/etc/grid-security/vomsdir/<VO>= directory. In a standard EMI UI installation, these settings should be already there. Make moreover sure you have in the directory =$HOME/.globus= your certificate/key pair, i.e. the following files: <verbatim> usercert.pem userkey.pem </verbatim> Note that file permissions are important: the two files must have respectively 0600 and 0400 permissions. Then you can issue the VOMS client command (you will be prompted for the pass-phrase): <verbatim> $ voms-proxy-init -voms dteam Enter GRID pass phrase: Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto Creating temporary proxy ............................................................................................................................................... Done Contacting voms2.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr] "dteam" Done Creating proxy .............................. Done Your proxy is valid until Sat Apr 30 05:05:49 2011 $ voms-proxy-info -all subject : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto/CN=proxy issuer : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto identity : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto type : proxy strength : 1024 bits path : /tmp/x509up_u500 timeleft : 11:59:55 key usage : Digital Signature, Key Encipherment, Data Encipherment === VO dteam extension information === VO : dteam subject : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto issuer : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr attribute : /dteam/Role=NULL/Capability=NULL attribute : /dteam/italy/Role=NULL/Capability=NULL attribute : /dteam/italy/INFN-PADOVA/Role=NULL/Capability=NULL timeleft : 11:59:55 uri : voms2.hellasgrid.gr:15004 </verbatim> ---## CREAM CLI commands The most relevant commands to interact with CREAM based CEs are: * glite-ce-job-submit <jdl_file> * glite-ce-delegate-proxy <delegation_Id> * glite-ce-job-status <job_Ids> * glite-ce-job-list <host[:port]> * glite-ce-job-cancel <job_Ids> * glite-ce-job-suspend <job_Ids> * glite-ce-job-resume <job_Ids> * glite-ce-job-output <job_Ids> * glite-ce-job-purge <job_Ids> * glite-ce-proxy-renew <delegation_Ids> * glite-ce-service-info <host[:port]> * glite-ce-get-cemon-url <host[:port]> * glite-ce-enable-submission <host[:port]> * glite-ce-disable-submission <host[:port]> * glite-ce-allowed-submission <host[:port]> Man pages are available for all the CREAM client commands. You can also access information about the usage of each command by issuing: <verbatim> $ <command> --help </verbatim> =glite-ce-job-submit= submits a job to a CREAM based CE. It requires a JDL file as input and returns a CREAM job identifier. =glite-ce-delegate-proxy= allows the user to delegate her proxy credential to the CREAM service. This delegated credential can then be used for job submissions. =glite-ce-job-status= displays information (in particular the states) of jobs previously submitted to CREAM based CEs. =glite-ce-job-list= lists the identifiers of jobs submitted to a CREAM based CE by the user issuing the command. =glite-ce-job-cancel= cancels one or more jobs previously submitted to CREAM based CEs. =glite-ce-job-suspend= suspends the execution of one or more jobs previously submitted to CREAM based CEs. =glite-ce-job-resume= resumes the execution of one or more jobs which have been previously suspended. =glite-ce-job-output= retrieves the output sandbox files of one or more jobs previously submitted to CREAM based CEs. =glite-ce-job-purge= clears one or more jobs from CREAM based CEs. After this operation the purged jobs can’t be managed anymore. =glite-ce-proxy-renew= renews delegations, and therefore refreshes the proxy of the jobs submitted to CREAM based CEs using the considered delegations. =glite-ce-service-info= returns information about the CREAM service (version, status, etc.). =glite-ce-get-cemon-url= returns the end-point of the CEMon service coupled with the considered CREAM CE. =glite-ce-enable-submission= (re-)enables job submissions on the specified CREAM CE. =glite-ce-disable-submission= disables job submissions on the specified CREAM CE. =glite-ce-allowed-submission= checks if jobs submissions on the specified CREAM CE are allowed or have been disabled. All these commands are described in the following sections. ---## Submitting jobs to CREAM based CEs To submit jobs to CREAM based CEs, the command =glite-ce-job-submit= must be used. The glite-ce-job-submit command requires as input a job description file, which describes the job characteristics and requirements through the JDL (Job Description Language). A typical example of a JDL job description file is: <verbatim> [ Type = "Job"; JobType = "Normal"; Executable = "myexe"; StdInput = "myinput.txt"; StdOutput = "message.txt"; StdError = "error.txt"; InputSandbox = {"/users/seredova/example/myinput.txt", "/users/seredova/example/myexe"}; OutputSandbox = {"message.txt", "error.txt"}; OutputSandboxBaseDestUri = "gsiftp://se.pd.infn.it/data/seredova"; ] </verbatim> Such a JDL would make the =myexe= executable be transferred on the remote CREAM CE and be run taking the =myinput.txt= file (also copied from the client node) as input. The standard streams of the job are redirected to files =message.txt= and =error.txt=, and when job completes its execution they are automatically uploaded on =gsiftp://se.pd.infn.it/data/seredova=. A detailed description of the available JDL attributes and of the rules for building correct JDL files is provided at http://wiki.italiangrid.org/twiki/bin/view/CREAM/JdlGuide. Each job submitted to a CREAM based CE is given the delegated credentials of the user who submitted it. These credentials can then be used when operations requiring security support has to be performed by the job. There are two possible options to deal with proxy delegation: * asking the “automatic” delegation of the credentials during the submission operation; * explicitly delegating credentials, and then asking to rely on these previously delegated credentials on the actual submission operations. It is highly suggested to rely on this latter mechanism, using the same delegated proxy for multiple job submissions, instead of delegating each time a proxy. Delegating a proxy, in fact, is an operation that can require a non negligible time. The command =glite-ce-delegate-proxy= is the command to be used to explicitly delegate the user credentials to a CREAM CE. The following shows an example of job submission, performed explicitly delegating credentials. So first of all the credentials are delegated to a CREAM based CE (whose endpoint is specified with the option =–endpoint (-e)=: <verbatim> > glite-ce-delegate-proxy -e cream-ce-01.pd.infn.it mydelid 2006-02-26 15:03:37,286 NOTICE - Proxy with delegation id [mydelid] successfully delegated to endpoint [https://cream-ce-01.pd.infn.it:8443//ce-cream/services/CREAMDelegation] </verbatim> The identifier of the delegation is then specified with the =–delegationId (-D)= option in the job submit operation: <verbatim> > glite-ce-job-submit -D mydelid -r cream-ce-01.pd.infn.it:8443/cream-lsf-grid02 myjob1.jdl </verbatim> The option =-r (–resource)= has been used to specify the identifier of the CREAM CE where the job has to be submitted to. =myjob1.jdl= is the JDL file describing the job to be submitted. The command returns the CREAM job identifier associated with this job (e.g. =https://cream-ce-01.pd.infn.it:8443/CREAM116j9vgnf=) which identifies it in clear and unique way all over the Grid system scope. ---## Monitoring jobs Passing the CREAM job id returned by the glite-ce-job-submit command to the =glite-ce-job-status= command, it is possible to monitor the submitted job. Several (static and dynamic) information can be shown, depending on the chosen verbosity level. The verbosity level can be 0 (less verbosity), 1 or 2 (most verbosity). Please note that specifying 0 as verbosity level means calling on the CREAM service a faster operation than when using 1 or 2 as verbosity level. The most relevant attribute is the job status. The following is an example of job status operation, specifying 1 as verbosity level: <verbatim> $ glite-ce-job-status -L 1 https://cream-02.pd.infn.it:8443/CREAM738582717 ****** JobID=[https://cream-02.pd.infn.it:8443/CREAM738582717] Current Status = [DONE-FAILED] ExitCode = [N/A] FailureReason = [lsf_reason=256; Cannot move ISB (${globus_transfer_cmd} gsiftp://cream-02.pd.infn.it//CREAMTests/Exe1/ssh1.sh file:///home/infngrid001/home_cream_738582717/CREAM738582717/ssh1.sh): error: globus_ftp_client: the server responded with an error 500 500-Command failed. : globus_l_gfs_file_open failed. 500-globus_xio: Unable to open file //CREAMTests/Exe1/ssh1.sh 500-globus_xio: System error in open: No such file or directory 500-globus_xio: A system call failed: No such file or directory 500 End.] Grid JobID = [N/A] Job status changes: ------------------- Status = [REGISTERED] - [Tue 22 Jan 2008 15:55:08] (1201013708) Status = [PENDING] - [Tue 22 Jan 2008 15:55:08] (1201013708) Status = [IDLE] - [Tue 22 Jan 2008 15:55:11] (1201013711) Status = [RUNNING] - [Tue 22 Jan 2008 15:55:18] (1201013718) Status = [DONE-FAILED] - [Tue 22 Jan 2008 16:03:10] (1201014190) Issued Commands: ------------------- *** Command Name = [JOB_REGISTER] Command Category = [JOB_MANAGEMENT] Command Status = [SUCCESSFULL] *** Command Name = [JOB_START] Command Category = [JOB_MANAGEMENT] Command Status = [SUCCESSFULL] </verbatim> In this example it is interesting to note that the job failed (as reported by the =Current Status= field) for the problem reported in the =FailureReason= field: the file to be transferred was not found. Instead of explicitly specifying the identifiers of the jobs to monitor, the user can also ask to monitor all her jobs, in case specifying conditions (on the submission date and/or on the job status) that must be met. For example to monitor all jobs, whose status is DONE-OK or DONE-FAILED, submitted to the grid005.pd.infn.it CREAM CE between July 23, 2005 10:00 and July 28, 2005 11:00, the following command must be issued: <verbatim> > glite-ce-job-status --all -e grid005.pd.infn.it:8443 --from ’2005-07-23 10:00:00’ --to ’2005-07-28 11:00:00’ -s DONE-OK:DONE-FAILED </verbatim> ---## Retrieving output of jobs User can choose to save the output sandbox (OSB) files on a remote server, or save them in the CREAM CE node. In the latter case these files can then be retrieved using the =glite-ce-job-output command. For example the following command retrieves the output sandbox files of the specified job from the relevant CREAM CE node: <verbatim> > glite-ce-job-output https://cream-38.pd.infn.it:8443/CREAM295728364 2011-01-29 10:09:50,394 INFO - For JobID [https://cream-38.pd.infn.it:8443/CREAM295728364] output will be stored in the dir ./cream-38.pd.infn.it_8443_CREAM295728364 </verbatim> ---## Getting job identifiers If a user is interested to get the identifiers of all her jobs submitted to a specific CREAM CE, she can use the =glite-ce-job-list= command. For example the following command returns the identifiers of all the jobs submitted to the specified CREAM CE, owned by the user issuing the command: <verbatim> > glite-ce-job-list grid005.pd.infn.it:8443 </verbatim> ---## Cancelling jobs In some cases it might be needed to cancel jobs which have been previously submitted to CREAM based CEs. This can be achieved via the =glite-ce-job-cancel= command. E.g., the command: <verbatim> > glite-ce-job-cancel https://grid005.pd.infn.it:8443/CREAM115j5vfnf </verbatim> cancels the specified job. ---## Suspending and resuming jobs A running or idle job can be suspended (i.e. its execution will be stopped), and be resumed (i.e. it will run again) later. This can be achieved with the =glite-ce-job-suspend= and =glite-ce-job-resume= commands. The following example shows that after having issued the glite-ce-job-suspend command, after a while the job status becomes =HELD=. <verbatim> > glite-ce-job-suspend https://cream-ce-01.pd.infn.it:8443/CREAM11a79tnb2 Are you sure you want to suspend specified job(s) [y/n]: y > glite-ce-job-status -L 0 https://cream-ce-01.pd.infn.it:8443/CREAM11a79tnb2 ****** JobID=[https://cream-ce-01.pd.infn.it:8443/CREAM11a79tnb2] Status = [HELD] </verbatim> Issuing the =glite-ce-job-resume= command, the job will run/will be idle again: <verbatim> > glite-ce-job-resume https://cream-ce-01.pd.infn.it:8443/CREAM11a79tnb2 Are you sure you want to resume specified job(s) [y/n]: y > glite-ce-job-status -L 0 https://cream-ce-01.pd.infn.it:8443/CREAM11a79tnb2 ****** JobID=[https://cream-ce-01.pd.infn.it:8443/CREAM11a79tnb2] Status = [REALLY-RUNNING] </verbatim> ---## Purging jobs A CREAM job can be monitored (via the glite-ce-job-status) even after it has completed its execution. A job gets “lost” (i.e. it is not possible to monitor or manage it anymore) only when the user who submitted it decides to explicitly clear it, or when the CREAM system administrator decides to do this purging operation. A user can purge her own jobs, using the =glite-ce-job-purge= command. E.g., after having issued the command: <verbatim> > glite-ce-job-purge https://cream-ce-01.pd.infn.it:8443/CREAM116jbi4o0 </verbatim> the specified job can’t be managed anymore (e.g. it is not possible to check its status anymore). ---## Renewing proxies It is possible that long jobs may outlive the validity of the initial delegated credentials; if so the job will die prematurely. To avoid this it is possible to renew the proxy of jobs submitted to CREAM CEs with the =glite-ce-proxy-renew= command. E.g. the following command: <verbatim> > glite-ce-proxy-renew -e cream-ce-01.pd.infn.it:8443 mydelid </verbat renews the proxy of all the jobs having =mydelid= as delegation id. It must be stressed that for jobs submitted to CREAM based CEs via the Workload Management System (WMS), proxy renewal is automatically dealt by the middleware. ---# Man pages for CREAM Command Line Interface ---## [[GliteCeJobSubmitMan][glite-ce-job-submit]] ---## [[GliteCeJobCancel][glite-ce-job-cancel]] ---# Forward of requirements to the batch system The CREAM CE allows to forward, via tha BLAH component, requirements to the batch system. For this purpose the JDL =CERequirements- attribute, described at http://wiki.italiangrid.org/twiki/bin/view/CREAM/JdlGuide#3_27_CERequirements, can be used. For direct submissions to the CREAM CE (e.g. jobs submitted to the CREAM CE using the CREAM CLI =glite-ce-job-submit= command) the CeRequirements attribute is supposed to be filled by the end-user. For jobs submitted to the CREAM CE via the WMS, the CeRequirements attribute is instead filled by the WMS, considering the JDL =Requirements= expression and the value of the =CeForwardParameters= attribute in the WMS configuration file. For example, if in the user JDL there is : <verbatim> Requirements= "other.GlueHostMainMemoryRAMSize > 100 && other.GlueCEImplementationName==\"CREAM\""; </verbatim> and if the WMS configuration file there is: <verbatim> CeForwardParameters = {"GlueHostMainMemoryVirtualSize","GlueHostMainMemoryRAMSize", "GlueCEPolicyMaxCPUTime"}; </verbatim> in the JDL sent by the WMS to CREAM there will be: <verbatim> CeRequirements= "other.GlueHostMainMemoryRAMSize > 100"; </verbatim> The CERequirements expression received by CREAM is then forwarded to BLAH. Basically BLAH manages the CERequirements expression setting some environment variables, which are available and can be properly used by the =/usr/bin/xxx_local_submit_attributes.sh= script (e.g. =/usr/bin/pbs_local_submit_attributes.sh= for PBS/Torque, =/usr/bin/lsf_local_submit_attributes.sh= for LSF). This script must be properly created by the site admin. For example, considering the following CeRequirements expression: <verbatim> CeRequirements="other.GlueHostMainMemoryRAMSize > 100 && other.GlueCEStateWaitingJobs <10 && other.GlueCEImplementationName==\"CREAM\" && other.GlueHostProcessorClockSpeed >= 2800 && (Member(\"FDTD\", other.GlueHostApplicationSoftwareRuntimeEnvironment))"; </verbatim> the following settings will be available in $GLITE_LOCATION/bin/xxx_local_submit_attributes.sh: <verbatim> GlueHostMainMemoryRAMSize_Min='100' GlueCEStateWaitingJobs_Max='10' GlueCEImplementationName='CREAM' GlueHostProcessorClockSpeed_Min='2800' GlueHostApplicationSoftwareRuntimeEnvironment='"FDTD"' </verbatim> What is printed by the =/usr/bin/xxx_local_submit_attributes.sh= script is automatically added to the submit command file. For example if the JDL Cerequirements expression is: <verbatim> CeRequirements = "(Member(\"FDTD\", other.GlueHostApplicationSoftwareRuntimeEnvironment))"; </verbatim> and the =/usr/bin/pbs_local_submit_attributes.sh= is: <verbatim> #!/bin/sh if [ "$other.GlueHostApplicationSoftwareRuntimeEnvironment" == "FDTD" ]; then echo "#PBS -l software=FDTD" fi </verbatim> then the PBS submit file that will be used will include: <verbatim> ... ... # PBS directives: #PBS -S /bin/bash #PBS -o /dev/null #PBS -e /dev/null #PBS -l software=FDTD .... .... </verbatim> where the line: <verbatim> #PBS -l software=FDTD </verbatim> is set via the =/usr/bin/pbs_local_submit_attributes.sh= script. Please note that there are no differences if in CeRequirements expresssion there is e.g. <verbatim> CeRequirements = other.xyz==\"ABC\" </verbatim> or: <verbatim> CeRequirements = "xyz==\"ABC\""; </verbatim> In both cases in =/usr/bin/xxx_local_submit_attributes.sh= the variable =xyz= will be set. As shown above, having =x>a= or =x>=a= doesn't make any difference in the setting of the environment variable =x= in the =/usr/bin/xxx_local_submit_attributes.sh= script. It will be in both cases: <verbatim> x_Min='a' </verbatim> </noautolink> -- Main.MassimoSgaravatto - 2011-04-07
Edit
|
Attach
|
PDF
|
H
istory
:
r17
|
r9
<
r8
<
r7
<
r6
|
B
acklinks
|
V
iew topic
|
More topic actions...
Topic revision: r7 - 2011-04-30
-
MassimoSgaravatto
Home
Site map
CEMon web
CREAM web
Cloud web
Cyclops web
DGAS web
EgeeJra1It web
Gows web
GridOversight web
IGIPortal web
IGIRelease web
MPI web
Main web
MarcheCloud web
MarcheCloudPilotaCNAF web
Middleware web
Operations web
Sandbox web
Security web
SiteAdminCorner web
TWiki web
Training web
UserSupport web
VOMS web
WMS web
WMSMonitor web
WeNMR web
General Doc
Functional Description
Batch System Support
CREAM and Information Service
Release Notes
Known Issues
Security in CREAM
Nagios Probes to monitor CREAM and WN
Papers
Presentations
User Doc
CREAM User Guide for EMI-1
CREAM User Guide for EMI-2
CREAM User Guide for EMI-3
CREAM JDL Guide
BLAH User Guide
Troubleshooting Guide
System Administrator Doc
System Administrator Guide for CREAM (EMI-3 release)
System Administrator Guide for CREAM (EMI-2 release)
System Administrator Guide for CREAM (EMI-1 release)
The CREAM configuration file
The CEMonitor configuration file
The CREAM CE Service Reference Card (EMI-2 release)
The CREAM CE Service Reference Card (EMI-1 release)
Batch System related documentation
Troubleshooting Guide
The guide for integrating EMIR in CREAM
]
Developers Doc
CREAM Client API C++ Documentation
CREAM Client API for Python
Other Doc
Contacts
Moving to CREAM from LCG-CE
Testing
Internal Collaboration Information
Credits
CREAM Web utilities
Create New Topic
Index
Search
Changes
Notifications
RSS Feed
Statistics
Preferences
P
View
Raw View
Print version
Find backlinks
History
More topic actions
Edit
Raw edit
Attach file or image
Edit topic preference settings
Set new parent
More topic actions
Account
Log In
Edit
Attach
Copyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki?
Send feedback