Difference: UserGuideEMI2 (16 vs. 17)

Revision 172012-03-27 - AlviseDorigo

Line: 1 to 1
 
META TOPICPARENT name="UserDocumentation"

CREAM User's Guide for EMI-2

Line: 1364 to 1364
 

Using an output file for the command glite-es-activity-create

Changed:
<
<
The activity creation command has the ability to write (just) the activity identifier(s) into an output file. When this file is first created the user has nothing to take care about. If the file already exists, the user has to make sure that the output file has been already used for the same endpoint of the current activity creation. En example is better than any explanation (the special ADL file ~/JDLs/activity_sleep60_multiple.adl cointains 3 descriptions for 3 activities):
>
>
The activity creation command has the ability to write the activity identifier(s) into an output file. If this file already exists, the user has to make sure that the output file has been already used for the same endpoint of the current activity creation. En example is better than any further explanation (the special ADL file ~/JDLs/activity_sleep60_multiple.adl cointains 3 descriptions for 3 activities):
 

$ glite-es-activity-create -e cream-05.pd.infn.it ~/JDLs/activity_sleep60_multiple.adl -o ids_cream-05 
Line: 1404 to 1404
 
Changed:
<
<
Note the option -o ids_cream-05 that instruct the command to write (only) the activity identifiers on the output file ids_cream-05. Let's take a look at this file:
>
>
Note the option -o ids_cream-05 that instructs the command to write (only) the activity identifiers on the output file ids_cream-05. Let's take a look at this file:
 
$ cat ids_cream-05 
Line: 1414 to 1414
 CR_ES276464760
Changed:
<
<
The header indicates the endpoint where the following identifiers has been created. If the user tries to create activites to a different endpoint but wants to write the output identifiers on the file ids_cream-05, he/she will get an error:
>
>
The header ##ACTIVITY_IDS@cream-05.pd.infn.it:8443## indicates the endpoint where the following identifiers has been created. If the user tries to create activites to a different endpoint but wants to write the output identifiers on the file ids_cream-05, he/she will get an error:
 

$ glite-es-activity-create -e cream-10.pd.infn.it ~/JDLs/activity_sleep60.adl -o ids_cream-05
Line: 1437 to 1437
 

Using an input file for the command glite-es-activity-status

Changed:
<
<
We've seen above that the command glite-es-activity-status accepts one or more activity identifiers as arguments. Alternatively, if the identifiers are many, it could be convenient to load them from an input file, previously created by the glite-es-activity-create command with the -o option. In this case the user must not specify the endpoint where the identifiers have been created, because the endpoint will be extracted from the input file:
>
>
We've seen above that the command glite-es-activity-status accepts one or more activity identifiers as arguments. Alternatively, if the identifiers are many, it could be convenient to load them from an input file (by mean of the -i option), previously created by the glite-es-activity-create command with the -o option. In this case the user must not specify the endpoint where the identifiers have been created, because the endpoint will be extracted from the input file:
 

$ glite-es-activity-status -e cream-05.pd.infn.it -i ids_cream-05 
Line: 1669 to 1669
  Where N must be >= 1 (i.e. at least one activity ID is a mandatory argument); unless the user specify an input file as explained below.
Changed:
<
<
The user can specify an input file (as for the glite-es-activity-status) instead of using <endpoint> and activity identifiers; or he/she can mixex input file and just activity identifiers. Then, the 3 possibilities of usage are:
>
>
The user can specify an input file (as for the glite-es-activity-status) instead of using <endpoint> and activity identifiers; or he/she can mixes input file and activity identifiers. Then, the 3 possibilities of usage are:
 

<command> -e cream-05.pd.infn.it CR_ES075634021 CR_ES342891609 CR_ES832095061
Line: 1687 to 1687
  -i cream-10_activity_list CR_ES081635022

Added:
>
>

Notifying service about user's operations

When the user's ADL contains files to be sent to the CE before the activity starts, or files that need to be retrieved after activity's termination, just after activity creation the CE puts the activity in a particular waiting state as we have seen above:
$ cat ~/JDLs/activity_files_push.adl 
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<CreateActivities>
    <ActivityDescription>

        <ActivityIdentification>
            <Name>CustomJob</Name>
            <Description>A job that to run needs a file to be sent and writes its elaboration on an output file to be retrieved.</Description>
            <Type>single</Type>
        </ActivityIdentification>

        <Application>
            <Executable>
                <Path>myjob.sh</Path>
                <Argument></Argument>
                <FailIfExitCodeNotEqualTo>0</FailIfExitCodeNotEqualTo>
            </Executable>
            <Environment>
              <Name>MY_ENV</Name>
              <Value>"my env"</Value>
            </Environment>
            <Error>JobError.txt</Error>
            <Output>JobOutput.txt</Output>
        </Application>
        <Resources>
            <QueueName>creamtest2</QueueName>
        </Resources>

        <DataStaging>

          <ClientDataPush>true</ClientDataPush>

          <OutputFile>
            <Name>JobError.txt</Name>
            <Target>
              <URI>gsiftp://cream-23.pd.infn.it//tmp/JobError.txt</URI>
              <DelegationID>04352064184726456</DelegationID>
            </Target>
          </OutputFile>
          <OutputFile>
            <Name>JobOutput.txt</Name>
            <Target>
              <URI>gsiftp://cream-23.pd.infn.it//tmp/JobOutput.txt</URI>
              <DelegationID>04352064184726456</DelegationID>
            </Target>
          </OutputFile>
        </DataStaging>

    </ActivityDescription>
</CreateActivities>

When the user submits this ADL the CE returns a particular status attribute saying that the CE is waiting for the user action (sending a file):

$ glite-es-activity-create --endpoint cream-52.pd.infn.it ~/JDLs/activity_files_push.adl
*****************************************
ActivityID     = CR_ES510328951
ActivityMgrURI = https://cream-52.pd.infn.it:8443/ce-cream-es/services/ActivityManagementService
Status         = PREPROCESSING
Status Attrs   = {CLIENT_STAGEIN_POSSIBLE}
Timestamp      = Tue Mar 27 09:22:05 2012
Description    = 
ETNSC          = 
STAGEIN Dir    = {gsiftp://cream-52.pd.infn.it/var/cream_es_sandbox/dteam/CN_Alvise_Dorigo_L_Padova_OU_Personal_Certificate_O_INFN_C_IT_dteam_Role_NULL_Capability_NULL_dteam021/51/CR_ES510328951/ISB}
SESSION Dir    = {}
STAGEOUT Dir   = {gsiftp://cream-52.pd.infn.it/var/cream_es_sandbox/dteam/CN_Alvise_Dorigo_L_Padova_OU_Personal_Certificate_O_INFN_C_IT_dteam_Role_NULL_Capability_NULL_dteam021/51/CR_ES510328951/OSB}

We've already seen above that the user will have to send the myjob.sh executable into the "STAGEIN Dir" and then notify the CE:

$ glite-es-notify-service -e cream-52.pd.infn.it CR_ES510328951:CLIENT-DATAPUSH-DONE

The syntax of the glite-es-notify-service command is simple: the endpoint is the same for all the other commands; the argument CR_ES510328951:CLIENT-DATAPUSH-DONE is simply the <Activity_Identifier>:<ACTION_TO_NOTIFY>. The <Activity_Identifier> is to indicate which activity in the CE is affected by the action performed by the user; the <ACTION_TO_NOTIFY> can have at the moment only these two values:

  • CLIENT-DATAPUSH-DONE
  • CLIENT-DATAPULL-DONE

We've just seen the meaning of the former. The CLIENT-DATAPULL-DONE is to be sent when the activity is terminated, and output files, available on the CE in the "STAGEOUT Dir", have been retrieved by the user.

 
 
This site is powered by the TWiki collaboration platformCopyright © 2008-2023 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback