Tags:
, view all tags

WMS Test Plan

YAIM-WMS Configuration Testing

WMS Job Submission/GetOutput Testing

Registers the job into the wmproxy, delivers it to the WM that submits it to the grid. Test job submission with the following type of jobs/jdl:

Normal Job

  • to JC
    [
    Type = "Job";
    JobType = "normal";
    InputSandbox = { "file://$HOME/test.sh"};
    VirtualOrganisation = "dteam";
    Executable="test.sh";
    Arguments="Hello ";
    Requirements = ( !RegExp("/cream-",other.GlueCEUniqueID));
    Rank = 0;
    fuzzyrank = true;
    StdOutput="message.txt";
    StdError="err.log";
    OutputSandbox={"message.txt","err.log",".BrokerInfo"};
    RetryCount = 0;
    usertags = [ jdl = "normal job" ];
    ShallowRetryCount = 3;
    ]
  • to ICE
    [
    Type = "Job";
    JobType = "normal";
    InputSandbox = { "file:///home/emolinari/test.sh"};
    VirtualOrganisation = "dteam";
    Executable="test.sh";
    Arguments="Hello ";
    Requirements = ( RegExp("/cream-",other.GlueCEUniqueID));
    Rank = 0;
    fuzzyrank = true;
    StdOutput="message.txt";
    StdError="err.log";
    OutputSandbox={"message.txt","err.log",".BrokerInfo"};
    usertags = [ jdl = "normal job to ICE" ];
    RetryCount = 0;
    ShallowRetryCount = 3;
    ]

Perusal job

Job perusal is the ability to view output from a job while it is running. In order to use this feature the JDL attributes PerusalFileEnable and PerusalTimeInterval must be declared when the job is submitted for example:
PerusalFileEnable = true;
PerusalTimeInterval = 120;
This makes the WN upload, at regular time intervals (PerusalTimeInterval), a copy of the output files specified using the glite-wms-job-perusal --set command to the WMS machine.
  • Following an example on how to test a perusal job:
    • Given the jdl file:
      [
      Executable = "perusal.sh" ;
      InputSandbox = {"file://$HOME/perusal.sh"};
      Stdoutput = "perusal.out" ;
      StdError =  "perusal.err" ;
      OutputSandbox ={ "perusal.out" ,"perusal.err"} ;
      usertags = [ jdl = "perusal" ];
      PerusalFileEnable = true;
      ]
      where perusal.sh is
      #!/bin/bash
      
      echo "Hi all, this is a perusal test program..."
      
      for (( i=0 ; $i < 50 ; i = $(($i + 1)) )); do
      
              echo "Step number $i"
              date
              sleep 10
      
      done
      
      echo ""
      echo "Bye"
    • Submit the job with
       glite-wms-job-submit --config glite_wms.conf -a perusal.jdl
    • Given the jobid of the previously submitted set the files that are to be perused with
      glite-wms-job-perusal --set -f perusal.out https://devel17.cnaf.infn.it:9000/HBPb0WwhMfhKbjdUkrasWg
    • When the job starts running on the WN get the files as here:
      glite-wms-job-perusal --get -f perusal.out https://devel17.cnaf.infn.it:9000/HBPb0WwhMfhKbjdUkrasWg

DAG job

to JC and IC as above specifying Requirements = ( RegExp("/cream-",other.GlueCEUniqueID)); or Requirements = ( RegExp("/cream-",other.GlueCEUniqueID)); respectively in the job jdl
  • DAG job jdls samples:
    [
    Type = "dag";
    VirtualOrganisation = "dteam";
    Max_nodes_running = 10;
    InputSandbox = "test.sh";
    FuzzyRank = true;
    Nodes = [
    nodeA = [
    file= "test_dag.jdl";
    ];
    nodeB = [
    file= "test_dag.jdl";
    ];
    nodeC = [
    file= "test_dag.jdl";
    ];
    nodeD = [
    file= "test_dag.jdl";
    ];
    nodeE = [
    file= "test_dag.jdl";
    ];
    nodeF= [
    file= "test_dag.jdl";
    ];
    ];
    Dependencies = {
    {{nodeA, nodeB}, nodeC},{nodeD,nodeE.nodeF}
    }
    ]
    [
        type = "dag";
        DefaultNodeShallowRetryCount = 3;
            nodes = [
          nodeA = [
            node_type = "edg-jdl";
            file ="myjob.jdl" ;
          ];
            nodeB = [
                                    node_type = "edg-jdl";
            file ="myjob.jdl" ;
           ];
             nodeC = [
             node_type = "edg-jdl";
             file ="myjob.jdl" ;
           ];
                            dependencies = {
            { nodeA, nodeB },
                                    { nodeA, nodeC }
           }
         ];
      ]

Parametric Job

Submit it to ICE and JC. Following a sample jdl:
// Test parametric job.
// The standar Output and Error are parametrizied such as the
// enviroment variable: MYPATH_PARAM_.
 [
  JobType = "parametric";
  Executable = "/usr/bin/env";
  Environment = {"MYPATH_PARAM_=$PATH:/bin:/usr/bin:$HOME"};
  StdOutput = "echo_PARAM_.out";
  StdError = "echo_PARAM_.err";
  OutputSandbox = {"echo_PARAM_.out","echo_PARAM_.err"};
  Requirements = ( RegExp("/cream-",other.GlueCEUniqueID));
  Parameters =  5;
        usertags = [ jdl = "parametric cream" ];
 ]

Collection Job

WMS Job shallow and deep re-submission

  • For the shallow resubmission add the following attributes to your favourite jdl file:
    Prologue = "/bin/false"; ShallowRetryCount = 3;
  • For the deep resubmission add the following attributes:
     Prologue = "/bin/false"; RetryCount = 3;

WMS Job List-match Testing

-- ElisabettaMolinari - 2010-02-24

Edit | Attach | PDF | History: r30 | r6 < r5 < r4 < r3 | Backlinks | Raw View | More topic actions...
Topic revision: r4 - 2010-02-25 - ElisabettaMolinari
 
  • Edit
  • Attach
This site is powered by the TWiki collaboration platformCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback