Difference: TestPlan (1 vs. 2)

Revision 22011-07-13 - AlessioGianelle

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
Line: 11 to 10
 

Deployment tests

Changed:
<
<

Repository

The EMI-1 RC4 repository can be found under:

http://emisoft.web.cern.ch/emisoft/dist/EMI/1/RC4/sl5/x86_64
>
>

Generic repository

 
Deleted:
<
<
Other repositories:
 
  • epel.repo
Changed:
<
<
  • lcg-CA.repo
>
>
  • EGI-trustanchors
 
  • sl.repo
  • sl-security.repo

Installation test

Changed:
<
<
First of all, install the yum-protectbase rpm:
yum install yum-protectbase.noarch
>
>
First of all, install the yum-protectbase rpm: yum install yum-protectbase.noarch
  Then proceed with the installation of the CA certificates by issuing:
Deleted:
<
<
yum install ca-policy-egi-core
 
Changed:
<
<
Install the WMS metapackage:
yum install emi-wms
>
>
yum install ca-policy-egi-core
 
Changed:
<
<
Configure the WMS:
/opt/glite/yaim/bin/yaim -c -s site-info.def -n WMS  
>
>
Install the WMS metapackage:
 
Changed:
<
<

Update test

>
>
yum install emi-wms
 
Changed:
<
<
N/A
>
>
After the definition of the site-info.def file configure the WMS:
 
Changed:
<
<

Functionality tests

>
>
/opt/glite/yaim/bin/yaim -c -s site-info.def -n WMS
 
Changed:
<
<

Features/Scenarios to be tested

>
>

Update test

 
Changed:
<
<

YAIM-WMS Configuration Testing

  • Installation and configuration starting from a cleaning machine (i.e. only OS)
  • Update and configuration from a previous version
>
>
Starting from a production WMS add the patch repository then issue:
 
Changed:
<
<

WMS Job Submission/GetOutput Testing

Submit a job to the WMS service and when finished retrieve the output. Test job submission with the following type of jobs:
>
>
yum update
 
Added:
>
>
If necessary reconfigure the WMS:
 
Changed:
<
<
Bunch #1: submission of normal jobs.
>
>
/opt/glite/yaim/bin/yaim -c -s site-info.def -n WMS
 
Changed:
<
<
Test 1.1: Test this JDL:
>
>

Functionality tests

 
Changed:
<
<
[ Executable = "/bin/echo"; Arguments = "Hello"; StdOutput = "out.log"; StdError = "err.log"; InputSandbox = {"Test.sh"}; OutputSandbox = {"out.log", "err.log"}; requirements = RegExp("cream.*", other.GlueCEUniqueID);; AllowZippedISB = false; rank=0; myproxyserver="myproxy.cnaf.infn.it"; RetryCount = 0; ShallowRetryCount = 1; ]
>
>

Features/Scenarios to be tested

 
Changed:
<
<
Test 1.10 submit a job with an empty file in ISB
>
>

Test job cycle (from submission to output retrieve)

 
Changed:
<
<
Normal Job
  • Test submission of normal jobs with different options and situation Implemented
>
>
Submit a job to the WMS service and when finished retrieve the output; a the end the final status of the jobs should be Cleared.
 
Changed:
<
<
  • Test the complete cycle with the two types of CEs: lcg and Cream Implemented
>
>
Test job submission with the following type of jobs:
 
Added:
>
>
Normal Job
  • Test the complete cycle submitting to the two types of CE: lcg and Cream Implemented.
 More different jdls can added in the future.

Perusal job
Changed:
<
<
Job perusal is the ability to view output from a job while it is running. Implemented
>
>
Job perusal is the ability to view output from a job while it is running. Implemented.
 
DAG job
Changed:
<
<
Directed Acyclic Graphs (a set of jobs where the input/output/execution of one of more jobs may depend on one or more other jobs).

  • Try with JDL:

[
  type = "dag";
  DefaultNodeShallowRetryCount = 3;
  nodes = [
    nodeA = [
      node_type = "edg-jdl";
      file ="jdl/arg.jdl" ; 
    ];
    nodeB = [
      node_type = "edg-jdl";
      file ="jdl/arg.jdl" ; 
    ];
    nodeC = [
      node_type = "edg-jdl";
      file ="jdl/arg.jdl" ; 
    ];
    dependencies = {
      { nodeA, nodeB },
      { nodeA, nodeC }
    }
  ];
]

  • When dag finishes retrieve the output files
  • Check the final status of the dag (all nodes and parent should be "Cleared")
>
>
Directed Acyclic Graphs (a set of jobs where the input/output/execution of one of more jobs may depend on one or more other jobs). Implemented.
 
Added:
>
>
  • Also the nodes should be in state Cleared
 More different jdls can added in the future.

Parametric Job
Changed:
<
<
Multiple jobs with one parametrized description. Implemented
>
>
Multiple jobs with one parametrized description. Implemented.
 
Collection Job
Changed:
<
<
Multiple jobs with a common description. There are two ways to submit collection: you can create a single jdl with all the jdls of nodes or you can submit all the jdls stored in a directory (bulk submission)

  • Submit a jdl like this one:
[
nodes = {
   [
   file="jdl/arg.jdl";
   ],
   [
  executable="/bin/env";
  ShallowRetryCount = 0;
  RetryCount = 0;
  Stdoutput = "file.out" ;
  StdError =  "file.err" ;
  OutputSandbox ={ "file.out" ,"file.err"} ;
  FuzzyRank = true;
   ],
   [
   NodeName="nodeA";
   executable="/bin/ls" ;
  Stdoutput = "file.out" ;
  OutputSandbox ={ "file.out"} ;
   ]
};
Type = "Collection" ;
requirements =  other.GlueCEStateStatus == "Production" ;
rank = -other.GlueCEStateEstimatedResponseTime ;
]

  • When collection finishes retrieve the output files
  • Check the final status of the collectionall nodes and parent should be "Cleared")

  • To test bulk submission use option "--collection" of glite-wms-job-submit command.
  • When collection finishes retrieve the output files
  • Check the final status of the collection (all nodes and parent should be "Cleared")

More different jdls can added in the future.

>
>
Multiple jobs with a common description. There are two ways to submit collection:
  • you can create a single jdl with all the jdls of nodes TBD
  • you can submit all the jdls stored in a directory (bulk submission) Implemented.
 
Parallel Job
Changed:
<
<
Jobs that can be running in one or more cpus in parallel.

  • Submit a jdl like this one:
[
Executable = "cpi";
CpuNumber = 2;
Stdoutput = "cpi.out" ;
StdError =  "cpi.err" ;
OutputSandbox = { "cpi.out" ,"cpi.err"} ;
InputSandbox = { "exe/cpi" };
FuzzyRank = true;
usertags = [ exe = "cpi" ];
]

  • When job finishes retrieve the output files
  • Check the final status of the job
>
>
Jobs that can be running in one or more cpus in parallel. Implemented.
 

Delegation

Deleted:
<
<
Explicit delegation, automatic delegation
 
Changed:
<
<

Brokerinfo

>
>
  • There are two types of delegation: the automatic ones or you can delegate before submission. Submit jdls using both methods Implemented.
 
Changed:
<
<

WMS Job shallow and deep re-submission

>
>
  • Make a delegation with an expired proxy. Command should fails. Implemented.
 
Changed:
<
<
There two type of resubmission; the first is defined deep occurs when the user's job has stardted running on the WN and then the job itself or the WMS JobWrapper has failed. The second one is called shallow and occurs when the WMS JobWrapper has failed before starting the actual user's job. Implemented
>
>
  • Submit with an expired delegation. Command should fails. Implemented.
 
Changed:
<
<

Replanning

MaxReplansCount in glite_wms.conf
>
>

Shallow and deep re-submission

 
Changed:
<
<

WMS Job List-match Testing

>
>
There two type of resubmission; the first is defined deep occurs when the user's job has stardted running on the WN and then the job itself or the WMS JobWrapper has failed. The second one is called shallow and occurs when the WMS JobWrapper has failed before starting the actual user's job. Implemented.
 
Changed:
<
<
Without data
>
>

Job List-match Testing

 
Changed:
<
<
Test job-list-command and its option Implemented
>
>
Test various matching requests Implemented.
 
Changed:
<
<
With data
>
>
With data

Test matchmaking using data requests (gang matchmaking) TBD

 
  • You need to register a file on an SE, then submit a jdl like this one (as InputData put the lfn(s) registered before):
Line: 239 to 126
 }; DataAccessProtocol = "gsiftp";
Deleted:
<
<
 
  • Then try a list-match, the listed CEs should be the ones "close" to the used SE

WMS Job Cancel Testing

Changed:
<
<
Test the cancellation of these type of jobs (final status should be cleared):

Normal job

Submit and cancel a normal job Implementd

DAG job

Submit a dag job and then cancel it (the parent)

Collection
>
>
Test the cancellation of these type of jobs (final status should be Cancelled):
 
Changed:
<
<
Submit a collection job and then cancel it (the parent)
>
>
  • Submit and cancel a normal job Implemented.
 
Changed:
<
<
Node of a collection
>
>
  • Submit a dag job and then cancel it (the parent) Implemented.
 
Changed:
<
<
Submit a collection job and then some of its nodes
>
>
  • Submit a dag job and then cancel some of its nodes Implemented.
 
Changed:
<
<

Others

>
>
  • Submit a collection job and then cancel it (the parent) Implemented.
 
Changed:
<
<
Delegation Testing
>
>
  • Submit a collection job and then cancel some of its nodes Implemented.
 
Changed:
<
<
Test the delegation command and its options Implementd
>
>
  • Cancellation of a Done job should fails. Implemented.
 
Changed:
<
<
Job-info Testing
>
>

Prologue and Epilogue jobs

 
Changed:
<
<
Test the job-info command and its options Implementd
>
>
In the jdl you can specify two attributes prologue and epilogue which are scripts that are execute respectively before and after the user's job. Implemented.
 
Changed:
<
<
Logging-info Testing

Test the logging-info command and its options Implemented

Job Status Testing

Test the job-status commend and its options Implemented

Prologue and Epilogue jobs

In the jdl you can specify two attributes prologue and epilogue which are scripts that are execute respectively before and after the user's job. Implemented

>
>

Proxy renewal

 
Added:
>
>
  • Submit a long job with myproxyserver set using a short proxy to both CE (lcg and CREAM). Job should finishes Done (Success) Implemented.
  • Submit a long job without setting myproxyserver using a short proxy to both CE (lcg and CREAM). Job should finishes Aborted with reason "proxy expired" Implemented.
 

Performance tests

Line: 300 to 168
 
  • Four users
  • Use LCG-CEs and CREAM-CEs (with different batch systems)
  • Use automatic-delegation
Changed:
<
<
  • The job is a "sleep random(672)"
>
>
  • The job is a "sleep random(666)"
 
  • Resubmission is enabled
  • Enable proxy renewal

Regression tests

Complete list of Rfc tests

Added:
>
>

Note

Implemented. means that an automatic test exists. Otherwise test must be execute by hand.

Revision 12011-07-13 - AlessioGianelle

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="WebHome"

WMS Test Plan

Unit tests

N/A

Deployment tests

Repository

The EMI-1 RC4 repository can be found under:

http://emisoft.web.cern.ch/emisoft/dist/EMI/1/RC4/sl5/x86_64

Other repositories:

  • epel.repo
  • lcg-CA.repo
  • sl.repo
  • sl-security.repo

Installation test

First of all, install the yum-protectbase rpm:

yum install yum-protectbase.noarch

Then proceed with the installation of the CA certificates by issuing:

yum install ca-policy-egi-core

Install the WMS metapackage:

yum install emi-wms

Configure the WMS:

/opt/glite/yaim/bin/yaim -c -s site-info.def -n WMS  

Update test

N/A

Functionality tests

Features/Scenarios to be tested

YAIM-WMS Configuration Testing

  • Installation and configuration starting from a cleaning machine (i.e. only OS)
  • Update and configuration from a previous version

WMS Job Submission/GetOutput Testing

Submit a job to the WMS service and when finished retrieve the output. Test job submission with the following type of jobs:

Bunch #1: submission of normal jobs.

Test 1.1: Test this JDL:

[ Executable = "/bin/echo"; Arguments = "Hello"; StdOutput = "out.log"; StdError = "err.log"; InputSandbox = {"Test.sh"}; OutputSandbox = {"out.log", "err.log"}; requirements = RegExp("cream.*", other.GlueCEUniqueID);; AllowZippedISB = false; rank=0; myproxyserver="myproxy.cnaf.infn.it"; RetryCount = 0; ShallowRetryCount = 1; ]

Test 1.10 submit a job with an empty file in ISB

Normal Job
  • Test submission of normal jobs with different options and situation Implemented

  • Test the complete cycle with the two types of CEs: lcg and Cream Implemented

More different jdls can added in the future.

Perusal job

Job perusal is the ability to view output from a job while it is running. Implemented

DAG job

Directed Acyclic Graphs (a set of jobs where the input/output/execution of one of more jobs may depend on one or more other jobs).

  • Try with JDL:

[
  type = "dag";
  DefaultNodeShallowRetryCount = 3;
  nodes = [
    nodeA = [
      node_type = "edg-jdl";
      file ="jdl/arg.jdl" ; 
    ];
    nodeB = [
      node_type = "edg-jdl";
      file ="jdl/arg.jdl" ; 
    ];
    nodeC = [
      node_type = "edg-jdl";
      file ="jdl/arg.jdl" ; 
    ];
    dependencies = {
      { nodeA, nodeB },
      { nodeA, nodeC }
    }
  ];
]

  • When dag finishes retrieve the output files
  • Check the final status of the dag (all nodes and parent should be "Cleared")

More different jdls can added in the future.

Parametric Job

Multiple jobs with one parametrized description. Implemented

Collection Job

Multiple jobs with a common description. There are two ways to submit collection: you can create a single jdl with all the jdls of nodes or you can submit all the jdls stored in a directory (bulk submission)

  • Submit a jdl like this one:
[
nodes = {
   [
   file="jdl/arg.jdl";
   ],
   [
  executable="/bin/env";
  ShallowRetryCount = 0;
  RetryCount = 0;
  Stdoutput = "file.out" ;
  StdError =  "file.err" ;
  OutputSandbox ={ "file.out" ,"file.err"} ;
  FuzzyRank = true;
   ],
   [
   NodeName="nodeA";
   executable="/bin/ls" ;
  Stdoutput = "file.out" ;
  OutputSandbox ={ "file.out"} ;
   ]
};
Type = "Collection" ;
requirements =  other.GlueCEStateStatus == "Production" ;
rank = -other.GlueCEStateEstimatedResponseTime ;
]

  • When collection finishes retrieve the output files
  • Check the final status of the collectionall nodes and parent should be "Cleared")

  • To test bulk submission use option "--collection" of glite-wms-job-submit command.
  • When collection finishes retrieve the output files
  • Check the final status of the collection (all nodes and parent should be "Cleared")

More different jdls can added in the future.

Parallel Job

Jobs that can be running in one or more cpus in parallel.

  • Submit a jdl like this one:
[
Executable = "cpi";
CpuNumber = 2;
Stdoutput = "cpi.out" ;
StdError =  "cpi.err" ;
OutputSandbox = { "cpi.out" ,"cpi.err"} ;
InputSandbox = { "exe/cpi" };
FuzzyRank = true;
usertags = [ exe = "cpi" ];
]

  • When job finishes retrieve the output files
  • Check the final status of the job

Delegation

Explicit delegation, automatic delegation

Brokerinfo

WMS Job shallow and deep re-submission

There two type of resubmission; the first is defined deep occurs when the user's job has stardted running on the WN and then the job itself or the WMS JobWrapper has failed. The second one is called shallow and occurs when the WMS JobWrapper has failed before starting the actual user's job. Implemented

Replanning

MaxReplansCount in glite_wms.conf

WMS Job List-match Testing

Without data

Test job-list-command and its option Implemented

With data

  • You need to register a file on an SE, then submit a jdl like this one (as InputData put the lfn(s) registered before):
###########################################
#      JDL with Data Requirements         #
###########################################

Executable = "calc-pi.sh";
Arguments = "1000";
StdOutput = "std.out";
StdError = "std.err";
Prologue = "prologue.sh";
InputSandbox = {"calc-pi.sh", "fileA", "fileB","prologue.sh"};
OutputSandbox = {"std.out", "std.err","out-PI.txt","out-e.txt"};
Requirements = true;

DataRequirements = {
[
DataCatalogType = "DLI";
DataCatalog = "http://lfcserver.cnaf.infn.it:8085";
InputData = {"lfn:/grid/infngrid/cesini/PI_1M.txt","lfn:/grid/infngrid/cesini/e-2M.txt"};
]
};
DataAccessProtocol = "gsiftp";

  • Then try a list-match, the listed CEs should be the ones "close" to the used SE

WMS Job Cancel Testing

Test the cancellation of these type of jobs (final status should be cleared):

Normal job

Submit and cancel a normal job Implementd

DAG job

Submit a dag job and then cancel it (the parent)

Collection

Submit a collection job and then cancel it (the parent)

Node of a collection

Submit a collection job and then some of its nodes

Others

Delegation Testing

Test the delegation command and its options Implementd

Job-info Testing

Test the job-info command and its options Implementd

Logging-info Testing

Test the logging-info command and its options Implemented

Job Status Testing

Test the job-status commend and its options Implemented

Prologue and Epilogue jobs

In the jdl you can specify two attributes prologue and epilogue which are scripts that are execute respectively before and after the user's job. Implemented

Performance tests

Collection of 1000 nodes

Submit a collection of 1000 nodes.

Stress test

This could be an example of stress test

  • 2880 collections each of 20 jobs
  • One collection every 60 seconds
  • Four users
  • Use LCG-CEs and CREAM-CEs (with different batch systems)
  • Use automatic-delegation
  • The job is a "sleep random(672)"
  • Resubmission is enabled
  • Enable proxy renewal

Regression tests

Complete list of Rfc tests

 
This site is powered by the TWiki collaboration platformCopyright © 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback