CREAM Test work plan
Unit tests
TBD
Deployment tests
Installation
The following should be tested:
- Fresh installation of a CREAM-CE node, doing what is documented in the CREAM System administratror guide (Sections: "Installation of a CREAM CE node in no cluster mode" and "Installation of a CREAM CE node in cluster mode")
- Update from a previous version of the same EMI major release. This should be tested doing a simple
yum update
.
- Update from a previous EMI major release. TBD
The following installation and update scenarios should be tested:
- CREAM CE in no cluster mode
- CREAM CE in cluster mode with gLite-cluster deployed in the same CREAM-CE node
Configuration
The configuration via yaim should be tested.
The following scenarios should be tested
(they are documented in CREAM System Administrator Guide):
- CREAM CE configured in no cluster mode
- CREAM configured in cluster mode with gLite-cluster deployed in the same CREAM-CE node
- CREAM configured in cluster mode with gLite-cluster deployed on another node
- CREAM configured using gJAF as authorization system
- CREAM configured using ARGUS as authorization system
- CREAM using the new BLAH blparser
- CREAM using the old BLAH blparser
- CREAM using Torque as batch system
- CREAM using LSF as batch system
- CREAM using GE as batch system
System tests
Basic functionality tests for legacy interface
Simple Submission Test
Submit a job which simply does "/bin/uname -a".
Check the final job status which should be DONE-OK.
Status:
Implemented in the Robot based testsuite
Execute a bash shell script.
The script is stored on the UI.
The jdl attribute
InputSandbox is used.
Firstly submit the jdl and then wait for final job state which should be DONE-OK.
Status:
Implemented in the Robot based testsuite
Execute a bash shell script.
The script is stored in a gridftp server.
The jdl attribute
InputSandbox is used.
First upload the executable script.
Then submit the jdl and check the the final state which should be DONE-OK.
Status:
Implemented in the Robot based testsuite
Submit a job where the executable is a bash shell script.
The script is stored in a gridftp server.
The jdl attribute
InputSandboxBaseURI is used.
First upload the executable script in the considered gridftp server.
Then submit the jdl.
Check the final state which should be DONE-OK.
Status:
Implemented in the Robot based testsuite
OutputSandbox localhost and get-output Test
- Write a simple job which executes =/bin/uname = and Ssore std out and error streams on "gsiftp://localhost"
- Submit the jdl and wait for final job state which should be DONE-OK
- Download the produced files with glite-ce-job-ouput
- Get the output file list from the jdl file and compare the two file lists
Status:
Implemented in the Robot based testsuite
- Write a simple job which executes
/bin/uname -a
and stores std out and error streams on a gridftp server.The jdl attribute OutputSandboxDestURI is used.
- Get output file list from jdl,base dest uri.
- lcg-ls base dest uri and files shouldn't exist
- Submit job and wait for its completion
- lcg-ls base dest uri and files should exist
- Delete files (cleanup)
- lcg-ls base dest uri and files shouldn't exist
Status:
Implemented in the Robot based testsuite
- Write a simple job which simply executes
/bin/uname -a.
and stores std out and error streams on a gridftp server. The jdl attribute OutputSandboxBaseDestURI is used.
- Get output file list from jdl base dest uri.
- lcg-ls base dest uri and files shouldn't exist
- Submit job and wait for its completion
- lcg-ls base dest uri and verify that files exist
- Delete files (cleanup)
- lcg-ls base dest uri and files shouldn't exist
Status:
Implemented in the Robot based testsuite
Try
OutputData JDL attributes considering all the possible combinations.
To do that, you can consider a JDL such as this one:
[
executable="/bin/sleep";
arguments="25";
Stdoutput = "env.out" ;
StdError = "env.err" ;
InputSandbox = {
"/etc/passwd",
"ssh1.sh",
"gsiftp://cream-18.pd.infn.it:2811/etc/hostname.txt",
"file:///etc/fstab"
};
outputsandbox={"out.out", "passwd", "ssh1.sh", "hostname.txt", "fstab"};
OutputSandboxBaseDestURI="gsiftp://localhost";
OutputData={
[
OutputFile="passwd";
LogicalFileName = "lfn:/sgaravat/passwd";
],
[
OutputFile="hostname.txt";
],
[
OutputFile="ssh1.sh";
StorageElement="prod-se-01.pd.infn.it";
],
[
OutputFile="fstab";
LogicalFileName="lfn:/sgaravat/fstab";
StorageElement="prod-se-01.pd.infn.it";
]
};
]
Once the job is in
DONE-OK
, retrieve the OSB files using the
glite-ce-job-output
command.
Check the result of the data upload operations in the file
DSUpload_.out
Not Implemented
Sandbox staging Test
This test case tests the two different Sandbox staging operations available for the /etc/glite-ce-cream/cream-config.xml, LRMS and GSIFTP.
- Change the setting (if GSIFTP, set to LRMS, if LRMS, set to GSIFTP)
- Run the "OutputSandbox localhost and get-output" Test
- Reset the setting to its original and rerun the "OutputSandbox localhost and get-output" Test
Status:
Implemented in the Robot based testsuite
Delegation tests
- Delegate a proxy with glite-ce-delegaet-proxy and submit a job referring to the previously delegate proxy. Job final state should be DONE-OK
- Try to delegate a a proxy using an already used delegationid. This should fail
- Try to submit a job referring to a non-existing delegationid. This should fail
Status:
Implemented in the Robot based testsuite
Environment test
Consider a job where the jdl sets an environmental variable and the executable is a script which prints the value of that variable.
Submit the jdl.
Check the the final state which should be DONE-OK.
Check the output file which should have the designated variable.
Status:
Implemented in the Robot based testsuite
Epilogue test
Execute two bash shell scripts, one for the job and the other as an epilogue.
The epilogue script creates a file with a certain string, which is collected by the job as an output file
Submit the jdl.
Check the final state which should be DONE-OK.
Check the output file which should have the designated string.
Status:
Implemented in the Robot based testsuite
Prologue test
Execute two bash shell scripts, one for the job and the other as a prologue.
The prologue script creates a file with a certain string, which is collected by the job as an output file
Submit the jdl.
Check the final state which should be DONE-OK.
Check the output file which should have the designated string.
Status:
Implemented in the Robot based testsuite
CPU Allocation Test 1
Submit a job where in the JDL there is:
WholeNodes=true;
SMPGranularity=G;
Hostnumber=H;
with H > 1.
Verify that in the submission script there is
BSUB -n S*H
BSUB -R "span[ptile=S]
BSUB -x
PBS -l nodes=H:ppn=S
PBS -W x=NACCESSPOLICY:SINGLEJOB
with S equal to the value published as
GlueHostArchitectureSMPSize
.
See:
http://wiki.italiangrid.it/twiki/bin/view/CREAM/TroubleshootingGuide#5_1_Saving_the_batch_job_submiss
to understand how to save and check the submission scripts.
Status:
Not Implemented
CPU Allocation Test 2
Submit a job where in the JDL there is:
WholeNodes=true;
SMPGranularity=G;
Verify that in the submission script there is
BSUB -n S
BSUB -R "span[hosts=1]"
BSUB -x
PBS -l nodes=1:ppn=S
PBS -W x=NACCESSPOLICY:SINGLEJOB
with S equal to the value published as
GlueHostArchitectureSMPSize
.
Status:
Not Implemented
CPU Allocation Test 3
Submit a job where in the JDL there is:
WholeNodes=true;
HostNumber=H;
with H>1.
Verify that in the submission script there is
BSUB -n S*H
BSUB -R "span[ptile=S]"
BSUB -x
PBS -l nodes=H:ppn=S
PBS -W x=NACCESSPOLICY:SINGLEJOB
with S equal to the value published as
GlueHostArchitectureSMPSize
.
Status:
Not Implemented
CPU Allocation Test 4
Submit a job where in the JDL there is:
WholeNodes=false;
SMPGranularity=G;
CPUNumber=C;
Verify that in the submission script there is
BSUB -n C
BSUB -R "span[ptile=G]"
PBS -l nodes=N:ppn=G { [+1:ppn=R] if r>0 }
with:
N = C / G
R = C % G
Status:
Not Implemented
CPU Allocation Test 5
Submit a job where in the JDL there is:
WholeNodes=false;
HostNumber=H;
CPUNumber=C;
with H>1.
Verify that in the submission script there is
BSUB -n C
BSUB -R "span[ptile={ N if R=0 ; N+1 if R>0 }]"
PBS -l nodes=H-R:ppn=N { [+R:ppn=N+1] if R>0 }
with:
N = C / H
R = C % H
Status:
Not Implemented
CPU Allocation Test 6
Submit a job where in the JDL there is:
WholeNodes=false;
CPUNumber=C;
Verify that in the submission script there is
BSUB -n C
PBS -l nodes=C
Status:
Not Implemented
Verify the proper functionality of the glite-ce-job-status command with different level of verbosity and different options.
Verify that with
-L 0
it is reported the "Status".
Verify that with
-L 1
it is reported the "Status", the "Job status changes" and the "Issued commands".
Verify that with
-L 1
it is reported the "Status", the "Job status changes", the "Issued commands", the "Worker Node", the "Deleg Proxy ID", the "DelegProxyInfo", "Local User", "JDL".
Verify the status of 2 jobs, with ids specified in the command line and with ids read from a file (option
-i
).
Verify the proper functionality of the
--from
and
-to
option. Specify a time range where a single job was submitted, and verify that only the status of this job is returned.
Status:
Not Implemented
List test
Perform a glite-ce-job-list.
Perform a job submission
Perform again a glite-ce-job-list
The returned jobid shouldn't exist in the in the job list command output before the submission and should exist afterwards.
Status:
Implemented in the Robot based testsuite
Cancel test
- Submit a job
- Cancel it using the
glite-ce-job-cancel
command
- Check job status: it should eventually be
CANCELLED
- Try to cancel again the aforementioned job. A failure should be returned
Status:
Implemented in the Robot based testsuite
Manual cancel test
- Submit a job and then manually qdel/bkill it on the CE host
- Check the final status which should be either "DONE-FAILED" or "CANCELLED"
Status:
Implemented in the Robot based testsuite
Suspend/Resume Test
Submit a job.
Wait until it's state is REALLY-RUNNING and then suspend it.
Case LSF batch system: It's state should be HELD. Then resume the same job.It's status should be REALLY-RUNNING.The final status should be DONE-OK.
Case Torque batch system: It's state should still be REALLY-RUNNING.Resume the job and expect an error.It's status should be REALLY-RUNNING.The final status should be DONE-OK.
Status:
Implemented in the Robot based testsuite
Enable/disable job submission test
his test case tests the proper functionality of the glite-ce-allowed-submission and glite-ce-enable/disable-submission commands, for admin and non-admin users. The test executes the following steps:
- Check with
glite-ce-allowed-submission
if submissions are enabled (they should)
- Executes a simple job (it should succeed)
- Try to Disable submission using
glite-ce-disable-submission
command considering that the current user is not a CREAM admin. This should fail
- Add the current user as a CREAM admin
- Try to disable submission using
glite-ce-disable-submission
command. This time it should succeed)
- Check with
glite-ce-allowed-submission
if submission is allowed (it shouldn't)
- Try a simple job submission (it should fail)
- Enable submission with
glite-ce-enable-submission
command
- Try the submission of a simple job (it should succeed)
- Remove current user from CREAM admin
- Try to disable submission using glite-ce-disable-submission= command (should fail, since the user isn't a CREAM admin anymore).
Status:
Implemented in the Robot based testsuite
Purge test
- Purges a job
- Then searches for its status and its sandbox dirs on the cream node. The job shouldn't be found by the status command and the sandbox dirs shouldn't exist.
Status:
Implemented in the Robot based testsuite
Automatic job purger test
Modify the
JOB_PURGE_POLICY
attribute in the configuration file
/etc/glite-ce-cream/cream-config.xml
setting
0 days
for
CANCELLED
.Then restart tomcat.
Submit a long job and then cancel it. Verify that the job status is cancelled.
After 24 hours recheck the status of the job with
glite-ce-job-status
. The job should not be found. Verify in the CREAM CE in the CREAM sandbox dir that a directory for this job doesn't exist anymore.
Status:
Not Implemented
Proxy renewal test
Verify the functionality of the
glite-ce-proxy-renew
command.
Submit a job lasting 20 minutes using a proxy valid 10 minutes.
After 10 minutes create on the UI a long proxy, and renew the proxy with the
glite-ce-proxy-renew
command.
Verify the final job status which should eventually be
DONE-OK
Status:
Not Implemented
Automatic proxy purger test
Delegate a proxy valid 1 hour
Submit a job using that delegatioid. This should work.
Try again the submission using the same delegationid after 24 hours. This should not work (the error message should say that the delegation was not found)
Status:
Not Implemented
Authz test
Try functionality of CREAM with both Argus based authorizations and gJAF based authorizations.
To do that simply perform the "Simple Submission Test", "Delegation test", "List test" and "Cancel test" first using a CREAM configured using gJAF, and then using a CREAM configured using ARGUS.
Resource BDII test
Check if the resource bdii publishes all the relevant objectclasses for both glue1 and glue2
- Check if the resource BDII publishes glue 1 GlueCE objectclasses. There should be one GlueCE objectclass per each queue:
ldapsearch -h <CREAM CE hostname> -x -p 2170 -b "o=grid" objectclass=GlueCE
- Check if the resource BDII publishes glue 1 GlueVOView objectclasses. There should be one GlueVOView objectclass per each queue-VO:
ldapsearch -h <CREAM CE hostname> -x -p 2170 -b "o=grid" objectclass=GlueVOView
- Check if the resource BDII publishes glue 1 GlueCluster objectclasses. If the CE is configured in no cluster mode there should be one GlueCluster objectclass. If the CE is configured in cluster mode and the gLite-CLUSTER is deployed on a different node, there shouldn't be any GlueCluster objectclass.
ldapsearch -h <CREAM CE hostname> -x -p 2170 -b "o=grid" objectclass=GlueCluster
- Check if the resource BDII publishes glue 1 GlueSubCluster objectclasses. If the CE is configured in no cluster mode there should be one GlueSubCluster objectclass. If the CE is configured in cluster mode and the gLite-CLUSTER is deployed on a different node, there shouldn't be any GlueSubCluster objectclass.
ldapsearch -h <CREAM CE hostname> -x -p 2170 -b "o=grid" objectclass=GlueSubCluster
ldapsearch -h <CREAM CE hostname> -x -p 2170 -b "o=glue" objectclass=GLUE2ComputingService
ldapsearch -h <CREAM CE hostname> -x -p 2170 -b "o=glue" "(&(objectclass=GLUE2ComputingEndPoint)(GLUE2EndpointInterfaceName=org.glite.ce.CREAM))"
- Check if the resource BDII publishes glue 2 GLUE2Manager objectclasses. If the CE is configured in no cluster mode there should be one GLUE2Manager objectclass. If the CE is configured in cluster mode and the gLite-CLUSTER is deployed on a different node, there shouldn't be any GLUE2Manager objectclass.
ldapsearch -h <CREAM CE hostname> -x -p 2170 -b "o=glue" objectclass=GLUE2Manager
- Check if the resource BDII publishes glue 2 GLUE2Share objectclasses. If the CE is configured in no cluster mode there should be one GLUE2Share objectclass per each VOview. If the CE is configured in cluster mode and the gLite-CLUSTER is deployed on a different node, there shouldn't be any GLUE2Share objectclass.
ldapsearch -h <CREAM CE hostname> -x -p 2170 -b "o=glue" objectclass=GLUE2Share
ldapsearch -h <CREAM CE hostname> -x -p 2170 -b "o=glue" objectclass=GLUE2ExecutionEnvironment
- Check if the resource BDII publishes glue 2 GLUE2ComputingEndPoint objectclasses with GLUE2EndpointInterfaceName equal to org.glite.ce.ApplicationPublisher. If the CE is configured in no cluster mode there should be one of such objectclass. If the CE is configured in cluster mode and the gLite-CLUSTER is deployed on a different node, there shouldn't be any of such objectclasses.
ldapsearch -h <CREAM CE hostname> -x -p 2170 -b "o=glue" "(&(objectclass=GLUE2ComputingEndPoint)(GLUE2EndpointInterfaceName=org.glite.ce.ApplicationPublisher))"
Status:
Not Implemented
Banning test
For a CREAM-CE configured using gJAF:
- Add the user's DN to
/etc/lcas/ban_users.db
on the cream host
- Try to submit job: you should expect failure
- Remove DN from the aforementioned file
- Submit another job: you should expect a successfull submission
The test should not be done using a CREAM-CE configured using ARGUS
Status:
Implemented in the Robot based testsuite
Admin manage job test
Check that a user who is not admin can't check the status of jobs submitted by another user.
Check that a user who is admin can check the status of jobs submitted by another user.
Status:
Implemented in the Robot based testsuite
Service-info test
Verify the output of glite-ce-service-info command for both verbosity levels.
TBD
Limiter test
Set a limiter threshold to a value which will trigger it.
Wait for the limiter script to be run automatically.
Submit and job and expect failure.
Reset the limiter to the original values.
Wait for the limiter script to be run automatically.
Submit a command and expect success
Status:
Implemented in the Robot based testsuite
Blparser restart test
Configure a CREAM CE using the old blah blparser.
Stop the blparser and after a few seconds (e.g. 10) restart it.
Wait for 2 minute and try a job submission. The should run and complete properly.
Status:
Not Implemented in the Robot based testsuite
LB test
Submit a job to CREAM through the gLite-WMS and wait till the job is Done status
Check the logging-info output of this job: among the logged events there should be 3 events logged as
LRMS
as source:
- Running
- Really-Running
- Done
Status:
Not Implemented in the Robot based testsuite
Basic functionality tests for EMI-ES interface
Delegation
Delegate a proxy with the
glite-es-delegate-proxy
command.
It should return a delegationid.
E.g.:
glite-es-delegate-proxy -e cream-38.pd.infn.it
DelegationID = 5655008298193933
Check with the
glite-es-delegation-info
command about this delegationid. In particular check the
lifetime
field which should match the expiration time of the proxy in the UI:
$ glite-es-delegation-info -e cream-38.pd.infn.it 5655008298193933
Lifetime = Tue Feb 28 21:14:46 2012
Issuer = CN=proxy,CN=proxy,CN=Massimo Sgaravatto,L=Padova,OU=Personal Certificate,O=INFN,C=IT
Subject = CN=Massimo Sgaravatto,L=Padova,OU=Personal Certificate,O=INFN,C=IT
Proxy renewal
TBD
Activity submission without data staging
Submit an activity where in the Application part there is just the Executable without
failifExitCodeNotEqualTo
.
$ cat activity_sleep60.adl
<ActivityDescription>
<ActivityIdentification>
<Name>sleep60</Name>
<Description>sleep 60</Description>
<Type>single</Type>
</ActivityIdentification>
<Application>
<Executable>
<Path>/bin/sleep</Path>
<Argument>60</Argument>
</Executable>
</Application>
<Resources>
<QueueName>creamtest2</QueueName>
</Resources>
</ActivityDescription>
$ glite-es-activity-create -e cream-38.pd.infn.it activity_sleep60.adl
*****************************************
ActivityID = CR_ES096117205
ActivityMgrURI = https://cream-38.pd.infn.it:8443/ce-cream-es/services/ActivityManagementService
Status = ACCEPTED
Status Attrs = {}
Timestamp = Tue Feb 28 15:42:05 2012
Description =
ETNSC =
STAGEIN Dir = {}
SESSION Dir = {}
STAGEOUT Dir = {}
Eventually check with
glite-es-activity-status
the state which should be
TERMINAL
without attributes:
$ glite-es-activity-status -e cream-38.pd.infn.it CR_ES096117205
Sending ActivityStatus request for ActivityID(s) {CR_ES096117205} to service [https://cream-38.pd.infn.it:8443/ce-cream-es/services/ActivityManagementService]
*****************************************
JobID = CR_ES096117205
Status = TERMINAL
Attributes = {}
Timestamp = Tue Feb 28 15:43:10 2012
Description = reason=0
Status:
Not Implemented
Activity submission with failifExitCodeNotEqualTo
Submit an activity where in the Application part there is just the Executable with
failifExitCodeNotEqualTo
.
$ cat activity_sleep60_failif.adl
<ActivityDescription>
<ActivityIdentification>
<Name>sleep60</Name>
<Description>sleep 60</Description>
<Type>single</Type>
</ActivityIdentification>
<Application>
<Executable>
<Path>/bin/sleep</Path>
<Argument>60</Argument>
<FailIfExitCodeNotEqualTo>1</FailIfExitCodeNotEqualTo>
</Executable>
</Application>
<Resources>
<QueueName>creamtest2</QueueName>
</Resources>
</ActivityDescription>
$ glite-es-activity-create -e cream-38.pd.infn.it activity_sleep60_failif.adl
*****************************************
ActivityID = CR_ES776810952
ActivityMgrURI = https://cream-38.pd.infn.it:8443/ce-cream-es/services/ActivityManagementService
Status = ACCEPTED
Status Attrs = {}
Timestamp = Tue Feb 28 15:52:51 2012
Description =
ETNSC =
STAGEIN Dir = {}
SESSION Dir = {}
STAGEOUT Dir = {}
Eventually check with
glite-es-activity-status
the state which should be
TERMINAL
with attribute
APP-FAILURE
:
$ glite-es-activity-status -e cream-38.pd.infn.it CR_ES776810952
Sending ActivityStatus request for ActivityID(s) {CR_ES776810952} to service [https://cream-38.pd.infn.it:8443/ce-cream-es/services/ActivityManagementService]
*****************************************
JobID = CR_ES776810952
Status = TERMINAL
Attributes = {APP_FAILURE}
Timestamp = Tue Feb 28 15:53:54 2012
Description = Executable exitCode 0 instead of 1
Status:
Not Implemented
Activity submission with Input, Output, Client Data Push and Client Data Pull
Activity submission with Environment
Regression tests
See:
http://wiki.italiangrid.it/twiki/bin/view/CREAM/RegressionTestWorkPlan
Performance and scalability tests
Submission and cancellation with polling
Submit 1000 simple jobs (without sandbox transfers) in bunch of 50 jobs and then cancel them.
Check the status of these jobs using the queryevent operation.
All jobs should eventually have CANCELLED as final state.
Status:
Implemented in the old testsuite
cream-test-monitored-cancel -r 30 -n 1000 -m 50 -C 50 -l log4py.conf -j simple.jdl -R cream-35.pd.infn.it:8443/cream-lsf-creamtest2
Submission with polling and sandbox transfer
Submit 1000 jobs (with sandbox transfers) in bunch of 50 jobs.
Check the status of these jobs using the queryevent operation.
All jobs should eventually have DONE-OK as final state.
Status:
Implemented in the old testsuite
cream-test-monitored-submit -r 30 -n 1000 -m 50 -C 50 -l log4py.conf -j testsandbox.jdl -R cream-35.pd.infn.it:8443/cream-lsf-creamtest2
Simple submission with polling, lease updated and proxy renewal
Submit 100 jobs lasting 6000 seconds in bunch of 10 jobs, with an initial lease of 1200 seconds and with an initial proxy valid 20 minutes.
Renew the lease and the proxy.
Check the status of these jobs using the queryevent operation.
All jobs should eventually have DONE-OK as final state.
Status:
Implemented in the old testsuite
cream-test-monitored-lease-updated -r 60 -n 100 -m 10 -C 10 -l log4py.conf -W 1200 --vo dteam --valid 00:20 -R cream-35.pd.infn.it:8443/cream-lsf-creamtest2
Standard compliance and conformance tests
Glue 1 compliance
Glue 1 compliance of the information published by the CREAM-CE resource bdii should be tested. This should be done using [[https://tomtools.cern.ch/confluence/display/IS/GLUEValidator]GlueValidator]].
Status:
Implemented in the Robot based testsuite
Glue 2 compliance
Glue 2 compliance of the information published by the CREAM-CE resource bdii should be tested. This should be done using [[https://tomtools.cern.ch/confluence/display/IS/GLUEValidator]GlueValidator]].
Status:
Implemented in the Robot based testsuite
Nagios probe
For testing nagios probes for worker nodes see
here, tests description for nagios probes for direct cream job submission is
here
Robot based Testsuite
Information about the Robot Based Testsuite is available at:
https://twiki.cern.ch/twiki/bin/view/EMI/CREAMRobotFuncTests
Old Testsuite
The old CREAM testsuite is a set of python scripts that interacts with the CREAM command line tools in order to perform several functional tests.
Installation and setup
The easiest way to deploy the old testsuite is to have a UI node already available.
The old testsuite can then be installed using yum creating a file called testsuites.repo in the yum.repos.d directory with the following definition:
[ETICS-name-CREAM-service-testsuite]
name=ETICS Repository of CREAM-service-testsuite
baseurl=http://etics-repository.cern.ch:8080/repository/pm/registered/repomd/name/CREAM-service-testsuite_1_0_7
protect=0
enabled=1
gpgcheck=0
and then running
yum update
yum install glite-testsuites-cream
After the installation it is suggested to check if the environment variable
GLITE_LOCATION points to the correct gLite installation directory, i.e. the one containing the bin folder with the CREAM clients.
Old Testsuite and ETICS
The old testsuite can be downloaded and build directly from the ETICS framework. For any details concerning the ETICS client installation refer to the
User Guide or to the
twiki site.
The following list of commands is required in order to retrieve and build the latest stable release:
etics-workspace-setup
etics-get-project org.glite
etics-checkout --config glite-testsuites-cream_R_1_0_7_0 --project-config glite_branch_3_1_0 org.glite.testsuites.cream
etics-build org.glite.testsuites.cream
Releases and Changelog
- 1.0.3.0 (2009-02-24):
- Fixed bug: bad termination condition in cream-monitored-* tests with failures during the submission
- 1.0.4.0 (2009-03-05):
- Changed command line of the CE monitor client, this version can be used only with the clients whose version is 1.11 or higher
- 1.0.5.0 (2009-04-17):
- Fixed minor issues
- New options "nopurge" and "sotimeout" are available
- 1.0.6.0 (2009-05-26):
- Fixed man pages in rpm
- Fixed minor issues
- 1.0.7.0 (2009-03-02):
Old Testsuite description
The set of test can be divided into two main categories, according to the type of status detection mechanism for a job:
- with polling (synchronous):
- with notifications (asynchronous):
Proxy management
Each test can be run using either an external user voms-proxy or with an internal management of the voms-proxy.
The default mechanism is using an external voms-proxy. The external voms-proxy is not renewed by the test, the user is responsible for updating that credential if required in such a way that it does not interfere with tests, for example "renaming" the new proxy into the old one.
The external voms-proxy path is defined with the environment variable
X509_USER_PROXY, the main default is /tmp/x509_uuid.
With the internal management of the voms-proxy the testsuite keeps track of the renewal of that credential, the user has just to provide a valid personal certificate and private key. The environment variables required for enabling this mode are:
X509_USER_CERT and
X509_USER_KEY; when this mode is selected each test requires the definition of the VO to be used (option
--vo) and the voms-proxy-init client must be properly configured.
Delegation management
Each test can be run either using a single delegated proxy on the CE for all the submissions or with one delegated proxy per submitted job, see
--delegationType option.
The tester must be aware of using the second mode can overload the entire system, both the service and the testsuite, so the tuning of the test parameters must consider that issue.
Isolation and interference tests
One of the basic requirements for a test is to be isolated; each test handles only its own jobs and resources in general (delegated proxies, lease tokens, etc.) with no interferences from other tests.
Multiple tests can be executed in parallel in order to detect interferences on the service side; for example the
cream-test-monitored-lease-expired
test can be run together with
cream-test-monitored-submit for discovering a wrong lease management for the jobs submitted by the latter.
Hints for test tuning
- Due to unavoidable delays in the command scheduling, the life-time of the job submitted with the cream-test-notified-cancel has to be at least 3 times longer than the rate parameter. With shorter life-time the job can terminate before the testsuite is able to detect the change of its status.
- In order to test the internal proxy renewal mechanism of the CREAM service it's strongly suggested to run at least one test with notifications and proxy renewal all alone, with no other tests running on the same CE.
- It is strongly suggested to configure the socket timeout according to the latency of the network using the --sotimeout option (available since version 1.0.5)
Proposed tests
The set of tests described in this section is used for the pre-certification of the latest version of the CREAM CE and can be considered a template for external testers. In the following the CREAM CE under test is cream-12.pd.infn.it (port 8443) with Torque as batch system and cream_A as assigned queue, the hostname and port must be replaced with the correct values according to the arrangement of the testbed; the VO dteam is used.
The simple.jdl contains:
[ executable="/bin/sleep"; arguments="180";]
The long.jdl contains:
[ executable="/bin/sleep"; arguments="1800";]
The onehoursleep contains:<verbatim>[ executable="/bin/sleep"; arguments="3600";]</verbatim>
The testsandbox.jdl contains:
[
executable="/bin/cp";
arguments="grid-mapfile out-`date +%s%N`-`whoami`.txt";
inputsandbox={
"gsiftp://cream-12.pd.infn.it/etc/grid-security/grid-mapfile"
};
outputsandbox={
"out*.txt"
};
outputsandboxbasedesturi="gsiftp://cream-12.pd.infn.it/tmp";
epilogue="/bin/sleep";
epiloguearguments="60";
]
The log4py.conf contains:
[Default]
Format: %T %L %C [%F] - %M
Target: cream-test-cream-12.log
Ansi: False
LogLevel: Debug
Simple submission with polling |
cream-test-monitored-submit -r 30 -n 1000 -m 50 -C 50 -l log4py.conf -j simple.jdl -R cream-12.pd.infn.it:8443/cream-pbs-cream_A |
The renewal process is disabled, the variable X509_USER_PROXY must point to a 12-hours long proxy |
Simple submission with notifications |
cream-test-notified-submit -r 30 -n 1000 -m 50 -C 50 -p 9000 -l log4py.conf -j simple.jdl -R cream-12.pd.infn.it:8443/cream-pbs-cream_A |
The renewal process is disabled, the variable X509_USER_PROXY must point to a 12-hours long proxy |
Submission and cancellation with polling |
cream-test-monitored-cancel -r 30 -n 1000 -m 50 -C 50 -l log4py.conf -j simple.jdl -R cream-12.pd.infn.it:8443/cream-pbs-cream_A
|
The renewal process is disabled, the variable X509_USER_PROXY must point to a 12-hours long proxy. |
Submission and cancellation with notifications |
cream-test-notified-cancel -r 30 -n 1000 -m 50 -C 50 -p 9000 -l log4py.conf -j simple.jdl -R cream-12.pd.infn.it:8443/cream-pbs-cream_A |
The renewal process is disabled, the variable X509_USER_PROXY must point to a 12-hours long proxy. |
Submission with polling and sandbox transfer |
cream-test-monitored-submit -r 30 -n 1000 -m 50 -C 50 -l log4py.conf -j testsandbox.jdl -R cream-12.pd.infn.it:8443/cream-pbs-cream_A
|
The renewal process is disabled, the variable X509_USER_PROXY must point to a 12-hours long proxy. The output sandbox on the CE must be checked and removed manually. |
Submission with notifications and sandbox transfer |
cream-test-notified-submit -r 30 -n 1000 -m 50 -C 50 -p 9000 -l log4py.conf -j testsandbox.jdl -R cream-12.pd.infn.it:8443/cream-pbs-cream_A
|
The renewal process is disabled, the variable X509_USER_PROXY must point to a 12-hours long proxy. The output sandbox on the CE must be checked and removed manually. |
Simple submission with polling and proxy renewal |
cream-test-monitored-submit -r 60 -n 50 -m 10 -C 10 -l log4py.conf -j long.jdl --vo dteam --valid 00:20 -R cream-12.pd.infn.it:8443/cream-pbs-cream_A
|
The variables X509_USER_CERT and X509_USER_KEY must point to the location of the user certificate and private key, passphrase is required |
Simple submission with notifications and proxy renewal |
cream-test-notified-submit -r 60 -n 50 -m 10 -C 10 -p 9000 -l log4py.conf -j long.jdl --vo dteam --valid 00:20 -R cream-12.pd.infn.it:8443/cream-pbs-cream_A
|
The variables X509_USER_CERT and X509_USER_KEY must point to the location of the user certificate and private key, passphrase is required. |
Simple submission with polling and proxy expired |
cream-test-monitored-proxy-expired -r 60 -n 50 -m 10 -C 10 -l log4py.conf --vo dteam --valid 00:20 -R cream-12.pd.infn.it:8443/cream-pbs-cream_A
|
The variables X509_USER_CERT and X509_USER_KEY must point to the location of the user certificate and private key, passphrase is required. |
Simple submission with polling and lease expired |
cream-test-monitored-lease-expired -r 60 -n 50 -m 10 -C 10 -l log4py.conf -W 1200 -R cream-12.pd.infn.it:8443/cream-pbs-cream_A
|
The renewal process is disabled, the variable X509_USER_PROXY must point to a 12-hours long proxy. One lease token is defined for all jobs. |
Simple submission with notifications and lease expired |
cream-test-notified-lease-expired -r 60 -n 50 -m 10 -C 10 -l log4py.conf -W 1200 -p 9000 -R cream-12.pd.infn.it:8443/cream-pbs-cream_A
|
The renewal process is disabled, the variable X509_USER_PROXY must point to a 12-hours long proxy.One lease token is defined for all jobs |
Simple submission with polling, lease expired and proxy renewal |
cream-test-monitored-lease-expired -r 60 -n 50 -m 10 -C 10 -l log4py.conf -W 1800 --vo dteam --valid 00:20 -R cream-12.pd.infn.it:8443/cream-pbs-cream_A
|
The variables X509_USER_CERT and X509_USER_KEY must point to the location of the user certificate and private key, passphrase is required.One lease token is defined for all jobs. |
Simple submission with notifications, lease expired and proxy renewal |
cream-test-notified-lease-expired -r 60 -n 50 -m 10 -C 10 -l log4py.conf -W 1800 -p 9000 --vo dteam --valid 00:20 -R cream-12.pd.infn.it:8443/cream-pbs-cream_A |
The variables X509_USER_CERT and X509_USER_KEY must point to the location of the user certificate and private key, passphrase is required.One lease token is defined for all jobs. |
Simple submission with polling and lease updated |
cream-test-monitored-lease-updated -r 60 -n 50 -m 10 -C 10 -l log4py.conf -W 1200 -R cream-12.pd.infn.it:8443/cream-pbs-cream_A
|
The renewal process is disabled, the variable X509_USER_PROXY must point to a 12-hours long proxy.One lease token is defined for all jobs. |
Simple submission with notifications and lease updated |
cream-test-notified-lease-updated -r 60 -n 50 -m 10 -C 10 -l log4py.conf -W 1200 -p 9000 -R cream-12.pd.infn.it:8443/cream-pbs-cream_A
|
The renewal process is disabled, the variable X509_USER_PROXY must point to a 12-hours long proxy.One lease token is defined for all jobs. |
Simple submission with polling, lease updated and proxy renewal |
cream-test-monitored-lease-updated -r 60 -n 50 -m 10 -C 10 -l log4py.conf -W 1800 --vo dteam --valid 00:20 -R cream-12.pd.infn.it:8443/cream-pbs-cream_A
|
The variables X509_USER_CERT and X509_USER_KEY must point to the location of the user certificate and private key, passphrase is required.One lease token is defined for all jobs. |
Simple submission with notifications, lease updated and proxy renewal |
cream-test-notified-lease-updated -r 60 -n 50 -m 10 -C 10 -l log4py.conf -W 1800 -p 9000 --vo dteam --valid 00:20 -R cream-12.pd.infn.it:8443/cream-pbs-cream_A
|
The variables X509_USER_CERT and X509_USER_KEY must point to the location of the user certificate and private key, passphrase is required.One lease token is defined for all jobs. |
Proxy renewal for job in IDLE |
cream-test-monitored-submit -r 60 -n $TOTAL -m $PART -C $PART -l log4py.conf -j onehoursleep.jdl --vo dteam --valid 00:45 -R cream-12.pd.infn.it:8443/cream-pbs-cream_A
|
The variables X509_USER_CERT and X509_USER_KEY must point to the location of the user certificate and private key, passphrase is required. The variable $PART is an integer greater than the number of jobs which can be run concurrently in the batch system, the variable $TOTAL is a multiple of $PART. |