Difference: WmsTestPlan (1 vs. 30)

Revision 302011-07-01 - FabioCapannini

Line: 1 to 1
 

WMS Test Plan

Line: 33 to 33
 Install the WMS metapackage:
yum install emi-wms
Deleted:
<
<
(see log file)
 Configure the WMS:
/opt/glite/yaim/bin/yaim -c -s site-info.def -n WMS  
Deleted:
<
<
(see log file)
 

Update test

N/A

Revision 292011-07-01 - AlessioGianelle

Line: 1 to 1
 

WMS Test Plan

Line: 6 to 6
  N/A
Added:
>
>

Deployment tests

 

Repository

The EMI-1 RC4 repository can be found under:

Line: 39 to 41
  (see log file)
Deleted:
<
<

Deployment tests

 

Update test

N/A

Revision 282011-06-29 - FabioCapannini

Line: 1 to 1
 

WMS Test Plan

Line: 542 to 542
 

bug #45883: Optimization of resubmission

Changed:
<
<
Description of the test.
>
>
A proper way to test this bug was not found.
 

bug #48636: job wrapper should log events for truncated files

Line: 2301 to 2301
 IsmIiLDAPCEFilterExt = "(|(GlueCEAccessControlBaseRule=VO:atlas)(GlueCEAccessControlBaseRule=VOMS:/atlas/)(GlueCEAccessControlBaseRule=VO:alice)(GlueCEAccessControlBaseRule=VOMS:/alice/)(GlueCEAccessControlBaseRule=VO:lhcb)(GlueCEAccessControlBaseRule=VOMS:/lhcb/)(GlueCEAccessControlBaseRule=VO:cms)(GlueCEAccessControlBaseRule=VOMS:/cms/)(GlueCEAccessControlBaseRule=VO:dteam)(GlueCEAccessControlBaseRule=VOMS:/dteam/)(GlueCEAccessControlBaseRule=VO:biomed)(GlueCEAccessControlBaseRule=VOMS:/biomed/)(GlueCEAccessControlBaseRule=VO:ops)(GlueCEAccessControlBaseRule=VOMS:/ops/)(GlueCEAccessControlBaseRule=VO:infngrid)(GlueCEAccessControlBaseRule=VOMS:/infngrid/))";
Added:
>
>

bug #78484: [ YAIM_WMS ] Multiple parameter configuration added in condor_config.local

Test description.

[root@cream-44 ~]# cat /opt/condor-c/local.cream-44/condor_config.local | grep GRID_MONITOR
GRID_MONITOR_HEARTBEAT_TIMEOUT = 300
GRID_MONITOR_RETRY_DURATION = 31536000
GRID_MONITOR = $(RELEASE_DIR)/libexec/glite/grid_monitor.sh
ENABLE_GRID_MONITOR = TRUE

[root@cream-44 ~]# /opt/glite/yaim/bin/yaim -c -s /usr/local/nfs/jra1_test/cream-44/site-info_wms.def -n WMS
WARNING:
WARNING: *****************************************************************************
WARNING: Your siteinfo directory is world readable. This is generally a bad idea
WARNING: as configuration files may contain passwords and other sensitive information.
WARNING: *****************************************************************************
WARNING:
[ ... ]
DEBUG: Skipping function: config_glite_initd_setenv because it is not defined
INFO: Executing function: config_glite_initd
INFO: Configuration Complete. [ OK ]
INFO: YAIM terminated succesfully.

[root@cream-44 ~]# cat /opt/condor-c/local.cream-44/condor_config.local | grep GRID_MONITOR
GRID_MONITOR_HEARTBEAT_TIMEOUT = 300
GRID_MONITOR_RETRY_DURATION = 31536000
GRID_MONITOR = $(RELEASE_DIR)/libexec/glite/grid_monitor.sh
ENABLE_GRID_MONITOR = TRUE

Considered successful if the line GRID_MONITOR is not added after configuring.

bug #79141: various bugs about parametric jobs

Test description.

[ale@cream-12 UI]$ cat 79141.jdl
[
JobType = "Parametric";
Parameters = 1;
ParameterStart = 0;
ParameterStep = 1;
Executable = "/bin/ls";
requirements = other.GlueCEStateStatus == "Production";
Arguments = "-la";
Stdoutput = "output.txt";
InputSandbox = {"Src_PARAM_/Makefile", "Test_PARAM_"};
OutputSandbox = {"output.txt"};
]
[ale@cream-12 UI]$ glite-wms-job-submit -a -c etc/wmp_devel11.conf 79141.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel11.cnaf.infn.it:9000/5...

==========================================================================

[ale@cream-12 UI]$ glite-wms-job-status https://devel11.cnaf.infn.it:9000/5...

======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel11.cnaf.infn.it:9000/5...
Current Status: Waiting
Submitted: Wed Apr 20 09:37:09 2011 CEST
==========================================================================

- Nodes information for:
Status info for the Job : https://devel11.cnaf.infn.it:9000/S...
Current Status: Ready
Status Reason: unavailable
Destination: ce-enmr.chemie.uni-frankfurt.de:2119/jobmanager-lcgpbs-cert
Submitted: Wed Apr 20 09:37:09 2011 CEST
==========================================================================

[ale@cream-12 UI]$ glite-wms-job-status https://devel11.cnaf.infn.it:9000/5...

======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel11.cnaf.infn.it:9000/5...
Current Status: Done (Success)
Exit code: 0
Submitted: Wed Apr 20 09:37:09 2011 CEST
==========================================================================

- Nodes information for:
Status info for the Job : https://devel11.cnaf.infn.it:9000/S...
Current Status: Done (Success)
Logged Reason(s):
-
- Job terminated successfully
Exit code: 0
Status Reason: Job terminated successfully
Destination: ce-enmr.chemie.uni-frankfurt.de:2119/jobmanager-lcgpbs-cert
Submitted: Wed Apr 20 09:37:09 2011 CEST
==========================================================================

[ale@cream-12 UI]$ glite-wms-job-output https://devel11.cnaf.infn.it:9000/5...

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

================================================================================

JOB GET OUTPUT OUTCOME

Output sandbox files for the DAG/Collection :
https://devel11.cnaf.infn.it:9000/5...
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/ale_5x11U1aJFoKSDbpwu4-apw

================================================================================

[ale@cream-12 UI]$ cat /tmp/jobOutput/ale_5x11U1aJFoKSDbpwu4-apw/Node_0/output.txt
total 32
drwxr-xr-x 2 dteam035 dteam 4096 Apr 20 09:38 .
drwx------ 5 dteam035 dteam 4096 Apr 20 09:37 ..
-rw-r--r-- 1 dteam035 dteam 2150 Apr 20 09:37 .BrokerInfo
-rw-r--r-- 1 dteam035 dteam 118 Apr 20 09:38 https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2fShaETXWYzsKgV_5f0sd8o9uQ.output
-rw-r--r-- 1 dteam035 dteam 0 Apr 20 09:37 Makefile
-rw-r--r-- 1 dteam035 dteam 0 Apr 20 09:38 output.txt
-rw-r--r-- 1 dteam035 dteam 0 Apr 20 09:37 Test0
-rw------- 1 dteam035 dteam 103 Apr 20 09:38 tmp.fzuGl19574
-rw------- 1 dteam035 dteam 2 Apr 20 09:38 tmp.GxlYw19573
-rw------- 1 dteam035 dteam 103 Apr 20 09:38 tmp.mwAGt19459
-rw------- 1 dteam035 dteam 0 Apr 20 09:38 tmp.OfMof19572
-rw------- 1 dteam035 dteam 2 Apr 20 09:38 tmp.oizWr19458 

Considered successful if the parametric job is successfully executed and the ouptut correctly retrieved.

 -- ElisabettaMolinari - 2010-02-24

Revision 272011-06-28 - FabioCapannini

Line: 1 to 1
 

WMS Test Plan

Line: 1428 to 1428
 

bug #68944: Bug in ICE's start/stop script

Added:
>
>
Test description.

[root@devel09 ~]# ps aux |grep ice
glite     2716  0.0  0.0 220996  2192 ?        Ss   09:36   0:00 /usr/bin/glite-wms-ice-safe --conf glite_wms.conf --daemon /tmp/glite-wms-ice-safe.pid
glite     2720  0.0  0.0  63856  1108 ?        S    09:36   0:00 sh -c /usr/bin/glite-wms-ice --conf glite_wms.conf /var/log/wms/ice_console.log 2>&1
glite     2721  0.0  0.1 483464  7404 ?        Sl   09:36   0:02 /usr/bin/glite-wms-ice --conf glite_wms.conf /var/log/wms/ice_console.log
root     31253  0.0  0.0  61184   788 pts/1    S+   14:29   0:00 grep ice
[root@devel09 ~]# /etc/init.d/glite-wms-ice restart
stopping ICE... ok
starting ICE... ok
[root@devel09 ~]# ps aux |grep ice
glite    31318  0.0  0.0 220996  2196 ?        Ss   14:30   0:00 /usr/bin/glite-wms-ice-safe --conf glite_wms.conf --daemon /tmp/glite-wms-ice-safe.pid
glite    31322  0.0  0.0  63856  1112 ?        S    14:30   0:00 sh -c /usr/bin/glite-wms-ice --conf glite_wms.conf /var/log/wms/ice_console.log 2>&1
glite    31323  1.0  0.1 483464  7352 ?        Sl   14:30   0:00 /usr/bin/glite-wms-ice --conf glite_wms.conf /var/log/wms/ice_console.log
root     31363  0.0  0.0  61180   784 pts/1    S+   14:30   0:00 grep ice

Considered successful if ice processes are restarted.

bug #70061: WMS hates collections with 192 nodes!

Test description.

[mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf coll_192.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel11.cnaf.infn.it:9000/T...

==========================================================================

Considered successful if the collection is submitted and executed correctly.

bug #70331: glite-wms-create-proxy "ambiguous redirect"

Test description.

[root@cream-44 cron.d]# su - glite
[glite@cream-44 ~]$ /usr/sbin/glite-wms-create-proxy.sh /var/wms.proxy /var/log/wms/create_proxy.log
[glite@cream-44 ~]$

Considered successful if the command works with no error messages.

bug #70824: environment values in JDL cannot have spaces

Test description.

Submit this jdl:

[
Executable = "/bin/env";
Stdoutput = "env.out" ;
StdError = "env.err" ;
shallowretrycount = 2;
retrycount = 3;
OutputSandbox = { "env.out" ,"env.err" } ;
Environment = {
"ATHENA_RUN_EVENTS='[(152345L, 216721L)]'"
};

]

======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel09.cnaf.infn.it:9000/g...
Current Status: Done (Success)
Exit code: 0
Status Reason: Job terminated successfully
Destination: gridgate.ucc.ie:2119/jobmanager-pbs-sixhour
Submitted: Thu Apr 21 17:11:46 2011 CEST
==========================================================================

After output retrieval:

[ale@cream-12 UI]$ glite-wms-job-output https://devel09.cnaf.infn.it:9000/g...

Connecting to the service https://devel09.cnaf.infn.it:7443/g...

================================================================================

JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://devel09.cnaf.infn.it:9000/g...
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/ale_gvpbgkBN_KkLhly-uNjpxQ

================================================================================

Check the output file:

[ale@cream-12 UI]$ cat /tmp/jobOutput/ale_gvpbgkBN_KkLhly-uNjpxQ/env.out | grep ATHENA
ATHENA_RUN_EVENTS=[(152345L, 216721L)] 

Considered successful if the environment contains the variable with spaces.

bug #71863: JobWrapper tries to use "test -eq" for string comparison

Test description.

Looking into a JW:

fatal_error() # 1 - reason, 2 - transfer OSB
{
if [ "x${LM_done_failed_reason}" != "x" ]; then
jw_echo "LM_log_done_begin" # parsed 'as is' by LM
jw_echo "${LM_done_failed_reason}"
jw_echo "LM_log_done_end" # parsed 'as is' by LM
fi
jw_echo "$1"
log_done_failed 1 "$1"
if [ "x$2" == "xOSB" ]; then
OSB_transfer
fi
doExit 1
} 

Considered successful if the comparison is done via ==

bug #73192: Submission failed due to a credential problem

Test description.

Simply try a submit:

[ale@cream-12 UI]$ glite-wms-job-submit -a -c etc/wmp_devel09.conf jdl/deepresub.jdl

Connecting to the service https://devel09.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel09.cnaf.infn.it:9000/L...

========================================================================== 

Considered successful if the job is correctly submitted.

bug #73699: Wrong retry count computation

Test description.

Submit this jdl:

[
requirements = (other.GlueCEStateStatus == "Production");
Rank = -2 * other.GlueCEStateWaitingJobs;
Executable = "/bin/ls";
prologue = "/bin/false";
shallowretrycount = -1;
RetryCount = 3;
usertags = [ exe = "deepresub" ];
requirements=regexp("8443/cream", other.GlueCEUniqueID);

]

The job should file after 3 resubmissions:

======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel09.cnaf.infn.it:9000/L...
Current Status: Aborted
Logged Reason(s):
- Prologue failed with error 1
- reason=1; Prologue failed with error 1
- Transfer to CREAM failed due to exception: Failed to create a delegation id for job https://devel09.cnaf.infn.it:9000/L...: reason is Received NULL fault; the error is due to another cause: FaultString=[] - FaultCode=[SOAP-ENV:Server.generalException] - FaultSubCode=[SOAP-ENV:Server.generalException] - FaultDetail=[<ns1:DelegationException><msg>User C=IT,O=INFN,OU=Personal Certificate,L=Padova,CN=Alessio Gianelle not authorized for operation {http://www.gridsite.org/namespaces/...</msg></ns1:DelegationException><ns2:exceptionName xmlns:ns2="http://xml.apache.org/axis/">org.glite.security.delegation.DelegationException</ns2:exceptionName><ns3:hostname xmlns:ns3="http://xml.apache.org/axis/">cluster50.knu.ac.kr</ns3:hostname>]
- Prologue failed with error 1
- Prologue failed with error 1
- Prologue failed with error 1
- reason=1; Prologue failed with error 1
Status Reason: hit job retry count (3)
Destination: infnlns-ce-01.ct.pi2s2.it:8443/cream-lsf-cert
Submitted: Thu Apr 21 16:47:33 2011 CEST

Infact:

[ale@cream-12 UI]$ glite-wms-job-logging-info --event Resubmission https://devel09.cnaf.infn.it:9000/L...

===================== glite-wms-job-logging-info Success =====================

LOGGING INFORMATION:

Printing info for the Job : https://devel09.cnaf.infn.it:9000/L...

---
Event: Resubmission
- Result = WILLRESUB
- Source = LogMonitor
- Timestamp = Thu Apr 21 16:49:44 2011 CEST
---
Event: Resubmission
- Result = WILLRESUB
- Source = WorkloadManager
- Timestamp = Thu Apr 21 16:49:45 2011 CEST
---
Event: Resubmission
- Result = WILLRESUB
- Source = LogMonitor
- Timestamp = Thu Apr 21 16:49:50 2011 CEST
---
Event: Resubmission
- Result = WILLRESUB
- Source = WorkloadManager
- Timestamp = Thu Apr 21 16:49:51 2011 CEST
---
Event: Resubmission
- Result = WILLRESUB
- Source = LogMonitor
- Timestamp = Thu Apr 21 16:52:00 2011 CEST
---
Event: Resubmission
- Result = WILLRESUB
- Source = WorkloadManager
- Timestamp = Thu Apr 21 16:52:01 2011 CEST
---
Event: Resubmission
- Result = WILLRESUB
- Source = LogMonitor
- Timestamp = Thu Apr 21 16:54:11 2011 CEST
==========================================================================

(Two events one from LM and the other from WM for every resubmission, plus the last try which failed because limit is reached.)

Considered successful if the job is resubmitted the right number of times (three in this case).

bug #73711: edg_wll_SetLoggingJobProxy with empty sequence code returns "no state in DB"

Test description.

This bug represents a development issue. It is simply verified by checking that submission for both a job and a collection returns the jobid.

[mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf job.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel11.cnaf.infn.it:9000/B...

==========================================================================

[mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf coll_10.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel11.cnaf.infn.it:9000/T...

========================================================================== 

Considered successful if the jobids are correctly returned.

bug #73715: missing ReallyRunning event from LogMonitor

Test description.

[ale@cream-12 UI]$ glite-wms-job-logging-info -v 3 --event ReallyRunning https://devel09.cnaf.infn.it:9000/v...

===================== glite-wms-job-logging-info Success =====================

LOGGING INFORMATION:

Printing info for the Job : https://devel09.cnaf.infn.it:9000/v...

---
Event: ReallyRunning
- Arrived = Thu Apr 21 16:08:28 2011 CEST
- Host = grid003.fc.up.pt
- Level = SYSTEM
- Priority = synchronous
- Seqcode = UI=000000:NS=0000000004:WM=000005:BH=0000000000:JSS=000002:LM=000002:LRMS=000003:APP=000000:LBS=000000
- Source = LRMS
- Timestamp = Thu Apr 21 16:08:28 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
---
Event: ReallyRunning
- Arrived = Thu Apr 21 16:09:43 2011 CEST
- Host = devel09.cnaf.infn.it
- Level = SYSTEM
- Priority = synchronous
- Seqcode = UI=000000:NS=0000000004:WM=000005:BH=0000000000:JSS=000002:LM=000007:LRMS=000000:APP=000000:LBS=000000
- Source = LogMonitor
- Timestamp = Thu Apr 21 16:09:43 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy
- Wn seq = UI=000000:NS=0000000004:WM=000005:BH=0000000000:JSS=000002:LM=000002:LRMS=000000:APP=000000:LBS=000000
   Alessio Gianelle <ale>
Project Member

Considered successful if the reallyrunning event is effectively logged.

bug #74259: Previous matches information is not taken into account if direct submission is used

Test description.

Submit directly (i.e. use option -r ) a job which trigger a resubmission (e.g. prologue="/bin/false").

Then, when it fails, check the "UserTag" events:

[ale@cream-12 UI]$ glite-wms-job-logging-info -v 2 --event UserTag https://devel09.cnaf.infn.it:9000/Q...
===================== glite-wms-job-logging-info Success =====================

LOGGING INFORMATION:

Printing info for the Job : https://devel09.cnaf.infn.it:9000/Q...

---
Event: UserTag
- Arrived = Fri Apr 22 10:10:37 2011 CEST
- Host = devel09.cnaf.infn.it
- Name = exe
- Source = NetworkServer
- Src instance = https://devel09.cnaf.infn.it:7443/g...
- Timestamp = Fri Apr 22 10:10:37 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
- Value = deepresub
---
Event: UserTag
- Arrived = Fri Apr 22 10:10:39 2011 CEST
- Host = devel09.cnaf.infn.it
- Name = CEInfoHostName
- Source = WorkloadManager
- Src instance = 21624
- Timestamp = Fri Apr 22 10:10:39 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
- Value = alice16.spbu.ru
---
Event: UserTag
- Arrived = Fri Apr 22 10:10:41 2011 CEST
- Host = devel09.cnaf.infn.it
- Name = CEInfoHostName
- Source = WorkloadManager
- Src instance = 21624
- Timestamp = Fri Apr 22 10:10:41 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
- Value = alice16.spbu.ru
---
Event: UserTag
- Arrived = Fri Apr 22 10:10:43 2011 CEST
- Host = devel09.cnaf.infn.it
- Name = CEInfoHostName
- Source = WorkloadManager
- Src instance = 21624
- Timestamp = Fri Apr 22 10:10:43 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
- Value = alice16.spbu.ru
---
Event: UserTag
- Arrived = Fri Apr 22 10:10:45 2011 CEST
- Host = devel09.cnaf.infn.it
- Name = CEInfoHostName
- Source = WorkloadManager
- Src instance = 21624
- Timestamp = Fri Apr 22 10:10:45 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
- Value = alice16.spbu.ru
==========================================================================

Considered successful if the tag "CEInfoHostName" reports the same value (equal to the chosen CE) for all the events.

bug #74577: Wrong counter in ICE database is set at the job creation

Test description.

we can check that the first time a job is submitted the counter should be set to 1:

[ale@cream-12 UI]$ glite-wms-job-submit -a -c etc/wmp_devel09.conf test.jdl

Connecting to the service https://devel09.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel09.cnaf.infn.it:9000/Z...

==========================================================================

and on the WMS:
[root@devel09 persist_dir]# sqlite3 ice.db "select counter from proxy where myproxyurl='myproxy.cern.ch';"
1 

Considered successful if the counter is set to 1.

bug #75223: wrong reason logged

Test description.

submit a jdl like this one:

executable="/bin/sleep";
arguments="6";
Stdoutput = "std.out";
OutputSandbox = {"std.out"};
requirements=regexp("8443/cream", other.GlueCEUniqueID);
ShallowRetryCount = 2;
prologue="/bin/false";

and look for the Done failed reason in the logging info:

[ale@cream-12 UI]$ glite-wms-job-logging-info -v 2 --event Done https://devel09.cnaf.infn.it:9000/T...

===================== glite-wms-job-logging-info Success =====================

LOGGING INFORMATION:

Printing info for the Job : https://devel09.cnaf.infn.it:9000/T...

---
Event: Done
- Arrived = Thu Apr 21 13:31:27 2011 CEST
- Exit code = 0
- Host = lcg1371.gridpp.rl.ac.uk
- Reason = Prologue failed with error 1
- Source = LRMS
- Status code = FAILED
- Timestamp = Thu Apr 21 13:31:27 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
---
Event: Done
- Arrived = Thu Apr 21 13:32:27 2011 CEST
- Exit code = 0
- Host = devel09.cnaf.infn.it
- Reason = Prologue failed with error 1
- Source = LogMonitor
- Status code = FAILED
- Timestamp = Thu Apr 21 13:32:27 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy 

Considered successful if the Reason contains only "Prologue failed with error 1".

bug #75368: ICE should log a DONE_FAILED to LB every time the job is going to be resubmitted

Test description.

Submit a jdl to a failure Cream CE (i.e. a CE where you know that for some reasons job will fail) and check the logging-info, before "Resubmission" event ICE should log a "Done Failed" with the failure reason.

Event: Done
- Arrived = Thu Apr 21 13:24:09 2011 CEST
- Exit code = 0
- Host = devel09.cnaf.infn.it
- Reason = Transfer to CREAM failed due to exception: Failed to create a delegation id for job https://devel09.cnaf.infn.it:9000/6...: reason is Received NULL fault; the error is due to another cause: FaultString=[] - FaultCode=[SOAP-ENV:Server.generalException] - FaultSubCode=[SOAP-ENV:Server.generalException] - FaultDetail=[<ns1:DelegationException><msg>User C=IT,O=INFN,OU=Personal Certificate,L=Padova,CN=Alessio Gianelle not authorized for operation {http://www.gridsite.org/namespaces/...</msg></ns1:DelegationException><ns2:exceptionName xmlns:ns2="http://xml.apache.org/axis/">org.glite.security.delegation.DelegationException</ns2:exceptionName><ns3:hostname xmlns:ns3="http://xml.apache.org/axis/">foam.grid.kiae.ru</ns3:hostname>]
- Source = LogMonitor
- Status code = FAILED
- Timestamp = Thu Apr 21 13:24:09 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy
---
Event: Resubmission
- Arrived = Thu Apr 21 13:24:09 2011 CEST
- Host = devel09.cnaf.infn.it
- Reason = Resubmitting because of exception Failed to create a delegation id for job https://devel09.cnaf.infn.it:9000/6...: reason is Received NULL fault; the error is due to another cause: FaultString=[] - FaultCode=[SOAP-ENV:Server.generalException] - FaultSubCode=[SOAP-ENV:Server.generalException] - FaultDetail=[<ns1:DelegationException><msg>User C=IT,O=INFN,OU=Personal Certificate,L=Padova,CN=Alessio Gianelle not authorized for operation {http://www.gridsite.org/namespaces/...</msg></ns1:DelegationException><ns2:exceptionName xmlns:ns2="http://xml.apache.org/axis/">org.glite.security.delegation.DelegationException</ns2:exceptionName><ns3:hostname xmlns:ns3="http://xml.apache.org/axis/">foam.grid.kiae.ru</ns3:hostname>] CEUrl https://foam.grid.kiae.ru:8443/ce-c...
- Result = WILLRESUB
- Source = LogMonitor
- Tag = unavailable
- Timestamp = Thu Apr 21 13:24:09 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy

bug #75402: Synchronization loss between real validity of proxy and exp. time saved in ICE's database

Test description.

Rarely means that is not easy to reproduce, so we consider it hopefully fixed.

bug #77004: Wrong myproxyserver string processing in ICE

Test description.

Submit this jdl: executable="/bin/sleep"; arguments="6"; Stdoutput = "std.out"; OutputSandbox = {"std.out"}; requirements=regexp("8443/cream", other.GlueCEUniqueID); MyProxyServer = " ";

Then check the database's table "proxy" of ice in the WMS to see if the proxy is associated to a fake myproxy server:

[root@devel09 persist_dir]# sqlite3 ice.db "select * from proxy;" /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle-/dteam/Role=NULL/Capability=NULL||/var/ice/persist_dir/B23D0D7177A8B6234F1985493FA09FF41A4FA98C.proxy|1303415475|0

(the name of myproxy server should be in the second field)

bug #77055: "MyProxyServer: wrong type caught for attribute" for parametric jobs

Test description.

[ale@cream-12 UI]$ cat param.jdl
// Test parametric job.
// The standar Output and Error are parametrizied such as the
// enviroment variable: MYPATH_PARAM_.
[
JobType = "parametric";
Executable = "/bin/env";
Environment = {"MYPATH_PARAM_=$PATH:/bin:/usr/bin:$HOME"};
StdOutput = "echo_PARAM_.out";
StdError = "echo_PARAM_.err";
OutputSandbox = {"echo_PARAM_.out","echo_PARAM_.err"};
ParameterStart = 0;
ParameterStep = 2;
Parameters = 10;
myproxyserver="";
]
[ale@cream-12 UI]$ glite-wms-job-submit -a -c etc/wmp_cream-44.conf -o test.out param.jdl

Connecting to the service https://cream-44.pd.infn.it:7443/gl...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://cream-44.pd.infn.it:9000/H5...

The job identifier has been saved in the following file:
/home/ale/UI/test.out

========================================================================== 

Considered successful if the job is correctly submitted.

bug #77694: Resource BDII for WMS needs to be revisit

Test description.

[glite@cream-44 ~]$ cat /opt/glite/yaim/node-info.d/glite-wms | grep config_gip_only [glite@cream-44 ~]$

Considered successful if the function config_gip_only has been removed.

bug #77876: While purging DAGs/Collections the CLEAR event is only logged for the parent node

Test description.

Forcing a run of the cron purger you can then check the logging info of a "parent" node:

[ale@cream-12 UI]$ glite-wms-job-logging-info -v 2 https://cream-44.pd.infn.it:9000/vR...

===================== glite-wms-job-logging-info Success =====================

LOGGING INFORMATION:

Printing info for the Job : https://cream-44.pd.infn.it:9000/vR...

---
Event: RegJob
- Arrived = Wed Apr 20 14:01:25 2011 CEST
- Host = cream-44.pd.infn.it
- Jobtype = COLLECTION
- Ns = https://cream-44.pd.infn.it:7443/gl...
- Nsubjobs = 100
- Seed = WMPROXY
- Source = NetworkServer
- Src instance = https://cream-44.pd.infn.it:7443/gl...
- Timestamp = Wed Apr 20 14:01:25 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
---
Event: Accepted
- Arrived = Wed Apr 20 14:01:34 2011 CEST
- From = NetworkServer
- From host = cream-12.pd.infn.it
- Host = cream-44.pd.infn.it
- Source = NetworkServer
- Src instance = https://cream-44.pd.infn.it:7443/gl...
- Timestamp = Wed Apr 20 14:01:34 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
---
Event: EnQueued
- Arrived = Wed Apr 20 14:01:34 2011 CEST
- Host = cream-44.pd.infn.it
- Queue = /var/workload_manager/jobdir
- Result = START
- Source = NetworkServer
- Src instance = https://cream-44.pd.infn.it:7443/gl...
- Timestamp = Wed Apr 20 14:01:34 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
---
Event: EnQueued
- Arrived = Wed Apr 20 14:01:57 2011 CEST
- Host = cream-44.pd.infn.it
- Queue = /var/workload_manager/jobdir
- Result = OK
- Source = NetworkServer
- Src instance = https://cream-44.pd.infn.it:7443/gl...
- Timestamp = Wed Apr 20 14:01:57 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
---
Event: DeQueued
- Arrived = Wed Apr 20 14:05:03 2011 CEST
- Host = cream-44.pd.infn.it
- Queue = /var/workload_manager/jobdir
- Source = WorkloadManager
- Src instance = 24901
- Timestamp = Wed Apr 20 14:05:03 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy
---
Event: CollectionState
- Arrived = Wed Apr 20 14:05:11 2011 CEST
- Child = https://cream-44.pd.infn.it:9000/_y...
- Child event = Running
- Done code = 0
- Histogram = Submitted=0, Waiting=0, Ready=0, Scheduled=0, Running=1, Done=0, Cleared=0, Aborted=0, Cancelled=0, Unknown=99, Purged=0
- Host = cream-44.pd.infn.it
- Source = LBServer
- State = Running
- Timestamp = Wed Apr 20 14:05:11 2011 CEST
- User = /C=IT/O=INFN/OU=Host/L=Padova/CN=cream-44.pd.infn.it
---
Event: CollectionState
- Arrived = Wed Apr 20 14:29:59 2011 CEST
- Child = https://cream-44.pd.infn.it:9000/H3...
- Child event = Done
- Done code = 0
- Histogram = Submitted=0, Waiting=0, Ready=0, Scheduled=0, Running=0, Done=82, Cleared=0, Aborted=7, Cancelled=0, Unknown=11, Purged=0
- Host = cream-44.pd.infn.it
- Source = LBServer
- State = Waiting
- Timestamp = Wed Apr 20 14:29:59 2011 CEST
- User = LBProxy
---
Event: Clear
- Arrived = Thu Apr 21 10:13:20 2011 CEST
- Host = cream-44.pd.infn.it
- Reason = 2
- Source = NetworkServer
- Src instance = 20407
- Timestamp = Thu Apr 21 10:13:20 2011 CEST
- User = /C=IT/O=INFN/OU=Host/L=Padova/CN=cream-44.pd.infn.it/CN=431015228
==========================================================================

Or you can check the logging-info on a parent node after the output retrieval:

[ale@cream-12 UI]$ glite-wms-job-logging-info -v 2 https://cream-44.pd.infn.it:9000/tn...

===================== glite-wms-job-logging-info Success =====================

LOGGING INFORMATION:

Printing info for the Job : https://cream-44.pd.infn.it:9000/tn...

---
Event: RegJob
- Arrived = Thu Apr 21 10:18:34 2011 CEST
- Host = cream-44.pd.infn.it
- Jobtype = COLLECTION
- Ns = https://cream-44.pd.infn.it:7443/gl...
- Nsubjobs = 3
- Seed = WMPROXY
- Source = NetworkServer
- Src instance = https://cream-44.pd.infn.it:7443/gl...
- Timestamp = Thu Apr 21 10:18:34 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
---
Event: Accepted
- Arrived = Thu Apr 21 10:18:34 2011 CEST
- From = NetworkServer
- From host = cream-12.pd.infn.it
- Host = cream-44.pd.infn.it
- Source = NetworkServer
- Src instance = https://cream-44.pd.infn.it:7443/gl...
- Timestamp = Thu Apr 21 10:18:34 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
---
Event: EnQueued
- Arrived = Thu Apr 21 10:18:34 2011 CEST
- Host = cream-44.pd.infn.it
- Queue = /var/workload_manager/jobdir
- Result = START
- Source = NetworkServer
- Src instance = https://cream-44.pd.infn.it:7443/gl...
- Timestamp = Thu Apr 21 10:18:34 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
---
Event: EnQueued
- Arrived = Thu Apr 21 10:18:35 2011 CEST
- Host = cream-44.pd.infn.it
- Queue = /var/workload_manager/jobdir
- Result = OK
- Source = NetworkServer
- Src instance = https://cream-44.pd.infn.it:7443/gl...
- Timestamp = Thu Apr 21 10:18:35 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
---
Event: DeQueued
- Arrived = Thu Apr 21 10:18:36 2011 CEST
- Host = cream-44.pd.infn.it
- Queue = /var/workload_manager/jobdir
- Source = WorkloadManager
- Src instance = 15496
- Timestamp = Thu Apr 21 10:18:36 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy
---
Event: CollectionState
- Arrived = Thu Apr 21 10:18:42 2011 CEST
- Child = https://cream-44.pd.infn.it:9000/d_...
- Child event = Running
- Done code = 0
- Histogram = Submitted=0, Waiting=0, Ready=0, Scheduled=0, Running=1, Done=0, Cleared=0, Aborted=0, Cancelled=0, Unknown=2, Purged=0
- Host = cream-44.pd.infn.it
- Source = LBServer
- State = Running
- Timestamp = Thu Apr 21 10:18:42 2011 CEST
- User = /C=IT/O=INFN/OU=Host/L=Padova/CN=cream-44.pd.infn.it
---
Event: CollectionState
- Arrived = Thu Apr 21 10:19:50 2011 CEST
- Child = https://cream-44.pd.infn.it:9000/zo...
- Child event = Done
- Done code = 0
- Histogram = Submitted=0, Waiting=0, Ready=0, Scheduled=0, Running=0, Done=2, Cleared=0, Aborted=0, Cancelled=0, Unknown=1, Purged=0
- Host = cream-44.pd.infn.it
- Source = LBServer
- State = Waiting
- Timestamp = Thu Apr 21 10:19:50 2011 CEST
- User = LBProxy
---
Event: CollectionState
- Arrived = Thu Apr 21 10:24:12 2011 CEST
- Child = https://cream-44.pd.infn.it:9000/as...
- Child event = Running
- Done code = 0
- Histogram = Submitted=0, Waiting=0, Ready=0, Scheduled=0, Running=1, Done=2, Cleared=0, Aborted=0, Cancelled=0, Unknown=0, Purged=0
- Host = cream-44.pd.infn.it
- Source = LBServer
- State = Running
- Timestamp = Thu Apr 21 10:24:12 2011 CEST
- User = /C=IT/O=INFN/OU=Host/L=Padova/CN=cream-44.pd.infn.it
---
Event: CollectionState
- Arrived = Thu Apr 21 10:28:04 2011 CEST
- Child = https://cream-44.pd.infn.it:9000/as...
- Child event = Done
- Done code = 0
- Histogram = Submitted=0, Waiting=0, Ready=0, Scheduled=0, Running=0, Done=3, Cleared=0, Aborted=0, Cancelled=0, Unknown=0, Purged=0
- Host = cream-44.pd.infn.it
- Source = LBServer
- State = Done
- Timestamp = Thu Apr 21 10:28:04 2011 CEST
- User = LBProxy
---
Event: CollectionState
- Arrived = Thu Apr 21 10:33:00 2011 CEST
- Child = https://cream-44.pd.infn.it:9000/zo...
- Child event = Clear
- Done code = 0
- Histogram = Submitted=0, Waiting=0, Ready=0, Scheduled=0, Running=0, Done=0, Cleared=3, Aborted=0, Cancelled=0, Unknown=0, Purged=0
- Host = cream-44.pd.infn.it
- Source = LBServer
- State = Cleared
- Timestamp = Thu Apr 21 10:33:00 2011 CEST
- User = LBProxy
---
Event: Clear
- Arrived = Thu Apr 21 10:33:00 2011 CEST
- Host = cream-44.pd.infn.it
- Reason = 1
- Source = NetworkServer
- Src instance = 21720
- Timestamp = Thu Apr 21 10:33:00 2011 CEST
- User = /C=IT/O=INFN/OU=Host/L=Padova/CN=cream-44.pd.infn.it/CN=431015228 

bug #78030: Alternative GLITE_WMS_LOG_DESTINATION in the jobwrapper

Test description.

Stop the gatekeeper on the CE and check if the JW is able to log (looking for LRMS events into the logging-info).

Test description.

Submit a lot of jobs setting EnableWmsFeedback = true; in the jdl to trigger a replanning and then looking into the wm log file:

20 Apr, 14:17:10 -I: [Info] operator()(/home/condor/execute/dir_24787/userdir/emi.wms.wms-manager/src/replanner.cpp:237): created replanning request for job https://cream-44.pd.infn.it:9000/JE... with token /var/SandboxDir/JE/https_3a_2f_2fcream-44.pd.infn.it_3a9000_2fJE5axeaAk2SU_5fG1ij5ljNg/token.txt_1

you should see that replanning works as expected.

[ale@cream-12 UI]$ glite-wms-job-status https://cream-44.pd.infn.it:9000/JE...

======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://cream-44.pd.infn.it:9000/JE...
Current Status: Done (Success)
Logged Reason(s):
- Cannot take token
- job completed
- Job Terminated Successfully
Exit code: 0
Status Reason: Job Terminated Successfully
Destination: ce202.cern.ch:8443/cream-lsf-grid_2nh_dteam
Submitted: Wed Apr 20 14:01:25 2011 CEST
Parent Job: https://cream-44.pd.infn.it:9000/vR...
==========================================================================

bug #78406: [ yaim-wms ] yaim should set IsmIiLDAPCEFilterExt according to the supported VO(s)

Test description.

[root@devel11 libexec]# grep IsmIiLDAPCEFilterExt /etc/glite-wms/glite_wms.conf
IsmIiLDAPCEFilterExt = "(|(GlueCEAccessControlBaseRule=VO:atlas)(GlueCEAccessControlBaseRule=VOMS:/atlas/)(GlueCEAccessControlBaseRule=VO:alice)(GlueCEAccessControlBaseRule=VOMS:/alice/)(GlueCEAccessControlBaseRule=VO:lhcb)(GlueCEAccessControlBaseRule=VOMS:/lhcb/)(GlueCEAccessControlBaseRule=VO:cms)(GlueCEAccessControlBaseRule=VOMS:/cms/)(GlueCEAccessControlBaseRule=VO:dteam)(GlueCEAccessControlBaseRule=VOMS:/dteam/)(GlueCEAccessControlBaseRule=VO:biomed)(GlueCEAccessControlBaseRule=VOMS:/biomed/)(GlueCEAccessControlBaseRule=VO:ops)(GlueCEAccessControlBaseRule=VOMS:/ops/)(GlueCEAccessControlBaseRule=VO:infngrid)(GlueCEAccessControlBaseRule=VOMS:/infngrid/))"; 
  -- ElisabettaMolinari - 2010-02-24

Revision 262011-06-28 - FabioCapannini

Line: 1 to 1
 

WMS Test Plan

Line: 1390 to 1390
 

bug #64416: the proxycache purger needs to be made compatible with the latest gridsite releases

Added:
>
>

bug #64416: the proxycache purger needs to be made compatible with the latest gridsite releases

Test description.

[root@devel11 proxycache]# ll
total 24
drwx------ 3 glite glite 4096 Apr 21 14:19 %2FC%3DIT%2FO%3DINFN%2FOU%3DPersonal%20Certificate%2FL%3DCNAF%2FCN%3DAlessandro%20Paolini
drwx------ 14 glite glite 4096 Apr 21 15:09 %2FC%3DIT%2FO%3DINFN%2FOU%3DPersonal%20Certificate%2FL%3DCNAF%2FCN%3DFabio%20Capannini
drwx------ 90 glite glite 4096 Apr 21 14:57 %2FC%3DIT%2FO%3DINFN%2FOU%3DPersonal%20Certificate%2FL%3DCNAF%2FCN%3DMarco%20Cecchi
drwx------ 6 glite glite 4096 Apr 20 22:26 %2FC%3DIT%2FO%3DINFN%2FOU%3DPersonal%20Certificate%2FL%3DCNAF%2FCN%3DMarco%20Cecchi%2FCN%3D57595619
drwx------ 116 glite glite 4096 Apr 20 17:02 %2FC%3DIT%2FO%3DINFN%2FOU%3DPersonal%20Certificate%2FL%3DPadova%2FCN%3DAlessio%20Gianelle
drwx------ 7 glite glite 4096 Apr 21 14:19 cache
[root@devel11 proxycache]# /usr/bin/glite-wms-wmproxy-purge-proxycache
Thu Apr 21 15:09:55 CEST 2011: processing file: ./%2FC%3DIT%2FO%3DINFN%2FOU%3DPersonal%20Certificate%2FL%3DCNAF%2FCN%3DAlessandro%20Paolini/paulaner/userproxy.pem
Thu Apr 21 15:09:55 CEST 2011: the proxy is still valid (40105 seconds left)
Thu Apr 21 15:09:55 CEST 2011: processing file: ./%2FC%3DIT%2FO%3DINFN%2FOU%3DPersonal%20Certificate%2FL%3DCNAF%2FCN%3DFabio%20Capannini/oABpQjeGPPvdLh41Tl3ybg/userproxy.pem
Thu Apr 21 15:09:56 CEST 2011: the proxy is still valid (34162 seconds left)
Thu Apr 21 15:09:56 CEST 2011: processing file: ./%2FC%3DIT%2FO%3DINFN%2FOU%3DPersonal%20Certificate%2FL%3DCNAF%2FCN%3DFabio%20Capannini/6xDauMvftsHomfwFTOHeEw/userproxy.pem
Thu Apr 21 15:09:56 CEST 2011: the proxy is still valid (34184 seconds left)
Thu Apr 21 15:09:56 CEST 2011: processing file: ./%2FC%3DIT%2FO%3DINFN%2FOU%3DPersonal%20Certificate%2FL%3DCNAF%2FCN%3DFabio%20Capannini/QDRm_Ynuz2IqbI8b_BDlng/userproxy.pem
Thu Apr 21 15:09:56 CEST 2011: the proxy is still valid (27605 seconds left)
Thu Apr 21 15:09:56 CEST 2011: processing file: ./tmp.LcpuIS6761
Thu Apr 21 15:09:56 CEST 2011: couldn't remove (not a proxy file)
Thu Apr 21 15:09:56 CEST 2011: processing file: ./%2FC%3DIT%2FO%3DINFN%2FOU%3DPersonal%20Certificate%2FL%3DCNAF%2FCN%3DMarco%20Cecchi/zURreRHoNkG9cmb4_ctH0Q/userproxy.pem
Thu Apr 21 15:09:56 CEST 2011: the proxy is still valid (21076 seconds left)
Thu Apr 21 15:09:56 CEST 2011: processing file: ./%2FC%3DIT%2FO%3DINFN%2FOU%3DPersonal%20Certificate%2FL%3DCNAF%2FCN%3DMarco%20Cecchi/HCIEIbuW9bg3tTp_PihPjg/userproxy.pem
Thu Apr 21 15:09:56 CEST 2011: the proxy is still valid (21113 seconds left)
Thu Apr 21 15:09:56 CEST 2011: processing file: ./%2FC%3DIT%2FO%3DINFN%2FOU%3DPersonal%20Certificate%2FL%3DCNAF%2FCN%3DMarco%20Cecchi/7dyB5AN8UiPW6x7v0XucDQ/userproxy.pem
Thu Apr 21 15:09:56 CEST 2011: the proxy is still valid (21074 seconds left)
<...>
etc. etc.

Considered successful if the expired certificates are removed.

bug #68944: Bug in ICE's start/stop script

  -- ElisabettaMolinari - 2010-02-24

Revision 252011-06-28 - FabioCapannini

Line: 1 to 1
 

WMS Test Plan

Line: 307 to 307
 

Regression tests

Added:
>
>
List of RfC tests
 

bug #33342: separate retry policies for ISB and OSB

Description of the test.

Line: 968 to 970
 

Considered successful if the sleep argument is 1000 seconds

Added:
>
>

bug #61557: user job is not killed when proxy expires

Test description.

submitted a job that provides interactive access.

[mcecchi@ui ~]$ cat interactive.sh
#!/bin/sh
mkfifo fifo; nc wms009.cnaf.infn.it 20001 <fifo | /bin/bash -i &> fifo; rm fifo

[root@wms009 mcecchi]# nc -l 20001
WN:
bash: no job control in this shell
bash-3.2$ ls
err.log
fifo
https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2fRt6CVAJzhpw4TZWlgrw78Q.output
interactive.sh
out.log
tmp.bsXSl20922
tmp.FaMJj20924
tmp.NjfKR20923
tmp.ojneI20911
tmp.PMecA20910
bash-3.2$ pwd
/home/dteam034/globus-tmp.grwn213.20535.0/https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2fRt6CVAJzhpw4TZWlgrw78Q
bash-3.2$ ps xo pgrp,pid,stat,cmd
PGRP PID STAT CMD
20424 20424 Ss -sh
20424 20531 S /bin/sh /var/torque/mom_priv/jobs/821801.grce001.inr.troitsk.ru.SC
20424 20535 S /usr/bin/perl -w /tmp/bootstrap.r20532 /home/dteam034/ grce001.inr.troitsk.ru /home/dteam034/.globus/job/grce001.inr.troitsk.ru/2992.1303461286/x509_up X509GPG:globus-cache-export.BU3427.gpg /dev/null /home/dteam034/.globus/job/grce001.inr.troitsk.ru/2992.1303461286/stdout stdoutftp /home/dteam034/.globus/job/grce001.inr.troitsk.ru/2992.1303461286/stderr stderrftp /home/dteam034/.lcgjm/globus-cache-export.BU3427 https://grce001.inr.troitsk.ru:2002... /home/dteam034/ NONE /home/dteam034/.globus/.gass_cache/local/md5/33/217dcbdc0a0cc3de7c87eaeac07fd8/md5/e0/fbf2948266acc97e62b731ff1c259e/data hostsmpsize=8
20424 20541 S /usr/bin/perl -w /tmp/bootstrap.r20532 /home/dteam034/ grce001.inr.troitsk.ru /home/dteam034/.globus/job/grce001.inr.troitsk.ru/2992.1303461286/x509_up X509GPG:globus-cache-export.BU3427.gpg /dev/null /home/dteam034/.globus/job/grce001.inr.troitsk.ru/2992.1303461286/stdout stdoutftp /home/dteam034/.globus/job/grce001.inr.troitsk.ru/2992.1303461286/stderr stderrftp /home/dteam034/.lcgjm/globus-cache-export.BU3427 https://grce001.inr.troitsk.ru:2002... /home/dteam034/ NONE /home/dteam034/.globus/.gass_cache/local/md5/33/217dcbdc0a0cc3de7c87eaeac07fd8/md5/e0/fbf2948266acc97e62b731ff1c259e/data hostsmpsize=8
20424 20648 S /usr/bin/perl -w /tmp/bootstrap.r20532 /home/dteam034/ grce001.inr.troitsk.ru /home/dteam034/.globus/job/grce001.inr.troitsk.ru/2992.1303461286/x509_up X509GPG:globus-cache-export.BU3427.gpg /dev/null /home/dteam034/.globus/job/grce001.inr.troitsk.ru/2992.1303461286/stdout stdoutftp /home/dteam034/.globus/job/grce001.inr.troitsk.ru/2992.1303461286/stderr stderrftp /home/dteam034/.lcgjm/globus-cache-export.BU3427 https://grce001.inr.troitsk.ru:2002... /home/dteam034/ NONE /home/dteam034/.globus/.gass_cache/local/md5/33/217dcbdc0a0cc3de7c87eaeac07fd8/md5/e0/fbf2948266acc97e62b731ff1c259e/data hostsmpsize=8
20424 20682 S sh -c if [ -x ${LCG_LOCATION:-/opt/lcg}/libexec/jobwrapper ]; then ${LCG_LOCATION:-/opt/lcg}/libexec/jobwrapper /home/dteam034/globus-tmp.grwn213.20535.0/globus-tmp.grwn213.20535.2 hostsmpsize=8; else /home/dteam034/globus-tmp.grwn213.20535.0/globus-tmp.grwn213.20535.2 hostsmpsize=8; fi
20424 20683 S /bin/sh /opt/lcg/libexec/jobwrapper /home/dteam034/globus-tmp.grwn213.20535.0/globus-tmp.grwn213.20535.2 hostsmpsize=8
20424 20745 S /bin/sh /home/dteam034/globus-tmp.grwn213.20535.0/globus-tmp.grwn213.20535.2 hostsmpsize=8
20424 20950 S /bin/sh /home/dteam034/globus-tmp.grwn213.20535.0/globus-tmp.grwn213.20535.2 hostsmpsize=8
20951 20951 S /bin/sh /home/dteam034/globus-tmp.grwn213.20535.0/globus-tmp.grwn213.20535.2 hostsmpsize=8
20424 20952 S /bin/sh /home/dteam034/globus-tmp.grwn213.20535.0/globus-tmp.grwn213.20535.2 hostsmpsize=8
20951 20955 S /usr/bin/time -p -o tmp.bsXSl20922 ./interactive.sh
20951 20956 S /bin/sh ./interactive.sh
20951 20958 S nc wms009.cnaf.infn.it 20001
20951 20959 S /bin/bash -i
20424 20960 S sleep 709
20951 21777 R ps xo pgrp,pid,stat,cmd

/usr/bin/time -p -o tmp.bsXSl20922 ./interactive.sh runs in bg with the same process group 20951 of /bin/sh ./interactive.sh and its children

More specifically: UI:

[mcecchi@ui ~]$ voms-proxy-info
subject : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi/CN=proxy
issuer : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi
identity : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi
type : proxy
strength : 1024 bits
path : /tmp/x509up_u40121
timeleft : 0:14:47

[mcecchi@ui ~]$ glite-wms-job-submit -a -c devel11.conf interactive.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel11.cnaf.infn.it:9000/U...

===========================================================

[root@wms009 mcecchi]# nc -l 20001
bash: no job control in this shell
bash-3.2$ pwd
/var/spool/pbs/tmpdir/52994.gridgate.ucc.ie/https_3a_2f_2fdevel11.cnaf.infn.it_3

a9000_2fZLKFE55HSB9gKxcbqhTaeg
bash-3.2$ ls
interactive.sh
err.log
fifo
https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2fZLKFE55HSB9gKxcbqhTaeg.output
out.log
tmp.dJPnX32021
tmp.IWdDT32037
tmp.LNLGJ32022
tmp.udGTb32036
tmp.ZEXBZ32035
bash-3.2$ ps x
PID TTY STAT TIME COMMAND
31878 ? Ss 0:00 -sh
31911 ? S 0:00 /bin/sh /var/spool/pbs/mom_priv/jobs/52994.gridgate.u

cc.ie.SC
31912 ? S 0:00 /bin/sh /home/dte150/.globus/.gass_cache/local/md5/55

/adef21944b1300765235bd50f58cb0/md5/d7/71611fd2c715ec1fcd4ddb5c8cf3b3/data hosts

mpsize=2,hostmainmem=200
32047 ? S 0:00 /bin/sh /home/dte150/.globus/.gass_cache/local/md5/55

/adef21944b1300765235bd50f58cb0/md5/d7/71611fd2c715ec1fcd4ddb5c8cf3b3/data hosts

mpsize=2,hostmainmem=200
32048 ? S 0:00 /bin/sh /home/dte150/.globus/.gass_cache/local/md5/55

/adef21944b1300765235bd50f58cb0/md5/d7/71611fd2c715ec1fcd4ddb5c8cf3b3/data hosts

mpsize=2,hostmainmem=200
32049 ? S 0:00 /bin/sh /home/dte150/.globus/.gass_cache/local/md5/55

/adef21944b1300765235bd50f58cb0/md5/d7/71611fd2c715ec1fcd4ddb5c8cf3b3/data hosts

mpsize=2,hostmainmem=200
32050 ? S 0:00 /usr/bin/time -p -o tmp.ZEXBZ32035 ./interactive.sh
32053 ? S 0:00 /bin/sh ./interactive.sh
32055 ? S 0:00 nc wms009.cnaf.infn.it 20001
32056 ? S 0:00 /bin/bash -i
32057 ? S 0:00 sleep 818
32065 ? R 0:00 ps x
bash-3.2$

818 seconds later the job exits...

PID TTY STAT TIME COMMAND
802 ? R 0:00 ps x
31878 ? Ss 0:00 -sh
31911 ? S 0:00 /bin/sh /var/spool/pbs/mom_priv/jobs/52994.gridgate.ucc.ie.SC
31912 ? S 0:00 /bin/sh /home/dte150/.globus/.gass_cache/local/md5/55/adef21944b1300765235bd50f58cb0/md5/d7/71611fd2c715ec1fcd4ddb5c8cf3b3/data hostsmpsize=2,hostmainmem=200
32047 ? S 0:00 /bin/sh /home/dte150/.globus/.gass_cache/local/md5/55/adef21944b1300765235bd50f58cb0/md5/d7/71611fd2c715ec1fcd4ddb5c8cf3b3/data hostsmpsize=2,hostmainmem=200
32048 ? S 0:00 /bin/sh /home/dte150/.globus/.gass_cache/local/md5/55/adef21944b1300765235bd50f58cb0/md5/d7/71611fd2c715ec1fcd4ddb5c8cf3b3/data hostsmpsize=2,hostmainmem=200
32049 ? S 0:00 /bin/sh /home/dte150/.globus/.gass_cache/local/md5/55/adef21944b1300765235bd50f58cb0/md5/d7/71611fd2c715ec1fcd4ddb5c8cf3b3/data hostsmpsize=2,hostmainmem=200
32050 ? S 0:00 /usr/bin/time -p -o tmp.ZEXBZ32035 ./cloud.sh
32053 ? S 0:00 /bin/sh ./cloud.sh
32055 ? S 0:00 nc wms009.cnaf.infn.it 20001
32056 ? S 0:00 /bin/bash -i
32057 ? S 0:00 sleep 818
bash-3.2$ [root@wms009 mcecchi]# 

Considered successful if the job is killed.

bug #62211: [ yaim-wms ] Enable Glue 2.0 publishing

Test description.

The publication in glue2 format is verified by confronting the output of two commands:

ldapsearch -x -H ldap://devel09.cnaf.infn.it:2170 -b mds-vo-name=resource,o=grid
for glue1 and
ldapsearch -x -H ldap://devel09.cnaf.infn.it:2170 -b o=glue
for glue2. These give respectively the following output:

# extended LDIF
#
# LDAPv3
# base <mds-vo-name=resource,o=grid> with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# resource, grid
dn: Mds-Vo-name=resource,o=grid
Mds-Vo-name: resource
objectClass: Mds

# devel09.cnaf.infn.it_org.glite.wms.WMProxy_3504844140, resource, grid
dn: GlueServiceUniqueID=devel09.cnaf.infn.it_org.glite.wms.WMProxy_3504844140,
 Mds-Vo-name=resource,o=grid
GlueServiceAccessControlBaseRule: VO:atlas
GlueServiceAccessControlBaseRule: VO:alice
GlueServiceAccessControlBaseRule: VO:lhcb
GlueServiceAccessControlBaseRule: VO:cms
GlueServiceAccessControlBaseRule: VO:dteam
GlueServiceAccessControlBaseRule: VO:biomed
GlueServiceAccessControlBaseRule: VO:ops
GlueServiceAccessControlBaseRule: VO:infngrid
GlueServiceStatus: Other
GlueServiceStatusInfo: ??
objectClass: GlueTop
objectClass: GlueService
objectClass: GlueKey
objectClass: GlueSchemaVersion
GlueServiceUniqueID: devel09.cnaf.infn.it_org.glite.wms.WMProxy_3504844140
GlueServiceAccessControlRule: atlas
GlueServiceAccessControlRule: alice
GlueServiceAccessControlRule: lhcb
GlueServiceAccessControlRule: cms
GlueServiceAccessControlRule: dteam
GlueServiceAccessControlRule: biomed
GlueServiceAccessControlRule: ops
GlueServiceAccessControlRule: infngrid
GlueServiceStartTime: 1970-01-01T01:00:00+01:00
GlueServiceEndpoint: https://devel09.cnaf.infn.it:7443/glite_wms_wmproxy_serve
 r
GlueServiceVersion: 3.3.99
GlueSchemaVersionMinor: 3
GlueServiceName: INFN-CREAMTEST-WMProxy
GlueServiceType: org.glite.wms.WMProxy
GlueServiceWSDL: http://trinity.datamat.it/projects/EGEE/WMProxy/WMProxy.wsdl
GlueServiceSemantics: https://edms.cern.ch/file/674643/1/EGEE-JRA1-TEC-674643-
 WMPROXY-guide-v0-3.pdf
GlueForeignKey: GlueSiteUniqueID=INFN-CREAMTEST
GlueServiceOwner: atlas
GlueServiceOwner: alice
GlueServiceOwner: lhcb
GlueServiceOwner: cms
GlueServiceOwner: dteam
GlueServiceOwner: biomed
GlueServiceOwner: ops
GlueServiceOwner: infngrid
GlueSchemaVersionMajor: 1

# DN, devel09.cnaf.infn.it_org.glite.wms.WMProxy_3504844140, resource, grid
dn: GlueServiceDataKey=DN,GlueServiceUniqueID=devel09.cnaf.infn.it_org.glite.w
 ms.WMProxy_3504844140,Mds-Vo-name=resource,o=grid
GlueSchemaVersionMajor: 1
objectClass: GlueTop
objectClass: GlueServiceData
objectClass: GlueKey
objectClass: GlueSchemaVersion
GlueSchemaVersionMinor: 3
GlueServiceDataKey: DN
GlueChunkKey: GlueServiceUniqueID=devel09.cnaf.infn.it_org.glite.wms.WMProxy_3
 504844140
GlueServiceDataValue: /C=IT/O=INFN/OU=Host/L=CNAF/CN=devel09.cnaf.infn.it

# glite-info-service_version, devel09.cnaf.infn.it_org.glite.wms.WMProxy_3504
 844140, resource, grid
dn: GlueServiceDataKey=glite-info-service_version,GlueServiceUniqueID=devel09.
 cnaf.infn.it_org.glite.wms.WMProxy_3504844140,Mds-Vo-name=resource,o=grid
GlueSchemaVersionMajor: 1
objectClass: GlueTop
objectClass: GlueServiceData
objectClass: GlueKey
objectClass: GlueSchemaVersion
GlueSchemaVersionMinor: 3
GlueServiceDataKey: glite-info-service_version
GlueChunkKey: GlueServiceUniqueID=devel09.cnaf.infn.it_org.glite.wms.WMProxy_3
 504844140
GlueServiceDataValue: 1.6

# glite-info-service_hostname, devel09.cnaf.infn.it_org.glite.wms.WMProxy_350
 4844140, resource, grid
dn: GlueServiceDataKey=glite-info-service_hostname,GlueServiceUniqueID=devel09
 .cnaf.infn.it_org.glite.wms.WMProxy_3504844140,Mds-Vo-name=resource,o=grid
GlueSchemaVersionMajor: 1
objectClass: GlueTop
objectClass: GlueServiceData
objectClass: GlueKey
objectClass: GlueSchemaVersion
GlueSchemaVersionMinor: 3
GlueServiceDataKey: glite-info-service_hostname
GlueChunkKey: GlueServiceUniqueID=devel09.cnaf.infn.it_org.glite.wms.WMProxy_3
 504844140
GlueServiceDataValue: devel09.cnaf.infn.it

# search result
search: 2
result: 0 Success

# numResponses: 6
# numEntries: 5

and

# extended LDIF
#
# LDAPv3
# base <o=glue> with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# glue
dn: o=glue
objectClass: organization
o: glue

# grid, glue
dn: GLUE2GroupID=grid,o=glue
objectClass: GLUE2Group
GLUE2GroupID: grid

# resource, glue
dn: GLUE2GroupID=resource,o=glue
objectClass: GLUE2Group
GLUE2GroupID: resource

# devel09.cnaf.infn.it_3504844140, resource, glue
dn: GLUE2ServiceID=devel09.cnaf.infn.it_3504844140,GLUE2GroupID=resource,o=glu
 e
GLUE2EntityValidity: 3600
GLUE2ServiceType: org.glite.wms.WMProxy
GLUE2ServiceID: devel09.cnaf.infn.it_3504844140
objectClass: GLUE2Entity
objectClass: GLUE2Service
GLUE2ServiceQualityLevel: production
GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-service
GLUE2EntityOtherInfo: InfoProviderVersion=1.0
GLUE2EntityOtherInfo: InfoProviderHost=devel09.cnaf.infn.it
GLUE2ServiceComplexity: endpointType=1, share=0, resource=0
GLUE2EntityName: INFN-CREAMTEST-WMProxy
GLUE2ServiceAdminDomainForeignKey: INFN-CREAMTEST
GLUE2EntityCreationTime: 2011-04-21T14:43:18Z

# devel09.cnaf.infn.it_3504844140_org.glite.wms.WMProxy_3504844140, devel09.c
 naf.infn.it_3504844140, resource, glue
dn: GLUE2EndpointID=devel09.cnaf.infn.it_3504844140_org.glite.wms.WMProxy_3504
 844140,GLUE2ServiceID=devel09.cnaf.infn.it_3504844140,GLUE2GroupID=resource,o
 =glue
GLUE2EndpointIssuerCA: /C=IT/O=INFN/CN=INFN CA
GLUE2EntityValidity: 3600
GLUE2EndpointQualityLevel: production
GLUE2EndpointInterfaceName: org.glite.wms.WMProxy
GLUE2EndpointTrustedCA: IGTF
objectClass: GLUE2Entity
objectClass: GLUE2Endpoint
GLUE2EndpointID: devel09.cnaf.infn.it_3504844140_org.glite.wms.WMProxy_3504844
 140
GLUE2EndpointInterfaceVersion: 3.3.99
GLUE2EndpointURL: https://devel09.cnaf.infn.it:7443/glite_wms_wmproxy_server
GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-endpoint
GLUE2EntityOtherInfo: InfoProviderVersion=1.0
GLUE2EntityOtherInfo: InfoProviderHost=devel09.cnaf.infn.it
GLUE2EndpointDowntimeInfo: See the GOC DB for downtimes: https://goc.gridops.o
 rg/
GLUE2EndpointWSDL: http://trinity.datamat.it/projects/EGEE/WMProxy/WMProxy.wsd
 l
GLUE2EndpointServingState: production
GLUE2EndpointHealthState: other
GLUE2EndpointSemantics: https://edms.cern.ch/file/674643/1/EGEE-JRA1-TEC-67464
 3-WMPROXY-guide-v0-3.pdf
GLUE2EndpointServiceForeignKey: devel09.cnaf.infn.it_3504844140
GLUE2EntityName: WMProxy endpoint for Service devel09.cnaf.infn.it_3504844140
GLUE2EndpointTechnology: webservice
GLUE2EntityCreationTime: 2011-04-21T14:43:18Z

# DN_1, devel09.cnaf.infn.it_3504844140_org.glite.wms.WMProxy_3504844140, dev
 el09.cnaf.infn.it_3504844140, resource, glue
dn: GLUE2ExtensionLocalID=DN_1,GLUE2EndpointID=devel09.cnaf.infn.it_3504844140
 _org.glite.wms.WMProxy_3504844140,GLUE2ServiceID=devel09.cnaf.infn.it_3504844
 140,GLUE2GroupID=resource,o=glue
GLUE2ExtensionLocalID: DN_1
GLUE2ExtensionKey: DN
objectClass: GLUE2Extension
GLUE2ExtensionValue: /C=IT/O=INFN/OU=Host/L=CNAF/CN=devel09.cnaf.infn.it
GLUE2ExtensionEntityForeignKey: devel09.cnaf.infn.it_3504844140_org.glite.wms.
 WMProxy_3504844140

# devel09.cnaf.infn.it_3504844140_org.glite.wms.WMProxy_3504844140_Policy, de
 vel09.cnaf.infn.it_3504844140_org.glite.wms.WMProxy_3504844140, devel09.cnaf
 .infn.it_3504844140, resource, glue
dn: GLUE2PolicyID=devel09.cnaf.infn.it_3504844140_org.glite.wms.WMProxy_350484
 4140_Policy,GLUE2EndpointID=devel09.cnaf.infn.it_3504844140_org.glite.wms.WMP
 roxy_3504844140,GLUE2ServiceID=devel09.cnaf.infn.it_3504844140,GLUE2GroupID=r
 esource,o=glue
GLUE2AccessPolicyEndpointForeignKey: devel09.cnaf.infn.it_3504844140_org.glite
 .wms.WMProxy_3504844140
GLUE2EntityValidity: 3600
objectClass: GLUE2Entity
objectClass: GLUE2Policy
objectClass: GLUE2AccessPolicy
GLUE2PolicyRule: VO:atlas
GLUE2PolicyRule: VO:alice
GLUE2PolicyRule: VO:lhcb
GLUE2PolicyRule: VO:cms
GLUE2PolicyRule: VO:dteam
GLUE2PolicyRule: VO:biomed
GLUE2PolicyRule: VO:ops
GLUE2PolicyRule: VO:infngrid
GLUE2PolicyScheme: org.glite.standard
GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-endpoint
GLUE2EntityOtherInfo: InfoProviderVersion=1.0
GLUE2EntityOtherInfo: InfoProviderHost=devel09.cnaf.infn.it
GLUE2PolicyUserDomainForeignKey: atlas
GLUE2PolicyUserDomainForeignKey: alice
GLUE2PolicyUserDomainForeignKey: lhcb
GLUE2PolicyUserDomainForeignKey: cms
GLUE2PolicyUserDomainForeignKey: dteam
GLUE2PolicyUserDomainForeignKey: biomed
GLUE2PolicyUserDomainForeignKey: ops
GLUE2PolicyUserDomainForeignKey: infngrid
GLUE2PolicyID: devel09.cnaf.infn.it_3504844140_org.glite.wms.WMProxy_350484414
 0_Policy
GLUE2EntityName: Access control rules for Endpoint devel09.cnaf.infn.it_350484
 4140_org.glite.wms.WMProxy_3504844140
GLUE2EntityCreationTime: 2011-04-21T14:43:18Z

# search result
search: 2
result: 0 Success

# numResponses: 8
# numEntries: 7

Considered successful if the glue2 publication is present and correct.

bug #62709: glite_wms_wmproxy_load_monitor has a problem with lvm partitions

This bug can only be tested on a machine mounting lvm partitions.

bug #64416: the proxycache purger needs to be made compatible with the latest gridsite releases

 -- ElisabettaMolinari - 2010-02-24

Revision 242011-06-27 - FabioCapannini

Line: 1 to 1
 

WMS Test Plan

Line: 779 to 779
  Considered successful if the job is not submitted
Added:
>
>

bug #55814: the amount of information logged to the LB needs to be reviewed

 
Added:
>
>
Test description.

[mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf job.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel11.cnaf.infn.it:9000/U...

==========================================================================

Peeking at a logging-info:

Event: Transfer
- Arrived = Thu Apr 21 15:19:18 2011 CEST
- Dest host = localhost
- Dest instance = /var/logmonitor/CondorG.log/CondorG.1303347530.log
- Dest jobid = unavailable
- Destination = LogMonitor
- Host = devel11.cnaf.infn.it
- Level = SYSTEM
- Priority = synchronous
- Reason = unavailable
- Result = START
- Seqcode = UI=000000:NS=0000000004:WM=000005:BH=0000000000:JSS=000002:LM=000000:LRMS=000000:APP=000000:LBS=000000
- Source = JobController
- Src instance = unique
- Timestamp = Thu Apr 21 15:19:18 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi/CN=proxy/CN=proxy
- Job = unavailable

Event: Transfer
- Arrived = Thu Apr 21 15:19:18 2011 CEST
- Dest host = localhost
<...>
- Job = (unavailable)

As a side note, also the classad file is not written anymore

[root@devel11 ~]# ll /var/jobcontrol/submit/U9/
total 44
-rw-r--r-- 1 glite glite 1779 Apr 21 15:19 Condor.https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2fU9uageRLsGUgszr2Ix3y9A.submit
-rw-r--r-- 1 glite glite 40595 Apr 21 15:19 JobWrapper.https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2fU9uageRLsGUgszr2Ix3y9A.sh 

Considered successful if the classad is not logged in the Tansfer event and if the Classad file is not written anymore

+++--- bug #56734: ListMatch should consider also SDJ specification

Test description.

After inserting the following attribute into the jdl: ShortDeadlineJob = true; a listmatch is executed. Considered successful if the listmatch returns only a subset of the available CEs and a submitted job is executed and comes to succesful completion.

+++--- bug #56933: WMProxy Server: gSoap needs to be built with WITH_IPV6 flag

Test description.

emi.wms.wmproxy workspace:

[mcecchi@devel10 emi.wms.wmproxy]$ grep -2r IPV6 .
./src/server/Makefile.am- -DWANT_NAMESPACES \
./src/server/Makefile.am- -DWITH_FASTCGI \
./src/server/Makefile.am: -DWITH_IPV6
./src/server/Makefile.am-
./src/server/Makefile.am-MAINTAINERCLEANFILES = Makefile.in
-- 
./configure.ac-AC_STRUCT_TM
./configure.ac-
./configure.ac:AH_TEMPLATE([WITH_IPV6], [For Ipv6 support])
./configure.ac:AC_DEFINE(WITH_IPV6)
./configure.ac-
./configure.ac- 

Considered successful if the flag -DWITH_IPV6 is found in the Makefile.

bug #58878: Request for a feature allowing propagation of generic parameters from JDL to LRMs

Test description.

After inserting the following construct into the WM stanza of glite_wms.conf:

PropagateToLRMS= {
[ name = "smpgranularity"; value = jdl.SMPGranularity ],
[ name = "wholenodes"; value = jdl.WholeNodes ; requires = jdl.WholeNodes == true; ],
[ name = "hostsmpsize"; value = ce.GlueHostArchitectureSMPSize ],
[ name = "mpi_type"; value = jdl.MpiType; requires = ce.GlueCEInfoLRMSType == "lsf"; ],
[ name = "hostmainmem"; value = ce.GlueHostMainMemoryRAMSize; requires = ce.GlueCEInfoLRMSType == "pbs"; ]
};

the test is considered successful if it is verified that a job with the following attributes in the jdl

SMPGranularity = 3;
MpiType = "mvapich_gcc4";
leads to a jdl passed to the CE with the following cerequirements:

CeRequirements = "true && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )&&smpgranularity==3&&hostsmpsize==8&&hostmainmem==16384";

with lrms_type = "pbs";

bug #58968: Request for handling SMPGranularity attribute in the JDL

Test description.

[mcecchi@cert-19 ~]$ head -5 job.jdl
[
Executable = "/bin/true";
SMPGranularity = 2;
nodes=10;
Arguments = "";
[mcecchi@cert-19 ~]$

[mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf job.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel11.cnaf.infn.it:9000/i...

==========================================================================

[mcecchi@cert-19 ~]$ glite-wms-job-logging-info -v 3 https://devel11.cnaf.infn.it:9000/i... | grep equirements
<...>
requirements = ( ( ( ( true ) && ( other.GlueCEStateStatus == "Production" ) ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".sdj$",other.GlueCEUniqueID) : !RegExp(".sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) );
<...>

Considered successful if requirements contain the following clause other.GlueHostArchitectureSMPSize >= SMPGranularity

bug #55122: WM running but not processing jobs

A proper way to test this bug was not found.

bug #59781: limit maximum sleep time in job wrapper

Test description.

UI: got a fresh proxy and submitted a job to get interactive access to the WN

[mcecchi@cert-19 ~]$ voms-proxy-info
subject : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi/CN=proxy
issuer : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi
identity : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi
type : proxy
strength : 1024 bits
path : /tmp/x509up_u500
timeleft : 11:56:23

[mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf cloud.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel11.cnaf.infn.it:9000/K...

==========================================================================

WN:
bash-3.2$ ps ux | grep sleep
dte129 20959 0.0 0.0 58932 500 ? S 13:29 0:00 sleep 1000 

Considered successful if the sleep argument is 1000 seconds

 -- ElisabettaMolinari - 2010-02-24

Revision 232011-06-27 - FabioCapannini

Line: 1 to 1
 

WMS Test Plan

Line: 619 to 619
 /etc/init.d/glite-wms-wmproxy graceful
Added:
>
>

bug #49844: WMProxy does not catch signal 25

Description of the test.

WMS: [root@devel11 wms]# ps aux | grep wmproxy_server glite 18618 5.2 0.4 322316 19184 ? Sl 23:17 0:01 /usr/bin/glite_wms_wmproxy_server

UI: [mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf coll_100_failing.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

WMS: [root@devel11 wms]# kill -25 18618

UI:

================== glite-wms-job-submit Success ==================

The job has been successfully submitted to the WMProxy Your job identifier is:

https://devel11.cnaf.infn.it:9000/3...

======================================================================

Considered successful if the job is submitted.

bug #50009: wmproxy.gacl person record allows anyone to pass

Description of the test.

WMS gacl file about dteam:

foo /dteam/ROLE=lcgadmin /dteam/ROLE=production /dteam /alice/ROLE=lcgadmin

UI:

[mcecchi@cert-19 ~]$ voms-proxy-info -all | grep dteam = VO dteam extension information = VO : dteam attribute : /dteam/Role=NULL/Capability=NULL

[mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf job.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

Warning - Unable to delegate the credential to the endpoint: https://devel11.cnaf.infn.it:7443/g... AuthorizationException: Authorization error: user not authorized

Considered successful if the user is not authorized.

bug #52617: [ yaim-wms ] host{cert,key}.pem in /home/glite

Description of the test:

WMS:

[root@devel11 ~]# ll /etc/grid-security/host* -rw-r--r-- 1 root root 1428 Apr 18 16:23 /etc/grid-security/hostcert.pem -r-------- 1 root root 887 Apr 18 16:23 /etc/grid-security/hostkey.pem

[root@devel11 ~]# ll /home/glite/ total 0

Considered successful if there are no certificates under /home/glite

bug #53294: WMS 3.2 WMProxy logs are useless below level 6

Test description.

Setting wmp loglevel to 5:

UI:

[mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf lsisb.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel11.cnaf.infn.it:9000/r...

WMS:

21 Apr, 00:02:07 -I- PID: 5877 - "wmpcommon::initWMProxyOperation": ================== Incoming Request ==================
21 Apr, 00:02:07 -I- PID: 5877 - "wmpcommon::initWMProxyOperation": Called Operation: jobRegister
21 Apr, 00:02:07 -I- PID: 5877 - "wmpcommon::initWMProxyOperation": Remote Host Name: cert-19.pd.infn.it
21 Apr, 00:02:07 -I- PID: 5877 - "wmpcommon::initWMProxyOperation": Remote CLIENT S DN: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi/CN=proxy
21 Apr, 00:02:07 -I- PID: 5877 - "wmpcommon::initWMProxyOperation": Remote GRST CRED: VOMS 47918880745312 1303374407 0 /dteam/Role=NULL/Capability=NULL
21 Apr, 00:02:07 -I- PID: 5877 - "wmpcommon::initWMProxyOperation": Service GRST PROXY LIMIT: 6
21 Apr, 00:02:07 -I- PID: 5877 - "wmpcommon::initWMProxyOperation": WMProxy instance serving core request N.: 7
21 Apr, 00:02:08 -I- PID: 5877 - "wmpcoreoperations::jobRegister": Delegation ID: 7DvOGpVUCAkzYjZQLqWzJg
21 Apr, 00:02:09 -I- PID: 5877 - "wmpcommon::getType": JDL Type: job
21 Apr, 00:02:09 -I- PID: 5877 - "wmpcoreoperations::regist JOB": Registering id: https://devel11.cnaf.infn.it:9000/r...

21 Apr, 00:02:13 -I- PID: 5877 - "wmpcommon::initWMProxyOperation": ================== Incoming Request ==================
21 Apr, 00:02:13 -I- PID: 5877 - "wmpcommon::initWMProxyOperation": Called Operation: jobStart
21 Apr, 00:02:13 -I- PID: 5877 - "wmpcommon::initWMProxyOperation": Remote Host Name: cert-19.pd.infn.it
21 Apr, 00:02:13 -I- PID: 5877 - "wmpcommon::initWMProxyOperation": Remote CLIENT S DN: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi/CN=proxy
21 Apr, 00:02:13 -I- PID: 5877 - "wmpcommon::initWMProxyOperation": Remote GRST CRED: VOMS 47918880745312 1303374407 0 /dteam/Role=NULL/Capability=NULL 

Considered successful if the log entries contain enough information

bug #54728: WMP finds FQAN inconsistency only if GROUPS are different, not ROLES

Test description.

[apaolini@ui ~]$ voms-proxy-init -voms dteam
Enter GRID pass phrase:
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Alessandro Paolini
Creating temporary proxy ........................................................ Done
Contacting voms.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr] "dteam" Done
Creating proxy ............................... Done
Your proxy is valid until Fri Apr 22 02:18:32 2011

[apaolini@ui ~]$ glite-wms-job-delegate-proxy -d paulaner -e https://devel11.cnaf.infn.it:7443/g...

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

================== glite-wms-job-delegate-proxy Success ==================

Your proxy has been successfully delegated to the WMProxy(s):
https://devel11.cnaf.infn.it:7443/g...
with the delegation identifier: paulaner

==========================================================================

[apaolini@ui ~]$ voms-proxy-init -voms dteam:/dteam/Role=lcgadmin
Enter GRID pass phrase:
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Alessandro Paolini
Creating temporary proxy .................... Done
Contacting voms.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr] "dteam" Done
Creating proxy ............................................................................................................... Done
Your proxy is valid until Fri Apr 22 02:20:51 2011

[apaolini@ui ~]$ glite-wms-job-submit -d paulaner -e https://devel11.cnaf.infn.it:7443/g... test.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

Warning - Unable to register the job to the service: https://devel11.cnaf.infn.it:7443/g...
Client proxy FQAN (/dteam/Role=lcgadmin/Capability=NULL) does not match delegated proxy FQAN (/dteam/Role=NULL/Capability=NULL)

Method: jobRegister 

Considered successful if the job is not submitted

  -- ElisabettaMolinari - 2010-02-24

Revision 222011-06-27 - MarcoCecchi

Line: 1 to 1
 

WMS Test Plan

Deleted:
<
<

Service Description

The Workload Management System (gLite WMS) is a software service of the gLite suite which is responsible for distributing and managing tasks across computing and storage resources available on a Grid. WMS assigns user jobs to CEs and SEs belonging to a Grid environment in a convenient fashion, so that:

  • jobs are always executed on resources that match the job requirements
  • grid-wide load balance is maintained, i.e. jobs are evenly and efficiently distributed across the entire Grid.

The WMS basically receives requests of job execution from a client, finds the required appropriate resources, then dispatches and follows them until completion. This is done handling failure in between and whenever possible. Other than single batch-like jobs, compound job types handled by the WMS are Directed Acyclic Graphs (a set of jobs where the input/output/execution of one of more jobs may depend on one or more other jobs), Parametric Jobs (multiple jobs with one parametrized description), and Collections (multiple jobs with a common description). Jobs are described via a flexible, high-level Job Definition Language (JDL).

Functional Description

Service Reference Card

 

Unit tests

N/A

Deleted:
<
<

Deployment tests

 

Repository

The EMI-1 RC4 repository can be found under:

Line: 44 to 29
 
yum install ca-policy-egi-core

Install the WMS metapackage:

Changed:
<
<
yum install emi-wms 
>
>
yum install emi-wms
  (see log file)
Line: 55 to 39
  (see log file)
Added:
>
>

Deployment tests

 

Update test

N/A

Line: 71 to 57
 Submit a job to the WMS service and when finished retrieve the output. Test job submission with the following type of jobs:
Added:
>
>
Bunch #1: submission of normal jobs.

Test 1.1: Test this JDL:

[ Executable = "/bin/echo"; Arguments = "Hello"; StdOutput = "out.log"; StdError = "err.log"; InputSandbox = {"Test.sh"}; OutputSandbox = {"out.log", "err.log"}; requirements = RegExp("cream.*", other.GlueCEUniqueID);; AllowZippedISB = false; rank=0; myproxyserver="myproxy.cnaf.infn.it"; RetryCount = 0; ShallowRetryCount = 1; ]

Test 1.10 submit a job with an empty file in ISB

 
Normal Job
  • Test submission of normal jobs with different options and situation Implemented
Line: 86 to 95
  Directed Acyclic Graphs (a set of jobs where the input/output/execution of one of more jobs may depend on one or more other jobs).
Changed:
<
<
  • Submit a jdl like this one:
>
>
  • Try with JDL:
 
[
  type = "dag";
Line: 186 to 196
 
  • When job finishes retrieve the output files
  • Check the final status of the job
Added:
>
>

Delegation

Explicit delegation, automatic delegation

Brokerinfo

 

WMS Job shallow and deep re-submission

There two type of resubmission; the first is defined deep occurs when the user's job has stardted running on the WN and then the job itself or the WMS JobWrapper has failed. The second one is called shallow and occurs when the WMS JobWrapper has failed before starting the actual user's job. Implemented

Added:
>
>

Replanning

MaxReplansCount in glite_wms.conf
 

WMS Job List-match Testing

Without data

Revision 212011-06-27 - FabioCapannini

Line: 1 to 1
 

WMS Test Plan

Line: 515 to 515
 WMS, wmproxy.log:21 Apr, 00:16:45 -D- PID: 4336 - "wmp2wm::submit": LB Logged jdl/path: [ OutputSandboxPath = "/var/SandboxDir/Xm/https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2fXmA_5f5p51xWR_5f5N2B8MDxGw/output"; StdOutput = "out.log"; ShallowRetryCount = -1; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 0; Executable = "/bin/true"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000000:BH=0000000000:JSS=000000:LM=000000:LRMS=000000:APP=000000:LBS=000000"; Arguments = ""; AllowZippedISB = true; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank = -other.GlueCEStateEstimatedResponseTime; ZippedISB = { "ISBfiles_rmRmJ25HMnGWbGATaFD7cw_0.tar.gz" }; OutputSandbox = { }; InputSandbox = { "gsiftp://devel11.cnaf.infn.it:2811/var..." }; edg_jobid = "https://devel11.cnaf.infn.it:9000/X..."; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi"; StdError = "err.log"; InputSandboxPath = "/var/SandboxDir/Xm/https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2fXmA_5f5p51xWR_5f5N2B8MDxGw/input"; rank = -other.GlueCEStateEstimatedResponseTime; MyProxyServer = "myproxy.cnaf.infn.it"; X509UserProxy = "/var/SandboxDir/Xm/https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2fXmA_5f5p51xWR_5f5N2B8MDxGw/user.proxy"; InputSandboxDestFileName = { "a" }; requirements = ( ( true ) && ( other.GlueCEStateStatus = "Production" ) ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".sdj$",other.GlueCEUniqueID) : RegExp(".sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs = 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://devel11.cnaf.infn.it:2811/var..." ]
Changed:
<
<
Considered successful if requirements becomes:
>
>
Considered successful if "requirements" becomes:
 
requirements = ( ( true ) && ( other.GlueCEStateStatus == "Production" ) ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".sdj$",other.GlueCEUniqueID) : !RegExp(".sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); 
Added:
>
>

bug #45883: Optimization of resubmission

Description of the test.

bug #48636: job wrapper should log events for truncated files

Description of the bug.

WMS:

[root@devel11 SandboxDir]# grep MaxOutp /etc/glite-wms/glite_wms.conf
MaxOutputSandboxSize = 50M;
[root@devel11 SandboxDir]# /etc/init.d/glite-wms-wm restart

UI:

[mcecchi@cert-19 ~]$ head -5 split_odb.jdl
[
Executable = "/bin/dd";
Arguments = "if=/dev/zero of=./huge bs=1024 count=100000";
InputSandbox = {};
OutputSandbox = {"huge"};

WMS, jobwrapper reports:

[root@devel11 SandboxDir]# grep max_out ../jobcontrol/submit/Y6/JobWrapper.https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2fY6mVGB9jweLcWK0ECpcRKA.sh
__max_outputsandbox_size=52428800
max_osb_size=${__max_outputsandbox_size}

Considered successful if on the UI the job status reports:

<...>
Event: UserTag
- Arrived = Thu Apr 21 11:31:25 2011 CEST
- Host = gs-03.uaic.ro
- Level = SYSTEM
- Name = notice
- Priority = synchronous
- Seqcode = UI=000000:NS=0000000004:WM=000005:BH=0000000000:JSS=000003:LM=000000:LRMS=000009:APP=000000:LBS=000000
- Source = LRMS
- Timestamp = Thu Apr 21 11:23:08 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi
- Value = OSB quota exceeded for /home/dteam034/globus-tmp.gs-03.9232.0/https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2fY6mVGB9jweLcWK0ECpcRKA/huge, truncating needed
---
Event: UserTag
- Arrived = Thu Apr 21 11:31:25 2011 CEST
- Host = gs-03.uaic.ro
- Level = SYSTEM
- Name = notice
- Priority = synchronous
- Seqcode = UI=000000:NS=0000000004:WM=000005:BH=0000000000:JSS=000003:LM=000000:LRMS=000011:APP=000000:LBS=000000
- Source = LRMS
- Timestamp = Thu Apr 21 11:24:10 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi
- Value = Truncated last 52428800 bytes for file huge
---
<...>

bug #48640: glite-wms-wmproxy to support graceful command

The test is considered successful if the following processes:

root     16366     1  0 13:14 ?        00:00:01 /usr/sbin/httpd -k start -f /etc/glite-wms/glite_wms_wmproxy_httpd.conf
glite    16641 16366  0 13:16 ?        00:00:00 /usr/sbin/httpd -k start -f /etc/glite-wms/glite_wms_wmproxy_httpd.conf
glite    16642 16366  0 13:16 ?        00:00:00 /usr/sbin/httpd -k start -f /etc/glite-wms/glite_wms_wmproxy_httpd.conf
glite    16643 16366  0 13:16 ?        00:00:00 /usr/sbin/httpd -k start -f /etc/glite-wms/glite_wms_wmproxy_httpd.conf
glite    16644 16366  0 13:16 ?        00:00:00 /usr/sbin/httpd -k start -f /etc/glite-wms/glite_wms_wmproxy_httpd.conf
glite    16645 16366  0 13:16 ?        00:00:00 /usr/sbin/httpd -k start -f /etc/glite-wms/glite_wms_wmproxy_httpd.conf
glite    16646 16366  0 13:16 ?        00:00:00 /usr/sbin/httpd -k start -f /etc/glite-wms/glite_wms_wmproxy_httpd.conf
are restarted gracefully after issuing the command:
/etc/init.d/glite-wms-wmproxy graceful

 -- ElisabettaMolinari - 2010-02-24

Revision 202011-06-27 - FabioCapannini

Line: 1 to 1
 

WMS Test Plan

Line: 291 to 291
 

bug #33342: separate retry policies for ISB and OSB

Added:
>
>
Description of the test.
 Of course we're speaking of submission to the lcg-CE, as CREAM uses its own jobwrapper.

ISB:

Line: 316 to 318
 [root@devel11 input]# pwd /var/SandboxDir/-h/https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2f-h4MRDzYkufRu71MKfF1pw/input
Changed:
<
<
after a while, maradona reports:
>
>
after a while, maradona file reports:
 [root@devel11 https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2f-h4MRDzYkufRu71MKfF1pw]# cat Maradona.output LM_log_done_begin Wed Apr 20 22:08:51 CEST 2011: lcg-jobwrapper-hook.sh not readable or not present
Line: 343 to 345
  ======================================================================
Changed:
<
<
after more than twenty minutes Maradona hasn't returned yet and the job is running, meaning that other defaults are in place (the ones previously used for both ISB and OSB)
>
>
Considered successful if: after more than twenty minutes Maradona file hasn't returned yet and the job is running, meaning that other defaults are in place (the ones previously used for both ISB and OSB)
 

bug #36292: Not all attributes of a SA/SE coul be used in a gangmatching

Added:
>
>
Description of the test.
 Fix certified doing a listmatch with the following expression in the jdl:

Requirements = regexp(".in2p3.fr:2119.*",other.GlueCEUniqueID) && anyMatch(other.storage.CloseSEs,target.GlueSEImplementationVersion=="1.9.5-24");

Line: 388 to 393
 

bug #40982: When a collection is aborted the "Abort" event should be logged for the sub-nodes as well /2

Added:
>
>
Description of the test.
 coll_10.jdl is a ten nodes collection, only the first node having non empty ISB.
Line: 455 to 462
 ======================================================================
Changed:
<
<
After some seconds:
>
>
Considered successful if:
 
Changed:
<
<
[mcecchi@cert-19 ~]$ glite-wms-job-status https://devel11.cnaf.infn.it:9000/M... Aborted|wc -l
>
>
[mcecchi@cert-19 ~]$ glite-wms-job-status https://devel11.cnaf.infn.it:9000/MKfaYgbm4wDSKh0iO191rA|grep Aborted|wc -l
 11
Added:
>
>

bug #44599: WMS should consider "MaxTotalJobs"

Description of the test.

The wmp configuration section has a new entry:

WmsRequirements = ((ShortDeadlineJob =?= TRUE) ? RegExp(".sdj$", other.GlueCEUniqueID) : !RegExp(".sdj$", other.GlueCEUniqueID)) && (other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs) && (EnableWmsFeedback =?= TRUE ? RegExp("cream", other.GlueCEImplementationName, "i") : true);

WmsRequirements value is appended in && to the user requirements

UI:

[mcecchi@cert-19 ~]$ cat job.jdl
[
Executable = "/bin/true";
Arguments = "";
StdOutput = "out.log";
StdError = "err.log";
OutputSandbox = {};
InputSandbox = {"a"};
rank = - other.GlueCEStateEstimatedResponseTime ;
//Requirements = !RegExp("cream.*", other.GlueCEUniqueID);
Requirements = true;
RetryCount = 0;
ShallowRetryCount = -1;
]

[mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf job.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel11.cnaf.infn.it:9000/X...

==========================================================================

WMS, wmproxy.log:21 Apr, 00:16:45 -D- PID: 4336 - "wmp2wm::submit": LB Logged jdl/path: [ OutputSandboxPath = "/var/SandboxDir/Xm/https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2fXmA_5f5p51xWR_5f5N2B8MDxGw/output"; StdOutput = "out.log"; ShallowRetryCount = -1; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 0; Executable = "/bin/true"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000000:BH=0000000000:JSS=000000:LM=000000:LRMS=000000:APP=000000:LBS=000000"; Arguments = ""; AllowZippedISB = true; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank = -other.GlueCEStateEstimatedResponseTime; ZippedISB = { "ISBfiles_rmRmJ25HMnGWbGATaFD7cw_0.tar.gz" }; OutputSandbox = { }; InputSandbox = { "gsiftp://devel11.cnaf.infn.it:2811/var..." }; edg_jobid = "https://devel11.cnaf.infn.it:9000/X..."; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi"; StdError = "err.log"; InputSandboxPath = "/var/SandboxDir/Xm/https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2fXmA_5f5p51xWR_5f5N2B8MDxGw/input"; rank = -other.GlueCEStateEstimatedResponseTime; MyProxyServer = "myproxy.cnaf.infn.it"; X509UserProxy = "/var/SandboxDir/Xm/https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2fXmA_5f5p51xWR_5f5N2B8MDxGw/user.proxy"; InputSandboxDestFileName = { "a" }; requirements = ( ( true ) && ( other.GlueCEStateStatus == "Production" ) ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".sdj$",other.GlueCEUniqueID) : !RegExp(".sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://devel11.cnaf.infn.it:2811/var..." ]

Considered successful if requirements becomes:

requirements = ( ( true ) && ( other.GlueCEStateStatus == "Production" ) ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".sdj$",other.GlueCEUniqueID) : !RegExp(".sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); 
 -- ElisabettaMolinari - 2010-02-24

Revision 192011-06-24 - FabioCapannini

Line: 1 to 1
 

WMS Test Plan

Line: 12 to 12
 The WMS basically receives requests of job execution from a client, finds the required appropriate resources, then dispatches and follows them until completion. This is done handling failure in between and whenever possible. Other than single batch-like jobs, compound job types handled by the WMS are Directed Acyclic Graphs (a set of jobs where the input/output/execution of one of more jobs may depend on one or more other jobs), Parametric Jobs (multiple jobs with one parametrized description), and Collections (multiple jobs with a common description). Jobs are described via a flexible, high-level Job Definition Language (JDL).
Changed:
<
<

Deployment scenarios

>
>
Functional Description
 
Changed:
<
<
TBD
>
>
Service Reference Card

Unit tests

N/A

Deployment tests

Repository

The EMI-1 RC4 repository can be found under:

http://emisoft.web.cern.ch/emisoft/dist/EMI/1/RC4/sl5/x86_64

Other repositories:

  • epel.repo
  • lcg-CA.repo
  • sl.repo
  • sl-security.repo

Installation test

First of all, install the yum-protectbase rpm:

yum install yum-protectbase.noarch

Then proceed with the installation of the CA certificates by issuing:

yum install ca-policy-egi-core

Install the WMS metapackage:

yum install emi-wms 

(see log file)

Configure the WMS:

/opt/glite/yaim/bin/yaim -c -s site-info.def -n WMS  

(see log file)

Update test

N/A

 

Functionality tests

Line: 244 to 287
 
  • Resubmission is enabled
  • Enable proxy renewal
Added:
>
>

Regression tests

 
Added:
>
>

bug #33342: separate retry policies for ISB and OSB

 
Added:
>
>
Of course we're speaking of submission to the lcg-CE, as CREAM uses its own jobwrapper.

ISB: https://devel11.cnaf.infn.it:9000/a...

submitted a job and then removed its ISB

[mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf lsisb.jdl
Connecting to the service https://devel11.cnaf.infn.it:7443/g...
 
Added:
>
>
================== glite-wms-job-submit Success ==================
 
Added:
>
>
The job has been successfully submitted to the WMProxy Your job identifier is:
 
Added:
>
>
https://devel11.cnaf.infn.it:9000/-...

======================================================================

server side: [root@devel11 input]# rm -f a [root@devel11 input]# pwd /var/SandboxDir/-h/https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2f-h4MRDzYkufRu71MKfF1pw/input

after a while, maradona reports: [root@devel11 https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2f-h4MRDzYkufRu71MKfF1pw]# cat Maradona.output LM_log_done_begin Wed Apr 20 22:08:51 CEST 2011: lcg-jobwrapper-hook.sh not readable or not present Wed Apr 20 22:08:52 CEST 2011: Error during transfer Wed Apr 20 22:09:53 CEST 2011: Error during transfer Wed Apr 20 22:11:54 CEST 2011: Error during transfer

LM_log_done_end Cannot download a from gsiftp://devel11.cnaf.infn.it:2811/var... Killing log watchdog (pid=21047)... jw exit status = 1

OSB: [mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf lsosb.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

================== glite-wms-job-submit Success ==================

The job has been successfully submitted to the WMProxy Your job identifier is:

https://devel11.cnaf.infn.it:9000/3...

====================================================================== after more than twenty minutes Maradona hasn't returned yet and the job is running, meaning that other defaults are in place (the ones previously used for both ISB and OSB)

bug #36292: Not all attributes of a SA/SE coul be used in a gangmatching

Fix certified doing a listmatch with the following expression in the jdl:

Requirements = regexp(".in2p3.fr:2119.*",other.GlueCEUniqueID) && anyMatch(other.storage.CloseSEs,target.GlueSEImplementationVersion=="1.9.5-24");

which returns:

- cclcgceli02.in2p3.fr:2119/jobmanager-bqs-short
- cclcgceli04.in2p3.fr:2119/jobmanager-bqs-short
- cclcgceli06.in2p3.fr:2119/jobmanager-bqs-short
- cclcgceli09.in2p3.fr:2119/jobmanager-bqs-short
- cclcgceli02.in2p3.fr:2119/jobmanager-bqs-medium
- cclcgceli04.in2p3.fr:2119/jobmanager-bqs-long
- cclcgceli06.in2p3.fr:2119/jobmanager-bqs-medium
- cclcgceli09.in2p3.fr:2119/jobmanager-bqs-medium
- cclcgceli02.in2p3.fr:2119/jobmanager-bqs-long
- cclcgceli06.in2p3.fr:2119/jobmanager-bqs-long
- cclcgceli04.in2p3.fr:2119/jobmanager-bqs-medium
- cclcgceli09.in2p3.fr:2119/jobmanager-bqs-long
 
Added:
>
>
Double checking that the correct "GlueSEImplementationVersion" is picked up:

lcg-infosites --vo dteam closeSE >closeses.txt

gives the following closeSEs:

cclcgceli02.in2p3.fr:2119/jobmanager-bqs-long
ccsrm.in2p3.fr
ccsrm02.in2p3.fr

and ldapsearch -x -H ldap://lcg-bdii.cern.ch:2170 -b 'Mds-vo-name=local,o=Grid' '(GlueSEUniqueId=ccsrm.in2p3.fr)'
returns:

...
GlueSEImplementationVersion: 1.9.5-24
...

bug #40982: When a collection is aborted the "Abort" event should be logged for the sub-nodes as well /2

coll_10.jdl is a ten nodes collection, only the first node having non empty ISB.

[mcecchi@cert-19 ~]$ head -25 coll_10.jdl
[
Type = "collection";
InputSandbox = {"/home/mcecchi/Test.sh"};
RetryCount = 1;
Requirements = ( random(1.0) < 0.5 );
ShallowRetryCount = 2;
nodes = {
[
JobType = "Normal";
Zippedisb=true;
Executable = "Test.sh";
StdOutput = "test.out";
StdError = "test.err";
InputSandbox = {"a"};
OutputSandbox = {};
],
[
JobType = "Normal";
Executable = "Test.sh";
StdOutput = "test.out";
StdError = "test.err";
OutputSandbox = {};
],
[
JobType = "Normal";

We register the collection:

[mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf --register-only coll_10.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully registered to the WMProxy
Your job identifier is:

https://devel11.cnaf.infn.it:9000/M...

==========================================================================

To complete the operation, the following file containing the InputSandbox of the job needs to be transferred:
==========================================================================================================
ISB ZIP file : /tmp/ISBfiles_aoIPOxSR3GFuEcTxqJ6_Mg_0.tar.gz
Destination : gsiftp://devel11.cnaf.infn.it:2811/var...

We do NOT transfer ISB for the first node and start the job.

[mcecchi@cert-19 ~]$ glite-wms-job-submit --start https://devel11.cnaf.infn.it:9000/M...
Connecting to the service https://devel11.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully started to the WMProxy
Your job identifier is:

https://devel11.cnaf.infn.it:9000/M...

==========================================================================

After some seconds:

[mcecchi@cert-19 ~]$ glite-wms-job-status https://devel11.cnaf.infn.it:9000/M... Aborted|wc -l
11
 -- ElisabettaMolinari - 2010-02-24

Revision 182011-04-29 - AlessioGianelle

Line: 1 to 1
 

WMS Test Plan

Line: 225 to 225
 In the jdl you can specify two attributes prologue and epilogue which are scripts that are execute respectively before and after the user's job. Implemented
Changed:
<
<

Stress tests

>
>

Performance tests

 

Collection of 1000 nodes

Changed:
<
<

Two days test

>
>
Submit a collection of 1000 nodes.

Stress test

This could be an example of stress test

 
Deleted:
<
<
Description
 
  • 2880 collections each of 20 jobs
  • One collection every 60 seconds
  • Four users

Revision 172011-04-27 - AlessioGianelle

Line: 1 to 1
 

WMS Test Plan

Line: 69 to 69
 ]
Changed:
<
<
  • When job finished retrieve the output files
>
>
  • When dag finishes retrieve the output files
 
  • Check the final status of the dag (all nodes and parent should be "Cleared")

More different jdls can added in the future.

Line: 82 to 82
  Multiple jobs with a common description. There are two ways to submit collection: you can create a single jdl with all the jdls of nodes or you can submit all the jdls stored in a directory (bulk submission)
Changed:
<
<
>
>
  • Submit a jdl like this one:
[
nodes = {
   [
   file="jdl/arg.jdl";
   ],
   [
  executable="/bin/env";
  ShallowRetryCount = 0;
  RetryCount = 0;
  Stdoutput = "file.out" ;
  StdError =  "file.err" ;
  OutputSandbox ={ "file.out" ,"file.err"} ;
  FuzzyRank = true;
   ],
   [
   NodeName="nodeA";
   executable="/bin/ls" ;
  Stdoutput = "file.out" ;
  OutputSandbox ={ "file.out"} ;
   ]
};
Type = "Collection" ;
requirements =  other.GlueCEStateStatus == "Production" ;
rank = -other.GlueCEStateEstimatedResponseTime ;
]

  • When collection finishes retrieve the output files
  • Check the final status of the collectionall nodes and parent should be "Cleared")

  • To test bulk submission use option "--collection" of glite-wms-job-submit command.
  • When collection finishes retrieve the output files
  • Check the final status of the collection (all nodes and parent should be "Cleared")

More different jdls can added in the future.

 
Parallel Job
Added:
>
>
Jobs that can be running in one or more cpus in parallel.
 
Changed:
<
<

WMS Job shallow and deep re-submission

  • For the shallow resubmission submit the following jdl file:
>
>
  • Submit a jdl like this one:
 [
Changed:
<
<
requirements = (other.GlueCEStateStatus == "Production"); Rank = -2 * other.GlueCEStateWaitingJobs; Executable = "/bin/ls"; prologue = "/bin/false"; shallowretrycount = 3; RetryCount = 0; usertags = [ exe = "shallowresub" ]; ]
    • The job should Aborted with message: hit job shallow retry count (3)
    • Check the logging-info (-v 2) of the job, you should find 3 events from the WorkloadManager like this one:
      Event: Resubmission
      - Arrived                    =    Fri Jun 11 13:11:22 2010 CEST
      - Host                       =    devel18.cnaf.infn.it
      - Reason                     =    token still exists
      - Result                     =    SHALLOW
      - Source                     =    WorkloadManager
      - Src instance               =    13035
      - Tag                        =    /var/glite/SandboxDir/LW/https_3a_2f_2fdevel07.cnaf.infn.it_3a9000_2fLWVHkUN2cBOiEcWR7Rc6ew/token.txt
      - Timestamp                  =    Fri Jun 11 13:11:22 2010 CEST
>
>
Executable = "cpi"; CpuNumber = 2; Stdoutput = "cpi.out" ; StdError = "cpi.err" ; OutputSandbox = { "cpi.out" ,"cpi.err"} ; InputSandbox = { "exe/cpi" }; FuzzyRank = true; usertags = [ exe = "cpi" ]; ]
 
Changed:
<
<
  • For the deep resubmission submit the following jdl:
    [
      requirements =  (other.GlueCEStateStatus == "Production");
      Rank = -2 * other.GlueCEStateWaitingJobs;
      Executable = "/bin/ls";
      prologue = "/bin/false";
      shallowretrycount = -1; 
      RetryCount = 3;
      usertags = [ exe = "deepresub" ];
    ]
    • The job should Aborted with message: hit job retry count (3)
    • Check the logging-info (-v 2) of the job, you should find 3 events from the WorkloadManager like this one:
          Event: Resubmission
      - Arrived                    =    Fri Jun 11 14:15:19 2010 CEST
      - Host                       =    devel18.cnaf.infn.it
      - Level                      =    SYSTEM
      - Priority                   =    synchronous
      - Reason                     =    shallow resubmission is disabled
      - Result                     =    WILLRESUB
      - Seqcode                    =    UI=000000:NS=0000000005:WM=000016:BH=0000000000:JSS=000009:LM=000018:LRMS=000000:APP=000000:LBS=000000
      - Source                     =    WorkloadManager
>
>
  • When job finishes retrieve the output files
  • Check the final status of the job
 
Added:
>
>

WMS Job shallow and deep re-submission

 
Added:
>
>
There two type of resubmission; the first is defined deep occurs when the user's job has stardted running on the WN and then the job itself or the WMS JobWrapper has failed. The second one is called shallow and occurs when the WMS JobWrapper has failed before starting the actual user's job. Implemented
 

WMS Job List-match Testing

Without data
Added:
>
>
Test job-list-command and its option Implemented
 
With data
Changed:
<
<
  • Set these variables:
      export LCG_GFAL_INFOSYS=<BDII set in the wms conf file> (e.g. cert-bdii-04.cnaf.infn.it:2170)
      export LFC_HOST=<lfc host name> (e.g. lfcserver.cnaf.infn.it)
      export LFC_HOME=<lfc home directory> (e.g. lfcserver.cnaf.infn.it:/grid/infngrid)
  • Register a file to a SE using lcg-cr command:
      lcg-cr --vo <VO name> -d <SE host> -l lfn:<logical file name> file:<local file path>
  • Using the lcg-rep you can create some replicas:
      lcg-rep  --vo <VO name> -d <SE host> lfn:<logical file name>
  • lcg-lr is useful to see the list of replicas:
      lcg-lr --vo <VO name>  lfn:<logical file name>
  • Submit a list-match command with a jdl likes this one (as InputData put the lfn(s) registered before):
>
>
  • You need to register a file on an SE, then submit a jdl like this one (as InputData put the lfn(s) registered before):
 ########################################### # JDL with Data Requirements # ###########################################
Line: 172 to 178
 ] }; DataAccessProtocol = "gsiftp";
Changed:
<
<
  • You should obtain an output like this one:
    [ale@cream-15 DataReq]$ glite-wms-job-list-match -a --rank --config ~/UI/etc/wmp_wms007.conf data-req.jdl
    Connecting to the service https://wms007.cnaf.infn.it:7443/glite_wms_wmproxy_server
    
    ==========================================================================
    
             COMPUTING ELEMENT IDs LIST
     The following CE(s) matching your job requirements have been found:
    
      *CEId*                                                   *Rank*
    
     - cert-15.pd.infn.it:8443/cream-lsf-cert                           0
     - gridit-ce-001.cnaf.infn.it:2119/jobmanager-lcgpbs-cert           0
     - prod-ce-01.pd.infn.it:8443/cream-lsf-cert                        0
     - prod-ce-02.pd.infn.it:2119/jobmanager-lcglsf-cert                0
     - t2-ce-01.to.infn.it:8443/cream-pbs-cert                          0
     - t2-ce-01.to.infn.it:8443/cream-pbs-short                         0
     - t2-ce-02.to.infn.it:2119/jobmanager-lcgpbs-cert                  0
     - t2-ce-02.to.infn.it:2119/jobmanager-lcgpbs-short                 0
     - test7200a.cnaf.infn.it:2119/jobmanager-lcgpbs-cert               0
     - test7200a.cnaf.infn.it:2119/jobmanager-lcgpbs-parallel           0
     - devce.cnaf.infn.it:8443/cream-pbs-cert                           -700634
    
    ==========================================================================
  • Finally you can check if these CEs are really closed to the SEs where you replicate your file(s)
    • First list the SEs:
      [ale@cream-15 DataReq]$ lcg-lr --vo dteam  lfn:/grid/infngrid/cesini/PI_1M.txt
      
      srm://aliserv6.ct.infn.it/dpm/ct.infn.it/home/infngrid/generated/2009-11-24/file0b6a613c-e1f1-4e53-a328-2152e44a8576
      srm://gridit-se-01.cnaf.infn.it/dpm/cnaf.infn.it/home/infngrid/generated/2009-11-24/fileb9f3b972-806c-48a6-92fc-a9677901403a
      srm://prod-se-01.pd.infn.it/infngrid/generated/2009-11-24/file10b5fe11-ad95-41e6-a3a3-14bc5cb9694b
      srm://t2-se-00.to.infn.it/dpm/to.infn.it/home/infngrid/generated/2009-11-24/file1bb28727-1356-4fb1-bb73-5362f93e766f
    • then for each SEs in the list use this command to find the "Close" CEs:
      ldapsearch -x -h <BDII set in the WMS conf file> -p 2170 -b mds-vo-name=local,o=grid "(&(objectclass=GlueCESEBindGroup)(GlueCESEBindGroupSEUniqueID=<SE HOST>))" | grep ^GlueCESEBindGroupCEUniqueID
      
      GlueCESEBindGroupCEUniqueID: devce.cnaf.infn.it:8443/cream-pbs-cert
      GlueCESEBindGroupCEUniqueID: gridit-ce-001.cnaf.infn.it:2119/jobmanager-lcgpbs
      GlueCESEBindGroupCEUniqueID: cert-08.cnaf.infn.it:8443/cream-lsf-pps
      GlueCESEBindGroupCEUniqueID: cert-05.cnaf.infn.it:8443/cream-lsf-pps
      GlueCESEBindGroupCEUniqueID: test7200a.cnaf.infn.it:2119/jobmanager-lcgpbs-par
      GlueCESEBindGroupCEUniqueID: gridit-ce-001.cnaf.infn.it:2119/jobmanager-lcgpbs
      GlueCESEBindGroupCEUniqueID: cert-07.cnaf.infn.it:8443/cream-lsf-pps
      GlueCESEBindGroupCEUniqueID: cert-09.cnaf.infn.it:8443/cream-lsf-pps
      GlueCESEBindGroupCEUniqueID: gridit-ce-001.cnaf.infn.it:2119/jobmanager-lcgpbs
      GlueCESEBindGroupCEUniqueID: cert-06.cnaf.infn.it:8443/cream-lsf-pps
      GlueCESEBindGroupCEUniqueID: cert-04.cnaf.infn.it:8443/cream-lsf-pps
      GlueCESEBindGroupCEUniqueID: test7200a.cnaf.infn.it:2119/jobmanager-lcgpbs-cer
      GlueCESEBindGroupCEUniqueID: cert-13.cnaf.infn.it:8443/cream-lsf-pps
    • You can find more CEs because you should also filter using the "vo" attribute.
>
>
  • Then try a list-match, the listed CEs should be the ones "close" to the used SE
 

WMS Job Cancel Testing

Changed:
<
<
Normal job
>
>
Test the cancellation of these type of jobs (final status should be cleared):
 
Changed:
<
<
DAG job
>
>
Normal job
 
Changed:
<
<
Collection
>
>
Submit and cancel a normal job Implementd
 
Changed:
<
<
Node of a collection
>
>
DAG job
 
Changed:
<
<

Delegation Testing

>
>
Submit a dag job and then cancel it (the parent)
 
Changed:
<
<

Logging-info Testing

>
>
Collection
 
Changed:
<
<

Job Status Testing

>
>
Submit a collection job and then cancel it (the parent)
 
Added:
>
>
Node of a collection
 
Added:
>
>
Submit a collection job and then some of its nodes
 

Others

Changed:
<
<
BrokerInfo
  • Use a jdl like this one to verify its creation:
    ###########################################
    #    JDL to test Brokerinfo Creation      #
    ###########################################
>
>
Delegation Testing
 
Changed:
<
<
Executable = "/bin/ls"; Arguments = "-la"; StdOutput = "std.out"; StdError = "std.err"; FuzzyRank = true; InputSandbox = {"calc-pi.sh", "fileA", "fileB"}; OutputSandbox = {"std.out", "std.err",".BrokerInfo"};
>
>
Test the delegation command and its options Implementd
 
Changed:
<
<
Requirements = (other.GlueCEInfoHostName = "spacin-ce1.dma.unina.it") && regexp("8443/cream", other.GlueCEUniqueID) ;
>
>
Job-info Testing
 
Changed:
<
<
DataRequirements = { [ DataCatalogType = "DLI"; DataCatalog = "http://lfcserver.cnaf.infn.it:8085"; InputData = {"lfn:/grid/infngrid/cesini/PI_1M.txt","lfn:/grid/infngrid/cesini/e-2M.txt"}; ] };

DataAccessProtocol = "gsiftp";

  • After the retrieving of the output files you should see these files:
    [ale@cream-15 DataReq]$ ls -la /tmp/ale_5XbbEhIAtjO5uC5wRyCmyA
    total 1264
    drwxr-xr-x   2 ale  ale     4096 Mar  4 10:31 .
    drwxrwxrwt  17 root root 1269760 Mar  4 10:31 ..
    -rw-rw-r--   1 ale  ale     7804 Mar  4 10:31 .BrokerInfo
    -rw-rw-r--   1 ale  ale        0 Mar  4 10:31 std.err
    -rw-rw-r--   1 ale  ale      624 Mar  4 10:31 std.out
And also:
[ale@cream-15 DataReq]$ cat /tmp/ale_5XbbEhIAtjO5uC5wRyCmyA/std.out
total 32
drwxr-xr-x  2 dteam038 dteam 4096 Mar  4 10:27 .
drwx------  5 dteam038 dteam 4096 Mar  4 10:27 ..
-rw-r--r--  1 dteam038 dteam 7804 Mar  4 10:27 .BrokerInfo
-rw-r--r--  1 dteam038 dteam  216 Mar  4 10:27 calc-pi.sh
-rw-r--r--  1 dteam038 dteam   17 Mar  4 10:27 fileA
-rw-r--r--  1 dteam038 dteam   17 Mar  4 10:27 fileB
-rw-r--r--  1 dteam038 dteam  166 Mar  4 10:27 https_3a_2f_2fdevel17.cnaf.infn.it_3a9000_2f5XbbEhIAtjO5uC5wRyCmyA.output
-rw-r--r--  1 dteam038 dteam    0 Mar  4 10:27 std.err
-rw-r--r--  1 dteam038 dteam    0 Mar  4 10:27 std.out
-rw-------  1 dteam038 dteam    0 Mar  4 10:27 tmp.LesGC30313

  • Check the goodness of the .BrokerInfo file generated:
    • glite-brokerinfo getCE -f < .Brokerinfo file path >
      • It should return the matched CE
    • glite-brokerinfo getDataAccessProtocol -f < .Brokerinfo file path >
      • It should return the DataAccessProtocol parameter as specified in the jdl
    • glite-brokerinfo getInputData -f < .Brokerinfo file path >
      • It should return the InputData files specified in the jdl
    • glite-brokerinfo getSEs -f < .Brokerinfo file path >
      • It should return all the SEs (seen by the BDII of the WMS) that store all the files specified in the InputData attribute (You can chek it using the command lcg-lr --vo <VO name> <logical file name>)
    • glite-brokerinfo getCloseSEs -f < .Brokerinfo file path >
      • It should return only the CloseSe to the matched CE (to see them use the command: ldapsearch -x -h <specify the BDII host name> -p 2170 -b mds-vo-name=local,o=grid "(&(objectclass=GlueCESEBindGroup)(GlueCESEBindGroupCEUniqueID=<specify the CE name>))" | grep GlueCESEBindGroupSEUniqueID)
    • glite-brokerinfo getLFN2SFN <LFN of the files in InputData > -f < .Brokerinfo file path >
      • It should return the output of the command lcg-lr --vo <VO name> <logical file name>
    • glite-brokerinfo getVirtualOrganization -f < .Brokerinfo file path >
      • It should return your VO
>
>
Test the job-info command and its options Implementd
 
Added:
>
>
Logging-info Testing
 
Changed:
<
<
Prologue and Epilogue jobs
  • Submit a jdl like this one:
    ###########################################
    #     JDL with Prologue and Epilogue      #
    ###########################################
    
    Executable = "exe.sh";
    Arguments = "1000";
    StdOutput = "std.out";
    StdError = "std.err";
    Prologue = "prologue.sh";
    Epilogue = "epilogue.sh";
    FuzzyRank = true;
>
>
Test the logging-info command and its options Implemented
 
Changed:
<
<
InputSandbox = {"exe/exe.sh", "data/pippo", "exe/epilogue.sh","exe/prologue.sh"}; OutputSandbox = {"std.out", "std.err", "prologue.out", "epilogue.out"};
>
>
Job Status Testing
 
Changed:
<
<
#Requirements = regexp("8443/cream", other.GlueCEUniqueID); #Requirements = regexp("2119/jobmanager", other.GlueCEUniqueID);
>
>
Test the job-status commend and its options Implemented
 
Changed:
<
<
RetryCount = 1; ShallowRetryCount = 2;
  • Where:
    [ale@cream-15 UI]$ cat exe/exe.sh
    #!/bin/sh
    
    date
    echo "Hello world!"
    echo "My prologue said:" 
    cat prologue.out
    echo "My argument is $1"
    
    [ale@cream-15 UI]$ cat exe/prologue.sh 
    #!/bin/sh
    
    date > prologue.out
    echo "I'm the prologue" >> prologue.out
    echo "##########################" >> prologue.out
    echo "This is the output of an ls command" >> prologue.out
    ls >> prologue.out
    echo "##########################" >> prologue.out
    
    [ale@cream-15 UI]$ cat exe/epilogue.sh 
    #!/bin/sh
    
    echo " This is the output of the job: `cat std.out`" > epilogue.out
    echo "#########################" >> epilogue.out
    echo "We finish at `date`" >> epilogue.out
    echo "#########################" >> epilogue.out
    echo "All the jokes are done!" >> epilogue.out

  • Use the requirements expression to choose an LCG or a CREAM ce
  • When the job is Done retrieve the output sandbox, it should contains these files:
    • epilogue.out
    • prologue.out
    • std.err and std.out
  • This should be the contenent of the files:
    [ale@cream-15 UI]$ cat prologue.out
    Fri Jun 11 10:24:15 BST 2010
    I'm the prologue
    ##########################
    This is the output of an ls command
    epilogue.sh
    exe.sh
    pippo
    prologue.out
    prologue.sh
    ##########################
    
    [ale@cream-15 UI]$ cat std.out
    Fri Jun 11 10:24:15 BST 2010
    Hello world!
    My prologue said:
    Fri Jun 11 10:24:15 BST 2010
    I'm the prologue
    ##########################
    This is the output of an ls command
    epilogue.sh
    exe.sh
    pippo
    prologue.out
    prologue.sh
    ##########################
    My argument is 1000
    
    [ale@cream-15 UI]$ cat epilogue.out
     This is the output of the job: Fri Jun 11 10:24:15 BST 2010
    Hello world!
    My prologue said:
    Fri Jun 11 10:24:15 BST 2010
    I'm the prologue
    ##########################
    This is the output of an ls command
    epilogue.sh
    exe.sh
    pippo
    prologue.out
    prologue.sh
    ##########################
    My argument is 1000
    #########################
    We finish at Fri Jun 11 10:24:15 BST 2010
    #########################
    All the jokes are done!
>
>
Prologue and Epilogue jobs
 
Added:
>
>
In the jdl you can specify two attributes prologue and epilogue which are scripts that are execute respectively before and after the user's job. Implemented
 

Stress tests

Revision 162011-04-27 - AlessioGianelle

Line: 1 to 1
 

WMS Test Plan

Service Description

Changed:
<
<
TBD
>
>
The Workload Management System (gLite WMS) is a software service of the gLite suite which is responsible for distributing and managing tasks across computing and storage resources available on a Grid. WMS assigns user jobs to CEs and SEs belonging to a Grid environment in a convenient fashion, so that:

  • jobs are always executed on resources that match the job requirements
  • grid-wide load balance is maintained, i.e. jobs are evenly and efficiently distributed across the entire Grid.

The WMS basically receives requests of job execution from a client, finds the required appropriate resources, then dispatches and follows them until completion. This is done handling failure in between and whenever possible. Other than single batch-like jobs, compound job types handled by the WMS are Directed Acyclic Graphs (a set of jobs where the input/output/execution of one of more jobs may depend on one or more other jobs), Parametric Jobs (multiple jobs with one parametrized description), and Collections (multiple jobs with a common description). Jobs are described via a flexible, high-level Job Definition Language (JDL).

 

Deployment scenarios

Line: 15 to 21
 

Features/Scenarios to be tested

YAIM-WMS Configuration Testing

Added:
>
>
  • Installation and configuration starting from a cleaning machine (i.e. only OS)
  • Update and configuration from a previous version
 

WMS Job Submission/GetOutput Testing

Changed:
<
<
Registers the job into the wmproxy, delivers it to the WM that submits it to the grid. Test job submission with the following type of jobs/jdl:
Normal Job
  • to JC
    [
    Type = "Job";
    JobType = "normal";
    InputSandbox = { "file://$HOME/test.sh"};
    VirtualOrganisation = "dteam";
    Executable="test.sh";
    Arguments="Hello ";
    Requirements = ( !RegExp("/cream-",other.GlueCEUniqueID));
    Rank = 0;
    fuzzyrank = true;
    StdOutput="message.txt";
    StdError="err.log";
    OutputSandbox={"message.txt","err.log",".BrokerInfo"};
    RetryCount = 0;
    usertags = [ jdl = "normal job" ];
    ShallowRetryCount = 3;
    ]
  • to ICE
    [
    Type = "Job";
    JobType = "normal";
    InputSandbox = { "file:///home/emolinari/test.sh"};
    VirtualOrganisation = "dteam";
    Executable="test.sh";
    Arguments="Hello ";
    Requirements = ( RegExp("/cream-",other.GlueCEUniqueID));
    Rank = 0;
    fuzzyrank = true;
    StdOutput="message.txt";
    StdError="err.log";
    OutputSandbox={"message.txt","err.log",".BrokerInfo"};
    usertags = [ jdl = "normal job to ICE" ];
    RetryCount = 0;
    ShallowRetryCount = 3;
    ]

Perusal job
Job perusal is the ability to view output from a job while it is running. In order to use this feature the JDL attributes PerusalFileEnable and PerusalTimeInterval must be declared when the job is submitted for example:
PerusalFileEnable = true;
PerusalTimeInterval = 120;
This makes the WN upload, at regular time intervals (PerusalTimeInterval), a copy of the output files specified using the glite-wms-job-perusal --set command to the WMS machine.
  • Following an example on how to test a perusal job:
    • Given the jdl file:
      [
      Executable = "perusal.sh" ;
      InputSandbox = {"file://$HOME/perusal.sh"};
      Stdoutput = "perusal.out" ;
      StdError =  "perusal.err" ;
      OutputSandbox ={ "perusal.out" ,"perusal.err"} ;
      usertags = [ jdl = "perusal" ];
      PerusalFileEnable = true;
      ]
      where perusal.sh is
      #!/bin/bash
>
>
Submit a job to the WMS service and when finished retrieve the output. Test job submission with the following type of jobs:
 
Changed:
<
<
echo "Hi all, this is a perusal test program..."

for (( i=0 ; $i < 50 ; i = $(($i + 1)) )); do

>
>
Normal Job
  • Test submission of normal jobs with different options and situation Implemented
 
Changed:
<
<
echo "Step number $i" date sleep 10
>
>
  • Test the complete cycle with the two types of CEs: lcg and Cream Implemented
 
Changed:
<
<
done
>
>
More different jdls can added in the future.
 
Changed:
<
<
echo "" echo "Bye"
    • Submit the job with
       glite-wms-job-submit --config glite_wms.conf -a perusal.jdl
    • Given the jobid of the previously submitted set the files that are to be perused with
      glite-wms-job-perusal --set -f perusal.out https://devel17.cnaf.infn.it:9000/HBPb0WwhMfhKbjdUkrasWg
    • When the job starts running on the WN get the files as here:
      glite-wms-job-perusal --get -f perusal.out https://devel17.cnaf.infn.it:9000/HBPb0WwhMfhKbjdUkrasWg
>
>
Perusal job
 
Added:
>
>
Job perusal is the ability to view output from a job while it is running. Implemented
 
DAG job
Changed:
<
<
  • DAG job jdls samples:
    [
    Type = "dag";
    VirtualOrganisation = "dteam";
    Max_nodes_running = 10;
    InputSandbox = "test.sh";
    FuzzyRank = true;
    Nodes = [
    nodeA = [
    file= "test_dag.jdl";
    ];
    nodeB = [
    file= "test_dag.jdl";
    ];
    nodeC = [
    file= "test_dag.jdl";
    ];
    nodeD = [
    file= "test_dag.jdl";
    ];
    nodeE = [
    file= "test_dag.jdl";
    ];
    nodeF= [
    file= "test_dag.jdl";
    ];
    ];
    Dependencies = {
    {{nodeA, nodeB}, nodeC},{nodeD,nodeE.nodeF}
    }
    ]
    [
>
>
Directed Acyclic Graphs (a set of jobs where the input/output/execution of one of more jobs may depend on one or more other jobs).

  • Submit a jdl like this one:
[
  type = "dag"; DefaultNodeShallowRetryCount = 3; nodes = [ nodeA = [ node_type = "edg-jdl";
Changed:
<
<
file ="myjob.jdl" ;
>
>
file ="jdl/arg.jdl" ;
  ]; nodeB = [ node_type = "edg-jdl";
Changed:
<
<
file ="myjob.jdl" ;
>
>
file ="jdl/arg.jdl" ;
  ]; nodeC = [ node_type = "edg-jdl";
Changed:
<
<
file ="myjob.jdl" ;
>
>
file ="jdl/arg.jdl" ;
  ]; dependencies = { { nodeA, nodeB }, { nodeA, nodeC } } ];
Changed:
<
<
]
>
>
]

  • When job finished retrieve the output files
  • Check the final status of the dag (all nodes and parent should be "Cleared")

More different jdls can added in the future.

 
Parametric Job
Changed:
<
<
Submit it to ICE and JC. Following a sample jdl:
// Test parametric job.
// The standar Output and Error are parametrizied such as the
// enviroment variable: MYPATH_PARAM_.
 [
  JobType = "parametric";
  Executable = "/usr/bin/env";
  Environment = {"MYPATH_PARAM_=$PATH:/bin:/usr/bin:$HOME"};
  StdOutput = "echo_PARAM_.out";
  StdError = "echo_PARAM_.err";
  OutputSandbox = {"echo_PARAM_.out","echo_PARAM_.err"};
  Requirements = ( RegExp("/cream-",other.GlueCEUniqueID));
  Parameters =  5;
        usertags = [ jdl = "parametric cream" ];
 ]
>
>
Multiple jobs with one parametrized description. Implemented
 
Collection Job
Added:
>
>
Multiple jobs with a common description. There are two ways to submit collection: you can create a single jdl with all the jdls of nodes or you can submit all the jdls stored in a directory (bulk submission)

 
Parallel Job

Revision 152011-02-04 - MassimoSgaravatto

Line: 1 to 1
 

WMS Test Plan

Line: 160 to 160
 
Collection Job
Added:
>
>
Parallel Job
 

WMS Job shallow and deep re-submission

  • For the shallow resubmission submit the following jdl file:
    [

Revision 142011-02-03 - MassimoSgaravatto

Line: 1 to 1
 

WMS Test Plan

Changed:
<
<

YAIM-WMS Configuration Testing

>
>

Service Description

 
Changed:
<
<

WMS Job Submission/GetOutput Testing

>
>
TBD

Deployment scenarios

TBD

Functionality tests

Features/Scenarios to be tested

YAIM-WMS Configuration Testing

WMS Job Submission/GetOutput Testing

 Registers the job into the wmproxy, delivers it to the WM that submits it to the grid. Test job submission with the following type of jobs/jdl:
Changed:
<
<

Normal Job

>
>
Normal Job
 
  • to JC
    [
    Type = "Job";
    JobType = "normal";
Line: 43 to 55
 ShallowRetryCount = 3; ]
Changed:
<
<

Perusal job

>
>
Perusal job
 Job perusal is the ability to view output from a job while it is running. In order to use this feature the JDL attributes PerusalFileEnable and PerusalTimeInterval must be declared when the job is submitted for example:
PerusalFileEnable = true;
PerusalTimeInterval = 120;
This makes the WN upload, at regular time intervals (PerusalTimeInterval), a copy of the output files specified using the glite-wms-job-perusal --set command to the WMS machine.
Line: 75 to 87
 
    • When the job starts running on the WN get the files as here:
      glite-wms-job-perusal --get -f perusal.out https://devel17.cnaf.infn.it:9000/HBPb0WwhMfhKbjdUkrasWg
Changed:
<
<

DAG job

>
>
DAG job
 
  • DAG job jdls samples:
    [
    Type = "dag";
Line: 129 to 141
  ]; ]
Changed:
<
<

Parametric Job

>
>
Parametric Job
 Submit it to ICE and JC. Following a sample jdl:
// Test parametric job.
// The standar Output and Error are parametrizied such as the
Line: 146 to 158
  usertags = [ jdl = "parametric cream" ]; ]
Changed:
<
<

Collection Job

>
>
Collection Job
 
Changed:
<
<

WMS Job shallow and deep re-submission

>
>

WMS Job shallow and deep re-submission

 
  • For the shallow resubmission submit the following jdl file:
    [
      requirements =  (other.GlueCEStateStatus == "Production");
Line: 193 to 205
 - Seqcode = UI=000000:NS=0000000005:WM=000016:BH=0000000000:JSS=000009:LM=000018:LRMS=000000:APP=000000:LBS=000000 - Source = WorkloadManager
Changed:
<
<

WMS Job List-match Testing

>
>

WMS Job List-match Testing

Without data
 
Changed:
<
<

With data

>
>
With data
 
  • Set these variables:
      export LCG_GFAL_INFOSYS=<BDII set in the wms conf file> (e.g. cert-bdii-04.cnaf.infn.it:2170)
Line: 279 to 293
 GlueCESEBindGroupCEUniqueID: cert-13.cnaf.infn.it:8443/cream-lsf-pps
    • You can find more CEs because you should also filter using the "vo" attribute.
Added:
>
>

WMS Job Cancel Testing

Normal job

DAG job

Collection
 
Changed:
<
<

Others

>
>
Node of a collection
 
Changed:
<
<

BrokerInfo

>
>

Delegation Testing

Logging-info Testing

Job Status Testing

Others

BrokerInfo
 
  • Use a jdl like this one to verify its creation:
    ###########################################
    #    JDL to test Brokerinfo Creation      #
Line: 348 to 379
 
      • It should return your VO
Changed:
<
<

Prologue and Epilogue jobs

>
>
Prologue and Epilogue jobs
 
  • Submit a jdl like this one:
    ###########################################
    #     JDL with Prologue and Epilogue      #
Line: 455 to 486
 
Added:
>
>

Stress tests

Collection of 1000 nodes

Two days test

Description

  • 2880 collections each of 20 jobs
  • One collection every 60 seconds
  • Four users
  • Use LCG-CEs and CREAM-CEs (with different batch systems)
  • Use automatic-delegation
  • The job is a "sleep random(672)"
  • Resubmission is enabled
  • Enable proxy renewal
 

Revision 132010-10-20 - AlessioGianelle

Line: 1 to 1
 

WMS Test Plan

Revision 122010-07-30 - AlessioGianelle

Line: 1 to 1
 

WMS Test Plan

Line: 332 to 332
 

  • Check the goodness of the .BrokerInfo file generated:
Changed:
<
<
    • glite-brokerinfo getCE -f glite-brokerinfo getCE -f < .Brokerinfo file path >
>
>
    • glite-brokerinfo getCE -f < .Brokerinfo file path >
 
      • It should return the matched CE
    • glite-brokerinfo getDataAccessProtocol -f < .Brokerinfo file path >
      • It should return the DataAccessProtocol parameter as specified in the jdl

Revision 112010-06-14 - AlessioGianelle

Line: 1 to 1
 

WMS Test Plan

Line: 76 to 76
 

DAG job

Changed:
<
<
to JC and IC as above specifying Requirements = ( RegExp("/cream-",other.GlueCEUniqueID)); or Requirements = ( RegExp("/cream-",other.GlueCEUniqueID)); respectively in the job jdl
>
>
 
  • DAG job jdls samples:
    [
    Type = "dag";
    VirtualOrganisation = "dteam";

Revision 102010-06-11 - AlessioGianelle

Line: 1 to 1
 

WMS Test Plan

Line: 76 to 76
 

DAG job

Changed:
<
<
to JC and IC as above specifying Requirements = ( RegExp("/cream-",other.GlueCEUniqueID)); or Requirements = ( RegExp("/cream-",other.GlueCEUniqueID)); respectively in the job jdl
>
>
to JC and IC as above specifying Requirements = ( RegExp("/cream-",other.GlueCEUniqueID)); or Requirements = ( RegExp("/cream-",other.GlueCEUniqueID)); respectively in the job jdl
 
  • DAG job jdls samples:
    [
    Type = "dag";
    VirtualOrganisation = "dteam";
Line: 171 to 171
 - Tag = /var/glite/SandboxDir/LW/https_3a_2f_2fdevel07.cnaf.infn.it_3a9000_2fLWVHkUN2cBOiEcWR7Rc6ew/token.txt - Timestamp = Fri Jun 11 13:11:22 2010 CEST
Changed:
<
<
  • For the deep resubmission add the following attributes:
     Prologue = "/bin/false"; RetryCount = 3;
>
>
  • For the deep resubmission submit the following jdl:
    [
      requirements =  (other.GlueCEStateStatus == "Production");
      Rank = -2 * other.GlueCEStateWaitingJobs;
      Executable = "/bin/ls";
      prologue = "/bin/false";
      shallowretrycount = -1; 
      RetryCount = 3;
      usertags = [ exe = "deepresub" ];
    ]
    • The job should Aborted with message: hit job retry count (3)
    • Check the logging-info (-v 2) of the job, you should find 3 events from the WorkloadManager like this one:
          Event: Resubmission
      - Arrived                    =    Fri Jun 11 14:15:19 2010 CEST
      - Host                       =    devel18.cnaf.infn.it
      - Level                      =    SYSTEM
      - Priority                   =    synchronous
      - Reason                     =    shallow resubmission is disabled
      - Result                     =    WILLRESUB
      - Seqcode                    =    UI=000000:NS=0000000005:WM=000016:BH=0000000000:JSS=000009:LM=000018:LRMS=000000:APP=000000:LBS=000000
      - Source                     =    WorkloadManager
 

WMS Job List-match Testing

Revision 92010-06-11 - AlessioGianelle

Line: 1 to 1
 

WMS Test Plan

Line: 149 to 149
 

Collection Job

WMS Job shallow and deep re-submission

Changed:
<
<
  • For the shallow resubmission add the following attributes to your favourite jdl file:
    Prologue = "/bin/false"; ShallowRetryCount = 3;
>
>
  • For the shallow resubmission submit the following jdl file:
    [
      requirements =  (other.GlueCEStateStatus == "Production");
      Rank = -2 * other.GlueCEStateWaitingJobs;
      Executable = "/bin/ls";
      prologue = "/bin/false";
      shallowretrycount = 3;
      RetryCount = 0;
      usertags = [ exe = "shallowresub" ];
    ]
    • The job should Aborted with message: hit job shallow retry count (3)
    • Check the logging-info (-v 2) of the job, you should find 3 events from the WorkloadManager like this one:
      Event: Resubmission
      - Arrived                    =    Fri Jun 11 13:11:22 2010 CEST
      - Host                       =    devel18.cnaf.infn.it
      - Reason                     =    token still exists
      - Result                     =    SHALLOW
      - Source                     =    WorkloadManager
      - Src instance               =    13035
      - Tag                        =    /var/glite/SandboxDir/LW/https_3a_2f_2fdevel07.cnaf.infn.it_3a9000_2fLWVHkUN2cBOiEcWR7Rc6ew/token.txt
      - Timestamp                  =    Fri Jun 11 13:11:22 2010 CEST
 
  • For the deep resubmission add the following attributes:
     Prologue = "/bin/false"; RetryCount = 3;

WMS Job List-match Testing

Revision 82010-06-11 - AlessioGianelle

Line: 1 to 1
 

WMS Test Plan

Line: 307 to 307
 
      • It should return your VO
Added:
>
>

Prologue and Epilogue jobs

  • Submit a jdl like this one:
    ###########################################
    #     JDL with Prologue and Epilogue      #
    ###########################################
    
    Executable = "exe.sh";
    Arguments = "1000";
    StdOutput = "std.out";
    StdError = "std.err";
    Prologue = "prologue.sh";
    Epilogue = "epilogue.sh";
    FuzzyRank = true;
    
    InputSandbox = {"exe/exe.sh", "data/pippo", "exe/epilogue.sh","exe/prologue.sh"};
    OutputSandbox = {"std.out", "std.err", "prologue.out", "epilogue.out"};
    
    #Requirements = regexp("8443/cream", other.GlueCEUniqueID);
    #Requirements = regexp("2119/jobmanager", other.GlueCEUniqueID);
    
    RetryCount = 1;
    ShallowRetryCount = 2;
  • Where:
    [ale@cream-15 UI]$ cat exe/exe.sh
    #!/bin/sh
    
    date
    echo "Hello world!"
    echo "My prologue said:" 
    cat prologue.out
    echo "My argument is $1"
    
    [ale@cream-15 UI]$ cat exe/prologue.sh 
    #!/bin/sh
    
    date > prologue.out
    echo "I'm the prologue" >> prologue.out
    echo "##########################" >> prologue.out
    echo "This is the output of an ls command" >> prologue.out
    ls >> prologue.out
    echo "##########################" >> prologue.out
    
    [ale@cream-15 UI]$ cat exe/epilogue.sh 
    #!/bin/sh
    
    echo " This is the output of the job: `cat std.out`" > epilogue.out
    echo "#########################" >> epilogue.out
    echo "We finish at `date`" >> epilogue.out
    echo "#########################" >> epilogue.out
    echo "All the jokes are done!" >> epilogue.out

  • Use the requirements expression to choose an LCG or a CREAM ce
  • When the job is Done retrieve the output sandbox, it should contains these files:
    • epilogue.out
    • prologue.out
    • std.err and std.out
  • This should be the contenent of the files:
    [ale@cream-15 UI]$ cat prologue.out
    Fri Jun 11 10:24:15 BST 2010
    I'm the prologue
    ##########################
    This is the output of an ls command
    epilogue.sh
    exe.sh
    pippo
    prologue.out
    prologue.sh
    ##########################
    
    [ale@cream-15 UI]$ cat std.out
    Fri Jun 11 10:24:15 BST 2010
    Hello world!
    My prologue said:
    Fri Jun 11 10:24:15 BST 2010
    I'm the prologue
    ##########################
    This is the output of an ls command
    epilogue.sh
    exe.sh
    pippo
    prologue.out
    prologue.sh
    ##########################
    My argument is 1000
    
    [ale@cream-15 UI]$ cat epilogue.out
     This is the output of the job: Fri Jun 11 10:24:15 BST 2010
    Hello world!
    My prologue said:
    Fri Jun 11 10:24:15 BST 2010
    I'm the prologue
    ##########################
    This is the output of an ls command
    epilogue.sh
    exe.sh
    pippo
    prologue.out
    prologue.sh
    ##########################
    My argument is 1000
    #########################
    We finish at Fri Jun 11 10:24:15 BST 2010
    #########################
    All the jokes are done!

 

Revision 72010-03-04 - AlessioGianelle

Line: 1 to 1
 

WMS Test Plan

Line: 241 to 241
 

Others

Changed:
<
<

BrokerInfo

>
>

BrokerInfo

 
  • Use a jdl like this one to verify its creation:
Changed:
<
<
[
>
>
########################################### # JDL to test Brokerinfo Creation # ###########################################
 Executable = "/bin/ls" ;
Changed:
<
<
Arguments = "-lha " ; Stdoutput = "ls.out" ; OutputSandbox = { ".BrokerInfo", "ls.out" } ; InputSandbox = { "data/input.txt" }; ]
  • After the retrieving of the output files you should find two files:
    [ale@cream-15 UI]$ ls -la /tmp/ale_m5xVaFwXoOAkUlQgTGTZxw
    total 1272
    drwxr-xr-x   2 ale  ale     4096 Mar  3 18:23 .
    drwxrwxrwt  16 root root 1269760 Mar  3 18:23 ..
    -rw-rw-r--   1 ale  ale    14432 Mar  3 18:23 .BrokerInfo
    -rw-rw-r--   1 ale  ale      646 Mar  3 18:23 ls.out
  • In the file ls.out there should be listed also the .BrokerInfo:
    [ale@cream-15 UI]$ cat /tmp/ale_m5xVaFwXoOAkUlQgTGTZxw/ls.out
    drwxr-xr-x  2 dteam009 cg 4.0K Mar  3 17:49 .
    drwx------  5 dteam009 cg 4.0K Mar  3 17:49 ..
    -rw-r--r--  1 dteam009 cg  15K Mar  3 17:49 .BrokerInfo
    -rw-r--r--  1 dteam009 cg  166 Mar  3 17:49 https_3a_2f_2fdevel17.cnaf.infn.it_3a9000_2fm5xVaFwXoOAkUlQgTGTZxw.output
    -rw-r--r--  1 dteam009 cg    9 Mar  3 17:49 input.txt
    -rw-r--r--  1 dteam009 cg    0 Mar  3 17:49 ls.out
    -rw-------  1 dteam009 cg    0 Mar  3 17:49 tmp.MsRYWZ5040
>
>
Arguments = "-la"; StdOutput = "std.out"; StdError = "std.err"; FuzzyRank = true; InputSandbox = {"calc-pi.sh", "fileA", "fileB"}; OutputSandbox = {"std.out", "std.err",".BrokerInfo"};

Requirements = (other.GlueCEInfoHostName = "spacin-ce1.dma.unina.it") && regexp("8443/cream", other.GlueCEUniqueID) ;

DataRequirements = { [ DataCatalogType = "DLI"; DataCatalog = "http://lfcserver.cnaf.infn.it:8085"; InputData = {"lfn:/grid/infngrid/cesini/PI_1M.txt","lfn:/grid/infngrid/cesini/e-2M.txt"}; ] };

DataAccessProtocol = "gsiftp";

  • After the retrieving of the output files you should see these files:
    [ale@cream-15 DataReq]$ ls -la /tmp/ale_5XbbEhIAtjO5uC5wRyCmyA
    total 1264
    drwxr-xr-x   2 ale  ale     4096 Mar  4 10:31 .
    drwxrwxrwt  17 root root 1269760 Mar  4 10:31 ..
    -rw-rw-r--   1 ale  ale     7804 Mar  4 10:31 .BrokerInfo
    -rw-rw-r--   1 ale  ale        0 Mar  4 10:31 std.err
    -rw-rw-r--   1 ale  ale      624 Mar  4 10:31 std.out
And also:
[ale@cream-15 DataReq]$ cat /tmp/ale_5XbbEhIAtjO5uC5wRyCmyA/std.out
total 32
drwxr-xr-x  2 dteam038 dteam 4096 Mar  4 10:27 .
drwx------  5 dteam038 dteam 4096 Mar  4 10:27 ..
-rw-r--r--  1 dteam038 dteam 7804 Mar  4 10:27 .BrokerInfo
-rw-r--r--  1 dteam038 dteam  216 Mar  4 10:27 calc-pi.sh
-rw-r--r--  1 dteam038 dteam   17 Mar  4 10:27 fileA
-rw-r--r--  1 dteam038 dteam   17 Mar  4 10:27 fileB
-rw-r--r--  1 dteam038 dteam  166 Mar  4 10:27 https_3a_2f_2fdevel17.cnaf.infn.it_3a9000_2f5XbbEhIAtjO5uC5wRyCmyA.output
-rw-r--r--  1 dteam038 dteam    0 Mar  4 10:27 std.err
-rw-r--r--  1 dteam038 dteam    0 Mar  4 10:27 std.out
-rw-------  1 dteam038 dteam    0 Mar  4 10:27 tmp.LesGC30313
 
  • Check the goodness of the .BrokerInfo file generated:
Changed:
<
<
    • glite-brokerinfo getCE -f /tmp/ale_m5xVaFwXoOAkUlQgTGTZxw/.BrokerInfo
      • Returns the matched CE
    • glite-brokerinfo getVirtualOrganization -f /tmp/ale_m5xVaFwXoOAkUlQgTGTZxw/.BrokerInfo
      • Returns your VO
>
>
    • glite-brokerinfo getCE -f glite-brokerinfo getCE -f < .Brokerinfo file path >
      • It should return the matched CE
    • glite-brokerinfo getDataAccessProtocol -f < .Brokerinfo file path >
      • It should return the DataAccessProtocol parameter as specified in the jdl
    • glite-brokerinfo getInputData -f < .Brokerinfo file path >
      • It should return the InputData files specified in the jdl
    • glite-brokerinfo getSEs -f < .Brokerinfo file path >
      • It should return all the SEs (seen by the BDII of the WMS) that store all the files specified in the InputData attribute (You can chek it using the command lcg-lr --vo <VO name> <logical file name>)
    • glite-brokerinfo getCloseSEs -f < .Brokerinfo file path >
      • It should return only the CloseSe to the matched CE (to see them use the command: ldapsearch -x -h <specify the BDII host name> -p 2170 -b mds-vo-name=local,o=grid "(&(objectclass=GlueCESEBindGroup)(GlueCESEBindGroupCEUniqueID=<specify the CE name>))" | grep GlueCESEBindGroupSEUniqueID)
    • glite-brokerinfo getLFN2SFN <LFN of the files in InputData > -f < .Brokerinfo file path >
      • It should return the output of the command lcg-lr --vo <VO name> <logical file name>
    • glite-brokerinfo getVirtualOrganization -f < .Brokerinfo file path >
      • It should return your VO
 

Revision 62010-03-03 - AlessioGianelle

Line: 1 to 1
 

WMS Test Plan

Line: 239 to 239
 
    • You can find more CEs because you should also filter using the "vo" attribute.
Added:
>
>

Others

BrokerInfo

  • Use a jdl like this one to verify its creation:
     [
    Executable = "/bin/ls" ;
    Arguments = "-lha " ;
    Stdoutput = "ls.out" ;
    OutputSandbox = { ".BrokerInfo", "ls.out" } ;
    InputSandbox = { "data/input.txt" };
    ]
  • After the retrieving of the output files you should find two files:
    [ale@cream-15 UI]$ ls -la /tmp/ale_m5xVaFwXoOAkUlQgTGTZxw
    total 1272
    drwxr-xr-x   2 ale  ale     4096 Mar  3 18:23 .
    drwxrwxrwt  16 root root 1269760 Mar  3 18:23 ..
    -rw-rw-r--   1 ale  ale    14432 Mar  3 18:23 .BrokerInfo
    -rw-rw-r--   1 ale  ale      646 Mar  3 18:23 ls.out
  • In the file ls.out there should be listed also the .BrokerInfo:
    [ale@cream-15 UI]$ cat /tmp/ale_m5xVaFwXoOAkUlQgTGTZxw/ls.out
    drwxr-xr-x  2 dteam009 cg 4.0K Mar  3 17:49 .
    drwx------  5 dteam009 cg 4.0K Mar  3 17:49 ..
    -rw-r--r--  1 dteam009 cg  15K Mar  3 17:49 .BrokerInfo
    -rw-r--r--  1 dteam009 cg  166 Mar  3 17:49 https_3a_2f_2fdevel17.cnaf.infn.it_3a9000_2fm5xVaFwXoOAkUlQgTGTZxw.output
    -rw-r--r--  1 dteam009 cg    9 Mar  3 17:49 input.txt
    -rw-r--r--  1 dteam009 cg    0 Mar  3 17:49 ls.out
    -rw-------  1 dteam009 cg    0 Mar  3 17:49 tmp.MsRYWZ5040

  • Check the goodness of the .BrokerInfo file generated:
    • glite-brokerinfo getCE -f /tmp/ale_m5xVaFwXoOAkUlQgTGTZxw/.BrokerInfo
      • Returns the matched CE
    • glite-brokerinfo getVirtualOrganization -f /tmp/ale_m5xVaFwXoOAkUlQgTGTZxw/.BrokerInfo
      • Returns your VO

 

Revision 52010-03-02 - AlessioGianelle

Line: 1 to 1
 

WMS Test Plan

Line: 154 to 154
 

WMS Job List-match Testing

Added:
>
>

With data

  • Set these variables:
      export LCG_GFAL_INFOSYS=<BDII set in the wms conf file> (e.g. cert-bdii-04.cnaf.infn.it:2170)
      export LFC_HOST=<lfc host name> (e.g. lfcserver.cnaf.infn.it)
      export LFC_HOME=<lfc home directory> (e.g. lfcserver.cnaf.infn.it:/grid/infngrid)
  • Register a file to a SE using lcg-cr command:
      lcg-cr --vo <VO name> -d <SE host> -l lfn:<logical file name> file:<local file path>
  • Using the lcg-rep you can create some replicas:
      lcg-rep  --vo <VO name> -d <SE host> lfn:<logical file name>
  • lcg-lr is useful to see the list of replicas:
      lcg-lr --vo <VO name>  lfn:<logical file name>
  • Submit a list-match command with a jdl likes this one (as InputData put the lfn(s) registered before):
    ###########################################
    #      JDL with Data Requirements         #
    ###########################################
    
    Executable = "calc-pi.sh";
    Arguments = "1000";
    StdOutput = "std.out";
    StdError = "std.err";
    Prologue = "prologue.sh";
    InputSandbox = {"calc-pi.sh", "fileA", "fileB","prologue.sh"};
    OutputSandbox = {"std.out", "std.err","out-PI.txt","out-e.txt"};
    Requirements = true;
    
    DataRequirements = {
    [
    DataCatalogType = "DLI";
    DataCatalog = "http://lfcserver.cnaf.infn.it:8085";
    InputData = {"lfn:/grid/infngrid/cesini/PI_1M.txt","lfn:/grid/infngrid/cesini/e-2M.txt"};
    ]
    };
    DataAccessProtocol = "gsiftp";
  • You should obtain an output like this one:
    [ale@cream-15 DataReq]$ glite-wms-job-list-match -a --rank --config ~/UI/etc/wmp_wms007.conf data-req.jdl
    Connecting to the service https://wms007.cnaf.infn.it:7443/glite_wms_wmproxy_server
    
    ==========================================================================
    
             COMPUTING ELEMENT IDs LIST
     The following CE(s) matching your job requirements have been found:
    
      *CEId*                                                   *Rank*
    
     - cert-15.pd.infn.it:8443/cream-lsf-cert                           0
     - gridit-ce-001.cnaf.infn.it:2119/jobmanager-lcgpbs-cert           0
     - prod-ce-01.pd.infn.it:8443/cream-lsf-cert                        0
     - prod-ce-02.pd.infn.it:2119/jobmanager-lcglsf-cert                0
     - t2-ce-01.to.infn.it:8443/cream-pbs-cert                          0
     - t2-ce-01.to.infn.it:8443/cream-pbs-short                         0
     - t2-ce-02.to.infn.it:2119/jobmanager-lcgpbs-cert                  0
     - t2-ce-02.to.infn.it:2119/jobmanager-lcgpbs-short                 0
     - test7200a.cnaf.infn.it:2119/jobmanager-lcgpbs-cert               0
     - test7200a.cnaf.infn.it:2119/jobmanager-lcgpbs-parallel           0
     - devce.cnaf.infn.it:8443/cream-pbs-cert                           -700634
    
    ==========================================================================
  • Finally you can check if these CEs are really closed to the SEs where you replicate your file(s)
    • First list the SEs:
      [ale@cream-15 DataReq]$ lcg-lr --vo dteam  lfn:/grid/infngrid/cesini/PI_1M.txt
      
      srm://aliserv6.ct.infn.it/dpm/ct.infn.it/home/infngrid/generated/2009-11-24/file0b6a613c-e1f1-4e53-a328-2152e44a8576
      srm://gridit-se-01.cnaf.infn.it/dpm/cnaf.infn.it/home/infngrid/generated/2009-11-24/fileb9f3b972-806c-48a6-92fc-a9677901403a
      srm://prod-se-01.pd.infn.it/infngrid/generated/2009-11-24/file10b5fe11-ad95-41e6-a3a3-14bc5cb9694b
      srm://t2-se-00.to.infn.it/dpm/to.infn.it/home/infngrid/generated/2009-11-24/file1bb28727-1356-4fb1-bb73-5362f93e766f
    • then for each SEs in the list use this command to find the "Close" CEs:
      ldapsearch -x -h <BDII set in the WMS conf file> -p 2170 -b mds-vo-name=local,o=grid "(&(objectclass=GlueCESEBindGroup)(GlueCESEBindGroupSEUniqueID=<SE HOST>))" | grep ^GlueCESEBindGroupCEUniqueID
      
      GlueCESEBindGroupCEUniqueID: devce.cnaf.infn.it:8443/cream-pbs-cert
      GlueCESEBindGroupCEUniqueID: gridit-ce-001.cnaf.infn.it:2119/jobmanager-lcgpbs
      GlueCESEBindGroupCEUniqueID: cert-08.cnaf.infn.it:8443/cream-lsf-pps
      GlueCESEBindGroupCEUniqueID: cert-05.cnaf.infn.it:8443/cream-lsf-pps
      GlueCESEBindGroupCEUniqueID: test7200a.cnaf.infn.it:2119/jobmanager-lcgpbs-par
      GlueCESEBindGroupCEUniqueID: gridit-ce-001.cnaf.infn.it:2119/jobmanager-lcgpbs
      GlueCESEBindGroupCEUniqueID: cert-07.cnaf.infn.it:8443/cream-lsf-pps
      GlueCESEBindGroupCEUniqueID: cert-09.cnaf.infn.it:8443/cream-lsf-pps
      GlueCESEBindGroupCEUniqueID: gridit-ce-001.cnaf.infn.it:2119/jobmanager-lcgpbs
      GlueCESEBindGroupCEUniqueID: cert-06.cnaf.infn.it:8443/cream-lsf-pps
      GlueCESEBindGroupCEUniqueID: cert-04.cnaf.infn.it:8443/cream-lsf-pps
      GlueCESEBindGroupCEUniqueID: test7200a.cnaf.infn.it:2119/jobmanager-lcgpbs-cer
      GlueCESEBindGroupCEUniqueID: cert-13.cnaf.infn.it:8443/cream-lsf-pps
    • You can find more CEs because you should also filter using the "vo" attribute.

 -- ElisabettaMolinari - 2010-02-24

Revision 42010-02-25 - ElisabettaMolinari

Line: 1 to 1
 

WMS Test Plan

Line: 46 to 46
 

Perusal job

Job perusal is the ability to view output from a job while it is running. In order to use this feature the JDL attributes PerusalFileEnable and PerusalTimeInterval must be declared when the job is submitted for example:
PerusalFileEnable = true;
PerusalTimeInterval = 120;
Changed:
<
<
This makes the WN upload, at regular time intervals (PerusalTimeInterval), a copy of the output files specified using the glite-wms-job-perusal --set command to the WMS machine Here's a sample perusal jdl file:
[
>
>
This makes the WN upload, at regular time intervals (PerusalTimeInterval), a copy of the output files specified using the glite-wms-job-perusal --set command to the WMS machine.
  • Following an example on how to test a perusal job:
    • Given the jdl file:
      [
 Executable = "perusal.sh" ; InputSandbox = {"file://$HOME/perusal.sh"}; Stdoutput = "perusal.out" ;
Line: 69 to 70
  echo "" echo "Bye"
Added:
>
>
    • Submit the job with
       glite-wms-job-submit --config glite_wms.conf -a perusal.jdl
    • Given the jobid of the previously submitted set the files that are to be perused with
      glite-wms-job-perusal --set -f perusal.out https://devel17.cnaf.infn.it:9000/HBPb0WwhMfhKbjdUkrasWg
    • When the job starts running on the WN get the files as here:
      glite-wms-job-perusal --get -f perusal.out https://devel17.cnaf.infn.it:9000/HBPb0WwhMfhKbjdUkrasWg
 

DAG job

Line: 144 to 148
 

Collection Job

Added:
>
>

WMS Job shallow and deep re-submission

  • For the shallow resubmission add the following attributes to your favourite jdl file:
    Prologue = "/bin/false"; ShallowRetryCount = 3;
  • For the deep resubmission add the following attributes:
     Prologue = "/bin/false"; RetryCount = 3;
 

WMS Job List-match Testing

Revision 32010-02-24 - ElisabettaMolinari

Line: 1 to 1
 

WMS Test Plan

Line: 43 to 43
 ShallowRetryCount = 3; ]
Added:
>
>

Perusal job

Job perusal is the ability to view output from a job while it is running. In order to use this feature the JDL attributes PerusalFileEnable and PerusalTimeInterval must be declared when the job is submitted for example:
PerusalFileEnable = true;
PerusalTimeInterval = 120;
This makes the WN upload, at regular time intervals (PerusalTimeInterval), a copy of the output files specified using the glite-wms-job-perusal --set command to the WMS machine Here's a sample perusal jdl file:
[
Executable = "perusal.sh" ;
InputSandbox = {"file://$HOME/perusal.sh"};
Stdoutput = "perusal.out" ;
StdError =  "perusal.err" ;
OutputSandbox ={ "perusal.out" ,"perusal.err"} ;
usertags = [ jdl = "perusal" ];
PerusalFileEnable = true;
]
where perusal.sh is
#!/bin/bash

echo "Hi all, this is a perusal test program..."

for (( i=0 ; $i < 50 ; i = $(($i + 1)) )); do

        echo "Step number $i"
        date
        sleep 10

done

echo ""
echo "Bye"
 

DAG job

to JC and IC as above specifying Requirements = ( RegExp("/cream-",other.GlueCEUniqueID)); or Requirements = ( RegExp("/cream-",other.GlueCEUniqueID)); respectively in the job jdl
  • DAG job jdls samples:
    [
Line: 116 to 144
 

Collection Job

Added:
>
>
 

WMS Job List-match Testing

-- ElisabettaMolinari - 2010-02-24

Revision 22010-02-24 - ElisabettaMolinari

Line: 1 to 1
 

WMS Test Plan

Line: 7 to 7
 

WMS Job Submission/GetOutput Testing

Registers the job into the wmproxy, delivers it to the WM that submits it to the grid. Test job submission with the following type of jobs/jdl:
Changed:
<
<
  • Normal
>
>

Normal Job

 
    • to JC
      [
      Type = "Job";
      JobType = "normal";
Line: 42 to 42
 RetryCount = 0; ShallowRetryCount = 3; ]
Deleted:
<
<
where tests.sh for both the above jdl is
#/bin/sh
 
Changed:
<
<
echo "Hello...";

  • DAG to JC and IC as above specifying Requirements = ( RegExp("/cream-",other.GlueCEUniqueID)); or Requirements = ( RegExp("/cream-",other.GlueCEUniqueID)); respectively in the job jdl
>
>

DAG job

to JC and IC as above specifying Requirements = ( RegExp("/cream-",other.GlueCEUniqueID)); or Requirements = ( RegExp("/cream-",other.GlueCEUniqueID)); respectively in the job jdl
 
    • DAG job jdls samples:
      [
      Type = "dag";
      VirtualOrganisation = "dteam";
Line: 76 to 74
 Dependencies = { {{nodeA, nodeB}, nodeC},{nodeD,nodeE.nodeF} }
Added:
>
>
]
[
    type = "dag";
    DefaultNodeShallowRetryCount = 3;
        nodes = [
      nodeA = [
        node_type = "edg-jdl";
        file ="myjob.jdl" ;
      ];
        nodeB = [
                                node_type = "edg-jdl";
        file ="myjob.jdl" ;
       ];
         nodeC = [
         node_type = "edg-jdl";
         file ="myjob.jdl" ;
       ];
                        dependencies = {
        { nodeA, nodeB },
                                { nodeA, nodeC }
       }
     ];
  ]

Parametric Job

Submit it to ICE and JC. Following a sample jdl:
// Test parametric job.
// The standar Output and Error are parametrizied such as the
// enviroment variable: MYPATH_PARAM_.
 [
  JobType = "parametric";
  Executable = "/usr/bin/env";
  Environment = {"MYPATH_PARAM_=$PATH:/bin:/usr/bin:$HOME"};
  StdOutput = "echo_PARAM_.out";
  StdError = "echo_PARAM_.err";
  OutputSandbox = {"echo_PARAM_.out","echo_PARAM_.err"};
  Requirements = ( RegExp("/cream-",other.GlueCEUniqueID));
  Parameters =  5;
        usertags = [ jdl = "parametric cream" ];
 ]

Added:
>
>

Collection Job

 

WMS Job List-match Testing

-- ElisabettaMolinari - 2010-02-24

Revision 12010-02-24 - ElisabettaMolinari

Line: 1 to 1
Added:
>
>

WMS Test Plan

YAIM-WMS Configuration Testing

WMS Job Submission/GetOutput Testing

Registers the job into the wmproxy, delivers it to the WM that submits it to the grid. Test job submission with the following type of jobs/jdl:
  • Normal
    • to JC
      [
      Type = "Job";
      JobType = "normal";
      InputSandbox = { "file://$HOME/test.sh"};
      VirtualOrganisation = "dteam";
      Executable="test.sh";
      Arguments="Hello ";
      Requirements = ( !RegExp("/cream-",other.GlueCEUniqueID));
      Rank = 0;
      fuzzyrank = true;
      StdOutput="message.txt";
      StdError="err.log";
      OutputSandbox={"message.txt","err.log",".BrokerInfo"};
      RetryCount = 0;
      usertags = [ jdl = "normal job" ];
      ShallowRetryCount = 3;
      ]
  • to ICE
    [
    Type = "Job";
    JobType = "normal";
    InputSandbox = { "file:///home/emolinari/test.sh"};
    VirtualOrganisation = "dteam";
    Executable="test.sh";
    Arguments="Hello ";
    Requirements = ( RegExp("/cream-",other.GlueCEUniqueID));
    Rank = 0;
    fuzzyrank = true;
    StdOutput="message.txt";
    StdError="err.log";
    OutputSandbox={"message.txt","err.log",".BrokerInfo"};
    usertags = [ jdl = "normal job to ICE" ];
    RetryCount = 0;
    ShallowRetryCount = 3;
    ]
where tests.sh for both the above jdl is
#/bin/sh
                                                                                                                    
echo "Hello...";

  • DAG to JC and IC as above specifying Requirements = ( RegExp("/cream-",other.GlueCEUniqueID)); or Requirements = ( RegExp("/cream-",other.GlueCEUniqueID)); respectively in the job jdl
    • DAG job jdls samples:
      [
      Type = "dag";
      VirtualOrganisation = "dteam";
      Max_nodes_running = 10;
      InputSandbox = "test.sh";
      FuzzyRank = true;
      Nodes = [
      nodeA = [
      file= "test_dag.jdl";
      ];
      nodeB = [
      file= "test_dag.jdl";
      ];
      nodeC = [
      file= "test_dag.jdl";
      ];
      nodeD = [
      file= "test_dag.jdl";
      ];
      nodeE = [
      file= "test_dag.jdl";
      ];
      nodeF= [
      file= "test_dag.jdl";
      ];
      ];
      Dependencies = {
      {{nodeA, nodeB}, nodeC},{nodeD,nodeE.nodeF}
      }
      ]

WMS Job List-match Testing

-- ElisabettaMolinari - 2010-02-24

 
This site is powered by the TWiki collaboration platformCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback