Deployment tests

bug #78484: [ YAIM_WMS ] Multiple parameter configuration added in condor_config.local

Test description.

[root@cream-44 ~]# cat /opt/condor-c/local.cream-44/condor_config.local | grep GRID_MONITOR
GRID_MONITOR_HEARTBEAT_TIMEOUT = 300
GRID_MONITOR_RETRY_DURATION = 31536000
GRID_MONITOR = $(RELEASE_DIR)/libexec/glite/grid_monitor.sh
ENABLE_GRID_MONITOR = TRUE

[root@cream-44 ~]# /opt/glite/yaim/bin/yaim -c -s /usr/local/nfs/jra1_test/cream-44/site-info_wms.def -n WMS
WARNING:
WARNING: *****************************************************************************
WARNING: Your siteinfo directory is world readable. This is generally a bad idea
WARNING: as configuration files may contain passwords and other sensitive information.
WARNING: *****************************************************************************
WARNING:
[ ... ]
DEBUG: Skipping function: config_glite_initd_setenv because it is not defined
INFO: Executing function: config_glite_initd
INFO: Configuration Complete. [ OK ]
INFO: YAIM terminated succesfully.

[root@cream-44 ~]# cat /opt/condor-c/local.cream-44/condor_config.local | grep GRID_MONITOR
GRID_MONITOR_HEARTBEAT_TIMEOUT = 300
GRID_MONITOR_RETRY_DURATION = 31536000
GRID_MONITOR = $(RELEASE_DIR)/libexec/glite/grid_monitor.sh
ENABLE_GRID_MONITOR = TRUE

Considered successful if the line GRID_MONITOR is not added after configuring.

bug #79141: various bugs about parametric jobs

Test description.

[ale@cream-12 UI]$ cat 79141.jdl
[
JobType = "Parametric";
Parameters = 1;
ParameterStart = 0;
ParameterStep = 1;
Executable = "/bin/ls";
requirements = other.GlueCEStateStatus == "Production";
Arguments = "-la";
Stdoutput = "output.txt";
InputSandbox = {"Src_PARAM_/Makefile", "Test_PARAM_"};
OutputSandbox = {"output.txt"};
]
[ale@cream-12 UI]$ glite-wms-job-submit -a -c etc/wmp_devel11.conf 79141.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel11.cnaf.infn.it:9000/5...

==========================================================================

[ale@cream-12 UI]$ glite-wms-job-status https://devel11.cnaf.infn.it:9000/5...

======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel11.cnaf.infn.it:9000/5...
Current Status: Waiting
Submitted: Wed Apr 20 09:37:09 2011 CEST
==========================================================================

- Nodes information for:
Status info for the Job : https://devel11.cnaf.infn.it:9000/S...
Current Status: Ready
Status Reason: unavailable
Destination: ce-enmr.chemie.uni-frankfurt.de:2119/jobmanager-lcgpbs-cert
Submitted: Wed Apr 20 09:37:09 2011 CEST
==========================================================================

[ale@cream-12 UI]$ glite-wms-job-status https://devel11.cnaf.infn.it:9000/5...

======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel11.cnaf.infn.it:9000/5...
Current Status: Done (Success)
Exit code: 0
Submitted: Wed Apr 20 09:37:09 2011 CEST
==========================================================================

- Nodes information for:
Status info for the Job : https://devel11.cnaf.infn.it:9000/S...
Current Status: Done (Success)
Logged Reason(s):
-
- Job terminated successfully
Exit code: 0
Status Reason: Job terminated successfully
Destination: ce-enmr.chemie.uni-frankfurt.de:2119/jobmanager-lcgpbs-cert
Submitted: Wed Apr 20 09:37:09 2011 CEST
==========================================================================

[ale@cream-12 UI]$ glite-wms-job-output https://devel11.cnaf.infn.it:9000/5...

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

================================================================================

JOB GET OUTPUT OUTCOME

Output sandbox files for the DAG/Collection :
https://devel11.cnaf.infn.it:9000/5...
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/ale_5x11U1aJFoKSDbpwu4-apw

================================================================================

[ale@cream-12 UI]$ cat /tmp/jobOutput/ale_5x11U1aJFoKSDbpwu4-apw/Node_0/output.txt
total 32
drwxr-xr-x 2 dteam035 dteam 4096 Apr 20 09:38 .
drwx------ 5 dteam035 dteam 4096 Apr 20 09:37 ..
-rw-r--r-- 1 dteam035 dteam 2150 Apr 20 09:37 .BrokerInfo
-rw-r--r-- 1 dteam035 dteam 118 Apr 20 09:38 https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2fShaETXWYzsKgV_5f0sd8o9uQ.output
-rw-r--r-- 1 dteam035 dteam 0 Apr 20 09:37 Makefile
-rw-r--r-- 1 dteam035 dteam 0 Apr 20 09:38 output.txt
-rw-r--r-- 1 dteam035 dteam 0 Apr 20 09:37 Test0
-rw------- 1 dteam035 dteam 103 Apr 20 09:38 tmp.fzuGl19574
-rw------- 1 dteam035 dteam 2 Apr 20 09:38 tmp.GxlYw19573
-rw------- 1 dteam035 dteam 103 Apr 20 09:38 tmp.mwAGt19459
-rw------- 1 dteam035 dteam 0 Apr 20 09:38 tmp.OfMof19572
-rw------- 1 dteam035 dteam 2 Apr 20 09:38 tmp.oizWr19458 

Considered successful if the parametric job is successfully executed and the ouptut correctly retrieved.

[root@devel09 ~]# ps aux |grep ice
glite     2716  0.0  0.0 220996  2192 ?        Ss   09:36   0:00 /usr/bin/glite-wms-ice-safe --conf glite_wms.conf --daemon /tmp/glite-wms-ice-safe.pid
glite     2720  0.0  0.0  63856  1108 ?        S    09:36   0:00 sh -c /usr/bin/glite-wms-ice --conf glite_wms.conf /var/log/wms/ice_console.log 2>&1
glite     2721  0.0  0.1 483464  7404 ?        Sl   09:36   0:02 /usr/bin/glite-wms-ice --conf glite_wms.conf /var/log/wms/ice_console.log
root     31253  0.0  0.0  61184   788 pts/1    S+   14:29   0:00 grep ice
[root@devel09 ~]# /etc/init.d/glite-wms-ice restart
stopping ICE... ok
starting ICE... ok
[root@devel09 ~]# ps aux |grep ice
glite    31318  0.0  0.0 220996  2196 ?        Ss   14:30   0:00 /usr/bin/glite-wms-ice-safe --conf glite_wms.conf --daemon /tmp/glite-wms-ice-safe.pid
glite    31322  0.0  0.0  63856  1112 ?        S    14:30   0:00 sh -c /usr/bin/glite-wms-ice --conf glite_wms.conf /var/log/wms/ice_console.log 2>&1
glite    31323  1.0  0.1 483464  7352 ?        Sl   14:30   0:00 /usr/bin/glite-wms-ice --conf glite_wms.conf /var/log/wms/ice_console.log
root     31363  0.0  0.0  61180   784 pts/1    S+   14:30   0:00 grep ice

Considered successful if ice processes are restarted.

bug #70061: WMS hates collections with 192 nodes!

Test description.

[mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf coll_192.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel11.cnaf.infn.it:9000/T...

==========================================================================

Considered successful if the collection is submitted and executed correctly.

bug #70331: glite-wms-create-proxy "ambiguous redirect"

Test description.

[root@cream-44 cron.d]# su - glite
[glite@cream-44 ~]$ /usr/sbin/glite-wms-create-proxy.sh /var/wms.proxy /var/log/wms/create_proxy.log
[glite@cream-44 ~]$

Considered successful if the command works with no error messages.

bug #70824: environment values in JDL cannot have spaces

Test description.

Submit this jdl:

[
Executable = "/bin/env";
Stdoutput = "env.out" ;
StdError = "env.err" ;
shallowretrycount = 2;
retrycount = 3;
OutputSandbox = { "env.out" ,"env.err" } ;
Environment = {
"ATHENA_RUN_EVENTS='[(152345L, 216721L)]'"
};

]

======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel09.cnaf.infn.it:9000/g...
Current Status: Done (Success)
Exit code: 0
Status Reason: Job terminated successfully
Destination: gridgate.ucc.ie:2119/jobmanager-pbs-sixhour
Submitted: Thu Apr 21 17:11:46 2011 CEST
==========================================================================

After output retrieval:

[ale@cream-12 UI]$ glite-wms-job-output https://devel09.cnaf.infn.it:9000/g...

Connecting to the service https://devel09.cnaf.infn.it:7443/g...

================================================================================

JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://devel09.cnaf.infn.it:9000/g...
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/ale_gvpbgkBN_KkLhly-uNjpxQ

================================================================================

Check the output file:

[ale@cream-12 UI]$ cat /tmp/jobOutput/ale_gvpbgkBN_KkLhly-uNjpxQ/env.out | grep ATHENA
ATHENA_RUN_EVENTS=[(152345L, 216721L)] 

Considered successful if the environment contains the variable with spaces.

bug #71863: JobWrapper tries to use "test -eq" for string comparison

Test description.

Looking into a JW:

fatal_error() # 1 - reason, 2 - transfer OSB
{
if [ "x${LM_done_failed_reason}" != "x" ]; then
jw_echo "LM_log_done_begin" # parsed 'as is' by LM
jw_echo "${LM_done_failed_reason}"
jw_echo "LM_log_done_end" # parsed 'as is' by LM
fi
jw_echo "$1"
log_done_failed 1 "$1"
if [ "x$2" == "xOSB" ]; then
OSB_transfer
fi
doExit 1
} 

Considered successful if the comparison is done via ==

bug #73192: Submission failed due to a credential problem

Test description.

Simply try a submit:

[ale@cream-12 UI]$ glite-wms-job-submit -a -c etc/wmp_devel09.conf jdl/deepresub.jdl

Connecting to the service https://devel09.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel09.cnaf.infn.it:9000/L...

========================================================================== 

Considered successful if the job is correctly submitted.

bug #73699: Wrong retry count computation

Test description.

Submit this jdl:

[
requirements = (other.GlueCEStateStatus == "Production");
Rank = -2 * other.GlueCEStateWaitingJobs;
Executable = "/bin/ls";
prologue = "/bin/false";
shallowretrycount = -1;
RetryCount = 3;
usertags = [ exe = "deepresub" ];
requirements=regexp("8443/cream", other.GlueCEUniqueID);

]

The job should file after 3 resubmissions:

======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel09.cnaf.infn.it:9000/L...
Current Status: Aborted
Logged Reason(s):
- Prologue failed with error 1
- reason=1; Prologue failed with error 1
- Transfer to CREAM failed due to exception: Failed to create a delegation id for job https://devel09.cnaf.infn.it:9000/L...: reason is Received NULL fault; the error is due to another cause: FaultString=[] - FaultCode=[SOAP-ENV:Server.generalException] - FaultSubCode=[SOAP-ENV:Server.generalException] - FaultDetail=[<ns1:DelegationException><msg>User C=IT,O=INFN,OU=Personal Certificate,L=Padova,CN=Alessio Gianelle not authorized for operation {http://www.gridsite.org/namespaces/...</msg></ns1:DelegationException><ns2:exceptionName xmlns:ns2="http://xml.apache.org/axis/">org.glite.security.delegation.DelegationException</ns2:exceptionName><ns3:hostname xmlns:ns3="http://xml.apache.org/axis/">cluster50.knu.ac.kr</ns3:hostname>]
- Prologue failed with error 1
- Prologue failed with error 1
- Prologue failed with error 1
- reason=1; Prologue failed with error 1
Status Reason: hit job retry count (3)
Destination: infnlns-ce-01.ct.pi2s2.it:8443/cream-lsf-cert
Submitted: Thu Apr 21 16:47:33 2011 CEST

Infact:

[ale@cream-12 UI]$ glite-wms-job-logging-info --event Resubmission https://devel09.cnaf.infn.it:9000/L...

===================== glite-wms-job-logging-info Success =====================

LOGGING INFORMATION:

Printing info for the Job : https://devel09.cnaf.infn.it:9000/L...

---
Event: Resubmission
- Result = WILLRESUB
- Source = LogMonitor
- Timestamp = Thu Apr 21 16:49:44 2011 CEST
---
Event: Resubmission
- Result = WILLRESUB
- Source = WorkloadManager
- Timestamp = Thu Apr 21 16:49:45 2011 CEST
---
Event: Resubmission
- Result = WILLRESUB
- Source = LogMonitor
- Timestamp = Thu Apr 21 16:49:50 2011 CEST
---
Event: Resubmission
- Result = WILLRESUB
- Source = WorkloadManager
- Timestamp = Thu Apr 21 16:49:51 2011 CEST
---
Event: Resubmission
- Result = WILLRESUB
- Source = LogMonitor
- Timestamp = Thu Apr 21 16:52:00 2011 CEST
---
Event: Resubmission
- Result = WILLRESUB
- Source = WorkloadManager
- Timestamp = Thu Apr 21 16:52:01 2011 CEST
---
Event: Resubmission
- Result = WILLRESUB
- Source = LogMonitor
- Timestamp = Thu Apr 21 16:54:11 2011 CEST
==========================================================================

(Two events one from LM and the other from WM for every resubmission, plus the last try which failed because limit is reached.)

Considered successful if the job is resubmitted the right number of times (three in this case).

bug #73711: edg_wll_SetLoggingJobProxy with empty sequence code returns "no state in DB"

Test description.

This bug represents a development issue. It is simply verified by checking that submission for both a job and a collection returns the jobid.

[mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf job.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel11.cnaf.infn.it:9000/B...

==========================================================================

[mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf coll_10.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel11.cnaf.infn.it:9000/T...

========================================================================== 

Considered successful if the jobids are correctly returned.

bug #73715: missing ReallyRunning event from LogMonitor

Test description.

[ale@cream-12 UI]$ glite-wms-job-logging-info -v 3 --event ReallyRunning https://devel09.cnaf.infn.it:9000/v...

===================== glite-wms-job-logging-info Success =====================

LOGGING INFORMATION:

Printing info for the Job : https://devel09.cnaf.infn.it:9000/v...

---
Event: ReallyRunning
- Arrived = Thu Apr 21 16:08:28 2011 CEST
- Host = grid003.fc.up.pt
- Level = SYSTEM
- Priority = synchronous
- Seqcode = UI=000000:NS=0000000004:WM=000005:BH=0000000000:JSS=000002:LM=000002:LRMS=000003:APP=000000:LBS=000000
- Source = LRMS
- Timestamp = Thu Apr 21 16:08:28 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
---
Event: ReallyRunning
- Arrived = Thu Apr 21 16:09:43 2011 CEST
- Host = devel09.cnaf.infn.it
- Level = SYSTEM
- Priority = synchronous
- Seqcode = UI=000000:NS=0000000004:WM=000005:BH=0000000000:JSS=000002:LM=000007:LRMS=000000:APP=000000:LBS=000000
- Source = LogMonitor
- Timestamp = Thu Apr 21 16:09:43 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy
- Wn seq = UI=000000:NS=0000000004:WM=000005:BH=0000000000:JSS=000002:LM=000002:LRMS=000000:APP=000000:LBS=000000
   Alessio Gianelle <ale>
Project Member

Considered successful if the reallyrunning event is effectively logged.

bug #74259: Previous matches information is not taken into account if direct submission is used

Test description.

Submit directly (i.e. use option -r ) a job which trigger a resubmission (e.g. prologue="/bin/false").

Then, when it fails, check the "UserTag" events:

[ale@cream-12 UI]$ glite-wms-job-logging-info -v 2 --event UserTag https://devel09.cnaf.infn.it:9000/Q...
===================== glite-wms-job-logging-info Success =====================

LOGGING INFORMATION:

Printing info for the Job : https://devel09.cnaf.infn.it:9000/Q...

---
Event: UserTag
- Arrived = Fri Apr 22 10:10:37 2011 CEST
- Host = devel09.cnaf.infn.it
- Name = exe
- Source = NetworkServer
- Src instance = https://devel09.cnaf.infn.it:7443/g...
- Timestamp = Fri Apr 22 10:10:37 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
- Value = deepresub
---
Event: UserTag
- Arrived = Fri Apr 22 10:10:39 2011 CEST
- Host = devel09.cnaf.infn.it
- Name = CEInfoHostName
- Source = WorkloadManager
- Src instance = 21624
- Timestamp = Fri Apr 22 10:10:39 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
- Value = alice16.spbu.ru
---
Event: UserTag
- Arrived = Fri Apr 22 10:10:41 2011 CEST
- Host = devel09.cnaf.infn.it
- Name = CEInfoHostName
- Source = WorkloadManager
- Src instance = 21624
- Timestamp = Fri Apr 22 10:10:41 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
- Value = alice16.spbu.ru
---
Event: UserTag
- Arrived = Fri Apr 22 10:10:43 2011 CEST
- Host = devel09.cnaf.infn.it
- Name = CEInfoHostName
- Source = WorkloadManager
- Src instance = 21624
- Timestamp = Fri Apr 22 10:10:43 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
- Value = alice16.spbu.ru
---
Event: UserTag
- Arrived = Fri Apr 22 10:10:45 2011 CEST
- Host = devel09.cnaf.infn.it
- Name = CEInfoHostName
- Source = WorkloadManager
- Src instance = 21624
- Timestamp = Fri Apr 22 10:10:45 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
- Value = alice16.spbu.ru
==========================================================================

Considered successful if the tag "CEInfoHostName" reports the same value (equal to the chosen CE) for all the events.

bug #74577: Wrong counter in ICE database is set at the job creation

Test description.

we can check that the first time a job is submitted the counter should be set to 1:

[ale@cream-12 UI]$ glite-wms-job-submit -a -c etc/wmp_devel09.conf test.jdl

Connecting to the service https://devel09.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel09.cnaf.infn.it:9000/Z...

==========================================================================

and on the WMS:
[root@devel09 persist_dir]# sqlite3 ice.db "select counter from proxy where myproxyurl='myproxy.cern.ch';"
1 

Considered successful if the counter is set to 1.

bug #75223: wrong reason logged

Test description.

submit a jdl like this one:

executable="/bin/sleep";
arguments="6";
Stdoutput = "std.out";
OutputSandbox = {"std.out"};
requirements=regexp("8443/cream", other.GlueCEUniqueID);
ShallowRetryCount = 2;
prologue="/bin/false";

and look for the Done failed reason in the logging info:

[ale@cream-12 UI]$ glite-wms-job-logging-info -v 2 --event Done https://devel09.cnaf.infn.it:9000/T...

===================== glite-wms-job-logging-info Success =====================

LOGGING INFORMATION:

Printing info for the Job : https://devel09.cnaf.infn.it:9000/T...

---
Event: Done
- Arrived = Thu Apr 21 13:31:27 2011 CEST
- Exit code = 0
- Host = lcg1371.gridpp.rl.ac.uk
- Reason = Prologue failed with error 1
- Source = LRMS
- Status code = FAILED
- Timestamp = Thu Apr 21 13:31:27 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
---
Event: Done
- Arrived = Thu Apr 21 13:32:27 2011 CEST
- Exit code = 0
- Host = devel09.cnaf.infn.it
- Reason = Prologue failed with error 1
- Source = LogMonitor
- Status code = FAILED
- Timestamp = Thu Apr 21 13:32:27 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy 

Considered successful if the Reason contains only "Prologue failed with error 1".

bug #75368: ICE should log a DONE_FAILED to LB every time the job is going to be resubmitted

Test description.

Submit a jdl to a failure Cream CE (i.e. a CE where you know that for some reasons job will fail) and check the logging-info, before "Resubmission" event ICE should log a "Done Failed" with the failure reason.

Event: Done
- Arrived = Thu Apr 21 13:24:09 2011 CEST
- Exit code = 0
- Host = devel09.cnaf.infn.it
- Reason = Transfer to CREAM failed due to exception: Failed to create a delegation id for job https://devel09.cnaf.infn.it:9000/6...: reason is Received NULL fault; the error is due to another cause: FaultString=[] - FaultCode=[SOAP-ENV:Server.generalException] - FaultSubCode=[SOAP-ENV:Server.generalException] - FaultDetail=[<ns1:DelegationException><msg>User C=IT,O=INFN,OU=Personal Certificate,L=Padova,CN=Alessio Gianelle not authorized for operation {http://www.gridsite.org/namespaces/...</msg></ns1:DelegationException><ns2:exceptionName xmlns:ns2="http://xml.apache.org/axis/">org.glite.security.delegation.DelegationException</ns2:exceptionName><ns3:hostname xmlns:ns3="http://xml.apache.org/axis/">foam.grid.kiae.ru</ns3:hostname>]
- Source = LogMonitor
- Status code = FAILED
- Timestamp = Thu Apr 21 13:24:09 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy
---
Event: Resubmission
- Arrived = Thu Apr 21 13:24:09 2011 CEST
- Host = devel09.cnaf.infn.it
- Reason = Resubmitting because of exception Failed to create a delegation id for job https://devel09.cnaf.infn.it:9000/6...: reason is Received NULL fault; the error is due to another cause: FaultString=[] - FaultCode=[SOAP-ENV:Server.generalException] - FaultSubCode=[SOAP-ENV:Server.generalException] - FaultDetail=[<ns1:DelegationException><msg>User C=IT,O=INFN,OU=Personal Certificate,L=Padova,CN=Alessio Gianelle not authorized for operation {http://www.gridsite.org/namespaces/...</msg></ns1:DelegationException><ns2:exceptionName xmlns:ns2="http://xml.apache.org/axis/">org.glite.security.delegation.DelegationException</ns2:exceptionName><ns3:hostname xmlns:ns3="http://xml.apache.org/axis/">foam.grid.kiae.ru</ns3:hostname>] CEUrl https://foam.grid.kiae.ru:8443/ce-c...
- Result = WILLRESUB
- Source = LogMonitor
- Tag = unavailable
- Timestamp = Thu Apr 21 13:24:09 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy

bug #75402: Synchronization loss between real validity of proxy and exp. time saved in ICE's database

Test description.

Rarely means that is not easy to reproduce, so we consider it hopefully fixed.

bug #77004: Wrong myproxyserver string processing in ICE

Test description.

Submit this jdl: executable="/bin/sleep"; arguments="6"; Stdoutput = "std.out"; OutputSandbox = {"std.out"}; requirements=regexp("8443/cream", other.GlueCEUniqueID); MyProxyServer = " ";

Then check the database's table "proxy" of ice in the WMS to see if the proxy is associated to a fake myproxy server:

[root@devel09 persist_dir]# sqlite3 ice.db "select * from proxy;" /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle-/dteam/Role=NULL/Capability=NULL||/var/ice/persist_dir/B23D0D7177A8B6234F1985493FA09FF41A4FA98C.proxy|1303415475|0

(the name of myproxy server should be in the second field)

bug #77055: "MyProxyServer: wrong type caught for attribute" for parametric jobs

Test description.

[ale@cream-12 UI]$ cat param.jdl
// Test parametric job.
// The standar Output and Error are parametrizied such as the
// enviroment variable: MYPATH_PARAM_.
[
JobType = "parametric";
Executable = "/bin/env";
Environment = {"MYPATH_PARAM_=$PATH:/bin:/usr/bin:$HOME"};
StdOutput = "echo_PARAM_.out";
StdError = "echo_PARAM_.err";
OutputSandbox = {"echo_PARAM_.out","echo_PARAM_.err"};
ParameterStart = 0;
ParameterStep = 2;
Parameters = 10;
myproxyserver="";
]
[ale@cream-12 UI]$ glite-wms-job-submit -a -c etc/wmp_cream-44.conf -o test.out param.jdl

Connecting to the service https://cream-44.pd.infn.it:7443/gl...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://cream-44.pd.infn.it:9000/H5...

The job identifier has been saved in the following file:
/home/ale/UI/test.out

========================================================================== 

Considered successful if the job is correctly submitted.

bug #77694: Resource BDII for WMS needs to be revisit

Test description.

[glite@cream-44 ~]$ cat /opt/glite/yaim/node-info.d/glite-wms | grep config_gip_only [glite@cream-44 ~]$

Considered successful if the function config_gip_only has been removed.

bug #77876: While purging DAGs/Collections the CLEAR event is only logged for the parent node

Test description.

Forcing a run of the cron purger you can then check the logging info of a "parent" node:

[ale@cream-12 UI]$ glite-wms-job-logging-info -v 2 https://cream-44.pd.infn.it:9000/vR...

===================== glite-wms-job-logging-info Success =====================

LOGGING INFORMATION:

Printing info for the Job : https://cream-44.pd.infn.it:9000/vR...

---
Event: RegJob
- Arrived = Wed Apr 20 14:01:25 2011 CEST
- Host = cream-44.pd.infn.it
- Jobtype = COLLECTION
- Ns = https://cream-44.pd.infn.it:7443/gl...
- Nsubjobs = 100
- Seed = WMPROXY
- Source = NetworkServer
- Src instance = https://cream-44.pd.infn.it:7443/gl...
- Timestamp = Wed Apr 20 14:01:25 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
---
Event: Accepted
- Arrived = Wed Apr 20 14:01:34 2011 CEST
- From = NetworkServer
- From host = cream-12.pd.infn.it
- Host = cream-44.pd.infn.it
- Source = NetworkServer
- Src instance = https://cream-44.pd.infn.it:7443/gl...
- Timestamp = Wed Apr 20 14:01:34 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
---
Event: EnQueued
- Arrived = Wed Apr 20 14:01:34 2011 CEST
- Host = cream-44.pd.infn.it
- Queue = /var/workload_manager/jobdir
- Result = START
- Source = NetworkServer
- Src instance = https://cream-44.pd.infn.it:7443/gl...
- Timestamp = Wed Apr 20 14:01:34 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
---
Event: EnQueued
- Arrived = Wed Apr 20 14:01:57 2011 CEST
- Host = cream-44.pd.infn.it
- Queue = /var/workload_manager/jobdir
- Result = OK
- Source = NetworkServer
- Src instance = https://cream-44.pd.infn.it:7443/gl...
- Timestamp = Wed Apr 20 14:01:57 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
---
Event: DeQueued
- Arrived = Wed Apr 20 14:05:03 2011 CEST
- Host = cream-44.pd.infn.it
- Queue = /var/workload_manager/jobdir
- Source = WorkloadManager
- Src instance = 24901
- Timestamp = Wed Apr 20 14:05:03 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy
---
Event: CollectionState
- Arrived = Wed Apr 20 14:05:11 2011 CEST
- Child = https://cream-44.pd.infn.it:9000/_y...
- Child event = Running
- Done code = 0
- Histogram = Submitted=0, Waiting=0, Ready=0, Scheduled=0, Running=1, Done=0, Cleared=0, Aborted=0, Cancelled=0, Unknown=99, Purged=0
- Host = cream-44.pd.infn.it
- Source = LBServer
- State = Running
- Timestamp = Wed Apr 20 14:05:11 2011 CEST
- User = /C=IT/O=INFN/OU=Host/L=Padova/CN=cream-44.pd.infn.it
---
Event: CollectionState
- Arrived = Wed Apr 20 14:29:59 2011 CEST
- Child = https://cream-44.pd.infn.it:9000/H3...
- Child event = Done
- Done code = 0
- Histogram = Submitted=0, Waiting=0, Ready=0, Scheduled=0, Running=0, Done=82, Cleared=0, Aborted=7, Cancelled=0, Unknown=11, Purged=0
- Host = cream-44.pd.infn.it
- Source = LBServer
- State = Waiting
- Timestamp = Wed Apr 20 14:29:59 2011 CEST
- User = LBProxy
---
Event: Clear
- Arrived = Thu Apr 21 10:13:20 2011 CEST
- Host = cream-44.pd.infn.it
- Reason = 2
- Source = NetworkServer
- Src instance = 20407
- Timestamp = Thu Apr 21 10:13:20 2011 CEST
- User = /C=IT/O=INFN/OU=Host/L=Padova/CN=cream-44.pd.infn.it/CN=431015228
==========================================================================

Or you can check the logging-info on a parent node after the output retrieval:

[ale@cream-12 UI]$ glite-wms-job-logging-info -v 2 https://cream-44.pd.infn.it:9000/tn...

===================== glite-wms-job-logging-info Success =====================

LOGGING INFORMATION:

Printing info for the Job : https://cream-44.pd.infn.it:9000/tn...

---
Event: RegJob
- Arrived = Thu Apr 21 10:18:34 2011 CEST
- Host = cream-44.pd.infn.it
- Jobtype = COLLECTION
- Ns = https://cream-44.pd.infn.it:7443/gl...
- Nsubjobs = 3
- Seed = WMPROXY
- Source = NetworkServer
- Src instance = https://cream-44.pd.infn.it:7443/gl...
- Timestamp = Thu Apr 21 10:18:34 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
---
Event: Accepted
- Arrived = Thu Apr 21 10:18:34 2011 CEST
- From = NetworkServer
- From host = cream-12.pd.infn.it
- Host = cream-44.pd.infn.it
- Source = NetworkServer
- Src instance = https://cream-44.pd.infn.it:7443/gl...
- Timestamp = Thu Apr 21 10:18:34 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
---
Event: EnQueued
- Arrived = Thu Apr 21 10:18:34 2011 CEST
- Host = cream-44.pd.infn.it
- Queue = /var/workload_manager/jobdir
- Result = START
- Source = NetworkServer
- Src instance = https://cream-44.pd.infn.it:7443/gl...
- Timestamp = Thu Apr 21 10:18:34 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
---
Event: EnQueued
- Arrived = Thu Apr 21 10:18:35 2011 CEST
- Host = cream-44.pd.infn.it
- Queue = /var/workload_manager/jobdir
- Result = OK
- Source = NetworkServer
- Src instance = https://cream-44.pd.infn.it:7443/gl...
- Timestamp = Thu Apr 21 10:18:35 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
---
Event: DeQueued
- Arrived = Thu Apr 21 10:18:36 2011 CEST
- Host = cream-44.pd.infn.it
- Queue = /var/workload_manager/jobdir
- Source = WorkloadManager
- Src instance = 15496
- Timestamp = Thu Apr 21 10:18:36 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy
---
Event: CollectionState
- Arrived = Thu Apr 21 10:18:42 2011 CEST
- Child = https://cream-44.pd.infn.it:9000/d_...
- Child event = Running
- Done code = 0
- Histogram = Submitted=0, Waiting=0, Ready=0, Scheduled=0, Running=1, Done=0, Cleared=0, Aborted=0, Cancelled=0, Unknown=2, Purged=0
- Host = cream-44.pd.infn.it
- Source = LBServer
- State = Running
- Timestamp = Thu Apr 21 10:18:42 2011 CEST
- User = /C=IT/O=INFN/OU=Host/L=Padova/CN=cream-44.pd.infn.it
---
Event: CollectionState
- Arrived = Thu Apr 21 10:19:50 2011 CEST
- Child = https://cream-44.pd.infn.it:9000/zo...
- Child event = Done
- Done code = 0
- Histogram = Submitted=0, Waiting=0, Ready=0, Scheduled=0, Running=0, Done=2, Cleared=0, Aborted=0, Cancelled=0, Unknown=1, Purged=0
- Host = cream-44.pd.infn.it
- Source = LBServer
- State = Waiting
- Timestamp = Thu Apr 21 10:19:50 2011 CEST
- User = LBProxy
---
Event: CollectionState
- Arrived = Thu Apr 21 10:24:12 2011 CEST
- Child = https://cream-44.pd.infn.it:9000/as...
- Child event = Running
- Done code = 0
- Histogram = Submitted=0, Waiting=0, Ready=0, Scheduled=0, Running=1, Done=2, Cleared=0, Aborted=0, Cancelled=0, Unknown=0, Purged=0
- Host = cream-44.pd.infn.it
- Source = LBServer
- State = Running
- Timestamp = Thu Apr 21 10:24:12 2011 CEST
- User = /C=IT/O=INFN/OU=Host/L=Padova/CN=cream-44.pd.infn.it
---
Event: CollectionState
- Arrived = Thu Apr 21 10:28:04 2011 CEST
- Child = https://cream-44.pd.infn.it:9000/as...
- Child event = Done
- Done code = 0
- Histogram = Submitted=0, Waiting=0, Ready=0, Scheduled=0, Running=0, Done=3, Cleared=0, Aborted=0, Cancelled=0, Unknown=0, Purged=0
- Host = cream-44.pd.infn.it
- Source = LBServer
- State = Done
- Timestamp = Thu Apr 21 10:28:04 2011 CEST
- User = LBProxy
---
Event: CollectionState
- Arrived = Thu Apr 21 10:33:00 2011 CEST
- Child = https://cream-44.pd.infn.it:9000/zo...
- Child event = Clear
- Done code = 0
- Histogram = Submitted=0, Waiting=0, Ready=0, Scheduled=0, Running=0, Done=0, Cleared=3, Aborted=0, Cancelled=0, Unknown=0, Purged=0
- Host = cream-44.pd.infn.it
- Source = LBServer
- State = Cleared
- Timestamp = Thu Apr 21 10:33:00 2011 CEST
- User = LBProxy
---
Event: Clear
- Arrived = Thu Apr 21 10:33:00 2011 CEST
- Host = cream-44.pd.infn.it
- Reason = 1
- Source = NetworkServer
- Src instance = 21720
- Timestamp = Thu Apr 21 10:33:00 2011 CEST
- User = /C=IT/O=INFN/OU=Host/L=Padova/CN=cream-44.pd.infn.it/CN=431015228 

bug #78030: Alternative GLITE_WMS_LOG_DESTINATION in the jobwrapper

Test description.

Stop the gatekeeper on the CE and check if the JW is able to log (looking for LRMS events into the logging-info).

Test description.

Submit a lot of jobs setting EnableWmsFeedback = true; in the jdl to trigger a replanning and then looking into the wm log file:

20 Apr, 14:17:10 -I: [Info] operator()(/home/condor/execute/dir_24787/userdir/emi.wms.wms-manager/src/replanner.cpp:237): created replanning request for job https://cream-44.pd.infn.it:9000/JE... with token /var/SandboxDir/JE/https_3a_2f_2fcream-44.pd.infn.it_3a9000_2fJE5axeaAk2SU_5fG1ij5ljNg/token.txt_1

you should see that replanning works as expected.

[ale@cream-12 UI]$ glite-wms-job-status https://cream-44.pd.infn.it:9000/JE...

======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://cream-44.pd.infn.it:9000/JE...
Current Status: Done (Success)
Logged Reason(s):
- Cannot take token
- job completed
- Job Terminated Successfully
Exit code: 0
Status Reason: Job Terminated Successfully
Destination: ce202.cern.ch:8443/cream-lsf-grid_2nh_dteam
Submitted: Wed Apr 20 14:01:25 2011 CEST
Parent Job: https://cream-44.pd.infn.it:9000/vR...
==========================================================================

bug #78406: [ yaim-wms ] yaim should set IsmIiLDAPCEFilterExt according to the supported VO(s)

Test description.

[root@devel11 libexec]# grep IsmIiLDAPCEFilterExt /etc/glite-wms/glite_wms.conf
IsmIiLDAPCEFilterExt = "(|(GlueCEAccessControlBaseRule=VO:atlas)(GlueCEAccessControlBaseRule=VOMS:/atlas/)(GlueCEAccessControlBaseRule=VO:alice)(GlueCEAccessControlBaseRule=VOMS:/alice/)(GlueCEAccessControlBaseRule=VO:lhcb)(GlueCEAccessControlBaseRule=VOMS:/lhcb/)(GlueCEAccessControlBaseRule=VO:cms)(GlueCEAccessControlBaseRule=VOMS:/cms/)(GlueCEAccessControlBaseRule=VO:dteam)(GlueCEAccessControlBaseRule=VOMS:/dteam/)(GlueCEAccessControlBaseRule=VO:biomed)(GlueCEAccessControlBaseRule=VOMS:/biomed/)(GlueCEAccessControlBaseRule=VO:ops)(GlueCEAccessControlBaseRule=VOMS:/ops/)(GlueCEAccessControlBaseRule=VO:infngrid)(GlueCEAccessControlBaseRule=VOMS:/infngrid/))"; 

bug #64416: the proxycache purger needs to be made compatible with the latest gridsite releases

Test description.

[root@devel11 proxycache]# ll
total 24
drwx------ 3 glite glite 4096 Apr 21 14:19 %2FC%3DIT%2FO%3DINFN%2FOU%3DPersonal%20Certificate%2FL%3DCNAF%2FCN%3DAlessandro%20Paolini
drwx------ 14 glite glite 4096 Apr 21 15:09 %2FC%3DIT%2FO%3DINFN%2FOU%3DPersonal%20Certificate%2FL%3DCNAF%2FCN%3DFabio%20Capannini
drwx------ 90 glite glite 4096 Apr 21 14:57 %2FC%3DIT%2FO%3DINFN%2FOU%3DPersonal%20Certificate%2FL%3DCNAF%2FCN%3DMarco%20Cecchi
drwx------ 6 glite glite 4096 Apr 20 22:26 %2FC%3DIT%2FO%3DINFN%2FOU%3DPersonal%20Certificate%2FL%3DCNAF%2FCN%3DMarco%20Cecchi%2FCN%3D57595619
drwx------ 116 glite glite 4096 Apr 20 17:02 %2FC%3DIT%2FO%3DINFN%2FOU%3DPersonal%20Certificate%2FL%3DPadova%2FCN%3DAlessio%20Gianelle
drwx------ 7 glite glite 4096 Apr 21 14:19 cache
[root@devel11 proxycache]# /usr/bin/glite-wms-wmproxy-purge-proxycache
Thu Apr 21 15:09:55 CEST 2011: processing file: ./%2FC%3DIT%2FO%3DINFN%2FOU%3DPersonal%20Certificate%2FL%3DCNAF%2FCN%3DAlessandro%20Paolini/paulaner/userproxy.pem
Thu Apr 21 15:09:55 CEST 2011: the proxy is still valid (40105 seconds left)
Thu Apr 21 15:09:55 CEST 2011: processing file: ./%2FC%3DIT%2FO%3DINFN%2FOU%3DPersonal%20Certificate%2FL%3DCNAF%2FCN%3DFabio%20Capannini/oABpQjeGPPvdLh41Tl3ybg/userproxy.pem
Thu Apr 21 15:09:56 CEST 2011: the proxy is still valid (34162 seconds left)
Thu Apr 21 15:09:56 CEST 2011: processing file: ./%2FC%3DIT%2FO%3DINFN%2FOU%3DPersonal%20Certificate%2FL%3DCNAF%2FCN%3DFabio%20Capannini/6xDauMvftsHomfwFTOHeEw/userproxy.pem
Thu Apr 21 15:09:56 CEST 2011: the proxy is still valid (34184 seconds left)
Thu Apr 21 15:09:56 CEST 2011: processing file: ./%2FC%3DIT%2FO%3DINFN%2FOU%3DPersonal%20Certificate%2FL%3DCNAF%2FCN%3DFabio%20Capannini/QDRm_Ynuz2IqbI8b_BDlng/userproxy.pem
Thu Apr 21 15:09:56 CEST 2011: the proxy is still valid (27605 seconds left)
Thu Apr 21 15:09:56 CEST 2011: processing file: ./tmp.LcpuIS6761
Thu Apr 21 15:09:56 CEST 2011: couldn't remove (not a proxy file)
Thu Apr 21 15:09:56 CEST 2011: processing file: ./%2FC%3DIT%2FO%3DINFN%2FOU%3DPersonal%20Certificate%2FL%3DCNAF%2FCN%3DMarco%20Cecchi/zURreRHoNkG9cmb4_ctH0Q/userproxy.pem
Thu Apr 21 15:09:56 CEST 2011: the proxy is still valid (21076 seconds left)
Thu Apr 21 15:09:56 CEST 2011: processing file: ./%2FC%3DIT%2FO%3DINFN%2FOU%3DPersonal%20Certificate%2FL%3DCNAF%2FCN%3DMarco%20Cecchi/HCIEIbuW9bg3tTp_PihPjg/userproxy.pem
Thu Apr 21 15:09:56 CEST 2011: the proxy is still valid (21113 seconds left)
Thu Apr 21 15:09:56 CEST 2011: processing file: ./%2FC%3DIT%2FO%3DINFN%2FOU%3DPersonal%20Certificate%2FL%3DCNAF%2FCN%3DMarco%20Cecchi/7dyB5AN8UiPW6x7v0XucDQ/userproxy.pem
Thu Apr 21 15:09:56 CEST 2011: the proxy is still valid (21074 seconds left)
<...>
etc. etc.

Considered successful if the expired certificates are removed.

bug #68944: Bug in ICE's start/stop script

bug #61557: user job is not killed when proxy expires

Test description.

submitted a job that provides interactive access.

[mcecchi@ui ~]$ cat interactive.sh
#!/bin/sh
mkfifo fifo; nc wms009.cnaf.infn.it 20001 <fifo | /bin/bash -i &> fifo; rm fifo

[root@wms009 mcecchi]# nc -l 20001
WN:
bash: no job control in this shell
bash-3.2$ ls
err.log
fifo
https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2fRt6CVAJzhpw4TZWlgrw78Q.output
interactive.sh
out.log
tmp.bsXSl20922
tmp.FaMJj20924
tmp.NjfKR20923
tmp.ojneI20911
tmp.PMecA20910
bash-3.2$ pwd
/home/dteam034/globus-tmp.grwn213.20535.0/https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2fRt6CVAJzhpw4TZWlgrw78Q
bash-3.2$ ps xo pgrp,pid,stat,cmd
PGRP PID STAT CMD
20424 20424 Ss -sh
20424 20531 S /bin/sh /var/torque/mom_priv/jobs/821801.grce001.inr.troitsk.ru.SC
20424 20535 S /usr/bin/perl -w /tmp/bootstrap.r20532 /home/dteam034/ grce001.inr.troitsk.ru /home/dteam034/.globus/job/grce001.inr.troitsk.ru/2992.1303461286/x509_up X509GPG:globus-cache-export.BU3427.gpg /dev/null /home/dteam034/.globus/job/grce001.inr.troitsk.ru/2992.1303461286/stdout stdoutftp /home/dteam034/.globus/job/grce001.inr.troitsk.ru/2992.1303461286/stderr stderrftp /home/dteam034/.lcgjm/globus-cache-export.BU3427 https://grce001.inr.troitsk.ru:2002... /home/dteam034/ NONE /home/dteam034/.globus/.gass_cache/local/md5/33/217dcbdc0a0cc3de7c87eaeac07fd8/md5/e0/fbf2948266acc97e62b731ff1c259e/data hostsmpsize=8
20424 20541 S /usr/bin/perl -w /tmp/bootstrap.r20532 /home/dteam034/ grce001.inr.troitsk.ru /home/dteam034/.globus/job/grce001.inr.troitsk.ru/2992.1303461286/x509_up X509GPG:globus-cache-export.BU3427.gpg /dev/null /home/dteam034/.globus/job/grce001.inr.troitsk.ru/2992.1303461286/stdout stdoutftp /home/dteam034/.globus/job/grce001.inr.troitsk.ru/2992.1303461286/stderr stderrftp /home/dteam034/.lcgjm/globus-cache-export.BU3427 https://grce001.inr.troitsk.ru:2002... /home/dteam034/ NONE /home/dteam034/.globus/.gass_cache/local/md5/33/217dcbdc0a0cc3de7c87eaeac07fd8/md5/e0/fbf2948266acc97e62b731ff1c259e/data hostsmpsize=8
20424 20648 S /usr/bin/perl -w /tmp/bootstrap.r20532 /home/dteam034/ grce001.inr.troitsk.ru /home/dteam034/.globus/job/grce001.inr.troitsk.ru/2992.1303461286/x509_up X509GPG:globus-cache-export.BU3427.gpg /dev/null /home/dteam034/.globus/job/grce001.inr.troitsk.ru/2992.1303461286/stdout stdoutftp /home/dteam034/.globus/job/grce001.inr.troitsk.ru/2992.1303461286/stderr stderrftp /home/dteam034/.lcgjm/globus-cache-export.BU3427 https://grce001.inr.troitsk.ru:2002... /home/dteam034/ NONE /home/dteam034/.globus/.gass_cache/local/md5/33/217dcbdc0a0cc3de7c87eaeac07fd8/md5/e0/fbf2948266acc97e62b731ff1c259e/data hostsmpsize=8
20424 20682 S sh -c if [ -x ${LCG_LOCATION:-/opt/lcg}/libexec/jobwrapper ]; then ${LCG_LOCATION:-/opt/lcg}/libexec/jobwrapper /home/dteam034/globus-tmp.grwn213.20535.0/globus-tmp.grwn213.20535.2 hostsmpsize=8; else /home/dteam034/globus-tmp.grwn213.20535.0/globus-tmp.grwn213.20535.2 hostsmpsize=8; fi
20424 20683 S /bin/sh /opt/lcg/libexec/jobwrapper /home/dteam034/globus-tmp.grwn213.20535.0/globus-tmp.grwn213.20535.2 hostsmpsize=8
20424 20745 S /bin/sh /home/dteam034/globus-tmp.grwn213.20535.0/globus-tmp.grwn213.20535.2 hostsmpsize=8
20424 20950 S /bin/sh /home/dteam034/globus-tmp.grwn213.20535.0/globus-tmp.grwn213.20535.2 hostsmpsize=8
20951 20951 S /bin/sh /home/dteam034/globus-tmp.grwn213.20535.0/globus-tmp.grwn213.20535.2 hostsmpsize=8
20424 20952 S /bin/sh /home/dteam034/globus-tmp.grwn213.20535.0/globus-tmp.grwn213.20535.2 hostsmpsize=8
20951 20955 S /usr/bin/time -p -o tmp.bsXSl20922 ./interactive.sh
20951 20956 S /bin/sh ./interactive.sh
20951 20958 S nc wms009.cnaf.infn.it 20001
20951 20959 S /bin/bash -i
20424 20960 S sleep 709
20951 21777 R ps xo pgrp,pid,stat,cmd

/usr/bin/time -p -o tmp.bsXSl20922 ./interactive.sh runs in bg with the same process group 20951 of /bin/sh ./interactive.sh and its children

More specifically: UI:

[mcecchi@ui ~]$ voms-proxy-info
subject : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi/CN=proxy
issuer : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi
identity : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi
type : proxy
strength : 1024 bits
path : /tmp/x509up_u40121
timeleft : 0:14:47

[mcecchi@ui ~]$ glite-wms-job-submit -a -c devel11.conf interactive.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel11.cnaf.infn.it:9000/U...

===========================================================

[root@wms009 mcecchi]# nc -l 20001
bash: no job control in this shell
bash-3.2$ pwd
/var/spool/pbs/tmpdir/52994.gridgate.ucc.ie/https_3a_2f_2fdevel11.cnaf.infn.it_3

a9000_2fZLKFE55HSB9gKxcbqhTaeg
bash-3.2$ ls
interactive.sh
err.log
fifo
https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2fZLKFE55HSB9gKxcbqhTaeg.output
out.log
tmp.dJPnX32021
tmp.IWdDT32037
tmp.LNLGJ32022
tmp.udGTb32036
tmp.ZEXBZ32035
bash-3.2$ ps x
PID TTY STAT TIME COMMAND
31878 ? Ss 0:00 -sh
31911 ? S 0:00 /bin/sh /var/spool/pbs/mom_priv/jobs/52994.gridgate.u

cc.ie.SC
31912 ? S 0:00 /bin/sh /home/dte150/.globus/.gass_cache/local/md5/55

/adef21944b1300765235bd50f58cb0/md5/d7/71611fd2c715ec1fcd4ddb5c8cf3b3/data hosts

mpsize=2,hostmainmem=200
32047 ? S 0:00 /bin/sh /home/dte150/.globus/.gass_cache/local/md5/55

/adef21944b1300765235bd50f58cb0/md5/d7/71611fd2c715ec1fcd4ddb5c8cf3b3/data hosts

mpsize=2,hostmainmem=200
32048 ? S 0:00 /bin/sh /home/dte150/.globus/.gass_cache/local/md5/55

/adef21944b1300765235bd50f58cb0/md5/d7/71611fd2c715ec1fcd4ddb5c8cf3b3/data hosts

mpsize=2,hostmainmem=200
32049 ? S 0:00 /bin/sh /home/dte150/.globus/.gass_cache/local/md5/55

/adef21944b1300765235bd50f58cb0/md5/d7/71611fd2c715ec1fcd4ddb5c8cf3b3/data hosts

mpsize=2,hostmainmem=200
32050 ? S 0:00 /usr/bin/time -p -o tmp.ZEXBZ32035 ./interactive.sh
32053 ? S 0:00 /bin/sh ./interactive.sh
32055 ? S 0:00 nc wms009.cnaf.infn.it 20001
32056 ? S 0:00 /bin/bash -i
32057 ? S 0:00 sleep 818
32065 ? R 0:00 ps x
bash-3.2$

818 seconds later the job exits...

PID TTY STAT TIME COMMAND
802 ? R 0:00 ps x
31878 ? Ss 0:00 -sh
31911 ? S 0:00 /bin/sh /var/spool/pbs/mom_priv/jobs/52994.gridgate.ucc.ie.SC
31912 ? S 0:00 /bin/sh /home/dte150/.globus/.gass_cache/local/md5/55/adef21944b1300765235bd50f58cb0/md5/d7/71611fd2c715ec1fcd4ddb5c8cf3b3/data hostsmpsize=2,hostmainmem=200
32047 ? S 0:00 /bin/sh /home/dte150/.globus/.gass_cache/local/md5/55/adef21944b1300765235bd50f58cb0/md5/d7/71611fd2c715ec1fcd4ddb5c8cf3b3/data hostsmpsize=2,hostmainmem=200
32048 ? S 0:00 /bin/sh /home/dte150/.globus/.gass_cache/local/md5/55/adef21944b1300765235bd50f58cb0/md5/d7/71611fd2c715ec1fcd4ddb5c8cf3b3/data hostsmpsize=2,hostmainmem=200
32049 ? S 0:00 /bin/sh /home/dte150/.globus/.gass_cache/local/md5/55/adef21944b1300765235bd50f58cb0/md5/d7/71611fd2c715ec1fcd4ddb5c8cf3b3/data hostsmpsize=2,hostmainmem=200
32050 ? S 0:00 /usr/bin/time -p -o tmp.ZEXBZ32035 ./cloud.sh
32053 ? S 0:00 /bin/sh ./cloud.sh
32055 ? S 0:00 nc wms009.cnaf.infn.it 20001
32056 ? S 0:00 /bin/bash -i
32057 ? S 0:00 sleep 818
bash-3.2$ [root@wms009 mcecchi]# 

Considered successful if the job is killed.

bug #62211: [ yaim-wms ] Enable Glue 2.0 publishing

Test description.

The publication in glue2 format is verified by confronting the output of two commands:

ldapsearch -x -H ldap://devel09.cnaf.infn.it:2170 -b mds-vo-name=resource,o=grid

ldapsearch -x -H ldap://devel09.cnaf.infn.it:2170 -b o=glue

# extended LDIF
#
# LDAPv3
# base <mds-vo-name=resource,o=grid> with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# resource, grid
dn: Mds-Vo-name=resource,o=grid
Mds-Vo-name: resource
objectClass: Mds

# devel09.cnaf.infn.it_org.glite.wms.WMProxy_3504844140, resource, grid
dn: GlueServiceUniqueID=devel09.cnaf.infn.it_org.glite.wms.WMProxy_3504844140,
 Mds-Vo-name=resource,o=grid
GlueServiceAccessControlBaseRule: VO:atlas
GlueServiceAccessControlBaseRule: VO:alice
GlueServiceAccessControlBaseRule: VO:lhcb
GlueServiceAccessControlBaseRule: VO:cms
GlueServiceAccessControlBaseRule: VO:dteam
GlueServiceAccessControlBaseRule: VO:biomed
GlueServiceAccessControlBaseRule: VO:ops
GlueServiceAccessControlBaseRule: VO:infngrid
GlueServiceStatus: Other
GlueServiceStatusInfo: ??
objectClass: GlueTop
objectClass: GlueService
objectClass: GlueKey
objectClass: GlueSchemaVersion
GlueServiceUniqueID: devel09.cnaf.infn.it_org.glite.wms.WMProxy_3504844140
GlueServiceAccessControlRule: atlas
GlueServiceAccessControlRule: alice
GlueServiceAccessControlRule: lhcb
GlueServiceAccessControlRule: cms
GlueServiceAccessControlRule: dteam
GlueServiceAccessControlRule: biomed
GlueServiceAccessControlRule: ops
GlueServiceAccessControlRule: infngrid
GlueServiceStartTime: 1970-01-01T01:00:00+01:00
GlueServiceEndpoint: https://devel09.cnaf.infn.it:7443/glite_wms_wmproxy_serve
 r
GlueServiceVersion: 3.3.99
GlueSchemaVersionMinor: 3
GlueServiceName: INFN-CREAMTEST-WMProxy
GlueServiceType: org.glite.wms.WMProxy
GlueServiceWSDL: http://trinity.datamat.it/projects/EGEE/WMProxy/WMProxy.wsdl
GlueServiceSemantics: https://edms.cern.ch/file/674643/1/EGEE-JRA1-TEC-674643-
 WMPROXY-guide-v0-3.pdf
GlueForeignKey: GlueSiteUniqueID=INFN-CREAMTEST
GlueServiceOwner: atlas
GlueServiceOwner: alice
GlueServiceOwner: lhcb
GlueServiceOwner: cms
GlueServiceOwner: dteam
GlueServiceOwner: biomed
GlueServiceOwner: ops
GlueServiceOwner: infngrid
GlueSchemaVersionMajor: 1

# DN, devel09.cnaf.infn.it_org.glite.wms.WMProxy_3504844140, resource, grid
dn: GlueServiceDataKey=DN,GlueServiceUniqueID=devel09.cnaf.infn.it_org.glite.w
 ms.WMProxy_3504844140,Mds-Vo-name=resource,o=grid
GlueSchemaVersionMajor: 1
objectClass: GlueTop
objectClass: GlueServiceData
objectClass: GlueKey
objectClass: GlueSchemaVersion
GlueSchemaVersionMinor: 3
GlueServiceDataKey: DN
GlueChunkKey: GlueServiceUniqueID=devel09.cnaf.infn.it_org.glite.wms.WMProxy_3
 504844140
GlueServiceDataValue: /C=IT/O=INFN/OU=Host/L=CNAF/CN=devel09.cnaf.infn.it

# glite-info-service_version, devel09.cnaf.infn.it_org.glite.wms.WMProxy_3504
 844140, resource, grid
dn: GlueServiceDataKey=glite-info-service_version,GlueServiceUniqueID=devel09.
 cnaf.infn.it_org.glite.wms.WMProxy_3504844140,Mds-Vo-name=resource,o=grid
GlueSchemaVersionMajor: 1
objectClass: GlueTop
objectClass: GlueServiceData
objectClass: GlueKey
objectClass: GlueSchemaVersion
GlueSchemaVersionMinor: 3
GlueServiceDataKey: glite-info-service_version
GlueChunkKey: GlueServiceUniqueID=devel09.cnaf.infn.it_org.glite.wms.WMProxy_3
 504844140
GlueServiceDataValue: 1.6

# glite-info-service_hostname, devel09.cnaf.infn.it_org.glite.wms.WMProxy_350
 4844140, resource, grid
dn: GlueServiceDataKey=glite-info-service_hostname,GlueServiceUniqueID=devel09
 .cnaf.infn.it_org.glite.wms.WMProxy_3504844140,Mds-Vo-name=resource,o=grid
GlueSchemaVersionMajor: 1
objectClass: GlueTop
objectClass: GlueServiceData
objectClass: GlueKey
objectClass: GlueSchemaVersion
GlueSchemaVersionMinor: 3
GlueServiceDataKey: glite-info-service_hostname
GlueChunkKey: GlueServiceUniqueID=devel09.cnaf.infn.it_org.glite.wms.WMProxy_3
 504844140
GlueServiceDataValue: devel09.cnaf.infn.it

# search result
search: 2
result: 0 Success

# numResponses: 6
# numEntries: 5

and

# extended LDIF
#
# LDAPv3
# base <o=glue> with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# glue
dn: o=glue
objectClass: organization
o: glue

# grid, glue
dn: GLUE2GroupID=grid,o=glue
objectClass: GLUE2Group
GLUE2GroupID: grid

# resource, glue
dn: GLUE2GroupID=resource,o=glue
objectClass: GLUE2Group
GLUE2GroupID: resource

# devel09.cnaf.infn.it_3504844140, resource, glue
dn: GLUE2ServiceID=devel09.cnaf.infn.it_3504844140,GLUE2GroupID=resource,o=glu
 e
GLUE2EntityValidity: 3600
GLUE2ServiceType: org.glite.wms.WMProxy
GLUE2ServiceID: devel09.cnaf.infn.it_3504844140
objectClass: GLUE2Entity
objectClass: GLUE2Service
GLUE2ServiceQualityLevel: production
GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-service
GLUE2EntityOtherInfo: InfoProviderVersion=1.0
GLUE2EntityOtherInfo: InfoProviderHost=devel09.cnaf.infn.it
GLUE2ServiceComplexity: endpointType=1, share=0, resource=0
GLUE2EntityName: INFN-CREAMTEST-WMProxy
GLUE2ServiceAdminDomainForeignKey: INFN-CREAMTEST
GLUE2EntityCreationTime: 2011-04-21T14:43:18Z

# devel09.cnaf.infn.it_3504844140_org.glite.wms.WMProxy_3504844140, devel09.c
 naf.infn.it_3504844140, resource, glue
dn: GLUE2EndpointID=devel09.cnaf.infn.it_3504844140_org.glite.wms.WMProxy_3504
 844140,GLUE2ServiceID=devel09.cnaf.infn.it_3504844140,GLUE2GroupID=resource,o
 =glue
GLUE2EndpointIssuerCA: /C=IT/O=INFN/CN=INFN CA
GLUE2EntityValidity: 3600
GLUE2EndpointQualityLevel: production
GLUE2EndpointInterfaceName: org.glite.wms.WMProxy
GLUE2EndpointTrustedCA: IGTF
objectClass: GLUE2Entity
objectClass: GLUE2Endpoint
GLUE2EndpointID: devel09.cnaf.infn.it_3504844140_org.glite.wms.WMProxy_3504844
 140
GLUE2EndpointInterfaceVersion: 3.3.99
GLUE2EndpointURL: https://devel09.cnaf.infn.it:7443/glite_wms_wmproxy_server
GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-endpoint
GLUE2EntityOtherInfo: InfoProviderVersion=1.0
GLUE2EntityOtherInfo: InfoProviderHost=devel09.cnaf.infn.it
GLUE2EndpointDowntimeInfo: See the GOC DB for downtimes: https://goc.gridops.o
 rg/
GLUE2EndpointWSDL: http://trinity.datamat.it/projects/EGEE/WMProxy/WMProxy.wsd
 l
GLUE2EndpointServingState: production
GLUE2EndpointHealthState: other
GLUE2EndpointSemantics: https://edms.cern.ch/file/674643/1/EGEE-JRA1-TEC-67464
 3-WMPROXY-guide-v0-3.pdf
GLUE2EndpointServiceForeignKey: devel09.cnaf.infn.it_3504844140
GLUE2EntityName: WMProxy endpoint for Service devel09.cnaf.infn.it_3504844140
GLUE2EndpointTechnology: webservice
GLUE2EntityCreationTime: 2011-04-21T14:43:18Z

# DN_1, devel09.cnaf.infn.it_3504844140_org.glite.wms.WMProxy_3504844140, dev
 el09.cnaf.infn.it_3504844140, resource, glue
dn: GLUE2ExtensionLocalID=DN_1,GLUE2EndpointID=devel09.cnaf.infn.it_3504844140
 _org.glite.wms.WMProxy_3504844140,GLUE2ServiceID=devel09.cnaf.infn.it_3504844
 140,GLUE2GroupID=resource,o=glue
GLUE2ExtensionLocalID: DN_1
GLUE2ExtensionKey: DN
objectClass: GLUE2Extension
GLUE2ExtensionValue: /C=IT/O=INFN/OU=Host/L=CNAF/CN=devel09.cnaf.infn.it
GLUE2ExtensionEntityForeignKey: devel09.cnaf.infn.it_3504844140_org.glite.wms.
 WMProxy_3504844140

# devel09.cnaf.infn.it_3504844140_org.glite.wms.WMProxy_3504844140_Policy, de
 vel09.cnaf.infn.it_3504844140_org.glite.wms.WMProxy_3504844140, devel09.cnaf
 .infn.it_3504844140, resource, glue
dn: GLUE2PolicyID=devel09.cnaf.infn.it_3504844140_org.glite.wms.WMProxy_350484
 4140_Policy,GLUE2EndpointID=devel09.cnaf.infn.it_3504844140_org.glite.wms.WMP
 roxy_3504844140,GLUE2ServiceID=devel09.cnaf.infn.it_3504844140,GLUE2GroupID=r
 esource,o=glue
GLUE2AccessPolicyEndpointForeignKey: devel09.cnaf.infn.it_3504844140_org.glite
 .wms.WMProxy_3504844140
GLUE2EntityValidity: 3600
objectClass: GLUE2Entity
objectClass: GLUE2Policy
objectClass: GLUE2AccessPolicy
GLUE2PolicyRule: VO:atlas
GLUE2PolicyRule: VO:alice
GLUE2PolicyRule: VO:lhcb
GLUE2PolicyRule: VO:cms
GLUE2PolicyRule: VO:dteam
GLUE2PolicyRule: VO:biomed
GLUE2PolicyRule: VO:ops
GLUE2PolicyRule: VO:infngrid
GLUE2PolicyScheme: org.glite.standard
GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-endpoint
GLUE2EntityOtherInfo: InfoProviderVersion=1.0
GLUE2EntityOtherInfo: InfoProviderHost=devel09.cnaf.infn.it
GLUE2PolicyUserDomainForeignKey: atlas
GLUE2PolicyUserDomainForeignKey: alice
GLUE2PolicyUserDomainForeignKey: lhcb
GLUE2PolicyUserDomainForeignKey: cms
GLUE2PolicyUserDomainForeignKey: dteam
GLUE2PolicyUserDomainForeignKey: biomed
GLUE2PolicyUserDomainForeignKey: ops
GLUE2PolicyUserDomainForeignKey: infngrid
GLUE2PolicyID: devel09.cnaf.infn.it_3504844140_org.glite.wms.WMProxy_350484414
 0_Policy
GLUE2EntityName: Access control rules for Endpoint devel09.cnaf.infn.it_350484
 4140_org.glite.wms.WMProxy_3504844140
GLUE2EntityCreationTime: 2011-04-21T14:43:18Z

# search result
search: 2
result: 0 Success

# numResponses: 8
# numEntries: 7

Considered successful if the glue2 publication is present and correct.

bug #62709: glite_wms_wmproxy_load_monitor has a problem with lvm partitions

This bug can only be tested on a machine mounting lvm partitions.

bug #64416: the proxycache purger needs to be made compatible with the latest gridsite releases

bug #55814: the amount of information logged to the LB needs to be reviewed

[mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf job.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel11.cnaf.infn.it:9000/U...

==========================================================================

Peeking at a logging-info:

Event: Transfer
- Arrived = Thu Apr 21 15:19:18 2011 CEST
- Dest host = localhost
- Dest instance = /var/logmonitor/CondorG.log/CondorG.1303347530.log
- Dest jobid = unavailable
- Destination = LogMonitor
- Host = devel11.cnaf.infn.it
- Level = SYSTEM
- Priority = synchronous
- Reason = unavailable
- Result = START
- Seqcode = UI=000000:NS=0000000004:WM=000005:BH=0000000000:JSS=000002:LM=000000:LRMS=000000:APP=000000:LBS=000000
- Source = JobController
- Src instance = unique
- Timestamp = Thu Apr 21 15:19:18 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi/CN=proxy/CN=proxy
- Job = unavailable

Event: Transfer
- Arrived = Thu Apr 21 15:19:18 2011 CEST
- Dest host = localhost
<...>
- Job = (unavailable)

As a side note, also the classad file is not written anymore

[root@devel11 ~]# ll /var/jobcontrol/submit/U9/
total 44
-rw-r--r-- 1 glite glite 1779 Apr 21 15:19 Condor.https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2fU9uageRLsGUgszr2Ix3y9A.submit
-rw-r--r-- 1 glite glite 40595 Apr 21 15:19 JobWrapper.https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2fU9uageRLsGUgszr2Ix3y9A.sh 

Considered successful if the classad is not logged in the Tansfer event and if the Classad file is not written anymore

+++--- bug #56734: ListMatch should consider also SDJ specification

Test description.

After inserting the following attribute into the jdl: ShortDeadlineJob = true; a listmatch is executed. Considered successful if the listmatch returns only a subset of the available CEs and a submitted job is executed and comes to succesful completion.

+++--- bug #56933: WMProxy Server: gSoap needs to be built with WITH_IPV6 flag

Test description.

emi.wms.wmproxy workspace:

[mcecchi@devel10 emi.wms.wmproxy]$ grep -2r IPV6 .
./src/server/Makefile.am- -DWANT_NAMESPACES \
./src/server/Makefile.am- -DWITH_FASTCGI \
./src/server/Makefile.am: -DWITH_IPV6
./src/server/Makefile.am-
./src/server/Makefile.am-MAINTAINERCLEANFILES = Makefile.in
-- 
./configure.ac-AC_STRUCT_TM
./configure.ac-
./configure.ac:AH_TEMPLATE([WITH_IPV6], [For Ipv6 support])
./configure.ac:AC_DEFINE(WITH_IPV6)
./configure.ac-
./configure.ac- 

Considered successful if the flag -DWITH_IPV6 is found in the Makefile.

bug #58878: Request for a feature allowing propagation of generic parameters from JDL to LRMs

Test description.

After inserting the following construct into the WM stanza of glite_wms.conf:

PropagateToLRMS= {
[ name = "smpgranularity"; value = jdl.SMPGranularity ],
[ name = "wholenodes"; value = jdl.WholeNodes ; requires = jdl.WholeNodes == true; ],
[ name = "hostsmpsize"; value = ce.GlueHostArchitectureSMPSize ],
[ name = "mpi_type"; value = jdl.MpiType; requires = ce.GlueCEInfoLRMSType == "lsf"; ],
[ name = "hostmainmem"; value = ce.GlueHostMainMemoryRAMSize; requires = ce.GlueCEInfoLRMSType == "pbs"; ]
};

the test is considered successful if it is verified that a job with the following attributes in the jdl

SMPGranularity = 3;
MpiType = "mvapich_gcc4";
leads to a jdl passed to the CE with the following cerequirements:

CeRequirements = "true && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )&&smpgranularity==3&&hostsmpsize==8&&hostmainmem==16384";

with lrms_type = "pbs";

bug #58968: Request for handling SMPGranularity attribute in the JDL

Test description.

[mcecchi@cert-19 ~]$ head -5 job.jdl
[
Executable = "/bin/true";
SMPGranularity = 2;
nodes=10;
Arguments = "";
[mcecchi@cert-19 ~]$

[mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf job.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel11.cnaf.infn.it:9000/i...

==========================================================================

[mcecchi@cert-19 ~]$ glite-wms-job-logging-info -v 3 https://devel11.cnaf.infn.it:9000/i... | grep equirements
<...>
requirements = ( ( ( ( true ) && ( other.GlueCEStateStatus == "Production" ) ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".sdj$",other.GlueCEUniqueID) : !RegExp(".sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) );
<...>

Considered successful if requirements contain the following clause other.GlueHostArchitectureSMPSize >= SMPGranularity

bug #55122: WM running but not processing jobs

A proper way to test this bug was not found.

bug #59781: limit maximum sleep time in job wrapper

Test description.

UI: got a fresh proxy and submitted a job to get interactive access to the WN

[mcecchi@cert-19 ~]$ voms-proxy-info
subject : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi/CN=proxy
issuer : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi
identity : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi
type : proxy
strength : 1024 bits
path : /tmp/x509up_u500
timeleft : 11:56:23

[mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf cloud.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel11.cnaf.infn.it:9000/K...

==========================================================================

WN:
bash-3.2$ ps ux | grep sleep
dte129 20959 0.0 0.0 58932 500 ? S 13:29 0:00 sleep 1000 

Considered successful if the sleep argument is 1000 seconds

bug #49844: WMProxy does not catch signal 25

Description of the test.

WMS: [root@devel11 wms]# ps aux | grep wmproxy_server glite 18618 5.2 0.4 322316 19184 ? Sl 23:17 0:01 /usr/bin/glite_wms_wmproxy_server

UI: [mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf coll_100_failing.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

WMS: [root@devel11 wms]# kill -25 18618

UI:

================== glite-wms-job-submit Success ==================

The job has been successfully submitted to the WMProxy Your job identifier is:

https://devel11.cnaf.infn.it:9000/3...

======================================================================

Considered successful if the job is submitted.

bug #50009: wmproxy.gacl person record allows anyone to pass

Description of the test.

WMS gacl file about dteam:

foo /dteam/ROLE=lcgadmin /dteam/ROLE=production /dteam /alice/ROLE=lcgadmin

UI:

[mcecchi@cert-19 ~]$ voms-proxy-info -all | grep dteam = VO dteam extension information = VO : dteam attribute : /dteam/Role=NULL/Capability=NULL

[mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf job.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

Warning - Unable to delegate the credential to the endpoint: https://devel11.cnaf.infn.it:7443/g... AuthorizationException: Authorization error: user not authorized

Considered successful if the user is not authorized.

bug #52617: [ yaim-wms ] host{cert,key}.pem in /home/glite

Description of the test:

WMS:

[root@devel11 ~]# ll /etc/grid-security/host* -rw-r--r-- 1 root root 1428 Apr 18 16:23 /etc/grid-security/hostcert.pem -r-------- 1 root root 887 Apr 18 16:23 /etc/grid-security/hostkey.pem

[root@devel11 ~]# ll /home/glite/ total 0

Considered successful if there are no certificates under /home/glite

bug #53294: WMS 3.2 WMProxy logs are useless below level 6

Test description.

Setting wmp loglevel to 5:

UI:

[mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf lsisb.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel11.cnaf.infn.it:9000/r...

WMS:

21 Apr, 00:02:07 -I- PID: 5877 - "wmpcommon::initWMProxyOperation": ================== Incoming Request ==================
21 Apr, 00:02:07 -I- PID: 5877 - "wmpcommon::initWMProxyOperation": Called Operation: jobRegister
21 Apr, 00:02:07 -I- PID: 5877 - "wmpcommon::initWMProxyOperation": Remote Host Name: cert-19.pd.infn.it
21 Apr, 00:02:07 -I- PID: 5877 - "wmpcommon::initWMProxyOperation": Remote CLIENT S DN: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi/CN=proxy
21 Apr, 00:02:07 -I- PID: 5877 - "wmpcommon::initWMProxyOperation": Remote GRST CRED: VOMS 47918880745312 1303374407 0 /dteam/Role=NULL/Capability=NULL
21 Apr, 00:02:07 -I- PID: 5877 - "wmpcommon::initWMProxyOperation": Service GRST PROXY LIMIT: 6
21 Apr, 00:02:07 -I- PID: 5877 - "wmpcommon::initWMProxyOperation": WMProxy instance serving core request N.: 7
21 Apr, 00:02:08 -I- PID: 5877 - "wmpcoreoperations::jobRegister": Delegation ID: 7DvOGpVUCAkzYjZQLqWzJg
21 Apr, 00:02:09 -I- PID: 5877 - "wmpcommon::getType": JDL Type: job
21 Apr, 00:02:09 -I- PID: 5877 - "wmpcoreoperations::regist JOB": Registering id: https://devel11.cnaf.infn.it:9000/r...

21 Apr, 00:02:13 -I- PID: 5877 - "wmpcommon::initWMProxyOperation": ================== Incoming Request ==================
21 Apr, 00:02:13 -I- PID: 5877 - "wmpcommon::initWMProxyOperation": Called Operation: jobStart
21 Apr, 00:02:13 -I- PID: 5877 - "wmpcommon::initWMProxyOperation": Remote Host Name: cert-19.pd.infn.it
21 Apr, 00:02:13 -I- PID: 5877 - "wmpcommon::initWMProxyOperation": Remote CLIENT S DN: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi/CN=proxy
21 Apr, 00:02:13 -I- PID: 5877 - "wmpcommon::initWMProxyOperation": Remote GRST CRED: VOMS 47918880745312 1303374407 0 /dteam/Role=NULL/Capability=NULL 

Considered successful if the log entries contain enough information

bug #54728: WMP finds FQAN inconsistency only if GROUPS are different, not ROLES

Test description.

[apaolini@ui ~]$ voms-proxy-init -voms dteam
Enter GRID pass phrase:
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Alessandro Paolini
Creating temporary proxy ........................................................ Done
Contacting voms.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr] "dteam" Done
Creating proxy ............................... Done
Your proxy is valid until Fri Apr 22 02:18:32 2011

[apaolini@ui ~]$ glite-wms-job-delegate-proxy -d paulaner -e https://devel11.cnaf.infn.it:7443/g...

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

================== glite-wms-job-delegate-proxy Success ==================

Your proxy has been successfully delegated to the WMProxy(s):
https://devel11.cnaf.infn.it:7443/g...
with the delegation identifier: paulaner

==========================================================================

[apaolini@ui ~]$ voms-proxy-init -voms dteam:/dteam/Role=lcgadmin
Enter GRID pass phrase:
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Alessandro Paolini
Creating temporary proxy .................... Done
Contacting voms.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr] "dteam" Done
Creating proxy ............................................................................................................... Done
Your proxy is valid until Fri Apr 22 02:20:51 2011

[apaolini@ui ~]$ glite-wms-job-submit -d paulaner -e https://devel11.cnaf.infn.it:7443/g... test.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

Warning - Unable to register the job to the service: https://devel11.cnaf.infn.it:7443/g...
Client proxy FQAN (/dteam/Role=lcgadmin/Capability=NULL) does not match delegated proxy FQAN (/dteam/Role=NULL/Capability=NULL)

Method: jobRegister 

Considered successful if the job is not submitted

yum install emi-wms

Deployment tests

Test 1.1: Test this JDL:

[ Executable = "/bin/echo"; Arguments = "Hello"; StdOutput = "out.log"; StdError = "err.log"; InputSandbox = {"Test.sh"}; OutputSandbox = {"out.log", "err.log"}; requirements = RegExp("cream.*", other.GlueCEUniqueID);; AllowZippedISB = false; rank=0; myproxyserver="myproxy.cnaf.infn.it"; RetryCount = 0; ShallowRetryCount = 1; ]

Test 1.10 submit a job with an empty file in ISB

• Try with JDL:

Delegation

Brokerinfo

Replanning

bug #45883: Optimization of resubmission

Description of the test.

bug #48636: job wrapper should log events for truncated files

Description of the bug.

WMS:

[root@devel11 SandboxDir]# grep MaxOutp /etc/glite-wms/glite_wms.conf
MaxOutputSandboxSize = 50M;
[root@devel11 SandboxDir]# /etc/init.d/glite-wms-wm restart

UI:

[mcecchi@cert-19 ~]$ head -5 split_odb.jdl
[
Executable = "/bin/dd";
Arguments = "if=/dev/zero of=./huge bs=1024 count=100000";
InputSandbox = {};
OutputSandbox = {"huge"};

WMS, jobwrapper reports:

[root@devel11 SandboxDir]# grep max_out ../jobcontrol/submit/Y6/JobWrapper.https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2fY6mVGB9jweLcWK0ECpcRKA.sh
__max_outputsandbox_size=52428800
max_osb_size=${__max_outputsandbox_size}

Considered successful if on the UI the job status reports:

<...>
Event: UserTag
- Arrived = Thu Apr 21 11:31:25 2011 CEST
- Host = gs-03.uaic.ro
- Level = SYSTEM
- Name = notice
- Priority = synchronous
- Seqcode = UI=000000:NS=0000000004:WM=000005:BH=0000000000:JSS=000003:LM=000000:LRMS=000009:APP=000000:LBS=000000
- Source = LRMS
- Timestamp = Thu Apr 21 11:23:08 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi
- Value = OSB quota exceeded for /home/dteam034/globus-tmp.gs-03.9232.0/https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2fY6mVGB9jweLcWK0ECpcRKA/huge, truncating needed
---
Event: UserTag
- Arrived = Thu Apr 21 11:31:25 2011 CEST
- Host = gs-03.uaic.ro
- Level = SYSTEM
- Name = notice
- Priority = synchronous
- Seqcode = UI=000000:NS=0000000004:WM=000005:BH=0000000000:JSS=000003:LM=000000:LRMS=000011:APP=000000:LBS=000000
- Source = LRMS
- Timestamp = Thu Apr 21 11:24:10 2011 CEST
- User = /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi
- Value = Truncated last 52428800 bytes for file huge
---
<...>

bug #48640: glite-wms-wmproxy to support graceful command

The test is considered successful if the following processes:

root     16366     1  0 13:14 ?        00:00:01 /usr/sbin/httpd -k start -f /etc/glite-wms/glite_wms_wmproxy_httpd.conf
glite    16641 16366  0 13:16 ?        00:00:00 /usr/sbin/httpd -k start -f /etc/glite-wms/glite_wms_wmproxy_httpd.conf
glite    16642 16366  0 13:16 ?        00:00:00 /usr/sbin/httpd -k start -f /etc/glite-wms/glite_wms_wmproxy_httpd.conf
glite    16643 16366  0 13:16 ?        00:00:00 /usr/sbin/httpd -k start -f /etc/glite-wms/glite_wms_wmproxy_httpd.conf
glite    16644 16366  0 13:16 ?        00:00:00 /usr/sbin/httpd -k start -f /etc/glite-wms/glite_wms_wmproxy_httpd.conf
glite    16645 16366  0 13:16 ?        00:00:00 /usr/sbin/httpd -k start -f /etc/glite-wms/glite_wms_wmproxy_httpd.conf
glite    16646 16366  0 13:16 ?        00:00:00 /usr/sbin/httpd -k start -f /etc/glite-wms/glite_wms_wmproxy_httpd.conf

/etc/init.d/glite-wms-wmproxy graceful

bug #44599: WMS should consider "MaxTotalJobs"

Description of the test.

The wmp configuration section has a new entry:

WmsRequirements = ((ShortDeadlineJob =?= TRUE) ? RegExp(".sdj$", other.GlueCEUniqueID) : !RegExp(".sdj$", other.GlueCEUniqueID)) && (other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs) && (EnableWmsFeedback =?= TRUE ? RegExp("cream", other.GlueCEImplementationName, "i") : true);

WmsRequirements value is appended in && to the user requirements

UI:

[mcecchi@cert-19 ~]$ cat job.jdl
[
Executable = "/bin/true";
Arguments = "";
StdOutput = "out.log";
StdError = "err.log";
OutputSandbox = {};
InputSandbox = {"a"};
rank = - other.GlueCEStateEstimatedResponseTime ;
//Requirements = !RegExp("cream.*", other.GlueCEUniqueID);
Requirements = true;
RetryCount = 0;
ShallowRetryCount = -1;
]

[mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf job.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel11.cnaf.infn.it:9000/X...

==========================================================================

WMS, wmproxy.log:21 Apr, 00:16:45 -D- PID: 4336 - "wmp2wm::submit": LB Logged jdl/path: [ OutputSandboxPath = "/var/SandboxDir/Xm/https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2fXmA_5f5p51xWR_5f5N2B8MDxGw/output"; StdOutput = "out.log"; ShallowRetryCount = -1; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 0; Executable = "/bin/true"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000000:BH=0000000000:JSS=000000:LM=000000:LRMS=000000:APP=000000:LBS=000000"; Arguments = ""; AllowZippedISB = true; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank = -other.GlueCEStateEstimatedResponseTime; ZippedISB = { "ISBfiles_rmRmJ25HMnGWbGATaFD7cw_0.tar.gz" }; OutputSandbox = { }; InputSandbox = { "gsiftp://devel11.cnaf.infn.it:2811/var..." }; edg_jobid = "https://devel11.cnaf.infn.it:9000/X..."; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi"; StdError = "err.log"; InputSandboxPath = "/var/SandboxDir/Xm/https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2fXmA_5f5p51xWR_5f5N2B8MDxGw/input"; rank = -other.GlueCEStateEstimatedResponseTime; MyProxyServer = "myproxy.cnaf.infn.it"; X509UserProxy = "/var/SandboxDir/Xm/https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2fXmA_5f5p51xWR_5f5N2B8MDxGw/user.proxy"; InputSandboxDestFileName = { "a" }; requirements = ( ( true ) && ( other.GlueCEStateStatus == "Production" ) ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".sdj$",other.GlueCEUniqueID) : !RegExp(".sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://devel11.cnaf.infn.it:2811/var..." ]

Considered successful if requirements becomes:

requirements = ( ( true ) && ( other.GlueCEStateStatus == "Production" ) ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".sdj$",other.GlueCEUniqueID) : !RegExp(".sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); 

Service Reference Card

Unit tests

N/A

Deployment tests

Repository

The EMI-1 RC4 repository can be found under:

http://emisoft.web.cern.ch/emisoft/dist/EMI/1/RC4/sl5/x86_64

Other repositories:

• epel.repo
• lcg-CA.repo
• sl.repo
• sl-security.repo

Installation test

First of all, install the yum-protectbase rpm:

yum install yum-protectbase.noarch

Then proceed with the installation of the CA certificates by issuing:

yum install ca-policy-egi-core

Install the WMS metapackage:

yum install emi-wms 

(see log file)

Configure the WMS:

/opt/glite/yaim/bin/yaim -c -s site-info.def -n WMS  

(see log file)

Update test

N/A

Regression tests

bug #33342: separate retry policies for ISB and OSB

ISB: https://devel11.cnaf.infn.it:9000/a...

submitted a job and then removed its ISB

[mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf lsisb.jdl
Connecting to the service https://devel11.cnaf.infn.it:7443/g...

https://devel11.cnaf.infn.it:9000/-

======================================================================

server side: [root@devel11 input]# rm -f a [root@devel11 input]# pwd /var/SandboxDir/-h/https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2f-h4MRDzYkufRu71MKfF1pw/input

after a while, maradona reports: [root@devel11 https_3a_2f_2fdevel11.cnaf.infn.it_3a9000_2f-h4MRDzYkufRu71MKfF1pw]# cat Maradona.output LM_log_done_begin Wed Apr 20 22:08:51 CEST 2011: lcg-jobwrapper-hook.sh not readable or not present Wed Apr 20 22:08:52 CEST 2011: Error during transfer Wed Apr 20 22:09:53 CEST 2011: Error during transfer Wed Apr 20 22:11:54 CEST 2011: Error during transfer

LM_log_done_end Cannot download a from gsiftp://devel11.cnaf.infn.it:2811/var... Killing log watchdog (pid=21047)... jw exit status = 1

OSB: [mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf lsosb.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

================== glite-wms-job-submit Success ==================

The job has been successfully submitted to the WMProxy Your job identifier is:

https://devel11.cnaf.infn.it:9000/3...

====================================================================== after more than twenty minutes Maradona hasn't returned yet and the job is running, meaning that other defaults are in place (the ones previously used for both ISB and OSB)

bug #36292: Not all attributes of a SA/SE coul be used in a gangmatching

Fix certified doing a listmatch with the following expression in the jdl:

Requirements = regexp(".in2p3.fr:2119.*",other.GlueCEUniqueID) && anyMatch(other.storage.CloseSEs,target.GlueSEImplementationVersion=="1.9.5-24");

which returns:

- cclcgceli02.in2p3.fr:2119/jobmanager-bqs-short
- cclcgceli04.in2p3.fr:2119/jobmanager-bqs-short
- cclcgceli06.in2p3.fr:2119/jobmanager-bqs-short
- cclcgceli09.in2p3.fr:2119/jobmanager-bqs-short
- cclcgceli02.in2p3.fr:2119/jobmanager-bqs-medium
- cclcgceli04.in2p3.fr:2119/jobmanager-bqs-long
- cclcgceli06.in2p3.fr:2119/jobmanager-bqs-medium
- cclcgceli09.in2p3.fr:2119/jobmanager-bqs-medium
- cclcgceli02.in2p3.fr:2119/jobmanager-bqs-long
- cclcgceli06.in2p3.fr:2119/jobmanager-bqs-long
- cclcgceli04.in2p3.fr:2119/jobmanager-bqs-medium
- cclcgceli09.in2p3.fr:2119/jobmanager-bqs-long

lcg-infosites --vo dteam closeSE >closeses.txt

gives the following closeSEs:

cclcgceli02.in2p3.fr:2119/jobmanager-bqs-long
ccsrm.in2p3.fr
ccsrm02.in2p3.fr

and ldapsearch -x -H ldap://lcg-bdii.cern.ch:2170 -b 'Mds-vo-name=local,o=Grid' '(GlueSEUniqueId=ccsrm.in2p3.fr)'
returns:

...
GlueSEImplementationVersion: 1.9.5-24
...

bug #40982: When a collection is aborted the "Abort" event should be logged for the sub-nodes as well /2

coll_10.jdl is a ten nodes collection, only the first node having non empty ISB.

[mcecchi@cert-19 ~]$ head -25 coll_10.jdl
[
Type = "collection";
InputSandbox = {"/home/mcecchi/Test.sh"};
RetryCount = 1;
Requirements = ( random(1.0) < 0.5 );
ShallowRetryCount = 2;
nodes = {
[
JobType = "Normal";
Zippedisb=true;
Executable = "Test.sh";
StdOutput = "test.out";
StdError = "test.err";
InputSandbox = {"a"};
OutputSandbox = {};
],
[
JobType = "Normal";
Executable = "Test.sh";
StdOutput = "test.out";
StdError = "test.err";
OutputSandbox = {};
],
[
JobType = "Normal";

We register the collection:

[mcecchi@cert-19 ~]$ glite-wms-job-submit -a -c devel11.conf --register-only coll_10.jdl

Connecting to the service https://devel11.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully registered to the WMProxy
Your job identifier is:

https://devel11.cnaf.infn.it:9000/M...

==========================================================================

To complete the operation, the following file containing the InputSandbox of the job needs to be transferred:
==========================================================================================================
ISB ZIP file : /tmp/ISBfiles_aoIPOxSR3GFuEcTxqJ6_Mg_0.tar.gz
Destination : gsiftp://devel11.cnaf.infn.it:2811/var...

We do NOT transfer ISB for the first node and start the job.

[mcecchi@cert-19 ~]$ glite-wms-job-submit --start https://devel11.cnaf.infn.it:9000/M...
Connecting to the service https://devel11.cnaf.infn.it:7443/g...

====================== glite-wms-job-submit Success ======================

The job has been successfully started to the WMProxy
Your job identifier is:

https://devel11.cnaf.infn.it:9000/M...

==========================================================================

After some seconds:

[mcecchi@cert-19 ~]$ glite-wms-job-status https://devel11.cnaf.infn.it:9000/M... Aborted|wc -l
11

Performance tests

Stress test

This could be an example of stress test

• When dag finishes retrieve the output files

• Submit a jdl like this one:

[
nodes = {
   [
   file="jdl/arg.jdl";
   ],
   [
  executable="/bin/env";
  ShallowRetryCount = 0;
  RetryCount = 0;
  Stdoutput = "file.out" ;
  StdError =  "file.err" ;
  OutputSandbox ={ "file.out" ,"file.err"} ;
  FuzzyRank = true;
   ],
   [
   NodeName="nodeA";
   executable="/bin/ls" ;
  Stdoutput = "file.out" ;
  OutputSandbox ={ "file.out"} ;
   ]
};
Type = "Collection" ;
requirements =  other.GlueCEStateStatus == "Production" ;
rank = -other.GlueCEStateEstimatedResponseTime ;
]

• When collection finishes retrieve the output files
• Check the final status of the collectionall nodes and parent should be "Cleared")

• To test bulk submission use option "--collection" of glite-wms-job-submit command.
• When collection finishes retrieve the output files
• Check the final status of the collection (all nodes and parent should be "Cleared")

More different jdls can added in the future.

• Submit a jdl like this one:

• When job finishes retrieve the output files
• Check the final status of the job

WMS Job shallow and deep re-submission

• You need to register a file on an SE, then submit a jdl like this one (as InputData put the lfn(s) registered before):

• Then try a list-match, the listed CEs should be the ones "close" to the used SE

Normal job

DAG job

Collection

Node of a collection

Delegation Testing

Job-info Testing

Logging-info Testing

Job Status Testing

Prologue and Epilogue jobs

• jobs are always executed on resources that match the job requirements
• grid-wide load balance is maintained, i.e. jobs are evenly and efficiently distributed across the entire Grid.

The WMS basically receives requests of job execution from a client, finds the required appropriate resources, then dispatches and follows them until completion. This is done handling failure in between and whenever possible. Other than single batch-like jobs, compound job types handled by the WMS are Directed Acyclic Graphs (a set of jobs where the input/output/execution of one of more jobs may depend on one or more other jobs), Parametric Jobs (multiple jobs with one parametrized description), and Collections (multiple jobs with a common description). Jobs are described via a flexible, high-level Job Definition Language (JDL).

• Installation and configuration starting from a cleaning machine (i.e. only OS)
• Update and configuration from a previous version

Normal Job

• Test submission of normal jobs with different options and situation Implemented

• Test the complete cycle with the two types of CEs: lcg and Cream Implemented

Perusal job

• Submit a jdl like this one:

[

• When job finished retrieve the output files
• Check the final status of the dag (all nodes and parent should be "Cleared")

More different jdls can added in the future.

Parallel Job

Service Description

Deployment scenarios

TBD

Functionality tests

Features/Scenarios to be tested

YAIM-WMS Configuration Testing

WMS Job Submission/GetOutput Testing

Normal Job

Perusal job

DAG job

Parametric Job

Collection Job

WMS Job shallow and deep re-submission

WMS Job List-match Testing

Without data

With data

WMS Job Cancel Testing

Normal job

DAG job

Collection

Node of a collection

Delegation Testing

Logging-info Testing

Job Status Testing

Others

BrokerInfo

Prologue and Epilogue jobs

Stress tests

Collection of 1000 nodes

Two days test

Description

• 2880 collections each of 20 jobs
• One collection every 60 seconds
• Four users
• Use LCG-CEs and CREAM-CEs (with different batch systems)
• Use automatic-delegation
• The job is a "sleep random(672)"
• Resubmission is enabled
• Enable proxy renewal

• • glite-brokerinfo getCE -f < .Brokerinfo file path >

• For the deep resubmission submit the following jdl:

  [
    requirements =  (other.GlueCEStateStatus == "Production");
    Rank = -2 * other.GlueCEStateWaitingJobs;
    Executable = "/bin/ls";
    prologue = "/bin/false";
    shallowretrycount = -1; 
    RetryCount = 3;
    usertags = [ exe = "deepresub" ];
  ]

  • The job should Aborted with message: hit job retry count (3)
  • Check the logging-info (-v 2) of the job, you should find 3 events from the WorkloadManager like this one:

        Event: Resubmission
    - Arrived                    =    Fri Jun 11 14:15:19 2010 CEST
    - Host                       =    devel18.cnaf.infn.it
    - Level                      =    SYSTEM
    - Priority                   =    synchronous
    - Reason                     =    shallow resubmission is disabled
    - Result                     =    WILLRESUB
    - Seqcode                    =    UI=000000:NS=0000000005:WM=000016:BH=0000000000:JSS=000009:LM=000018:LRMS=000000:APP=000000:LBS=000000
    - Source                     =    WorkloadManager

• For the shallow resubmission submit the following jdl file:

  [
    requirements =  (other.GlueCEStateStatus == "Production");
    Rank = -2 * other.GlueCEStateWaitingJobs;
    Executable = "/bin/ls";
    prologue = "/bin/false";
    shallowretrycount = 3;
    RetryCount = 0;
    usertags = [ exe = "shallowresub" ];
  ]

  • The job should Aborted with message: hit job shallow retry count (3)
  • Check the logging-info (-v 2) of the job, you should find 3 events from the WorkloadManager like this one:

    Event: Resubmission
    - Arrived                    =    Fri Jun 11 13:11:22 2010 CEST
    - Host                       =    devel18.cnaf.infn.it
    - Reason                     =    token still exists
    - Result                     =    SHALLOW
    - Source                     =    WorkloadManager
    - Src instance               =    13035
    - Tag                        =    /var/glite/SandboxDir/LW/https_3a_2f_2fdevel07.cnaf.infn.it_3a9000_2fLWVHkUN2cBOiEcWR7Rc6ew/token.txt
    - Timestamp                  =    Fri Jun 11 13:11:22 2010 CEST

Prologue and Epilogue jobs

• Submit a jdl like this one:

  ###########################################
  #     JDL with Prologue and Epilogue      #
  ###########################################
  
  Executable = "exe.sh";
  Arguments = "1000";
  StdOutput = "std.out";
  StdError = "std.err";
  Prologue = "prologue.sh";
  Epilogue = "epilogue.sh";
  FuzzyRank = true;
  
  InputSandbox = {"exe/exe.sh", "data/pippo", "exe/epilogue.sh","exe/prologue.sh"};
  OutputSandbox = {"std.out", "std.err", "prologue.out", "epilogue.out"};
  
  #Requirements = regexp("8443/cream", other.GlueCEUniqueID);
  #Requirements = regexp("2119/jobmanager", other.GlueCEUniqueID);
  
  RetryCount = 1;
  ShallowRetryCount = 2;

• Where:

  [ale@cream-15 UI]$ cat exe/exe.sh
  #!/bin/sh
  
  date
  echo "Hello world!"
  echo "My prologue said:" 
  cat prologue.out
  echo "My argument is $1"
  
  [ale@cream-15 UI]$ cat exe/prologue.sh 
  #!/bin/sh
  
  date > prologue.out
  echo "I'm the prologue" >> prologue.out
  echo "##########################" >> prologue.out
  echo "This is the output of an ls command" >> prologue.out
  ls >> prologue.out
  echo "##########################" >> prologue.out
  
  [ale@cream-15 UI]$ cat exe/epilogue.sh 
  #!/bin/sh
  
  echo " This is the output of the job: `cat std.out`" > epilogue.out
  echo "#########################" >> epilogue.out
  echo "We finish at `date`" >> epilogue.out
  echo "#########################" >> epilogue.out
  echo "All the jokes are done!" >> epilogue.out

• Use the requirements expression to choose an LCG or a CREAM ce
• When the job is Done retrieve the output sandbox, it should contains these files:
  • epilogue.out
  • prologue.out
  • std.err and std.out
• This should be the contenent of the files:

  [ale@cream-15 UI]$ cat prologue.out
  Fri Jun 11 10:24:15 BST 2010
  I'm the prologue
  ##########################
  This is the output of an ls command
  epilogue.sh
  exe.sh
  pippo
  prologue.out
  prologue.sh
  ##########################
  
  [ale@cream-15 UI]$ cat std.out
  Fri Jun 11 10:24:15 BST 2010
  Hello world!
  My prologue said:
  Fri Jun 11 10:24:15 BST 2010
  I'm the prologue
  ##########################
  This is the output of an ls command
  epilogue.sh
  exe.sh
  pippo
  prologue.out
  prologue.sh
  ##########################
  My argument is 1000
  
  [ale@cream-15 UI]$ cat epilogue.out
   This is the output of the job: Fri Jun 11 10:24:15 BST 2010
  Hello world!
  My prologue said:
  Fri Jun 11 10:24:15 BST 2010
  I'm the prologue
  ##########################
  This is the output of an ls command
  epilogue.sh
  exe.sh
  pippo
  prologue.out
  prologue.sh
  ##########################
  My argument is 1000
  #########################
  We finish at Fri Jun 11 10:24:15 BST 2010
  #########################
  All the jokes are done!

BrokerInfo

Requirements = (other.GlueCEInfoHostName = "spacin-ce1.dma.unina.it") && regexp("8443/cream", other.GlueCEUniqueID) ;

DataRequirements = { [ DataCatalogType = "DLI"; DataCatalog = "http://lfcserver.cnaf.infn.it:8085"; InputData = {"lfn:/grid/infngrid/cesini/PI_1M.txt","lfn:/grid/infngrid/cesini/e-2M.txt"}; ] };

DataAccessProtocol = "gsiftp";

• After the retrieving of the output files you should see these files:

  [ale@cream-15 DataReq]$ ls -la /tmp/ale_5XbbEhIAtjO5uC5wRyCmyA
  total 1264
  drwxr-xr-x   2 ale  ale     4096 Mar  4 10:31 .
  drwxrwxrwt  17 root root 1269760 Mar  4 10:31 ..
  -rw-rw-r--   1 ale  ale     7804 Mar  4 10:31 .BrokerInfo
  -rw-rw-r--   1 ale  ale        0 Mar  4 10:31 std.err
  -rw-rw-r--   1 ale  ale      624 Mar  4 10:31 std.out

[ale@cream-15 DataReq]$ cat /tmp/ale_5XbbEhIAtjO5uC5wRyCmyA/std.out
total 32
drwxr-xr-x  2 dteam038 dteam 4096 Mar  4 10:27 .
drwx------  5 dteam038 dteam 4096 Mar  4 10:27 ..
-rw-r--r--  1 dteam038 dteam 7804 Mar  4 10:27 .BrokerInfo
-rw-r--r--  1 dteam038 dteam  216 Mar  4 10:27 calc-pi.sh
-rw-r--r--  1 dteam038 dteam   17 Mar  4 10:27 fileA
-rw-r--r--  1 dteam038 dteam   17 Mar  4 10:27 fileB
-rw-r--r--  1 dteam038 dteam  166 Mar  4 10:27 https_3a_2f_2fdevel17.cnaf.infn.it_3a9000_2f5XbbEhIAtjO5uC5wRyCmyA.output
-rw-r--r--  1 dteam038 dteam    0 Mar  4 10:27 std.err
-rw-r--r--  1 dteam038 dteam    0 Mar  4 10:27 std.out
-rw-------  1 dteam038 dteam    0 Mar  4 10:27 tmp.LesGC30313

• • glite-brokerinfo getCE -f glite-brokerinfo getCE -f < .Brokerinfo file path >
    • It should return the matched CE
  • glite-brokerinfo getDataAccessProtocol -f < .Brokerinfo file path >
    • It should return the DataAccessProtocol parameter as specified in the jdl
  • glite-brokerinfo getInputData -f < .Brokerinfo file path >
    • It should return the InputData files specified in the jdl
  • glite-brokerinfo getSEs -f < .Brokerinfo file path >
    • It should return all the SEs (seen by the BDII of the WMS) that store all the files specified in the InputData attribute (You can chek it using the command lcg-lr --vo <VO name> <logical file name>)
  • glite-brokerinfo getCloseSEs -f < .Brokerinfo file path >
    • It should return only the CloseSe to the matched CE (to see them use the command: ldapsearch -x -h <specify the BDII host name> -p 2170 -b mds-vo-name=local,o=grid "(&(objectclass=GlueCESEBindGroup)(GlueCESEBindGroupCEUniqueID=<specify the CE name>))" | grep GlueCESEBindGroupSEUniqueID)
  • glite-brokerinfo getLFN2SFN <LFN of the files in InputData > -f < .Brokerinfo file path >
    • It should return the output of the command lcg-lr --vo <VO name> <logical file name>
  • glite-brokerinfo getVirtualOrganization -f < .Brokerinfo file path >
    • It should return your VO