Tags:
, view all tags

WMS test report

Summary

  • Component: WMS 3.3.4

  • ETICS Subsystem Configuration Name: emi-wms_R_3_3_6_1

  • VCS Tag: emi-ice_R_3_3_4_1, emi-jobsubmission_R_3_3_2_1, emi-wmproxy_R_3_3_4_1, emi-wms-brokerinfo_R_3_3_2_1, emi-wms-configuration_R_3_3_2_1, emi-wms-ism_R_3_3_2_1, emi-wms-matchmaking_R_3_3_2_1, emi-wms-purger_R_3_3_2_1, emi-yaim-wms_R_4_1_4_1

  • EMI Major Release: EMI 1 (Kebnekaise)

  • Platform: sl5 epel

  • Certification report: ...

  • Outcome: In certification

Clean installation

  • Starting from a SL5 x86_64 kernel machine:
[root@cream-44 ~]# cat /etc/redhat-release
Scientific Linux SL release 5.5 (Boron)
[root@cream-44 ~]# uname -a
Linux cream-44.pd.infn.it 2.6.18-238.5.1.el5xen #1 SMP Tue Mar 1 19:22:01 EST 2011 x86_64 x86_64 x86_64 GNU/Linux

  • /opt/glite/yaim/bin/yaim -c -s /usr/local/nfs/jra1_test/cream-44/site-info_wms.def -n WMS

Update

  • Starting from a production version:
    [root@cream-45 yum.repos.d]# etc/emi-version
    1.3.0-1
    

Test Report

Submission

Normal

[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 1 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: devel11.cnaf.infn.it
===> Started: 13:16:14
===> Ended  : 13:22:05
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111014131614.log

[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 1 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: devel11.cnaf.infn.it
===> Started: 17:16:39
===> Ended  : 17:22:59
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111014171639.log




[ale@cream-12 WMS-service]$ myproxy-init -d -s myproxy.cnaf.infn.it
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
Creating proxy ....................................................... Done
Proxy Verify OK
Your proxy is valid until: Fri Oct 21 17:20:00 2011
Enter MyProxy pass phrase:
Verifying - Enter MyProxy pass phrase:
Error entering passphrase.
[ale@cream-12 WMS-service]$ myproxy-init -d -s myproxy.cnaf.infn.it
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
Creating proxy .......................................................................................................................................................... Done
Proxy Verify OK
Your proxy is valid until: Fri Oct 21 17:20:13 2011
Enter MyProxy pass phrase:
Verifying - Enter MyProxy pass phrase:
A proxy valid for 168 hours (7.0 days) for user /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle now exists on myproxy.cnaf.infn.it.
[ale@cream-12 WMS-service]$ myproxy-logon -d -s myproxy.cnaf.infn.it --voms dteam
Enter MyProxy pass phrase:
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
Contacting  voms.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr] "dteam" Done
Creating proxy ..................................................................... Done

Your proxy is valid until Sat Oct 15 04:20:24 2011
A credential has been received for user /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle in /tmp/x509up_u500.
[ale@cream-12 WMS-service]$  voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u500
timeleft  : 10:59:48
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 11:58:47
uri       : voms.hellasgrid.gr:15004
[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 1
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: devel11.cnaf.infn.it
===> Started: 17:21:02
===> Ended  : 17:27:21
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111014172102.log

See gzipped log file

Jdl with DataRequirements

[ale@cream-12 WMS-service]$ cat JDLs/data-req.jdl
###########################################
#      JDL with Data Requirements         #
###########################################

Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";
Prologue = "prologue.sh";

FuzzyRank = true;
Environment={"LFC_HOST=prod-lfc-shared-central.cern.ch", "LFC_HOME=prod-lfc-shared-central.cern.ch:/grid/dteam/aleph"};
InputSandbox = {"JDLs/prologue.sh"};
OutputSandbox = {"std.out", "std.err","prologue.out", "file.txt"};

requirements = true;

DataRequirements = {
[
DataCatalogType = "DLI";
DataCatalog ="http://prod-lfc-shared-central.cern.ch:8085/"; 
InputData = { "guid:17c667b7-9622-4c95-be68-3d15d1eacadb" };
]
};

DataAccessProtocol = "gsiftp";
AllowZippedISB=true;
RetryCount = 1;
ShallowRetryCount = 2;

[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 1 -i -j JDLs/data-req.jdl 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: cream-45.pd.infn.it
===> Started: 16:39:27
===> Ended  : 17:07:30
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111020163927.log

Test log file is here

Jdl with ISBBaseURI and OSBDestURI

Test on CREAM ce:

[ale@cream-12 DataReq]$ cat URI.jdl 
###########################################
#      JDL with ISBURI & OSBDestURI       #
###########################################

Executable = "exe.sh";
StdOutput = "std.out";
StdError = "std.err";

InputSandboxBaseURI = "gsiftp://devel11.cnaf.infn.it:2811/tmp";

FuzzyRank = true;
InputSandbox = {"exe.sh", "test1", "test2"};
OutputSandbox = {"std.out", "std.err"};
OutputSandboxDestURI = {"gsiftp://cream-44.pd.infn.it:2811/tmp/myjobOutput", "std.err"};


requirements = other.GlueCEStateStatus == "Testing" &&  regexp("/cream", other.GlueCEUniqueID);

DataAccessProtocol = "gsiftp";
RetryCount = 1;
ShallowRetryCount = 2;


[ale@cream-12 DataReq]$ cat exe.sh 
#!/bin/sh

NAME=`echo $GLITE_WMS_JOBID | sed -e 's/.*\///'`

echo $NAME 

ls -l

[ale@cream-12 DataReq]$ glite-wms-job-submit -a -c /home/ale/UI/etc/wmp_cream-45.conf URI.jdl

Connecting to the service https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://cream-45.pd.infn.it:9000/JGxTge2N8vrsyuT7jhScnw

==========================================================================

[ale@cream-12 DataReq]$ glite-wms-job-status https://cream-45.pd.infn.it:9000/JGxTge2N8vrsyuT7jhScnw


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://cream-45.pd.infn.it:9000/JGxTge2N8vrsyuT7jhScnw
Current Status:     Done (Success)
Logged Reason(s):
    - job completed
    - Job Terminated Successfully
Exit code:          0
Status Reason:      Job Terminated Successfully
Destination:        cream-31.pd.infn.it:8443/cream-pbs-creamtest1
Submitted:          Fri Oct 21 12:40:25 2011 CEST
==========================================================================

[ale@cream-12 DataReq]$ glite-wms-job-output https://cream-45.pd.infn.it:9000/JGxTge2N8vrsyuT7jhScnw

Connecting to the service https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server


================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://cream-45.pd.infn.it:9000/JGxTge2N8vrsyuT7jhScnw
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/ale_JGxTge2N8vrsyuT7jhScnw

================================================================================


[ale@cream-12 DataReq]$ ls -l /tmp/jobOutput/ale_JGxTge2N8vrsyuT7jhScnw
total 0
-rw-rw-r-- 1 ale ale 0 Oct 21 12:46 std.err


Log on cream-44:

[root@cream-44 ~]# cat /tmp/myjobOutput 
JGxTge2N8vrsyuT7jhScnw
total 8
-rwxr-xr-x 1 dteam009 dteam 81 Oct 21 12:40 exe.sh
-rw-r--r-- 1 dteam009 dteam  0 Oct 21 12:40 std.err
-rw-r--r-- 1 dteam009 dteam 23 Oct 21 12:40 std.out
-rw-r--r-- 1 dteam009 dteam  0 Oct 21 12:40 test1
-rw-r--r-- 1 dteam009 dteam  0 Oct 21 12:40 test2

Same process with an LCG CE:

[ale@cream-12 DataReq]$ glite-wms-job-submit -a -c /home/ale/UI/etc/wmp_cream-45.conf URI.jdl

Connecting to the service https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://cream-45.pd.infn.it:9000/0quSOp8cK62SYKoR5v0IDQ

==========================================================================


[ale@cream-12 DataReq]$ glite-wms-job-status https://cream-45.pd.infn.it:9000/0quSOp8cK62SYKoR5v0IDQ


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://cream-45.pd.infn.it:9000/0quSOp8cK62SYKoR5v0IDQ
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job terminated successfully
Destination:        cream-11.pd.infn.it:2119/jobmanager-lcglsf-creamtest1
Submitted:          Fri Oct 21 12:48:32 2011 CEST
==========================================================================

[ale@cream-12 DataReq]$ glite-wms-job-output https://cream-45.pd.infn.it:9000/0quSOp8cK62SYKoR5v0IDQ

Connecting to the service https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server

ls -l 
================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://cream-45.pd.infn.it:9000/0quSOp8cK62SYKoR5v0IDQ
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/ale_0quSOp8cK62SYKoR5v0IDQ

================================================================================


[ale@cream-12 DataReq]$ ls -l /tmp/jobOutput/ale_0quSOp8cK62SYKoR5v0IDQ
total 0
-rw-rw-r-- 1 ale ale 0 Oct 21 12:51 std.err

Log on cream-44:
[root@cream-44 ~]# cat /tmp/myjobOutput 
0quSOp8cK62SYKoR5v0IDQ
total 28
-rwxr-xr-x  1 dteam003 dteam  81 Oct 21 12:49 exe.sh
-rw-r--r--  1 dteam003 dteam 118 Oct 21 12:49 https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f0quSOp8cK62SYKoR5v0IDQ.output
-rw-r--r--  1 dteam003 dteam   0 Oct 21 12:49 std.err
-rw-r--r--  1 dteam003 dteam  23 Oct 21 12:49 std.out
-rw-r--r--  1 dteam003 dteam   0 Oct 21 12:49 test1
-rw-r--r--  1 dteam003 dteam   0 Oct 21 12:49 test2
-rw-------  1 dteam003 dteam 103 Oct 21 12:49 tmp.bHXHsc5360
-rw-------  1 dteam003 dteam   2 Oct 21 12:49 tmp.DCRLdD5359
-rw-------  1 dteam003 dteam   2 Oct 21 12:49 tmp.iCdKAK5374
-rw-------  1 dteam003 dteam 103 Oct 21 12:49 tmp.iWCcAq5375
-rw-------  1 dteam003 dteam   0 Oct 21 12:49 tmp.osUtbu5373



Jdl with ISBBaseURI and OSBBaseDestURI

Submit to CREAM CE


[ale@cream-12 DataReq]$ cat BaseURI.jdl 
###########################################
#      JDL with ISBURI & OSBDestURI       #
###########################################

Executable = "exe.sh";
StdOutput = "std.out";
StdError = "std.err";

InputSandboxBaseURI = "gsiftp://devel11.cnaf.infn.it:2811/tmp";

FuzzyRank = true;
InputSandbox = {"exe.sh", "test1", "test2"};
OutputSandbox = {"std.out", "std.err"};
OutputSandboxBaseDestURI="gsiftp://cream-44.pd.infn.it:2811/tmp";

requirements = other.GlueCEStateStatus == "Testing" &&  regexp("/cream", other.GlueCEUniqueID);

RetryCount = 1;
ShallowRetryCount = 2;

[ale@cream-12 DataReq]$ glite-wms-job-submit -a -c /home/ale/UI/etc/wmp_cream-45.conf BaseURI.jdl 

Connecting to the service https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://cream-45.pd.infn.it:9000/WX6KjW4Z9yfYXNdhUnkNCQ

==========================================================================


[ale@cream-12 DataReq]$ glite-wms-job-status https://cream-45.pd.infn.it:9000/WX6KjW4Z9yfYXNdhUnkNCQ


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://cream-45.pd.infn.it:9000/WX6KjW4Z9yfYXNdhUnkNCQ
Current Status:     Done (Success)
Logged Reason(s):
    - job completed
    - Job Terminated Successfully
Exit code:          0
Status Reason:      Job Terminated Successfully
Destination:        cream-20.pd.infn.it:8443/cream-lsf-cert
Submitted:          Fri Oct 21 13:01:55 2011 CEST
==========================================================================

[ale@cream-12 DataReq]$ glite-wms-job-output https://cream-45.pd.infn.it:9000/WX6KjW4Z9yfYXNdhUnkNCQ

Connecting to the service https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server


================================================================================

         JOB GET OUTPUT OUTCOME

No output files to be retrieved for the job:
https://cream-45.pd.infn.it:9000/WX6KjW4Z9yfYXNdhUnkNCQ

================================================================================

Log on cream-44:

[root@cream-44 ~]# ls -l /tmp/std.*
-rw-r--r-- 1 dteam009 dteam   0 Oct 21 13:02 /tmp/std.err
-rw-r--r-- 1 dteam009 dteam 291 Oct 21 13:02 /tmp/std.out
[root@cream-44 ~]# cat /tmp/std.out 
WX6KjW4Z9yfYXNdhUnkNCQ
total 8
-rwxr-xr-x  1 dteam017 dteam 81 Oct 21 13:02 exe.sh
-rw-r--r--  1 dteam017 dteam  0 Oct 21 13:02 std.err
-rw-r--r--  1 dteam017 dteam 23 Oct 21 13:02 std.out
-rw-r--r--  1 dteam017 dteam  0 Oct 21 13:02 test1
-rw-r--r--  1 dteam017 dteam  0 Oct 21 13:02 test2

Submit to an LCG CE:


[ale@cream-12 DataReq]$ glite-wms-job-submit -a -c /home/ale/UI/etc/wmp_cream-45.conf BaseURI.jdl 

Connecting to the service https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://cream-45.pd.infn.it:9000/FsSdKXZ5kgYmelLYsqwO8A

==========================================================================

[ale@cream-12 DataReq]$ glite-wms-job-status https://cream-45.pd.infn.it:9000/FsSdKXZ5kgYmelLYsqwO8A


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://cream-45.pd.infn.it:9000/FsSdKXZ5kgYmelLYsqwO8A
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job terminated successfully
Destination:        cream-26.pd.infn.it:2119/jobmanager-lcgpbs-creamtest1
Submitted:          Fri Oct 21 13:06:46 2011 CEST
==========================================================================

[ale@cream-12 DataReq]$ glite-wms-job-output https://cream-45.pd.infn.it:9000/FsSdKXZ5kgYmelLYsqwO8A

Connecting to the service https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server


================================================================================

         JOB GET OUTPUT OUTCOME

No output files to be retrieved for the job:
https://cream-45.pd.infn.it:9000/FsSdKXZ5kgYmelLYsqwO8A

================================================================================

Log on cream-44:

[root@cream-44 ~]# ls -l /tmp/std.*
-rw-r--r-- 1 dteam009 dteam   0 Oct 21 13:07 /tmp/std.err
-rw-r--r-- 1 dteam009 dteam 721 Oct 21 13:07 /tmp/std.out
[root@cream-44 ~]# cat /tmp/std.out 
FsSdKXZ5kgYmelLYsqwO8A
total 28
-rwxr-xr-x  1 dteam003 dteam  81 Oct 21 13:07 exe.sh
-rw-r--r--  1 dteam003 dteam 118 Oct 21 13:07 https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fFsSdKXZ5kgYmelLYsqwO8A.output
-rw-r--r--  1 dteam003 dteam   0 Oct 21 13:07 std.err
-rw-r--r--  1 dteam003 dteam  23 Oct 21 13:07 std.out
-rw-r--r--  1 dteam003 dteam   0 Oct 21 13:07 test1
-rw-r--r--  1 dteam003 dteam   0 Oct 21 13:07 test2
-rw-------  1 dteam003 dteam 103 Oct 21 13:07 tmp.EoCkc12184
-rw-------  1 dteam003 dteam   0 Oct 21 13:07 tmp.GIFWQ12182
-rw-------  1 dteam003 dteam 103 Oct 21 13:07 tmp.knHTI12169
-rw-------  1 dteam003 dteam   2 Oct 21 13:07 tmp.vbISe12183
-rw-------  1 dteam003 dteam   2 Oct 21 13:07 tmp.vWbAp12168


Jdl with AllowZippedISB

[ale@cream-12 DataReq]$ cat AllowZip.jdl 
###########################################
#   JDL with AllowZippedISB attribute     #
###########################################

Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";

InputSandbox = {"fileA", "fileB", "fileC", "fileD", "fileE"};
OutputSandbox = {"std.out", "std.err"};

requirements = other.GlueCEStateStatus == "Testing" &&  regexp("/cream", other.GlueCEUniqueID);

AllowZippedISB=true;

RetryCount = 1;
ShallowRetryCount = 2;

[ale@cream-12 DataReq]$ glite-wms-job-submit -a --debug  -c /home/ale/UI/etc/wmp_cream-45.conf AllowZip.jdl 
-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - Vo read from proxy certificate extension
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - VirtualOrganisation value : dteam
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - Loaded user configuration file:
 /home/ale/UI/etc/wmp_cream-45.conf
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - Configuration file: //etc/glite-wms/dteam/glite_wmsui.conf - MyProxyServer attribute(s) no more supported outside JDL Default Attributes section "JdlDefaultAttributes"
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - Loaded Vo specific configuration file:
 //etc/glite-wms/dteam/glite_wmsui.conf
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - Loaded Configuration values: 
 [
  JdlDefaultAttributes = 
   [
    SignificantAttributes = { "Requirements","Rank" }; 
    RetryCount = 3; 
    VirtualOrganisation = "dteam"; 
    rank = 1; 
    MyProxyServer = "myproxy.cnaf.infn.it"; 
    requirements = ( other.GlueCEStateStatus == "Testing" ) || ( other.GlueCEStateStatus == "Production" )
   ]; 
  OutputStorage = "/tmp"; 
  ListenerStorage = "/tmp"; 
  ErrorStorage = "/tmp"; 
  WmProxyEndPoints = { "https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server" }
 ]
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - Getting Endpoint URL from configuration file 
-----------------------------------------

Connecting to the service https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server

-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - Calling the WMProxy getVersion service
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - WMProxy Version: 3.3.4 
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - Sending Proxy Request to https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - Calling the WMProxy delegationns__getProxyReq service
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - delegationns__getProxyReq -  The proxy has been successfully retrieved
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - Calling the WMProxy delegationns__putProxyReq service
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - The proxy has been successfully delegated with the identifier: tuSUwcytwLcBpNmsumDTJA
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - Calling the WMProxy getTransferProtocols service
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - getTransferProtocols -  Available protocols: gsiftp, https
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - FileTransferProtocol not specified; using the default protocol: gsiftp
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - The JDL file is: /home/ale/UI/DataReq/AllowZip.jdl
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - Retrieving the list of the local ISB files from the user JDL 
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - Total size of the ISB file(s) to be transferred to: 34
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - Max single job size of the ISB file(s) to be transferred to: 17
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - Checking the User-FreeQuota on the server 
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - Calling the WMProxy getFreeQuota service
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - getFreeQuota -  User freequota not set on the server
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - Getting the max ISB size from the server https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:34 -I- PID: 17989 (Debug) - Calling the WMProxy getMaxInputSandboxSize service
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:35 -I- PID: 17989 (Debug) - getMaxInputSandboxSize -  Max ISB size information successfully retrieved
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:35 -I- PID: 17989 (Debug) - The max job size (17 bytes) doesn't exceed the max size limit of 100000000 bytes: File transfer is allowed
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:35 -I- PID: 17989 (Debug) - File archiving and file compression allowed by user in the JDL
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:35 -I- PID: 17989 (Debug) - Registering JDL [ StdOutput = "std.out"; ShallowRetryCount = 2; SignificantAttributes = { "Requirements","Rank" }; RetryCount = 1; Executable = "/bin/ls"; Type = "job"; Arguments = "-la"; AllowZippedISB = true; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank =  -other.GlueCEStateEstimatedResponseTime; ZippedISB = { "ISBfiles_LFm0I3oE0tPp-9woGjYHaQ_0.tar.gz" }; OutputSandbox = { "std.out","std.err" }; InputSandbox = { "file:///home/ale/UI/DataReq/fileA","file:///home/ale/UI/DataReq/fileB","file:///home/ale/UI/DataReq/fileC","file:///home/ale/UI/DataReq/fileD","file:///home/ale/UI/DataReq/fileE" }; StdError = "std.err"; rank = 1; MyProxyServer = "myproxy.cnaf.infn.it"; requirements = ( other.GlueCEStateStatus == "Testing" && regexp("/cream",other.GlueCEUniqueID) ) && ( ( other.GlueCEStateStatus == "Testing" ) || ( other.GlueCEStateStatus == "Production" ) ) ]
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:35 -I- PID: 17989 (Debug) - Registering the job to the service https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:35 -I- PID: 17989 (Debug) - Calling the WMProxy jobRegister service
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:35 -I- PID: 17989 (Debug) - The JobId is:  https://cream-45.pd.infn.it:9000/rmnxumOwedNhAjvFx_C4Zg
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:35 -I- PID: 17989 (Debug) - Archiving the ISB files: /tmp/ISBfiles_LFm0I3oE0tPp-9woGjYHaQ_0.tar
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:35 -I- PID: 17989 (Debug) - This archive file has been successfully created: /tmp/ISBfiles_LFm0I3oE0tPp-9woGjYHaQ_0.tar
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:35 -I- PID: 17989 (Debug) - Compressing the file (.gz): /tmp/ISBfiles_LFm0I3oE0tPp-9woGjYHaQ_0.tar
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:35 -I- PID: 17989 (Debug) - ISB ZIPPED file successfully created: /tmp/ISBfiles_LFm0I3oE0tPp-9woGjYHaQ_0.tar.gz
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:35 -I- PID: 17989 (Debug) - Getting the SandboxDestinationURI from the service https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:35 -I- PID: 17989 (Debug) - Calling the WMProxy getSandboxDestURI service with gsiftp protocol 
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:35 -I- PID: 17989 (Debug) - getSandboxDestURI -  Destination URIs sucessfully retrieved
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:35 -I- PID: 17989 (Debug) - DestinationURI: gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/rm/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2frmnxumOwedNhAjvFx_5fC4Zg/input
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:35 -I- PID: 17989 (Debug) - FileTransfer (gsiftp): using globus-url-copy to transfer the local InputSandBox file(s) to the submission endpoint
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:35 -I- PID: 17989 (Debug) - File Transfer (gsiftp) 
 Command: /usr/bin/globus-url-copy
Source: file:///tmp/ISBfiles_LFm0I3oE0tPp-9woGjYHaQ_0.tar.gz
Destination: gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/rm/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2frmnxumOwedNhAjvFx_5fC4Zg/input/ISBfiles_LFm0I3oE0tPp-9woGjYHaQ_0.tar.gz
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:37 -I- PID: 17989 (Debug) - File Transfer (gsiftp) Transfer successfully done
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:37 -I- PID: 17989 (Debug) - Starting the job:  https://cream-45.pd.infn.it:9000/rmnxumOwedNhAjvFx_C4Zg
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:37 -I- PID: 17989 (Debug) - Calling the WMProxy jobStart service
-----------------------------------------
-----------------------------------------
21 Oct 2011, 13:20:38 -I- PID: 17989 (Debug) - jobStart -  The job has been successfully started
-----------------------------------------

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://cream-45.pd.infn.it:9000/rmnxumOwedNhAjvFx_C4Zg

==========================================================================

      *** Log file created ***
Possible Errors and Debug messages have been printed in the following file:
/tmp/glite-wms-job-submit_500_17989_20111021_13-20-34.log


Check in ZippedISB Attribute is added:

[ale@cream-12 WMS-service]$ glite-wms-job-info --jdl https://cream-45.pd.infn.it:9000/rmnxumOwedNhAjvFx_C4Zg

Connecting to the service https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server


======================= glite-wms-job-info Success =======================

The registered JDL
 for the job https://cream-45.pd.infn.it:9000/rmnxumOwedNhAjvFx_C4Zg :

 [
  OutputSandboxPath = "/var/SandboxDir/rm/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2frmnxumOwedNhAjvFx_5fC4Zg/output"; 
  StdOutput = "std.out"; 
  ShallowRetryCount = 2; 
  SignificantAttributes = { "Requirements","Rank" }; 
  RetryCount = 1; 
  Executable = "/bin/ls"; 
  Type = "job"; 
  LB_sequence_code = "UI=000000:NS=0000000004:WM=000000:BH=0000000000:JSS=000000:LM=000000:LRMS=000000:APP=000000:LBS=000000"; 
  Arguments = "-la"; 
  AllowZippedISB = true; 
  VirtualOrganisation = "dteam"; 
  JobType = "normal"; 
  DefaultRank =  -other.GlueCEStateEstimatedResponseTime; 
  ZippedISB = { "ISBfiles_LFm0I3oE0tPp-9woGjYHaQ_0.tar.gz" }; 
  OutputSandboxDestURI = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/rm/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2frmnxumOwedNhAjvFx_5fC4Zg/output/std.out","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/rm/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2frmnxumOwedNhAjvFx_5fC4Zg/output/std.err" }; 
  OutputSandbox = { "std.out","std.err" }; 
  InputSandbox = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/rm/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2frmnxumOwedNhAjvFx_5fC4Zg/input/fileA","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/rm/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2frmnxumOwedNhAjvFx_5fC4Zg/input/fileB","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/rm/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2frmnxumOwedNhAjvFx_5fC4Zg/input/fileC","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/rm/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2frmnxumOwedNhAjvFx_5fC4Zg/input/fileD","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/rm/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2frmnxumOwedNhAjvFx_5fC4Zg/input/fileE" }; 
  edg_jobid = "https://cream-45.pd.infn.it:9000/rmnxumOwedNhAjvFx_C4Zg"; 
  VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; 
  CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle"; 
  StdError = "std.err"; 
  InputSandboxPath = "/var/SandboxDir/rm/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2frmnxumOwedNhAjvFx_5fC4Zg/input"; 
  rank = 1; 
  MyProxyServer = "myproxy.cnaf.infn.it"; 
  X509UserProxy = "/var/SandboxDir/rm/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2frmnxumOwedNhAjvFx_5fC4Zg/user.proxy"; 
  InputSandboxDestFileName = { "fileA","fileB","fileC","fileD","fileE" }; 
  requirements = ( ( other.GlueCEStateStatus == "Testing" && regexp("/cream",other.GlueCEUniqueID) ) && ( ( other.GlueCEStateStatus == "Testing" ) || ( other.GlueCEStateStatus == "Production" ) ) ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) :  !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); 
  WMPInputSandboxBaseURI = "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/rm/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2frmnxumOwedNhAjvFx_5fC4Zg"
 ]

==========================================================================


[ale@cream-12 DataReq]$ glite-wms-job-status https://cream-45.pd.infn.it:9000/rmnxumOwedNhAjvFx_C4Zg


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://cream-45.pd.infn.it:9000/rmnxumOwedNhAjvFx_C4Zg
Current Status:     Done (Success)
Logged Reason(s):
    - job completed
    - Job Terminated Successfully
Exit code:          0
Status Reason:      Job Terminated Successfully
Destination:        cream-23.pd.infn.it:8443/cream-lsf-cert
Submitted:          Fri Oct 21 13:20:35 2011 CEST
==========================================================================

Retrieve oputput files:

[ale@cream-12 DataReq]$ glite-wms-job-output https://cream-45.pd.infn.it:9000/rmnxumOwedNhAjvFx_C4Zg

Connecting to the service https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server


================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://cream-45.pd.infn.it:9000/rmnxumOwedNhAjvFx_C4Zg
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/ale_rmnxumOwedNhAjvFx_C4Zg

================================================================================

Check std output to see if all the files are transfered to the CE:

[ale@cream-12 DataReq]$ cat /tmp/jobOutput/ale_rmnxumOwedNhAjvFx_C4Zg/std.out 
total 20
drwxr-xr-x  2 dteam017 dteam 4096 Oct 21 13:20 .
drwxr-xr-x  3 dteam017 dteam 4096 Oct 21 13:20 ..
-rw-r--r--  1 dteam017 dteam  361 Oct 21 13:20 .BrokerInfo
-rw-r--r--  1 dteam017 dteam   17 Oct 21 13:20 fileA
-rw-r--r--  1 dteam017 dteam   17 Oct 21 13:20 fileB
-rw-r--r--  1 dteam017 dteam    0 Oct 21 13:20 fileC
-rw-r--r--  1 dteam017 dteam    0 Oct 21 13:20 fileD
-rw-r--r--  1 dteam017 dteam    0 Oct 21 13:20 fileE
-rw-r--r--  1 dteam017 dteam    0 Oct 21 13:20 std.err
-rw-r--r--  1 dteam017 dteam    0 Oct 21 13:20 std.out
-rw-r--r--  1 dteam017 dteam    0 Oct 21 13:20 .tmp_file



Jdl with ExpiryTime

[ale@cream-12 DataReq]$ cat ExpiryTime.jdl 
###########################################
#       JDL with ExpiryTime attribute     #
###########################################

Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";

InputSandbox = {"fileA", "fileB", "fileC", "fileD", "fileE"};
OutputSandbox = {"std.out", "std.err"};

requirements = false;

ExpiryTime = 1319197000;

RetryCount = 1;
ShallowRetryCount = 2;

[ale@cream-12 DataReq]$ date +%s
1319196578
[ale@cream-12 DataReq]$ glite-wms-job-submit -a   -c /home/ale/UI/etc/wmp_cream-45.conf ExpiryTime.jdl 

Connecting to the service https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://cream-45.pd.infn.it:9000/WvB8J_CHOaEqOy2GjEyIMg

==========================================================================


[ale@cream-12 DataReq]$ glite-wms-job-status https://cream-45.pd.infn.it:9000/WvB8J_CHOaEqOy2GjEyIMg


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://cream-45.pd.infn.it:9000/WvB8J_CHOaEqOy2GjEyIMg
Current Status:     Waiting 
Status Reason:      BrokerHelper: no compatible resources
Submitted:          Fri Oct 21 13:29:48 2011 CEST
==========================================================================

[ale@cream-12 DataReq]$ date +%s
1319197024
[ale@cream-12 DataReq]$ glite-wms-job-status https://cream-45.pd.infn.it:9000/WvB8J_CHOaEqOy2GjEyIMg


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://cream-45.pd.infn.it:9000/WvB8J_CHOaEqOy2GjEyIMg
Current Status:     Waiting 
Status Reason:      BrokerHelper: no compatible resources
Submitted:          Fri Oct 21 13:29:48 2011 CEST
==========================================================================

[ale@cream-12 DataReq]$ date +%s
1319197201
[ale@cream-12 DataReq]$ glite-wms-job-status https://cream-45.pd.infn.it:9000/WvB8J_CHOaEqOy2GjEyIMg


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://cream-45.pd.infn.it:9000/WvB8J_CHOaEqOy2GjEyIMg
Current Status:     Aborted 
Status Reason:      request expired
Submitted:          Fri Oct 21 13:29:48 2011 CEST
==========================================================================


Jdl with ShortDeadlineJob

cat [ale@cream-12 DataReq]$ cat SDLJ.jdl 
###########################################
# JDL with ShortDeadlineJob attribute     #
###########################################

Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";

InputSandbox = {"fileA", "fileB", "fileC", "fileD", "fileE"};
OutputSandbox = {"std.out", "std.err"};

requirements = true;

ShortDeadlineJob = true;

RetryCount = 1;
ShallowRetryCount = 2;

[ale@cream-12 DataReq]$ glite-wms-job-list-match -a  -c /home/ale/UI/etc/wmp_cream-45.conf SDLJ.jdl 

Connecting to the service https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server

==========================================================================

           COMPUTING ELEMENT IDs LIST 
 The following CE(s) matching your job requirements have been found:

   *CEId*
 - ce01.ariagni.hellasgrid.gr:2119/jobmanager-pbs-dteam.sdj
 - ce01.marie.hellasgrid.gr:2119/jobmanager-pbs-sdj
 - cream-ce01.ariagni.hellasgrid.gr:8443/cream-pbs-dteam.sdj
 - cream-ce01.marie.hellasgrid.gr:8443/cream-pbs-sdj
 - grid4.mif.vu.lt:2119/jobmanager-lcgpbs-sdj
 - grive02.ibcp.fr:2119/jobmanager-pbs-sdj
 - mario.univ-lille1.fr:8443/cream-pbs-sdj

==========================================================================


[ale@cream-12 DataReq]$ glite-wms-job-submit -a   -c /home/ale/UI/etc/wmp_cream-45.conf SDLJ.jdl 

Connecting to the service https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://cream-45.pd.infn.it:9000/VkQ7F_5o-sl7_n3DiXH_kQ

==========================================================================


[ale@cream-12 DataReq]$ glite-wms-job-status https://cream-45.pd.infn.it:9000/VkQ7F_5o-sl7_n3DiXH_kQ


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://cream-45.pd.infn.it:9000/VkQ7F_5o-sl7_n3DiXH_kQ
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job Terminated Successfully
Destination:        cream-ce01.ariagni.hellasgrid.gr:8443/cream-pbs-dteam.sdj
Submitted:          Fri Oct 21 13:47:54 2011 CEST
==========================================================================

[ale@cream-12 DataReq]$ glite-wms-job-output https://cream-45.pd.infn.it:9000/VkQ7F_5o-sl7_n3DiXH_kQ

Connecting to the service https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server


================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://cream-45.pd.infn.it:9000/VkQ7F_5o-sl7_n3DiXH_kQ
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/ale_VkQ7F_5o-sl7_n3DiXH_kQ

================================================================================


[ale@cream-12 DataReq]$ ls -l /tmp/jobOutput/ale_VkQ7F_5o-sl7_n3DiXH_kQ
total 4
-rw-rw-r-- 1 ale ale   0 Oct 21 13:51 std.err
-rw-rw-r-- 1 ale ale 587 Oct 21 13:51 std.out

[ale@cream-12 DataReq]$ cat /tmp/jobOutput/ale_VkQ7F_5o-sl7_n3DiXH_kQ/std.out 
total 8
drwxr-xr-x 2 dteam070 dteam 4096 Oct 21 14:48 .
drwxr-xr-x 3 dteam070 dteam 4096 Oct 21 14:48 ..
-rw-r--r-- 1 dteam070 dteam 2131 Oct 21 14:48 .BrokerInfo
-rw-r--r-- 1 dteam070 dteam   17 Oct 21 14:48 fileA
-rw-r--r-- 1 dteam070 dteam   17 Oct 21 14:48 fileB
-rw-r--r-- 1 dteam070 dteam    0 Oct 21 14:48 fileC
-rw-r--r-- 1 dteam070 dteam    0 Oct 21 14:48 fileD
-rw-r--r-- 1 dteam070 dteam    0 Oct 21 14:48 fileE
-rw-r--r-- 1 dteam070 dteam    0 Oct 21 14:48 std.err
-rw-r--r-- 1 dteam070 dteam    0 Oct 21 14:48 std.out
-rw-r--r-- 1 dteam070 dteam    0 Oct 21 14:48 .tmp_file


Collection

Bulk submission

[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 2 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 2: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) 


 Test 2A: Submit to an LCG-CE 


 Test 2B: Submit to a CREAM CE 


 Test 2C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: devel11.cnaf.infn.it
===> Started: 17:36:04
===> Ended  : 17:42:06
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111014173604.log

[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 2 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 2: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) 


 Test 2A: Submit to an LCG-CE 


 Test 2B: Submit to a CREAM CE 


 Test 2C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: devel11.cnaf.infn.it
===> Started: 17:36:30
===> Ended  : 17:44:31
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111014173630.log

[ale@cream-12 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u500
timeleft  : 10:33:46
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 11:32:45
uri       : voms.hellasgrid.gr:15004
[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 2 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Set 2: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) 


 Test 2A: Submit to an LCG-CE 


 Test 2B: Submit to a CREAM CE 


 Test 2C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: devel11.cnaf.infn.it
===> Started: 17:46:43
===> Ended  : 17:52:45
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111014174643.log


See gzipped log file

All in a jdl


[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 7 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 7: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) using a single jdl with al the jdls of nodes 


 Test 7A: Submit to an LCG-CE 


 Test 7B: Submit to a CREAM CE 


 Test 7C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: devel11.cnaf.infn.it
===> Started: 18:17:34
===> Ended  : 18:25:09
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111018181734.log

===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 7: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) using a single jdl with al the jdls of nodes 


 Test 7A: Submit to an LCG-CE 


 Test 7B: Submit to a CREAM CE 


 Test 7C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: devel11.cnaf.infn.it
===> Started: 18:25:40
===> Ended  : 18:32:45
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111018182540.log

[ale@cream-12 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u500
timeleft  : 2:42:56
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 3:41:56
uri       : voms.hellasgrid.gr:15004
[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 7 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Set 7: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) using a single jdl with al the jdls of nodes 


 Test 7A: Submit to an LCG-CE 


 Test 7B: Submit to a CREAM CE 


 Test 7C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: devel11.cnaf.infn.it
===> Started: 18:25:54
===> Ended  : 18:32:35
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111018182554.log


See gzipped log file

Parametric job

[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 3 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 3: Submit a parametric job (3 cases LCG-CE, CREAM, general) 


 Test 3A: Submit to an LCG-CE 


 Test 3B: Submit to a CREAM CE 


 Test 3C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: devel11.cnaf.infn.it
===> Started: 09:48:11
===> Ended  : 09:55:05
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111018094811.log

[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 3 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 3: Submit a parametric job (3 cases LCG-CE, CREAM, general) 


 Test 3A: Submit to an LCG-CE 


 Test 3B: Submit to a CREAM CE 


 Test 3C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: devel11.cnaf.infn.it
===> Started: 09:58:13
===> Ended  : 10:07:06
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111018095813.log

[ale@cream-12 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u500
timeleft  : 10:50:45
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 11:49:45
uri       : voms2.hellasgrid.gr:15004


[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 3
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Set 3: Submit a parametric job (3 cases LCG-CE, CREAM, general) 


 Test 3A: Submit to an LCG-CE 


 Test 3B: Submit to a CREAM CE 


 Test 3C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: devel11.cnaf.infn.it
===> Started: 10:09:18
===> Ended  : 10:18:20
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111018100918.log



See gzipped log file

DAG job


[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 4 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 4: Submit a DAG job 

===> 
===> Test: WMS-job-cycle.py
===> WMS: devel11.cnaf.infn.it
===> Started: 10:20:01
===> Ended  : 10:25:16
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111018102001.log

[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 4 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 4: Submit a DAG job 

===> 
===> Test: WMS-job-cycle.py
===> WMS: devel11.cnaf.infn.it
===> Started: 10:36:52
===> Ended  : 10:42:06
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111018103652.log

[ale@cream-12 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u500
timeleft  : 9:50:38
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 10:49:38
uri       : voms.hellasgrid.gr:15004

[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 4 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Set 4: Submit a DAG job 

===> 
===> Test: WMS-job-cycle.py
===> WMS: devel11.cnaf.infn.it
===> Started: 11:42:46
===> Ended  : 11:47:59
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111018114246.log


See gzipped log file

Parallel job

[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 5 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 5: Submit a MPI job (3 cases LCG-CE, CREAM, general) 


 Test 5A: Submit to an LCG-CE 


 Test 5B: Submit to a CREAM CE 


 Test 5C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: devel11.cnaf.infn.it
===> Started: 16:18:55
===> Ended  : 16:26:49
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111019161855.log

[ale@cream-12 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u500
timeleft  : 9:38:05
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 10:37:05
uri       : voms2.hellasgrid.gr:15004

[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 5
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Set 5: Submit a MPI job (3 cases LCG-CE, CREAM, general) 


 Test 5A: Submit to an LCG-CE 


 Test 5B: Submit to a CREAM CE 


 Test 5C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: devel11.cnaf.infn.it
===> Started: 16:30:22
===> Ended  : 16:40:18
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111019163022.log


See gzipped log file

Testing forwarding parameters for parallel jobs

In order to test the passing of mpi parameters (wholenodes,smpgranularity,cpunumber,hostnumber) to the ce, an mpi jdl is submitted with varying values for the parameters, specifying a cream CE as destination. Then the cream jobid is retrieved via a glite-wms-job-logging-info -v 2 ("Dest jobid" field of a Transfer event).Finally the cream jdl is found via glite-ce-job-status -L 2 <cream-jobid>.

The following jdls were submitted:

1)

jdl:

Executable = "openmpi-wrapper.sh";
Arguments="hello";
StdOutput  = "hello.out";
StdError   = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
InputSandbox = {"/home/fabiok/mpi/openmpi-wrapper.sh","/home/fabiok/mpi/hello.c","/home/fabiok/mpi/Makefile"};
Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID);
WholeNodes = true;
SMPGranularity = 2;
CpuNumber=2;
Hostnumber = 1;

Resultim cream jdl:

JDL            = [[ CEInfoHostName = "ce-02.roma3.infn.it"; OutputSandboxPath = "/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/output"; StdOutput = "hello.out"; BatchSystem = "pbs"; GlobusResourceContactString = "ce-02.roma3.infn.it:8443/cream-pbs"; QueueName = "cert"; ShallowRetryCount = 10; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 3; Executable = "openmpi-wrapper.sh"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000011:BH=0000000000:JSS=000004:LM=000012:LRMS=000000:APP=000000:LBS=000000"; Arguments = "hello"; AllowZippedISB = true; maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank =  -other.GlueCEStateEstimatedResponseTime; ce_id = "ce-02.roma3.infn.it:8443/cream-pbs-cert"; ZippedISB = { "ISBfiles_IggVNRt3plhKwsv7zkIQRw_0.tar.gz" }; OutputSandboxDestURI = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/output/hello.out","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/output/hello.err" }; OutputSandbox = { "hello.out","hello.err" }; CeRequirements = "( true && ( true && ( true && ( true && ( true && ( true && Member(\"OPENMPI\",other.GlueHostApplicationSoftwareRunTimeEnvironment) ) ) ) ) ) ) && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )"; InputSandbox = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/input/openmpi-wrapper.sh","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/input/hello.c","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/input/Makefile","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/input/.BrokerInfo" }; edg_jobid = "https://cream-45.pd.infn.it:9000/gQF7dCwrxP9pfDne1RfyOA"; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini"; StdError = "hello.err"; SMPGranularity = 2; InputSandboxPath = "/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/input"; CeApplicationDir = "/storage/local/exp_soft/dteam"; ReallyRunningToken = "gsiftp://cream-45.pd.infn.it/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/token.txt_0"; rank =  -other.GlueCEStateEstimatedResponseTime; WholeNodes = true; CpuNumber = 2; edg_previous_matches = { "ce.scope.unina.it" }; WMSHostname = "cream-45.pd.infn.it"; lrms_type = "pbs"; MyProxyServer = "myproxy.cnaf.infn.it"; edg_previous_matches_ex = { [ timestamp = 1319039372; ce_id = "ce.scope.unina.it" ] }; X509UserProxy = "/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/user.proxy"; InputSandboxDestFileName = { "openmpi-wrapper.sh","hello.c","Makefile" }; Hostnumber = 1; requirements = ( ( ( ( ( ( Member("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*",other.GlueCEUniqueID) ) && ( other.GlueCEStateStatus == "Production" ) ) && other.GlueCEInfoTotalCPUs >= 2 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueCEInfoTotalCPUs >= 2 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) :  !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA" ]]

2)

jdl:

CpuNumber=1;
Executable = "openmpi-wrapper.sh";
Arguments="hello";
StdOutput  = "hello.out";
StdError   = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
InputSandbox = {"/home/fabiok/mpi/openmpi-wrapper.sh","/home/fabiok/mpi/hello.c","/home/fabiok/mpi/Makefile"};
Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID);
WholeNodes = true;
SMPGranularity = 2;

Resulting cream jdl:

JDL            = [[ CEInfoHostName = "ce-02.roma3.infn.it"; OutputSandboxPath = "/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/output"; StdOutput = "hello.out"; BatchSystem = "pbs"; GlobusResourceContactString = "ce-02.roma3.infn.it:8443/cream-pbs"; QueueName = "cert"; ShallowRetryCount = 10; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 3; Executable = "openmpi-wrapper.sh"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000011:BH=0000000000:JSS=000004:LM=000012:LRMS=000000:APP=000000:LBS=000000"; Arguments = "hello"; AllowZippedISB = true; maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank =  -other.GlueCEStateEstimatedResponseTime; ce_id = "ce-02.roma3.infn.it:8443/cream-pbs-cert"; ZippedISB = { "ISBfiles_A4rYVNupOqd7YmRathOHog_0.tar.gz" }; OutputSandboxDestURI = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/output/hello.out","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/output/hello.err" }; OutputSandbox = { "hello.out","hello.err" }; CeRequirements = "( true && ( true && ( true && ( true && ( true && ( true && Member(\"OPENMPI\",other.GlueHostApplicationSoftwareRunTimeEnvironment) ) ) ) ) ) ) && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )"; InputSandbox = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/input/openmpi-wrapper.sh","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/input/hello.c","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/input/Makefile","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/input/.BrokerInfo" }; edg_jobid = "https://cream-45.pd.infn.it:9000/94xFQjidomDV4P_pzEey5A"; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini"; StdError = "hello.err"; SMPGranularity = 2; InputSandboxPath = "/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/input"; CeApplicationDir = "/storage/local/exp_soft/dteam"; ReallyRunningToken = "gsiftp://cream-45.pd.infn.it/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/token.txt_0"; rank =  -other.GlueCEStateEstimatedResponseTime; WholeNodes = true; CpuNumber = 1; edg_previous_matches = { "gridsrv2-4.dir.garr.it" }; WMSHostname = "cream-45.pd.infn.it"; lrms_type = "pbs"; MyProxyServer = "myproxy.cnaf.infn.it"; edg_previous_matches_ex = { [ timestamp = 1319037684; ce_id = "gridsrv2-4.dir.garr.it" ] }; X509UserProxy = "/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/user.proxy"; InputSandboxDestFileName = { "openmpi-wrapper.sh","hello.c","Makefile" }; requirements = ( ( ( ( ( ( Member("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*",other.GlueCEUniqueID) ) && ( other.GlueCEStateStatus == "Production" ) ) && other.GlueCEInfoTotalCPUs >= 1 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueCEInfoTotalCPUs >= 1 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) :  !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A" ]]

3)

jdl:

Executable = "openmpi-wrapper.sh";
Arguments="hello";
StdOutput  = "hello.out";
StdError   = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
InputSandbox = {"/home/fabiok/mpi/openmpi-wrapper.sh","/home/fabiok/mpi/hello.c","/home/fabiok/mpi/Makefile"};
Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID);
WholeNodes = false;
SMPGranularity = 3;
CpuNumber=3;
Hostnumber = 1;

Submission result:

[fabiok@ui mpi]$ glite-wms-job-submit -a -c ../wms_cream-45.conf mpi.jdl 

Connecting to the service https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server


Warning - Unable to register the job to the service: https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server
SMPGranularity and HostNumber are mutually exclusive when WholeNodes allocation is not requested: wrong combination of values

Method: jobRegister


Error - Operation failed
Unable to find any endpoint where to perform service request

4)

jdl:

Executable = "openmpi-wrapper.sh";
Arguments="hello";
StdOutput  = "hello.out";
StdError   = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
InputSandbox = {"/home/fabiok/mpi/openmpi-wrapper.sh","/home/fabiok/mpi/hello.c","/home/fabiok/mpi/Makefile"};
Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID);
WholeNodes = false;
SMPGranularity = 3;
CpuNumber=3;
//Hostnumber = 1;

Resulting cream jdl:

JDL            = [[ CEInfoHostName = "ce-02.roma3.infn.it"; OutputSandboxPath = "/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/output"; StdOutput = "hello.out"; BatchSystem = "pbs"; GlobusResourceContactString = "ce-02.roma3.infn.it:8443/cream-pbs"; QueueName = "cert"; ShallowRetryCount = 10; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 3; Executable = "openmpi-wrapper.sh"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000017:BH=0000000000:JSS=000006:LM=000022:LRMS=000000:APP=000000:LBS=000000"; Arguments = "hello"; AllowZippedISB = true; maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank =  -other.GlueCEStateEstimatedResponseTime; ce_id = "ce-02.roma3.infn.it:8443/cream-pbs-cert"; ZippedISB = { "ISBfiles_km6imNWXB17PDZruVO82Mg_0.tar.gz" }; OutputSandboxDestURI = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/output/hello.out","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/output/hello.err" }; OutputSandbox = { "hello.out","hello.err" }; CeRequirements = "( true && ( true && ( true && ( true && ( true && ( true && Member(\"OPENMPI\",other.GlueHostApplicationSoftwareRunTimeEnvironment) ) ) ) ) ) ) && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )"; InputSandbox = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/input/openmpi-wrapper.sh","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/input/hello.c","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/input/Makefile","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/input/.BrokerInfo" }; edg_jobid = "https://cream-45.pd.infn.it:9000/n2xH3q9eUcG1o0x9ZgFNEA"; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini"; StdError = "hello.err"; SMPGranularity = 3; InputSandboxPath = "/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/input"; CeApplicationDir = "/storage/local/exp_soft/dteam"; ReallyRunningToken = "gsiftp://cream-45.pd.infn.it/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/token.txt_0"; rank =  -other.GlueCEStateEstimatedResponseTime; WholeNodes = false; CpuNumber = 3; edg_previous_matches = { "grisuce.scope.unina.it","ce.scope.unina.it" }; WMSHostname = "cream-45.pd.infn.it"; lrms_type = "pbs"; MyProxyServer = "myproxy.cnaf.infn.it"; edg_previous_matches_ex = { [ timestamp = 1319039871; ce_id = "grisuce.scope.unina.it" ],[ timestamp = 1319039874; ce_id = "ce.scope.unina.it" ] }; X509UserProxy = "/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/user.proxy"; InputSandboxDestFileName = { "openmpi-wrapper.sh","hello.c","Makefile" }; requirements = ( ( ( ( ( ( Member("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*",other.GlueCEUniqueID) ) && ( other.GlueCEStateStatus == "Production" ) ) && other.GlueCEInfoTotalCPUs >= 3 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueCEInfoTotalCPUs >= 3 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) :  !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA" ]]

5)

jdl:

Executable = "openmpi-wrapper.sh";
Arguments="hello";
StdOutput  = "hello.out";
StdError   = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
InputSandbox = {"/home/fabiok/mpi/openmpi-wrapper.sh","/home/fabiok/mpi/hello.c","/home/fabiok/mpi/Makefile"};
Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID);
WholeNodes = true;
SMPGranularity = 8;
CpuNumber=8;
Hostnumber = 2;

Resulting cream jdl:

JDL            = [[ CEInfoHostName = "prabi-ce5-gpu.ibcp.fr"; OutputSandboxPath = "/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/output"; StdOutput = "hello.out"; BatchSystem = "pbs"; GlobusResourceContactString = "prabi-ce5-gpu.ibcp.fr:8443/cream-pbs"; QueueName = "dteam"; ShallowRetryCount = 10; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 3; Executable = "openmpi-wrapper.sh"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000005:BH=0000000000:JSS=000002:LM=000002:LRMS=000000:APP=000000:LBS=000000"; Arguments = "hello"; AllowZippedISB = true; maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank =  -other.GlueCEStateEstimatedResponseTime; ce_id = "prabi-ce5-gpu.ibcp.fr:8443/cream-pbs-dteam"; ZippedISB = { "ISBfiles_4xXBPYiJ8Z6ZeIyiIUREnw_0.tar.gz" }; OutputSandboxDestURI = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/output/hello.out","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/output/hello.err" }; OutputSandbox = { "hello.out","hello.err" }; CeRequirements = "( true && ( true && ( true && ( true && ( true && ( true && Member(\"OPENMPI\",other.GlueHostApplicationSoftwareRunTimeEnvironment) ) ) ) ) ) ) && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )"; InputSandbox = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/input/openmpi-wrapper.sh","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/input/hello.c","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/input/Makefile","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/input/.BrokerInfo" }; edg_jobid = "https://cream-45.pd.infn.it:9000/HsKoKN4FdQ3oGzyhzrxLuw"; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini"; StdError = "hello.err"; SMPGranularity = 8; InputSandboxPath = "/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/input"; CeApplicationDir = "/swareas/dteam"; ReallyRunningToken = "gsiftp://cream-45.pd.infn.it/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/token.txt_0"; rank =  -other.GlueCEStateEstimatedResponseTime; WholeNodes = true; CpuNumber = 8; WMSHostname = "cream-45.pd.infn.it"; lrms_type = "pbs"; MyProxyServer = "myproxy.cnaf.infn.it"; X509UserProxy = "/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/user.proxy"; InputSandboxDestFileName = { "openmpi-wrapper.sh","hello.c","Makefile" }; Hostnumber = 2; requirements = ( ( ( ( ( ( Member("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*",other.GlueCEUniqueID) ) && ( other.GlueCEStateStatus == "Production" ) ) && other.GlueCEInfoTotalCPUs >= 8 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueCEInfoTotalCPUs >= 8 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) :  !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw" ]]

Perusal job

[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 6 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 6: Submit a perusal job 


 BEWARE default min perusal interval is 1000 secs, so this phase could take many minutes 

===> 
===> Test: WMS-job-cycle.py
===> WMS: devel11.cnaf.infn.it
===> Started: 14:40:13
===> Ended  : 15:13:43
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111018144013.log

[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 6 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 6: Submit a perusal job 


 BEWARE default min perusal interval is 1000 secs, so this phase could take many minutes 

===> 
===> Test: WMS-job-cycle.py
===> WMS: devel11.cnaf.infn.it
===> Started: 16:47:08
===> Ended  : 17:13:33
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111018164708.log

[ale@cream-12 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u500
timeleft  : 9:50:38
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 10:49:38
uri       : voms.hellasgrid.gr:15004


[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 6
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Set 6: Submit a perusal job 


 BEWARE default min perusal interval is 1000 secs, so this phase could take many minutes 

===> 
===> Test: WMS-job-cycle.py
===> WMS: devel11.cnaf.infn.it
===> Started: 17:48:26
===> Ended  : 18:15:50
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111018174826.log



See gzipped log file

Delegation


[ale@cream-12 WMS-service]$ python WMS-delegation-test.py -c wms-command.conf -V dteam -d 3 -l -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test delegation operation 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Check --autm-delegation option 


 Test 2: Check --delegationid option 


 Test 3: Try to delegate with a short proxy and check the validity 


 Test 4: Works with expired proxy 

===> 
===> Test: WMS-delegation-test.py
===> WMS: devel11.cnaf.infn.it
===> Started: 15:07:22
===> Ended  : 15:08:42
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111019150722.log
[ale@cream-12 WMS-service]$ python WMS-delegation-test.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test delegation operation 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Check --autm-delegation option 


 Test 2: Check --delegationid option 


 Test 3: Try to delegate with a short proxy and check the validity 


 Test 4: Works with expired proxy 

===> 
===> Test: WMS-delegation-test.py
===> WMS: devel11.cnaf.infn.it
===> Started: 15:12:36
===> Ended  : 15:13:52
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111019151236.log
[ale@cream-12 WMS-service]$ myproxy-logon -d -s myproxy.cnaf.infn.it --voms dteam
Enter MyProxy pass phrase:
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
Contacting  voms2.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr] "dteam" Done
Creating proxy .........................9........................ Done

Your proxy is valid until Thu Oct 20 02:19:21 2011
A credential has been received for user /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle in /tmp/x509up_u500.
[ale@cream-12 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u500
timeleft  : 10:59:50
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 11:58:50
uri       : voms2.hellasgrid.gr:15004
[ale@cream-12 WMS-service]$ python WMS-delegation-test.py -c wms-command.conf -V dteam -d 3 -l 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test delegation operation 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Test 1: Check --autm-delegation option 


 Test 2: Check --delegationid option 


 There are other two tests which require the user proxy password. Use -i option to enable them 

===> 
===> Test: WMS-delegation-test.py
===> WMS: devel11.cnaf.infn.it
===> Started: 15:19:37
===> Ended  : 15:19:42
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111019151937.log


See gzipped log file

Job List-Match

Without data

[ale@cream-12 WMS-service]$ python WMS-listmatch-test.py -c wms-command.conf -V dteam -d 3 -l -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job ListMatch Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: check if glite-wms-job-list-match command exists 


 Test 2: test a simple matching 


 Test 3: check --delegationid option 


 Test 4: exclude a CE from the match 


 Test 5: try a failure matching (Requirements == false) 

===> 
===> Test: WMS-listmatch-test.py
===> WMS: devel11.cnaf.infn.it
===> Started: 12:28:10
===> Ended  : 12:28:24
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111014122810.log

[ale@cream-12 WMS-service]$ python WMS-listmatch-test.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job ListMatch Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: check if glite-wms-job-list-match command exists 


 Test 2: test a simple matching 


 Test 3: check --delegationid option 


 Test 4: exclude a CE from the match 


 Test 5: try a failure matching (Requirements == false) 

===> 
===> Test: WMS-listmatch-test.py
===> WMS: devel11.cnaf.infn.it
===> Started: 12:28:44
===> Ended  : 12:28:57
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111014122844.log

[ale@cream-12 WMS-service]$ voms-proxy-init -rfc --voms dteam
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
Creating temporary proxy ........................ Done
Contacting  voms2.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr] "dteam" Done
Creating proxy .................................................................. Done

Your proxy is valid until Sat Oct 15 00:36:35 2011
[ale@cream-12 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=796998768
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
type      : RFC compliant proxy
strength  : 1024 bits
path      : /tmp/x509up_u500
timeleft  : 11:59:53
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 11:59:52
uri       : voms2.hellasgrid.gr:15004

[ale@cream-12 WMS-service]$ python WMS-listmatch-test.py -c wms-command.conf -V dteam -d 3 -l 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job ListMatch Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Test 1: check if glite-wms-job-list-match command exists 


 Test 2: test a simple matching 


 Test 3: check --delegationid option 


 Test 4: exclude a CE from the match 


 Test 5: try a failure matching (Requirements == false) 

===> 
===> Test: WMS-listmatch-test.py
===> WMS: devel11.cnaf.infn.it
===> Started: 12:37:09
===> Ended  : 12:37:19
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111014123709.log

[ale@cream-12 WMS-service]$ myproxy-init -d -s myproxy.cnaf.infn.it
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
Creating proxy ................................ Done
Proxy Verify OK
Your proxy is valid until: Fri Oct 21 12:46:06 2011
Enter MyProxy pass phrase:
Verifying - Enter MyProxy pass phrase:
A proxy valid for 168 hours (7.0 days) for user /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle now exists on myproxy.cnaf.infn.it.

[ale@cream-12 WMS-service]$ myproxy-logon -d -s myproxy.cnaf.infn.it --voms dteam
Enter MyProxy pass phrase:
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
Contacting  voms2.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr] "dteam" Done
Creating proxy ...................................... Done

Your proxy is valid until Fri Oct 14 23:47:19 2011
A credential has been received for user /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle in /tmp/x509up_u500.

[ale@cream-12 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u500
timeleft  : 10:50:45
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 11:49:45
uri       : voms2.hellasgrid.gr:15004

[ale@cream-12 WMS-service]$ python WMS-listmatch-test.py -c wms-command.conf -V dteam -d 3 -l 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job ListMatch Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Test 1: check if glite-wms-job-list-match command exists 


 Test 2: test a simple matching 


 Test 3: check --delegationid option 


 Test 4: exclude a CE from the match 


 Test 5: try a failure matching (Requirements == false) 

===> 
===> Test: WMS-listmatch-test.py
===> WMS: devel11.cnaf.infn.it
===> Started: 12:48:18
===> Ended  : 12:48:27
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111014124818.log


See gzipped log file

With data

Two files have been registered on the SE srm-public.cern.ch:

find an lfc for the dteam vo:
lcg-infosites --vo dteam lfc

set the lfc host:
export LFC_HOST=prod-lfc-shared-central.cern.ch

create the directory:
lfc-mkdir /grid/dteam/fcapanni

create the files:
lcg-cr --vo dteam -d srm://srm-public.cern.ch:8443/castor/cern.ch/grid/dteam/fcapanni/e-2M.txt -l lfn:/grid/dteam/fcapanni/e-2M.txt "file://$PWD/e-2M.txt"
lcg-cr --vo dteam -d srm://srm-public.cern.ch:8443/castor/cern.ch/grid/dteam/fcapanni/PI_1M.txt -l lfn:/grid/dteam/fcapanni/PI_1M.txt "file://$PWD/PI_1M.txt"


Then the following jdl was submitted:

###########################################
#      JDL with Data Requirements         #
###########################################

Executable = "calc-pi.sh";
# Arguments is the number of digits, must be < 1000000
Arguments = "1000";
StdOutput = "std.out";
StdError = "std.err";
Prologue = "prologue.sh";

FuzzyRank = true;

InputSandbox = {"calc-pi.sh", "fileA", "fileB","prologue.sh"};
OutputSandbox = {"std.out", "std.err","out-PI.txt","out-e.txt"};

#Requirements = other.GlueCEInfoHostName != "spacin-ce1.dma.unina.it";
Requirements = true;

DataRequirements = {
[
DataCatalogType = "DLI";
DataCatalog = "http://prod-lfc-shared-central.cern.ch:8085";
InputData = {"lfn:/grid/dteam/fcapanni/PI_1M.txt","lfn:/grid/dteam/fcapanni/e-2M.txt"};
]
};

DataAccessProtocol = "gsiftp";
VirtualOrganisation = "dteam";

RetryCount = 1;
ShallowRetryCount = 2;

The job was assigned to the CE ce133.cern.ch.

Verification that the SE srm-public.cern.ch is close to the CE ce133.cern.ch:

[fabiok@ui DataReq]$ lcg-infosites --vo dteam closeSE|grep -A 5 ce133
Name of the CE: ce133.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam
    srm-alice.cern.ch
    srm-atlas.cern.ch
    srm-cms.cern.ch
    srm-lhcb.cern.ch
    srm-public.cern.ch

Gangmatching

Tried a list-match considering the following JDL:

[
executable="ssh1.sh";
myproxyserver="";
SErequirements = anyMatch(other.storage.CloseSEs, target.GlueSAStateAvailableSpace > 20);
CErequirements = other.GlueCEStateStatus == "Production" && other.GlueCEInfoTotalCPUs >= 2;
requirements = CErequirements && SErequirements;
inputsandbox={"ssh1.sh"};
stdoutput="out3.out";
stderror="err2.err";
outputsandbox={"out3.out", "err2.err", "ssh1.sh", "a"};
]


$ glite-wms-job-list-match -a -c cream44.conf gang.jdl

Connecting to the service https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server

==========================================================================

           COMPUTING ELEMENT IDs LIST 
 The following CE(s) matching your job requirements have been found:

   *CEId*
 - cccreamceli05.in2p3.fr:8443/cream-sge-long
 - cccreamceli05.in2p3.fr:8443/cream-sge-medium
 - cccreamceli05.in2p3.fr:8443/cream-sge-short
 - cccreamceli06.in2p3.fr:8443/cream-sge-long
 - cccreamceli06.in2p3.fr:8443/cream-sge-medium
 - cccreamceli06.in2p3.fr:8443/cream-sge-short
 - cccreamceli10.in2p3.fr:8443/cream-sge-long
 - cccreamceli10.in2p3.fr:8443/cream-sge-medium
 - cccreamceli10.in2p3.fr:8443/cream-sge-short
 - cmsce01.na.infn.it:2119/jobmanager-lcgpbs-cert
 - grid010.ct.infn.it:2119/jobmanager-lcgpbs-cert
 - inaf-ce-01.ct.pi2s2.it:8443/cream-lsf-cert
 - virgo-ce.roma1.infn.it:2119/jobmanager-lcgpbs-cert
 - cccreamceli09.in2p3.fr:8443/cream-sge-long
 - cccreamceli09.in2p3.fr:8443/cream-sge-medium
 - cccreamceli09.in2p3.fr:8443/cream-sge-short
 - ce-01.grid.sissa.it:8443/cream-pbs-blade
 - cccreamceli02.in2p3.fr:8443/cream-bqs-short
 - cccreamceli02.in2p3.fr:8443/cream-bqs-medium
 - unict-dmi-ce-01.ct.pi2s2.it:2119/jobmanager-lcglsf-cert
 - cream-ce.pg.infn.it:8443/cream-pbs-cert

==========================================================================

Manually checked in the bdii some of matched CE and verified that they really meet the constraints.

Tried also modifying the JDL setting:

requirements = CErequirements;

Many more CEs get matched with a list-match.

Prologue and Epilogue jobs

[ale@cream-12 WMS-service]$ python WMS-prologue-epilogue.py -c wms-command.conf -V dteam -d 3 -l  -i 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS test prologue and epilogue attributes 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Test prologue attribute 


 Test 2: Test epilogue attribute 


 Test 3: Test prologue and epilogue attributes 

===> 
===> Test: WMS-prologue-epilogue.py
===> WMS: devel11.cnaf.infn.it
===> Started: 18:29:24
===> Ended  : 18:36:49
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111019182924.log
[ale@cream-12 WMS-service]$ python WMS-prologue-epilogue.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l  -i 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS test prologue and epilogue attributes 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Test prologue attribute 


 Test 2: Test epilogue attribute 


 Test 3: Test prologue and epilogue attributes 

===> 
===> Test: WMS-prologue-epilogue.py
===> WMS: devel11.cnaf.infn.it
===> Started: 09:40:11
===> Ended  : 09:47:05
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111020094011.log

[ale@cream-12 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u500
timeleft  : 10:59:57
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 11:58:56
uri       : voms2.hellasgrid.gr:15004
[ale@cream-12 WMS-service]$ python WMS-prologue-epilogue.py -c wms-command.conf -V dteam -d 3 -l  
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS test prologue and epilogue attributes 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Test 1: Test prologue attribute 


 Test 2: Test epilogue attribute 


 Test 3: Test prologue and epilogue attributes 

===> 
===> Test: WMS-prologue-epilogue.py
===> WMS: devel11.cnaf.infn.it
===> Started: 10:01:01
===> Ended  : 10:08:25
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111020100101.log


See gzipped log file

Shallow and deep re-submission

[ale@cream-12 WMS-service]$ python WMS-job-resubmit.py -c wms-command.conf -V dteam -d 3 -l -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job Resubmission Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Try a shallow resubmission 


 Test 2: Try a deep resubmission 

===> 
===> Test: WMS-job-resubmit.py
===> WMS: devel11.cnaf.infn.it
===> Started: 15:57:18
===> Ended  : 16:31:38
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111019155718.log

[ale@cream-12 WMS-service]$ python WMS-job-resubmit.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job Resubmission Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Try a shallow resubmission 


 Test 2: Try a deep resubmission 

===> 
===> Test: WMS-job-resubmit.py
===> WMS: devel11.cnaf.infn.it
===> Started: 17:53:58
===> Ended  : 18:09:10
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111019175358.log

[ale@cream-12 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u500
timeleft  : 10:29:06
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 11:28:05
uri       : voms2.hellasgrid.gr:15004


[ale@cream-12 WMS-service]$ python WMS-job-resubmit.py -c wms-command.conf -V dteam -d 3 -l 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job Resubmission Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Test 1: Try a shallow resubmission 


 Test 2: Try a deep resubmission 

===> 
===> Test: WMS-job-resubmit.py
===> WMS: devel11.cnaf.infn.it
===> Started: 18:10:04
===> Ended  : 18:24:43
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111019181004.log

See gzipped log file

WMS Job Cancel Testing

[ale@cream-12 WMS-service]$ python WMS-job-cancel.py -c wms-command.conf -V dteam -d 3 -l  -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job Cancel Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Try to cancel a normal job 


 Test 2: Try to cancel a DAG job 


 Test 3: Try to cancel one node of a DAG job 


 Test 4: Try to cancel a collection job 


 Test 5: Try to cancel one node of a collection job 


 Test 6: Try to cancel a parametric job 


 Test 7: Try to cancel one node of a parametric job 


 Test 8: Try to cancel a MPI job 

===> 
===> Test: WMS-job-cancel.py
===> WMS: devel11.cnaf.infn.it
===> Started: 17:54:06
===> Ended  : 17:58:22
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111019175406.log

[ale@cream-12 WMS-service]$ python WMS-job-cancel.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l  -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job Cancel Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Try to cancel a normal job 


 Test 2: Try to cancel a DAG job 


 Test 3: Try to cancel one node of a DAG job 


 Test 4: Try to cancel a collection job 


 Test 5: Try to cancel one node of a collection job 


 Test 6: Try to cancel a parametric job 


 Test 7: Try to cancel one node of a parametric job 


 Test 8: Try to cancel a MPI job 

===> 
===> Test: WMS-job-cancel.py
===> WMS: devel11.cnaf.infn.it
===> Started: 10:58:45
===> Ended  : 11:07:04
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111020105845.log

[ale@cream-12 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u500
timeleft  : 9:52:14
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 10:51:13
uri       : voms2.hellasgrid.gr:15004

[ale@cream-12 WMS-service]$ python WMS-job-cancel.py -c wms-command.conf -V dteam -d 3 -l  
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job Cancel Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Test 1: Try to cancel a normal job 


 Test 2: Try to cancel a DAG job 


 Test 3: Try to cancel one node of a DAG job 


 Test 4: Try to cancel a collection job 


 Test 5: Try to cancel one node of a collection job 


 Test 6: Try to cancel a parametric job 


 Test 7: Try to cancel one node of a parametric job 


 Test 8: Try to cancel a MPI job 

===> 
===> Test: WMS-job-cancel.py
===> WMS: devel11.cnaf.infn.it
===> Started: 11:21:13
===> Ended  : 11:32:02
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111020112113.log

See gzipped log file

Proxy renewal

[ale@cream-12 WMS-service]$ python WMS-proxy-renewal.py -c wms-command.conf -V dteam -d 3 -l  -i 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS test proxy renewal operation 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Test proxy renewal with LCG CE 


 Test 2: Test proxy renewal with CREAM CE 


 Test 3: Test proxy renewal with LCG CE (without setting MYPROXYSERVER) 


 Test 4: Test proxy renewal with CREAM CE (without setting MYPROXYSERVER) 

===> ERROR: Timeout reached while waiting the job https://devel19.cnaf.infn.it:9000/Ny0-wEesVaLnUjwdNP1lhA to finish
===> 
===> Test: WMS-proxy-renewal.py
===> WMS: devel11.cnaf.infn.it
===> Started: 12:04:35
===> Ended  : 13:45:07
===> 
===>     >>> TEST FAILED <<<
===> 
===>  >>> failure reason: 1 test(s) fail(s): ['Test 4: Test proxy renewal with CREAM CE (without setting MYPROXYSERVER)'] <<< 
===> 
===> Test log file is WMSService-TS_20111020120435.log
===> Error messages have been written in /home/ale/wms_tests/WMS-service/WMSService-Test_20111020120435/errors.log
===> 
===>  Test directory /home/ale/wms_tests/WMS-service/WMSService-Test_20111020120435 has not been cleaned for debug purpose


[ale@cream-12 WMS-service]$ python WMS-proxy-renewal.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l  -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS test proxy renewal operation 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Test proxy renewal with LCG CE 


 Test 2: Test proxy renewal with CREAM CE 


 Test 3: Test proxy renewal with LCG CE (without setting MYPROXYSERVER) 


 Test 4: Test proxy renewal with CREAM CE (without setting MYPROXYSERVER) 

===> ERROR: Timeout reached while waiting the job https://devel19.cnaf.infn.it:9000/ATCKDsyFkq3ayqmPmvx3iA to finish
===> 
===> Test: WMS-proxy-renewal.py
===> WMS: devel11.cnaf.infn.it
===> Started: 13:26:35
===> Ended  : 14:36:44
===> 
===>     >>> TEST FAILED <<<
===> 
===>  >>> failure reason: 1 test(s) fail(s): ['Test 4: Test proxy renewal with CREAM CE (without setting MYPROXYSERVER)'] <<< 
===> 
===> Test log file is WMSService-TS_20111020132635.log
===> Error messages have been written in /home/ale/wms_tests/WMS-service/WMSService-Test_20111020132635/errors.log
===> 
===>  Test directory /home/ale/wms_tests/WMS-service/WMSService-Test_20111020132635 has not been cleaned for debug purpose



NB Last Test fail for bug #84839

See gzipped log file

Configuration file

Verified that in the configuration file /etc/glite-wms/glite_wms.conf there are these hard-coded values:

For the common section:

DGUser = "\${GLITE_WMS_USER}"
HostProxyFile = "\${WMS_LOCATION_VAR}/glite/wms.proxy"
LBProxy = true

For the JobController section:

CondorSubmit = "${CONDORG_INSTALL_PATH}/bin/condor_submit"
CondorRemove = "${CONDORG_INSTALL_PATH}/bin/condor_rm"
CondorQuery = "${CONDORG_INSTALL_PATH}/bin/condor_q"
CondorRelease = "${CONDORG_INSTALL_PATH}/bin/condor_release"
CondorDagman = "${CONDORG_INSTALL_PATH}/bin/condor_dagman"
DagmanMaxPre = 10
SubmitFileDir = "${WMS_LOCATION_VAR}/jobcontrol/submit"
OutputFileDir = "${WMS_LOCATION_VAR}/jobcontrol/condorio"
InputType = "jobdir"
Input = "${WMS_LOCATION_VAR}/jobcontrol/jobdir/"
LockFile = "${WMS_LOCATION_VAR}/jobcontrol/lock"
LogFile = "\${WMS_LOCATION_LOG}/jobcontoller_events.log"
LogLevel = 5
MaximumTimeAllowedForCondorMatch = 1800
ContainerRefreshThreshold = 1000

For the NetworkServer section:

II_Port  = 2170
Gris_Port = 2170
II_Timeout = 100
Gris_Timeout = 20
II_DN = "mds-vo-name=local, o=grid"
Gris_DN = "mds-vo-name=local, o=grid"
BacklogSize = 64
ListeningPort = 7772
MasterThreads = 8
DispatcherThreads = 10
SandboxStagingPath = "${WMS_LOCATION_VAR}/SandboxDir"
LogFile = "${WMS_LOCATION_LOG}/networkserver_events.log"
LogLevel = 5
EnableQuotaManagement = false
MaxInputSandboxSize = 10000000
EnableDynamicQuotaAdjustment = false
QuotaAdjustmentAmount = 10000
QuotaInsensibleDiskPortion = 2.0
DLI_SI_CatalogTimeout = 60
ConnectionTimeout = 300

For the LogMonitor section:

JobsPerCondorLog = 1000
LockFile = "${WMS_LOCATION_VAR}/logmonitor/lock"
LogFile = "${WMS_LOCATION_LOG}/logmonitor_events.log"
LogLevel = 5
ExternalLogFile = "\${WMS_LOCATION_LOG}/logmonitor_external.log"
MainLoopDuration = 5
CondorLogDir = "${WMS_LOCATION_VAR}/logmonitor/CondorG.log"
CondorLogRecycleDir = "${WMS_LOCATION_VAR}/logmonitor/CondorG.log/recycle"
MonitorInternalDir = "${WMS_LOCATION_VAR}/logmonitor/internal"
IdRepositoryName = "irepository.dat"
AbortedJobsTimeout = 600
GlobusDownTimeout = 7200
RemoveJobFiles = true
ForceCancellationRetries = 2

For the Workloadmanager section:

PipeDepth = 200
WorkerThreads = 5
DispatcherType = "jobdir"
Input = "${WMS_LOCATION_VAR}/workload_manager/jobdir"
LogLevel = 5
LogFile  = "${WMS_LOCATION_LOG}/workload_manager_events.log"
MaxRetryCount = 10
CeMonitorServices = {}
CeMonitorAsynchPort = 0
IsmBlackList = {}
IsmUpdateRate = 600
IsmIiPurchasingRate = 480
JobWrapperTemplateDir = "${WMS_JOBWRAPPER_TEMPLATE}"
IsmThreads = false
IsmDump = "${WMS_LOCATION_VAR}/workload_manager/ismdump.fl"
SiServiceName = "org.glite.SEIndex"
DliServiceName = "data-location-interface"
MaxRetryCount = 10
DisablePurchasingFromGris = true
EnableBulkMM = true
CeForwardParameters = {"GlueHostMainMemoryVirtualSize","GlueHostMainMemoryRAMSize","GlueCEPolicyMaxCPUTime"}
MaxOutputSandboxSize = -1
EnableRecovery = true
QueueSize = 1000
ReplanGracePeriod = 3600
MaxReplansCount = 5
WmsRequirements  = ((ShortDeadlineJob =?= TRUE) ? RegExp(".*sdj$", other.GlueCEUniqueID) : !RegExp(".*sdj$", other.GlueCEUniqueID)) && (other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs) && (EnableWmsFeedback =?= TRUE ? RegExp("cream", other.GlueCEImplementationName, "i") : true)

For the WorkloadManagerProxy:

SandboxStagingPath = "${WMS_LOCATION_VAR}/SandboxDir"
LogFile = "${WMS_LOCATION_LOG}/wmproxy.log"
LogLevel = 5
MaxInputSandboxSize = 100000000
ListMatchRootPath = "/tmp"
GridFTPPort = 2811
LBLocalLogger = "localhost:9002"
MinPerusalTimeInterval = 1000
AsyncJobStart = true
EnableServiceDiscovery = false
LBServiceDiscoveryType = "org.glite.lb.server"
ServiceDiscoveryInfoValidity = 3600
WeightsCacheValidity = 86400
MaxServedRequests = 50
OperationLoadScripts = [
jobRegister = "${WMS_LOCATION_SBIN}/glite_wms_wmproxy_load_monitor --oper jobRegister --load1 22 --load5 20 --load15 18 --memusage 99 --diskusage 95 --fdnum 1000 --jdnum 1500 --ftpconn 300"
jobSubmit = "${WMS_LOCATION_SBIN}/glite_wms_wmproxy_load_monitor --oper jobSubmit --load1 22 --load5 20 --load15 18 --memusage 99 --diskusage 95 --fdnum 1000 --jdnum 1500  --ftpconn 300"
RuntimeMalloc = "/usr/lib64/libtcmalloc_minimal.so"
]

For the ICE section:

start_listener  =  false
start_lease_updater  =  false
logfile  =  "${WMS_LOCATION_LOG}/ice.log"
log_on_file = true
creamdelegation_url_prefix  =  "https://"
listener_enable_authz  =  true
poller_status_threshold_time  =  30*60
ice_topic  =  "CREAM_JOBS"
subscription_update_threshold_time  =  3600
lease_delta_time  =  0
notification_frequency  =  3*60
start_proxy_renewer  =  true
max_logfile_size  =  100*1024*1024
ice_host_cert  =  "${GLITE_HOST_CERT}"
Input  =  "${WMS_LOCATION_VAR}/ice/jobdir"
job_cancellation_threshold_time  =  300
poller_delay  =  2*60
persist_dir  =  "${WMS_LOCATION_VAR}/ice/persist_dir"
lease_update_frequency  =  20*60
log_on_console = false
cream_url_postfix  =  "/ce-cream/services/CREAM2"
subscription_duration  =  86400
bulk_query_size  =  100
purge_jobs  =  false
InputType  =  "jobdir"
listener_port  =  7010
listener_enable_authn  =  true
ice_host_key  =  "${GLITE_HOST_KEY}"
start_poller  =  true
creamdelegation_url_postfix  =  "/ce-cream/services/gridsite-delegation"
cream_url_prefix  =  "https://"
max_ice_threads  =  10
cemon_url_prefix  =  "https://"
start_subscription_updater  =  true
proxy_renewal_frequency  =  600
ice_log_level  =  700
soap_timeout  =  60
max_logfile_rotations  =  10
cemon_url_postfix  =  "/ce-monitor/services/CEMonitor"
max_ice_mem = 2096000
ice_empty_threshold = 600

Verified that:

  • The attribute II_Contact of NetworkServer section matches the value of the yaim variable BDII_HOST
  • The attribute WMExpiryPeriod of WorkloadManager section matches the value of yaim variable WMS_EXPIRY_PERIOD
  • The attribute MatchRetryPeriod of WorkloadManager section matches the value of yaim variable WMS_MATCH_RETRY_PERIOD
  • The attribute IsmIiLDAPCEFilterExt of WorkloadManager section is (|(GlueCEAccessControlBaseRule=VO:vo1)(GlueCEAccessControlBaseRule=VOMS:/vo1/*)(GlueCEAccessControlBaseRule=VO=vo2...
  • The attribute LBServer of the WorkloadManagerProxy section matches the value of yaim variable LB_HOST

WMS feedback

The following lines are inserted into glite_wms.conf and the wm is restarted:

MaxReplansCount  =  5;
ReplanGracePeriod  =  10;

Then a bunch of jobs with the following jdl is submitted on a restricted number of CREAM CEs:

[
Executable = "/bin/sleep";
Arguments = "3600";
retrycount = 0;
shallowretrycount = 0;
StdError = "stderr.log";
StdOutput = "stdout.log";
OutputSandbox = {"stderr.log", "stdout.log"};
requirements = RegExp(".*ce203.cern.ch.*", other.GlueCEUniqueID) || RegExp(".*ce204.cern.ch.*", other.GlueCEUniqueID);
EnableWMSFeedback = true;
]

After a while the jobs end up in the Scheduled state and the replan mechanism is triggered. These lines show up in workload_manager_events.log:

21 Oct, 10:55:52 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-manager-3.3.1/build/../src/replanner.cpp:125): replanning job https://devel19.cnaf.infn.it:9000/4vP_w_TcK5NCF2L-ZiSetg
21 Oct, 10:55:52 -D: [Debug] operator()(/builddir/build/BUILD/glite-wms-manager-3.3.1/build/../src/replanner.cpp:177): present token number for job https://devel19.cnaf.infn.it:9000/4vP_w_TcK5NCF2L-ZiSetg is 0
21 Oct, 10:55:52 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-manager-3.3.1/build/../src/replanner.cpp:237): created replanning request for job https://devel19.cnaf.infn.it:9000/4vP_w_TcK5NCF2L-ZiSetg with token /var/SandboxDir/4v/https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f4vP_5fw_5fTcK5NCF2L-ZiSetg/token.txt_1
21 Oct, 10:55:53 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-manager-3.3.1/build/../src/dispatcher_utils.cpp:228): new jobsubmit for https://devel19.cnaf.infn.it:9000/4vP_w_TcK5NCF2L-ZiSetg
21 Oct, 10:55:53 -D: [Debug] operator()(/builddir/build/BUILD/glite-wms-manager-3.3.1/build/../src/submit_request.cpp:239): considering (re)submit of https://devel19.cnaf.infn.it:9000/4vP_w_TcK5NCF2L-ZiSetg
21 Oct, 10:55:53 -D: [Debug] operator()(/builddir/build/BUILD/glite-wms-manager-3.3.1/build/../src/submit_request.cpp:458): found token number 1 for job https://devel19.cnaf.infn.it:9000/4vP_w_TcK5NCF2L-ZiSetg
21 Oct, 10:55:53 -D: [Debug] operator()(/builddir/build/BUILD/glite-wms-manager-3.3.1/build/../src/submit_request.cpp:509): this is a reschedule for job https://devel19.cnaf.infn.it:9000/4vP_w_TcK5NCF2L-ZiSetg

And a replan resubmission is logged for the corresponding job. A logging-info shows the following:

Event: Resubmission
- Arrived                    =    Fri Oct 21 10:55:52 2011 CEST
- Host                       =    devel11.cnaf.infn.it
- Reason                     =    token still exists
- Result                     =    SHALLOW
- Source                     =    WorkloadManager
- Src instance               =    29132
- Tag                        =    job will be replanned
- Timestamp                  =    Fri Oct 21 10:55:52 2011 CEST
- User                       =    /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy  

Limiter mechanism

The following line is inserted into glite_wms.conf and the wmproxy is restarted:

jobSubmit  =  "${WMS_LOCATION_SBIN}/glite_wms_wmproxy_load_monitor --oper jobSubmit --load1 0.001 --load5 20 --load15 18 --memusage 99 --diskusage 95 --fdnum 1000 --jdnum 150000  --ftpconn 300";

then trying to submit a job the following message is returned:

[fabiok@ui ~]$ glite-wms-job-submit -a -c wms_devel11.conf echo.jdl 

Connecting to the service https://devel11.cnaf.infn.it:7443/glite_wms_wmproxy_server


Warning - Unable to submit the job to the service: https://devel11.cnaf.infn.it:7443/glite_wms_wmproxy_server
System load is too high:
Threshold for Load Average(1 min): 0.001 => Detected value for Load Average(1 min):  0.04


Method: jobSubmit


Error - Operation failed
Unable to find any endpoint where to perform service request

If the following line is inserted:

jobSubmit  =  "${WMS_LOCATION_SBIN}/glite_wms_wmproxy_load_monitor --oper jobSubmit --load1 22 --load5 0.01 --load15 18 --memusage 99 --diskusage 95 --fdnum 1000 --jdnum 150000  --ftpconn 300";

the following message is returned:

[fabiok@ui ~]$ glite-wms-job-submit -a -c wms_devel11.conf echo.jdl 

Connecting to the service https://devel11.cnaf.infn.it:7443/glite_wms_wmproxy_server


Warning - Unable to submit the job to the service: https://devel11.cnaf.infn.it:7443/glite_wms_wmproxy_server
System load is too high:
Threshold for Load Average(5 min): 0.01 => Detected value for Load Average(5 min):  0.14


Method: jobSubmit


Error - Operation failed
Unable to find any endpoint where to perform service request

If the following parameters are inserted:

jobSubmit  =  "${WMS_LOCATION_SBIN}/glite_wms_wmproxy_load_monitor --oper jobSubmit --load1 22 --load5 20 --load15 18 --memusage 1 --diskusage 95 --fdnum 1000 --jdnum 150000  --ftpconn 300";

the following message is returned:

[fabiok@ui ~]$ glite-wms-job-submit -a -c wms_devel11.conf echo.jdl 

Connecting to the service https://devel11.cnaf.infn.it:7443/glite_wms_wmproxy_server


Warning - Unable to submit the job to the service: https://devel11.cnaf.infn.it:7443/glite_wms_wmproxy_server
System load is too high:
Threshold for Memory Usage: 1 => Detected value for Memory Usage: 86.02%


Method: jobSubmit


Error - Operation failed
Unable to find any endpoint where to perform service request

If the following parameters are inserted:

jobSubmit  =  "${WMS_LOCATION_SBIN}/glite_wms_wmproxy_load_monitor --oper jobSubmit --load1 22 --load5 20 --load15 18 --memusage 99 --diskusage 1 --fdnum 1000 --jdnum 150000  --ftpconn 300";

The following message is returned:

[fabiok@ui ~]$ glite-wms-job-submit -a -c wms_devel11.conf echo.jdl 

Connecting to the service https://devel11.cnaf.infn.it:7443/glite_wms_wmproxy_server


Warning - Unable to submit the job to the service: https://devel11.cnaf.infn.it:7443/glite_wms_wmproxy_server
System load is too high:
Threshold for Disk Usage: 1% => Detected value for Partition / : 13%


Method: jobSubmit


Error - Operation failed
Unable to find any endpoint where to perform service request

If the following parameters are inserted:

jobSubmit  =  "${WMS_LOCATION_SBIN}/glite_wms_wmproxy_load_monitor --oper jobSubmit --load1 22 --load5 20 --load15 18 --memusage 99 --diskusage 95 --fdnum 1000 --jdnum 150000  --ftpconn 0";

The following message is returned:

[fabiok@ui ~]$ glite-wms-job-submit -a -c wms_devel11.conf echo.jdl 

Connecting to the service https://devel11.cnaf.infn.it:7443/glite_wms_wmproxy_server


Warning - Unable to submit the job to the service: https://devel11.cnaf.infn.it:7443/glite_wms_wmproxy_server
System load is too high:
Threshold for FTP Connection: 0 => Detected value for FTP Connection: 1


Method: jobSubmit


Error - Operation failed
Unable to find any endpoint where to perform service request

As far as the other parameters are concerned, Bug #88024 has been opened:

1) It seems that glite_wms_wmproxy_load_monitor is called by wmproxy only for
the jobSubmit operation and not for the jobRegister even if the configuration
file reports the latter too.

2) If called by wmproxy with certain values for the parameters it seems to not take
them into account, therefore jobs are submitted even if some
threshold is exceeded. For example, by setting --fdnum 100 :

21 Oct, 12:31:07 -D- PID: 24147 - "wmpcommon::callLoadScriptFile": Executing
command:  /usr/sbin/glite_wms_wmproxy_load_monitor --oper jobSubmit --load1
22 -
-load5 20 --load15 18 --memusage 99 --diskusage 95 --fdnum 100 --jdnum 150000
--ftpconn 300
21 Oct, 12:31:07 -D- PID: 24147 - "wmpcommon::callLoadScriptFile": Script
error file: /tmp/wmpscriptcall.err.24147
21 Oct, 12:31:07 -D- PID: 24147 - "wmpcommon::callLoadScriptFile": Executing
load script file: /usr/sbin/glite_wms_wmproxy_load_monitor

If launched explicitly the script doesn't seem to fail.

Purging

Normal job cycle

Submit a job to a ce LCG:

[ale@cream-12 UI]$ glite-wms-job-submit -a   -c /home/ale/UI/etc/wmp_cream-45.conf test.jdl 

Connecting to the service https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://cream-45.pd.infn.it:9000/JDtP4ObMP5A_FCWjR6Eh4g

==========================================================================


Check the SBD and the others file used by the services:

[root@cream-45 ~]# ls -l /var/SandboxDir/JD/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fJDtP4ObMP5A_5fFCWjR6Eh4g/
total 24
drwxrwx--- 2 dteam009 glite 4096 Oct 21 15:00 input
-rw-r--r-- 1 glite    glite  703 Oct 21 15:00 JDLOriginal
-rw-r--r-- 1 glite    glite 2261 Oct 21 15:00 JDLStarted
drwxrwx--- 2 dteam009 glite 4096 Oct 21 15:00 output
drwxrwx--- 2 dteam009 glite 4096 Oct 21 15:00 peek
-rw-r--r-- 1 glite    glite    0 Oct 21 15:00 token.txt_0
lrwxrwxrwx 1 glite    glite   64 Oct 21 15:00 user.proxy -> /var/glite/spool/glite-renewd/3f82b966e8a77413044be1a9144a4af4.0


[root@cream-45 ~]# ls -l /var/jobcontrol/condorio/JD
total 4
drwxr-xr-x 2 glite glite 4096 Oct 21 15:00 https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fJDtP4ObMP5A_5fFCWjR6Eh4g

[root@cream-45 ~]# ls -l /var/jobcontrol/submit/JD/
total 44
-rw-r--r-- 1 glite glite  1784 Oct 21 15:00 Condor.https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fJDtP4ObMP5A_5fFCWjR6Eh4g.submit
-rw-r--r-- 1 glite glite 40672 Oct 21 15:00 JobWrapper.https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fJDtP4ObMP5A_5fFCWjR6Eh4g.sh


Wait until job finishes:

[ale@cream-12 WMS-service]$ glite-wms-job-status https://cream-45.pd.infn.it:9000/JDtP4ObMP5A_FCWjR6Eh4g


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://cream-45.pd.infn.it:9000/JDtP4ObMP5A_FCWjR6Eh4g
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job terminated successfully
Destination:        cream-11.pd.infn.it:2119/jobmanager-lcglsf-creamtest2
Submitted:          Fri Oct 21 15:00:35 2011 CEST
==========================================================================

Check on WMS auxiliary files should be removed:

[root@cream-45 ~]# ls -l /var/jobcontrol/submit/JD/
total 0
[root@cream-45 ~]# ls -l /var/jobcontrol/condorio/JD/
total 0

Retrieve job oputput:

[ale@cream-12 UI]$ glite-wms-job-output https://cream-45.pd.infn.it:9000/JDtP4ObMP5A_FCWjR6Eh4g

Connecting to the service https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server


================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://cream-45.pd.infn.it:9000/JDtP4ObMP5A_FCWjR6Eh4g
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/ale_JDtP4ObMP5A_FCWjR6Eh4g

================================================================================

Check that also the SBD has been removed on WMS:

[root@cream-45 ~]# ls -l /var/SandboxDir/JD/
total 0

-----------

Submit a job to a CREAM CE without setting myproxyserver:

[ale@cream-12 UI]$ glite-wms-job-submit -a   -c /home/ale/UI/etc/wmp_cream-45.conf test.jdl 

Connecting to the service https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://cream-45.pd.infn.it:9000/N9AYUlkkvG6RlRhyt_JaHA

==========================================================================

Check the creation of the SBD on WMS:

[root@cream-45 ~]# ls -l /var/SandboxDir/JD/
total 0
[root@cream-45 ~]# ls /var/SandboxDir/N9/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fN9AYUlkkvG6RlRhyt_5fJaHA/
input  JDLOriginal  JDLStarted  output  peek  user.proxy

Wait until job finishesi and retrieve output files:

[ale@cream-12 WMS-service]$ glite-wms-job-status https://cream-45.pd.infn.it:9000/N9AYUlkkvG6RlRhyt_JaHA


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://cream-45.pd.infn.it:9000/N9AYUlkkvG6RlRhyt_JaHA
Current Status:     Done (Success)
Logged Reason(s):
    - job completed
    - Job Terminated Successfully
Exit code:          0
Status Reason:      Job Terminated Successfully
Destination:        cream-32.pd.infn.it:8443/cream-pbs-creamtest2
Submitted:          Fri Oct 21 15:07:40 2011 CEST
==========================================================================

[ale@cream-12 UI]$ glite-wms-job-output https://cream-45.pd.infn.it:9000/N9AYUlkkvG6RlRhyt_JaHA

Connecting to the service https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server


================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://cream-45.pd.infn.it:9000/N9AYUlkkvG6RlRhyt_JaHA
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/ale_N9AYUlkkvG6RlRhyt_JaHA

================================================================================

Check that also the SBD has been removed on WMS:

[root@cream-45 ~]# ls /var/SandboxDir/N9/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fN9AYUlkkvG6RlRhyt_5fJaHA/
ls: /var/SandboxDir/N9/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fN9AYUlkkvG6RlRhyt_5fJaHA/: No such file or directory



Regression Tests

Bug #81376 (glite-wms-wmproxy.restart.cron MUST support graceful wmp restart) FIXED

[ale@cream-12 regression_tests]$ python execute_regTest.py -d 3 -b 81376 -l
+++++++++++++++++++++++++++++++++++++++++++++++++++++
+ Regression TestSuite ....                         +
+++++++++++++++++++++++++++++++++++++++++++++++++++++

 Testing bug 81376 


 All tests PASSED 

See log file

Bug #81568 (some inconsistencies in locations for logs and configuration) FIXED

[ale@cream-12 regression_tests]$ python execute_regTest.py -d 3 -b 81568 -l
+++++++++++++++++++++++++++++++++++++++++++++++++++++
+ Regression TestSuite ....                         +
+++++++++++++++++++++++++++++++++++++++++++++++++++++

 Testing bug 81568 


 All tests PASSED 

See log file

Verified that all WMS log files are in this directory:

# ls $WMS_LOCATION_LOG 
glite-lb-lcas.log                       glite-wms-wmproxy.restart.cron.log
glite-lb-lcas.log.1.gz                  httpd-wmproxy-access.log
glite-lb-lcas.log.2.gz                  httpd-wmproxy-errors.log
glite-lb-lcas.log.3.gz                  ice.log
glite-lb-lcas.log.4.gz                  jobcontoller_events.log
glite-lb-purger.log                     logmonitor_events.log
glite-lb-purger.log.1.gz                wmproxy.log
glite-wms-purgeStorage.log              wmproxy_logrotate.log
glite-wms-wmproxy-purge-proxycache.log  workload_manager_events.log

Bug #83062 (A different template should be used for glue2 publication) FIXED

Verified querying the resource BDII running on the WMS for Glue2, and verified that:

  • a GLUE2Service objectclass with GLUE2ServiceType: org.glite.wms.WMProxy
  • a GLUE2Endpoint objectclass with GLUE2EndpointInterfaceName: org.glite.wms.WMProxy
are published.

sgaravat@lxsgaravatto:~$ ldapsearch -h cream-44.pd.infn.it -x -p 2170 -b "o=glue"
# extended LDIF
#
# LDAPv3
# base <o=glue> with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# glue
dn: o=glue
objectClass: organization
o: glue

# grid, glue
dn: GLUE2GroupID=grid,o=glue
objectClass: GLUE2Group
GLUE2GroupID: grid

# resource, glue
dn: GLUE2GroupID=resource,o=glue
objectClass: GLUE2Group
GLUE2GroupID: resource

# cream-44.pd.infn.it_wmproxy_1185681625, resource, glue
dn: GLUE2ServiceID=cream-44.pd.infn.it_wmproxy_1185681625,GLUE2GroupID=resource,o=glue
GLUE2EntityValidity: 3600
GLUE2ServiceType: org.glite.wms.WMProxy
GLUE2ServiceID: cream-44.pd.infn.it_wmproxy_1185681625
objectClass: GLUE2Service
GLUE2ServiceQualityLevel: production
GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-service
GLUE2EntityOtherInfo: InfoProviderVersion=1.1
GLUE2EntityOtherInfo: InfoProviderHost=cream-44.pd.infn.it
GLUE2ServiceComplexity: endpointType=1, share=0, resource=0
GLUE2ServiceCapability: executionmanagement.candidatesetgenerator
GLUE2ServiceCapability: executionmanagement.jobdescription
GLUE2ServiceCapability: executionmanagement.jobmanager
GLUE2EntityName: JRA1-CREAM-TEST-WMProxy
GLUE2ServiceAdminDomainForeignKey: JRA1-CREAM-TEST
GLUE2EntityCreationTime: 2011-10-19T13:44:35Z

# cream-44.pd.infn.it_wmproxy_1185681625_org.glite.wms.WMProxy_1185681625, cream-44.pd.infn.it_wmproxy_1185681625, resource, glue
dn: GLUE2EndpointID=cream-44.pd.infn.it_wmproxy_1185681625_org.glite.wms.WMProxy_1185681625,GLUE2ServiceID=cream-44.pd.infn.it_wmproxy_1185681625,GLUE2GroupID=resource,o=glue
GLUE2EntityValidity: 3600
GLUE2EndpointQualityLevel: production
GLUE2EndpointImplementor: gLite
GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-endpoint
GLUE2EntityOtherInfo: InfoProviderVersion=1.2
GLUE2EntityOtherInfo: InfoProviderHost=cream-44.pd.infn.it
GLUE2EntityOtherInfo: MiddlewareName=EMI
GLUE2EntityOtherInfo: MiddlewareVersion=1.4.0-1
GLUE2EndpointCapability: executionmanagement.candidatesetgenerator
GLUE2EndpointCapability: executionmanagement.jobdescription
GLUE2EndpointCapability: executionmanagement.jobmanager
GLUE2EndpointServiceForeignKey: cream-44.pd.infn.it_wmproxy_1185681625
GLUE2EntityName: WMProxy endpoint for Service cream-44.pd.infn.it_wmproxy_1185681625
GLUE2EndpointTechnology: webservice
GLUE2EndpointWSDL: http://web.infn.it/gLiteWMS/images/WMS/Docs/wmproxy.wsdl
GLUE2EndpointInterfaceName: org.glite.wms.WMProxy
GLUE2EndpointURL: https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server
GLUE2EndpointDowntimeInfo: See the GOC DB for downtimes: https://goc.egi.eu/
GLUE2EndpointImplementationVersion: 3.3.4
GLUE2EndpointSemantics: http://web.infn.it/gLiteWMS/images/WMS/Docs/wmproxy-guide.pdf
GLUE2EndpointIssuerCA: /C=IT/O=INFN/CN=INFN CA
GLUE2EndpointServingState: production
objectClass: GLUE2Endpoint
GLUE2EndpointInterfaceVersion: 1.0
GLUE2EndpointImplementationName: WMS
GLUE2EndpointTrustedCA: IGTF
GLUE2EndpointID: cream-44.pd.infn.it_wmproxy_1185681625_org.glite.wms.WMProxy_1185681625
GLUE2EndpointHealthState: ok
GLUE2EndpointStartTime: 2011-10-18T22:00:02Z
GLUE2EndpointHealthStateInfo: WMProxy httpd listening on port 7443 httpd is running .... === WMProxy Server running instances: UID PID PPID C STIME TTY TIME CMD glite 3888 15787 0 12:20 ? 00:00:00 /usr/bin/glite_wms_wmproxy_server
GLUE2EntityCreationTime: 2011-10-19T13:44:35Z

# DN_1, cream-44.pd.infn.it_wmproxy_1185681625_org.glite.wms.WMProxy_1185681625, cream-44.pd.infn.it_wmproxy_1185681625, resource, glue
dn: GLUE2ExtensionLocalID=DN_1,GLUE2EndpointID=cream-44.pd.infn.it_wmproxy_1185681625_org.glite.wms.WMProxy_1185681625,GLUE2ServiceID=cream-44.pd.infn.it_wmproxy_1185681625,GLUE2GroupID=resource,o=glue
GLUE2ExtensionLocalID: DN_1
GLUE2ExtensionKey: DN
objectClass: GLUE2Extension
GLUE2ExtensionValue: /C=IT/O=INFN/OU=Host/L=Padova/CN=cream-44.pd.infn.it
GLUE2ExtensionEntityForeignKey: cream-44.pd.infn.it_wmproxy_1185681625_org.gli
 te.wms.WMProxy_1185681625

# cream-44.pd.infn.it_wmproxy_1185681625_org.glite.wms.WMProxy_1185681625_Policy, cream-44.pd.infn.it_wmproxy_1185681625_org.glite.wms.WMProxy_1185681625
 , cream-44.pd.infn.it_wmproxy_1185681625, resource, glue
dn: GLUE2PolicyID=cream-44.pd.infn.it_wmproxy_1185681625_org.glite.wms.WMProxy
 _1185681625_Policy,GLUE2EndpointID=cream-44.pd.infn.it_wmproxy_1185681625_org.glite.wms.WMProxy_1185681625,GLUE2ServiceID=cream-44.pd.infn.it_wmproxy_1185681625,GLUE2GroupID=resource,o=glue
GLUE2EntityValidity: 3600
objectClass: GLUE2Policy
objectClass: GLUE2AccessPolicy
GLUE2PolicyRule: ALL
GLUE2PolicyScheme: org.glite.standard
GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-endpoint
GLUE2EntityOtherInfo: InfoProviderVersion=1.2
GLUE2EntityOtherInfo: InfoProviderHost=cream-44.pd.infn.it
GLUE2AccessPolicyEndpointForeignKey: cream-44.pd.infn.it_wmproxy_1185681625_org.glite.wms.WMProxy_1185681625
GLUE2PolicyID: cream-44.pd.infn.it_wmproxy_1185681625_org.glite.wms.WMProxy_1185681625_Policy
GLUE2EntityName: Access control rules for Endpoint cream-44.pd.infn.it_wmproxy_1185681625_org.glite.wms.WMProxy_1185681625
GLUE2EntityCreationTime: 2011-10-19T13:44:35Z

# search result
search: 2
result: 0 Success

# numResponses: 8
# numEntries: 7

Verified that GlueValidor doesn't report errors:

sgaravat@lxsgaravatto:~/GlueValidator$ ./trunk/bin/glue-validator -t glue2 -b "o=glue" -h cream-44.pd.infn.it -p 2170 
test_data_types (validator.EntryTest.EntryTest)
Validating data types. ... ok
test_mandatory_attributes (validator.EntryTest.EntryTest)
Verifying the existence of mandatory attributes. ... ok
test_object_class (validator.EntryTest.EntryTest)
Verifying the object class ... ok
test_single_valued (validator.EntryTest.EntryTest)
Verifying single-valued attributes only have one value. ... ok
test_data_types (validator.EntryTest.EntryTest)
Validating data types. ... ok
test_mandatory_attributes (validator.EntryTest.EntryTest)
Verifying the existence of mandatory attributes. ... ok
test_object_class (validator.EntryTest.EntryTest)
Verifying the object class ... ok
test_single_valued (validator.EntryTest.EntryTest)
Verifying single-valued attributes only have one value. ... ok
test_data_types (validator.EntryTest.EntryTest)
Validating data types. ... ok
test_mandatory_attributes (validator.EntryTest.EntryTest)
Verifying the existence of mandatory attributes. ... ok
test_object_class (validator.EntryTest.EntryTest)
Verifying the object class ... ok
test_single_valued (validator.EntryTest.EntryTest)
Verifying single-valued attributes only have one value. ... ok
test_data_types (validator.EntryTest.EntryTest)
Validating data types. ... ok
test_mandatory_attributes (validator.EntryTest.EntryTest)
Verifying the existence of mandatory attributes. ... ok
test_object_class (validator.EntryTest.EntryTest)
Verifying the object class ... ok
test_single_valued (validator.EntryTest.EntryTest)
Verifying single-valued attributes only have one value. ... ok
test_data_types (validator.EntryTest.EntryTest)
Validating data types. ... ok
test_mandatory_attributes (validator.EntryTest.EntryTest)
Verifying the existence of mandatory attributes. ... ok
test_object_class (validator.EntryTest.EntryTest)
Verifying the object class ... ok
test_single_valued (validator.EntryTest.EntryTest)
Verifying single-valued attributes only have one value. ... ok
test_data_types (validator.EntryTest.EntryTest)
Validating data types. ... ok
test_mandatory_attributes (validator.EntryTest.EntryTest)
Verifying the existence of mandatory attributes. ... ok
test_object_class (validator.EntryTest.EntryTest)
Verifying the object class ... ok
test_single_valued (validator.EntryTest.EntryTest)
Verifying single-valued attributes only have one value. ... ok
test_data_types (validator.EntryTest.EntryTest)
Validating data types. ... ok
test_mandatory_attributes (validator.EntryTest.EntryTest)
Verifying the existence of mandatory attributes. ... ok
test_object_class (validator.EntryTest.EntryTest)
Verifying the object class ... ok
test_single_valued (validator.EntryTest.EntryTest)
Verifying single-valued attributes only have one value. ... ok

----------------------------------------------------------------------
Ran 28 tests in 0.005s

OK

Bug #85327 (glite-wms-wmproxy.restart.cron missing HOSTNAME environment variable) FIXED

[ale@cream-12 regression_tests]$ python execute_regTest.py -d 3 -b 85327 -l
+++++++++++++++++++++++++++++++++++++++++++++++++++++
+ Regression TestSuite ....                         +
+++++++++++++++++++++++++++++++++++++++++++++++++++++

 Testing bug 85327 


 All tests PASSED 

See log file

Bug #86485 (ICE doesn't send the iceId to CREAM) FIXED

Submitted a job through the WMS to a CREAM CE in debug mode.

Verified that in the CREAM log file the iceid is reported:

# grep selectToRetrieveJobStatusQuery /var/log/cream/glite-ce-cream.log 
20 Oct 2011 07:11:43,186 DEBUG org.glite.ce.cream.jobmanagement.db.table.JobStatusTable (JobStatusTable.java:419) - (TP-Processor25) selectToRetrieveJobStatusQuery = select job_status.type AS type,job_status.exitCode AS exitCode,job_status.failureReason AS failureReason,job_status.description AS description,job_status.time_stamp AS time_stamp,job_status.jobId AS jobId,job_status.id AS id,job.gridJobId AS gridJobId,job.workerNode AS workerNode from job_status, job where job_status.jobId = job.id and job_status.type IN ( '10', '5', '8', '7', '6', '4', '3') and job.userId = '_C_IT_O_INFN_OU_Personal_Certificate_L_Padova_CN_Massimo_Sgaravatto_dteam_Role_NULL_Capability_NULL' and job.iceId = 'C_IT_O_INFN_OU_Host_L_Padova_CN_cream-44.pd.infn.it' and job_status.id >= 0 and time_stamp >= '2011-10-19 15:46:36.0' ORDER BY job_status.id limit 500

Verified also that in the CREAMDB the iceid is set for this job:

mysql> use creamdb;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select iceId from job where id="CREAM806900601"
    -> ;
+-----------------------------------------------------+
| iceId                                               |
+-----------------------------------------------------+
| C_IT_O_INFN_OU_Host_L_Padova_CN_cream-44.pd.infn.it | 
+-----------------------------------------------------+
1 row in set (0.00 sec)

Bug #86490 (EMI WMS stops accepting jobs after 31999th submission for the same DN (on ext3)) FIXED

Verified, checking the /var/log/glite/glite-wms-wmproxy-purge-proxycache.log log, that the relevant cron job purges expired proxies and relevant directories (and checked that these are really removed).

Verified that in /var/proxycache there aren't proxies expired more than 6 hours ago and that there aren't empty directories.

Bug #86682 ( yaim changes ownership of /var) FIXED

[ale@cream-12 regression_tests]$ python execute_regTest.py -d 3 -b 86682 -l
+++++++++++++++++++++++++++++++++++++++++++++++++++++
+ Regression TestSuite ....                         +
+++++++++++++++++++++++++++++++++++++++++++++++++++++

 Testing bug 86682 


 All tests PASSED 

See log file

Bug #86769 (EMI WMS Update 6: WMS 'hangs' after several hours of operation) TBD

Bug #86956 (Match-matching of requests with access to catalogues can cause the WM to crash) TBD

Bug #87259 (WMS purger leaves sandboxes untouched in case LB status returns EIDRM) FIXED

For a job submitted to a LCG-CE, verified that in /var/log/glite/logmonitor_events.log the removing of job directory, submit file, classad file, and wrapper file is logged:

19 Oct, 17:57:05 -I- JobFilePurger::do_purge(...): Removing job directory: /var/jobcontrol/condorio/1y/https_3a_2f_2fcream-44.pd.infn.it_3a9000_2f1yPrTrq9B5gEXyabDPjg8w
19 Oct, 17:57:05 -I- JobFilePurger::do_purge(...): Removing submit file: /var/jobcontrol/submit/1y/Condor.https_3a_2f_2fcream-44.pd.infn.it_3a9000_2f1yPrTrq9B5gEXyabDPjg8w.submit
19 Oct, 17:57:05 -I- JobFilePurger::do_purge(...): Removing classad file: /var/jobcontrol/submit/1y/ClassAd.https_3a_2f_2fcream-44.pd.infn.it_3a9000_2f1yPrTrq9B5gEXyabDPjg8w
19 Oct, 17:57:05 -I- JobFilePurger::do_purge(...): Removing wrapper file: /var/jobcontrol/submit/1y/JobWrapper.https_3a_2f_2fcream-44.pd.infn.it_3a9000_2f1yPrTrq9B5gEXyabDPjg8w.sh

Verified that they have been actually removed:

# ls /var/jobcontrol/condorio/1y/https_3a_2f_2fcream-44.pd.infn.it_3a9000_2f1yPrTrq9B5gEXyabDPjg8w /var/jobcontrol/submit/1y/Condor.https_3a_2f_2fcream-44.pd.infn.it_3a9000_2f1yPrTrq9B5gEXyabDPjg8w.submit /var/jobcontrol/submit/1y/ClassAd.https_3a_2f_2fcream-44.pd.infn.it_3a9000_2f1yPrTrq9B5gEXyabDPjg8w /var/jobcontrol/submit/1y/JobWrapper.https_3a_2f_2fcream-44.pd.infn.it_3a9000_2f1yPrTrq9B5gEXyabDPjg8w.sh
ls: /var/jobcontrol/condorio/1y/https_3a_2f_2fcream-44.pd.infn.it_3a9000_2f1yPrTrq9B5gEXyabDPjg8w: No such file or directory
ls: /var/jobcontrol/submit/1y/Condor.https_3a_2f_2fcream-44.pd.infn.it_3a9000_2f1yPrTrq9B5gEXyabDPjg8w.submit: No such file or directory
ls: /var/jobcontrol/submit/1y/ClassAd.https_3a_2f_2fcream-44.pd.infn.it_3a9000_2f1yPrTrq9B5gEXyabDPjg8w: No such file or directory
ls: /var/jobcontrol/submit/1y/JobWrapper.https_3a_2f_2fcream-44.pd.infn.it_3a9000_2f1yPrTrq9B5gEXyabDPjg8w.sh: No such file or directory

Submitted a job:

$ glite-wms-job-submit -o wmsids -a -c cream44.conf oo.jdl

Connecting to the service https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://cream-44.pd.infn.it:9000/gcr-dFPacjwKFlk8CE8Gpw

The job identifier has been saved in the following file:
/home/sgaravat/JDLExamples/wmsids

==========================================================================

Copied the sandbox dir:

[root@cream-44 ~]# cd /var/SandboxDir/gc/
[root@cream-44 gc]# ls -al
total 12
drwxrwx-wx   3 glite    glite 4096 Oct 19 16:33 .
drwxrwx-wt 178 glite    glite 4096 Oct 19 16:33 ..
drwxrwx---   5 dteam004 glite 4096 Oct 19 16:34 https_3a_2f_2fcream-44.pd.infn.it_3a9000_2fgcr-dFPacjwKFlk8CE8Gpw
[root@cream-44 gc]# cp -r -p https_3a_2f_2fcream-44.pd.infn.it_3a9000_2fgcr-dFPacjwKFlk8CE8Gpw https_3a_2f_2fcream-44.pd.infn.it_3a9000_2fgcr-dFPacjwKFlk8CE8Gpw.old
[root@cream-44 gc]# ls
https_3a_2f_2fcream-44.pd.infn.it_3a9000_2fgcr-dFPacjwKFlk8CE8Gpw
https_3a_2f_2fcream-44.pd.infn.it_3a9000_2fgcr-dFPacjwKFlk8CE8Gpw.old

Retrieved the OSB when the job is done:

[sgaravat@cream-50 JDLExamples]$ glite-wms-job-status https://cream-44.pd.infn.it:9000/gcr-dFPacjwKFlk8CE8Gpw


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://cream-44.pd.infn.it:9000/gcr-dFPacjwKFlk8CE8Gpw
Current Status:     Done (Exit Code !=0)
Exit code:          23
Status Reason:      Job Terminated Successfully
Destination:        cream.sns.it:8443/cream-pbs-cert
Submitted:          Wed Oct 19 16:33:46 2011 CEST
==========================================================================

[sgaravat@cream-50 JDLExamples]$ glite-wms-job-output https://cream-44.pd.infn.it:9000/gcr-dFPacjwKFlk8CE8Gpw

Connecting to the service https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server


Warning - https://cream-44.pd.infn.it:9000/gcr-dFPacjwKFlk8CE8Gpw: the status is DONE (ExitCode != 0)

================================================================================

         JOB GET OUTPUT OUTCOME

No output files to be retrieved for the job:
https://cream-44.pd.infn.it:9000/gcr-dFPacjwKFlk8CE8Gpw

================================================================================

Restored the sandbox dir in the WMS:

[root@cream-44 gc]# mv https_3a_2f_2fcream-44.pd.infn.it_3a9000_2fgcr-dFPacjwKFlk8CE8Gpw.old https_3a_2f_2fcream-44.pd.infn.it_3a9000_2fgcr-dFPacjwKFlk8CE8Gpw
[root@cream-44 gc]# ls -l
total 4
drwxrwx--- 5 dteam004 glite 4096 Oct 19 16:34 https_3a_2f_2fcream-44.pd.infn.it_3a9000_2fgcr-dFPacjwKFlk8CE8Gpw

Purged the job from LBProxy and from LBServer:

[root@cream-44 ~]# echo "https://cream-44.pd.infn.it:9000/gcr-dFPacjwKFlk8CE8Gpw" > jobid.txt
[root@cream-44 ~]# /usr/bin/glite-lb-purge -m cream-44.pd.infn.it -x -j jobid.txt 
The jobs were not dumped.
[root@cream-44 ~]# /usr/bin/glite-lb-purge -m cream-44.pd.infn.it -j jobid.txt The jobs were not dumped.

Verified that job status returns Identifier removed (i.e. error EIDRM):

$ glite-wms-job-status https://cream-44.pd.infn.it:9000/gcr-dFPacjwKFlk8CE8Gpw



**** Error: API_NATIVE_ERROR ****  
Error while calling the "Job:getStatus" native api 
Unable to retrieve the status for: https://cream-44.pd.infn.it:9000/gcr-dFPacjwKFlk8CE8Gpw
glite.lb.Exception: edg_wll_JobStatus: Identifier removed: matching job already purged
   at glite::lb::Job::status[./src/Job.cpp:87]

On the WMS, purged the sandboxdir, as user glite, invoking the command glite-wms-purgeStorage.sh and verifying that Identifier removed (i.e. error 43) is returned:

$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/gc
19 Oct, 16:49:12 -E: [Error] query_job_status(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:134): https://cream-44.pd.infn.it:9000/gcr-dFPacjwKFlk8CE8Gpw: edg_wll_JobStat [43] Identifier removed(matching job already purged)
19 Oct, 16:49:12 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:306): https://cream-44.pd.infn.it:9000/gcr-dFPacjwKFlk8CE8Gpw: forced removal, unknown/removed L&B job

Verified that the sandboxdir has been removed:

[root@cream-44 ~]# ls /var/SandboxDir/gc
[root@cream-44 ~]# 
Topic attachments
I Attachment Action Size Date Who Comment
Unknown file formatlog 81376.log manage 5.1 K 2011-10-20 - 10:31 AlessioGianelle  
Unknown file formatlog 81568.log manage 5.7 K 2011-10-20 - 10:47 AlessioGianelle  
Unknown file formatlog 85327.log manage 5.8 K 2011-10-20 - 11:12 AlessioGianelle  
Unknown file formatlog 86682.log manage 5.4 K 2011-10-20 - 11:16 AlessioGianelle  
Unknown file formatlog WMSService-TS_20111020163927.log manage 68.1 K 2011-10-20 - 15:24 AlessioGianelle  
Compressed Zip archivetgz cancel.tgz manage 16.5 K 2011-10-20 - 09:45 AlessioGianelle  
Compressed Zip archivetgz collection.tgz manage 6.8 K 2011-10-19 - 11:40 AlessioGianelle  
Texttxt configure.txt manage 50.3 K 2011-10-14 - 09:02 AlessioGianelle Configure
Compressed Zip archivetgz dag.tgz manage 4.0 K 2011-10-19 - 11:41 AlessioGianelle  
Compressed Zip archivetgz delegation.tgz manage 4.2 K 2011-10-19 - 13:26 AlessioGianelle  
Compressed Zip archivetgz epilogue.tgz manage 7.0 K 2011-10-20 - 08:29 AlessioGianelle  
Texttxt install.txt manage 152.7 K 2011-10-14 - 09:02 AlessioGianelle  
Compressed Zip archivetgz listmatch.tgz manage 3.9 K 2011-10-14 - 16:01 AlessioGianelle  
Compressed Zip archivetgz parallel.tgz manage 4.3 K 2011-10-19 - 15:14 AlessioGianelle  
Compressed Zip archivetgz parametric.tgz manage 7.3 K 2011-10-19 - 11:41 AlessioGianelle  
Compressed Zip archivetgz perusal.tgz manage 9.1 K 2011-10-19 - 11:41 AlessioGianelle  
Compressed Zip archivetgz pr.tgz manage 21.3 K 2011-10-20 - 13:37 AlessioGianelle  
Compressed Zip archivetgz resubmit.tgz manage 9.7 K 2011-10-20 - 08:49 AlessioGianelle  
Compressed Zip archivetgz submitcoll.tgz manage 5.9 K 2011-10-14 - 16:01 AlessioGianelle  
Compressed Zip archivetgz submitnormal.tgz manage 5.4 K 2011-10-14 - 16:01 AlessioGianelle  
Texttxt update.txt manage 95.3 K 2011-10-14 - 09:29 AlessioGianelle Update log
Edit | Attach | PDF | History: r68 | r34 < r33 < r32 < r31 | Backlinks | Raw View | More topic actions...
Topic revision: r32 - 2011-10-21 - FabioCapannini
 
  • Edit
  • Attach
This site is powered by the TWiki collaboration platformCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback