Tags:
, view all tags

WMS test report

Summary

  • Component: WMS 3.3.5

  • ETICS Subsystem Configuration Name: emi-wms_R_3_3_7_1 emi-jdl_R_3_2_6_1

  • VCS Tag: emi-ice_R_3_3_4_1, emi-wms-manager_R_3_3_1_4, emi-jobsubmission_R_3_3_2_1, emi-wmproxy_R_3_3_5_1, emi-wms-brokerinfo_R_3_3_2_1, emi-wms-configuration_R_3_3_2_1, emi-wms-ism_R_3_3_2_1, emi-wms-matchmaking_R_3_3_2_1, emi-wms-purger_R_3_3_2_1, emi-yaim-wms_R_4_1_5_1, emi-jdl-api-cpp_R_3_2_6_1, emi-wms-nagios_R_1_0_0_0, emi-wms-mp_R_1_0_2_0

  • EMI Major Release: EMI 1 (Kebnekaise)

  • Platform: sl5 epel

  • Certification report: not yet available

  • Outcome: Certification under way

Deployment Tests

Clean installation

  • Starting from a SL5 x86_64 kernel machine:

[root@emi-demo11 ~]# cat /etc/redhat-release 
Scientific Linux SL release 5.7 (Boron)
[root@emi-demo11 ~]# uname -a
Linux emi-demo11.cnaf.infn.it 2.6.18-274.18.1.el5 #1 SMP Thu Feb 9 12:20:03 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
[root@emi-demo11 ~]# wget http://emisoft.web.cern.ch/emisoft/dist/EMI/1/RPM-GPG-KEY-emi -O /tmp/emi-key_gd.asc
[root@emi-demo11 ~]# rpm --import /tmp/emi-key_gd.asc

Update

[root@emi-demo12 yum.repos.d]# rpm -qa|grep wms-manager
glite-wms-manager-3.3.1-3.sl5

The wms emi-demo12 is configured in proxy mode, pointing to the LB devel17.cnaf.infn.it

Unit Tests

N/A: Tests will be available with EMI-2.

Basic Functionality tests

Submission

Where possible the tests should use internal testbed CEs (located in Padova). This is accomplished by adding the following line to wms-command.conf:
DEFAULTREQ= other.GlueCEStateStatus == "Testing"
This is not possible, for example, for parallel jobs.

Normal

[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 1 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 10:28:22
===> Ended  : 10:32:49
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120316102822.log
[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 1 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 10:30:51
===> Ended  : 10:35:08
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120316103051.log
   [capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V infngrid -d 3 -l -t 1 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 10:31:52
===> Ended  : 10:37:14
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120316103152.log
* submit.tgz: submit.tgz
Proxy mode
Pointing the testsuite at emi-demo12:

[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 1 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo12.cnaf.infn.it
===> Started: 10:50:40
===> Ended  : 10:56:37
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120316105040.log
* WMSService-TS_20120316105040.log: WMSService-TS_20120316105040.log

Jdl with DataRequirements

[capannini@devel15 WMS-service]$ cat ~/certification/data-req.jdl
###########################################
#      JDL with Data Requirements         #
###########################################

Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";
Prologue = "prologue.sh";

FuzzyRank = true;
Environment={"LFC_HOST=prod-lfc-shared-central.cern.ch", "LFC_HOME=prod-lfc-shared-central.cern.ch:/grid/dteam/aleph"};
InputSandbox = {"JDLs/prologue.sh"};
OutputSandbox = {"std.out", "std.err","prologue.out", "PI_1M.txt"};

DataRequirements = {
[
DataCatalogType = "DLI";
DataCatalog ="http://prod-lfc-shared-central.cern.ch:8085/"; 
InputData = { "lfn:/grid/dteam/fcapanni/PI_1M.txt" };
]
};

DataAccessProtocol = "gsiftp";
AllowZippedISB=true;
RetryCount = 1;
ShallowRetryCount = 2;

[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 1 -i -j ~/certification/data-req.jdl
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 11:33:22
===> Ended  : 11:54:21
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120316113322.log
* WMSService-TS_20120316113322.log: WMSService-TS_20120316113322.log

Jdl with ISBBaseURI and OSBDestURI

Test on CREAM ce:

[capannini@cert-17 certification]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf URI.jdl 

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://emi-demo11.cnaf.infn.it:9000/FNmMujUg_2yIhS0iY8h9YA

==========================================================================


[capannini@cert-17 certification]$ 
[capannini@cert-17 certification]$ 
[capannini@cert-17 certification]$ 
[capannini@cert-17 certification]$ 
[capannini@cert-17 certification]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/FNmMujUg_2yIhS0iY8h9YA


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/FNmMujUg_2yIhS0iY8h9YA
Current Status:     Done (Success)
Logged Reason(s):
    - Transfer to CREAM failed due to exception: Failed to create a delegation id for job https://emi-demo11.cnaf.infn.it:9000/FNmMujUg_2yIhS0iY8h9YA: reason is User C=IT,O=INFN,OU=Personal Certificate,L=CNAF,CN=Fabio Capannini not authorized for operation {http://www.gridsite.org/namespaces/delegation-2}getProxyReq
    - Job Terminated Successfully
Exit code:          0
Status Reason:      Job Terminated Successfully
Destination:        cccreamceli10.in2p3.fr:8443/cream-sge-short
Submitted:          Fri Mar 16 14:29:07 2012 CET
==========================================================================

[capannini@cert-17 certification]$ glite-wms-job-output https://emi-demo11.cnaf.infn.it:9000/FNmMujUg_2yIhS0iY8h9YA

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://emi-demo11.cnaf.infn.it:9000/FNmMujUg_2yIhS0iY8h9YA
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/capannini_FNmMujUg_2yIhS0iY8h9YA

================================================================================

[capannini@cert-17 certification]$ ls /tmp/jobOutput/capannini_FNmMujUg_2yIhS0iY8h9YA
std.err

Log on devel09.cnaf.infn.it:

[root@devel09 ~]# cat /tmp/myjobOutput 
FNmMujUg_2yIhS0iY8h9YA
total 76
-rwxr-xr-x 1 dteam009 dteam 80 Mar 16 14:33 exe.sh
-rw-r--r-- 1 dteam009 dteam  0 Mar 16 14:33 std.err
-rw-r--r-- 1 dteam009 dteam 23 Mar 16 14:33 std.out
-rw-r--r-- 1 dteam009 dteam  6 Mar 16 14:33 test1
-rw-r--r-- 1 dteam009 dteam  6 Mar 16 14:33 test2

===>   test PASSED


Same process with an LCG CE:

[capannini@cert-17 certification]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf URI.jdl 

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://emi-demo11.cnaf.infn.it:9000/nz4pKzrc453q3uSn-q8vsQ

==========================================================================

[capannini@cert-17 certification]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/nz4pKzrc453q3uSn-q8vsQ


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/nz4pKzrc453q3uSn-q8vsQ
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job terminated successfully
Destination:        gridce2.pi.infn.it:2119/jobmanager-lcglsf-cert
Submitted:          Fri Mar 16 14:51:57 2012 CET
==========================================================================

[capannini@cert-17 certification]$ 
[capannini@cert-17 certification]$ 
[capannini@cert-17 certification]$ 
[capannini@cert-17 certification]$ glite-wms-job-output https://emi-demo11.cnaf.infn.it:9000/nz4pKzrc453q3uSn-q8vsQ

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://emi-demo11.cnaf.infn.it:9000/nz4pKzrc453q3uSn-q8vsQ
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/capannini_nz4pKzrc453q3uSn-q8vsQ

================================================================================


[capannini@cert-17 certification]$ ls /tmp/jobOutput/capannini_nz4pKzrc453q3uSn-q8vsQ
std.err

Log on devel09:

[root@devel09 ~]# cat /tmp/myjobOutput 
nz4pKzrc453q3uSn-q8vsQ
total 96
-rwxr-xr-x 1 dteam016 dteam  80 Mar 16 14:53 exe.sh
-rw-r--r-- 1 dteam016 dteam 118 Mar 16 14:53 https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fnz4pKzrc453q3uSn-q8vsQ.output
-rw-r--r-- 1 dteam016 dteam   0 Mar 16 14:53 std.err
-rw-r--r-- 1 dteam016 dteam  23 Mar 16 14:53 std.out
-rw-r--r-- 1 dteam016 dteam   6 Mar 16 14:53 test1
-rw-r--r-- 1 dteam016 dteam   6 Mar 16 14:53 test2
-rw------- 1 dteam016 dteam 103 Mar 16 14:53 tmp.dSuMI19622
-rw------- 1 dteam016 dteam   2 Mar 16 14:53 tmp.iYrdk19621
-rw------- 1 dteam016 dteam   2 Mar 16 14:53 tmp.kpcpo19636
-rw------- 1 dteam016 dteam   0 Mar 16 14:53 tmp.nAPiv19635
-rw------- 1 dteam016 dteam 103 Mar 16 14:53 tmp.rUfjz19637

===>   test PASSED

Jdl with ISBBaseURI and OSBBaseDestURI

Submit to CREAM CE

[capannini@cert-17 certification]$ cat BaseURI.jdl 
###########################################
#  JDL with ISBURI & OSBBaseDestURI       #
###########################################

Executable = "exe.sh";
StdOutput = "std.out";
StdError = "std.err";

InputSandboxBaseURI = "gsiftp://emi-demo12.cnaf.infn.it:2811/tmp";

FuzzyRank = true;
InputSandbox = {"exe.sh", "test1", "test2"};
OutputSandbox = {"std.out", "std.err"};
OutputSandboxBaseDestURI="gsiftp://devel20.cnaf.infn.it:2811/tmp";

#requirements = other.GlueCEStateStatus == "Testing" &&  regexp("/cream", other.GlueCEUniqueID);
requirements = regexp("/cream", other.GlueCEUniqueID);

RetryCount = 1;
ShallowRetryCount = 2;

[capannini@cert-17 certification]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf BaseURI.jdl 

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://emi-demo11.cnaf.infn.it:9000/hlAzaQOlelzhpDmzgGL88g

==========================================================================

[capannini@cert-17 certification]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/hlAzaQOlelzhpDmzgGL88g


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/hlAzaQOlelzhpDmzgGL88g
Current Status:     Done (Success)
Logged Reason(s):
    - Transfer to CREAM failed due to exception: Failed to create a delegation id for job https://emi-demo11.cnaf.infn.it:9000/hlAzaQOlelzhpDmzgGL88g: reason is User C=IT,O=INFN,OU=Personal Certificate,L=CNAF,CN=Fabio Capannini not authorized for operation {http://www.gridsite.org/namespaces/delegation-2}getProxyReq
    - Job Terminated Successfully
Exit code:          0
Status Reason:      Job Terminated Successfully
Destination:        gridce0.pi.infn.it:8443/cream-lsf-cert
Submitted:          Fri Mar 16 16:13:56 2012 CET
==========================================================================

[capannini@cert-17 certification]$ 
[capannini@cert-17 certification]$ 
[capannini@cert-17 certification]$ glite-wms-job-output https://emi-demo11.cnaf.infn.it:9000/hlAzaQOlelzhpDmzgGL88g

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


================================================================================

         JOB GET OUTPUT OUTCOME

No output files to be retrieved for the job:
https://emi-demo11.cnaf.infn.it:9000/hlAzaQOlelzhpDmzgGL88g

================================================================================

Log on devel20:

[root@devel20 ~]# ls /tmp/std*
/tmp/std.err  /tmp/std.out
[root@devel20 ~]# 
[root@devel20 ~]# 
[root@devel20 ~]# cat /tmp/std.out 
hlAzaQOlelzhpDmzgGL88g
total 76
-rwxr-xr-x 1 dteam015 dteam 80 Mar 16 16:14 exe.sh
-rw-r--r-- 1 dteam015 dteam  0 Mar 16 16:14 std.err
-rw-r--r-- 1 dteam015 dteam 23 Mar 16 16:14 std.out
-rw-r--r-- 1 dteam015 dteam  6 Mar 16 16:14 test1
-rw-r--r-- 1 dteam015 dteam  6 Mar 16 16:14 test2

===>     === test PASSED === 

Submit to an LCG CE:

[capannini@cert-17 certification]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf BaseURI.jdl 

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://emi-demo11.cnaf.infn.it:9000/urEE8OAYVl2r9jUsAysuQw

==========================================================================

[capannini@cert-17 certification]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/urEE8OAYVl2r9jUsAysuQw


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/urEE8OAYVl2r9jUsAysuQw
Current Status:     Done (Success)
Logged Reason(s):
    - Job got an error while in the CondorG queue.
    - Job terminated successfully
Exit code:          0
Status Reason:      Job terminated successfully
Destination:        t2-ce-03.lnl.infn.it:2119/jobmanager-lcglsf-cert1
Submitted:          Fri Mar 16 16:25:36 2012 CET
==========================================================================

[capannini@cert-17 certification]$ 
[capannini@cert-17 certification]$ 
[capannini@cert-17 certification]$ glite-wms-job-output https://emi-demo11.cnaf.infn.it:9000/urEE8OAYVl2r9jUsAysuQw

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


================================================================================

         JOB GET OUTPUT OUTCOME

No output files to be retrieved for the job:
https://emi-demo11.cnaf.infn.it:9000/urEE8OAYVl2r9jUsAysuQw

================================================================================

Log on devel20:

[root@devel20 ~]# ls /tmp/std.*
/tmp/std.err  /tmp/std.out
[root@devel20 ~]# cat /tmp/std.out 
7A98_2qv4Qkqihilp2blrg
total 36
-rwxr-xr-x  1 dteam022 dteam  80 Mar 16 16:56 exe.sh
-rw-r--r--  1 dteam022 dteam 118 Mar 16 16:56 https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2f7A98_5f2qv4Qkqihilp2blrg.output
-rw-r--r--  1 dteam022 dteam   0 Mar 16 16:56 std.err
-rw-r--r--  1 dteam022 dteam  23 Mar 16 16:56 std.out
-rw-r--r--  1 dteam022 dteam   6 Mar 16 16:56 test1
-rw-r--r--  1 dteam022 dteam   6 Mar 16 16:56 test2
-rw-------  1 dteam022 dteam 103 Mar 16 16:56 tmp.boMqn22089
-rw-------  1 dteam022 dteam   2 Mar 16 16:56 tmp.lnMRO22073
-rw-------  1 dteam022 dteam 103 Mar 16 16:56 tmp.mxsln22074
-rw-------  1 dteam022 dteam   0 Mar 16 16:56 tmp.nBJFQ22087
-rw-------  1 dteam022 dteam   2 Mar 16 16:56 tmp.twwvt22088

===> Test PASSED

Jdl with AllowZippedISB

[capannini@cert-17 certification]$ cat AllowZIP.jdl 
###########################################
#   JDL with AllowZippedISB attribute     #
###########################################

Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";

InputSandbox = {"fileA", "fileB", "fileC", "fileD", "fileE"};
OutputSandbox = {"std.out", "std.err"};

requirements = other.GlueCEStateStatus == "Testing" && regexp("/cream", other.GlueCEUniqueID);

AllowZippedISB=true;

RetryCount = 1;
ShallowRetryCount = 2;

[capannini@cert-17 certification]$ glite-wms-job-submit --debug -a -c ~/wms_emi-demo11.conf AllowZIP.jdl 
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Vo read from proxy certificate extension
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - VirtualOrganisation value : dteam
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Configuration file: /home/capannini/wms_emi-demo11.conf - VirtualOrganisation, RetryCount, rank attribute(s) no more supported outside JDL Default Attributes section "JdlDefaultAttributes"
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Loaded user configuration file:
 /home/capannini/wms_emi-demo11.conf
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Configuration file: //etc/glite-wms/dteam/glite_wmsui.conf - VirtualOrganisation, MyProxyServer attribute(s) no more supported outside JDL Default Attributes section "JdlDefaultAttributes"
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Loaded Vo specific configuration file:
 //etc/glite-wms/dteam/glite_wmsui.conf
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Loaded Configuration values: 
 [
  JdlDefaultAttributes = 
   [
    PerusalFileEnable = false; 
    ShallowRetryCount = 10; 
    SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; 
    RetryCount = 3; 
    AllowZippedISB = true; 
    VirtualOrganisation = "dteam"; 
    rank =  -other.GlueCEStateEstimatedResponseTime; 
    MyProxyServer = "myproxy.cnaf.infn.it"
   ]; 
  OutputStorage = "/tmp"; 
  ListenerStorage = "/tmp"; 
  ErrorStorage = "/var/tmp"; 
  WmProxyEndPoints = { "https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server" }
 ]
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Getting Endpoint URL from configuration file 
-----------------------------------------

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server

-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Calling the WMProxy getVersion service
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - WMProxy Version: 3.3.5 
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Sending Proxy Request to https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Calling the WMProxy delegationns__getProxyReq service
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - delegationns__getProxyReq -  The proxy has been successfully retrieved
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Calling the WMProxy delegationns__putProxyReq service
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - The proxy has been successfully delegated with the identifier: QEfCo4sVtkysnpwMV4Br5w
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Calling the WMProxy getTransferProtocols service
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - getTransferProtocols -  Available protocols: gsiftp, https
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - FileTransferProtocol not specified; using the default protocol: gsiftp
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - The JDL file is: /home/capannini/certification/AllowZIP.jdl
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Retrieving the list of the local ISB files from the user JDL 
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Total size of the ISB file(s) to be transferred to: 1126406
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Max single job size of the ISB file(s) to be transferred to: 1024000
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Checking the User-FreeQuota on the server 
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Calling the WMProxy getFreeQuota service
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - getFreeQuota -  User freequota not set on the server
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Getting the max ISB size from the server https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Calling the WMProxy getMaxInputSandboxSize service
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - getMaxInputSandboxSize -  Max ISB size information successfully retrieved
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - The max job size (1024000 bytes) doesn't exceed the max size limit of 100000000 bytes: File transfer is allowed
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - File archiving and file compression allowed by user in the JDL
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Registering JDL [ StdOutput = "std.out"; ShallowRetryCount = 2; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 1; Executable = "/bin/ls"; Type = "job"; Arguments = "-la"; AllowZippedISB = true; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank =  -other.GlueCEStateEstimatedResponseTime; ZippedISB = { "ISBfiles_901aKw3LrX55xFsD4_xnTw_0.tar.gz" }; OutputSandbox = { "std.out","std.err" }; InputSandbox = { "file:///home/capannini/certification/fileA","file:///home/capannini/certification/fileB","file:///home/capannini/certification/fileC","file:///home/capannini/certification/fileD","file:///home/capannini/certification/fileE" }; StdError = "std.err"; rank =  -other.GlueCEStateEstimatedResponseTime; MyProxyServer = "myproxy.cnaf.infn.it"; requirements = other.GlueCEStateStatus == "Testing" && regexp("/cream",other.GlueCEUniqueID) ]
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Registering the job to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Calling the WMProxy jobRegister service
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - The JobId is:  https://emi-demo11.cnaf.infn.it:9000/KAx2BtFz9BkraoEKeaHNAw
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Archiving the ISB files: /tmp/ISBfiles_901aKw3LrX55xFsD4_xnTw_0.tar
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - This archive file has been successfully created: /tmp/ISBfiles_901aKw3LrX55xFsD4_xnTw_0.tar
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Compressing the file (.gz): /tmp/ISBfiles_901aKw3LrX55xFsD4_xnTw_0.tar
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - ISB ZIPPED file successfully created: /tmp/ISBfiles_901aKw3LrX55xFsD4_xnTw_0.tar.gz
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Getting the SandboxDestinationURI from the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Calling the WMProxy getSandboxDestURI service with gsiftp protocol 
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - getSandboxDestURI -  Destination URIs sucessfully retrieved
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - DestinationURI: gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/KA/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fKAx2BtFz9BkraoEKeaHNAw/input
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - FileTransfer (gsiftp): using globus-url-copy to transfer the local InputSandBox file(s) to the submission endpoint
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - File Transfer (gsiftp) 
 Command: /usr/bin/globus-url-copy
Source: file:///tmp/ISBfiles_901aKw3LrX55xFsD4_xnTw_0.tar.gz
Destination: gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/KA/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fKAx2BtFz9BkraoEKeaHNAw/input/ISBfiles_901aKw3LrX55xFsD4_xnTw_0.tar.gz
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - File Transfer (gsiftp) Transfer successfully done
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Starting the job:  https://emi-demo11.cnaf.infn.it:9000/KAx2BtFz9BkraoEKeaHNAw
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - Calling the WMProxy jobStart service
-----------------------------------------
-----------------------------------------
16 March 2012, 17:00:28 -I- PID: 3015 (Debug) - jobStart -  The job has been successfully started
-----------------------------------------

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://emi-demo11.cnaf.infn.it:9000/KAx2BtFz9BkraoEKeaHNAw

==========================================================================

      *** Log file created ***
Possible Errors and Debug messages have been printed in the following file:
/var/tmp/glite-wms-job-submit_504_3015_20120316_17-00-28.log

Check if AllowZippedISB has been included in the jdl:

[capannini@cert-17 certification]$ glite-wms-job-info --jdl https://emi-demo11.cnaf.infn.it:9000/KAx2BtFz9BkraoEKeaHNAw|grep Allow
  AllowZippedISB = true; 

[capannini@cert-17 certification]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/KAx2BtFz9BkraoEKeaHNAw


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/KAx2BtFz9BkraoEKeaHNAw
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job Terminated Successfully
Destination:        cream-31.pd.infn.it:8443/cream-pbs-creamtest1
Submitted:          Fri Mar 16 17:00:28 2012 CET
==========================================================================

[capannini@cert-17 certification]$ glite-wms-job-output https://emi-demo11.cnaf.infn.it:9000/KAx2BtFz9BkraoEKeaHNAw

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://emi-demo11.cnaf.infn.it:9000/KAx2BtFz9BkraoEKeaHNAw
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/capannini_KAx2BtFz9BkraoEKeaHNAw

================================================================================


[capannini@cert-17 certification]$ ls -l /tmp/jobOutput/capannini_KAx2BtFz9BkraoEKeaHNAw
totale 4
-rw-r--r-- 1 capannini emitestbed   0 16 mar 17:06 std.err
-rw-r--r-- 1 capannini emitestbed 623 16 mar 17:06 std.out
[capannini@cert-17 certification]$ cat /tmp/jobOutput/capannini_KAx2BtFz9BkraoEKeaHNAw/std.out 
total 1132
drwxr-xr-x 2 dteam026 dteam    4096 Mar 16 17:00 .
drwxr-xr-x 3 dteam026 dteam    4096 Mar 16 17:00 ..
-rw-r--r-- 1 dteam026 dteam    4077 Mar 16 17:00 .BrokerInfo
-rw-r--r-- 1 dteam026 dteam       2 Mar 16 17:00 fileA
-rw-r--r-- 1 dteam026 dteam       2 Mar 16 17:00 fileB
-rw-r--r-- 1 dteam026 dteam 1024000 Mar 16 17:00 fileC
-rw-r--r-- 1 dteam026 dteam  102400 Mar 16 17:00 fileD
-rw-r--r-- 1 dteam026 dteam       2 Mar 16 17:00 fileE
-rw-r--r-- 1 dteam026 dteam       0 Mar 16 17:00 std.err
-rw-r--r-- 1 dteam026 dteam       0 Mar 16 17:00 std.out
-rw-r--r-- 1 dteam026 dteam       0 Mar 16 17:00 .tmp_file

====> Test passed

Jdl with ExpiryTime

[capannini@cert-17 certification]$ cat ExpiryTime.jdl 
###########################################
#       JDL with ExpiryTime attribute     #
###########################################

Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";

InputSandbox = {"fileA", "fileB", "fileC", "fileD", "fileE"};
OutputSandbox = {"std.out", "std.err"};

requirements = false;

ExpiryTime = 1331914400;

RetryCount = 1;
ShallowRetryCount = 2;

[capannini@cert-17 certification]$ date +%s
1331914321

[capannini@cert-17 certification]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf ExpiryTime.jdl 

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://emi-demo11.cnaf.infn.it:9000/DLi-uEyoyTgKNYdsBfXMUQ

==========================================================================


[capannini@cert-17 certification]$ 
[capannini@cert-17 certification]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/DLi-uEyoyTgKNYdsBfXMUQ


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/DLi-uEyoyTgKNYdsBfXMUQ
Current Status:     Waiting 
Status Reason:      BrokerHelper: no compatible resources
Submitted:          Fri Mar 16 17:13:15 2012 CET
==========================================================================

[capannini@cert-17 certification]$ date +%s
1331915013

[capannini@cert-17 certification]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/DLi-uEyoyTgKNYdsBfXMUQ


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/DLi-uEyoyTgKNYdsBfXMUQ
Current Status:     Aborted 
Status Reason:      request expired
Submitted:          Fri Mar 16 17:13:15 2012 CET
==========================================================================

===> Test PASSED

Jdl with ShortDeadlineJob

To test SDJs the following bdii must be used: egee-bdii.cnaf.infn.it

[capannini@cert-17 certification]$ cat SDLJ.jdl 
###########################################
# JDL with ShortDeadlineJob attribute     #
###########################################

Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";

InputSandbox = {"fileA", "fileB", "fileC", "fileD", "fileE"};
OutputSandbox = {"std.out", "std.err"};

requirements = true;

ShortDeadlineJob = true;

RetryCount = 1;
ShallowRetryCount = 2;

[capannini@cert-17 certification]$ glite-wms-job-list-match -a -c ~/wms_emi-demo11.conf SDLJ.jdl 

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server

==========================================================================

           COMPUTING ELEMENT IDs LIST 
 The following CE(s) matching your job requirements have been found:

   *CEId*
 - ce01.ariagni.hellasgrid.gr:2119/jobmanager-pbs-dteam.sdj
 - ce01.marie.hellasgrid.gr:2119/jobmanager-pbs-sdj
 - ce02.marie.hellasgrid.gr:2119/jobmanager-pbs-sdj
 - cream-ce01.ariagni.hellasgrid.gr:8443/cream-pbs-dteam.sdj
 - cream-ce01.marie.hellasgrid.gr:8443/cream-pbs-sdj
 - grid4.mif.vu.lt:2119/jobmanager-lcgpbs-sdj
 - grive02.ibcp.fr:8443/cream-pbs-sdj
 - mario.univ-lille1.fr:8443/cream-pbs-sdj

==========================================================================

[capannini@cert-17 certification]$ python ~/wms_tests.old/WMS-service/WMS-job-cycle.py -c ~/wms_tests.old/WMS-service/wms-command.conf -V dteam -d 3 -l -t 1 -i -j SDLJ.jdl
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: /home/capannini/wms_tests.old/WMS-service/WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 17:32:05
===> Ended  : 17:43:26
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120316173205.log

-- FabioCapannini - 2012-03-15

Topic attachments
I Attachment Action Size Date Who CommentSorted ascending
Unknown file formatlog install.log manage 122.4 K 2012-03-15 - 11:11 FabioCapannini Installation log
Unknown file formatlog WMSService-TS_20120316105040.log manage 12.1 K 2012-03-16 - 10:30 FabioCapannini  
Unknown file formatlog WMSService-TS_20120316113322.log manage 34.8 K 2012-03-16 - 11:02 FabioCapannini  
Unknown file formatlog WMSService-TS_20120316173205.log manage 20.9 K 2012-03-16 - 16:49 FabioCapannini  
Compressed Zip archivetgz submit.tgz manage 3.0 K 2012-03-16 - 09:45 FabioCapannini  
Unknown file formatlog update.log manage 31.1 K 2012-03-15 - 11:48 FabioCapannini Update log
Unknown file formatlog yaim-config.log manage 9.0 K 2012-03-15 - 11:25 FabioCapannini yaim configuration log
Edit | Attach | PDF | History: r34 | r9 < r8 < r7 < r6 | Backlinks | Raw View | More topic actions...
Topic revision: r7 - 2012-03-16 - FabioCapannini
 
  • Edit
  • Attach
This site is powered by the TWiki collaboration platformCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback