(r6) WMSTask26837 < IGIRelease

Tags: , view all tags
%TOC%

---+ *WMS test report*

---++ Summary

   * Component: *WMS 3.3.5*

   * Savannah task: [[https://savannah.cern.ch/task/?26837][26837]]

   * ETICS Subsystem Configuration Name: *emi-wms_R_3_3_7_1* *emi-jdl_R_3_2_6_1*

   * VCS Tag: _emi-ice_R_3_3_4_1_, _emi-wms-manager_R_3_3_1_4_, _emi-jobsubmission_R_3_3_2_1_, _emi-wmproxy_R_3_3_5_1_, _emi-wms-brokerinfo_R_3_3_2_1_, _emi-wms-configuration_R_3_3_2_1_, _emi-wms-ism_R_3_3_2_1_, _emi-wms-matchmaking_R_3_3_2_1_, _emi-wms-purger_R_3_3_2_1_, _emi-yaim-wms_R_4_1_5_1_, _emi-jdl-api-cpp_R_3_2_6_1_, _emi-wms-nagios_R_1_0_0_0_, _emi-wms-mp_R_1_0_2_0_

   * EMI Major Release: *EMI 1 (Kebnekaise)*

   * Platform: sl5 epel

   * Certification report: not yet available

   * Outcome:<strong> %GREEN% *Certification under way* %ENDCOLOR% </strong>

---++ Deployment Tests 

---+++ Clean installation

   * Starting from a SL5 x86_64 kernel machine:

<verbatim>[root@emi-demo11 ~]# cat /etc/redhat-release 
Scientific Linux SL release 5.7 (Boron)
[root@emi-demo11 ~]# uname -a
Linux emi-demo11.cnaf.infn.it 2.6.18-274.18.1.el5 #1 SMP Thu Feb 9 12:20:03 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
[root@emi-demo11 ~]# wget http://emisoft.web.cern.ch/emisoft/dist/EMI/1/RPM-GPG-KEY-emi -O /tmp/emi-key_gd.asc
[root@emi-demo11 ~]# rpm --import /tmp/emi-key_gd.asc</verbatim>

   * cd /etc/yum.repos.d/
   * wget --no-check-certificate https://twiki.cern.ch/twiki/pub/EMI/EMI1Updates/testing.repo
   * yum clean all
   * yum install ca-policy-egi-core
   * yum install emi-wms 
      * [[%ATTACHURL%/install.log][install.log]]: Installation log
   * /opt/glite/yaim/bin/yaim -c -s siteinfo/site-info.def -n WMS
      * [[%ATTACHURL%/yaim-config.log][yaim-config.log]]: yaim configuration log   

---+++ Update
   * Starting from a production version, repo available at http://emisoft.web.cern.ch/emisoft/dist/EMI/1/sl5/x86_64/updates/emi-release-1.0.1-1.sl5.noarch.rpm
<verbatim>[root@emi-demo12 yum.repos.d]# rpm -qa|grep wms-manager
glite-wms-manager-3.3.1-3.sl5
</verbatim>

   * wget --no-check-certificate https://twiki.cern.ch/twiki/pub/EMI/EMI1Updates/testing.repo -O /etc/yum.repos.d/testing.repo
   * yum clean all
   * service gLite stop
   * yum update
      * [[%ATTACHURL%/update.log][update.log]]: Update log
   * /opt/glite/yaim/bin/yaim -c -s site-info.def -n WMS
   * Check final version: <verbatim>[root@emi-demo12 siteinfo]# rpm -qa|grep wms-manager
glite-wms-manager-3.3.1-4.sl5
</verbatim>

The wms emi-demo12 is configured in proxy mode, pointing to the LB devel17.cnaf.infn.it

---++ Unit Tests 

*N/A*: Tests will be available with EMI-2.

---++ Basic Functionality tests

---+++ Submission
Where possible the tests should use internal testbed CEs (located in Padova).
This is accomplished by adding the following line to wms-command.conf:
<verbatim>
DEFAULTREQ= other.GlueCEStateStatus == "Testing"
</verbatim>
This is not possible, for example, for parallel jobs.

---++++ Normal
<verbatim>[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 1 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 10:28:22
===> Ended  : 10:32:49
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120316102822.log
[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 1 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 10:30:51
===> Ended  : 10:35:08
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120316103051.log
   [capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V infngrid -d 3 -l -t 1 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 10:31:52
===> Ended  : 10:37:14
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120316103152.log
</verbatim>   
* [[%ATTACHURL%/submit.tgz][submit.tgz]]: submit.tgz
---+++++ Proxy mode
Pointing the testsuite at emi-demo12:

<verbatim>
[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 1 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo12.cnaf.infn.it
===> Started: 10:50:40
===> Ended  : 10:56:37
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120316105040.log
</verbatim>
 * [[%ATTACHURL%/WMSService-TS_20120316105040.log][WMSService-TS_20120316105040.log]]: WMSService-TS_20120316105040.log

---++++ Jdl with _DataRequirements_
<verbatim>
[capannini@devel15 WMS-service]$ cat ~/certification/data-req.jdl
###########################################
#      JDL with Data Requirements         #
###########################################

Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";
Prologue = "prologue.sh";

FuzzyRank = true;
Environment={"LFC_HOST=prod-lfc-shared-central.cern.ch", "LFC_HOME=prod-lfc-shared-central.cern.ch:/grid/dteam/aleph"};
InputSandbox = {"JDLs/prologue.sh"};
OutputSandbox = {"std.out", "std.err","prologue.out", "PI_1M.txt"};

DataRequirements = {
[
DataCatalogType = "DLI";
DataCatalog ="http://prod-lfc-shared-central.cern.ch:8085/"; 
InputData = { "lfn:/grid/dteam/fcapanni/PI_1M.txt" };
]
};

DataAccessProtocol = "gsiftp";
AllowZippedISB=true;
RetryCount = 1;
ShallowRetryCount = 2;

[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 1 -i -j ~/certification/data-req.jdl
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 11:33:22
===> Ended  : 11:54:21
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120316113322.log
</verbatim>
* [[%ATTACHURL%/WMSService-TS_20120316113322.log][WMSService-TS_20120316113322.log]]: WMSService-TS_20120316113322.log

---++++ Jdl with _ISBBaseURI_ and _OSBDestURI_
Test on CREAM ce:

<verbatim>
[capannini@cert-17 certification]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf URI.jdl 

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://emi-demo11.cnaf.infn.it:9000/FNmMujUg_2yIhS0iY8h9YA

==========================================================================


[capannini@cert-17 certification]$ 
[capannini@cert-17 certification]$ 
[capannini@cert-17 certification]$ 
[capannini@cert-17 certification]$ 
[capannini@cert-17 certification]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/FNmMujUg_2yIhS0iY8h9YA


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/FNmMujUg_2yIhS0iY8h9YA
Current Status:     Done (Success)
Logged Reason(s):
    - Transfer to CREAM failed due to exception: Failed to create a delegation id for job https://emi-demo11.cnaf.infn.it:9000/FNmMujUg_2yIhS0iY8h9YA: reason is User C=IT,O=INFN,OU=Personal Certificate,L=CNAF,CN=Fabio Capannini not authorized for operation {http://www.gridsite.org/namespaces/delegation-2}getProxyReq
    - Job Terminated Successfully
Exit code:          0
Status Reason:      Job Terminated Successfully
Destination:        cccreamceli10.in2p3.fr:8443/cream-sge-short
Submitted:          Fri Mar 16 14:29:07 2012 CET
==========================================================================

[capannini@cert-17 certification]$ glite-wms-job-output https://emi-demo11.cnaf.infn.it:9000/FNmMujUg_2yIhS0iY8h9YA

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


================================================================================

			JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://emi-demo11.cnaf.infn.it:9000/FNmMujUg_2yIhS0iY8h9YA
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/capannini_FNmMujUg_2yIhS0iY8h9YA

================================================================================

[capannini@cert-17 certification]$ ls /tmp/jobOutput/capannini_FNmMujUg_2yIhS0iY8h9YA
std.err

Log on devel09.cnaf.infn.it:

[root@devel09 ~]# cat /tmp/myjobOutput 
FNmMujUg_2yIhS0iY8h9YA
total 76
-rwxr-xr-x 1 dteam009 dteam 80 Mar 16 14:33 exe.sh
-rw-r--r-- 1 dteam009 dteam  0 Mar 16 14:33 std.err
-rw-r--r-- 1 dteam009 dteam 23 Mar 16 14:33 std.out
-rw-r--r-- 1 dteam009 dteam  6 Mar 16 14:33 test1
-rw-r--r-- 1 dteam009 dteam  6 Mar 16 14:33 test2

===>   test PASSED


</verbatim>

Same process with an LCG CE:
<verbatim>
[capannini@cert-17 certification]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf URI.jdl 

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://emi-demo11.cnaf.infn.it:9000/nz4pKzrc453q3uSn-q8vsQ

==========================================================================

[capannini@cert-17 certification]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/nz4pKzrc453q3uSn-q8vsQ


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/nz4pKzrc453q3uSn-q8vsQ
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job terminated successfully
Destination:        gridce2.pi.infn.it:2119/jobmanager-lcglsf-cert
Submitted:          Fri Mar 16 14:51:57 2012 CET
==========================================================================

[capannini@cert-17 certification]$ 
[capannini@cert-17 certification]$ 
[capannini@cert-17 certification]$ 
[capannini@cert-17 certification]$ glite-wms-job-output https://emi-demo11.cnaf.infn.it:9000/nz4pKzrc453q3uSn-q8vsQ

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


================================================================================

			JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://emi-demo11.cnaf.infn.it:9000/nz4pKzrc453q3uSn-q8vsQ
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/capannini_nz4pKzrc453q3uSn-q8vsQ

================================================================================


[capannini@cert-17 certification]$ ls /tmp/jobOutput/capannini_nz4pKzrc453q3uSn-q8vsQ
std.err

Log on devel09:

[root@devel09 ~]# cat /tmp/myjobOutput 
nz4pKzrc453q3uSn-q8vsQ
total 96
-rwxr-xr-x 1 dteam016 dteam  80 Mar 16 14:53 exe.sh
-rw-r--r-- 1 dteam016 dteam 118 Mar 16 14:53 https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fnz4pKzrc453q3uSn-q8vsQ.output
-rw-r--r-- 1 dteam016 dteam   0 Mar 16 14:53 std.err
-rw-r--r-- 1 dteam016 dteam  23 Mar 16 14:53 std.out
-rw-r--r-- 1 dteam016 dteam   6 Mar 16 14:53 test1
-rw-r--r-- 1 dteam016 dteam   6 Mar 16 14:53 test2
-rw------- 1 dteam016 dteam 103 Mar 16 14:53 tmp.dSuMI19622
-rw------- 1 dteam016 dteam   2 Mar 16 14:53 tmp.iYrdk19621
-rw------- 1 dteam016 dteam   2 Mar 16 14:53 tmp.kpcpo19636
-rw------- 1 dteam016 dteam   0 Mar 16 14:53 tmp.nAPiv19635
-rw------- 1 dteam016 dteam 103 Mar 16 14:53 tmp.rUfjz19637

===>   test PASSED
</verbatim>

---++++ Jdl with _ISBBaseURI_ and _OSBBaseDestURI_

Submit to CREAM CE
<verbatim>
[capannini@cert-17 certification]$ cat BaseURI.jdl 
###########################################
#  JDL with ISBURI & OSBBaseDestURI       #
###########################################

Executable = "exe.sh";
StdOutput = "std.out";
StdError = "std.err";

InputSandboxBaseURI = "gsiftp://emi-demo12.cnaf.infn.it:2811/tmp";

FuzzyRank = true;
InputSandbox = {"exe.sh", "test1", "test2"};
OutputSandbox = {"std.out", "std.err"};
OutputSandboxBaseDestURI="gsiftp://devel20.cnaf.infn.it:2811/tmp";

#requirements = other.GlueCEStateStatus == "Testing" &&  regexp("/cream", other.GlueCEUniqueID);
requirements = regexp("/cream", other.GlueCEUniqueID);

RetryCount = 1;
ShallowRetryCount = 2;

[capannini@cert-17 certification]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf BaseURI.jdl 

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://emi-demo11.cnaf.infn.it:9000/hlAzaQOlelzhpDmzgGL88g

==========================================================================

[capannini@cert-17 certification]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/hlAzaQOlelzhpDmzgGL88g


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/hlAzaQOlelzhpDmzgGL88g
Current Status:     Done (Success)
Logged Reason(s):
    - Transfer to CREAM failed due to exception: Failed to create a delegation id for job https://emi-demo11.cnaf.infn.it:9000/hlAzaQOlelzhpDmzgGL88g: reason is User C=IT,O=INFN,OU=Personal Certificate,L=CNAF,CN=Fabio Capannini not authorized for operation {http://www.gridsite.org/namespaces/delegation-2}getProxyReq
    - Job Terminated Successfully
Exit code:          0
Status Reason:      Job Terminated Successfully
Destination:        gridce0.pi.infn.it:8443/cream-lsf-cert
Submitted:          Fri Mar 16 16:13:56 2012 CET
==========================================================================

[capannini@cert-17 certification]$ 
[capannini@cert-17 certification]$ 
[capannini@cert-17 certification]$ glite-wms-job-output https://emi-demo11.cnaf.infn.it:9000/hlAzaQOlelzhpDmzgGL88g

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


================================================================================

			JOB GET OUTPUT OUTCOME

No output files to be retrieved for the job:
https://emi-demo11.cnaf.infn.it:9000/hlAzaQOlelzhpDmzgGL88g

================================================================================

Log on devel20:

[root@devel20 ~]# ls /tmp/std*
/tmp/std.err  /tmp/std.out
[root@devel20 ~]# 
[root@devel20 ~]# 
[root@devel20 ~]# cat /tmp/std.out 
hlAzaQOlelzhpDmzgGL88g
total 76
-rwxr-xr-x 1 dteam015 dteam 80 Mar 16 16:14 exe.sh
-rw-r--r-- 1 dteam015 dteam  0 Mar 16 16:14 std.err
-rw-r--r-- 1 dteam015 dteam 23 Mar 16 16:14 std.out
-rw-r--r-- 1 dteam015 dteam  6 Mar 16 16:14 test1
-rw-r--r-- 1 dteam015 dteam  6 Mar 16 16:14 test2

===>     === test PASSED === 

</verbatim>

Submit to an LCG CE:


-- Main.FabioCapannini - 2012-03-15
Attachments
Topic attachments
I	Attachment	Action	Size	Date	Who	Comment
log	WMSService-TS_20120316105040.log	manage	12.1 K	2012-03-16 - 10:30	FabioCapannini
log	WMSService-TS_20120316113322.log	manage	34.8 K	2012-03-16 - 11:02	FabioCapannini
log	install.log	manage	122.4 K	2012-03-15 - 11:11	FabioCapannini	Installation log
log	update.log	manage	31.1 K	2012-03-15 - 11:48	FabioCapannini	Update log
log	yaim-config.log	manage	9.0 K	2012-03-15 - 11:25	FabioCapannini	yaim configuration log
tgz	submit.tgz	manage	3.0 K	2012-03-16 - 09:45	FabioCapannini
Topic revision: r6 - 2012-03-16 - FabioCapannini
Account
- Log In
~~Edit~~
~~Attach~~