Tags:
, view all tags

WMS v. 3.4 test report

Summary

  • Component: WMS 3.4

  • ETICS Subsystem Configuration Name: emi-wms_R_3_4_0_7

  • VCS Tag:

  • EMI Major Release: EMI 1 (Kebnekaise)

  • Platform: sl5 epel

  • Certification report: not yet available

  • Outcome: Certification under way

Deployment Tests

Clean installation

  • Starting from a SL5 x86_64 kernel machine:

[root@cert-13 ~]# cat /etc/redhat-release 
Scientific Linux SL release 5.8 (Boron)
[root@cert13 ~]# uname -a
Linux cert-13.cnaf.infn.it 2.6.18-308.13.1.el5 

INSTALL LOG : * 20120831_151144_WMS3.4_SL5_EMI2INSTALL.log: 20120831_151144_WMS3.4_SL5_EMI2INSTALL.log

COMMAND SEQUENCE:

wget http://emisoft.web.cern.ch/emisoft/dist/EMI/2/RPM-GPG-KEY-emi -P /etc/pki/rpm-gpg/
yum -y install yum-protectbase
yum -y install yum-priorities
rpm -ivh http://emisoft.web.cern.ch/emisoft/dist/EMI/2/sl5/x86_64/base/emi-release-2.0.0-1.sl5.noarch.rpm
wget --no-check-certificate http://emisoft.web.cern.ch/emisoft/dist/EMI/certification/repos/emi2-cert-sl5-base.repo -O /etc/yum.repos.d/emi2-cert-sl5-base.repo
yum clean all
yum makecache
yum -y install emi-release
yum -y install yum-protectbase
yum -y install yum-priorities
yum -y install ca-policy-egi-core
yum -y install emi-wms 
yum -y install condor-emi
/opt/glite/yaim/bin/yaim -v -s site-info.def -n WMS -n LB 
/opt/glite/yaim/bin/yaim -c -d 6 -s site-info.def -n WMS -n LB
chown root:root /var /var/log 
source /usr/libexec/grid-env.sh; /usr/sbin/glite-wms-create-proxy.sh /var/glite/wms.proxy /var/log/wms/create_proxy.log
rpm -qa |grep emi |sort
service gLite stop
service gLite start
service gLite status

Update Installation

INSTALL LOG : * 20120903_180452_TEST_EMI1TOEMI2UPDATE.log: 20120903_180452_TEST_EMI1TOEMI2UPDATE.log

COMMAND SEQUENCE:

wget http://emisoft.web.cern.ch/emisoft/dist/EMI/1/RPM-GPG-KEY-emi -P /etc/pki/rpm-gpg/
yum -y install yum-protectbase
yum -y install yum-priorities
rpm -ivh http://emisoft.web.cern.ch/emisoft/dist/EMI/1/sl5/x86_64/base/emi-release-1.0.0-1.sl5.noarch.rpm
yum clean all
yum makecache
yum -y install emi-release
yum -y install yum-protectbase
yum -y install yum-priorities
yum -y install ca-policy-egi-core
yum -y install emi-wms
/opt/glite/yaim/bin/yaim -v -s site-info.def -n WMS -n LB 
/opt/glite/yaim/bin/yaim -c -d 6 -s site-info.def -n WMS -n LB
rpm -qa |grep emi
rpm -qa |grep wms
service gLite restart
rpm -e emi-release
rpm -ivh http://emisoft.web.cern.ch/emisoft/dist/EMI/2/sl5/x86_64/base/emi-release-2.0.0-1.sl5.noarch.rpm
wget --no-check-certificate http://emisoft.web.cern.ch/emisoft/dist/EMI/certification/repos/emi2-cert-sl5-base.repo -O /etc/yum.repos.d/emi2-cert-sl5-base.repo
yum clean all
yum makecache
yum -y remove condor condor-lcg
yum -y install emi-wms condor-emi
yum -y update
rpm -qa |grep emi
rpm -qa |grep wms
chown root:root /var /var/log 
source /usr/libexec/grid-env.sh; /usr/sbin/glite-wms-create-proxy.sh /var/glite/wms.proxy /var/log/wms/create_proxy.log 
/opt/glite/yaim/bin/yaim -v -s site-info.def -n WMS -n LB 
/opt/glite/yaim/bin/yaim -c -d 6 -s site-info.def -n WMS -n LB
service gLite restart
chmod a+r /etc/nsswitch.conf
/sbin/restorecon /etc/nsswitch.conf
/etc/init.d/glite-wms-wmproxy restart

Unit Tests

N/A: Tests will be available with EMI-2.

Basic Functionality tests

Submission

Normal

[dongiovanni@emitestbed08 ~]$ glite-wms-job-status  https://cert-13.cnaf.infn.it:9000/bJ-1NLTGcc3OjUldvbhZrg
======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:
Status info for the Job : https://cert-13.cnaf.infn.it:9000/bJ-1NLTGcc3OjUldvbhZrg
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job Terminated Successfully
Destination:        ctb04.gridctb.uoa.gr:8443/cream-pbs-emitesters
Submitted:          Fri Aug 31 15:51:39 2012 CEST
==========================================================================

SUBMISSION TESTSUITE: * WMSService-TS_20120831163229.log: WMSService-TS_20120831163229.log

[dongiovanni@cert-17 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 1 -i 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
Enter the user proxy password:
 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 
 Test 1A: Submit to an LCG-CE 
 Test 1B: Submit to a CREAM CE 
 Test 1C: Submit without restrictions 
===> 
===> Test: WMS-job-cycle.py
===> WMS: cert-13.cnaf.infn.it
===> Started: 16:32:29
===> Ended  : 16:57:23
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120831163229.log

Normal on EMI1->EMI2 UPDATE Installation

[dongiovanni@cert-17 ~]$ glite-wms-job-status https://cert-13.cnaf.infn.it:9000/4dapc9K3V1IWpGjjdREq2w 
======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:
Status info for the Job : https://cert-13.cnaf.infn.it:9000/4dapc9K3V1IWpGjjdREq2w
Current Status:     Scheduled 
Status Reason:      unavailable
Destination:        ctb04.gridctb.uoa.gr:8443/cream-pbs-emitesters
Submitted:          Tue Sep  4 12:06:01 2012 CEST
==========================================================================

SUBMISSION TESTSUITE: * WMSService-TS_20120904121133.log: WMSService-TS_20120904121133.log

[dongiovanni@cert-17 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 1 -i 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
Enter the user proxy password:
 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 
 Test 1A: Submit to an LCG-CE 
 Test 1B: Submit to a CREAM CE 
 Test 1C: Submit without restrictions 
===> ERROR: Command glite-wms-job-status --verbosity 0 https://cert-13.cnaf.infn.it:9000/8TrsHieqXo1ukD1Aaw9S2A failed. Failure message: **** Error: API_NATIVE_ERROR ****  
Error while calling the "Job:getStatus" native api 
Unable to retrieve the status for: https://cert-13.cnaf.infn.it:9000/8TrsHieqXo1ukD1Aaw9S2A
glite.lb.Exception: edg_wll_JobStatus: Connection refused: edg_wll_gss_connect()
        at glite::lb::Job::status[./src/Job.cpp:87]
===> 
===> Test: WMS-job-cycle.py
===> WMS: cert-13.cnaf.infn.it
===> Started: 12:11:33
===> Ended  : 12:58:05
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120904121133.log

Proxy Mode

SUBMISSION TESTLOG: * WMSService-TS_20120903160648.log: WMSService-TS_20120903160648.log

[dongiovanni@cert-17 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 1 -i 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
Enter the user proxy password:
 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 
 Test 1A: Submit to an LCG-CE 
 Test 1B: Submit to a CREAM CE 
 Test 1C: Submit without restrictions 
===> 
===> Test: WMS-job-cycle.py
===> WMS: cert-13.cnaf.infn.it
===> Started: 16:06:48
===> Ended  : 16:33:44
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120903160648.log

ARC CE
Job submitted on a production CE using the cms VO.
[capannini@cert-17 ~]$ lcg-infosites --is egee-bdii.cnaf.infn.it --vo cms ce |grep arc
     68         0           10         2        10   arc.univ.kiev.ua:2811/nordugrid-torque-arc
    768         0          909       600       309   gsiftp://jade-cms.hip.fi:2811/jobs?queue=arc
Jdl attributes
Some of the tests failed at first submission so they have been repeated until successful execution.
Define the following variables in wms-command.conf:
# For Jdl attributes tests
SE=gridsrm.pi.infn.it
LFC=prod-lfc-shared-central.cern.ch
ISB_DEST_HOSTNAME=emi-demo11.cnaf.infn.it
ISB_DEST_USERNAME=root
ISB_DEST_PASSWORD=***
OSB_DEST_HOSTNAME=emi-demo11.cnaf.infn.it
OSB_DEST_USERNAME=root
OSB_DEST_PASSWORD=***

[capannini@cert-17 WMS-service]$ python WMS-jdl-attributes-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -i -t 1,2,4,5,6,7
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle for normal job with non default jdl files 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Jdl with AllowZippedISB 


 Test 2: Jdl with ExpiryTime 


 Test 4: Jdl with DataRequirements 


 Submit to CREAM CE 


 Submit to LCG CE 


 Test 5: Jdl with InputSandboxBaseURI 


 Submit to CREAM CE 


 Submit to LCG CE 


 Test 6: Jdl with OutputSandboxBaseDestURI 


 Please set the required variables for OSB node in test's configuration file 


 Test 7: Jdl with OutputSandboxDestURI 


 Please set the required variables for OSB node in test's configuration file 


[capannini@cert-17 WMS-service]$ python WMS-jdl-attributes-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -i -t 6,7
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle for normal job with non default jdl files 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 6: Jdl with OutputSandboxBaseDestURI 


 Submit to CREAM CE 


 Submit to LCG CE 


 Test 7: Jdl with OutputSandboxDestURI 


 Submit to CREAM CE 

===> ERROR: Job finishes with status: Aborted cannot retrieve output

[capannini@cert-17 WMS-service]$ python WMS-jdl-attributes-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -i -t 7
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle for normal job with non default jdl files 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 7: Jdl with OutputSandboxDestURI 


 Submit to CREAM CE 


 Submit to LCG CE 

===> 
===> Test: WMS-jdl-attributes-job-cycle.py
===> WMS: cert-13.cnaf.infn.it
===> Started: 12:07:57
===> Ended  : 12:28:46
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120907120757.log

Collection

Bulk submission
[capannini@cert-17 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 2 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 2: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) 


 Test 2A: Submit to an LCG-CE 


 Test 2B: Submit to a CREAM CE 


 Test 2C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: cert-13.cnaf.infn.it
===> Started: 14:26:13
===> Ended  : 15:35:35
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120907142613.log

[capannini@cert-17 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 2 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 2: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) 


 Test 2A: Submit to an LCG-CE 


 Test 2B: Submit to a CREAM CE 


 Test 2C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: cert-13.cnaf.infn.it
===> Started: 14:26:31
===> Ended  : 15:06:12
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120907142631.log

[capannini@cert-17 WMS-service]$ myproxy-init -d
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini
Enter GRID pass phrase for this identity:
Creating proxy ............................................................................................................................................. Done
Proxy Verify OK
Your proxy is valid until: Fri Sep 14 12:29:57 2012
Enter MyProxy pass phrase:
Verifying - Enter MyProxy pass phrase:
A proxy valid for 168 hours (7.0 days) for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini now exists on myproxy.cnaf.infn.it.
[capannini@cert-17 WMS-service]$ myproxy-logon -d --voms dteam
Enter MyProxy pass phrase:
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
Contacting  voms.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr] "dteam" Done
Creating proxy ......................................... Done

Your proxy is valid until Fri Sep  7 23:30:28 2012
A credential has been received for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini in /tmp/x509up_u504.
[capannini@cert-17 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u504
timeleft  : 10:59:51
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 11:58:50
uri       : voms.hellasgrid.gr:15004
[capannini@cert-17 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -d 3 -l -t 2
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Set 2: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) 


 Test 2A: Submit to an LCG-CE 


 Test 2B: Submit to a CREAM CE 


 Test 2C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: cert-13.cnaf.infn.it
===> Started: 12:30:53
===> Ended  : 13:18:02
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20120907123053.log

All in a jdl

Performance test

Stress test

  • 2880 collection of 25 nodes each (total: 72000 jobs)
  • One collection every 60 seconds
  • one user
  • Use LCG-CEs and CREAM-CEs (with different batch systems)
  • Use automatic-delegation
  • The job is a `ls -l`
  • Resubmission is enabled
  • Enable proxy renewal

[capannini@cert-17 StressTest]$ ./StressTest.sh -t 2880 -n 25 -c ~/wms_cert-13.conf -j ls.jdl
Test starts on ven ago 31 17:32:10 CEST 2012: submit 2880 collections (of 25 nodes) with a frequency of 60 seconds using 1 users.

Results

2880 jobs submitted in 9286 seconds: 1/3/35 (min/avg/max)
Test finishes at dom set 2 17:31:34 CEST 2012

[capannini@cert-17 output]$ ../../loginfo.sh -e 57

 Results:
DONE   :      67701 
NOTDone:         13 
Aborted:        386 
------------------------
Total  :      68100 

[capannini@cert-17 output]$ cat *.done|wc -l
67701
[capannini@cert-17 output]$ cat *.done|grep '8443/cream' |wc -l
65744
[capannini@cert-17 output]$ cat *.done|grep '2119/jobmanager'|wc -l
1953

Done jobs on CREAM CE: 65744 (97.1%)
Done jobs on LCG CE: 1953 (2.9%)

-- DaniloDongiovanni - 2012-08-31

Topic attachments
I Attachment Action Size Date WhoSorted ascending Comment
Unknown file formatlog 20120831_151144_WMS3.4_SL5_EMI2INSTALL.log manage 529.0 K 2012-08-31 - 14:00 DaniloDongiovanni  
Unknown file formatlog 20120903_180452_TEST_EMI1TOEMI2UPDATE.log manage 778.7 K 2012-09-04 - 10:12 DaniloDongiovanni  
Unknown file formatlog WMSService-TS_20120831163229.log manage 40.0 K 2012-09-03 - 09:16 DaniloDongiovanni  
Unknown file formatlog WMSService-TS_20120903160648.log manage 42.8 K 2012-09-03 - 14:55 DaniloDongiovanni  
Unknown file formatlog WMSService-TS_20120904121133.log manage 71.2 K 2012-09-04 - 11:39 DaniloDongiovanni  
Compressed Zip archivetgz coll.tgz manage 13.3 K 2012-09-07 - 14:05 FabioCapannini  
Compressed Zip archivetgz jdl.tgz manage 13.3 K 2012-09-07 - 12:11 FabioCapannini  
Edit | Attach | PDF | History: r54 | r14 < r13 < r12 < r11 | Backlinks | Raw View | More topic actions...
Topic revision: r12 - 2012-09-07 - FabioCapannini
 
  • Edit
  • Attach
This site is powered by the TWiki collaboration platformCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback