Certification report patch 4880
Author(s): Alessio Gianelle
Outcome: In certification
Information
Clean installation
- Starting from an sl4 we install the WMS:
- wget http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/3.1/lcg-CA.repo -O /etc/yum.repos.d/lcg-CA.repo
- yum install lcg-CA
- wget http://etics-repository.cern.ch/repository/pm/registered/repomd/name/patch_4880_1/etics-registered-build-by-name.repo -O /etc/yum.repos.d/glite.repo
- yum install glite-WMS
- /opt/glite/yaim/bin/yaim -c -s site-info.def -n WMS
- wget http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/3.1/lcg-CA.repo
Upgrade from production
- Starting from a Production WMS:
[root@devel18 etc]# yum update Loading "kernel-module" plugin Loading "versionlock" plugin Loading "installonlyn" plugin Loading "changelog" plugin Loading "tsflags" plugin Loading "protectbase" plugin Setting up Update Process Setting up repositories dag 100% |=========================| 1.1 kB 00:00 slc-update 100% |=========================| 951 B 00:00 slc4-base 100% |=========================| 951 B 00:00 slc4-updates 100% |=========================| 951 B 00:00 EGI-trustanchors 100% |=========================| 1.9 kB 00:00 sl-base 100% |=========================| 1.1 kB 00:00 slc4-extras 100% |=========================| 951 B 00:00 glite-WMS 100% |=========================| 951 B 00:00 slc-base 100% |=========================| 1.1 kB 00:00 CA 100% |=========================| 951 B 00:00 ETICS-name-patch_4880_1 764 B 00:00 cnaf-local 100% |=========================| 951 B 00:00 Reading repository metadata in from local files primary.xml.gz 100% |=========================| 4.4 MB 00:00 dag : ################################################## 12410/12410 Added 12410 new packages, deleted 0 old in 50.16 seconds primary.xml.gz 100% |=========================| 2.7 MB 00:00 slc-update: ################################################## 8677/8677 Added 8677 new packages, deleted 0 old in 46.93 seconds primary.xml.gz 100% |=========================| 707 kB 00:00 slc4-base : ################################################## 1923/1923 Added 1923 new packages, deleted 0 old in 10.18 seconds primary.xml.gz 100% |=========================| 3.2 MB 00:00 slc4-updat: ################################################## 8573/8573 Added 8573 new packages, deleted 0 old in 45.44 seconds primary.xml.gz 100% |=========================| 19 kB 00:00 EGI-trusta: ################################################## 96/96 Added 96 new packages, deleted 0 old in 0.28 seconds primary.xml.gz 100% |=========================| 599 kB 00:00 sl-base : ################################################## 1848/1848 Added 1848 new packages, deleted 0 old in 9.98 seconds primary.xml.gz 100% |=========================| 233 kB 00:00 slc4-extra: ################################################## 1442/1442 Added 1442 new packages, deleted 0 old in 3.17 seconds primary.xml.gz 100% |=========================| 75 kB 00:00 glite-WMS : ################################################## 279/279 Added 279 new packages, deleted 0 old in 1.70 seconds primary.xml.gz 100% |=========================| 604 kB 00:00 slc-base : ################################################## 1923/1923 Added 1923 new packages, deleted 0 old in 8.46 seconds primary.xml.gz 100% |=========================| 15 kB 00:00 CA : ################################################## 95/95 Added 95 new packages, deleted 0 old in 0.21 seconds primary.xml.gz 35 kB 00:00 ETICS-name: ################################################## 94/94 Added 94 new packages, deleted 0 old in 0.48 seconds primary.xml.gz 100% |=========================| 2.7 kB 00:00 cnaf-local: ################################################## 17/17 Added 17 new packages, deleted 0 old in 0.03 seconds Reading version lock configuration 108 packages excluded due to repository protections Resolving Dependencies --> Populating transaction set with selected packages. Please wait. ---> Downloading header for glite-jdl-api-cpp to pack into transaction set. glite-jdl-api-cpp-3.1.18- 100% |=========================| 5.3 kB 00:00 ---> Package glite-jdl-api-cpp.i386 0:3.1.18-0.slc4 set to be updated ---> Downloading header for glite-WMS to pack into transaction set. glite-WMS-3.1.32-0.slc4.i 100% |=========================| 4.4 kB 00:00 ---> Package glite-WMS.i386 0:3.1.32-0.slc4 set to be updated --> Running transaction check Dependencies Resolved ============================================================================= Package Arch Version Repository Size ============================================================================= Updating: glite-WMS i386 3.1.32-0.slc4 ETICS-name-patch_4880_1 5.3 k glite-jdl-api-cpp i386 3.1.18-0.slc4 ETICS-name-patch_4880_1 2.6 M Transaction Summary ============================================================================= Install 0 Package(s) Update 2 Package(s) Remove 0 Package(s) Total download size: 2.6 M Is this ok [y/N]: y Downloading Packages: (1/2): glite-jdl-api-cpp-3.1.18-0.slc4.i386.rpm 2.6 MB 00:00 (2/2): glite-WMS-3.1.32-0 100% |=========================| 5.3 kB 00:00 Running Transaction Test Finished Transaction Test Transaction Test Succeeded Running Transaction Updating : glite-jdl-api-cpp ######################### [1/4] Updating : glite-WMS ######################### [2/4] Cleanup : glite-jdl-api-cpp ######################### [3/4] Cleanup : glite-WMS ######################### [4/4] Updated: glite-WMS.i386 0:3.1.32-0.slc4 glite-jdl-api-cpp.i386 0:3.1.18-0.slc4 Complete!
Test Report
The test report has been produced following the guidelines from here
List Match
List match without data
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the gLite-WMS service ===> + Description: WMS Job List-match Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: check if command glite-wms-job-list-match exists Test 2: test a simple matching Test 3: check --delegationid option Test 4: match only CREAM CE Test 5: try a failure matching (Requirements == false) ===> ===> Test: WMS-listmatch-test.py ===> WMS: cream-44.pd.infn.it ===> Started: 15:42:43 ===> Ended : 15:42:55 ===> ===> === test PASSED ===
List match with data
[ale@cream-12 DataReq]$ glite-wms-job-list-match -a -c ../etc/wmp_cream-44.conf data-req.jdl
Connecting to the service https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server
==========================================================================
COMPUTING ELEMENT IDs LIST
The following CE(s) matching your job requirements have been found:
*CEId*
- cert-15.pd.infn.it:8443/cream-lsf-cert
- cert-37.pd.infn.it:8443/cream-lsf-cert
- cream-19.pd.infn.it:8443/cream-lsf-cert
- cream-19.pd.infn.it:8443/cream-lsf-creamcert1
- cream-19.pd.infn.it:8443/cream-lsf-creamcert2
- cream-24.pd.infn.it:8443/cream-lsf-cert
- cream-24.pd.infn.it:8443/cream-lsf-testbedB_1
- cream-24.pd.infn.it:8443/cream-lsf-testbedB_2
- cream-26:2119/jobmanager-lcgpbs-cert
- cream-26:2119/jobmanager-lcgpbs-creamtest1
- cream-26:2119/jobmanager-lcgpbs-creamtest2
- cream-32.pd.infn.it:8443/cream-pbs-creamtest1
- cream-32.pd.infn.it:8443/cream-pbs-creamtest2
- grid010.ct.infn.it:2119/jobmanager-lcgpbs-cert
- prod-ce-01.pd.infn.it:8443/cream-lsf-cert
- prod-ce-02.pd.infn.it:2119/jobmanager-lcglsf-cert
==========================================================================
Submission
Normal, Collection, parametric, dag and parallel jobs
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the gLite-WMS command line interface ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: Submit a normal job to an LCG-CE Test 2: Submit a normal job to a CREAM CE Test 3: Submit a normal job Test 4: Submit a job collection to an LCG-CE Test 5: Submit a job collection to CREAM CE Test 6: Submit a job collection Test 7: Submit a parametric job to an LCG-CE Test 8: Submit a parametric job to CREAM CE Test 9: Submit a parametric job Test 10: Submit a DAG job Test 11: Submit MPI job to an LCG-CE Test 12: Submit MPI job to CREAM CE Test 13: Submit MPI job ===> ===> Test: WMS-job-cycle.py ===> Started: 10:19:43 ===> Ended : 14:48:37 ===> ===> >>> TEST SUCCESS <<<
Perusal
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the gLite-WMS command line interface ===> + Description: test glite-wms-job-persual commmad ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1 ===> >>> Check if command glite-wms-job-perusal exists Test 2 ===> >>> ===> >>> Test --version option ===> >>> We are testing WMS User Interface version 3.3.2 Test 3 ===> >>> ===> >>> Test --set option Test 4 ===> >>> ===> >>> Test --unset option Test 5 ===> >>> ===> >>> Test --set and --config options Test 6 ===> >>> ===> >>> Test --set and --output options Test 7 ===> >>> ===> >>> Test --set and --logfile options Test 8 ===> >>> ===> >>> Test --set --logfile and --debug option Test 9 ===> >>> ===> >>> Test --set and --nodisplay options ===> >>> ===> >>> Test --set option Test 10 ===> >>> BEWARE default min perusal interval is 1000 secs, so this phase could take many minutes ===> >>> Test --get option Test 11 ===> >>> ===> >>> Test --get option with --dir Test 12 ===> >>> ===> >>> Test --get option with --nodisplay and --all Test 13 ===> >>> ===> ===> Test: WMS-command_job-perusal.py ===> Started: 15:45:37 ===> Ended : 19:23:49 ===> ===> === test PASSED ===
Cancel
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the gLite-WMS command line interface ===> + Description: WMS Job Cancel Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: Try to cancel a normal job Test 2: Try to cancel a DAG job Test 3: Try to cancel one node of a DAG job Test 4: Try to cancel a collection job Test 5: Try to cancel one node of a collection job Test 6: Try to cancel a parametric job Test 7: Try to cancel one node of a parametric job Test 8: Try to cancel a MPI job Test 9: Try to cancel a DoneOK job ===> ===> Test: WMS-job-cancel.py ===> WMS: cream-44.pd.infn.it ===> Started: 12:05:03 ===> Ended : 12:40:19 ===> ===> === test PASSED ===
Delegation
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the gLite-WMS command line interface ===> + Description: test delegation operation ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: Check --autm-delegation option Test 2: Check --delegationid option Test 3: Try to delegate with a short proxy and check the validity Test 4: Works with expired proxy ===> ===> Test: WMS-delegation-test.py ===> WMS: cream-44.pd.infn.it ===> Started: 14:00:15 ===> Ended : 14:01:38 ===> ===> === test PASSED ===
Prologue/Epilogue
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the gLite-WMS command line interface ===> + Description: WMS test prologue and epilogue attributes ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: Test prologue attribute Test 2: Test epilogue attribute Test 3: Test prologue and epilogue attributes ===> ===> Test: WMS-prologue-epilogue.py ===> WMS: cream-44.pd.infn.it ===> Started: 17:38:28 ===> Ended : 17:43:57 ===> ===> === test PASSED ===
Test proxy renewal
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the gLite-WMS command line interface ===> + Description: WMS test proxy renewal operation ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: Test proxy renewal with LCG CE Test 2: Test proxy renewal with CREAM CE Test 3: Test proxy renewal with LCG CE (without setting MYPROXYSERVER) Test 4: Test proxy renewal with CREAM CE (without setting MYPROXYSERVER) ===> ===> Test: WMS-proxy-renewal.py ===> WMS: devel18.cnaf.infn.it ===> Started: 13:15:52 ===> Ended : 13:54:50 ===> ===> === test PASSED ===
Shallow/Deep Resubmission
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the gLite-WMS command line interface ===> + Description: WMS Job Resubmission Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: Try a shallow resubmission Test 2: Try a deep resubmission ===> ===> Test: WMS-job-resubmit.py ===> WMS: cream-44.pd.infn.it ===> Started: 18:17:05 ===> Ended : 18:46:54 ===> ===> === test PASSED ===
Performance test
Collection of 1000 nodes
- Collection ID: https://devel07.cnaf.infn.it:9000/0sqJfQjOX17cHiF5vONNDw
- Nodes are submitted to a production environment:
- 320 are executed on a LCG CE
- 680 are executed in a CREAM CE
- Final Status:
- 1000 Done (Success)
Stress Test
Description:- 355 collection of 25 nodes each (total: 8875 jobs)
- One collection every 60 seconds
- two users
- Use LCG-CEs and CREAM-CEs from production enviroment (with different batch systems)
- Use automatic-delegation
- The job is a "sleep random(666)"
- Resubmission is enabled
- Enable proxy renewal
- DONE OK: 8636 (97.3%)
- Aborted: 239 (proxy expired) (2.7%)
Check bugs
Bugs #82654: URLs in ISB are sometimes converted to lowercase by WMproxy FIXED
Submit this jdl:
[ale@cream-12 UI]$ cat 82654.jdl
[
Type = "collection";
Nodes = {
[
NodeName = "NodeName_0_0";
Executable = "submit.sh";
Arguments = "cmsdataops_JH_gLite_test_110519_113545-Sandbox.tar.bz2 9352";
StdOutput = "9352_0.stdout";
StdError = "9352_0.stderr";
OutputSandboxBaseDestURI = "gsiftp://vocms37.cern.ch/data/wmagent/osb/JobCache/cmsdataops_JH_gLite_test_110519_113545/DataProcessing/JobCollection_26_8/job_9352";
OutputSandbox = {"Report.0.pkl",".BrokerInfo", "9352_0.stdout","9352_0.stderr"};
InputSandbox = {"gsiftp://vocms37.cern.ch/tmp/glite.submit.fakefile.empty","gsiftp://vocms37.cern.ch/data/projects/v0.7.5/install/wmagent/WorkQueueManager/wf/cmsdataops_JH_gLite_test_110519_113545/batch_9228-0/JobPackage.pkl",root.inputsandbox[0],root.inputsandbox[1],root.inputsandbox[2]};
],
[
NodeName = "NodeName_1_0";
Executable = "submit.sh";
Arguments = "cmsdataops_JH_gLite_test_110519_113545-Sandbox.tar.bz2 6133";
StdOutput = "6133_0.stdout";
StdError = "6133_0.stderr";
OutputSandboxBaseDestURI = "gsiftp://vocms37.cern.ch/data/wmagent/osb/JobCache/cmsdataops_JH_gLite_test_110519_113545/DataProcessing/JobCollection_26_5/job_6133";
OutputSandbox = {"Report.0.pkl",".BrokerInfo", "6133_0.stdout","6133_0.stderr"};
InputSandbox = {"gsiftp://vocms37.cern.ch/tmp/glite.submit.fakefile.empty","gsiftp://vocms37.cern.ch/data/projects/v0.7.5/install/wmagent/WorkQueueManager/wf/cmsdataops_JH_gLite_test_110519_113545/batch_6057-0/JobPackage.pkl",root.inputsandbox[0],root.inputsandbox[1],root.inputsandbox[2]};
]
};
InputSandbox = {"gsiftp://vocms37.cern.ch/data/projects/v0.7.5/install/wmagent/WorkQueueManager/wf/cmsdataops_JH_gLite_test_110519_113545/cmsdataops_JH_gLite_test_110519_113545-Sandbox.tar.bz2","gsiftp://vocms37.cern.ch/data/projects/v0.7.5/apps/wmcore/etc/submit.sh","gsiftp://vocms37.cern.ch/data/wmagent/isb/Unpacker.py"};
Requirements = Member("VO-cms-slc5_ia32_gcc434", other.GlueHostApplicationSoftwareRunTimeEnvironment) && (other.GlueHostNetworkAdapterOutboundIP) && other.GlueCEStateStatus == "Production" && other.GlueCEPolicyMaxCPUTime>=130 && ( Member("srm.ciemat.es", other.GlueCESEBindGroupSEUniqueID)) ;
MyProxyServer = "myproxy.cern.ch";
VirtualOrganization = "cms";
RetryCount = 0;
DefaultNodeRetryCount = 0;
ShallowRetryCount = -1;
DefaultNodeShallowRetryCount = -1;
SignificantAttributes = {"Requirements", "Rank", "FuzzyRank"};
]
[ale@cream-12 UI]$ glite-wms-job-submit -a -e https://devel18.cnaf.infn.it:7443/glite_wms_wmproxy_server 82654.jdl
Connecting to the service https://devel18.cnaf.infn.it:7443/glite_wms_wmproxy_server
====================== glite-wms-job-submit Success ======================
The job has been successfully submitted to the WMProxy
Your job identifier is:
https://devel07.cnaf.infn.it:9000/VbYQLwwE3f4LFWEv6-xb0Q
==========================================================================
Then verify the InputSandbox in the submitted jdl:
[ale@cream-12 UI]$ glite-wms-job-info --jdl https://devel07.cnaf.infn.it:9000/VbYQLwwE3f4LFWEv6-xb0Q | grep "InputSandbox ="
InputSandbox = { "gsiftp://vocms37.cern.ch/tmp/glite.submit.fakefile.empty","gsiftp://vocms37.cern.ch/data/projects/v0.7.5/install/wmagent/WorkQueueManager/wf/cmsdataops_JH_gLite_test_110519_113545/batch_9228-0/JobPackage.pkl","gsiftp://vocms37.cern.ch/data/projects/v0.7.5/install/wmagent/WorkQueueManager/wf/cmsdataops_JH_gLite_test_110519_113545/cmsdataops_JH_gLite_test_110519_113545-Sandbox.tar.bz2","gsiftp://vocms37.cern.ch/data/projects/v0.7.5/apps/wmcore/etc/submit.sh","gsiftp://vocms37.cern.ch/data/wmagent/isb/Unpacker.py" };
InputSandbox = { "gsiftp://vocms37.cern.ch/tmp/glite.submit.fakefile.empty","gsiftp://vocms37.cern.ch/data/projects/v0.7.5/install/wmagent/WorkQueueManager/wf/cmsdataops_JH_gLite_test_110519_113545/batch_6057-0/JobPackage.pkl","gsiftp://vocms37.cern.ch/data/projects/v0.7.5/install/wmagent/WorkQueueManager/wf/cmsdataops_JH_gLite_test_110519_113545/cmsdataops_JH_gLite_test_110519_113545-Sandbox.tar.bz2","gsiftp://vocms37.cern.ch/data/projects/v0.7.5/apps/wmcore/etc/submit.sh","gsiftp://vocms37.cern.ch/data/wmagent/isb/Unpacker.py" };
InputSandbox = { "gsiftp://vocms37.cern.ch/data/projects/v0.7.5/install/wmagent/WorkQueueManager/wf/cmsdataops_JH_gLite_test_110519_113545/cmsdataops_JH_gLite_test_110519_113545-Sandbox.tar.bz2","gsiftp://vocms37.cern.ch/data/projects/v0.7.5/apps/wmcore/etc/submit.sh","gsiftp://vocms37.cern.ch/data/wmagent/isb/Unpacker.py","gsiftp://vocms37.cern.ch/tmp/glite.submit.fakefile.empty","gsiftp://vocms37.cern.ch/data/projects/v0.7.5/install/wmagent/WorkQueueManager/wf/cmsdataops_JH_gLite_test_110519_113545/batch_9228-0/JobPackage.pkl" };
Note, if you try in an old WMS (i.e. pre-patch) you should have:
[ale@cream-12 UI]$ glite-wms-job-info --jdl https://devel17.cnaf.infn.it:9000/Hh7KhY4cn49XIZHpgfLvpQ | grep "InputSandbox ="
InputSandbox = { "gsiftp://vocms37.cern.ch/tmp/glite.submit.fakefile.empty","gsiftp://vocms37.cern.ch/data/projects/v0.7.5/install/wmagent/WorkQueueManager/wf/cmsdataops_JH_gLite_test_110519_113545/batch_9228-0/JobPackage.pkl","gsiftp://vocms37.cern.ch/data/projects/v0.7.5/install/wmagent/WorkQueueManager/wf/cmsdataops_JH_gLite_test_110519_113545/cmsdataops_JH_gLite_test_110519_113545-Sandbox.tar.bz2","gsiftp://vocms37.cern.ch/data/projects/v0.7.5/apps/wmcore/etc/submit.sh","gsiftp://vocms37.cern.ch/data/wmagent/isb/Unpacker.py" };
InputSandbox = { "gsiftp://vocms37.cern.ch/tmp/glite.submit.fakefile.empty","gsiftp://vocms37.cern.ch/data/projects/v0.7.5/install/wmagent/workqueuemanager/wf/cmsdataops_jh_glite_test_110519_113545/batch_6057-0/jobpackage.pkl","gsiftp://vocms37.cern.ch/data/projects/v0.7.5/install/wmagent/WorkQueueManager/wf/cmsdataops_JH_gLite_test_110519_113545/cmsdataops_JH_gLite_test_110519_113545-Sandbox.tar.bz2","gsiftp://vocms37.cern.ch/data/projects/v0.7.5/apps/wmcore/etc/submit.sh","gsiftp://vocms37.cern.ch/data/wmagent/isb/Unpacker.py" };
InputSandbox = { "gsiftp://vocms37.cern.ch/data/projects/v0.7.5/install/wmagent/WorkQueueManager/wf/cmsdataops_JH_gLite_test_110519_113545/cmsdataops_JH_gLite_test_110519_113545-Sandbox.tar.bz2","gsiftp://vocms37.cern.ch/data/projects/v0.7.5/apps/wmcore/etc/submit.sh","gsiftp://vocms37.cern.ch/data/wmagent/isb/Unpacker.py","gsiftp://vocms37.cern.ch/tmp/glite.submit.fakefile.empty","gsiftp://vocms37.cern.ch/data/projects/v0.7.5/install/wmagent/WorkQueueManager/wf/cmsdataops_JH_gLite_test_110519_113545/batch_9228-0/JobPackage.pkl" };
As you can see the file gsiftp://vocms37.cern.ch/data/projects/v0.7.5/install/wmagent/workqueuemanager/wf/cmsdataops_jh_glite_test_110519_113545/batch_6057-0/jobpackage.pkl in the second entry is expressed in lower case
-- AlessioGianelle - 2011-06-16 
| I | Attachment | Action | Size | Date |
Who | Comment |
|---|---|---|---|---|---|---|
 txt |
install.txt | manage | 194.1 K | 2011-06-20 - 11:44 | AlessioGianelle | Installation log file |
Topic revision: r11 - 2011-06-28 - AlessioGianelle
Home 
Site map
CEMon web
Create New Topic
Index
Search
Changes
Notifications
RSS Feed
Statistics
Preferences
Account  |
 |
|
|
|
Copyright © 2008-2023 by the contributing authors. All material on this collaboration platform is the property of the contributing authors. Ideas, requests, problems regarding TWiki? Send feedback