WMS test report
Summary
- Component: WMS 3.3.2
- Savannah task: 20812
- ETICS Subsystem Configuration Name: emi-wms_R_3_3_3_0
- VCS Tag: emi-wmproxy_R_3_3_2_0 emi-wms-common_R_3_3_2_0 emi-wms-helper_R_3_3_2_0 emi-yaim-wms_R_4_1_3_0 emi-wms-ice_R_3_3_3_0
- EMI Major Release: EMI 1 (Kebnekaise)
- Platform: sl5 epel
- Build reports: http://etics-repository.cern.ch/repository/reports/id/d3db2292-4aa9-44e5-ac24-ea3380f100fe/sl5_x86_64_gcc412EPEL/-/reports/index.html
- Repository: http://etics-repository.cern.ch/repository/pm/registered/repomd/name/task__20812_2/index.html
- Certification report (TBD)
- Author: Alessio Gianelle
- Outcome: In certification
Clean installation
- cd /etc/yum.repos.d/
- wget --no-check-certificate https://twiki.cern.ch/twiki/pub/EMI/EMI1Updates/testing.repo
- yum clean all
- yum install ca-policy-egi-core
- yum install emi-wms
- /opt/glite/yaim/bin/yaim -c -s /usr/local/nfs1/jra1_test/cream-44/site-info_wms.def -n WMS
- See log file
Update
- Starting from a released version:
[root@devel11 ~]# cat /etc/emi-release EMI release 1.0 (Kebnekaise) [root@devel11 ~]# cat /etc/emi-version 1.0.2-1
- wget --no-check-certificate https://twiki.cern.ch/twiki/pub/EMI/EMI1Updates/testing.repo
- yum update
- /opt/glite/yaim/bin/yaim -c -s site-info.def -n WMS
- See log file
Test Report
List Match
List match without data
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Job ListMatch Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: check if glite-wms-job-list-match command exists Test 2: test a simple matching Test 3: check --delegationid option Test 4: exclude a CE from the match Test 5: try a failure matching (Requirements == false) ===> ===> Test: WMS-listmatch-test.py ===> WMS: devel11.cnaf.infn.it ===> Started: 16:56:28 ===> Ended : 16:56:42 ===> ===> === test PASSED ===Test log file is WMSService-TS_20110701165628.log
List match with data
[ale@cream-12 DataReq]$ glite-wms-job-list-match -a -c ../etc/wmp_devel11.conf data-req.jdl
Connecting to the service https://devel11.cnaf.infn.it:7443/glite_wms_wmproxy_server
==========================================================================
COMPUTING ELEMENT IDs LIST
The following CE(s) matching your job requirements have been found:
*CEId*
- cert-15.pd.infn.it:8443/cream-lsf-cert
- cert-37.pd.infn.it:8443/cream-lsf-cert
- grid010.ct.infn.it:2119/jobmanager-lcgpbs-cert
- prod-ce-01.pd.infn.it:8443/cream-lsf-cert
- prod-ce-02.pd.infn.it:2119/jobmanager-lcglsf-cert
==========================================================================
Submission
Normal job
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) Test 1A: Submit to an LCG-CE Test 1B: Submit to a CREAM CE Test 1C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: cream-44.pd.infn.it ===> Started: 16:53:19 ===> Ended : 17:00:06 ===> ===> === test PASSED ===Test log file is WMSService-TS_20110705165319.log
Collection job
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Set 2: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) Test 2A: Submit to an LCG-CE Test 2B: Submit to a CREAM CE Test 2C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: cream-44.pd.infn.it ===> Started: 17:36:59 ===> Ended : 17:46:57 ===> ===> === test PASSED ===Test log file is WMSService-TS_20110705173659.log
Parametric job
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Set 3: Submit a parametric job (3 cases LCG-CE, CREAM, general) Test 3A: Submit to an LCG-CE Test 3B: Submit to a CREAM CE Test 3C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: cream-44.pd.infn.it ===> Started: 17:49:13 ===> Ended : 17:56:05 ===> ===> === test PASSED ===Test log file is WMSService-TS_20110705174913.log
DAG job
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Set 4: Submit a DAG job ===> ===> Test: WMS-job-cycle.py ===> WMS: devel11.cnaf.infn.it ===> Started: 11:34:25 ===> Ended : 11:40:36 ===> ===> === test PASSED ===
Test log file is WMSService-TS_20110706113425.log
Parallel job
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Set 5: Submit a MPI job (3 cases LCG-CE, CREAM, general) Test 5A: Submit to an LCG-CE Test 5B: Submit to a CREAM CE Test 5C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: devel11.cnaf.infn.it ===> Started: 11:16:52 ===> Ended : 11:21:10 ===> ===> === test PASSED ===Test log file is WMSService-TS_20110706111652.log
Perusal job
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Set 6: Submit a perusal job BEWARE default min perusal interval is 1000 secs, so this phase could take many minutes ===> ===> Test: WMS-job-cycle.py ===> WMS: devel11.cnaf.infn.it ===> Started: 15:36:21 ===> Ended : 16:06:46 ===> ===> === test PASSED ===Test log file is WMSService-TS_20110706153621.log
Cancel
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Job Cancel Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: Try to cancel a normal job Test 2: Try to cancel a DAG job Test 3: Try to cancel one node of a DAG job Test 4: Try to cancel a collection job Test 5: Try to cancel one node of a collection job Test 6: Try to cancel a parametric job Test 7: Try to cancel one node of a parametric job Test 8: Try to cancel a MPI job ===> ===> Test: WMS-job-cancel.py ===> WMS: devel11.cnaf.infn.it ===> Started: 17:24:54 ===> Ended : 17:28:51 ===> ===> === test PASSED ===Test log file is WMSService-TS_20110704172454.log
Others
Delegation
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test delegation operation ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: Check --autm-delegation option Test 2: Check --delegationid option Test 3: Try to delegate with a short proxy and check the validity Test 4: Works with expired proxy ===> ===> Test: WMS-delegation-test.py ===> WMS: devel11.cnaf.infn.it ===> Started: 11:26:43 ===> Ended : 11:27:59 ===> ===> === test PASSED ===Test log file is WMSService-TS_20110704112643.log
Prologue/Epilogue
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS test prologue and epilogue attributes ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: Test prologue attribute Test 2: Test epilogue attribute Test 3: Test prologue and epilogue attributes ===> ===> Test: WMS-prologue-epilogue.py ===> WMS: cream-44.pd.infn.it ===> Started: 18:30:58 ===> Ended : 18:36:53 ===> ===> === test PASSED ===Test log file is WMSService-TS_20110705183058.log
Test proxy renewal
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS test proxy renewal operation ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: Test proxy renewal with LCG CE Test 2: Test proxy renewal with CREAM CE Test 3: Test proxy renewal with LCG CE (without setting MYPROXYSERVER) Test 4: Test proxy renewal with CREAM CE (without setting MYPROXYSERVER) ===> ===> Test: WMS-proxy-renewal.py ===> WMS: devel11.cnaf.infn.it ===> Started: 17:56:06 ===> Ended : 18:48:50 ===> ===> === test PASSED ===Test log file is WMSService-TS_20110701175606.log
Shallow/Deep Resubmission
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Job Resubmission Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: Try a shallow resubmission Test 2: Try a deep resubmission ===> ===> Test: WMS-job-resubmit.py ===> WMS: cream-44.pd.infn.it ===> Started: 15:17:25 ===> Ended : 15:29:02 ===> ===> === test PASSED ==Test log file is WMSService-TS_20110705151725.log
Published information
bdb database definitions
- ldapsearch -x -H ldap://cream-44.pd.infn.it:2170 -b 'o=grid'
# extended LDIF # # LDAPv3 # base <o=grid> with scope subtree # filter: (objectclass=*) # requesting: ALL # # grid dn: o=grid objectClass: organization o: grid # local, grid dn: Mds-Vo-name=local,o=grid Mds-Vo-name: local objectClass: Mds # resource, grid dn: Mds-Vo-name=resource,o=grid Mds-Vo-name: resource objectClass: Mds # cream-44.pd.infn.it_org.glite.wms.WMProxy_2846265655, resource, grid dn: GlueServiceUniqueID=cream-44.pd.infn.it_org.glite.wms.WMProxy_2846265655,M ds-Vo-name=resource,o=grid GlueServiceAccessControlBaseRule: VO:alice GlueServiceAccessControlBaseRule: VO:atlas GlueServiceAccessControlBaseRule: VO:cms GlueServiceAccessControlBaseRule: VO:lhcb GlueServiceAccessControlBaseRule: VO:dteam GlueServiceAccessControlBaseRule: VO:biomed GlueServiceAccessControlBaseRule: VO:ops GlueServiceAccessControlBaseRule: VO:testers.eu-emi.eu GlueServiceStatus: Other GlueServiceStatusInfo: ?? objectClass: GlueTop objectClass: GlueService objectClass: GlueKey objectClass: GlueSchemaVersion GlueServiceUniqueID: cream-44.pd.infn.it_org.glite.wms.WMProxy_2846265655 GlueServiceAccessControlRule: alice GlueServiceAccessControlRule: atlas GlueServiceAccessControlRule: cms GlueServiceAccessControlRule: lhcb GlueServiceAccessControlRule: dteam GlueServiceAccessControlRule: biomed GlueServiceAccessControlRule: ops GlueServiceAccessControlRule: testers.eu-emi.eu GlueServiceStartTime: 1970-01-01T01:00:00+01:00 GlueServiceEndpoint: https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server GlueServiceVersion: 3.3.2 GlueSchemaVersionMinor: 3 GlueServiceName: JRA1-CREAM-TEST-WMProxy GlueServiceType: org.glite.wms.WMProxy GlueServiceWSDL: http://trinity.datamat.it/projects/EGEE/WMProxy/WMProxy.wsdl GlueServiceSemantics: https://edms.cern.ch/file/674643/1/EGEE-JRA1-TEC-674643- WMPROXY-guide-v0-3.pdf GlueForeignKey: GlueSiteUniqueID=JRA1-CREAM-TEST GlueServiceOwner: alice GlueServiceOwner: atlas GlueServiceOwner: cms GlueServiceOwner: lhcb GlueServiceOwner: dteam GlueServiceOwner: biomed GlueServiceOwner: ops GlueServiceOwner: testers.eu-emi.eu GlueSchemaVersionMajor: 1 # DN, cream-44.pd.infn.it_org.glite.wms.WMProxy_2846265655, resource, grid dn: GlueServiceDataKey=DN,GlueServiceUniqueID=cream-44.pd.infn.it_org.glite.wm s.WMProxy_2846265655,Mds-Vo-name=resource,o=grid GlueSchemaVersionMajor: 1 objectClass: GlueTop objectClass: GlueServiceData objectClass: GlueKey objectClass: GlueSchemaVersion GlueSchemaVersionMinor: 3 GlueServiceDataKey: DN GlueChunkKey: GlueServiceUniqueID=cream-44.pd.infn.it_org.glite.wms.WMProxy_28 46265655 GlueServiceDataValue: /C=IT/O=INFN/OU=Host/L=Padova/CN=cream-44.pd.infn.it # glite-info-service_version, cream-44.pd.infn.it_org.glite.wms.WMProxy_28462 65655, resource, grid dn: GlueServiceDataKey=glite-info-service_version,GlueServiceUniqueID=cream-44 .pd.infn.it_org.glite.wms.WMProxy_2846265655,Mds-Vo-name=resource,o=grid GlueSchemaVersionMajor: 1 objectClass: GlueTop objectClass: GlueServiceData objectClass: GlueKey objectClass: GlueSchemaVersion GlueSchemaVersionMinor: 3 GlueServiceDataKey: glite-info-service_version GlueChunkKey: GlueServiceUniqueID=cream-44.pd.infn.it_org.glite.wms.WMProxy_28 46265655 GlueServiceDataValue: 1.6 # glite-info-service_hostname, cream-44.pd.infn.it_org.glite.wms.WMProxy_2846 265655, resource, grid dn: GlueServiceDataKey=glite-info-service_hostname,GlueServiceUniqueID=cream-4 4.pd.infn.it_org.glite.wms.WMProxy_2846265655,Mds-Vo-name=resource,o=grid GlueSchemaVersionMajor: 1 objectClass: GlueTop objectClass: GlueServiceData objectClass: GlueKey objectClass: GlueSchemaVersion GlueSchemaVersionMinor: 3 GlueServiceDataKey: glite-info-service_hostname GlueChunkKey: GlueServiceUniqueID=cream-44.pd.infn.it_org.glite.wms.WMProxy_28 46265655 GlueServiceDataValue: cream-44.pd.infn.it # search result search: 2 result: 0 Success # numResponses: 8 # numEntries: 7
Stats database definitions
- ldapsearch -x -H ldap://cream-44.pd.infn.it:2170 -b 'o=infosys'
# extended LDIF # # LDAPv3 # base <o=infosys> with scope subtree # filter: (objectclass=*) # requesting: ALL # # infosys dn: o=infosys objectClass: organization o: infosys # zip, infosys dn: CompressionType=zip,o=infosys objectClass: CompressedContent Hostname: localhost CompressionType: zip Data: file:///var/run/bdii/archive/11-07-06-11-54-55-snapshot.gz # localhost, infosys dn: Hostname=localhost,o=infosys objectClass: UpdateStats Hostname: localhost FailedDeletes: 0 ModifiedEntries: 5 DeletedEntries: 0 UpdateTime: 0 FailedAdds: 0 FailedModifies: 0 TotalEntries: 14 QueryTime: 0 NewEntries: 1 DBUpdateTime: 0 ReadTime: 0 PluginsTime: 0 ProvidersTime: 0 # search result search: 2 result: 0 Success # numResponses: 4 # numEntries: 3
GLUE 2.0 database definitions
- ldapsearch -x -H ldap://cream-44.pd.infn.it:2170 -b 'o=glue'
# extended LDIF # # LDAPv3 # base <o=glue> with scope subtree # filter: (objectclass=*) # requesting: ALL # # glue dn: o=glue objectClass: organization o: glue # grid, glue dn: GLUE2GroupID=grid,o=glue objectClass: GLUE2Group GLUE2GroupID: grid # resource, glue dn: GLUE2GroupID=resource,o=glue objectClass: GLUE2Group GLUE2GroupID: resource # cream-44.pd.infn.it_wmproxy_2846265655, resource, glue dn: GLUE2ServiceID=cream-44.pd.infn.it_wmproxy_2846265655,GLUE2GroupID=resourc e,o=glue GLUE2EntityValidity: 3600 GLUE2ServiceType: org.glite.wms.WMProxy GLUE2ServiceID: cream-44.pd.infn.it_wmproxy_2846265655 objectClass: GLUE2Service GLUE2ServiceQualityLevel: production GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-service GLUE2EntityOtherInfo: InfoProviderVersion=1.1 GLUE2EntityOtherInfo: InfoProviderHost=cream-44.pd.infn.it GLUE2ServiceComplexity: endpointType=1, share=0, resource=0 GLUE2EntityName: JRA1-CREAM-TEST-WMProxy GLUE2ServiceAdminDomainForeignKey: JRA1-CREAM-TEST GLUE2EntityCreationTime: 2011-07-06T09:56:56Z # cream-44.pd.infn.it_wmproxy_2846265655_org.glite.wms.WMProxy_2846265655, cr eam-44.pd.infn.it_wmproxy_2846265655, resource, glue dn: GLUE2EndpointID=cream-44.pd.infn.it_wmproxy_2846265655_org.glite.wms.WMPro xy_2846265655,GLUE2ServiceID=cream-44.pd.infn.it_wmproxy_2846265655,GLUE2Grou pID=resource,o=glue GLUE2EndpointIssuerCA: /C=IT/O=INFN/CN=INFN CA GLUE2EntityValidity: 3600 GLUE2EndpointQualityLevel: production GLUE2EndpointInterfaceName: org.glite.wms.WMProxy GLUE2EndpointTrustedCA: IGTF objectClass: GLUE2Endpoint GLUE2EndpointID: cream-44.pd.infn.it_wmproxy_2846265655_org.glite.wms.WMProxy_ 2846265655 GLUE2EndpointInterfaceVersion: 3.3.2 GLUE2EndpointURL: https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-endpoint GLUE2EntityOtherInfo: InfoProviderVersion=1.1 GLUE2EntityOtherInfo: InfoProviderHost=cream-44.pd.infn.it GLUE2EntityOtherInfo: MiddlewareName=EMI GLUE2EntityOtherInfo: MiddlewareVersion=1.0.2-1 GLUE2EndpointDowntimeInfo: See the GOC DB for downtimes: https://goc.egi.eu/ GLUE2EndpointWSDL: http://trinity.datamat.it/projects/EGEE/WMProxy/WMProxy.wsd l GLUE2EndpointServingState: production GLUE2EndpointHealthState: other GLUE2EndpointSemantics: https://edms.cern.ch/file/674643/1/EGEE-JRA1-TEC-67464 3-WMPROXY-guide-v0-3.pdf GLUE2EndpointServiceForeignKey: cream-44.pd.infn.it_wmproxy_2846265655 GLUE2EntityName: WMProxy endpoint for Service cream-44.pd.infn.it_wmproxy_2846 265655 GLUE2EndpointTechnology: webservice GLUE2EntityCreationTime: 2011-07-06T09:56:56Z # DN_1, cream-44.pd.infn.it_wmproxy_2846265655_org.glite.wms.WMProxy_28462656 55, cream-44.pd.infn.it_wmproxy_2846265655, resource, glue dn: GLUE2ExtensionLocalID=DN_1,GLUE2EndpointID=cream-44.pd.infn.it_wmproxy_284 6265655_org.glite.wms.WMProxy_2846265655,GLUE2ServiceID=cream-44.pd.infn.it_w mproxy_2846265655,GLUE2GroupID=resource,o=glue GLUE2ExtensionLocalID: DN_1 GLUE2ExtensionKey: DN objectClass: GLUE2Extension GLUE2ExtensionValue: /C=IT/O=INFN/OU=Host/L=Padova/CN=cream-44.pd.infn.it GLUE2ExtensionEntityForeignKey: cream-44.pd.infn.it_wmproxy_2846265655_org.gli te.wms.WMProxy_2846265655 # cream-44.pd.infn.it_wmproxy_2846265655_org.glite.wms.WMProxy_2846265655_Pol icy, cream-44.pd.infn.it_wmproxy_2846265655_org.glite.wms.WMProxy_2846265655 , cream-44.pd.infn.it_wmproxy_2846265655, resource, glue dn: GLUE2PolicyID=cream-44.pd.infn.it_wmproxy_2846265655_org.glite.wms.WMProxy _2846265655_Policy,GLUE2EndpointID=cream-44.pd.infn.it_wmproxy_2846265655_org .glite.wms.WMProxy_2846265655,GLUE2ServiceID=cream-44.pd.infn.it_wmproxy_2846 265655,GLUE2GroupID=resource,o=glue GLUE2AccessPolicyEndpointForeignKey: cream-44.pd.infn.it_wmproxy_2846265655_or g.glite.wms.WMProxy_2846265655 GLUE2EntityValidity: 3600 objectClass: GLUE2Policy objectClass: GLUE2AccessPolicy GLUE2PolicyRule: VO:alice GLUE2PolicyRule: VO:atlas GLUE2PolicyRule: VO:cms GLUE2PolicyRule: VO:lhcb GLUE2PolicyRule: VO:dteam GLUE2PolicyRule: VO:biomed GLUE2PolicyRule: VO:ops GLUE2PolicyRule: VO:testers.eu-emi.eu GLUE2PolicyScheme: org.glite.standard GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-endpoint GLUE2EntityOtherInfo: InfoProviderVersion=1.1 GLUE2EntityOtherInfo: InfoProviderHost=cream-44.pd.infn.it GLUE2PolicyUserDomainForeignKey: alice GLUE2PolicyUserDomainForeignKey: atlas GLUE2PolicyUserDomainForeignKey: cms GLUE2PolicyUserDomainForeignKey: lhcb GLUE2PolicyUserDomainForeignKey: dteam GLUE2PolicyUserDomainForeignKey: biomed GLUE2PolicyUserDomainForeignKey: ops GLUE2PolicyUserDomainForeignKey: testers.eu-emi.eu GLUE2PolicyID: cream-44.pd.infn.it_wmproxy_2846265655_org.glite.wms.WMProxy_28 46265655_Policy GLUE2EntityName: Access control rules for Endpoint cream-44.pd.infn.it_wmproxy _2846265655_org.glite.wms.WMProxy_2846265655 GLUE2EntityCreationTime: 2011-07-06T09:56:56Z # search result search: 2 result: 0 Success # numResponses: 8 # numEntries: 7
Performance test
Collection of 1000 nodes
Case A: production environment
- Collection ID: https://devel11.cnaf.infn.it:9000/Wty8qDpctah3lazxG6iViA
- Nodes are submitted to a production environment (397 CEs):
- 436 jobs are executed on a LCG CE
- 564 jobs are executed in a CREAM CE
- Final Status:
- 971: Done (Success)
- 29: Aborted
Case B: testing environment
- Collection ID: https://cream-44.pd.infn.it:9000/BJI8Q7RSNC050nCDqYHong
- Nodes are submitted to a testing environment (33 CEs):
- 147 jobs are executed on the single LCG CE
- 853 jobs are executed in a CREAM CE
- Final Status:
- 995: Done (Success)
- 5: Aborted
Stress test
Description
- 1440 collection of 25 nodes each (total: 36000 jobs)
- One collection every 60 seconds
- two users
- Use LCG-CEs and CREAM-CEs (with different batch systems)
- Use automatic-delegation
- The job is a "sleep random(666)"
- Resubmission is enabled
- Enable proxy renewal
Results
- 1321 collections correctly submitted in 29056 seconds: 1/21/157 (min/avg/max)
- Jobs correctly submitted: 33025 (48% to CREAM CEs and 52% to LCG CEs)
- DONE OK: 31500 (95.4%)
- Aborted: 1525 (4.6%)
)
Check Bugs (Regression tests)
Bug #69841: cancel requests can bounce between JC and LM FIXED not certified
This bug cannot be easily replicate.
Bug #82687: emi WMS problems with ISB tar file handling FIXED
Submit a jdl like this one:
[ale@cream-12 UI]$ cat 82687.jdl
[
Executable = "/bin/ls" ;
Arguments = "-lha " ;
Stdoutput = "ls.out" ;
InputSandbox = { "supercalifragilistichespiralidose.txt", "82687.jdl"} ;
OutputSandbox = { ".BrokerInfo", "ls.out"} ;
Retrycount = 2;
ShallowRetryCount = -1;
usertags = [ bug = "#82687" ];
VirtualOrganisation="dteam";
]
With a long file name as input file (and at least two files as input):
[ale@cream-12 UI]$ glite-wms-job-submit -a -c etc/wmp_cream-44.conf 82687.jdl Connecting to the service https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server ====================== glite-wms-job-submit Success ====================== The job has been successfully submitted to the WMProxy Your job identifier is: https://cream-44.pd.infn.it:9000/50rMqKVPm3Qe1cYH1L3Wmw ==========================================================================When the job arrives to Done (Success):
[ale@cream-12 UI]$ glite-wms-job-status https://cream-44.pd.infn.it:9000/50rMqKVPm3Qe1cYH1L3Wmw
======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:
Status info for the Job : https://cream-44.pd.infn.it:9000/50rMqKVPm3Qe1cYH1L3Wmw
Current Status: Done (Success)
Logged Reason(s):
- job completed
- Job Terminated Successfully
Exit code: 0
Status Reason: Job Terminated Successfully
Destination: cream-19.pd.infn.it:8443/cream-lsf-creamcert2
Submitted: Fri Jul 1 14:01:41 2011 CEST
==========================================================================
Check in the output file of the job if all the input files are listed:
[ale@cream-12 UI]$ glite-wms-job-output https://cream-44.pd.infn.it:9000/50rMqKVPm3Qe1cYH1L3Wmw
Connecting to the service https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server
================================================================================
JOB GET OUTPUT OUTCOME
Output sandbox files for the job:
https://cream-44.pd.infn.it:9000/50rMqKVPm3Qe1cYH1L3Wmw
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/ale_50rMqKVPm3Qe1cYH1L3Wmw
================================================================================
[ale@cream-12 UI]$ cat /tmp/jobOutput/ale_50rMqKVPm3Qe1cYH1L3Wmw/ls.out
total 20K
drwxr-xr-x 2 dteam017 dteam 4.0K Jul 1 14:01 .
drwxr-xr-x 3 dteam017 dteam 4.0K Jul 1 14:01 ..
-rw-r--r-- 1 dteam017 dteam 293 Jul 1 14:01 82687.jdl
-rw-r--r-- 1 dteam017 dteam 2.3K Jul 1 14:01 .BrokerInfo
-rw-r--r-- 1 dteam017 dteam 0 Jul 1 14:01 ls.out
-rw-r--r-- 1 dteam017 dteam 17 Jul 1 14:01 supercalifragilistichespiralidose.txt
-rw-r--r-- 1 dteam017 dteam 0 Jul 1 14:01 .tmp_file
Bug #82776: typo in WMS jobwrapper FIXED
Simply check the JobWrapper template on the WMS:
[root@cream-44 ~]# cat /usr/share/glite-wms/jobwrapper.template.sh | grep "Taken token ${__shallow_resubmission_token}"
push_in_LM_done_reason "Taken token ${__shallow_resubmission_token}"
Bug #82983: [yaim-wms] authorization problem in WMS EMI-1 FIXED
Simply check that in the file glite_wms_wmproxy.gacl file there are not entry with the word ROLE in upper case:
[root@devel11 ~]# grep ROLE /etc/glite-wms/glite_wms_wmproxy.gacl [root@devel11 ~]#
Bug #84155: Proxy renewal fails in ICE FIXED
Take a long jdl where MyProxyServer attribute is set and that require a cream CE as destination; like this one:
[ale@cream-12 UI]$ cat test.jdl
[
Executable = "/bin/sleep" ;
Stdoutput = "env.out" ;
StdError = "env.err" ;
InputSandbox = { "/home/ale/UI/data/pippo" };
OutputSandbox = { "env.out" ,"env.err"} ;
Arguments="900";
Environment={"VARIABLE=test"};
usertags = [ exe = "env" ];
FuzzyRank = true;
ShallowRetryCount = 3;
RetryCount = 2;
Myproxyserver="myproxy.cnaf.infn.it";
requirements = RegExp("/cream-",other.GlueCEUniqueID);
]
Submit it with a shorter proxy:[ale@cream-12 UI]$ voms-proxy-init -voms dteam --valid 00:15 Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle Creating temporary proxy .............................................. Done Contacting voms.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr] "dteam" Done Creating proxy ......................................................................................................................... Done Your proxy is valid until Tue Jul 12 15:30:22 2011 [ale@cream-12 UI]$ glite-wms-job-submit -a -c etc/wmp_devel11.conf test.jdl Connecting to the service https://devel11.cnaf.infn.it:7443/glite_wms_wmproxy_server ====================== glite-wms-job-submit Success ====================== The job has been successfully submitted to the WMProxy Your job identifier is: https://devel11.cnaf.infn.it:9000/tfn-DjwWIGZOg9zUvXs5CQ ==========================================================================Check that its proxy is valid for no more that 15 minutes:
[ale@cream-12 UI]$ glite-wms-job-info -p https://devel11.cnaf.infn.it:9000/tfn-DjwWIGZOg9zUvXs5CQ Connecting to the service https://devel11.cnaf.infn.it:7443/glite_wms_wmproxy_server ======================= glite-wms-job-info Success ======================= Your proxy delegated to the endpoint https://devel11.cnaf.infn.it:7443/glite_wms_wmproxy_server for the job https://devel11.cnaf.infn.it:9000/tfn-DjwWIGZOg9zUvXs5CQ : Subject : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy Issuer : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy Identity : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy Type : proxy Strength : 512 StartDate : 12 July 2011 - 15:13:45 Expiration : 12 July 2011 - 15:29:45 Timeleft : 07 min 17 sec === VO dteam extension information === VO : dteam Subject : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle Issuer : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr URI : voms.hellasgrid.gr:15004 Attribute : /dteam/Role=NULL/Capability=NULL Attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL StartTime : 12 July 2011 - 15:15:22 Expiration : 12 July 2011 - 15:30:22 Timeleft : 07 min 54 secWait until it finishes successfully:
[ale@cream-12 UI]$ glite-wms-job-status https://devel11.cnaf.infn.it:9000/tfn-DjwWIGZOg9zUvXs5CQ
======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:
Status info for the Job : https://devel11.cnaf.infn.it:9000/tfn-DjwWIGZOg9zUvXs5CQ
Current Status: Done (Success)
Logged Reason(s):
- job completed
- Job Terminated Successfully
Exit code: 0
Status Reason: Job Terminated Successfully
Destination: grid01.erciyes.edu.tr:8443/cream-pbs-dteam
Submitted: Tue Jul 12 15:18:46 2011 CEST
==========================================================================
We can also check directly the command on the WMS. First of all find the proxy associated with the job in the persist directory of ice:
[root@devel11 persist_dir]# openssl x509 -noout -text -in /var/ice/persist_dir/2A9DAF04C398C21D6ADF7E884BC192ED95AF554C.betterproxy
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 1310476725 (0x4e1c49b5)
Signature Algorithm: md5WithRSAEncryption
Issuer: C=IT, O=INFN, OU=Personal Certificate, L=Padova, CN=Alessio Gianelle, CN=proxy
Validity
Not Before: Jul 12 13:13:45 2011 GMT
Not After : Jul 12 13:29:45 2011 GMT
Subject: C=IT, O=INFN, OU=Personal Certificate, L=Padova, CN=Alessio Gianelle, CN=proxy, CN=proxy
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public Key: (512 bit)
Modulus (512 bit):
00:b7:78:00:10:30:78:4e:82:49:87:25:9d:65:51:
ff:4a:4e:5b:37:9e:78:05:84:eb:74:af:ae:11:bb:
60:00:f6:17:80:e7:e6:5d:1d:09:61:fd:71:59:64:
bc:10:34:65:54:f8:69:12:c2:61:78:be:29:09:2d:
17:ae:9c:7d:0f
Exponent: 65537 (0x10001)
Signature Algorithm: md5WithRSAEncryption
a9:44:e7:b7:a5:9d:be:e3:65:cf:68:fa:ce:8d:b7:b2:44:30:
31:a1:5a:f6:ee:fa:dc:1c:08:7b:be:6e:af:a7:f9:7f:d9:7f:
6f:3f:44:40:a7:2b:68:86:b7:c6:bd:35:49:49:4a:3a:db:a7:
26:4f:d4:93:b6:10:0d:55:58:e3:82:d4:92:44:b9:b0:ac:9d:
5c:6a:a2:45:c0:61:5f:e4:c1:5a:af:9e:61:60:fb:62:14:9f:
b6:c2:d1:ed:2c:6c:c8:9c:1b:3b:e6:c1:d9:8e:a0:4c:31:64:
9f:52:fe:ad:81:04:4b:e5:6b:d1:95:ae:f9:28:5f:53:73:57:
5e:97
Then logged as user glite and execute this command: export X509_USER_CERT=/var/wms.proxy; export X509_USER_KEY=/var/wms.proxy; /usr/bin/glite-wms-ice-proxy-renew -s myproxy.cnaf.infn.it -p /var/ice/persist_dir/<user proxy>.betterproxy -o <output file> where /var/ice/persist_dir/<user proxy>.betterproxy is the proxy associate to the job and in <output file> you should find the renewed proxy: [root@devel11 persist_dir]# su - glite [glite@devel11 ~]$ export X509_USER_CERT=/var/wms.proxy; export X509_USER_KEY=/var/wms.proxy; /usr/bin/glite-wms-ice-proxy-renew -s myproxy.cnaf.infn.it -p /var/ice/persist_dir/2A9DAF04C398C21D6ADF7E884BC192ED95AF554C.betterproxy -o /var/ice/persist_dir/2A9DAF04C398C21D6ADF7E884BC192ED95AF554C.betterproxy.renewed /var/ice/persist_dir/2A9DAF04C398C21D6ADF7E884BC192ED95AF554C.betterproxy.voms.xX3leH [glite@devel11 ~]$ ls -l /var/ice/persist_dir/ total 68 -rw-r--r-- 1 glite glite 9767 Jul 12 15:28 2A9DAF04C398C21D6ADF7E884BC192ED95AF554C.betterproxy -rw------- 1 glite glite 9767 Jul 12 15:29 2A9DAF04C398C21D6ADF7E884BC192ED95AF554C.betterproxy.renewed -rw-r--r-- 1 glite glite 44032 Jul 12 15:28 ice.dbCheck the validity of the new proxy:
[root@devel11 persist_dir]# openssl x509 -noout -text -in /var/ice/persist_dir/2A9DAF04C398C21D6ADF7E884BC192ED95AF554C.betterproxy.renewed
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 19275 (0x4b4b)
Signature Algorithm: md5WithRSAEncryption
Issuer: C=IT, O=INFN, OU=Personal Certificate, L=Padova, CN=Alessio Gianelle, CN=proxy, CN=proxy, CN=proxy
Validity
Not Before: Jul 12 13:24:29 2011 GMT
Not After : Jul 12 23:29:36 2011 GMT
Subject: C=IT, O=INFN, OU=Personal Certificate, L=Padova, CN=Alessio Gianelle, CN=proxy, CN=proxy, CN=proxy, CN=proxy
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public Key: (512 bit)
Modulus (512 bit):
00:ca:23:d8:f0:c3:c1:61:33:4b:05:62:42:2d:72:
22:94:9f:d7:da:a5:38:27:45:b8:49:cb:b9:89:dc:
a0:ad:55:18:1e:1e:fe:cb:ce:89:73:18:65:8e:38:
ad:cf:ad:f4:b4:e9:cb:d1:82:2d:1f:e9:b8:bf:40:
d9:43:bb:aa:59
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Extended Key Usage:
TLS Web Client Authentication, E-mail Protection
X509v3 Key Usage: critical
Digital Signature, Key Encipherment, Data Encipherment
1.3.6.1.4.1.8005.100.100.5:
0...U. 0..F0..B0..>0..&...0s.q0k.i0g1.0...U....IT1
..U....Padova1.0...U....Alessio Gianelle..KK.]0[.Y0W1.0...U....GR1.0...U.
.
.......+...}.Fq..=].....voms.hellasgrid.gr0
0"..20110712132928Z..20110713012928Z0..0...
+.....Edd.1s0q.". dteam://voms.hellasgrid.gr:150040K. /dteam/Role=NULL/Capability=NULL.'/dteam/NGI_IT/Role=NULL/Capability=NULL0...0..
+.....Edd...0.0.0...U.8....0...U.#..0....y...^.....f.b....V.0..;.
+.....Edd
.....0c1.0...U....GR1.0...U.q0
.
111008093327Z0W1.0...U....GR1.0...U.uthorities1.0...U....HellasGrid CA 20060..
.
..........0..1.0...U....voms.hellasgrid.gr0.."0
.......l{!.&?.}7...{...N.....c.....f....|........_.Q..7%L4......e.nn.....].>./.i'.A>.pb..s.rw#M...ac.6..R..7=3.l..'N.1...n..).../C.|g.21...l.=J
0.R..=.N=..e.~....a'[..eN..(F7..N..T...h.....*..Ph..$...m...jPl..uo.OD*..](.2+#.%.?...!Z..s/.=.8.......9a]8yj..l..............0...0...U.......0.0...U...........0...U.......y...^.....f.b....V.0....U.#...0.......]..:.l.C..=O<.f^&.l.j0h1.0...U....GR1.0...U.
.
HellasGrid1"0 ..U....Certification Authorities1 0...U....HellasGrid Root CA 2006...0%..U....0...hellasgrid-ca@grid.auth.gr0...U....0...voms.hellasgrid.gr0...+......E.....0c0a..+........Uhttp://www.grid.auth.gr/pki/hellasgrid-ca-2006/documents/cps/HellasGrid-CA-CP-CPS.pdf0H..U...A0?0=.;.9.7http://crl.grid.auth.gr/...........V....~..!Z..5.........Y...4...[.m.>&......*..........R..i5.7.u..~>;M.v....T.(..jk
......t...uJ....+..{CD...ec.q...-..L....BS..cAcP.)/[......^.i..V.};T.w#.GF(~.kcL..HP..h.....Y...J\..........#60...n.....Kx.{!Vx......?J.$..F.`)5yLl..).6...,.T&EM.5...y..-
...........M,.R...%..l.].$v.,..d.B....5.=m...
D".f..?..Qz.Rk.k..ZS..f...(....)8.....0f.........&..J....nV.C.I.bj..>Mz>..a..../N+.....0....w...6.U4......ap8.. ...u..kQh.._.sUYa.....?..1.A.E.m...3.A.....i.../....}..`.A.....h...U.0....*.r]....eocH....54
Signature Algorithm: md5WithRSAEncryption
34:d3:75:cd:a0:f2:3e:64:60:00:05:88:8d:4b:a2:e6:75:33:
ca:37:cc:1a:93:8a:48:73:72:ed:6e:df:e3:40:f9:27:2f:ce:
16:f7:43:83:df:cb:16:54:6f:d4:2a:d9:af:a4:30:4d:99:23:
94:60:95:3d:ed:17:27:d2:cc:de:42:d1:b4:df:51:e7:63:10:
e5:52:5f:f0:b8:2b:be:bd:3c:e2:98:b4:2a:ec:5a:70:5e:05:
03:69:aa:d7:48:ea:d2:4f:09:70:99:df:fe:98:04:74:af:e7:
31:67:33:4a:7e:27:8c:8d:88:e9:15:02:3b:0b:b9:e1:98:e2:
7f:ed:8b:31:27:80:f3:27:78:53:80:da:5c:4b:86:68:84:63:
c6:f5:f8:40:61:b4:f8:95:dd:19:f9:83:21:fc:18:bb:82:87:
b0:ba:5b:70:5c:92:56:ce:bd:f7:f6:04:08:bc:76:5b:d7:0e:
d2:e5:cb:fa:f2:f1:41:6d:6e:c7:21:3f:4f:b0:6d:56:8a:07:
5e:e1:73:0d:e0:23:54:e7:ef:0b:b8:d7:86:7f:4b:43:07:98:
3b:db:db:7c:30:6c:06:15:d5:c3:bc:03:17:ee:71:d9:df:9d:
de:47:47:35:09:5b:89:4c:68:3c:9d:fa:1c:9c:a8:bb:55:b8:
39:90:8d:74
-- AlessioGianelle - 2011-07-01 Topic revision: r18 - 2011-07-12 - AlessioGianelle
Home 
Site map
CEMon web
Create New Topic
Index
Search
Changes
Notifications
RSS Feed
Statistics
Preferences
Account  |
 |
|
|
|
|
Copyright © 2008-2023 by the contributing authors. All material on this collaboration platform is the property of the contributing authors. Ideas, requests, problems regarding TWiki? Send feedback