Tags:
,
view all tags
%TOC% ---+ *WMS test report* ---++ Summary * Component: *WMS 3.3.4* * Savannah task: [[https://savannah.cern.ch/task/index.php?22847][22847]] * ETICS Subsystem Configuration Name: *emi-wms_R_3_3_6_3* *emi-jdl_R_3_2_6_1* * VCS Tag: _emi-ice_R_3_3_4_1_, _emi-wms-manager_R_3_3_1_4_, _emi-jobsubmission_R_3_3_2_1_, _emi-wmproxy_R_3_3_4_2_, _emi-wms-brokerinfo_R_3_3_2_1_, _emi-wms-configuration_R_3_3_2_1_, _emi-wms-ism_R_3_3_2_1_, _emi-wms-matchmaking_R_3_3_2_1_, _emi-wms-purger_R_3_3_2_1_, _emi-yaim-wms_R_4_1_4_3_, _emi-jdl-api-cpp_R_3_2_6_1_, _emi-wms-nagios_R_1_0_0_0_, _emi-wms-mp_R_1_0_2_0_ * EMI Major Release: *EMI 1 (Kebnekaise)* * Platform: sl5 epel * Certification report: ... * Outcome:<strong> %ORANGE%In certification%ENDCOLOR% </strong> ---++ Deployment Tests ---+++ Clean installation * Starting from a SL5 x86_64 kernel machine: <verbatim> [root@cream-45 ~]# cat /etc/redhat-release Scientific Linux SL release 5.7 (Boron) [root@cream-45 ~]# uname -a Linux cream-45.pd.infn.it 2.6.18-238.5.1.el5xen #1 SMP Tue Mar 1 19:22:01 EST 2011 x86_64 x86_64 x86_64 GNU/Linux </verbatim> * cd /etc/yum.repos.d/ * wget --no-check-certificate https://twiki.cern.ch/twiki/pub/EMI/EMI1Updates/testing.repo * yum clean all * yum install ca-policy-egi-core * yum install emi-wms * See [[%ATTACHURL%/install.txt][log]] file * /opt/glite/yaim/bin/yaim -c -s /usr/local/nfs/jra1_test/cream-45/site-info_wms.def -n WMS * See [[%ATTACHURL%/configure.txt][log]] file ---+++ Update * Starting from a production version: <verbatim>[root@cream-45 yum.repos.d]# etc/emi-version 1.6.0-1 </verbatim> * wget --no-check-certificate https://twiki.cern.ch/twiki/pub/EMI/EMI1Updates/testing.repo -O /etc/yum.repos.d/testing.repo * yum clean all * service gLite stop * yum update * /opt/glite/yaim/bin/yaim -c -s site-info.def -n WMS * See [[%ATTACHURL%/update.txt][log]] file * Check final version: <verbatim>[root@cream-45 ~]# cat /etc/emi-version 1.6.0-1</verbatim> ---++ Unit Tests *N/A*: Tests will be available with EMI-2. ---++ Basic Functionality tests ---+++ Submission ---++++ Normal <verbatim> [ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 1 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) Test 1A: Submit to an LCG-CE Test 1B: Submit to a CREAM CE Test 1C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: cream-44.pd.infn.it ===> Started: 09:42:34 ===> Ended : 09:48:32 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111117094234.log [ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 1 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) Test 1A: Submit to an LCG-CE Test 1B: Submit to a CREAM CE Test 1C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: cream-44.pd.infn.it ===> Started: 09:59:32 ===> Ended : 10:06:54 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111117095932.log [ale@cream-12 WMS-service]$ voms-proxy-info -all subject : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy issuer : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy identity : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy type : proxy strength : 2048 bits path : /tmp/x509up_u500 timeleft : 10:59:48 key usage : Digital Signature, Key Encipherment, Data Encipherment === VO dteam extension information === VO : dteam subject : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle issuer : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr attribute : /dteam/Role=NULL/Capability=NULL attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL timeleft : 11:58:48 uri : voms2.hellasgrid.gr:15004 [ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 1 ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) Test 1A: Submit to an LCG-CE Test 1B: Submit to a CREAM CE Test 1C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: cream-44.pd.infn.it ===> Started: 11:56:43 ===> Ended : 12:03:02 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111117115643.log </verbatim> See [[%ATTACHURL%/submitnormal.tgz][gzipped log]] file ---+++++ Jdl with _DataRequirements_ <verbatim> [ale@cream-12 WMS-service]$ cat JDLs/data-req.jdl ########################################### # JDL with Data Requirements # ########################################### Executable = "/bin/ls"; Arguments = "-la"; StdOutput = "std.out"; StdError = "std.err"; Prologue = "prologue.sh"; FuzzyRank = true; Environment={"LFC_HOST=prod-lfc-shared-central.cern.ch", "LFC_HOME=prod-lfc-shared-central.cern.ch:/grid/dteam/aleph"}; InputSandbox = {"JDLs/prologue.sh"}; OutputSandbox = {"std.out", "std.err","prologue.out", "file.txt"}; requirements = true; DataRequirements = { [ DataCatalogType = "DLI"; DataCatalog ="http://prod-lfc-shared-central.cern.ch:8085/"; InputData = { "guid:17c667b7-9622-4c95-be68-3d15d1eacadb" }; ] }; DataAccessProtocol = "gsiftp"; AllowZippedISB=true; RetryCount = 1; ShallowRetryCount = 2; [ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 1 -i -j JDLs/data-req.jdl ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) Test 1A: Submit to an LCG-CE Test 1B: Submit to a CREAM CE Test 1C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: cream-44.pd.infn.it ===> Started: 12:06:28 ===> Ended : 12:12:56 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111117120628.log </verbatim> Test log file is [[%ATTACHURL%/WMSService-TS_20111117120628.log][here]] ---+++++ Jdl with _ISBBaseURI_ and _OSBDestURI_ Test on CREAM ce: <verbatim> [ale@cream-12 DataReq]$ cat URI.jdl ########################################### # JDL with ISBURI & OSBDestURI # ########################################### Executable = "exe.sh"; StdOutput = "std.out"; StdError = "std.err"; InputSandboxBaseURI = "gsiftp://devel11.cnaf.infn.it:2811/tmp"; FuzzyRank = true; InputSandbox = {"exe.sh", "test1", "test2"}; OutputSandbox = {"std.out", "std.err"}; OutputSandboxDestURI = {"gsiftp://cream-44.pd.infn.it:2811/tmp/myjobOutput", "std.err"}; requirements = other.GlueCEStateStatus == "Testing" && regexp("/cream", other.GlueCEUniqueID); DataAccessProtocol = "gsiftp"; RetryCount = 1; ShallowRetryCount = 2; [ale@cream-12 DataReq]$ cat exe.sh #!/bin/sh NAME=`echo $GLITE_WMS_JOBID | sed -e 's/.*\///'` echo $NAME ls -l [ale@cream-12 DataReq]$ glite-wms-job-submit -a -c /home/ale/UI/etc/wmp_devel19.conf URI.jdl Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server ====================== glite-wms-job-submit Success ====================== The job has been successfully submitted to the WMProxy Your job identifier is: https://devel19.cnaf.infn.it:9000/c0pisHDY4fELX0ycBTdNsA ========================================================================== [ale@cream-12 DataReq]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/c0pisHDY4fELX0ycBTdNsA ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://devel19.cnaf.infn.it:9000/c0pisHDY4fELX0ycBTdNsA Current Status: Done (Success) Exit code: 0 Status Reason: Job Terminated Successfully Destination: cream-32.pd.infn.it:8443/cream-pbs-cert Submitted: Tue Nov 22 12:02:48 2011 CET ========================================================================== [ale@cream-12 DataReq]$ glite-wms-job-output https://devel19.cnaf.infn.it:9000/c0pisHDY4fELX0ycBTdNsA Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server ================================================================================ JOB GET OUTPUT OUTCOME Output sandbox files for the job: https://devel19.cnaf.infn.it:9000/c0pisHDY4fELX0ycBTdNsA have been successfully retrieved and stored in the directory: /tmp/jobOutput/ale_c0pisHDY4fELX0ycBTdNsA ================================================================================ [ale@cream-12 DataReq]$ ls -l /tmp/jobOutput/ale_c0pisHDY4fELX0ycBTdNsA total 0 -rw-rw-r-- 1 ale ale 0 Nov 22 12:17 std.err Log on cream-44: [root@cream-44 ~]# cat /tmp/myjobOutput c0pisHDY4fELX0ycBTdNsA total 8 -rwxr-xr-x 1 dteam002 dteam 81 Nov 22 12:03 exe.sh -rw-r--r-- 1 dteam002 dteam 0 Nov 22 12:03 std.err -rw-r--r-- 1 dteam002 dteam 23 Nov 22 12:03 std.out -rw-r--r-- 1 dteam002 dteam 0 Nov 22 12:03 test1 -rw-r--r-- 1 dteam002 dteam 0 Nov 22 12:03 test2 ===> test PASSED </verbatim> Same process with an LCG CE: <verbatim> [ale@cream-12 DataReq]$ glite-wms-job-submit -a -c /home/ale/UI/etc/wmp_devel19.conf URI.jdl Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server ====================== glite-wms-job-submit Success ====================== The job has been successfully submitted to the WMProxy Your job identifier is: https://devel19.cnaf.infn.it:9000/AqhMc7zmXX01vPti6o-H_A ========================================================================== [ale@cream-12 DataReq]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/AqhMc7zmXX01vPti6o-H_A ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://devel19.cnaf.infn.it:9000/AqhMc7zmXX01vPti6o-H_A Current Status: Done (Success) Exit code: 0 Status Reason: Job terminated successfully Destination: cream-26.pd.infn.it:2119/jobmanager-lcgpbs-creamtest2 Submitted: Tue Nov 22 12:52:58 2011 CET ========================================================================== [ale@cream-12 DataReq]$ glite-wms-job-output https://devel19.cnaf.infn.it:9000/AqhMc7zmXX01vPti6o-H_A Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server ================================================================================ JOB GET OUTPUT OUTCOME Output sandbox files for the job: https://devel19.cnaf.infn.it:9000/AqhMc7zmXX01vPti6o-H_A have been successfully retrieved and stored in the directory: /tmp/jobOutput/ale_AqhMc7zmXX01vPti6o-H_A ================================================================================ [ale@cream-12 DataReq]$ ls -l /tmp/jobOutput/ale_AqhMc7zmXX01vPti6o-H_A total 0 -rw-rw-r-- 1 ale ale 0 Nov 22 12:57 std.err Log on cream-44: [root@cream-44 ~]# cat /tmp/myjobOutput AqhMc7zmXX01vPti6o-H_A total 28 -rwxr-xr-x 1 dteam003 dteam 81 Nov 22 12:53 exe.sh -rw-r--r-- 1 dteam003 dteam 118 Nov 22 12:53 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2fAqhMc7zmXX01vPti6o-H_5fA.output -rw-r--r-- 1 dteam003 dteam 0 Nov 22 12:53 std.err -rw-r--r-- 1 dteam003 dteam 23 Nov 22 12:53 std.out -rw-r--r-- 1 dteam003 dteam 0 Nov 22 12:53 test1 -rw-r--r-- 1 dteam003 dteam 0 Nov 22 12:53 test2 -rw------- 1 dteam003 dteam 103 Nov 22 12:53 tmp.flIel23998 -rw------- 1 dteam003 dteam 2 Nov 22 12:53 tmp.fUCFN23997 -rw------- 1 dteam003 dteam 2 Nov 22 12:53 tmp.IZYca24012 -rw------- 1 dteam003 dteam 103 Nov 22 12:53 tmp.OLIhC24013 -rw------- 1 dteam003 dteam 0 Nov 22 12:53 tmp.RJYqf24011 ===> === test PASSED === </verbatim> ---+++++ Jdl with _ISBBaseURI_ and _OSBBaseDestURI_ Submit to CREAM CE <verbatim> [ale@cream-12 DataReq]$ cat BaseURI.jdl ########################################### # JDL with ISBURI & OSBBaseDestURI # ########################################### Executable = "exe.sh"; StdOutput = "std.out"; StdError = "std.err"; InputSandboxBaseURI = "gsiftp://devel11.cnaf.infn.it:2811/tmp"; FuzzyRank = true; InputSandbox = {"exe.sh", "test1", "test2"}; OutputSandbox = {"std.out", "std.err"}; OutputSandboxBaseDestURI="gsiftp://cream-44.pd.infn.it:2811/tmp"; requirements = other.GlueCEStateStatus == "Testing" && regexp("/cream", other.GlueCEUniqueID); RetryCount = 1; ShallowRetryCount = 2; [ale@cream-12 DataReq]$ glite-wms-job-submit -a -c /home/ale/UI/etc/wmp_devel19.conf BaseURI.jdl Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server ====================== glite-wms-job-submit Success ====================== The job has been successfully submitted to the WMProxy Your job identifier is: https://devel19.cnaf.infn.it:9000/D5jZSqbE7r94hHpKwUI8SA ========================================================================== [ale@cream-12 DataReq]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/D5jZSqbE7r94hHpKwUI8SA ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://devel19.cnaf.infn.it:9000/D5jZSqbE7r94hHpKwUI8SA Current Status: Done (Success) Exit code: 0 Status Reason: Job Terminated Successfully Destination: cream-29.pd.infn.it:8443/cream-lsf-creamcert2 Submitted: Tue Nov 22 13:05:26 2011 CET ========================================================================== [ale@cream-12 DataReq]$ glite-wms-job-output https://devel19.cnaf.infn.it:9000/D5jZSqbE7r94hHpKwUI8SA Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server ================================================================================ JOB GET OUTPUT OUTCOME No output files to be retrieved for the job: https://devel19.cnaf.infn.it:9000/D5jZSqbE7r94hHpKwUI8SA ================================================================================ [ale@cream-12 DataReq]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/D5jZSqbE7r94hHpKwUI8SA ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://devel19.cnaf.infn.it:9000/D5jZSqbE7r94hHpKwUI8SA Current Status: Cleared Status Reason: user retrieved output sandbox Destination: cream-29.pd.infn.it:8443/cream-lsf-creamcert2 Submitted: Tue Nov 22 13:05:26 2011 CET ========================================================================== Log on cream-44: [root@cream-44 ~]# ls -l /tmp/std.* -rw-r--r-- 1 dteam009 dteam 0 Nov 22 13:05 /tmp/std.err -rw-r--r-- 1 dteam009 dteam 286 Nov 22 13:05 /tmp/std.out [root@cream-44 ~]# cat /tmp/std.out D5jZSqbE7r94hHpKwUI8SA total 8 -rwxr-xr-x 1 dteam017 dteam 81 Nov 22 13:05 exe.sh -rw-r--r-- 1 dteam017 dteam 0 Nov 22 13:05 std.err -rw-r--r-- 1 dteam017 dteam 23 Nov 22 13:05 std.out -rw-r--r-- 1 dteam017 dteam 0 Nov 22 13:05 test1 -rw-r--r-- 1 dteam017 dteam 0 Nov 22 13:05 test2 ===> === test PASSED === </verbatim> Submit to an LCG CE: <verbatim> [ale@cream-12 DataReq]$ glite-wms-job-submit -a -c /home/ale/UI/etc/wmp_devel19.conf BaseURI.jdl Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server ====================== glite-wms-job-submit Success ====================== The job has been successfully submitted to the WMProxy Your job identifier is: https://devel19.cnaf.infn.it:9000/kCtn3ymc2rgeYYn_GFD-XA ========================================================================== [ale@cream-12 DataReq]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/kCtn3ymc2rgeYYn_GFD-XA ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://devel19.cnaf.infn.it:9000/kCtn3ymc2rgeYYn_GFD-XA Current Status: Done (Success) Exit code: 0 Status Reason: Job terminated successfully Destination: cream-26.pd.infn.it:2119/jobmanager-lcgpbs-creamtest2 Submitted: Tue Nov 22 13:11:26 2011 CET ========================================================================== [ale@cream-12 DataReq]$ glite-wms-job-output https://devel19.cnaf.infn.it:9000/kCtn3ymc2rgeYYn_GFD-XA Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server ================================================================================ JOB GET OUTPUT OUTCOME No output files to be retrieved for the job: https://devel19.cnaf.infn.it:9000/kCtn3ymc2rgeYYn_GFD-XA ================================================================================ [ale@cream-12 DataReq]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/kCtn3ymc2rgeYYn_GFD-XA ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://devel19.cnaf.infn.it:9000/kCtn3ymc2rgeYYn_GFD-XA Current Status: Cleared Status Reason: user retrieved output sandbox Destination: cream-26.pd.infn.it:2119/jobmanager-lcgpbs-creamtest2 Submitted: Tue Nov 22 13:11:26 2011 CET ========================================================================== Log on cream-44: [root@cream-44 ~]# ls -l /tmp/std.* -rw-r--r-- 1 dteam009 dteam 0 Nov 22 13:12 /tmp/std.err -rw-r--r-- 1 dteam009 dteam 724 Nov 22 13:12 /tmp/std.out [root@cream-44 ~]# cat /tmp/std.out kCtn3ymc2rgeYYn_GFD-XA total 28 -rwxr-xr-x 1 dteam003 dteam 81 Nov 22 13:12 exe.sh -rw-r--r-- 1 dteam003 dteam 118 Nov 22 13:12 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2fkCtn3ymc2rgeYYn_5fGFD-XA.output -rw-r--r-- 1 dteam003 dteam 0 Nov 22 13:12 std.err -rw-r--r-- 1 dteam003 dteam 23 Nov 22 13:12 std.out -rw-r--r-- 1 dteam003 dteam 0 Nov 22 13:12 test1 -rw-r--r-- 1 dteam003 dteam 0 Nov 22 13:12 test2 -rw------- 1 dteam003 dteam 103 Nov 22 13:12 tmp.eOONQ24674 -rw------- 1 dteam003 dteam 2 Nov 22 13:12 tmp.qmVcP24673 -rw------- 1 dteam003 dteam 2 Nov 22 13:12 tmp.vzbZX24688 -rw------- 1 dteam003 dteam 0 Nov 22 13:12 tmp.WFASP24687 -rw------- 1 dteam003 dteam 103 Nov 22 13:12 tmp.WNCMx24689 ===> === test PASSED === </verbatim> ---+++++ Jdl with _AllowZippedISB_ <verbatim> [ale@cream-12 DataReq]$ cat AllowZip.jdl ########################################### # JDL with AllowZippedISB attribute # ########################################### Executable = "/bin/ls"; Arguments = "-la"; StdOutput = "std.out"; StdError = "std.err"; InputSandbox = {"fileA", "fileB", "fileC", "fileD", "fileE"}; OutputSandbox = {"std.out", "std.err"}; requirements = other.GlueCEStateStatus == "Testing" && regexp("/cream", other.GlueCEUniqueID); AllowZippedISB=true; RetryCount = 1; ShallowRetryCount = 2; [ale@cream-12 DataReq]$ glite-wms-job-submit -a --debug -c /home/ale/UI/etc/wmp_cream-44.conf AllowZip.jdl ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Vo read from proxy certificate extension ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - VirtualOrganisation value : dteam ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Loaded user configuration file: /home/ale/UI/etc/wmp_cream-44.conf ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Configuration file: //etc/glite-wms/dteam/glite_wmsui.conf - MyProxyServer attribute(s) no more supported outside JDL Default Attributes section "JdlDefaultAttributes" ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Loaded Vo specific configuration file: //etc/glite-wms/dteam/glite_wmsui.conf ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Loaded Configuration values: [ JdlDefaultAttributes = [ SignificantAttributes = { "Requirements","Rank" }; RetryCount = 3; AllowZippedISB = true; VirtualOrganisation = "dteam"; rank = 1; MyProxyServer = "myproxy.cnaf.infn.it" ]; OutputStorage = "/tmp"; ListenerStorage = "/tmp"; ErrorStorage = "/tmp"; WmProxyEndPoints = { "https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server" } ] ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Getting Endpoint URL from configuration file ----------------------------------------- Connecting to the service https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Calling the WMProxy getVersion service ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - WMProxy Version: 3.3.4 ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Sending Proxy Request to https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Calling the WMProxy delegationns__getProxyReq service ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - delegationns__getProxyReq - The proxy has been successfully retrieved ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Calling the WMProxy delegationns__putProxyReq service ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - The proxy has been successfully delegated with the identifier: uehC70_noJCQm8Mb6-SBgQ ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Calling the WMProxy getTransferProtocols service ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - getTransferProtocols - Available protocols: gsiftp, https ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - FileTransferProtocol not specified; using the default protocol: gsiftp ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - The JDL file is: /home/ale/UI/DataReq/AllowZip.jdl ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Retrieving the list of the local ISB files from the user JDL ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Total size of the ISB file(s) to be transferred to: 3000105 ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Max single job size of the ISB file(s) to be transferred to: 2000069 ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Checking the User-FreeQuota on the server ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Calling the WMProxy getFreeQuota service ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - getFreeQuota - User freequota not set on the server ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Getting the max ISB size from the server https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Calling the WMProxy getMaxInputSandboxSize service ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - getMaxInputSandboxSize - Max ISB size information successfully retrieved ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - The max job size (2000069 bytes) doesn't exceed the max size limit of 100000000 bytes: File transfer is allowed ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - File archiving and file compression allowed by user in the JDL ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Registering JDL [ StdOutput = "std.out"; ShallowRetryCount = 2; SignificantAttributes = { "Requirements","Rank" }; RetryCount = 1; Executable = "/bin/ls"; Type = "job"; Arguments = "-la"; AllowZippedISB = true; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank = -other.GlueCEStateEstimatedResponseTime; ZippedISB = { "ISBfiles_c4KJb6dqpljoq0YCiadrEg_0.tar.gz" }; OutputSandbox = { "std.out","std.err" }; InputSandbox = { "file:///home/ale/UI/DataReq/fileA","file:///home/ale/UI/DataReq/fileB","file:///home/ale/UI/DataReq/fileC","file:///home/ale/UI/DataReq/fileD","file:///home/ale/UI/DataReq/fileE" }; StdError = "std.err"; rank = 1; MyProxyServer = "myproxy.cnaf.infn.it"; requirements = other.GlueCEStateStatus == "Testing" && regexp("/cream",other.GlueCEUniqueID) ] ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Registering the job to the service https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Calling the WMProxy jobRegister service ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - The JobId is: https://cream-44.pd.infn.it:9000/bsuo8b3PaOq3xF0TEzaqrw ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Archiving the ISB files: /tmp/ISBfiles_c4KJb6dqpljoq0YCiadrEg_0.tar ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - This archive file has been successfully created: /tmp/ISBfiles_c4KJb6dqpljoq0YCiadrEg_0.tar ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Compressing the file (.gz): /tmp/ISBfiles_c4KJb6dqpljoq0YCiadrEg_0.tar ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:21 -I- PID: 17785 (Debug) - ISB ZIPPED file successfully created: /tmp/ISBfiles_c4KJb6dqpljoq0YCiadrEg_0.tar.gz ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:21 -I- PID: 17785 (Debug) - Getting the SandboxDestinationURI from the service https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:21 -I- PID: 17785 (Debug) - Calling the WMProxy getSandboxDestURI service with gsiftp protocol ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:22 -I- PID: 17785 (Debug) - getSandboxDestURI - Destination URIs sucessfully retrieved ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:22 -I- PID: 17785 (Debug) - DestinationURI: gsiftp://cream-44.pd.infn.it:2811/var/SandboxDir/bs/https_3a_2f_2fcream-44.pd.infn.it_3a9000_2fbsuo8b3PaOq3xF0TEzaqrw/input ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:22 -I- PID: 17785 (Debug) - FileTransfer (gsiftp): using globus-url-copy to transfer the local InputSandBox file(s) to the submission endpoint ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:22 -I- PID: 17785 (Debug) - File Transfer (gsiftp) Command: /usr/bin/globus-url-copy Source: file:///tmp/ISBfiles_c4KJb6dqpljoq0YCiadrEg_0.tar.gz Destination: gsiftp://cream-44.pd.infn.it:2811/var/SandboxDir/bs/https_3a_2f_2fcream-44.pd.infn.it_3a9000_2fbsuo8b3PaOq3xF0TEzaqrw/input/ISBfiles_c4KJb6dqpljoq0YCiadrEg_0.tar.gz ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:23 -I- PID: 17785 (Debug) - File Transfer (gsiftp) Transfer successfully done ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:23 -I- PID: 17785 (Debug) - Starting the job: https://cream-44.pd.infn.it:9000/bsuo8b3PaOq3xF0TEzaqrw ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:23 -I- PID: 17785 (Debug) - Calling the WMProxy jobStart service ----------------------------------------- ----------------------------------------- 22 Nov 2011, 13:34:23 -I- PID: 17785 (Debug) - jobStart - The job has been successfully started ----------------------------------------- ====================== glite-wms-job-submit Success ====================== The job has been successfully submitted to the WMProxy Your job identifier is: https://cream-44.pd.infn.it:9000/bsuo8b3PaOq3xF0TEzaqrw ========================================================================== *** Log file created *** Possible Errors and Debug messages have been printed in the following file: /tmp/glite-wms-job-submit_500_17785_20111122_13-34-20.log Check if ZippedISB Attribute is added: [ale@cream-12 DataReq]$ glite-wms-job-info --jdl https://cream-44.pd.infn.it:9000/bsuo8b3PaOq3xF0TEzaqrw | grep Allow AllowZippedISB = true; [ale@cream-12 DataReq]$ glite-wms-job-status https://cream-44.pd.infn.it:9000/bsuo8b3PaOq3xF0TEzaqrw ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://cream-44.pd.infn.it:9000/bsuo8b3PaOq3xF0TEzaqrw Current Status: Done (Success) Exit code: 0 Status Reason: Job Terminated Successfully Destination: cream-29.pd.infn.it:8443/cream-lsf-creamcert2 Submitted: Tue Nov 22 13:34:20 2011 CET ========================================================================== Retrieve oputput files: [ale@cream-12 DataReq]$ glite-wms-job-output https://cream-44.pd.infn.it:9000/bsuo8b3PaOq3xF0TEzaqrw Connecting to the service https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server ================================================================================ JOB GET OUTPUT OUTCOME Output sandbox files for the job: https://cream-44.pd.infn.it:9000/bsuo8b3PaOq3xF0TEzaqrw have been successfully retrieved and stored in the directory: /tmp/jobOutput/ale_bsuo8b3PaOq3xF0TEzaqrw ================================================================================ Check std output to see if all the files are transfered to the CE: [ale@cream-12 DataReq]$ cat /tmp/jobOutput/ale_bsuo8b3PaOq3xF0TEzaqrw/std.out total 2964 drwxr-xr-x 2 dteam017 dteam 4096 Nov 22 13:34 . drwxr-xr-x 3 dteam017 dteam 4096 Nov 22 13:34 .. -rw-r--r-- 1 dteam017 dteam 2282 Nov 22 13:34 .BrokerInfo -rw-r--r-- 1 dteam017 dteam 17 Nov 22 13:34 fileA -rw-r--r-- 1 dteam017 dteam 17 Nov 22 13:34 fileB -rw-r--r-- 1 dteam017 dteam 0 Nov 22 13:34 fileC -rw-r--r-- 1 dteam017 dteam 1000002 Nov 22 13:34 fileD -rw-r--r-- 1 dteam017 dteam 2000069 Nov 22 13:34 fileE -rw-r--r-- 1 dteam017 dteam 0 Nov 22 13:34 std.err -rw-r--r-- 1 dteam017 dteam 0 Nov 22 13:34 std.out -rw-r--r-- 1 dteam017 dteam 0 Nov 22 13:34 .tmp_file ===> === test PASSED === </verbatim> ---+++++ Jdl with _ExpiryTime_ <verbatim> [ale@cream-12 DataReq]$ cat ExpiryTime.jdl ########################################### # JDL with ExpiryTime attribute # ########################################### Executable = "/bin/ls"; Arguments = "-la"; StdOutput = "std.out"; StdError = "std.err"; InputSandbox = {"fileA", "fileB", "fileC", "fileD", "fileE"}; OutputSandbox = {"std.out", "std.err"}; requirements = false; ExpiryTime = 1321966000; RetryCount = 1; ShallowRetryCount = 2; [ale@cream-12 DataReq]$ date +%s 1321965735 [ale@cream-12 DataReq]$ glite-wms-job-submit -a -c /home/ale/UI/etc/wmp_cream-44.conf ExpiryTime.jdl Connecting to the service https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server ====================== glite-wms-job-submit Success ====================== The job has been successfully submitted to the WMProxy Your job identifier is: https://cream-44.pd.infn.it:9000/KlcFaVR5qp7DMIRLOsz_ag ========================================================================== [ale@cream-12 DataReq]$ glite-wms-job-status https://cream-44.pd.infn.it:9000/KlcFaVR5qp7DMIRLOsz_ag ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://cream-44.pd.infn.it:9000/KlcFaVR5qp7DMIRLOsz_ag Current Status: Waiting Status Reason: BrokerHelper: no compatible resources Submitted: Tue Nov 22 13:42:20 2011 CET ========================================================================== [ale@cream-12 DataReq]$ date +%s 1321966381 [ale@cream-12 DataReq]$ glite-wms-job-status https://cream-44.pd.infn.it:9000/KlcFaVR5qp7DMIRLOsz_ag ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://cream-44.pd.infn.it:9000/KlcFaVR5qp7DMIRLOsz_ag Current Status: Aborted Status Reason: request expired Submitted: Tue Nov 22 13:42:20 2011 CET ========================================================================== ===> === test PASSED === </verbatim> ---+++++ Jdl with _ShortDeadlineJob_ <verbatim> [ale@cream-12 DataReq]$ cat JDLs/SDLJ.jdl ########################################### # JDL with ShortDeadlineJob attribute # ########################################### Executable = "/bin/ls"; Arguments = "-la"; StdOutput = "std.out"; StdError = "std.err"; InputSandbox = {"fileA", "fileB", "fileC", "fileD", "fileE"}; OutputSandbox = {"std.out", "std.err"}; requirements = true; ShortDeadlineJob = true; RetryCount = 1; ShallowRetryCount = 2; [ale@cream-12 WMS-service]$ glite-wms-job-list-match -a -c /home/ale/UI/etc/wmp_devel19.conf JDLs/SDLJ.jdl Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server ========================================================================== COMPUTING ELEMENT IDs LIST The following CE(s) matching your job requirements have been found: *CEId* - ce01.ariagni.hellasgrid.gr:2119/jobmanager-pbs-dteam.sdj - ce01.marie.hellasgrid.gr:2119/jobmanager-pbs-sdj - ce02.marie.hellasgrid.gr:2119/jobmanager-pbs-sdj - cream-ce01.ariagni.hellasgrid.gr:8443/cream-pbs-dteam.sdj - cream-ce01.marie.hellasgrid.gr:8443/cream-pbs-sdj - grid4.mif.vu.lt:2119/jobmanager-lcgpbs-sdj - grive02.ibcp.fr:8443/cream-pbs-sdj - mario.univ-lille1.fr:8443/cream-pbs-sdj ========================================================================== [ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 1 -i -j JDLs/SDLJ.jdl ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) Test 1A: Submit to an LCG-CE Test 1B: Submit to a CREAM CE Test 1C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: devel19.cnaf.infn.it ===> Started: 15:47:17 ===> Ended : 15:55:47 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111122154717.log </verbatim> Test log file is [[%ATTACHURL%/WMSService-TS_20111122154717.log][here]] ---++++ Collection ---+++++ Bulk submission <verbatim> [ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 2 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 2: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) Test 2A: Submit to an LCG-CE Test 2B: Submit to a CREAM CE Test 2C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: cream-44.pd.infn.it ===> Started: 12:13:08 ===> Ended : 12:19:10 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111117121308.log You have new mail in /var/spool/mail/ale [ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 2 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 2: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) Test 2A: Submit to an LCG-CE Test 2B: Submit to a CREAM CE Test 2C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: cream-44.pd.infn.it ===> Started: 15:38:31 ===> Ended : 15:44:31 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111117153831.log [ale@cream-12 WMS-service]$ voms-proxy-info -all subject : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy issuer : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy identity : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy type : proxy strength : 2048 bits path : /tmp/x509up_u500 timeleft : 6:49:04 key usage : Digital Signature, Key Encipherment, Data Encipherment === VO dteam extension information === VO : dteam subject : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle issuer : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr attribute : /dteam/Role=NULL/Capability=NULL attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL timeleft : 7:48:04 uri : voms2.hellasgrid.gr:15004 [ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 2 ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Set 2: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) Test 2A: Submit to an LCG-CE Test 2B: Submit to a CREAM CE Test 2C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: cream-44.pd.infn.it ===> Started: 15:45:22 ===> Ended : 15:53:23 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111117154522.log </verbatim> See [[%ATTACHURL%/submitcoll.tgz][gzipped log]] file ---+++++ All in a jdl <verbatim> [ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 7 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 7: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) using a single jdl with al the jdls of nodes Test 7A: Submit to an LCG-CE Test 7B: Submit to a CREAM CE Test 7C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: cream-44.pd.infn.it ===> Started: 16:51:46 ===> Ended : 16:59:22 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111117165146.log [ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 7 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 7: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) using a single jdl with al the jdls of nodes Test 7A: Submit to an LCG-CE Test 7B: Submit to a CREAM CE Test 7C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: cream-44.pd.infn.it ===> Started: 17:00:03 ===> Ended : 17:06:41 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111117170003.log [ale@cream-12 WMS-service]$ voms-proxy-info -all subject : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy issuer : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy identity : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy type : proxy strength : 2048 bits path : /tmp/x509up_u500 timeleft : 5:46:00 key usage : Digital Signature, Key Encipherment, Data Encipherment === VO dteam extension information === VO : dteam subject : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle issuer : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr attribute : /dteam/Role=NULL/Capability=NULL attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL timeleft : 6:45:00 uri : voms2.hellasgrid.gr:15004 [ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 7 ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Set 7: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) using a single jdl with al the jdls of nodes Test 7A: Submit to an LCG-CE Test 7B: Submit to a CREAM CE Test 7C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: cream-44.pd.infn.it ===> Started: 17:10:15 ===> Ended : 17:16:25 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111117171015.log </verbatim> See [[%ATTACHURL%/collection.tgz][gzipped log]] file ---++++ Parametric job <verbatim> [ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 3 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 3: Submit a parametric job (3 cases LCG-CE, CREAM, general) Test 3A: Submit to an LCG-CE Test 3B: Submit to a CREAM CE Test 3C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: cream-44.pd.infn.it ===> Started: 17:21:01 ===> Ended : 17:29:29 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111117172101.log [ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 3 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 3: Submit a parametric job (3 cases LCG-CE, CREAM, general) Test 3A: Submit to an LCG-CE Test 3B: Submit to a CREAM CE Test 3C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: cream-44.pd.infn.it ===> Started: 17:31:41 ===> Ended : 17:40:42 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111117173141.log [ale@cream-12 WMS-service]$ voms-proxy-info -all subject : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy issuer : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy identity : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy type : proxy strength : 2048 bits path : /tmp/x509up_u500 timeleft : 5:08:06 key usage : Digital Signature, Key Encipherment, Data Encipherment === VO dteam extension information === VO : dteam subject : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle issuer : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr attribute : /dteam/Role=NULL/Capability=NULL attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL timeleft : 6:07:06 uri : voms2.hellasgrid.gr:15004 [ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 3 ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Set 3: Submit a parametric job (3 cases LCG-CE, CREAM, general) Test 3A: Submit to an LCG-CE Test 3B: Submit to a CREAM CE Test 3C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: cream-44.pd.infn.it ===> Started: 17:48:08 ===> Ended : 17:54:45 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111117174808.log </verbatim> See [[%ATTACHURL%/parametric.tgz][gzipped log]] file ---++++ DAG job <verbatim> [ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 4 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 4: Submit a DAG job ===> ===> Test: WMS-job-cycle.py ===> WMS: cream-44.pd.infn.it ===> Started: 17:58:23 ===> Ended : 18:03:39 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111117175823.log [ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 4 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 4: Submit a DAG job ===> ===> Test: WMS-job-cycle.py ===> WMS: cream-44.pd.infn.it ===> Started: 18:04:03 ===> Ended : 18:09:16 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111117180403.log [ale@cream-12 WMS-service]$ voms-proxy-info -all subject : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy issuer : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy identity : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy type : proxy strength : 2048 bits path : /tmp/x509up_u500 timeleft : 4:45:13 key usage : Digital Signature, Key Encipherment, Data Encipherment === VO dteam extension information === VO : dteam subject : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle issuer : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr attribute : /dteam/Role=NULL/Capability=NULL attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL timeleft : 5:44:13 uri : voms2.hellasgrid.gr:15004 [ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 4 ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Set 4: Submit a DAG job ===> ===> Test: WMS-job-cycle.py ===> WMS: cream-44.pd.infn.it ===> Started: 18:11:01 ===> Ended : 18:16:12 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111117181101.log </verbatim> See [[%ATTACHURL%/dag.tgz][gzipped log]] file ---++++ Parallel job <verbatim> [ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 5 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 5: Submit a MPI job (3 cases LCG-CE, CREAM, general) Test 5A: Submit to an LCG-CE Test 5B: Submit to a CREAM CE Test 5C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: cream-44.pd.infn.it ===> Started: 13:25:11 ===> Ended : 13:36:07 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111118132511.log [ale@cream-12 WMS-service]$ voms-proxy-info -all subject : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy issuer : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy identity : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy type : proxy strength : 2048 bits path : /tmp/x509up_u500 timeleft : 10:00:27 key usage : Digital Signature, Key Encipherment, Data Encipherment === VO dteam extension information === VO : dteam subject : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle issuer : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr attribute : /dteam/Role=NULL/Capability=NULL attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL timeleft : 10:59:27 uri : voms2.hellasgrid.gr:15004 [ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 5 ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Set 5: Submit a MPI job (3 cases LCG-CE, CREAM, general) Test 5A: Submit to an LCG-CE Test 5B: Submit to a CREAM CE Test 5C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: cream-44.pd.infn.it ===> Started: 15:18:11 ===> Ended : 15:24:04 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111118151811.log </verbatim> See [[%ATTACHURL%/parallel.tgz][gzipped log]] file ---+++++ Testing forwarding parameters for parallel jobs In order to test the passing of mpi parameters (wholenodes,smpgranularity,cpunumber,hostnumber) to the ce, an mpi jdl is submitted with varying values for the parameters, specifying a cream CE as destination. Then the cream jobid is retrieved via a glite-wms-job-logging-info -v 2 ("Dest jobid" field of a Transfer event).Finally the cream jdl is found via glite-ce-job-status -L 2 <cream-jobid>. The following jdls were submitted: 1) jdl: <verbatim>Executable = "openmpi-wrapper.sh"; Arguments="hello"; StdOutput = "hello.out"; StdError = "hello.err"; OutputSandbox = {"hello.out","hello.err"}; InputSandbox = {"/home/fabiok/mpi/openmpi-wrapper.sh","/home/fabiok/mpi/hello.c","/home/fabiok/mpi/Makefile"}; Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID); WholeNodes = true; SMPGranularity = 2; CpuNumber=2; Hostnumber = 1;</verbatim> Resultim cream jdl: <verbatim>JDL = [[ CEInfoHostName = "ce-02.roma3.infn.it"; OutputSandboxPath = "/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/output"; StdOutput = "hello.out"; BatchSystem = "pbs"; GlobusResourceContactString = "ce-02.roma3.infn.it:8443/cream-pbs"; QueueName = "cert"; ShallowRetryCount = 10; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 3; Executable = "openmpi-wrapper.sh"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000011:BH=0000000000:JSS=000004:LM=000012:LRMS=000000:APP=000000:LBS=000000"; Arguments = "hello"; AllowZippedISB = true; maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank = -other.GlueCEStateEstimatedResponseTime; ce_id = "ce-02.roma3.infn.it:8443/cream-pbs-cert"; ZippedISB = { "ISBfiles_IggVNRt3plhKwsv7zkIQRw_0.tar.gz" }; OutputSandboxDestURI = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/output/hello.out","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/output/hello.err" }; OutputSandbox = { "hello.out","hello.err" }; CeRequirements = "( true && ( true && ( true && ( true && ( true && ( true && Member(\"OPENMPI\",other.GlueHostApplicationSoftwareRunTimeEnvironment) ) ) ) ) ) ) && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )"; InputSandbox = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/input/openmpi-wrapper.sh","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/input/hello.c","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/input/Makefile","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/input/.BrokerInfo" }; edg_jobid = "https://cream-45.pd.infn.it:9000/gQF7dCwrxP9pfDne1RfyOA"; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini"; StdError = "hello.err"; SMPGranularity = 2; InputSandboxPath = "/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/input"; CeApplicationDir = "/storage/local/exp_soft/dteam"; ReallyRunningToken = "gsiftp://cream-45.pd.infn.it/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/token.txt_0"; rank = -other.GlueCEStateEstimatedResponseTime; WholeNodes = true; CpuNumber = 2; edg_previous_matches = { "ce.scope.unina.it" }; WMSHostname = "cream-45.pd.infn.it"; lrms_type = "pbs"; MyProxyServer = "myproxy.cnaf.infn.it"; edg_previous_matches_ex = { [ timestamp = 1319039372; ce_id = "ce.scope.unina.it" ] }; X509UserProxy = "/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/user.proxy"; InputSandboxDestFileName = { "openmpi-wrapper.sh","hello.c","Makefile" }; Hostnumber = 1; requirements = ( ( ( ( ( ( Member("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*",other.GlueCEUniqueID) ) && ( other.GlueCEStateStatus == "Production" ) ) && other.GlueCEInfoTotalCPUs >= 2 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueCEInfoTotalCPUs >= 2 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) : !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA" ]]</verbatim> 2) jdl: <verbatim>CpuNumber=1; Executable = "openmpi-wrapper.sh"; Arguments="hello"; StdOutput = "hello.out"; StdError = "hello.err"; OutputSandbox = {"hello.out","hello.err"}; InputSandbox = {"/home/fabiok/mpi/openmpi-wrapper.sh","/home/fabiok/mpi/hello.c","/home/fabiok/mpi/Makefile"}; Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID); WholeNodes = true; SMPGranularity = 2;</verbatim> Resulting cream jdl: <verbatim>JDL = [[ CEInfoHostName = "ce-02.roma3.infn.it"; OutputSandboxPath = "/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/output"; StdOutput = "hello.out"; BatchSystem = "pbs"; GlobusResourceContactString = "ce-02.roma3.infn.it:8443/cream-pbs"; QueueName = "cert"; ShallowRetryCount = 10; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 3; Executable = "openmpi-wrapper.sh"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000011:BH=0000000000:JSS=000004:LM=000012:LRMS=000000:APP=000000:LBS=000000"; Arguments = "hello"; AllowZippedISB = true; maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank = -other.GlueCEStateEstimatedResponseTime; ce_id = "ce-02.roma3.infn.it:8443/cream-pbs-cert"; ZippedISB = { "ISBfiles_A4rYVNupOqd7YmRathOHog_0.tar.gz" }; OutputSandboxDestURI = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/output/hello.out","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/output/hello.err" }; OutputSandbox = { "hello.out","hello.err" }; CeRequirements = "( true && ( true && ( true && ( true && ( true && ( true && Member(\"OPENMPI\",other.GlueHostApplicationSoftwareRunTimeEnvironment) ) ) ) ) ) ) && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )"; InputSandbox = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/input/openmpi-wrapper.sh","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/input/hello.c","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/input/Makefile","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/input/.BrokerInfo" }; edg_jobid = "https://cream-45.pd.infn.it:9000/94xFQjidomDV4P_pzEey5A"; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini"; StdError = "hello.err"; SMPGranularity = 2; InputSandboxPath = "/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/input"; CeApplicationDir = "/storage/local/exp_soft/dteam"; ReallyRunningToken = "gsiftp://cream-45.pd.infn.it/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/token.txt_0"; rank = -other.GlueCEStateEstimatedResponseTime; WholeNodes = true; CpuNumber = 1; edg_previous_matches = { "gridsrv2-4.dir.garr.it" }; WMSHostname = "cream-45.pd.infn.it"; lrms_type = "pbs"; MyProxyServer = "myproxy.cnaf.infn.it"; edg_previous_matches_ex = { [ timestamp = 1319037684; ce_id = "gridsrv2-4.dir.garr.it" ] }; X509UserProxy = "/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/user.proxy"; InputSandboxDestFileName = { "openmpi-wrapper.sh","hello.c","Makefile" }; requirements = ( ( ( ( ( ( Member("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*",other.GlueCEUniqueID) ) && ( other.GlueCEStateStatus == "Production" ) ) && other.GlueCEInfoTotalCPUs >= 1 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueCEInfoTotalCPUs >= 1 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) : !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A" ]]</verbatim> 3) jdl: <verbatim>Executable = "openmpi-wrapper.sh"; Arguments="hello"; StdOutput = "hello.out"; StdError = "hello.err"; OutputSandbox = {"hello.out","hello.err"}; InputSandbox = {"/home/fabiok/mpi/openmpi-wrapper.sh","/home/fabiok/mpi/hello.c","/home/fabiok/mpi/Makefile"}; Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID); WholeNodes = false; SMPGranularity = 3; CpuNumber=3; Hostnumber = 1;</verbatim> Submission result: <verbatim>[fabiok@ui mpi]$ glite-wms-job-submit -a -c ../wms_cream-45.conf mpi.jdl Connecting to the service https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server Warning - Unable to register the job to the service: https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server SMPGranularity and HostNumber are mutually exclusive when WholeNodes allocation is not requested: wrong combination of values Method: jobRegister Error - Operations failed Unable to find any endpoint where to perform service request</verbatim> 4) jdl: <verbatim>Executable = "openmpi-wrapper.sh"; Arguments="hello"; StdOutput = "hello.out"; StdError = "hello.err"; OutputSandbox = {"hello.out","hello.err"}; InputSandbox = {"/home/fabiok/mpi/openmpi-wrapper.sh","/home/fabiok/mpi/hello.c","/home/fabiok/mpi/Makefile"}; Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID); WholeNodes = false; SMPGranularity = 3; CpuNumber=3; </verbatim> Resulting cream jdl: <verbatim>JDL = [[ CEInfoHostName = "ce-02.roma3.infn.it"; OutputSandboxPath = "/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/output"; StdOutput = "hello.out"; BatchSystem = "pbs"; GlobusResourceContactString = "ce-02.roma3.infn.it:8443/cream-pbs"; QueueName = "cert"; ShallowRetryCount = 10; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 3; Executable = "openmpi-wrapper.sh"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000017:BH=0000000000:JSS=000006:LM=000022:LRMS=000000:APP=000000:LBS=000000"; Arguments = "hello"; AllowZippedISB = true; maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank = -other.GlueCEStateEstimatedResponseTime; ce_id = "ce-02.roma3.infn.it:8443/cream-pbs-cert"; ZippedISB = { "ISBfiles_km6imNWXB17PDZruVO82Mg_0.tar.gz" }; OutputSandboxDestURI = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/output/hello.out","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/output/hello.err" }; OutputSandbox = { "hello.out","hello.err" }; CeRequirements = "( true && ( true && ( true && ( true && ( true && ( true && Member(\"OPENMPI\",other.GlueHostApplicationSoftwareRunTimeEnvironment) ) ) ) ) ) ) && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )"; InputSandbox = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/input/openmpi-wrapper.sh","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/input/hello.c","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/input/Makefile","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/input/.BrokerInfo" }; edg_jobid = "https://cream-45.pd.infn.it:9000/n2xH3q9eUcG1o0x9ZgFNEA"; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini"; StdError = "hello.err"; SMPGranularity = 3; InputSandboxPath = "/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/input"; CeApplicationDir = "/storage/local/exp_soft/dteam"; ReallyRunningToken = "gsiftp://cream-45.pd.infn.it/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/token.txt_0"; rank = -other.GlueCEStateEstimatedResponseTime; WholeNodes = false; CpuNumber = 3; edg_previous_matches = { "grisuce.scope.unina.it","ce.scope.unina.it" }; WMSHostname = "cream-45.pd.infn.it"; lrms_type = "pbs"; MyProxyServer = "myproxy.cnaf.infn.it"; edg_previous_matches_ex = { [ timestamp = 1319039871; ce_id = "grisuce.scope.unina.it" ],[ timestamp = 1319039874; ce_id = "ce.scope.unina.it" ] }; X509UserProxy = "/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/user.proxy"; InputSandboxDestFileName = { "openmpi-wrapper.sh","hello.c","Makefile" }; requirements = ( ( ( ( ( ( Member("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*",other.GlueCEUniqueID) ) && ( other.GlueCEStateStatus == "Production" ) ) && other.GlueCEInfoTotalCPUs >= 3 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueCEInfoTotalCPUs >= 3 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) : !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA" ]]</verbatim> 5) jdl: <pre>Executable = "openmpi-wrapper.sh"; Arguments="hello"; StdOutput = "hello.out"; StdError = "hello.err"; OutputSandbox = {"hello.out","hello.err"}; InputSandbox = {"/home/fabiok/mpi/openmpi-wrapper.sh","/home/fabiok/mpi/hello.c","/home/fabiok/mpi/Makefile"}; Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID); WholeNodes = true; SMPGranularity = 8; CpuNumber=8; Hostnumber = 2;</pre> Resulting cream jdl: <pre>JDL = [[ CEInfoHostName = "prabi-ce5-gpu.ibcp.fr"; OutputSandboxPath = "/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/output"; StdOutput = "hello.out"; BatchSystem = "pbs"; GlobusResourceContactString = "prabi-ce5-gpu.ibcp.fr:8443/cream-pbs"; QueueName = "dteam"; ShallowRetryCount = 10; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 3; Executable = "openmpi-wrapper.sh"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000005:BH=0000000000:JSS=000002:LM=000002:LRMS=000000:APP=000000:LBS=000000"; Arguments = "hello"; AllowZippedISB = true; maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank = -other.GlueCEStateEstimatedResponseTime; ce_id = "prabi-ce5-gpu.ibcp.fr:8443/cream-pbs-dteam"; ZippedISB = { "ISBfiles_4xXBPYiJ8Z6ZeIyiIUREnw_0.tar.gz" }; OutputSandboxDestURI = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/output/hello.out","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/output/hello.err" }; OutputSandbox = { "hello.out","hello.err" }; CeRequirements = "( true && ( true && ( true && ( true && ( true && ( true && Member(\"OPENMPI\",other.GlueHostApplicationSoftwareRunTimeEnvironment) ) ) ) ) ) ) && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )"; InputSandbox = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/input/openmpi-wrapper.sh","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/input/hello.c","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/input/Makefile","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/input/.BrokerInfo" }; edg_jobid = "https://cream-45.pd.infn.it:9000/HsKoKN4FdQ3oGzyhzrxLuw"; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini"; StdError = "hello.err"; SMPGranularity = 8; InputSandboxPath = "/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/input"; CeApplicationDir = "/swareas/dteam"; ReallyRunningToken = "gsiftp://cream-45.pd.infn.it/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/token.txt_0"; rank = -other.GlueCEStateEstimatedResponseTime; WholeNodes = true; CpuNumber = 8; WMSHostname = "cream-45.pd.infn.it"; lrms_type = "pbs"; MyProxyServer = "myproxy.cnaf.infn.it"; X509UserProxy = "/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/user.proxy"; InputSandboxDestFileName = { "openmpi-wrapper.sh","hello.c","Makefile" }; Hostnumber = 2; requirements = ( ( ( ( ( ( Member("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*",other.GlueCEUniqueID) ) && ( other.GlueCEStateStatus == "Production" ) ) && other.GlueCEInfoTotalCPUs >= 8 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueCEInfoTotalCPUs >= 8 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) : !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw" ]]</pre> ===> test PASSED ---++++ Perusal job <verbatim> [ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 6 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 6: Submit a perusal job BEWARE default min perusal interval is 1000 secs, so this phase could take many minutes ===> ===> Test: WMS-job-cycle.py ===> WMS: cream-44.pd.infn.it ===> Started: 18:18:54 ===> Ended : 18:46:20 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111117181854.log [ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 6 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 6: Submit a perusal job BEWARE default min perusal interval is 1000 secs, so this phase could take many minutes ===> ===> Test: WMS-job-cycle.py ===> WMS: cream-44.pd.infn.it ===> Started: 09:46:54 ===> Ended : 10:14:18 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111118094654.log [ale@cream-12 WMS-service]$ voms-proxy-info -all subject : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy issuer : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy identity : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy type : proxy strength : 2048 bits path : /tmp/x509up_u500 timeleft : 10:59:57 key usage : Digital Signature, Key Encipherment, Data Encipherment === VO dteam extension information === VO : dteam subject : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle issuer : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr attribute : /dteam/Role=NULL/Capability=NULL attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL timeleft : 11:58:57 uri : voms2.hellasgrid.gr:15004 [ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 6 ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Set 6: Submit a perusal job BEWARE default min perusal interval is 1000 secs, so this phase could take many minutes ===> ===> Test: WMS-job-cycle.py ===> WMS: cream-44.pd.infn.it ===> Started: 12:45:02 ===> Ended : 13:11:24 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111118124502.log </verbatim> See [[%ATTACHURL%/perusal.tgz][gzipped log]] file ---+++ Delegation <verbatim> [ale@cream-12 WMS-service]$ python WMS-delegation-test.py -c wms-command.conf -V dteam -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test delegation operation ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: Check --autm-delegation option Test 2: Check --delegationid option Test 3: Try to delegate with a short proxy and check the validity Test 4: Works with expired proxy ===> ===> Test: WMS-delegation-test.py ===> WMS: cream-44.pd.infn.it ===> Started: 17:05:09 ===> Ended : 17:06:24 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111118170509.log [ale@cream-12 WMS-service]$ python WMS-delegation-test.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test delegation operation ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: Check --autm-delegation option Test 2: Check --delegationid option Test 3: Try to delegate with a short proxy and check the validity Test 4: Works with expired proxy ===> ===> Test: WMS-delegation-test.py ===> WMS: cream-44.pd.infn.it ===> Started: 17:06:50 ===> Ended : 17:08:00 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111118170650.log [ale@cream-12 WMS-service]$ voms-proxy-info -all subject : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy issuer : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy identity : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy type : proxy strength : 2048 bits path : /tmp/x509up_u500 timeleft : 10:59:55 key usage : Digital Signature, Key Encipherment, Data Encipherment === VO dteam extension information === VO : dteam subject : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle issuer : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr attribute : /dteam/Role=NULL/Capability=NULL attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL timeleft : 11:58:55 uri : voms2.hellasgrid.gr:15004 [ale@cream-12 WMS-service]$ python WMS-delegation-test.py -c wms-command.conf -V dteam -d 3 -l ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test delegation operation ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: Check --autm-delegation option Test 2: Check --delegationid option There are other two tests which require the user proxy password. Use -i option to enable them ===> ===> Test: WMS-delegation-test.py ===> WMS: cream-44.pd.infn.it ===> Started: 17:19:56 ===> Ended : 17:20:00 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111118171956.log </verbatim> See [[%ATTACHURL%/delegation.tgz][gzipped log]] file ---+++ Job List-Match ---++++ Without data <verbatim> [ale@cream-12 WMS-service]$ python WMS-listmatch-test.py -c wms-command.conf -V dteam -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Job ListMatch Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: check if glite-wms-job-list-match command exists Test 2: test a simple matching Test 3: check --delegationid option Test 4: exclude a CE from the match Test 5: try a failure matching (Requirements == false) Test 6: try a matching with EnableWMSFeedback attribute true ===> ===> Test: WMS-listmatch-test.py ===> WMS: cream-44.pd.infn.it ===> Started: 17:49:39 ===> Ended : 17:49:49 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111118174939.log [ale@cream-12 WMS-service]$ python WMS-listmatch-test.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Job ListMatch Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: check if glite-wms-job-list-match command exists Test 2: test a simple matching Test 3: check --delegationid option Test 4: exclude a CE from the match Test 5: try a failure matching (Requirements == false) Test 6: try a matching with EnableWMSFeedback attribute true ===> ===> Test: WMS-listmatch-test.py ===> WMS: cream-44.pd.infn.it ===> Started: 17:49:51 ===> Ended : 17:50:02 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111118174951.log [ale@cream-12 WMS-service]$ voms-proxy-info -all subject : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy issuer : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy identity : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy type : proxy strength : 2048 bits path : /tmp/x509up_u500 timeleft : 10:29:30 key usage : Digital Signature, Key Encipherment, Data Encipherment === VO dteam extension information === VO : dteam subject : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle issuer : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr attribute : /dteam/Role=NULL/Capability=NULL attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL timeleft : 11:28:30 uri : voms2.hellasgrid.gr:15004 [ale@cream-12 WMS-service]$ python WMS-listmatch-test.py -c wms-command.conf -V dteam -d 3 -l ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Job ListMatch Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: check if glite-wms-job-list-match command exists Test 2: test a simple matching Test 3: check --delegationid option Test 4: exclude a CE from the match Test 5: try a failure matching (Requirements == false) Test 6: try a matching with EnableWMSFeedback attribute true ===> ===> Test: WMS-listmatch-test.py ===> WMS: cream-44.pd.infn.it ===> Started: 17:50:19 ===> Ended : 17:50:25 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111118175019.log </verbatim> See [[%ATTACHURL%/listmatch.tgz][gzipped log]] file ---++++ With data Two files have been registered on the SE srm-public.cern.ch: <verbatim>find an lfc for the dteam vo: lcg-infosites --vo dteam lfc set the lfc host: export LFC_HOST=prod-lfc-shared-central.cern.ch create the directory: lfc-mkdir /grid/dteam/fcapanni create the files: lcg-cr --vo dteam -d srm://srm-public.cern.ch:8443/castor/cern.ch/grid/dteam/fcapanni/e-2M.txt -l lfn:/grid/dteam/fcapanni/e-2M.txt "file://$PWD/e-2M.txt" lcg-cr --vo dteam -d srm://srm-public.cern.ch:8443/castor/cern.ch/grid/dteam/fcapanni/PI_1M.txt -l lfn:/grid/dteam/fcapanni/PI_1M.txt "file://$PWD/PI_1M.txt" </verbatim> Then the following jdl was submitted: <verbatim>########################################### # JDL with Data Requirements # ########################################### Executable = "calc-pi.sh"; # Arguments is the number of digits, must be < 1000000 Arguments = "1000"; StdOutput = "std.out"; StdError = "std.err"; Prologue = "prologue.sh"; FuzzyRank = true; InputSandbox = {"calc-pi.sh", "fileA", "fileB","prologue.sh"}; OutputSandbox = {"std.out", "std.err","out-PI.txt","out-e.txt"}; #Requirements = other.GlueCEInfoHostName != "spacin-ce1.dma.unina.it"; Requirements = true; DataRequirements = { [ DataCatalogType = "DLI"; DataCatalog = "http://prod-lfc-shared-central.cern.ch:8085"; InputData = {"lfn:/grid/dteam/fcapanni/PI_1M.txt","lfn:/grid/dteam/fcapanni/e-2M.txt"}; ] }; DataAccessProtocol = "gsiftp"; VirtualOrganisation = "dteam"; RetryCount = 1; ShallowRetryCount = 2;</verbatim> The job was assigned to the CE ce133.cern.ch. Verification that the SE srm-public.cern.ch is close to the CE ce133.cern.ch: <verbatim>[fabiok@ui DataReq]$ lcg-infosites --vo dteam closeSE|grep -A 5 ce133 Name of the CE: ce133.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam srm-alice.cern.ch srm-atlas.cern.ch srm-cms.cern.ch srm-lhcb.cern.ch srm-public.cern.ch</verbatim> ===> test PASSED ---++++ Gangmatching Tried a list-match considering the following !JDL: <verbatim> [ executable="ssh1.sh"; myproxyserver=""; SErequirements = anyMatch(other.storage.CloseSEs, target.GlueSAStateAvailableSpace > 20); CErequirements = other.GlueCEStateStatus == "Production" && other.GlueCEInfoTotalCPUs >= 2; requirements = CErequirements && SErequirements; inputsandbox={"ssh1.sh"}; stdoutput="out3.out"; stderror="err2.err"; outputsandbox={"out3.out", "err2.err", "ssh1.sh", "a"}; ] $ glite-wms-job-list-match -a -c cream44.conf gang.jdl Connecting to the service https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server ========================================================================== COMPUTING ELEMENT IDs LIST The following CE(s) matching your job requirements have been found: *CEId* - cccreamceli05.in2p3.fr:8443/cream-sge-long - cccreamceli05.in2p3.fr:8443/cream-sge-medium - cccreamceli05.in2p3.fr:8443/cream-sge-short - cccreamceli06.in2p3.fr:8443/cream-sge-long - cccreamceli06.in2p3.fr:8443/cream-sge-medium - cccreamceli06.in2p3.fr:8443/cream-sge-short - cccreamceli10.in2p3.fr:8443/cream-sge-long - cccreamceli10.in2p3.fr:8443/cream-sge-medium - cccreamceli10.in2p3.fr:8443/cream-sge-short - cmsce01.na.infn.it:2119/jobmanager-lcgpbs-cert - grid010.ct.infn.it:2119/jobmanager-lcgpbs-cert - inaf-ce-01.ct.pi2s2.it:8443/cream-lsf-cert - virgo-ce.roma1.infn.it:2119/jobmanager-lcgpbs-cert - cccreamceli09.in2p3.fr:8443/cream-sge-long - cccreamceli09.in2p3.fr:8443/cream-sge-medium - cccreamceli09.in2p3.fr:8443/cream-sge-short - ce-01.grid.sissa.it:8443/cream-pbs-blade - cccreamceli02.in2p3.fr:8443/cream-bqs-short - cccreamceli02.in2p3.fr:8443/cream-bqs-medium - unict-dmi-ce-01.ct.pi2s2.it:2119/jobmanager-lcglsf-cert - cream-ce.pg.infn.it:8443/cream-pbs-cert ========================================================================== </verbatim> Manually checked in the bdii some of matched CE and verified that they really meet the constraints. Tried also modifying the JDL setting: <verbatim> requirements = CErequirements; </verbatim> Many more CEs get matched with a list-match. ===> test PASSED ---+++ Prologue and Epilogue jobs <verbatim> [ale@cream-12 WMS-service]$ python WMS-prologue-epilogue.py -c wms-command.conf -V dteam -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS test prologue and epilogue attributes ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: Test prologue attribute Test 2: Test epilogue attribute Test 3: Test prologue and epilogue attributes ===> ===> Test: WMS-prologue-epilogue.py ===> WMS: cream-44.pd.infn.it ===> Started: 17:56:10 ===> Ended : 18:03:36 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111118175610.log [ale@cream-12 WMS-service]$ python WMS-prologue-epilogue.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS test prologue and epilogue attributes ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: Test prologue attribute Test 2: Test epilogue attribute Test 3: Test prologue and epilogue attributes ===> ===> Test: WMS-prologue-epilogue.py ===> WMS: cream-44.pd.infn.it ===> Started: 18:15:28 ===> Ended : 18:21:51 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111118181528.log [ale@cream-12 WMS-service]$ python WMS-prologue-epilogue.py -c wms-command.conf -V dteam -d 3 -l ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS test prologue and epilogue attributes ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: Test prologue attribute Test 2: Test epilogue attribute Test 3: Test prologue and epilogue attributes ===> ===> Test: WMS-prologue-epilogue.py ===> WMS: cream-44.pd.infn.it ===> Started: 11:59:49 ===> Ended : 12:07:40 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111121115949.log </verbatim> See [[%ATTACHURL%/epilogue.tgz][gzipped log]] file ---+++ Shallow and deep re-submission <verbatim> [ale@cream-12 WMS-service]$ python WMS-job-resubmit.py -c wms-command.conf -V dteam -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Job Resubmission Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: Try a shallow resubmission Test 2: Try a deep resubmission ===> ===> Test: WMS-job-resubmit.py ===> WMS: cream-44.pd.infn.it ===> Started: 12:17:57 ===> Ended : 12:28:09 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111121121757.log You have new mail in /var/spool/mail/ale [ale@cream-12 WMS-service]$ python WMS-job-resubmit.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Job Resubmission Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: Try a shallow resubmission Test 2: Try a deep resubmission ===> ===> Test: WMS-job-resubmit.py ===> WMS: cream-44.pd.infn.it ===> Started: 12:28:27 ===> Ended : 12:40:38 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111121122827.log [ale@cream-12 WMS-service]$ python WMS-job-resubmit.py -c wms-command.conf -V dteam -d 3 -l ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Job Resubmission Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: Try a shallow resubmission Test 2: Try a deep resubmission ===> ===> Test: WMS-job-resubmit.py ===> WMS: cream-44.pd.infn.it ===> Started: 12:40:42 ===> Ended : 12:49:48 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111121124042.log </verbatim> See [[%ATTACHURL%/resubmit.tgz][gzipped log]] file ---+++ WMS Job Cancel Testing <verbatim> [ale@cream-12 WMS-service]$ python WMS-job-cancel.py -c wms-command.conf -V dteam -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Job Cancel Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: Try to cancel a normal job Test 2: Try to cancel a DAG job Test 3: Try to cancel one node of a DAG job Test 4: Try to cancel a collection job Test 5: Try to cancel one node of a collection job Test 6: Try to cancel a parametric job Test 7: Try to cancel one node of a parametric job Test 8: Try to cancel a MPI job ===> ===> Test: WMS-job-cancel.py ===> WMS: devel19.cnaf.infn.it ===> Started: 13:21:24 ===> Ended : 13:29:22 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111121132124.log [ale@cream-12 WMS-service]$ python WMS-job-cancel.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Job Cancel Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: Try to cancel a normal job Test 2: Try to cancel a DAG job Test 3: Try to cancel one node of a DAG job Test 4: Try to cancel a collection job Test 5: Try to cancel one node of a collection job Test 6: Try to cancel a parametric job Test 7: Try to cancel one node of a parametric job Test 8: Try to cancel a MPI job ===> ===> Test: WMS-job-cancel.py ===> WMS: devel19.cnaf.infn.it ===> Started: 13:30:54 ===> Ended : 13:39:55 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111121133054.log [ale@cream-12 WMS-service]$ voms-proxy-info -all subject : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy issuer : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy identity : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy type : proxy strength : 2048 bits path : /tmp/x509up_u500 timeleft : 7:51:40 key usage : Digital Signature, Key Encipherment, Data Encipherment === VO dteam extension information === VO : dteam subject : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle issuer : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr attribute : /dteam/Role=NULL/Capability=NULL attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL timeleft : 8:50:40 uri : voms2.hellasgrid.gr:15004 [ale@cream-12 WMS-service]$ python WMS-job-cancel.py -c wms-command.conf -V dteam -d 3 -l ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Job Cancel Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: Try to cancel a normal job Test 2: Try to cancel a DAG job Test 3: Try to cancel one node of a DAG job Test 4: Try to cancel a collection job Test 5: Try to cancel one node of a collection job Test 6: Try to cancel a parametric job Test 7: Try to cancel one node of a parametric job Test 8: Try to cancel a MPI job ===> ===> Test: WMS-job-cancel.py ===> WMS: devel19.cnaf.infn.it ===> Started: 14:41:56 ===> Ended : 14:50:00 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111121144156.log </verbatim> See [[%ATTACHURL%/cancel.tgz][gzipped log]] file ---+++ Proxy renewal <verbatim> [ale@cream-12 WMS-service]$ python WMS-proxy-renewal.py -c wms-command2.conf -V dteam -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS test proxy renewal operation ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: Test proxy renewal with LCG CE Test 2: Test proxy renewal with CREAM CE Test 3: Test proxy renewal with LCG CE (without setting MYPROXYSERVER) Test 4: Test proxy renewal with CREAM CE (without setting MYPROXYSERVER) ===> ERROR: Timeout reached while waiting the job https://devel11.cnaf.infn.it:9000/ezPwvHZibxuCyErO_FIauQ to finish ===> ===> Test: WMS-proxy-renewal.py ===> WMS: devel11.cnaf.infn.it ===> Started: 10:36:38 ===> Ended : 11:47:05 ===> ===> >>> TEST FAILED <<< ===> ===> >>> failure reason: 1 test(s) fail(s): ['Test 4: Test proxy renewal with CREAM CE (without setting MYPROXYSERVER)'] <<< ===> ===> Test log file is WMSService-TS_20111122103638.log ===> Error messages have been written in /home/ale/wms_tests/WMS-service/WMSService-Test_20111122103638/errors.log ===> ===> Test directory /home/ale/wms_tests/WMS-service/WMSService-Test_20111122103638 has not been cleaned for debug purpose [ale@cream-12 WMS-service]$ python WMS-proxy-renewal.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS test proxy renewal operation ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: Test proxy renewal with LCG CE Test 2: Test proxy renewal with CREAM CE Test 3: Test proxy renewal with LCG CE (without setting MYPROXYSERVER) Test 4: Test proxy renewal with CREAM CE (without setting MYPROXYSERVER) ===> ERROR: Timeout reached while waiting the job https://devel19.cnaf.infn.it:9000/ekn6N9tFatImb64X7Rwy6Q to finish ===> ===> Test: WMS-proxy-renewal.py ===> WMS: devel19.cnaf.infn.it ===> Started: 10:18:16 ===> Ended : 11:30:09 ===> ===> >>> TEST FAILED <<< ===> ===> >>> failure reason: 1 test(s) fail(s): ['Test 4: Test proxy renewal with CREAM CE (without setting MYPROXYSERVER)'] <<< ===> ===> Test log file is WMSService-TS_20111122101816.log ===> Error messages have been written in /home/ale/wms_tests/WMS-service/WMSService-Test_20111122101816/errors.log ===> ===> Test directory /home/ale/wms_tests/WMS-service/WMSService-Test_20111122101816 has not been cleaned for debug purpose </verbatim> NB Last Test fail for bug [[https://savannah.cern.ch/bugs/?84839][#84839]] See [[%ATTACHURL%/pr.tgz][gzipped log]] file ---+++ Configuration file Verified that in the configuration file =/etc/glite-wms/glite_wms.conf= there are these hard-coded values: For the common section: <verbatim> DGUser = "\${GLITE_WMS_USER}" HostProxyFile = "\${WMS_LOCATION_VAR}/glite/wms.proxy" LBProxy = true </verbatim> For the JobController section: <verbatim> CondorSubmit = "${CONDORG_INSTALL_PATH}/bin/condor_submit" CondorRemove = "${CONDORG_INSTALL_PATH}/bin/condor_rm" CondorQuery = "${CONDORG_INSTALL_PATH}/bin/condor_q" CondorRelease = "${CONDORG_INSTALL_PATH}/bin/condor_release" CondorDagman = "${CONDORG_INSTALL_PATH}/bin/condor_dagman" DagmanMaxPre = 10 SubmitFileDir = "${WMS_LOCATION_VAR}/jobcontrol/submit" OutputFileDir = "${WMS_LOCATION_VAR}/jobcontrol/condorio" InputType = "jobdir" Input = "${WMS_LOCATION_VAR}/jobcontrol/jobdir/" LockFile = "${WMS_LOCATION_VAR}/jobcontrol/lock" LogFile = "\${WMS_LOCATION_LOG}/jobcontoller_events.log" LogLevel = 5 MaximumTimeAllowedForCondorMatch = 1800 ContainerRefreshThreshold = 1000 </verbatim> For the NetworkServer section: <verbatim> II_Port = 2170 Gris_Port = 2170 II_Timeout = 100 Gris_Timeout = 20 II_DN = "mds-vo-name=local, o=grid" Gris_DN = "mds-vo-name=local, o=grid" BacklogSize = 64 ListeningPort = 7772 MasterThreads = 8 DispatcherThreads = 10 SandboxStagingPath = "${WMS_LOCATION_VAR}/SandboxDir" LogFile = "${WMS_LOCATION_LOG}/networkserver_events.log" LogLevel = 5 EnableQuotaManagement = false MaxInputSandboxSize = 10000000 EnableDynamicQuotaAdjustment = false QuotaAdjustmentAmount = 10000 QuotaInsensibleDiskPortion = 2.0 DLI_SI_CatalogTimeout = 60 ConnectionTimeout = 300 </verbatim> For the LogMonitor section: <verbatim> JobsPerCondorLog = 1000 LockFile = "${WMS_LOCATION_VAR}/logmonitor/lock" LogFile = "${WMS_LOCATION_LOG}/logmonitor_events.log" LogLevel = 5 ExternalLogFile = "\${WMS_LOCATION_LOG}/logmonitor_external.log" MainLoopDuration = 5 CondorLogDir = "${WMS_LOCATION_VAR}/logmonitor/CondorG.log" CondorLogRecycleDir = "${WMS_LOCATION_VAR}/logmonitor/CondorG.log/recycle" MonitorInternalDir = "${WMS_LOCATION_VAR}/logmonitor/internal" IdRepositoryName = "irepository.dat" AbortedJobsTimeout = 600 GlobusDownTimeout = 7200 RemoveJobFiles = true ForceCancellationRetries = 2 </verbatim> For the Workloadmanager section: <verbatim> PipeDepth = 200 WorkerThreads = 5 DispatcherType = "jobdir" Input = "${WMS_LOCATION_VAR}/workload_manager/jobdir" LogLevel = 5 LogFile = "${WMS_LOCATION_LOG}/workload_manager_events.log" MaxRetryCount = 10 CeMonitorServices = {} CeMonitorAsynchPort = 0 IsmBlackList = {} IsmUpdateRate = 600 IsmIiPurchasingRate = 480 JobWrapperTemplateDir = "${WMS_JOBWRAPPER_TEMPLATE}" IsmThreads = false IsmDump = "${WMS_LOCATION_VAR}/workload_manager/ismdump.fl" SiServiceName = "org.glite.SEIndex" DliServiceName = "data-location-interface" MaxRetryCount = 10 DisablePurchasingFromGris = true EnableBulkMM = true CeForwardParameters = {"GlueHostMainMemoryVirtualSize","GlueHostMainMemoryRAMSize","GlueCEPolicyMaxCPUTime"} MaxOutputSandboxSize = -1 EnableRecovery = true QueueSize = 1000 ReplanGracePeriod = 3600 MaxReplansCount = 5 WmsRequirements = ((ShortDeadlineJob =?= TRUE) ? RegExp(".*sdj$", other.GlueCEUniqueID) : !RegExp(".*sdj$", other.GlueCEUniqueID)) && (other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs) && (EnableWmsFeedback =?= TRUE ? RegExp("cream", other.GlueCEImplementationName, "i") : true) </verbatim> For the WorkloadManagerProxy: <verbatim> SandboxStagingPath = "${WMS_LOCATION_VAR}/SandboxDir" LogFile = "${WMS_LOCATION_LOG}/wmproxy.log" LogLevel = 5 MaxInputSandboxSize = 100000000 ListMatchRootPath = "/tmp" GridFTPPort = 2811 LBLocalLogger = "localhost:9002" MinPerusalTimeInterval = 1000 AsyncJobStart = true EnableServiceDiscovery = false LBServiceDiscoveryType = "org.glite.lb.server" ServiceDiscoveryInfoValidity = 3600 WeightsCacheValidity = 86400 MaxServedRequests = 50 OperationsLoadScripts = [ jobRegister = "${WMS_LOCATION_SBIN}/glite_wms_wmproxy_load_monitor --oper jobRegister --load1 22 --load5 20 --load15 18 --memusage 99 --diskusage 95 --fdnum 1000 --jdnum 1500 --ftpconn 300" jobSubmit = "${WMS_LOCATION_SBIN}/glite_wms_wmproxy_load_monitor --oper jobSubmit --load1 22 --load5 20 --load15 18 --memusage 99 --diskusage 95 --fdnum 1000 --jdnum 1500 --ftpconn 300" RuntimeMalloc = "/usr/lib64/libtcmalloc_minimal.so" ] </verbatim> For the ICE section: <verbatim> start_listener = false start_lease_updater = false logfile = "${WMS_LOCATION_LOG}/ice.log" log_on_file = true creamdelegation_url_prefix = "https://" listener_enable_authz = true poller_status_threshold_time = 30*60 ice_topic = "CREAM_JOBS" subscription_update_threshold_time = 3600 lease_delta_time = 0 notification_frequency = 3*60 start_proxy_renewer = true max_logfile_size = 100*1024*1024 ice_host_cert = "${GLITE_HOST_CERT}" Input = "${WMS_LOCATION_VAR}/ice/jobdir" job_cancellation_threshold_time = 300 poller_delay = 2*60 persist_dir = "${WMS_LOCATION_VAR}/ice/persist_dir" lease_update_frequency = 20*60 log_on_console = false cream_url_postfix = "/ce-cream/services/CREAM2" subscription_duration = 86400 bulk_query_size = 100 purge_jobs = false InputType = "jobdir" listener_port = 7010 listener_enable_authn = true ice_host_key = "${GLITE_HOST_KEY}" start_poller = true creamdelegation_url_postfix = "/ce-cream/services/gridsite-delegation" cream_url_prefix = "https://" max_ice_threads = 10 cemon_url_prefix = "https://" start_subscription_updater = true proxy_renewal_frequency = 600 ice_log_level = 700 soap_timeout = 60 max_logfile_rotations = 10 cemon_url_postfix = "/ce-monitor/services/CEMonitor" max_ice_mem = 2096000 ice_empty_threshold = 600 </verbatim> Verified that: * The attribute =II_Contact= of =NetworkServer= section matches the value of the yaim variable =BDII_HOST= * The attribute =WMExpiryPeriod= of =WorkloadManager= section matches the value of yaim variable =WMS_EXPIRY_PERIOD= * The attribute =MatchRetryPeriod= of =WorkloadManager= section matches the value of yaim variable =WMS_MATCH_RETRY_PERIOD= * The attribute =IsmIiLDAPCEFilterExt= of =WorkloadManager= section is =(|(GlueCEAccessControlBaseRule=VO:vo1)(GlueCEAccessControlBaseRule=VOMS:/vo1/*)(GlueCEAccessControlBaseRule=VO=vo2...= * The attribute =LBServer= of the =WorkloadManagerProxy= section matches the value of yaim variable =LB_HOST= ===> test PASSED ---+++ WMS feedback <verbatim> [ale@cream-12 WMS-service]$ python WMS-feedback-test.py -c wms-command.conf -V dteam -d 3 -l ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Feedback ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: Test WMS feedback feature ===> ===> Test: WMS-feedback-test.py ===> WMS: cream-44.pd.infn.it ===> Started: 12:22:27 ===> Ended : 12:25:53 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111123122227.log </verbatim> Test log file is [[%ATTACHURL%/WMSService-TS_20111123122227.log][here]] ---+++ Limiter mechanism <verbatim> [ale@cream-12 WMS-service]$ python WMS-limiter-mechanism.py -c wms-command.conf -V dteam -d 3 -l -t 1,4,6-10 ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Limiter Mechanism ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: Test option --load1 threshold for load average (1 min) Test 4: Test option --memusage threshold for memory usage (%) Test 6: Test option --ftpconn threshold for number of FTP connections Test 7: Test option --diskusage threshold for disk usage (%) Test 8: Test option --fdnum threshold for used file descriptor Test 9: Test option --jdnum threshold for number of unprocessed jobs (for jobdir) Test 10: Test option --jdsize threshold for input jobdir size (KB) ===> ===> Test: WMS-limiter-mechanism.py ===> WMS: cream-44.pd.infn.it ===> Started: 11:59:45 ===> Ended : 12:00:15 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111123115945.log </verbatim> Test log file is [[%ATTACHURL%/WMSService-TS_20111123115945.log][here]] ---+++ Purge ---++++ Normal job cycle <verbatim> [ale@cream-12 WMS-service]$ python WMS-check-purger.py -c wms-command.conf -V dteam -d 3 -l ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Purge Mechanism ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: Test purge , normal job cycle ( Submit to LCG CE ) Test 2: Test purge, normal job cycle ( Submit to CREAM CE ) Test 3: Test proxy cache purging ===> ===> Test: WMS-check-purger.py ===> WMS: cream-44.pd.infn.it ===> Started: 12:57:02 ===> Ended : 13:09:03 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111123125702.log </verbatim> Test log file is [[%ATTACHURL%/WMSService-TS_20111123125702.log][here]] ---++++ SandBoxDir cron purger <verbatim> The purge of the SandBox dir is done by a cron job - Submit a job and force its purge before it finishes: [ale@cream-12 UI]$ glite-wms-job-submit -a -c /home/ale/UI/etc/wmp_devel19.conf test.jdl Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server ====================== glite-wms-job-submit Success ====================== The job has been successfully submitted to the WMProxy Your job identifier is: https://devel19.cnaf.infn.it:9000/qDSMCo6BvbAurBg7uy-R_w ========================================================================== [ale@cream-12 UI]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/qDSMCo6BvbAurBg7uy-R_w ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://devel19.cnaf.infn.it:9000/qDSMCo6BvbAurBg7uy-R_w Current Status: Scheduled Status Reason: unavailable Destination: cream-24.pd.infn.it:8443/cream-lsf-testbedB_1 Submitted: Wed Nov 23 13:07:12 2011 CET ========================================================================== [root@devel19 ~]# su - glite [glite@devel19 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/qD -s 23 Nov, 13:08:01 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/qDSMCo6BvbAurBg7uy-R_w: removed SCHEDULED job [ale@cream-12 UI]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/qDSMCo6BvbAurBg7uy-R_w ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://devel19.cnaf.infn.it:9000/qDSMCo6BvbAurBg7uy-R_w Current Status: Cleared Status Reason: timed out, resource purge forced Destination: cream-24.pd.infn.it:8443/cream-lsf-testbedB_1 Submitted: Wed Nov 23 13:07:12 2011 CET ========================================================================== ===> === test PASSED === - Purge a done job: [ale@cream-12 UI]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/ZZ7OoZhwB-6tvBaxTrheVw ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://devel19.cnaf.infn.it:9000/ZZ7OoZhwB-6tvBaxTrheVw Current Status: Done (Success) Exit code: 0 Status Reason: Job Terminated Successfully Destination: cream-23.pd.infn.it:8443/cream-lsf-cert Submitted: Sat Nov 19 15:03:32 2011 CET Parent Job: https://devel19.cnaf.infn.it:9000/jfna3yAaluQf4CfBZp38tQ ========================================================================== [glite@devel19 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/ZZ | grep ZZ7OoZhwB-6tvBaxTrheVw 23 Nov, 13:10:25 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/ZZ7OoZhwB-6tvBaxTrheVw: removed DONE job [ale@cream-12 UI]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/ZZ7OoZhwB-6tvBaxTrheVw ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://devel19.cnaf.infn.it:9000/ZZ7OoZhwB-6tvBaxTrheVw Current Status: Cleared Status Reason: timed out, resource purge forced Destination: cream-23.pd.infn.it:8443/cream-lsf-cert Submitted: Sat Nov 19 15:03:32 2011 CET Parent Job: https://devel19.cnaf.infn.it:9000/jfna3yAaluQf4CfBZp38tQ ========================================================================== - Purge only jobs older than 12 hours: [glite@devel19 ~]$ ls -l /var/SandboxDir/9O total 68 drwxrwx--- 5 dteam042 glite 4096 Nov 20 03:19 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9O0DOmLWZKbIix2SZrBQBA drwxrwx--- 5 dteam042 glite 4096 Nov 20 08:55 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9O1zK64Gl9qEaPFw-LHPbQ drwxrwx--- 5 dteam042 glite 4096 Nov 20 15:56 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9O2NaEZENXOsQ5ARurVy9Q drwxrwx--- 5 dteam042 glite 4096 Nov 19 21:11 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9O46dhW7QO5mihD_5f3arqUg drwxrwx--- 5 dteam002 glite 4096 Nov 19 08:30 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OAG_5fKMvemu6G6_5fh1QtsXw drwxrwx--- 5 dteam002 glite 4096 Nov 19 03:48 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9Of9YBbvsamRURogbFCTog drwxrwx--- 5 dteam042 glite 4096 Nov 19 10:15 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OIA4NvNIy_5fFqGDxSgq36A drwxrwx--- 5 dteam002 glite 4096 Nov 20 06:43 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OJKWi0W9tAqMEvk1KMpXg drwxrwx--- 5 dteam002 glite 4096 Nov 18 21:02 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OJpHAsHmN68qRMgRR79bQ drwxrwx--- 5 dteam002 glite 4096 Nov 19 18:42 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OkIA_5fHWdJYQ2jhWtzN2qQ drwxrwx--- 5 dteam042 glite 4096 Nov 23 13:19 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OMsNaP2bzC3oiToyAu5hA drwxrwx--- 5 dteam042 glite 4096 Nov 20 02:29 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OoF8o2i76X5cOEWO9_5fezw drwxrwx--- 5 dteam002 glite 4096 Nov 19 08:18 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OPevUO6LxxqDeoyUjrgjw drwxrwx--- 5 dteam042 glite 4096 Nov 20 00:28 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OsqPkEq2A0BjaxsS83DxQ drwxrwx--- 5 dteam042 glite 4096 Nov 20 07:54 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OvoOJROG6hsz-dAJsAGRQ drwxrwx--- 5 dteam042 glite 4096 Nov 20 14:26 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OWbkU0KtgP1fWYqA2SHuw drwxrwx--- 5 dteam002 glite 4096 Nov 20 08:34 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OYsYtPw4B6qBu-N3G4cNw [glite@devel19 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/9O -t 43200 23 Nov, 13:19:43 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9O1zK64Gl9qEaPFw-LHPbQ: removed DONE job 23 Nov, 13:19:43 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9OIA4NvNIy_FqGDxSgq36A: removed DONE job 23 Nov, 13:19:43 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9O2NaEZENXOsQ5ARurVy9Q: removed DONE job 23 Nov, 13:19:43 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9OYsYtPw4B6qBu-N3G4cNw: removed DONE job 23 Nov, 13:19:43 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9OsqPkEq2A0BjaxsS83DxQ: removed DONE job 23 Nov, 13:19:43 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9OkIA_HWdJYQ2jhWtzN2qQ: removed DONE job 23 Nov, 13:19:43 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9OJKWi0W9tAqMEvk1KMpXg: removed DONE job 23 Nov, 13:19:43 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9OAG_KMvemu6G6_h1QtsXw: removed DONE job 23 Nov, 13:19:44 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9OoF8o2i76X5cOEWO9_ezw: removed DONE job 23 Nov, 13:19:44 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9OJpHAsHmN68qRMgRR79bQ: removed DONE job 23 Nov, 13:19:44 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9OvoOJROG6hsz-dAJsAGRQ: removed DONE job 23 Nov, 13:19:44 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9OPevUO6LxxqDeoyUjrgjw: removed DONE job 23 Nov, 13:19:44 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9Of9YBbvsamRURogbFCTog: removed DONE job 23 Nov, 13:19:44 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9OWbkU0KtgP1fWYqA2SHuw: removed DONE job 23 Nov, 13:19:44 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9O0DOmLWZKbIix2SZrBQBA: removed DONE job 23 Nov, 13:19:44 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9O46dhW7QO5mihD_3arqUg: removed DONE job Look if one SandBox dir has not been cancelled: [glite@devel19 ~]$ ls -l /var/SandboxDir/9O total 4 drwxrwx--- 5 dteam042 glite 4096 Nov 23 13:19 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OMsNaP2bzC3oiToyAu5hA ===> === test PASSED === - Purge a job which proxy is already expired [glite@cream-45 ~]$ ls /var/SandboxDir/Z6/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fZ6bSUBxN_5fkPNQNs73Vymog/ input JDLOriginal JDLStarted output peek user.proxy [glite@cream-45 ~]$ openssl x509 -noout -text -in /var/SandboxDir/Z6/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fZ6bSUBxN_5fkPNQNs73Vymog/user.proxy Certificate: Data: Version: 3 (0x2) Serial Number: 1319202904 (0x4ea17058) Signature Algorithm: md5WithRSAEncryption Issuer: C=IT, O=INFN, OU=Personal Certificate, L=Padova, CN=Alessio Gianelle, CN=proxy Validity Not Before: Oct 21 13:10:04 2011 GMT Not After : Oct 21 13:29:04 2011 GMT Subject: C=IT, O=INFN, OU=Personal Certificate, L=Padova, CN=Alessio Gianelle, CN=proxy, CN=proxy Subject Public Key Info: Public Key Algorithm: rsaEncryption RSA Public Key: (512 bit) Modulus (512 bit): 00:c5:b8:6e:a8:21:5d:48:da:23:10:e7:c0:13:3c: 97:a1:df:ef:10:27:30:fa:77:52:5f:1e:52:b8:fc: 74:1b:76:12:6d:63:e4:25:56:3e:6d:eb:61:13:5a: a6:47:0d:bb:13:4e:b5:b2:2a:7f:22:6b:49:f9:a7: b6:2c:b4:65:31 Exponent: 65537 (0x10001) Signature Algorithm: md5WithRSAEncryption a1:82:7a:84:59:92:6b:ba:94:a6:54:a0:17:60:f6:95:31:f8: c0:86:b5:3b:8f:6e:3f:7e:09:80:c4:39:3b:3b:d2:f7:42:fb: f6:a6:fd:67:4b:fd:92:3c:23:46:f6:e1:f4:4d:60:26:71:92: a6:aa:b2:f5:52:eb:25:cd:25:df:11:73:20:ad:c3:79:78:b2: 3f:19:2a:11:50:3b:43:42:9d:33:fc:36:38:19:8b:6c:ee:23: ce:fd:17:82:8f:01:d9:5b:14:55:ad:95:4c:d2:16:59:d5:a0: 22:90:57:25:10:01:71:1d:ce:cf:24:4f:37:8f:65:4a:e1:9a: 9f:50 [glite@cream-45 ~]$ date Fri Oct 21 15:35:19 CEST 2011 [glite@cream-45 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/Z6 21 Oct, 15:35:39 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://cream-45.pd.infn.it:9000/Z6bSUBxN_kPNQNs73Vymog: removed DONE job [ale@cream-12 UI]$ glite-wms-job-status https://cream-45.pd.infn.it:9000/Z6bSUBxN_kPNQNs73Vymog ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://cream-45.pd.infn.it:9000/Z6bSUBxN_kPNQNs73Vymog Current Status: Cleared Status Reason: timed out, resource purge forced Destination: cream-22.pd.infn.it:8443/cream-lsf-creamcert1 Submitted: Fri Oct 21 15:15:05 2011 CEST ========================================================================== The last Clear logged event should be logged with the host proxy (not user proxy): [ale@cream-12 WMS-service]$ glite-wms-job-logging-info --event Clear -v 2 https://cream-45.pd.infn.it:9000/Z6bSUBxN_kPNQNs73Vymog ===================== glite-wms-job-logging-info Success ===================== LOGGING INFORMATION: Printing info for the Job : https://cream-45.pd.infn.it:9000/Z6bSUBxN_kPNQNs73Vymog --- Event: Clear - Arrived = Fri Oct 21 15:35:39 2011 CEST - Host = cream-45.pd.infn.it - Reason = 2 - Source = NetworkServer - Src instance = 11882 - Timestamp = Fri Oct 21 15:35:39 2011 CEST - User = /C=IT/O=INFN/OU=Host/L=Padova/CN=cream-45.pd.infn.it/CN=244681262 ========================================================================== ===> === test PASSED === - Purge a dag job [glite@devel19 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/8r 23 Nov, 13:28:40 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:354): https://devel19.cnaf.infn.it:9000/8rdSy5Z2JdTiD4_hKVPoRQ: 25/25 nodes removed 23 Nov, 13:28:40 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:366): https://devel19.cnaf.infn.it:9000/8rdSy5Z2JdTiD4_hKVPoRQ: removed ABORTED dag Check SBD of some nodes: [glite@devel19 ~]$ ls -l /var/SandboxDir/1p/https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f1pW78m9XmY-w-26zFbsoPA ls: /var/SandboxDir/1p/https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f1pW78m9XmY-w-26zFbsoPA: No such file or directory [glite@devel19 ~]$ ls -l /var/SandboxDir/9g/https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9gTb01U4t3xqz0OZYzjw6Q ls: /var/SandboxDir/9g/https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9gTb01U4t3xqz0OZYzjw6Q: No such file or directory Check final status: [ale@cream-12 UI]$ glite-wms-job-status -v 0 https://devel19.cnaf.infn.it:9000/8rdSy5Z2JdTiD4_hKVPoRQ ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://devel19.cnaf.infn.it:9000/8rdSy5Z2JdTiD4_hKVPoRQ Current Status: Aborted - Nodes information for: Status info for the Job : https://devel19.cnaf.infn.it:9000/1543Cn1u5bYvdkZhto1W_g Current Status: Cleared Status info for the Job : https://devel19.cnaf.infn.it:9000/1pW78m9XmY-w-26zFbsoPA Current Status: Aborted Status info for the Job : https://devel19.cnaf.infn.it:9000/9gTb01U4t3xqz0OZYzjw6Q Current Status: Cleared Status info for the Job : https://devel19.cnaf.infn.it:9000/A7ElSgilrBeJG9GOyJfgEw Current Status: Cleared Status info for the Job : https://devel19.cnaf.infn.it:9000/Azs0x-XOpjKeH4qr6r8Rpw Current Status: Cleared Status info for the Job : https://devel19.cnaf.infn.it:9000/D5QhuhNEAN-NO_cvqDJq1g Current Status: Cleared Status info for the Job : https://devel19.cnaf.infn.it:9000/DON4ZQ1vyc3G8bfeXvi-yg Current Status: Cleared Status info for the Job : https://devel19.cnaf.infn.it:9000/H2CYH-muEWP7r6LQ8b2oLA Current Status: Cleared Status info for the Job : https://devel19.cnaf.infn.it:9000/M2byYUQLbZcOQ_7JOrkoPA Current Status: Cleared Status info for the Job : https://devel19.cnaf.infn.it:9000/R2NSChOyvZW4fegwaMGKwQ Current Status: Cleared Status info for the Job : https://devel19.cnaf.infn.it:9000/RECPyDf9WM1XWhk4b2d9-A Current Status: Cleared Status info for the Job : https://devel19.cnaf.infn.it:9000/SdbGoobqGLxTXoBkizcVQg Current Status: Aborted Status info for the Job : https://devel19.cnaf.infn.it:9000/WvPCqVsDgPHYicPOkBjgCQ Current Status: Aborted Status info for the Job : https://devel19.cnaf.infn.it:9000/by-jOy0FVhAfJrbpIRNvhA Current Status: Cleared Status info for the Job : https://devel19.cnaf.infn.it:9000/c47GgReIgki2uz9PDEHfMg Current Status: Cleared Status info for the Job : https://devel19.cnaf.infn.it:9000/dPva2oOwQHINZ6KFyOqYMw Current Status: Cleared Status info for the Job : https://devel19.cnaf.infn.it:9000/esNNVXHVT0smIq6eg8ko0A Current Status: Aborted Status info for the Job : https://devel19.cnaf.infn.it:9000/gaPt3wsKBnMCuW5C0rFUVg Current Status: Cleared Status info for the Job : https://devel19.cnaf.infn.it:9000/iqRZmlOIyaDP-sZ6pTyOnQ Current Status: Cleared Status info for the Job : https://devel19.cnaf.infn.it:9000/jHTFVQE8IU_tamz1S3DD-A Current Status: Aborted Status info for the Job : https://devel19.cnaf.infn.it:9000/jchwYeMEgSVaEosWk9N50Q Current Status: Cleared Status info for the Job : https://devel19.cnaf.infn.it:9000/k52iK4koVGkqOH2WY-EANw Current Status: Cleared Status info for the Job : https://devel19.cnaf.infn.it:9000/kCw2bybHh6vzdla7UuqBMQ Current Status: Cleared Status info for the Job : https://devel19.cnaf.infn.it:9000/lr4WSE3bZ4EITw0XnKdksA Current Status: Cleared Status info for the Job : https://devel19.cnaf.infn.it:9000/nAiA3XlVyjLTL43c9q5HIA Current Status: Cleared ========================================================================== ===> === test PASSED === </verbatim> ---++ Published information ---+++ bdb database definitions ldapsearch -x -H ldap://devel19.cnaf.infn.it:2170 -b 'o=grid' <verbatim> # extended LDIF # # LDAPv3 # base <o=grid> with scope subtree # filter: (objectclass=*) # requesting: ALL # # grid dn: o=grid objectClass: organization o: grid # local, grid dn: Mds-Vo-name=local,o=grid Mds-Vo-name: local objectClass: Mds # resource, grid dn: Mds-Vo-name=resource,o=grid Mds-Vo-name: resource objectClass: Mds # devel19.cnaf.infn.it_org.glite.wms.WMProxy_841415637, resource, grid dn: GlueServiceUniqueID=devel19.cnaf.infn.it_org.glite.wms.WMProxy_841415637,M ds-Vo-name=resource,o=grid GlueServiceAccessControlBaseRule: VO:atlas GlueServiceAccessControlBaseRule: VO:alice GlueServiceAccessControlBaseRule: VO:lhcb GlueServiceAccessControlBaseRule: VO:cms GlueServiceAccessControlBaseRule: VO:dteam GlueServiceAccessControlBaseRule: VO:biomed GlueServiceAccessControlBaseRule: VO:ops GlueServiceAccessControlBaseRule: VO:infngrid GlueServiceAccessControlBaseRule: VO:testers.eu-emi.eu GlueServiceStatus: Other GlueServiceStatusInfo: ?? objectClass: GlueTop objectClass: GlueService objectClass: GlueKey objectClass: GlueSchemaVersion GlueServiceUniqueID: devel19.cnaf.infn.it_org.glite.wms.WMProxy_841415637 GlueServiceAccessControlRule: atlas GlueServiceAccessControlRule: alice GlueServiceAccessControlRule: lhcb GlueServiceAccessControlRule: cms GlueServiceAccessControlRule: dteam GlueServiceAccessControlRule: biomed GlueServiceAccessControlRule: ops GlueServiceAccessControlRule: infngrid GlueServiceAccessControlRule: testers.eu-emi.eu GlueServiceEndpoint: https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_serve r GlueServiceVersion: 3.3.4 GlueSchemaVersionMinor: 3 GlueServiceName: INFN-CREAMTEST-WMProxy GlueServiceType: org.glite.wms.WMProxy GlueServiceWSDL: http://web.infn.it/gLiteWMS/images/WMS/Docs/wmproxy.wsdl GlueServiceSemantics: http://web.infn.it/gLiteWMS/images/WMS/Docs/wmproxy-guid e.pdf GlueForeignKey: GlueSiteUniqueID=INFN-CREAMTEST GlueServiceOwner: atlas GlueServiceOwner: alice GlueServiceOwner: lhcb GlueServiceOwner: cms GlueServiceOwner: dteam GlueServiceOwner: biomed GlueServiceOwner: ops GlueServiceOwner: infngrid GlueServiceOwner: testers.eu-emi.eu GlueSchemaVersionMajor: 1 GlueServiceStartTime: 2011-11-30T09:28:22+01:00 # DN, devel19.cnaf.infn.it_org.glite.wms.WMProxy_841415637, resource, grid dn: GlueServiceDataKey=DN,GlueServiceUniqueID=devel19.cnaf.infn.it_org.glite.w ms.WMProxy_841415637,Mds-Vo-name=resource,o=grid GlueSchemaVersionMajor: 1 objectClass: GlueTop objectClass: GlueServiceData objectClass: GlueKey objectClass: GlueSchemaVersion GlueSchemaVersionMinor: 3 GlueServiceDataKey: DN GlueChunkKey: GlueServiceUniqueID=devel19.cnaf.infn.it_org.glite.wms.WMProxy_8 41415637 GlueServiceDataValue: /C=IT/O=INFN/OU=Host/L=CNAF/CN=devel19.cnaf.infn.it # glite-info-service_version, devel19.cnaf.infn.it_org.glite.wms.WMProxy_8414 15637, resource, grid dn: GlueServiceDataKey=glite-info-service_version,GlueServiceUniqueID=devel19. cnaf.infn.it_org.glite.wms.WMProxy_841415637,Mds-Vo-name=resource,o=grid GlueSchemaVersionMajor: 1 objectClass: GlueTop objectClass: GlueServiceData objectClass: GlueKey objectClass: GlueSchemaVersion GlueSchemaVersionMinor: 3 GlueServiceDataKey: glite-info-service_version GlueChunkKey: GlueServiceUniqueID=devel19.cnaf.infn.it_org.glite.wms.WMProxy_8 41415637 GlueServiceDataValue: 1.6 # glite-info-service_hostname, devel19.cnaf.infn.it_org.glite.wms.WMProxy_841 415637, resource, grid dn: GlueServiceDataKey=glite-info-service_hostname,GlueServiceUniqueID=devel19 .cnaf.infn.it_org.glite.wms.WMProxy_841415637,Mds-Vo-name=resource,o=grid GlueSchemaVersionMajor: 1 objectClass: GlueTop objectClass: GlueServiceData objectClass: GlueKey objectClass: GlueSchemaVersion GlueSchemaVersionMinor: 3 GlueServiceDataKey: glite-info-service_hostname GlueChunkKey: GlueServiceUniqueID=devel19.cnaf.infn.it_org.glite.wms.WMProxy_8 41415637 GlueServiceDataValue: devel19.cnaf.infn.it # search result search: 2 result: 0 Success # numResponses: 8 # numEntries: 7 </verbatim> ---+++ Stats database definitions ldapsearch -x -H ldap://devel19.cnaf.infn.it:2170 -b 'o=infosys' <verbatim> # extended LDIF # # LDAPv3 # base <o=infosys> with scope subtree # filter: (objectclass=*) # requesting: ALL # # infosys dn: o=infosys objectClass: organization o: infosys # zip, infosys dn: CompressionType=zip,o=infosys objectClass: CompressedContent Hostname: localhost CompressionType: zip Data: file:///var/run/bdii/archive/11-11-30-11-27-58-snapshot.gz # localhost, infosys dn: Hostname=localhost,o=infosys objectClass: UpdateStats Hostname: localhost FailedDeletes: 0 ModifiedEntries: 5 DeletedEntries: 0 UpdateTime: 0 FailedAdds: 0 FailedModifies: 0 TotalEntries: 14 QueryTime: 0 NewEntries: 1 DBUpdateTime: 0 ReadTime: 0 PluginsTime: 0 ProvidersTime: 0 # search result search: 2 result: 0 Success # numResponses: 4 # numEntries: 3 </verbatim> ---+++ Stats database definitions ldapsearch -x -H ldap://devel19.cnaf.infn.it:2170 -b 'o=glue' <verbatim> # extended LDIF # # LDAPv3 # base <o=glue> with scope subtree # filter: (objectclass=*) # requesting: ALL # # glue dn: o=glue objectClass: organization o: glue # grid, glue dn: GLUE2GroupID=grid,o=glue objectClass: GLUE2Group GLUE2GroupID: grid # resource, glue dn: GLUE2GroupID=resource,o=glue objectClass: GLUE2Group GLUE2GroupID: resource # devel19.cnaf.infn.it_wmproxy_2598325561, resource, glue dn: GLUE2ServiceID=devel19.cnaf.infn.it_wmproxy_2598325561,GLUE2GroupID=resour ce,o=glue GLUE2EntityValidity: 3600 GLUE2ServiceType: org.glite.wms.WMProxy GLUE2ServiceID: devel19.cnaf.infn.it_wmproxy_2598325561 objectClass: GLUE2Service GLUE2ServiceQualityLevel: production GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-service GLUE2EntityOtherInfo: InfoProviderVersion=1.1 GLUE2EntityOtherInfo: InfoProviderHost=devel19.cnaf.infn.it GLUE2ServiceComplexity: endpointType=1, share=0, resource=0 GLUE2ServiceCapability: executionmanagement.candidatesetgenerator GLUE2ServiceCapability: executionmanagement.jobdescription GLUE2ServiceCapability: executionmanagement.jobmanager GLUE2EntityName: INFN-CREAMTEST-WMProxy GLUE2ServiceAdminDomainForeignKey: INFN-CREAMTEST GLUE2EntityCreationTime: 2011-11-30T10:29:59Z # devel19.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMProxy_2598325561, d evel19.cnaf.infn.it_wmproxy_2598325561, resource, glue dn: GLUE2EndpointID=devel19.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMPr oxy_2598325561,GLUE2ServiceID=devel19.cnaf.infn.it_wmproxy_2598325561,GLUE2Gr oupID=resource,o=glue GLUE2EntityValidity: 3600 GLUE2EndpointQualityLevel: production GLUE2EndpointImplementor: gLite GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-endpoint GLUE2EntityOtherInfo: InfoProviderVersion=1.2 GLUE2EntityOtherInfo: InfoProviderHost=devel19.cnaf.infn.it GLUE2EntityOtherInfo: MiddlewareName=EMI GLUE2EntityOtherInfo: MiddlewareVersion=1.5.0-1 GLUE2EndpointCapability: executionmanagement.candidatesetgenerator GLUE2EndpointCapability: executionmanagement.jobdescription GLUE2EndpointCapability: executionmanagement.jobmanager GLUE2EndpointServiceForeignKey: devel19.cnaf.infn.it_wmproxy_2598325561 GLUE2EntityName: WMProxy endpoint for Service devel19.cnaf.infn.it_wmproxy_259 8325561 GLUE2EndpointTechnology: webservice GLUE2EndpointWSDL: http://web.infn.it/gLiteWMS/images/WMS/Docs/wmproxy.wsdl GLUE2EndpointInterfaceName: org.glite.wms.WMProxy GLUE2EndpointURL: https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server GLUE2EndpointDowntimeInfo: See the GOC DB for downtimes: https://goc.egi.eu/ GLUE2EndpointImplementationVersion: 3.3.4 GLUE2EndpointSemantics: http://web.infn.it/gLiteWMS/images/WMS/Docs/wmproxy-gu ide.pdf GLUE2EndpointIssuerCA: /C=IT/O=INFN/CN=INFN CA GLUE2EndpointServingState: production objectClass: GLUE2Endpoint GLUE2EndpointInterfaceVersion: 1.0 GLUE2EndpointImplementationName: WMS GLUE2EndpointTrustedCA: IGTF GLUE2EndpointID: devel19.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMProxy _2598325561 GLUE2EndpointHealthState: ok GLUE2EndpointStartTime: 2011-11-30T08:28:22Z GLUE2EndpointHealthStateInfo: WMProxy httpd listening on port 7443 httpd is ru nning .... === WMProxy Server running instances: UID PID PPID C STIME TTY TIM E CMD glite 29819 29794 0 09:28 ? 00:00:01 /usr/bin/glite_wms_wmproxy_server GLUE2EntityCreationTime: 2011-11-30T10:29:59Z # DN_1, devel19.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMProxy_2598325 561, devel19.cnaf.infn.it_wmproxy_2598325561, resource, glue dn: GLUE2ExtensionLocalID=DN_1,GLUE2EndpointID=devel19.cnaf.infn.it_wmproxy_25 98325561_org.glite.wms.WMProxy_2598325561,GLUE2ServiceID=devel19.cnaf.infn.it _wmproxy_2598325561,GLUE2GroupID=resource,o=glue GLUE2ExtensionLocalID: DN_1 GLUE2ExtensionKey: DN objectClass: GLUE2Extension GLUE2ExtensionValue: /C=IT/O=INFN/OU=Host/L=CNAF/CN=devel19.cnaf.infn.it GLUE2ExtensionEntityForeignKey: devel19.cnaf.infn.it_wmproxy_2598325561_org.gl ite.wms.WMProxy_2598325561 # devel19.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMProxy_2598325561_Po licy, devel19.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMProxy_25983255 61, devel19.cnaf.infn.it_wmproxy_2598325561, resource, glue dn: GLUE2PolicyID=devel19.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMProx y_2598325561_Policy,GLUE2EndpointID=devel19.cnaf.infn.it_wmproxy_2598325561_o rg.glite.wms.WMProxy_2598325561,GLUE2ServiceID=devel19.cnaf.infn.it_wmproxy_2 598325561,GLUE2GroupID=resource,o=glue GLUE2AccessPolicyEndpointForeignKey: devel19.cnaf.infn.it_wmproxy_2598325561_o rg.glite.wms.WMProxy_2598325561 GLUE2EntityValidity: 3600 objectClass: GLUE2Policy objectClass: GLUE2AccessPolicy GLUE2PolicyRule: VO:atlas GLUE2PolicyRule: VO:alice GLUE2PolicyRule: VO:lhcb GLUE2PolicyRule: VO:cms GLUE2PolicyRule: VO:dteam GLUE2PolicyRule: VO:biomed GLUE2PolicyRule: VO:ops GLUE2PolicyRule: VO:infngrid GLUE2PolicyRule: VO:testers.eu-emi.eu GLUE2PolicyScheme: org.glite.standard GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-endpoint GLUE2EntityOtherInfo: InfoProviderVersion=1.2 GLUE2EntityOtherInfo: InfoProviderHost=devel19.cnaf.infn.it GLUE2PolicyUserDomainForeignKey: atlas GLUE2PolicyUserDomainForeignKey: alice GLUE2PolicyUserDomainForeignKey: lhcb GLUE2PolicyUserDomainForeignKey: cms GLUE2PolicyUserDomainForeignKey: dteam GLUE2PolicyUserDomainForeignKey: biomed GLUE2PolicyUserDomainForeignKey: ops GLUE2PolicyUserDomainForeignKey: infngrid GLUE2PolicyUserDomainForeignKey: testers.eu-emi.eu GLUE2PolicyID: devel19.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMProxy_2 598325561_Policy GLUE2EntityName: Access control rules for Endpoint devel19.cnaf.infn.it_wmprox y_2598325561_org.glite.wms.WMProxy_2598325561 GLUE2EntityCreationTime: 2011-11-30T10:29:59Z # search result search: 2 result: 0 Success # numResponses: 8 # numEntries: 7 </verbatim> ---++ Performance test ---+++ Collection of 1000 nodes * Description: submission of a collection with 1000 nodes * Collection ID: https://devel19.cnaf.infn.it:9000/EFsebwUWZUWMYi0GgFrntA * Final Status: * 1000 Done (Success) ===> test PASSED ---+++ Stress test ---++++ Description * 2880 collection of 25 nodes each (total: 72000 jobs) * One collection every 60 seconds * two users * Use LCG-CEs and CREAM-CEs (with different batch systems) * Use automatic-delegation * The job is a "sleep random(666)" * Resubmission is enabled * Enable proxy renewal ---++++ Results * 2880 collections submitted in 3200 seconds: 0/1/13 (min/avg/max) * Jobs correctly submitted: 72000 (85.4% to CREAM and 14.6% to LCG ces) * DONE OK: 71600 (99.44%) * Aborted: 400 (0.56%) (Proxy expired) ===> test PASSED ---++ Regression Tests ---+++ [[https://savannah.cern.ch/bugs/?81376][Bug #81376]] (glite-wms-wmproxy.restart.cron MUST support graceful wmp restart) %GREEN% FIXED %ENDCOLOR% <verbatim> [ale@cream-12 regression_tests]$ python execute_regTest.py -d 3 -b 81376 -l +++++++++++++++++++++++++++++++++++++++++++++++++++++ + Regression TestSuite .... + +++++++++++++++++++++++++++++++++++++++++++++++++++++ Testing bug 81376 All tests PASSED </verbatim> See [[%ATTACHURL%/81376.log][log]] file ---+++ [[https://savannah.cern.ch/bugs/?81568][Bug #81568]] (some inconsistencies in locations for logs and configuration) %GREEN% FIXED %ENDCOLOR% <verbatim> [ale@cream-12 regression_tests]$ python execute_regTest.py -d 3 -b 81568 -l +++++++++++++++++++++++++++++++++++++++++++++++++++++ + Regression TestSuite .... + +++++++++++++++++++++++++++++++++++++++++++++++++++++ Testing bug 81568 All tests PASSED </verbatim> See [[%ATTACHURL%/81568.log][log]] file Verified that all WMS log files are in this directory: <verbatim> [root@cream-44 ~]# ls $WMS_LOCATION_LOG glite-lb-lcas.log glite-lb-lcas.log.4.gz glite-wms-wmproxy-purge-proxycache.log ice.log wmproxy.log glite-lb-lcas.log.1.gz glite-lb-purger.log glite-wms-wmproxy.restart.cron.log jobcontoller_events.log wmproxy_logrotate.log glite-lb-lcas.log.2.gz glite-lb-purger.log.1.gz httpd-wmproxy-access.log lcmaps.log workload_manager_events.log glite-lb-lcas.log.3.gz glite-wms-purgeStorage.log httpd-wmproxy-errors.log logmonitor_events.log </verbatim> ---+++ [[https://savannah.cern.ch/bugs/?83062][Bug #83062]] (A different template should be used for glue2 publication) %GREEN% FIXED %ENDCOLOR% <verbatim> [ale@cream-12 regression_tests]$ python execute_regTest.py -d 3 -b 83062 -l +++++++++++++++++++++++++++++++++++++++++++++++++++++ + Regression TestSuite .... + +++++++++++++++++++++++++++++++++++++++++++++++++++++ Testing bug 83062 All tests PASSED </verbatim> See [[%ATTACHURL%/83062.log][log]] file ---+++ [[https://savannah.cern.ch/bugs/?85327][Bug #85327]] (glite-wms-wmproxy.restart.cron missing HOSTNAME environment variable) %GREEN% FIXED %ENDCOLOR% <verbatim> [ale@cream-12 regression_tests]$ python execute_regTest.py -d 3 -b 85327 -l +++++++++++++++++++++++++++++++++++++++++++++++++++++ + Regression TestSuite .... + +++++++++++++++++++++++++++++++++++++++++++++++++++++ Testing bug 85327 All tests PASSED </verbatim> See [[%ATTACHURL%/85327.log][log]] file ---+++ [[https://savannah.cern.ch/bugs/?86485][Bug #86485]] (ICE doesn't send the iceId to CREAM) %GREEN% FIXED %ENDCOLOR% Submitted a job through the WMS to a CREAM CE in debug mode. Verified that in the CREAM log file the iceid is reported: <verbatim> # grep selectToRetrieveJobStatusQuery /var/log/cream/glite-ce-cream.log 20 Oct 2011 07:11:43,186 DEBUG org.glite.ce.cream.jobmanagement.db.table.JobStatusTable (JobStatusTable.java:419) - (TP-Processor25) selectToRetrieveJobStatusQuery = select job_status.type AS type,job_status.exitCode AS exitCode,job_status.failureReason AS failureReason,job_status.description AS description,job_status.time_stamp AS time_stamp,job_status.jobId AS jobId,job_status.id AS id,job.gridJobId AS gridJobId,job.workerNode AS workerNode from job_status, job where job_status.jobId = job.id and job_status.type IN ( '10', '5', '8', '7', '6', '4', '3') and job.userId = '_C_IT_O_INFN_OU_Personal_Certificate_L_Padova_CN_Massimo_Sgaravatto_dteam_Role_NULL_Capability_NULL' and job.iceId = 'C_IT_O_INFN_OU_Host_L_Padova_CN_cream-44.pd.infn.it' and job_status.id >= 0 and time_stamp >= '2011-10-19 15:46:36.0' ORDER BY job_status.id limit 500 </verbatim> Verified also that in the CREAMDB the iceid is set for this job: <verbatim> mysql> use creamdb; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> select iceId from job where id="CREAM806900601" -> ; +-----------------------------------------------------+ | iceId | +-----------------------------------------------------+ | C_IT_O_INFN_OU_Host_L_Padova_CN_cream-44.pd.infn.it | +-----------------------------------------------------+ 1 row in set (0.00 sec) </verbatim> ===> test PASSED ---+++ [[https://savannah.cern.ch/bugs/?86490][Bug #86490]] (EMI WMS stops accepting jobs after 31999th submission for the same DN (on ext3)) %GREEN% FIXED %ENDCOLOR% <verbatim> [ale@cream-12 regression_tests]$ python execute_regTest.py -d 3 -b 86490 -l +++++++++++++++++++++++++++++++++++++++++++++++++++++ + Regression TestSuite .... + +++++++++++++++++++++++++++++++++++++++++++++++++++++ Testing bug 86490 All tests PASSED </verbatim> See [[%ATTACHURL%/86490.log][log]] file ---+++ [[https://savannah.cern.ch/bugs/?86682][Bug #86682]] ( yaim changes ownership of /var) %GREEN% FIXED %ENDCOLOR% <verbatim> [ale@cream-12 regression_tests]$ python execute_regTest.py -d 3 -b 86682 -l +++++++++++++++++++++++++++++++++++++++++++++++++++++ + Regression TestSuite .... + +++++++++++++++++++++++++++++++++++++++++++++++++++++ Testing bug 86682 All tests PASSED </verbatim> See [[%ATTACHURL%/86682.log][log]] file ---+++ [[https://savannah.cern.ch/bugs/?86769][Bug #86769]] (EMI WMS Update 6: WMS 'hangs' after several hours of operation) %GREEN% FIX NOT CERTIFIED %ENDCOLOR% It is not easy to verify this bug, but the WMS has passed some stress test of one or two days of submissions. It also has passed a test done by the user who posted the bug. ---+++ [[https://savannah.cern.ch/bugs/?86956][Bug #86956]] (Match-matching of requests with access to catalogues can cause the WM to crash) %GREEN% FIX NOT CERTIFIED %ENDCOLOR% To use the function specified in the bug we need a jdl with DataRequirements. In any case the error is not deterministic, so you can only set this bug as hopefully fixed. <verbatim> [ale@cream-12 DataReq]$ cat data-req.jdl ########################################### # JDL with Data Requirements # ########################################### Executable = "/bin/ls"; Arguments = "-la"; StdOutput = "std.out"; StdError = "std.err"; Prologue = "prologue.sh"; FuzzyRank = true; Environment={"LFC_HOST=prod-lfc-shared-central.cern.ch", "LFC_HOME=prod-lfc-shared-central.cern.ch:/grid/dteam/aleph"}; InputSandbox = {"prologue.sh"}; OutputSandbox = {"std.out", "std.err","prologue.out", "file.txt"}; requirements = true; DataRequirements = { [ DataCatalogType = "DLI"; DataCatalog ="http://prod-lfc-shared-central.cern.ch:8085/"; InputData = { "guid:17c667b7-9622-4c95-be68-3d15d1eacadb" }; ] }; DataAccessProtocol = "gsiftp"; AllowZippedISB=true; RetryCount = 1; ShallowRetryCount = 2; [ale@cream-12 DataReq]$ glite-wms-job-list-match -a -c /home/ale/UI/etc/wmp_devel19.conf data-req.jdl Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server ========================================================================== COMPUTING ELEMENT IDs LIST The following CE(s) matching your job requirements have been found: *CEId* - cert-15.pd.infn.it:8443/cream-lsf-cert - cert-37.pd.infn.it:8443/cream-lsf-cert - cream-19.pd.infn.it:8443/cream-lsf-cert - cream-19.pd.infn.it:8443/cream-lsf-creamcert1 - cream-19.pd.infn.it:8443/cream-lsf-creamcert2 - cream-26.pd.infn.it:2119/jobmanager-lcgpbs-cert - cream-26.pd.infn.it:2119/jobmanager-lcgpbs-creamtest1 - cream-26.pd.infn.it:2119/jobmanager-lcgpbs-creamtest2 - cream-31.pd.infn.it:8443/cream-pbs-creamtest1 - cream-31.pd.infn.it:8443/cream-pbs-creamtest2 - cream-38.pd.infn.it:8443/cream-pbs-creamtest1 - cream-38.pd.infn.it:8443/cream-pbs-creamtest2 - prod-ce-01.pd.infn.it:8443/cream-lsf-cert - prod-ce-02.pd.infn.it:2119/jobmanager-lcglsf-cert - cream-24.pd.infn.it:8443/cream-lsf-cert - cream-24.pd.infn.it:8443/cream-lsf-testbedB_1 - cream-24.pd.infn.it:8443/cream-lsf-testbedB_2 - cream-41.pd.infn.it:8443/cream-pbs-creamtest1 - cream-32.pd.infn.it:8443/cream-pbs-cert - cream-32.pd.infn.it:8443/cream-pbs-creamtest1 - cream-32.pd.infn.it:8443/cream-pbs-creamtest2 - ce130.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam - ce130.cern.ch:2119/jobmanager-lcglsf-grid_dteam - ce131.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam - ce131.cern.ch:2119/jobmanager-lcglsf-grid_dteam - ce132.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam - ce132.cern.ch:2119/jobmanager-lcglsf-grid_dteam - ce133.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam - ce133.cern.ch:2119/jobmanager-lcglsf-grid_dteam - ce203.cern.ch:8443/cream-lsf-grid_2nh_dteam - ce203.cern.ch:8443/cream-lsf-grid_dteam - ce204.cern.ch:8443/cream-lsf-grid_2nh_dteam - ce204.cern.ch:8443/cream-lsf-grid_dteam - ce205.cern.ch:8443/cream-lsf-grid_2nh_dteam - ce205.cern.ch:8443/cream-lsf-grid_dteam - ce206.cern.ch:8443/cream-lsf-grid_2nh_dteam - ce206.cern.ch:8443/cream-lsf-grid_dteam - ce207.cern.ch:8443/cream-lsf-grid_2nh_dteam - ce207.cern.ch:8443/cream-lsf-grid_dteam - ce208.cern.ch:8443/cream-lsf-grid_2nh_dteam - ce208.cern.ch:8443/cream-lsf-grid_dteam - cream-11.pd.infn.it:2119/jobmanager-lcglsf-cert - cream-11.pd.infn.it:2119/jobmanager-lcglsf-creamtest1 - cream-11.pd.infn.it:2119/jobmanager-lcglsf-creamtest2 - cream-20.pd.infn.it:8443/cream-lsf-cert - cream-20.pd.infn.it:8443/cream-lsf-creamcert1 - cream-20.pd.infn.it:8443/cream-lsf-creamcert2 - cream-31.pd.infn.it:8443/cream-pbs-cert - cream-41.pd.infn.it:8443/cream-pbs-cert - cream-41.pd.infn.it:8443/cream-pbs-creamtest2 ========================================================================== </verbatim> ---+++ [[https://savannah.cern.ch/bugs/?87259][Bug #87259]] (WMS purger errors in jobcontroller/logmonitor) %GREEN% FIXED %ENDCOLOR% For a job submitted to a LCG-CE, verified that in =/var/log/glite/logmonitor_events.log= the removing of job directory, submit file, classad file, and wrapper file is logged: <verbatim> 23 Nov, 15:27:44 -I- EventTerminated::process_event(): Got job terminated event for cluster 183 23 Nov, 15:27:44 -I- EventTerminated::processNormalJob(...): Job id = https://cream-44.pd.infn.it:9000/tQFtQrDkWGls48jUFr0nQw 23 Nov, 15:27:44 -I- EventTerminated::processNormalJob(...): Return value = 0 23 Nov, 15:27:44 -H- JobWrapperOutputParser::parse_file(...): Going to parse standard output file. 23 Nov, 15:27:44 -I- EventTerminated::processNormalJob(...): Real return code: 0 23 Nov, 15:27:44 -V- ProxyUnregistrar::unregister(): Unregistering user proxy... 23 Nov, 15:27:44 -I- JobFilePurger::do_purge(...): Removing job directory: /var/jobcontrol/condorio/tQ/https_3a_2f_2fcream-44.pd.infn.it_3a9000_2ftQFtQrDkWGls48jUFr0nQw 23 Nov, 15:27:44 -U- JobFilePurger::do_purge(...): Removed 3 files. 23 Nov, 15:27:44 -I- JobFilePurger::do_purge(...): Removing submit file: /var/jobcontrol/submit/tQ/Condor.https_3a_2f_2fcream-44.pd.infn.it_3a9000_2ftQFtQrDkWGls48jUFr0nQw.submit 23 Nov, 15:27:44 -U- JobFilePurger::do_purge(...): Removed... 23 Nov, 15:27:44 -I- JobFilePurger::do_purge(...): Removing classad file: /var/jobcontrol/submit/tQ/ClassAd.https_3a_2f_2fcream-44.pd.infn.it_3a9000_2ftQFtQrDkWGls48jUFr0nQw 23 Nov, 15:27:44 -U- JobFilePurger::do_purge(...): Removed... 23 Nov, 15:27:44 -I- JobFilePurger::do_purge(...): Removing wrapper file: /var/jobcontrol/submit/tQ/JobWrapper.https_3a_2f_2fcream-44.pd.infn.it_3a9000_2ftQFtQrDkWGls48jUFr0nQw.sh 23 Nov, 15:27:44 -U- JobFilePurger::do_purge(...): Removed... 23 Nov, 15:27:44 -I- MonitorLoop::run(): Spent 0 seconds in the last file loop. 23 Nov, 15:27:44 -I- MonitorLoop::run(): Must wait for other 6 seconds. 23 Nov, 15:27:50 -I- MonitorLoop::run(): No new event found, going to sleep. 23 Nov, 15:27:50 -I- MonitorLoop::run(): Checking each 5 seconds for new events. </verbatim> Verified that they have been actually removed: <verbatim> [root@cream-44 ~]# ls /var/jobcontrol/condorio/tQ/ [root@cream-44 ~]# ls /var/jobcontrol/submit/tQ/ [root@cream-44 ~]# </verbatim> Submitted a job: <verbatim> [ale@cream-12 UI]$ glite-wms-job-submit -a -c /home/ale/UI/etc/wmp_cream-44.conf test.jdl Connecting to the service https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server ====================== glite-wms-job-submit Success ====================== The job has been successfully submitted to the WMProxy Your job identifier is: https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA ========================================================================== [ale@cream-12 UI]$ glite-wms-job-status https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA Current Status: Scheduled Status Reason: unavailable Destination: cream-40.pd.infn.it:8443/cream-pbs-creamtest2 Submitted: Wed Nov 23 15:42:23 2011 CET ========================================================================== </verbatim> Copied the sandbox dir: <verbatim> [root@cream-44 uX]# cd /var/SandboxDir/tY/ [root@cream-44 tY]# ls -l total 4 drwxrwx--- 5 dteam009 glite 4096 Nov 23 15:42 https_3a_2f_2fcream-44.pd.infn.it_3a9000_2ftYM93krTdeUb7BX5qJoLDA [root@cream-44 tY]# cp -r -p https_3a_2f_2fcream-44.pd.infn.it_3a9000_2ftYM93krTdeUb7BX5qJoLDA/ https_3a_2f_2fcream-44.pd.infn.it_3a9000_2ftYM93krTdeUb7BX5qJoLDA.old [root@cream-44 tY]# ls -l total 8 drwxrwx--- 5 dteam009 glite 4096 Nov 23 15:42 https_3a_2f_2fcream-44.pd.infn.it_3a9000_2ftYM93krTdeUb7BX5qJoLDA drwxrwx--- 5 dteam009 glite 4096 Nov 23 15:42 https_3a_2f_2fcream-44.pd.infn.it_3a9000_2ftYM93krTdeUb7BX5qJoLDA.old </verbatim> Retrieved the OSB when the job is done: <verbatim> [ale@cream-12 UI]$ glite-wms-job-status https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA Current Status: Done (Success) Exit code: 0 Status Reason: Job Terminated Successfully Destination: cream-40.pd.infn.it:8443/cream-pbs-creamtest2 Submitted: Wed Nov 23 15:42:23 2011 CET ========================================================================== [ale@cream-12 UI]$ glite-wms-job-output https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA Connecting to the service https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server ================================================================================ JOB GET OUTPUT OUTCOME Output sandbox files for the job: https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA have been successfully retrieved and stored in the directory: /tmp/jobOutput/ale_tYM93krTdeUb7BX5qJoLDA ================================================================================ [ale@cream-12 UI]$ cat /tmp/jobOutput/ale_tYM93krTdeUb7BX5qJoLDA/out.log total 12 drwxr-xr-x 2 dteam009 dteam 4096 Nov 23 15:42 . drwxr-xr-x 3 dteam009 dteam 4096 Nov 23 15:42 .. -rw-r--r-- 1 dteam009 dteam 367 Nov 23 15:42 .BrokerInfo -rw-r--r-- 1 dteam009 dteam 0 Nov 23 15:42 out.log -rw-r--r-- 1 dteam009 dteam 0 Nov 23 15:42 .tmp_file [ale@cream-12 UI]$ glite-wms-job-status https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA Current Status: Cleared Status Reason: user retrieved output sandbox Destination: cream-40.pd.infn.it:8443/cream-pbs-creamtest2 Submitted: Wed Nov 23 15:42:23 2011 CET ========================================================================== </verbatim> Restored the sandbox dir in the WMS: <verbatim> [root@cream-44 tY]# mv https_3a_2f_2fcream-44.pd.infn.it_3a9000_2ftYM93krTdeUb7BX5qJoLDA.old/ https_3a_2f_2fcream-44.pd.infn.it_3a9000_2ftYM93krTdeUb7BX5qJoLDA [root@cream-44 tY]# ls -l total 4 drwxrwx--- 5 dteam009 glite 4096 Nov 23 15:42 https_3a_2f_2fcream-44.pd.infn.it_3a9000_2ftYM93krTdeUb7BX5qJoLDA </verbatim> Purged the job from LBProxy and from LBServer: <verbatim> [root@cream-44 ~]# echo "https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA" > jobid.txt [root@cream-44 ~]# /usr/bin/glite-lb-purge -m cream-44.pd.infn.it -x -j jobid.txt The jobs were not dumped. [root@cream-44 ~]# /usr/bin/glite-lb-purge -m cream-44.pd.infn.it -j jobid.txt The jobs were not dumped. </verbatim> Verified that job status returns =Identifier removed= (i.e. error =EIDRM=): <verbatim> [ale@cream-12 UI]$ glite-wms-job-status https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA **** Error: API_NATIVE_ERROR **** Error while calling the "Job:getStatus" native api Unable to retrieve the status for: https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA glite.lb.Exception: edg_wll_JobStatus: Identifier removed: matching job already purged at glite::lb::Job::status[./src/Job.cpp:87] </verbatim> On the WMS, purged the sandboxdir, as user glite, invoking the command =glite-wms-purgeStorage.sh= and verifying that =Identifier removed= (i.e. error 43) is returned: <verbatim> [glite@cream-44 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/tY 23 Nov, 15:49:35 -E: [Error] query_job_status(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:134): https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA: edg_wll_JobStat [43] Identifier removed(matching job already purged) 23 Nov, 15:49:35 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:306): https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA: forced removal, unknown/removed L&B job </verbatim> Verified that the sandboxdir has been removed: <verbatim> [root@cream-44 ~]# ls -l /var/SandboxDir/tY total 0 [root@cream-44 ~]# </verbatim> ===> test PASSED ---+++ [[https://savannah.cern.ch/bugs/?87994][Bug #87994]] yaim-wms doesn't fill the attributes get_acbr and get_owner in the conf file for Glue2 %GREEN% FIXED %ENDCOLOR% Check the attributes get_acbr and get_owner in /etc/glite/info/service/glite-info-glue2-wmproxy.conf <verbatim> [root@cream-44 ~]# grep get_acbr /etc/glite/info/service/glite-info-glue2-wmproxy.conf get_acbr = echo -e "VO:alice\nVO:atlas\nVO:cms\nVO:lhcb\nVO:dteam\nVO:testers.eu-emi.eu" [root@cream-44 ~]# grep get_owner /etc/glite/info/service/glite-info-glue2-wmproxy.conf get_owner = echo -e "alice\natlas\ncms\nlhcb\ndteam\ntesters.eu-emi.eu" </verbatim> ===> test PASSED ---+++ [[https://savannah.cern.ch/bugs/?88072][Bug #88072]] EMI-1 WMS cannot submit to ARC CEs %GREEN% FIXED %ENDCOLOR% <verbatim> -bash-3.2$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/pl6ZSNy7MNWNSH6x8650DQ ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://devel19.cnaf.infn.it:9000/pl6ZSNy7MNWNSH6x8650DQ Current Status: Done (Success) Exit code: 0 Status Reason: Job terminated successfully Destination: jade-cms.hip.fi:2811/nordugrid-GE-arc Submitted: Mon Nov 21 16:01:15 2011 CET ========================================================================== -bash-3.2$ glite-wms-job-output https://devel19.cnaf.infn.it:9000/pl6ZSNy7MNWNSH6x8650DQ Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server ================================================================================ JOB GET OUTPUT OUTCOME Output sandbox files for the job: https://devel19.cnaf.infn.it:9000/pl6ZSNy7MNWNSH6x8650DQ have been successfully retrieved and stored in the directory: /tmp/jobOutput/traldi_pl6ZSNy7MNWNSH6x8650DQ ================================================================================ -bash-3.2$ cat /tmp/jobOutput/traldi_pl6ZSNy7MNWNSH6x8650DQ/out.txt total 32 drwx------ 2 cmsuser43 cmsuser 4096 Nov 21 17:02 . drwx------ 3 cmsuser43 cmsuser 4096 Nov 21 17:02 .. -rw------- 1 cmsuser43 cmsuser 118 Nov 21 17:02 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2fpl6ZSNy7MNWNSH6x8650DQ.output -rw-r--r-- 1 cmsuser43 cmsuser 0 Nov 21 17:02 out.txt -rw-r--r-- 1 cmsuser43 cmsuser 1359 Nov 21 17:02 pippo.jdl -rw------- 1 cmsuser43 cmsuser 2 Nov 21 17:02 tmp.bUujW32646 -rw------- 1 cmsuser43 cmsuser 103 Nov 21 17:02 tmp.cNkaG32632 -rw------- 1 cmsuser43 cmsuser 103 Nov 21 17:02 tmp.exhlZ32647 -rw------- 1 cmsuser43 cmsuser 2 Nov 21 17:02 tmp.gZIii32631 -rw------- 1 cmsuser43 cmsuser 0 Nov 21 17:02 tmp.SdwGF32645 [ale@cream-12 WMS-service]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/pl6ZSNy7MNWNSH6x8650DQ ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://devel19.cnaf.infn.it:9000/pl6ZSNy7MNWNSH6x8650DQ Current Status: Cleared Status Reason: user retrieved output sandbox Destination: jade-cms.hip.fi:2811/nordugrid-GE-arc Submitted: Mon Nov 21 16:01:15 2011 CET ========================================================================== </verbatim> ===> test PASSED ---+++ [[https://savannah.cern.ch/bugs/?88096][Bug #88096]] CA certificate verification failed %GREEN% FIXED NOT CERTIFIED %ENDCOLOR% It is not easy to verify this bug, but the WMS has passed some stress test of one or two days of submissions. ---+++ [[https://savannah.cern.ch/bugs/?88097][Bug #88097]] Collection not terminated %GREEN% FIXED NOT CERTIFIED %ENDCOLOR% It is not easy to verify this bug, but the WMS has passed some stress test of one or two days of submissions. ---+++ [[https://savannah.cern.ch/bugs/?88450][Bug #88450]] Ice configuration section doesn't use environment variable %GREEN%FIXED%ENDCOLOR% <verbatim> [root@cream-44 ~]# cat /etc/glite-wms/glite_wms.conf | grep -i " logfile" logfile = "${WMS_LOCATION_LOG}/ice.log"; LogFile = "${WMS_LOCATION_LOG}/wmproxy.log"; LogFile = "${WMS_LOCATION_LOG}/networkserver_events.log"; LogFile = "${WMS_LOCATION_LOG}/workload_manager_events.log"; LogFile = "${WMS_LOCATION_LOG}/logmonitor_events.log"; LogFile = "${WMS_LOCATION_LOG}/jobcontoller_events.log"; </verbatim> ===> test PASSED ---+++ [[https://savannah.cern.ch/bugs/?88558][Bug #88558]] WM creates submit requests with old sequence code upon a replan %GREEN% FIXED %ENDCOLOR% <verbatim> Event: Resubmission - Arrived = Wed Nov 23 19:16:16 2011 CET - Host = devel19.cnaf.infn.it - Level = SYSTEM - Priority = synchronous - Reason = token still exists - Result = SHALLOW - Seqcode = UI=000000:NS=0000000004:WM=000007:BH=0000000000:JSS=000002:LM=000000:LRMS=000000:APP=000000:LBS=000000 - Source = WorkloadManager - Src instance = 13159 - Tag = job will be replanned - Timestamp = Wed Nov 23 19:16:16 2011 CET - User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy [...] Event: EnQueued - Arrived = Wed Nov 23 19:16:16 2011 CET - Host = devel19.cnaf.infn.it - Level = SYSTEM - Priority = synchronous - Queue = /var/workload_manager/jobdir - Result = OK - Seqcode = UI=000000:NS=0000000004:WM=000009:BH=0000000000:JSS=000002:LM=000000:LRMS=000000:APP=000000:LBS=000000 - Source = WorkloadManager - Src instance = 13159 - Timestamp = Wed Nov 23 19:16:16 2011 CET - User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy - Job = [ version = "1.0.0"; arguments = [ ad = [ NodeName = "Node_jdl_953"; FuzzyRank = true; CEInfoHostName = "cream-31.pd.infn.it"; OutputSandboxPath = "/var/SandboxDir/ji/https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2fjiHL26jdUVDO7Bl6BJlhEQ/output"; StdOutput = "out.log"; GlobusResourceContactString = "cream-31.pd.infn.it:8443/cream-pbs"; QueueName = "creamtest2"; ShallowRetryCount = 3; RetryCount = 2; Executable = "/bin/sleep"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000009:BH=0000000000:JSS=000002:LM=000000:LRMS=000000:APP=000000:LBS=000000"; </verbatim> ===> test PASSED ---+++ [[https://savannah.cern.ch/bugs/?88569][Bug #88569]] WMS: LCMAPS should log on file only and not on syslog %GREEN% FIXED %ENDCOLOR% Check that no messages about LCMAPS have been logged by wmproxy on syslog: <verbatim> [root@cream-44 ~]# grep glite_wms_wmproxy_server /var/log/messages* | grep -i "LCMAPS" [root@cream-44 ~]# </verbatim> And also that there is a log file for lcmaps: <verbatim> [root@cream-44 ~]# ls -l /var/log/glite/lcmaps.log -rw-r--r-- 1 glite glite 659344 Nov 21 15:43 /var/log/glite/lcmaps.log </verbatim> ===> test PASSED ---++ Nagios probes test ---+++ Clean installation ---++++ Environment Clean SL5 installation plus an EMI-UI according to EMI guidelines. <verbatim> [root@devel15 ~]# cat /etc/emi-version 1.6.0-1 </verbatim> ---++++ Repositories * wget --no-check-certificate https://twiki.cern.ch/twiki/pub/EMI/EMI1Updates/testing.repo -O /etc/yum.repos.d/testing.repo * wget http://repository.egi.eu/sw/production/sam/1/repofiles/sam.repo -O /etc/yum.repos.d/sam.repo ---++++ Install yum install emi-wms-nagios <verbatim> [root@devel15 ~]# yum install emi-wms-nagios Loaded plugins: downloadonly, kernel-module, priorities, protect-packages, protectbase, security, verify, versionlock Repository EGI-trustanchors is listed more than once in the configuration EMI-1-base | 1.9 kB 00:00 EMI-1-testing-base | 1.9 kB 00:00 EMI-1-third-party | 1.9 kB 00:00 EMI-1-updates | 1.9 kB 00:00 Oracle | 1.9 kB 00:00 core | 1.1 kB 00:00 egi-sam | 1.9 kB 00:00 extras | 3.7 kB 00:00 4168 packages excluded due to repository priority protections 23 packages excluded due to repository protections Reading version lock configuration Setting up Install Process Resolving Dependencies --> Running transaction check ---> Package emi-wms-nagios.noarch 0:1.0.0-1.el5 set to be updated --> Processing Dependency: nagios-submit-conf >= 0.2 for package: emi-wms-nagios --> Processing Dependency: python-GridMon >= 1.1.10 for package: emi-wms-nagios --> Processing Dependency: python-ldap for package: emi-wms-nagios --> Running transaction check ---> Package nagios-submit-conf.noarch 0:0.2-2.el5 set to be updated ---> Package python-GridMon.noarch 0:1.1.12-1.el5 set to be updated ---> Package python-ldap.x86_64 0:2.2.0-2.1 set to be updated --> Finished Dependency Resolution Beginning Kernel Module Plugin Finished Kernel Module Plugin Dependencies Resolved ============================================================================================================================================================= Package Arch Version Repository Size ============================================================================================================================================================= Installing: emi-wms-nagios noarch 1.0.0-1.el5 EMI-1-testing-base 61 k Installing for dependencies: nagios-submit-conf noarch 0.2-2.el5 egi-sam 3.4 k python-GridMon noarch 1.1.12-1.el5 egi-sam 153 k python-ldap x86_64 2.2.0-2.1 sl-base 123 k Transaction Summary ============================================================================================================================================================= Install 4 Package(s) Upgrade 0 Package(s) Total download size: 340 k Is this ok [y/N]: y Downloading Packages: (1/4): nagios-submit-conf-0.2-2.el5.noarch.rpm | 3.4 kB 00:00 (2/4): emi-wms-nagios-1.0.0-1.el5.noarch.rpm | 61 kB 00:00 (3/4): python-ldap-2.2.0-2.1.x86_64.rpm | 123 kB 00:00 (4/4): python-GridMon-1.1.12-1.el5.noarch.rpm | 153 kB 00:00 ------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 85 kB/s | 340 kB 00:04 Running rpm_check_debug Running Transaction Test Finished Transaction Test Transaction Test Succeeded Running Transaction Installing : python-ldap 1/4 Installing : nagios-submit-conf 2/4 Installing : python-GridMon 3/4 Installing : emi-wms-nagios 4/4 Installed: emi-wms-nagios.noarch 0:1.0.0-1.el5 Dependency Installed: nagios-submit-conf.noarch 0:0.2-2.el5 python-GridMon.noarch 0:1.1.12-1.el5 python-ldap.x86_64 0:2.2.0-2.1 Complete! </verbatim> ---+++ Upgrade from production *N/A*: This is the initial release. No previous version to upgrade from. ---+++ Unit Tests *N/A*: The nagios probes are interpretted python scripts. Unit tests do not apply. ---+++ Regression Tests *N/A*: This is the initial release. No associated bugs exist. ---+++ Functional Tests ---++++ Preparation Log as a user and create a valid voms proxy <verbatim> [root@devel15 ~]# su - gianelle [gianelle@devel15 ~]$ voms-proxy-init -voms dteam Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle Creating temporary proxy ....................... Done Contacting voms.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr] "dteam" Done Creating proxy ............................................. Done Your proxy is valid until Sat Nov 26 01:46:46 2011 </verbatim> ---++++ WMSProbe * State + Monit <verbatim> [gianelle@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u502 -H devel19.cnaf.infn.it -m emi.wms.WMS-JobState OK: OK: Testing from: devel15.cnaf.infn.it DN: /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy VOMS FQANs: /dteam/Role=NULL/Capability=NULL, /dteam/NGI_IT/Role=NULL/Capability=NULL Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server ====================== glite-wms-job-submit Success ====================== The job has been successfully submitted to the WMProxy Your job identifier is: https://devel19.cnaf.infn.it:9000/YQCH-0vFjnHyPTKVfY9EeA The job identifier has been saved in the following file: /var/lib/gridprobes/dteam/emi.wms/WMS/devel19.cnaf.infn.it/jobID ========================================================================== [gianelle@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u502 -H devel19.cnaf.infn.it -m emi.wms.WMS-JobMonit --pass-check-dest active metric results >>> <devel19.cnaf.infn.it,emi.wms.WMS-JobState-dteam> OK: [Scheduled] https://devel19.cnaf.infn.it:9000/YQCH-0vFjnHyPTKVfY9EeA OK: [Scheduled] https://devel19.cnaf.infn.it:9000/YQCH-0vFjnHyPTKVfY9EeA glite-wms-job-status https://devel19.cnaf.infn.it:9000/YQCH-0vFjnHyPTKVfY9EeA ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://devel19.cnaf.infn.it:9000/YQCH-0vFjnHyPTKVfY9EeA Current Status: Scheduled Status Reason: unavailable Destination: devce.cnaf.infn.it:8443/cream-pbs-cert Submitted: Fri Nov 25 13:53:47 2011 CET ========================================================================== OK: Jobs processed - 1 OK: Jobs processed - 1 [Scheduled] : 1|jobs_processed=1;; DONE=0;; RUNNING=0;; SCHEDULED=1;; SUBMITTED=0;; READY=0;; WAITING=0;; ABORTED=0;; CANCELLED=0;; CLEARED=0;; MISSED=0;; UNDETERMINED=0;; unknown=0;1;2 [gianelle@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u502 -H devel19.cnaf.infn.it -m emi.wms.WMS-JobStateOK: Active job - Scheduled [2011-11-25T12:54:11Z] OK: Active job - Scheduled [2011-11-25T12:54:11Z] Testing from: devel15.cnaf.infn.it DN: /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy VOMS FQANs: /dteam/Role=NULL/Capability=NULL, /dteam/NGI_IT/Role=NULL/Capability=NULL Active job - Scheduled [2011-11-25T12:54:11Z] [gianelle@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u502 -H devel19.cnaf.infn.it -m emi.wms.WMS-JobMonit --pass-check-dest active metric results >>> <devel19.cnaf.infn.it,emi.wms.WMS-JobSubmit-dteam> OK: success. glite-wms-job-output --noint --nosubdir --dir /var/lib/gridprobes/dteam/emi.wms/WMS/devel19.cnaf.infn.it/jobOutput https://devel19.cnaf.infn.it:9000/YQCH-0vFjnHyPTKVfY9EeA 2>&1 Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server Warning - option --nosubdir specified: output files with same name will be overridden Warning - Directory already exists: /var/lib/gridprobes/dteam/emi.wms/WMS/devel19.cnaf.infn.it/jobOutput ================================================================================ JOB GET OUTPUT OUTCOME Output sandbox files for the job: https://devel19.cnaf.infn.it:9000/YQCH-0vFjnHyPTKVfY9EeA have been successfully retrieved and stored in the directory: /var/lib/gridprobes/dteam/emi.wms/WMS/devel19.cnaf.infn.it/jobOutput ================================================================================ metric results >>> <devel19.cnaf.infn.it,emi.wms.WMS-JobState-dteam> OK: success. glite-wms-job-output --noint --nosubdir --dir /var/lib/gridprobes/dteam/emi.wms/WMS/devel19.cnaf.infn.it/jobOutput https://devel19.cnaf.infn.it:9000/YQCH-0vFjnHyPTKVfY9EeA 2>&1 Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server Warning - option --nosubdir specified: output files with same name will be overridden Warning - Directory already exists: /var/lib/gridprobes/dteam/emi.wms/WMS/devel19.cnaf.infn.it/jobOutput ================================================================================ JOB GET OUTPUT OUTCOME Output sandbox files for the job: https://devel19.cnaf.infn.it:9000/YQCH-0vFjnHyPTKVfY9EeA have been successfully retrieved and stored in the directory: /var/lib/gridprobes/dteam/emi.wms/WMS/devel19.cnaf.infn.it/jobOutput ================================================================================ OK: Jobs processed - 1 OK: Jobs processed - 1 Done : 1|jobs_processed=1;; DONE=1;; RUNNING=0;; SCHEDULED=0;; SUBMITTED=0;; READY=0;; WAITING=0;; ABORTED=0;; CANCELLED=0;; CLEARED=0;; MISSED=0;; UNDETERMINED=0;; unknown=0;1;2 [gianelle@devel15 ~]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/YQCH-0vFjnHyPTKVfY9EeA ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://devel19.cnaf.infn.it:9000/YQCH-0vFjnHyPTKVfY9EeA Current Status: Cleared Status Reason: user retrieved output sandbox Destination: devce.cnaf.infn.it:8443/cream-pbs-cert Submitted: Fri Nov 25 13:53:47 2011 CET ========================================================================== [gianelle@devel15 ~]$ cat /var/lib/gridprobes/dteam/emi.wms/WMS/devel19.cnaf.infn.it/jobOutput/gridjob.out vgrid01.cnaf.infn.it </verbatim> Summary: %GREEN% *PASSED* %ENDCOLOR% * State + Monit + Cancel <verbatim> [gianelle@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u502 -H devel19.cnaf.infn.it -m emi.wms.WMS-JobStateOK: OK: Testing from: devel15.cnaf.infn.it DN: /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy VOMS FQANs: /dteam/Role=NULL/Capability=NULL, /dteam/NGI_IT/Role=NULL/Capability=NULL Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server ====================== glite-wms-job-submit Success ====================== The job has been successfully submitted to the WMProxy Your job identifier is: https://devel19.cnaf.infn.it:9000/WwIF0DsmFHJsdqPYE-VEfA The job identifier has been saved in the following file: /var/lib/gridprobes/dteam/emi.wms/WMS/devel19.cnaf.infn.it/jobID ========================================================================== [gianelle@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u502 -H devel19.cnaf.infn.it -m emi.wms.WMS-JobMonit --pass-check-dest active metric results >>> <devel19.cnaf.infn.it,emi.wms.WMS-JobState-dteam> OK: [Scheduled] https://devel19.cnaf.infn.it:9000/WwIF0DsmFHJsdqPYE-VEfA OK: [Scheduled] https://devel19.cnaf.infn.it:9000/WwIF0DsmFHJsdqPYE-VEfA glite-wms-job-status https://devel19.cnaf.infn.it:9000/WwIF0DsmFHJsdqPYE-VEfA ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://devel19.cnaf.infn.it:9000/WwIF0DsmFHJsdqPYE-VEfA Current Status: Scheduled Status Reason: unavailable Destination: cremino.cnaf.infn.it:8443/cream-pbs-cloudtf Submitted: Fri Nov 25 13:56:16 2011 CET ========================================================================== OK: Jobs processed - 1 OK: Jobs processed - 1 [Scheduled] : 1|jobs_processed=1;; DONE=0;; RUNNING=0;; SCHEDULED=1;; SUBMITTED=0;; READY=0;; WAITING=0;; ABORTED=0;; CANCELLED=0;; CLEARED=0;; MISSED=0;; UNDETERMINED=0;; unknown=0;1;2 [gianelle@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u502 -H devel19.cnaf.infn.it -m emi.wms.WMS-JobCancel OK: job cancelled OK: job cancelled Testing from: devel15.cnaf.infn.it DN: /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy VOMS FQANs: /dteam/Role=NULL/Capability=NULL, /dteam/NGI_IT/Role=NULL/Capability=NULL Job cancellation request sent: glite-wms-job-cancel --noint -i /var/lib/gridprobes/dteam/emi.wms/WMS/devel19.cnaf.infn.it/jobID Job bookkeeping files deleted. [gianelle@devel15 ~]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/WwIF0DsmFHJsdqPYE-VEfA ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://devel19.cnaf.infn.it:9000/WwIF0DsmFHJsdqPYE-VEfA Current Status: Cancelled Destination: cremino.cnaf.infn.it:8443/cream-pbs-cloudtf Submitted: Fri Nov 25 13:56:16 2011 CET ========================================================================== </verbatim> Summary: %GREEN% *PASSED* %ENDCOLOR%
Attachments
Attachments
Topic attachments
I
Attachment
Action
Size
Date
Who
Comment
log
81376.log
manage
5.1 K
2011-11-23 - 15:30
AlessioGianelle
log
81568.log
manage
5.7 K
2011-11-23 - 15:31
AlessioGianelle
log
83062.log
manage
14.0 K
2011-11-23 - 15:32
AlessioGianelle
log
85327.log
manage
5.8 K
2011-11-23 - 15:31
AlessioGianelle
log
86490.log
manage
171.6 K
2011-11-23 - 15:32
AlessioGianelle
log
86682.log
manage
5.4 K
2011-11-23 - 15:31
AlessioGianelle
log
WMSService-TS_20111020163927.log
manage
68.1 K
2011-10-20 - 15:24
AlessioGianelle
log
WMSService-TS_20111117120628.log
manage
24.0 K
2011-11-22 - 10:51
AlessioGianelle
log
WMSService-TS_20111122154717.log
manage
27.5 K
2011-11-22 - 15:00
AlessioGianelle
log
WMSService-TS_20111123115945.log
manage
58.1 K
2011-11-23 - 11:03
AlessioGianelle
log
WMSService-TS_20111123122227.log
manage
41.8 K
2011-11-23 - 11:30
AlessioGianelle
log
WMSService-TS_20111123125702.log
manage
208.0 K
2011-11-23 - 12:35
AlessioGianelle
tgz
cancel.tgz
manage
15.5 K
2011-11-22 - 10:52
AlessioGianelle
tgz
collection.tgz
manage
6.7 K
2011-11-22 - 10:52
AlessioGianelle
txt
configure.txt
manage
44.8 K
2011-12-02 - 09:58
AlessioGianelle
Configure
tgz
dag.tgz
manage
4.0 K
2011-11-22 - 10:53
AlessioGianelle
tgz
delegation.tgz
manage
4.1 K
2011-11-22 - 10:53
AlessioGianelle
tgz
epilogue.tgz
manage
6.8 K
2011-11-22 - 10:54
AlessioGianelle
txt
install.txt
manage
168.4 K
2011-12-02 - 09:58
AlessioGianelle
tgz
listmatch.tgz
manage
7.5 K
2011-11-22 - 10:54
AlessioGianelle
tgz
parallel.tgz
manage
4.6 K
2011-11-22 - 10:54
AlessioGianelle
tgz
parametric.tgz
manage
7.8 K
2011-11-22 - 10:55
AlessioGianelle
tgz
perusal.tgz
manage
8.8 K
2011-11-22 - 10:55
AlessioGianelle
tgz
pr.tgz
manage
19.4 K
2011-11-22 - 10:55
AlessioGianelle
tgz
resubmit.tgz
manage
6.5 K
2011-11-22 - 10:57
AlessioGianelle
tgz
submitcoll.tgz
manage
5.9 K
2011-11-22 - 10:56
AlessioGianelle
tgz
submitnormal.tgz
manage
5.5 K
2011-11-22 - 10:56
AlessioGianelle
txt
update.txt
manage
72.6 K
2011-12-02 - 15:52
AlessioGianelle
Edit
|
Attach
|
PDF
|
H
istory
:
r68
<
r67
<
r66
<
r65
<
r64
|
B
acklinks
|
V
iew topic
|
More topic actions...
Topic revision: r66 - 2011-12-05
-
AlessioGianelle
Home
Site map
CEMon web
CREAM web
Cloud web
Cyclops web
DGAS web
EgeeJra1It web
Gows web
GridOversight web
IGIPortal web
IGIRelease web
MPI web
Main web
MarcheCloud web
MarcheCloudPilotaCNAF web
Middleware web
Operations web
Sandbox web
Security web
SiteAdminCorner web
TWiki web
Training web
UserSupport web
VOMS web
WMS web
WMSMonitor web
WeNMR web
WMS Web
Create New Topic
Index
Search
Changes
Notifications
RSS Feed
Statistics
Preferences
View
Raw View
Print version
Find backlinks
History
More topic actions
Edit
Raw edit
Attach file or image
Edit topic preference settings
Set new parent
More topic actions
Account
Log In
Edit
Attach
Copyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki?
Send feedback