- WMS test report
- Summary
- Deployment Tests
- Unit Tests
- Basic Functionality tests
- Published information
- Performance test
- Regression tests
- Bug #81376 (glite-wms-wmproxy.restart.cron MUST support graceful wmp restart) FIXED
- Bug #81568 (some inconsistencies in locations for logs and configuration) FIXED
- Bug #83062 (A different template should be used for glue2 publication) FIXED
- Bug #85327 (glite-wms-wmproxy.restart.cron missing HOSTNAME environment variable) FIXED
- Bug #86485 (ICE doesn't send the iceId to CREAM) FIXED
- Bug #86490 (EMI WMS stops accepting jobs after 31999th submission for the same DN (on ext3)) FIXED
- Bug #86682 ( yaim changes ownership of /var) FIXED
- Bug #86769 (EMI WMS Update 6: WMS 'hangs' after several hours of operation) FIX NOT CERTIFIED
- Bug #86956 (Match-matching of requests with access to catalogues can cause the WM to crash) FIX NOT CERTIFIED
- Bug #87259 (WMS purger errors in jobcontroller/logmonitor) FIXED
- Bug #87994 yaim-wms doesn't fill the attributes get_acbr and get_owner in the conf file for Glue2 FIXED
- Bug #88072 EMI-1 WMS cannot submit to ARC CEs FIXED
- Bug #88096 CA certificate verification failed FIXED NOT CERTIFIED
- Bug #88097 Collection not terminated FIXED NOT CERTIFIED
- Bug #88450 Ice configuration section doesn't use environment variable FIXED
- Bug #88558 WM creates submit requests with old sequence code upon a replan FIXED
- Bug #88569 WMS: LCMAPS should log on file only and not on syslog FIXED
- Nagios probes test
WMS test report
Summary
- Component: WMS 3.3.4
- Savannah task: 22847
- ETICS Subsystem Configuration Name: emi-wms_R_3_3_6_3 emi-jdl_R_3_2_6_1
- VCS Tag: emi-ice_R_3_3_4_1, emi-wms-manager_R_3_3_1_4, emi-jobsubmission_R_3_3_2_1, emi-wmproxy_R_3_3_4_2, emi-wms-brokerinfo_R_3_3_2_1, emi-wms-configuration_R_3_3_2_1, emi-wms-ism_R_3_3_2_1, emi-wms-matchmaking_R_3_3_2_1, emi-wms-purger_R_3_3_2_1, emi-yaim-wms_R_4_1_4_3, emi-jdl-api-cpp_R_3_2_6_1, emi-wms-nagios_R_1_0_0_0, emi-wms-mp_R_1_0_2_0
- EMI Major Release: EMI 1 (Kebnekaise)
- Platform: sl5 epel
- Certification report: not yet available
- Outcome: Certification under way
Deployment Tests
Clean installation
- Starting from a SL5 x86_64 kernel machine:
[root@emi-demo11 ~]# cat /etc/redhat-release Scientific Linux SL release 5.7 (Boron) [root@emi-demo11 ~]# uname -a Linux emi-demo11.cnaf.infn.it 2.6.18-274.12.1.el5 #1 SMP Tue Nov 29 11:14:39 EST 2011 x86_64 x86_64 x86_64 GNU/Linux
- Import EMI rpm gpg key
[root@emi-demo11 ~]# wget http://emisoft.web.cern.ch/emisoft/dist/EMI/1/RPM-GPG-KEY-emi -O /tmp/emi-key_gd.asc [root@emi-demo11 ~]# rpm --import /tmp/emi-key_gd.asc
- cd /etc/yum.repos.d/
- wget --no-check-certificate https://twiki.cern.ch/twiki/pub/EMI/EMI1Updates/testing.repo
- yum clean all
- yum install ca-policy-egi-core
- yum install emi-wms
- See install.log: Clean install log
- /opt/glite/yaim/bin/yaim -c -s siteinfo/site-info.def -n WMS
- See yaim-install.log: yaim install log
Update
- Starting from a production version, repo available at http://emisoft.web.cern.ch/emisoft/dist/EMI/1/sl5/x86_64/updates/emi-release-1.0.1-1.sl5.noarch.rpm
[root@emi-demo12 yum.repos.d]# rpm -qa|grep wms-manager glite-wms-manager-3.3.1-3.sl5
- wget --no-check-certificate https://twiki.cern.ch/twiki/pub/EMI/EMI1Updates/testing.repo -O /etc/yum.repos.d/testing.repo
- yum clean all
- service gLite stop
- yum update
- /opt/glite/yaim/bin/yaim -c -s site-info.def -n WMS
- update.log: Update log
- Check final version:
[root@emi-demo12 siteinfo]# rpm -qa|grep wms-manager glite-wms-manager-3.3.1-4.sl5
Proxy mode
The same update procedure has been applied on a wms configured in proxy mode. emi-demo12 being the wms and emi-demo11 the LB server. In order to configure emi-demo12 in proxy mode the following lines have been defined in the site-info.def file:# Hostname of the WMS WMS_HOST=emi-demo12.$MY_DOMAIN LB_HOST=emi-demo11.$MY_DOMAINand the following lines in siteinfo/services/glite-wms :
LB_HOST="emi-demo11.cnaf.infn.it:9000" #GLITE_LB_TYPE=both GLITE_LB_TYPE=proxyOn emi-demo11 the following lines have been defined in the site-info.def file:
GLITE_LB_WMS_DN="/C=IT/O=INFN/OU=Host/L=CNAF/CN=emi-demo12.cnaf.infn.it" GLITE_LB_AUTHZ_REGISTER_JOBS=".*"
Unit Tests
N/A: Tests will be available with EMI-2.Basic Functionality tests
Submission
Where possible the tests should use internal testbed CEs (located in Padova). This is accomplished by adding the following line to wms-command.conf:DEFAULTREQ= other.GlueCEStateStatus == "Testing"This is not possible, for example, for parallel jobs.
Normal
[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 1 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) Test 1A: Submit to an LCG-CE Test 1B: Submit to a CREAM CE Test 1C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 17:09:32 ===> Ended : 17:21:55 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111228170932.log [capannini@devel15 WMS-service]$ voms-proxy-init --voms dteam:/dteam/NGI_IT:operator Enter GRID pass phrase: Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini Creating temporary proxy .................................................................. Done Contacting voms2.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr] "dteam" Done Creating proxy ........................................................................... Done Your proxy is valid until Thu Dec 29 23:31:47 2011 [capannini@devel15 WMS-service]$ voms-proxy-info --all subject : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy issuer : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini identity : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini type : proxy strength : 1024 bits path : /tmp/x509up_u504 timeleft : 11:59:51 key usage : Digital Signature, Key Encipherment, Data Encipherment === VO dteam extension information === VO : dteam subject : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini issuer : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr attribute : /dteam/NGI_IT/Role=operator/Capability=NULL attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL attribute : /dteam/Role=NULL/Capability=NULL timeleft : 11:59:51 uri : voms2.hellasgrid.gr:15004 [capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 1 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) Test 1A: Submit to an LCG-CE Test 1B: Submit to a CREAM CE Test 1C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 11:32:22 ===> Ended : 11:40:10 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111229113222.log [capannini@devel15 WMS-service]$ voms-proxy-init --voms testers.eu-emi.eu Enter GRID pass phrase: Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini Creating temporary proxy ................................... Done Contacting emitestbed07.cnaf.infn.it:15002 [/C=IT/O=INFN/OU=Host/L=CNAF/CN=emitestbed07.cnaf.infn.it] "testers.eu-emi.eu" Done Creating proxy ...................................................... Done [capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 1 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) Test 1A: Submit to an LCG-CE Test 1B: Submit to a CREAM CE Test 1C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 11:45:01 ===> Ended : 11:52:20 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111229114501.log [fabiok@ui WMS-service]$ voms-proxy-init --voms infngrid Enter GRID pass phrase: Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini Creating temporary proxy ..................................................... Done Contacting voms.cnaf.infn.it:15000 [/C=IT/O=INFN/OU=Host/L=CNAF/CN=voms.cnaf.infn.it] "infngrid" Done Creating proxy .................................................... Done Your proxy is valid until Thu Dec 29 23:45:58 2011 [fabiok@ui WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V infngrid -d 3 -l -t 1 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) Test 1A: Submit to an LCG-CE Test 1B: Submit to a CREAM CE Test 1C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 11:46:25 ===> Ended : 11:52:40 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20111229114625.logSee submit.log.tgz: Normal submission logs
Proxy mode
There is a problem with the configuration of emi-demo12 (wms in proxy mode) pointing to emi-demo11 (wms in both mode): the events Running,ReallyRunning and Done Ok are not logged by emi-demo11 (and don't appear in the logging-info), even if they are present in the lbserver20 db of emi-demo12. These events are instead correctly logged if emi-demo11 is configured as LB in server mode. Thus, in order to properly test a wms in proxy mode, then it should point to a LB server not in both mode.ARC CE
Job submitted on a production CE using the cms VO.
[capannini@cert-17 regression_tests]$ lcg-infosites --is egee-bdii.cnaf.infn.it --vo cms ce |grep arc
80 0 95 5 95 arc.univ.kiev.ua:2811/nordugrid-torque-arc
768 707 196 61 196 jade-cms.hip.fi:2811/nordugrid-GE-arc
256 0 0 0 0 korundi.grid.helsinki.fi:2811/nordugrid-GE-arc
[cristofori@ui ~]$ glite-wms-job-submit -e https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server -a -r jade-cms.hip.fi:2811/nordugrid-GE-arc firsttest.jdl
Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
====================== glite-wms-job-submit Success ======================
The job has been successfully submitted to the WMProxy
Your job identifier is:
https://emi-demo11.cnaf.infn.it:9000/e4f1VtGXy_I_2MS2dVUs4g
==========================================================================
[cristofori@ui ~]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/e4f1VtGXy_I_2MS2dVUs4g
======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/e4f1VtGXy_I_2MS2dVUs4g
Current Status: Done (Success)
Exit code: 0
Status Reason: Job terminated successfully
Destination: jade-cms.hip.fi:2811/nordugrid-GE-arc
Submitted: Tue Mar 27 14:27:13 2012 CEST
==========================================================================
[cristofori@ui ~]$ glite-wms-job-output --dir ./ https://emi-demo11.cnaf.infn.it:9000/e4f1VtGXy_I_2MS2dVUs4g
Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
================================================================================
JOB GET OUTPUT OUTCOME
Output sandbox files for the job:
https://emi-demo11.cnaf.infn.it:9000/e4f1VtGXy_I_2MS2dVUs4g
have been successfully retrieved and stored in the directory:
/home/GUEST/cristofori/cristofori_e4f1VtGXy_I_2MS2dVUs4g
================================================================================
[cristofori@ui ~]$ cat /home/GUEST/cristofori/cristofori_e4f1VtGXy_I_2MS2dVUs4g/stderr.log
[cristofori@ui ~]$ cat /home/GUEST/cristofori/cristofori_e4f1VtGXy_I_2MS2dVUs4g/stdout.log
cmsuser07
Jdl with DataRequirements
In case of problems with lcg-ce, it i possible to create replicas of the file on other SEs close to different lcg-ces.
[capannini@devel15 WMS-service]$ lcg-rep --vo dteam -d srm://gridsrm.pi.infn.it:8444/dteam/fcapanni/e-2M.txt lfn:/grid/dteam/fcapanni/e-2M.txt
GSIFTP: source: set up FTP mode. DCAU disabled. Streams = 1, Tcp BS = 0
GSIFTP: dest: set up FTP mode. DCAU disabled. Streams = 1, Tcp BS = 0
Transfer completed succesfully
[capannini@devel15 WMS-service]$ lcg-rep --vo dteam -d srm://gridsrm.pi.infn.it:8444/dteam/fcapanni/PI_1M.txt lfn:/grid/dteam/fcapanni/PI_1M.txt
GSIFTP: source: set up FTP mode. DCAU disabled. Streams = 1, Tcp BS = 0
GSIFTP: dest: set up FTP mode. DCAU disabled. Streams = 1, Tcp BS = 0
Transfer completed succesfully
The ce/se couple can be found looking for "/jobmanager" in the output of the following command:
[capannini@devel15 WMS-service]$ lcg-infosites --vo dteam closeSE
The GlueVOInfoPath of the storage area to be used in the srm surl after the endpoint can be found with the following search:
ldapsearch -x -LLL -H ldap://egee-bdii.cnaf.infn.it:2170 -b mds-vo-name=local,o=grid '(&(objectClass=GlueVOInfo)(GlueChunkKey=GlueSEUniqueID=gridsrm.pi.infn.it))' GlueVOInfoPath
(/dteam was picked up in this test)
the lcg-ce to be excluded can be specified in the testsuite configuration file (wms-command.conf) with the following attribute:
DEFAULTREQ=!regexp(".*ce130.*", other.GlueCEUniqueID) && !regexp(".*ce131.*", other.GlueCEUniqueID) && !regexp(".*ce133.*", other.GlueCEUniqueID)
In the following test the lcg-ce was the following (by inspecting the log):
Destination CE is ce132.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam
[capannini@devel15 WMS-service]$ cat JDLs/data-req.jdl
###########################################
# JDL with Data Requirements #
###########################################
Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";
Prologue = "prologue.sh";
FuzzyRank = true;
Environment={"LFC_HOST=prod-lfc-shared-central.cern.ch", "LFC_HOME=prod-lfc-shared-central.cern.ch:/grid/dteam/aleph"};
InputSandbox = {"JDLs/prologue.sh"};
OutputSandbox = {"std.out", "std.err","prologue.out", "PI_1M.txt"};
DataRequirements = {
[
DataCatalogType = "DLI";
DataCatalog ="http://prod-lfc-shared-central.cern.ch:8085/";
InputData = { "lfn:/grid/dteam/fcapanni/PI_1M.txt" };
]
};
DataAccessProtocol = "gsiftp";
AllowZippedISB=true;
RetryCount = 1;
ShallowRetryCount = 2;
[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 1 -i -j JDLs/data-req.jdl
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service
===> + Description: Test a complete job cycle: from submission to get output
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
Enter the user proxy password:
Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general)
Test 1A: Submit to an LCG-CE
Test 1B: Submit to a CREAM CE
Test 1C: Submit without restrictions
===>
===> Test: WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 12:24:32
===> Ended : 12:36:28
===>
===> === test PASSED ===
===>
===> Test log file is WMSService-TS_20120110122432.log
Jdl with ISBBaseURI and OSBDestURI
Test on CREAM ce:
[capannini@devel15 certificazione]$ cat URI.jdl
###########################################
# JDL with ISBURI & OSBDestURI #
###########################################
Executable = "exe.sh";
StdOutput = "std.out";
StdError = "std.err";
InputSandboxBaseURI = "gsiftp://devel11.cnaf.infn.it:2811/tmp";
FuzzyRank = true;
InputSandbox = {"exe.sh", "test1", "test2"};
OutputSandbox = {"std.out", "std.err"};
OutputSandboxDestURI = {"gsiftp://devel09.cnaf.infn.it:2811/tmp/myjobOutput", "std.err"};
#requirements = other.GlueCEStateStatus == "Testing" && regexp("/cream", other.GlueCEUniqueID);
requirements = regexp("/cream", other.GlueCEUniqueID);
DataAccessProtocol = "gsiftp";
RetryCount = 1;
ShallowRetryCount = 2;
[capannini@devel15 certificazione]$ cat tmp/exe.sh
#!/bin/sh
NAME=`echo $GLITE_WMS_JOBID | sed -e 's/.*\///'`
echo $NAME
ls -l
[capannini@devel15 certificazione]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf URI.jdl
Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
====================== glite-wms-job-submit Success ======================
The job has been successfully submitted to the WMProxy
Your job identifier is:
https://emi-demo11.cnaf.infn.it:9000/NAwVzPxlPqGYz-Kf2xWLNg
==========================================================================
[capannini@devel15 certificazione]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/NAwVzPxlPqGYz-Kf2xWLNg
======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/NAwVzPxlPqGYz-Kf2xWLNg
Current Status: Done (Success)
Exit code: 0
Status Reason: Job Terminated Successfully
Destination: inaf-ce-01.ct.pi2s2.it:8443/cream-lsf-cert
Submitted: Thu Dec 29 16:20:11 2011 CET
==========================================================================
[capannini@devel15 certificazione]$ glite-wms-job-output https://emi-demo11.cnaf.infn.it:9000/NAwVzPxlPqGYz-Kf2xWLNg
Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
================================================================================
JOB GET OUTPUT OUTCOME
Output sandbox files for the job:
https://emi-demo11.cnaf.infn.it:9000/NAwVzPxlPqGYz-Kf2xWLNg
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/capannini_NAwVzPxlPqGYz-Kf2xWLNg
================================================================================
[capannini@devel15 certificazione]$ ls /tmp/jobOutput/capannini_NAwVzPxlPqGYz-Kf2xWLNg
std.err
Log on devel09.cnaf.infn.it:
[root@devel09 ~]# cat /tmp/myjobOutput
NAwVzPxlPqGYz-Kf2xWLNg
total 16
-rwxr-xr-x 1 dteam002 dteam 80 Dec 29 16:20 exe.sh
-rw-r--r-- 1 dteam002 dteam 0 Dec 29 16:20 std.err
-rw-r--r-- 1 dteam002 dteam 23 Dec 29 16:20 std.out
-rw-r--r-- 1 dteam002 dteam 6 Dec 29 16:20 test1
-rw-r--r-- 1 dteam002 dteam 6 Dec 29 16:20 test2
===> test PASSED
Same process with an LCG CE:
[capannini@devel15 certificazione]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf URI.jdl
Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
====================== glite-wms-job-submit Success ======================
The job has been successfully submitted to the WMProxy
Your job identifier is:
https://emi-demo11.cnaf.infn.it:9000/d4akLe8H2jynZpCPe5jGxA
==========================================================================
[capannini@devel15 certificazione]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/d4akLe8H2jynZpCPe5jGxA
======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/d4akLe8H2jynZpCPe5jGxA
Current Status: Done (Success)
Logged Reason(s):
- Job got an error while in the CondorG queue.
- Job terminated successfully
Exit code: 0
Status Reason: Job terminated successfully
Destination: gridce.sns.it:2119/jobmanager-lcgpbs-cert
Submitted: Fri Dec 30 11:43:28 2011 CET
==========================================================================
[capannini@devel15 certificazione]$ glite-wms-job-output https://emi-demo11.cnaf.infn.it:9000/d4akLe8H2jynZpCPe5jGxA
Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
================================================================================
JOB GET OUTPUT OUTCOME
Output sandbox files for the job:
https://emi-demo11.cnaf.infn.it:9000/d4akLe8H2jynZpCPe5jGxA
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/capannini_d4akLe8H2jynZpCPe5jGxA
================================================================================
[capannini@devel15 certificazione]$ ls -l /tmp/jobOutput/capannini_d4akLe8H2jynZpCPe5jGxA
total 0
-rw-r--r-- 1 capannini emitestbed 0 Dec 30 13:08 std.err
Log on devel09.cnaf.infn.it:
[root@devel09 tmp]# ls -l myjobOutput
-rw-r--r-- 1 dteam042 dteam 714 30 dic 13:01 myjobOutput
[root@devel09 tmp]# cat myjobOutput
d4akLe8H2jynZpCPe5jGxA
total 36
-rwxr-xr-x 1 dteam003 dteam 80 Dec 30 13:00 exe.sh
-rw-r--r-- 1 dteam003 dteam 154 Dec 30 13:01 https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fd4akLe8H2jynZpCPe5jGxA.output
-rw-r--r-- 1 dteam003 dteam 0 Dec 30 13:01 std.err
-rw-r--r-- 1 dteam003 dteam 23 Dec 30 13:01 std.out
-rw-r--r-- 1 dteam003 dteam 6 Dec 30 13:00 test1
-rw-r--r-- 1 dteam003 dteam 6 Dec 30 13:00 test2
-rw------- 1 dteam003 dteam 2 Dec 30 13:01 tmp.GPdus14280
-rw------- 1 dteam003 dteam 2 Dec 30 13:01 tmp.OAref14000
-rw------- 1 dteam003 dteam 103 Dec 30 13:01 tmp.QtSZI14001
-rw------- 1 dteam003 dteam 0 Dec 30 13:01 tmp.vFipq14279
-rw------- 1 dteam003 dteam 103 Dec 30 13:01 tmp.YUTON14281
===> test PASSED
Jdl with ISBBaseURI and OSBBaseDestURI
Submit to CREAM CE
[capannini@devel15 certificazione]$ cat BaseURI.jdl
###########################################
# JDL with ISBURI & OSBBaseDestURI #
###########################################
Executable = "exe.sh";
StdOutput = "std.out";
StdError = "std.err";
InputSandboxBaseURI = "gsiftp://devel11.cnaf.infn.it:2811/tmp";
FuzzyRank = true;
InputSandbox = {"exe.sh", "test1", "test2"};
OutputSandbox = {"std.out", "std.err"};
OutputSandboxBaseDestURI="gsiftp://devel09.cnaf.infn.it:2811/tmp";
#requirements = other.GlueCEStateStatus == "Testing" && regexp("/cream", other.GlueCEUniqueID);
requirements = regexp("/cream", other.GlueCEUniqueID);
RetryCount = 1;
ShallowRetryCount = 2;
[capannini@devel15 certificazione]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf BaseURI.jdl
Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
====================== glite-wms-job-submit Success ======================
The job has been successfully submitted to the WMProxy
Your job identifier is:
https://emi-demo11.cnaf.infn.it:9000/bRxuYfMUA79tvRXdrpgJdQ
==========================================================================
[capannini@devel15 certificazione]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/bRxuYfMUA79tvRXdrpgJdQ
======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/bRxuYfMUA79tvRXdrpgJdQ
Current Status: Done (Success)
Exit code: 0
Status Reason: Job Terminated Successfully
Destination: t2-ce-01.to.infn.it:8443/cream-pbs-short
Submitted: Fri Dec 30 12:47:43 2011 CET
==========================================================================
[capannini@devel15 certificazione]$ glite-wms-job-output https://emi-demo11.cnaf.infn.it:9000/bRxuYfMUA79tvRXdrpgJdQ
Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
================================================================================
JOB GET OUTPUT OUTCOME
No output files to be retrieved for the job:
https://emi-demo11.cnaf.infn.it:9000/bRxuYfMUA79tvRXdrpgJdQ
================================================================================
[capannini@devel15 certificazione]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/bRxuYfMUA79tvRXdrpgJdQ
======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/bRxuYfMUA79tvRXdrpgJdQ
Current Status: Cleared
Status Reason: user retrieved output sandbox
Destination: t2-ce-01.to.infn.it:8443/cream-pbs-short
Submitted: Fri Dec 30 12:47:43 2011 CET
==========================================================================
Log on devel09.cnaf.infn.it:
[root@devel09 tmp]# ls std*
std.err std.out
[root@devel09 tmp]# cat std.out
bRxuYfMUA79tvRXdrpgJdQ
total 16
-rwxr-xr-x 1 dteam029 dteam 80 Dec 30 12:48 exe.sh
-rw-r--r-- 1 dteam029 dteam 0 Dec 30 12:48 std.err
-rw-r--r-- 1 dteam029 dteam 23 Dec 30 12:48 std.out
-rw-r--r-- 1 dteam029 dteam 6 Dec 30 12:48 test1
-rw-r--r-- 1 dteam029 dteam 6 Dec 30 12:48 test2
===> === test PASSED ===
Submit to an LCG CE:
[capannini@devel15 certificazione]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf BaseURI.jdl
Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
====================== glite-wms-job-submit Success ======================
The job has been successfully submitted to the WMProxy
Your job identifier is:
https://emi-demo11.cnaf.infn.it:9000/pX2yRjupthmRSr8bqf5_Bg
==========================================================================
[capannini@devel15 certificazione]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/pX2yRjupthmRSr8bqf5_Bg
======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/pX2yRjupthmRSr8bqf5_Bg
Current Status: Done (Success)
Exit code: 0
Status Reason: Job terminated successfully
Destination: grid-ce.lns.infn.it:2119/jobmanager-lcgpbs-short
Submitted: Fri Dec 30 13:18:38 2011 CET
==========================================================================
[capannini@devel15 certificazione]$ glite-wms-job-output https://emi-demo11.cnaf.infn.it:9000/pX2yRjupthmRSr8bqf5_Bg
Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
================================================================================
JOB GET OUTPUT OUTCOME
No output files to be retrieved for the job:
https://emi-demo11.cnaf.infn.it:9000/pX2yRjupthmRSr8bqf5_Bg
================================================================================
[capannini@devel15 certificazione]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/pX2yRjupthmRSr8bqf5_Bg
======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/pX2yRjupthmRSr8bqf5_Bg
Current Status: Cleared
Status Reason: user retrieved output sandbox
Destination: grid-ce.lns.infn.it:2119/jobmanager-lcgpbs-short
Submitted: Fri Dec 30 13:18:38 2011 CET
==========================================================================
Log on devel09.cnaf.infn.it:
[root@devel09 tmp]# ls -l std*
-rw-r--r-- 1 dteam042 dteam 0 30 dic 13:19 std.err
-rw-r--r-- 1 dteam042 dteam 727 30 dic 13:19 std.out
[root@devel09 tmp]# cat std.out
pX2yRjupthmRSr8bqf5_Bg
total 36
-rwxr-xr-x 1 dteam050 dteam 80 Dec 30 13:19 exe.sh
-rw-r--r-- 1 dteam050 dteam 118 Dec 30 13:19 https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fpX2yRjupthmRSr8bqf5_5fBg.output
-rw-r--r-- 1 dteam050 dteam 0 Dec 30 13:19 std.err
-rw-r--r-- 1 dteam050 dteam 23 Dec 30 13:19 std.out
-rw-r--r-- 1 dteam050 dteam 6 Dec 30 13:19 test1
-rw-r--r-- 1 dteam050 dteam 6 Dec 30 13:19 test2
-rw------- 1 dteam050 dteam 103 Dec 30 13:19 tmp.csGdY18153
-rw------- 1 dteam050 dteam 0 Dec 30 13:19 tmp.FBSue18151
-rw------- 1 dteam050 dteam 2 Dec 30 13:19 tmp.FZDLk18152
-rw------- 1 dteam050 dteam 2 Dec 30 13:19 tmp.HdslV18137
-rw------- 1 dteam050 dteam 103 Dec 30 13:19 tmp.hSlSL18138
Jdl with AllowZippedISB
[capannini@devel15 certificazione]$ cat AllowZIP.jdl
###########################################
# JDL with AllowZippedISB attribute #
###########################################
Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";
InputSandbox = {"fileA", "fileB", "fileC", "fileD", "fileE"};
OutputSandbox = {"std.out", "std.err"};
#requirements = other.GlueCEStateStatus == "Testing" && regexp("/cream", other.GlueCEUniqueID);
requirements = regexp("/cream", other.GlueCEUniqueID);
AllowZippedISB=true;
RetryCount = 1;
ShallowRetryCount = 2;
[capannini@devel15 certificazione]$ glite-wms-job-submit --debug -a -c ~/wms_emi-demo11.conf AllowZIP.jdl
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Vo read from proxy certificate extension
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - VirtualOrganisation value : dteam
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Configuration file: /home/capannini/wms_emi-demo11.conf - VirtualOrganisation, RetryCount, rank, requirements attribute(s) no more supported outside JDL Default Attributes section "JdlDefaultAttributes"
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Loaded user configuration file:
/home/capannini/wms_emi-demo11.conf
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Configuration file: //etc/glite-wms/dteam/glite_wmsui.conf - VirtualOrganisation, MyProxyServer attribute(s) no more supported outside JDL Default Attributes section "JdlDefaultAttributes"
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Loaded Vo specific configuration file:
//etc/glite-wms/dteam/glite_wmsui.conf
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Loaded Configuration values:
[
JdlDefaultAttributes =
[
PerusalFileEnable = false;
ShallowRetryCount = 10;
SignificantAttributes = { "Requirements","Rank","FuzzyRank" };
RetryCount = 3;
AllowZippedISB = true;
VirtualOrganisation = "dteam";
rank = -other.GlueCEStateEstimatedResponseTime;
MyProxyServer = "myproxy.cnaf.infn.it";
requirements = other.GlueCEStateStatus == "Production"
];
OutputStorage = "/tmp";
ListenerStorage = "/tmp";
ErrorStorage = "/var/tmp";
WmProxyEndPoints = { "https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server" }
]
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Getting Endpoint URL from configuration file
-----------------------------------------
Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Calling the WMProxy getVersion service
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - WMProxy Version: 3.3.4
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Sending Proxy Request to https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Calling the WMProxy delegationns__getProxyReq service
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - delegationns__getProxyReq - The proxy has been successfully retrieved
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Calling the WMProxy delegationns__putProxyReq service
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - The proxy has been successfully delegated with the identifier: oHGkZ5pxjZH7-LDxL2NIMw
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Calling the WMProxy getTransferProtocols service
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - getTransferProtocols - Available protocols: gsiftp, https
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - FileTransferProtocol not specified; using the default protocol: gsiftp
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - The JDL file is: /home/capannini/certificazione/AllowZIP.jdl
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Retrieving the list of the local ISB files from the user JDL
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Total size of the ISB file(s) to be transferred to: 1126406
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Max single job size of the ISB file(s) to be transferred to: 1024000
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Checking the User-FreeQuota on the server
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:29 -I- PID: 23649 (Debug) - Calling the WMProxy getFreeQuota service
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - getFreeQuota - User freequota not set on the server
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - Getting the max ISB size from the server https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - Calling the WMProxy getMaxInputSandboxSize service
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - getMaxInputSandboxSize - Max ISB size information successfully retrieved
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - The max job size (1024000 bytes) doesn't exceed the max size limit of 100000000 bytes: File transfer is allowed
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - File archiving and file compression allowed by user in the JDL
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - Registering JDL [ StdOutput = "std.out"; ShallowRetryCount = 2; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 1; Executable = "/bin/ls"; Type = "job"; Arguments = "-la"; AllowZippedISB = true; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank = -other.GlueCEStateEstimatedResponseTime; ZippedISB = { "ISBfiles_y8ovpm9W5-W_kUqpeNiCKA_0.tar.gz" }; OutputSandbox = { "std.out","std.err" }; InputSandbox = { "file:///home/capannini/certificazione/fileA","file:///home/capannini/certificazione/fileB","file:///home/capannini/certificazione/fileC","file:///home/capannini/certificazione/fileD","file:///home/capannini/certificazione/fileE" }; StdError = "std.err"; rank = -other.GlueCEStateEstimatedResponseTime; MyProxyServer = "myproxy.cnaf.infn.it"; requirements = ( regexp("/cream",other.GlueCEUniqueID) ) && ( other.GlueCEStateStatus == "Production" ) ]
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - Registering the job to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - Calling the WMProxy jobRegister service
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - The JobId is: https://emi-demo11.cnaf.infn.it:9000/nGo2GsPtKD3urj66sBnHLA
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - Archiving the ISB files: /tmp/ISBfiles_y8ovpm9W5-W_kUqpeNiCKA_0.tar
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - This archive file has been successfully created: /tmp/ISBfiles_y8ovpm9W5-W_kUqpeNiCKA_0.tar
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - Compressing the file (.gz): /tmp/ISBfiles_y8ovpm9W5-W_kUqpeNiCKA_0.tar
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - ISB ZIPPED file successfully created: /tmp/ISBfiles_y8ovpm9W5-W_kUqpeNiCKA_0.tar.gz
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - Getting the SandboxDestinationURI from the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - Calling the WMProxy getSandboxDestURI service with gsiftp protocol
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - getSandboxDestURI - Destination URIs sucessfully retrieved
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - DestinationURI: gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/nG/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fnGo2GsPtKD3urj66sBnHLA/input
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - FileTransfer (gsiftp): using globus-url-copy to transfer the local InputSandBox file(s) to the submission endpoint
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - File Transfer (gsiftp)
Command: /usr/bin/globus-url-copy
Source: file:///tmp/ISBfiles_y8ovpm9W5-W_kUqpeNiCKA_0.tar.gz
Destination: gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/nG/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fnGo2GsPtKD3urj66sBnHLA/input/ISBfiles_y8ovpm9W5-W_kUqpeNiCKA_0.tar.gz
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - File Transfer (gsiftp) Transfer successfully done
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - Starting the job: https://emi-demo11.cnaf.infn.it:9000/nGo2GsPtKD3urj66sBnHLA
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - Calling the WMProxy jobStart service
-----------------------------------------
-----------------------------------------
30 Dec 2011, 16:54:30 -I- PID: 23649 (Debug) - jobStart - The job has been successfully started
-----------------------------------------
====================== glite-wms-job-submit Success ======================
The job has been successfully submitted to the WMProxy
Your job identifier is:
https://emi-demo11.cnaf.infn.it:9000/nGo2GsPtKD3urj66sBnHLA
==========================================================================
*** Log file created ***
Possible Errors and Debug messages have been printed in the following file:
/var/tmp/glite-wms-job-submit_504_23649_20111230_16-54-29.log
Check if AllowZippedISB has been included in the jdl:
[capannini@devel15 certificazione]$ glite-wms-job-info --jdl https://emi-demo11.cnaf.infn.it:9000/nGo2GsPtKD3urj66sBnHLA|grep Allow
AllowZippedISB = true;
[capannini@devel15 certificazione]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/nGo2GsPtKD3urj66sBnHLA
======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/nGo2GsPtKD3urj66sBnHLA
Current Status: Done (Success)
Exit code: 0
Status Reason: Job Terminated Successfully
Destination: cremino.cnaf.infn.it:8443/cream-pbs-cert
Submitted: Fri Dec 30 16:54:30 2011 CET
==========================================================================
[capannini@devel15 certificazione]$
[capannini@devel15 certificazione]$
[capannini@devel15 certificazione]$
[capannini@devel15 certificazione]$ glite-wms-job-output https://emi-demo11.cnaf.infn.it:9000/nGo2GsPtKD3urj66sBnHLA
Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
================================================================================
JOB GET OUTPUT OUTCOME
Output sandbox files for the job:
https://emi-demo11.cnaf.infn.it:9000/nGo2GsPtKD3urj66sBnHLA
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/capannini_nGo2GsPtKD3urj66sBnHLA
================================================================================
[capannini@devel15 certificazione]$ ls -l /tmp/jobOutput/capannini_nGo2GsPtKD3urj66sBnHLA
total 4
-rw-r--r-- 1 capannini emitestbed 0 Dec 30 17:00 std.err
-rw-r--r-- 1 capannini emitestbed 623 Dec 30 17:00 std.out
[capannini@devel15 certificazione]$ cat /tmp/jobOutput/capannini_nGo2GsPtKD3urj66sBnHLA/std.out
total 1132
drwxr-xr-x 2 dteam011 dteam 4096 Dec 30 16:54 .
drwxr-xr-x 3 dteam011 dteam 4096 Dec 30 16:54 ..
-rw-r--r-- 1 dteam011 dteam 4037 Dec 30 16:54 .BrokerInfo
-rw-r--r-- 1 dteam011 dteam 2 Dec 30 16:54 fileA
-rw-r--r-- 1 dteam011 dteam 2 Dec 30 16:54 fileB
-rw-r--r-- 1 dteam011 dteam 1024000 Dec 30 16:54 fileC
-rw-r--r-- 1 dteam011 dteam 102400 Dec 30 16:54 fileD
-rw-r--r-- 1 dteam011 dteam 2 Dec 30 16:54 fileE
-rw-r--r-- 1 dteam011 dteam 0 Dec 30 16:54 std.err
-rw-r--r-- 1 dteam011 dteam 0 Dec 30 16:54 std.out
-rw-r--r-- 1 dteam011 dteam 0 Dec 30 16:54 .tmp_file
===> === test PASSED ===
Jdl with ExpiryTime
[capannini@devel15 certificazione]$ cat ExpiryTime.jdl
###########################################
# JDL with ExpiryTime attribute #
###########################################
Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";
InputSandbox = {"fileA", "fileB", "fileC", "fileD", "fileE"};
OutputSandbox = {"std.out", "std.err"};
requirements = false;
ExpiryTime = 1325261887;
RetryCount = 1;
ShallowRetryCount = 2;
[capannini@devel15 certificazione]$ date +%s
1325261324
[capannini@devel15 certificazione]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf ExpiryTime.jdl
Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
====================== glite-wms-job-submit Success ======================
The job has been successfully submitted to the WMProxy
Your job identifier is:
https://emi-demo11.cnaf.infn.it:9000/O6MWj0UPsVQblxDjgmo-gQ
==========================================================================
[capannini@devel15 certificazione]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/O6MWj0UPsVQblxDjgmo-gQ
======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/O6MWj0UPsVQblxDjgmo-gQ
Current Status: Waiting
Status Reason: BrokerHelper: no compatible resources
Submitted: Fri Dec 30 17:08:58 2011 CET
==========================================================================
[capannini@devel15 certificazione]$ date +%s
1325261928
[capannini@devel15 certificazione]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/O6MWj0UPsVQblxDjgmo-gQ
======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/O6MWj0UPsVQblxDjgmo-gQ
Current Status: Aborted
Status Reason: request expired
Submitted: Fri Dec 30 17:08:58 2011 CET
==========================================================================
===> === test PASSED ===
Jdl with ShortDeadlineJob
To test SDJs the following bdii must be used: egee-bdii.cnaf.infn.it
[capannini@devel15 certificazione]$ cat SDLJ.jdl
###########################################
# JDL with ShortDeadlineJob attribute #
###########################################
Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";
InputSandbox = {"fileA", "fileB", "fileC", "fileD", "fileE"};
OutputSandbox = {"std.out", "std.err"};
requirements = true;
ShortDeadlineJob = true;
RetryCount = 1;
ShallowRetryCount = 2;
[capannini@devel15 certificazione]$ glite-wms-job-list-match -a -c ~/wms_emi-demo11.conf SDLJ.jdl
Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
==========================================================================
COMPUTING ELEMENT IDs LIST
The following CE(s) matching your job requirements have been found:
*CEId*
- ce01.marie.hellasgrid.gr:2119/jobmanager-pbs-sdj
- cream-ce01.marie.hellasgrid.gr:8443/cream-pbs-sdj
- grid4.mif.vu.lt:2119/jobmanager-lcgpbs-sdj
- grive02.ibcp.fr:8443/cream-pbs-sdj
- mario.univ-lille1.fr:8443/cream-pbs-sdj
- ce01.ariagni.hellasgrid.gr:2119/jobmanager-pbs-dteam.sdj
- cream-ce01.ariagni.hellasgrid.gr:8443/cream-pbs-dteam.sdj
==========================================================================
[capannini@devel15 certificazione]$ python ~/wms_tests/WMS-service/WMS-job-cycle.py -c ~/wms_tests/WMS-service/wms-command.conf -V dteam -d 3 -l -t 1 -i -j SDLJ.jdl
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service
===> + Description: Test a complete job cycle: from submission to get output
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
Enter the user proxy password:
Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general)
Test 1A: Submit to an LCG-CE
Test 1B: Submit to a CREAM CE
Test 1C: Submit without restrictions
===>
===> Test: /home/capannini/wms_tests/WMS-service/WMS-job-cycle.py
===> WMS: emi-demo11.cnaf.infn.it
===> Started: 11:58:16
===> Ended : 12:04:06
===>
===> === test PASSED ===
===>
===> Test log file is WMSService-TS_20120116115816.log
Log is here: WMSService-TS_20120116115816.log: sdj log
Collection
Bulk submission
[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 2 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 2: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) Test 2A: Submit to an LCG-CE Test 2B: Submit to a CREAM CE Test 2C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 15:15:00 ===> Ended : 15:24:31 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120116151500.logSee log: WMSService-TS_20120116151500.log: bulk collection log
[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 2 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 2: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) Test 2A: Submit to an LCG-CE Test 2B: Submit to a CREAM CE Test 2C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 15:46:24 ===> Ended : 15:56:56 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120116154624.logSee log: WMSService-TS_20120116154624.log: bulk collection log
[capannini@devel15 WMS-service]$ myproxy-init -d Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini Enter GRID pass phrase for this identity: Creating proxy .................................................................. Done Proxy Verify OK Your proxy is valid until: Mon Jan 23 16:04:28 2012 Enter MyProxy pass phrase: Verifying - Enter MyProxy pass phrase: A proxy valid for 168 hours (7.0 days) for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini now exists on myproxy.cnaf.infn.it. [capannini@devel15 WMS-service]$ myproxy-logon -d --voms dteam Enter MyProxy pass phrase: Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy Contacting voms.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr] "dteam" Done Creating proxy ......................................................................................................................... Done Your proxy is valid until Tue Jan 17 03:32:47 2012 A credential has been received for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini in /tmp/x509up_u504. [capannini@devel15 WMS-service]$ voms-proxy-info -all subject : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy/CN=proxy issuer : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy identity : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy type : proxy strength : 2048 bits path : /tmp/x509up_u504 timeleft : 10:54:56 key usage : Digital Signature, Key Encipherment, Data Encipherment === VO dteam extension information === VO : dteam subject : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini issuer : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr attribute : /dteam/Role=NULL/Capability=NULL attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL timeleft : 11:53:55 uri : voms.hellasgrid.gr:15004 [capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -d 3 -l -t 2 ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Set 2: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) Test 2A: Submit to an LCG-CE Test 2B: Submit to a CREAM CE Test 2C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 16:32:57 ===> Ended : 16:41:22 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120116163257.logSee log: WMSService-TS_20120116163257.log: bulk collection log
All in a jdl
[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 7 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 7: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) using a single jdl with al the jdls of nodes Test 7A: Submit to an LCG-CE Test 7B: Submit to a CREAM CE Test 7C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 10:04:17 ===> Ended : 10:36:27 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120117100417.log [capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 7 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 7: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) using a single jdl with al the jdls of nodes Test 7A: Submit to an LCG-CE Test 7B: Submit to a CREAM CE Test 7C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 10:24:08 ===> Ended : 10:38:44 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120117102408.log [capannini@devel15 WMS-service]$ myproxy-init -d Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini Enter GRID pass phrase for this identity: Creating proxy ..................................................................................................................................................................................................... Done Proxy Verify OK Your proxy is valid until: Tue Jan 24 11:18:37 2012 Enter MyProxy pass phrase: Verifying - Enter MyProxy pass phrase: A proxy valid for 168 hours (7.0 days) for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini now exists on myproxy.cnaf.infn.it. [capannini@devel15 WMS-service]$ myproxy-logon -d --voms dteam Enter MyProxy pass phrase: Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy Contacting voms2.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr] "dteam" Done Creating proxy ............................................................................................................................................................................................................................................. Done Your proxy is valid until Tue Jan 17 22:19:16 2012 A credential has been received for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini in /tmp/x509up_u504. [capannini@devel15 WMS-service]$ voms-proxy-info -all subject : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy/CN=proxy issuer : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy identity : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy type : proxy strength : 2048 bits path : /tmp/x509up_u504 timeleft : 10:59:33 key usage : Digital Signature, Key Encipherment, Data Encipherment === VO dteam extension information === VO : dteam subject : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini issuer : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr attribute : /dteam/Role=NULL/Capability=NULL attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL timeleft : 11:58:32 uri : voms2.hellasgrid.gr:15004 [capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 7 ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Set 7: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) using a single jdl with al the jdls of nodes Test 7A: Submit to an LCG-CE Test 7B: Submit to a CREAM CE Test 7C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 11:19:59 ===> Ended : 11:29:26 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120117111959.logSee logs: coll.tgz: all in a jdl collection logs
Parametric job
[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 3 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 3: Submit a parametric job (3 cases LCG-CE, CREAM, general) Test 3A: Submit to an LCG-CE Test 3B: Submit to a CREAM CE Test 3C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 10:46:35 ===> Ended : 10:53:22 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120118104635.log [capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 3 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 3: Submit a parametric job (3 cases LCG-CE, CREAM, general) Test 3A: Submit to an LCG-CE Test 3B: Submit to a CREAM CE Test 3C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 10:47:30 ===> Ended : 10:53:46 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120118104730.log [capannini@devel15 WMS-service]$ myproxy-init -d Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini Enter GRID pass phrase for this identity: Creating proxy ............... Done Proxy Verify OK Your proxy is valid until: Wed Jan 25 10:48:21 2012 Enter MyProxy pass phrase: Verifying - Enter MyProxy pass phrase: A proxy valid for 168 hours (7.0 days) for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini now exists on myproxy.cnaf.infn.it. [capannini@devel15 WMS-service]$ [capannini@devel15 WMS-service]$ [capannini@devel15 WMS-service]$ [capannini@devel15 WMS-service]$ myproxy-logon -d --voms dteam Enter MyProxy pass phrase: Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy Contacting voms.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr] "dteam" Done Creating proxy ................................................................................................................................................ Done Your proxy is valid until Wed Jan 18 21:48:49 2012 A credential has been received for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini in /tmp/x509up_u504. [capannini@devel15 WMS-service]$ [capannini@devel15 WMS-service]$ [capannini@devel15 WMS-service]$ voms-proxy-info -all subject : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy/CN=proxy issuer : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy identity : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy type : proxy strength : 2048 bits path : /tmp/x509up_u504 timeleft : 10:59:51 key usage : Digital Signature, Key Encipherment, Data Encipherment === VO dteam extension information === VO : dteam subject : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini issuer : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr attribute : /dteam/Role=NULL/Capability=NULL attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL timeleft : 11:58:50 uri : voms.hellasgrid.gr:15004 [capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 3 ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Set 3: Submit a parametric job (3 cases LCG-CE, CREAM, general) Test 3A: Submit to an LCG-CE Test 3B: Submit to a CREAM CE Test 3C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 10:49:51 ===> Ended : 10:55:10 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120118104951.logSee logs: parametric.tgz: parametric logs
DAG job
[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 4 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 4: Submit a DAG job ===> ===> Test: WMS-job-cycle.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 12:05:40 ===> Ended : 12:10:56 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120118120540.log [capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 4 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 4: Submit a DAG job ===> ===> Test: WMS-job-cycle.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 12:06:00 ===> Ended : 12:11:15 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120118120600.log [capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 4 ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Set 4: Submit a DAG job ===> ===> Test: WMS-job-cycle.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 12:06:30 ===> Ended : 12:12:39 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120118120630.logSee logs: dag.tgz: dag logs
Parallel job
Setting II_Contact = "egee-bdii.cnaf.infn.it"; in glite_wms.conf and commenting #DEFAULTREQ= other.GlueCEStateStatus == "Testing"[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 5 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 5: Submit a MPI job (3 cases LCG-CE, CREAM, general) Test 5A: Submit to an LCG-CE Test 5B: Submit to a CREAM CE Test 5C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 16:23:09 ===> Ended : 16:27:29 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120118162309.log [capannini@devel15 WMS-service]$ voms-proxy-info -all subject : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy/CN=proxy issuer : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy identity : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy type : proxy strength : 2048 bits path : /tmp/x509up_u504 timeleft : 5:59:20 key usage : Digital Signature, Key Encipherment, Data Encipherment === VO dteam extension information === VO : dteam subject : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini issuer : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr attribute : /dteam/Role=NULL/Capability=NULL attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL timeleft : 6:58:19 uri : voms.hellasgrid.gr:15004 [capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 5 ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Set 5: Submit a MPI job (3 cases LCG-CE, CREAM, general) Test 5A: Submit to an LCG-CE Test 5B: Submit to a CREAM CE Test 5C: Submit without restrictions ===> ===> Test: WMS-job-cycle.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 16:23:22 ===> Ended : 16:29:12 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120118162322.logSee logs: parallel.tgz: parallel job logs
Testing forwarding parameters for parallel jobs
In order to test the passing of mpi parameters (wholenodes,smpgranularity,cpunumber,hostnumber) to the ce, an mpi jdl is submitted with varying values for the parameters, specifying a cream CE as destination. Then the cream jobid is retrieved via a glite-wms-job-logging-info -v 2 ("Dest jobid" field of a Transfer event).Finally the cream jdl is found via glite-ce-job-status -L 2 <cream-jobid>. The following jdls were submitted: 1)
Executable = "openmpi-wrapper.sh";
Arguments="hello";
StdOutput = "hello.out";
StdError = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
InputSandbox = {"/home/capannini/certification/mpi/openmpi-wrapper.sh","/home/capannini/certification/mpi/hello.c","/home/capannini/certification/mpi/Makefile"};
Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID);
WholeNodes = true;
SMPGranularity = 2;
CpuNumber=2;
Hostnumber = 1;
Resulting cream JDL:
JDL = [[ CEInfoHostName = "ce-qamar.utmgrid.utm.my"; OutputSandboxPath = "/var/SandboxDir/iY/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fiYM9Zl0NcKDqT1HqZiYN_5fg/output"; StdOutput = "hello.out"; BatchSystem = "pbs"; GlobusResourceContactString = "ce-qamar.utmgrid.utm.my:8443/cream-pbs"; QueueName = "dteam"; ShallowRetryCount = 10; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 3; Executable = "openmpi-wrapper.sh"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000011:BH=0000000000:JSS=000004:LM=000012:LRMS=000000:APP=000000:LBS=000000"; Arguments = "hello"; AllowZippedISB = true; maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank = -other.GlueCEStateEstimatedResponseTime; ce_id = "ce-qamar.utmgrid.utm.my:8443/cream-pbs-dteam"; ZippedISB = { "ISBfiles_Gmr4VWTb4-nGOH4-Bd092w_0.tar.gz" }; OutputSandboxDestURI = { "gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/iY/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fiYM9Zl0NcKDqT1HqZiYN_5fg/output/hello.out","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/iY/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fiYM9Zl0NcKDqT1HqZiYN_5fg/output/hello.err" }; OutputSandbox = { "hello.out","hello.err" }; CeRequirements = "( true && ( true && ( true && ( true && ( true && ( true && Member(\"OPENMPI\",other.GlueHostApplicationSoftwareRunTimeEnvironment) ) ) ) ) ) ) && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )"; InputSandbox = { "gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/iY/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fiYM9Zl0NcKDqT1HqZiYN_5fg/input/openmpi-wrapper.sh","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/iY/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fiYM9Zl0NcKDqT1HqZiYN_5fg/input/hello.c","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/iY/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fiYM9Zl0NcKDqT1HqZiYN_5fg/input/Makefile","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/iY/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fiYM9Zl0NcKDqT1HqZiYN_5fg/input/.BrokerInfo" }; edg_jobid = "https://emi-demo11.cnaf.infn.it:9000/iYM9Zl0NcKDqT1HqZiYN_g"; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini"; StdError = "hello.err"; SMPGranularity = 2; InputSandboxPath = "/var/SandboxDir/iY/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fiYM9Zl0NcKDqT1HqZiYN_5fg/input"; CeApplicationDir = "/opt/exp_soft/dteam"; ReallyRunningToken = "gsiftp://emi-demo11.cnaf.infn.it/var/SandboxDir/iY/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fiYM9Zl0NcKDqT1HqZiYN_5fg/token.txt_0"; rank = -other.GlueCEStateEstimatedResponseTime; WholeNodes = true; CpuNumber = 2; edg_previous_matches = { "gb-ce-emc.erasmusmc.nl" }; ReplansCount = 0; WMSHostname = "emi-demo11.cnaf.infn.it"; lrms_type = "torque"; MyProxyServer = "myproxy.cnaf.infn.it"; edg_previous_matches_ex = { [ timestamp = 1326965618; ce_id = "gb-ce-emc.erasmusmc.nl" ] }; X509UserProxy = "/var/SandboxDir/iY/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fiYM9Zl0NcKDqT1HqZiYN_5fg/user.proxy"; InputSandboxDestFileName = { "openmpi-wrapper.sh","hello.c","Makefile" }; Hostnumber = 1; requirements = ( ( ( ( ( ( Member("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*",other.GlueCEUniqueID) ) && ( other.GlueCEStateStatus == "Production" ) ) && other.GlueCEInfoTotalCPUs >= 2 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueCEInfoTotalCPUs >= 2 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) : !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/iY/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fiYM9Zl0NcKDqT1HqZiYN_5fg" ]]
2)
Executable = "openmpi-wrapper.sh";
Arguments="hello";
StdOutput = "hello.out";
StdError = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
InputSandbox = {"/home/capannini/certification/mpi/openmpi-wrapper.sh","/home/capannini/certification/mpi/hello.c","/home/capannini/certification/mpi/Makefile"};
Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID);
WholeNodes = true;
SMPGranularity = 2;
Resulting cream jdl:
JDL = [[ CEInfoHostName = "ce-02.roma3.infn.it"; OutputSandboxPath = "/var/SandboxDir/mR/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fmRQj0IPxfJ5ZjuBWrcDn1g/output"; StdOutput = "hello.out"; BatchSystem = "pbs"; GlobusResourceContactString = "ce-02.roma3.infn.it:8443/cream-pbs"; QueueName = "cert"; ShallowRetryCount = 10; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 3; Executable = "openmpi-wrapper.sh"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000017:BH=0000000000:JSS=000006:LM=000022:LRMS=000000:APP=000000:LBS=000000"; Arguments = "hello"; AllowZippedISB = true; maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank = -other.GlueCEStateEstimatedResponseTime; ce_id = "ce-02.roma3.infn.it:8443/cream-pbs-cert"; ZippedISB = { "ISBfiles_zpOEExFPS0a9q8TGstx7hQ_0.tar.gz" }; OutputSandboxDestURI = { "gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/mR/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fmRQj0IPxfJ5ZjuBWrcDn1g/output/hello.out","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/mR/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fmRQj0IPxfJ5ZjuBWrcDn1g/output/hello.err" }; OutputSandbox = { "hello.out","hello.err" }; CeRequirements = "( true && ( true && ( true && ( true && Member(\"OPENMPI\",other.GlueHostApplicationSoftwareRunTimeEnvironment) ) ) ) ) && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )"; InputSandbox = { "gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/mR/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fmRQj0IPxfJ5ZjuBWrcDn1g/input/openmpi-wrapper.sh","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/mR/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fmRQj0IPxfJ5ZjuBWrcDn1g/input/hello.c","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/mR/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fmRQj0IPxfJ5ZjuBWrcDn1g/input/Makefile","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/mR/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fmRQj0IPxfJ5ZjuBWrcDn1g/input/.BrokerInfo" }; edg_jobid = "https://emi-demo11.cnaf.infn.it:9000/mRQj0IPxfJ5ZjuBWrcDn1g"; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini"; StdError = "hello.err"; SMPGranularity = 2; InputSandboxPath = "/var/SandboxDir/mR/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fmRQj0IPxfJ5ZjuBWrcDn1g/input"; CeApplicationDir = "/storage/local/exp_soft/dteam"; ReallyRunningToken = "gsiftp://emi-demo11.cnaf.infn.it/var/SandboxDir/mR/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fmRQj0IPxfJ5ZjuBWrcDn1g/token.txt_0"; rank = -other.GlueCEStateEstimatedResponseTime; WholeNodes = true; edg_previous_matches = { "ce.scope.unina.it","grisuce.scope.unina.it" }; ReplansCount = 0; WMSHostname = "emi-demo11.cnaf.infn.it"; lrms_type = "pbs"; MyProxyServer = "myproxy.cnaf.infn.it"; edg_previous_matches_ex = { [ timestamp = 1326967114; ce_id = "ce.scope.unina.it" ],[ timestamp = 1326967117; ce_id = "grisuce.scope.unina.it" ] }; X509UserProxy = "/var/SandboxDir/mR/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fmRQj0IPxfJ5ZjuBWrcDn1g/user.proxy"; InputSandboxDestFileName = { "openmpi-wrapper.sh","hello.c","Makefile" }; requirements = ( ( ( ( Member("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*",other.GlueCEUniqueID) ) && ( other.GlueCEStateStatus == "Production" ) ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) : !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/mR/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fmRQj0IPxfJ5ZjuBWrcDn1g" ]]
3)
Executable = "openmpi-wrapper.sh";
Arguments="hello";
StdOutput = "hello.out";
StdError = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
InputSandbox = {"/home/capannini/certification/mpi/openmpi-wrapper.sh","/home/capannini/certification/mpi/hello.c","/home/capannini/certification/mpi/Makefile"};
Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID);
WholeNodes = false;
SMPGranularity = 3;
CpuNumber=3;
Hostnumber = 1;
Submission result:
[capannini@devel15 mpi]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf mpi.jdl
Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
Error -
SMPGranularity and HostNumber are mutually exclusive when WholeNodes allocation is not requested: wrong combination of values
4)
Executable = "openmpi-wrapper.sh";
Arguments="hello";
StdOutput = "hello.out";
StdError = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
InputSandbox = {"/home/capannini/certification/mpi/openmpi-wrapper.sh","/home/capannini/certification/mpi/hello.c","/home/capannini/certification/mpi/Makefile"};
Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID);
WholeNodes = false;
SMPGranularity = 3;
CpuNumber=3;
Resulting cream jdl:
JDL = [[ CEInfoHostName = "ce-02.roma3.infn.it"; OutputSandboxPath = "/var/SandboxDir/l-/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fl-na1nvxx06q_5fy6klpEnQQ/output"; StdOutput = "hello.out"; BatchSystem = "pbs"; GlobusResourceContactString = "ce-02.roma3.infn.it:8443/cream-pbs"; QueueName = "cert"; ShallowRetryCount = 10; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 3; Executable = "openmpi-wrapper.sh"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000017:BH=0000000000:JSS=000006:LM=000022:LRMS=000000:APP=000000:LBS=000000"; Arguments = "hello"; AllowZippedISB = true; maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank = -other.GlueCEStateEstimatedResponseTime; ce_id = "ce-02.roma3.infn.it:8443/cream-pbs-cert"; ZippedISB = { "ISBfiles_gQrxvLonwNUGPUr_WRZxrg_0.tar.gz" }; OutputSandboxDestURI = { "gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/l-/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fl-na1nvxx06q_5fy6klpEnQQ/output/hello.out","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/l-/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fl-na1nvxx06q_5fy6klpEnQQ/output/hello.err" }; OutputSandbox = { "hello.out","hello.err" }; CeRequirements = "( true && ( true && ( true && ( true && ( true && ( true && Member(\"OPENMPI\",other.GlueHostApplicationSoftwareRunTimeEnvironment) ) ) ) ) ) ) && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )"; InputSandbox = { "gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/l-/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fl-na1nvxx06q_5fy6klpEnQQ/input/openmpi-wrapper.sh","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/l-/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fl-na1nvxx06q_5fy6klpEnQQ/input/hello.c","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/l-/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fl-na1nvxx06q_5fy6klpEnQQ/input/Makefile","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/l-/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fl-na1nvxx06q_5fy6klpEnQQ/input/.BrokerInfo" }; edg_jobid = "https://emi-demo11.cnaf.infn.it:9000/l-na1nvxx06q_y6klpEnQQ"; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini"; StdError = "hello.err"; SMPGranularity = 3; InputSandboxPath = "/var/SandboxDir/l-/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fl-na1nvxx06q_5fy6klpEnQQ/input"; CeApplicationDir = "/storage/local/exp_soft/dteam"; ReallyRunningToken = "gsiftp://emi-demo11.cnaf.infn.it/var/SandboxDir/l-/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fl-na1nvxx06q_5fy6klpEnQQ/token.txt_0"; rank = -other.GlueCEStateEstimatedResponseTime; WholeNodes = false; CpuNumber = 3; edg_previous_matches = { "grisuce.scope.unina.it","ce.scope.unina.it" }; ReplansCount = 0; WMSHostname = "emi-demo11.cnaf.infn.it"; lrms_type = "pbs"; MyProxyServer = "myproxy.cnaf.infn.it"; edg_previous_matches_ex = { [ timestamp = 1326968771; ce_id = "grisuce.scope.unina.it" ],[ timestamp = 1326968773; ce_id = "ce.scope.unina.it" ] }; X509UserProxy = "/var/SandboxDir/l-/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fl-na1nvxx06q_5fy6klpEnQQ/user.proxy"; InputSandboxDestFileName = { "openmpi-wrapper.sh","hello.c","Makefile" }; requirements = ( ( ( ( ( ( Member("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*",other.GlueCEUniqueID) ) && ( other.GlueCEStateStatus == "Production" ) ) && other.GlueCEInfoTotalCPUs >= 3 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueCEInfoTotalCPUs >= 3 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) : !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/l-/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fl-na1nvxx06q_5fy6klpEnQQ" ]]
5)
Executable = "openmpi-wrapper.sh";
Arguments="hello";
StdOutput = "hello.out";
StdError = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
InputSandbox = {"/home/capannini/certification/mpi/openmpi-wrapper.sh","/home/capannini/certification/mpi/hello.c","/home/capannini/certification/mpi/Makefile"};
Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID);
WholeNodes = true;
SMPGranularity = 8;
CpuNumber=8;
Hostnumber = 2;
Resulting ce jdl:
JDL = [[ CEInfoHostName = "ce-02.roma3.infn.it"; OutputSandboxPath = "/var/SandboxDir/Oc/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fOc3Gbgez4On6im_5fB63zOkQ/output"; StdOutput = "hello.out"; BatchSystem = "pbs"; GlobusResourceContactString = "ce-02.roma3.infn.it:8443/cream-pbs"; QueueName = "cert"; ShallowRetryCount = 10; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 3; Executable = "openmpi-wrapper.sh"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000005:BH=0000000000:JSS=000002:LM=000002:LRMS=000000:APP=000000:LBS=000000"; Arguments = "hello"; AllowZippedISB = true; maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank = -other.GlueCEStateEstimatedResponseTime; ce_id = "ce-02.roma3.infn.it:8443/cream-pbs-cert"; ZippedISB = { "ISBfiles_dz-M5EA3o9uj857y8CkqMg_0.tar.gz" }; OutputSandboxDestURI = { "gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/Oc/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fOc3Gbgez4On6im_5fB63zOkQ/output/hello.out","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/Oc/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fOc3Gbgez4On6im_5fB63zOkQ/output/hello.err" }; OutputSandbox = { "hello.out","hello.err" }; CeRequirements = "( true && ( true && ( true && ( true && ( true && ( true && Member(\"OPENMPI\",other.GlueHostApplicationSoftwareRunTimeEnvironment) ) ) ) ) ) ) && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )"; InputSandbox = { "gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/Oc/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fOc3Gbgez4On6im_5fB63zOkQ/input/openmpi-wrapper.sh","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/Oc/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fOc3Gbgez4On6im_5fB63zOkQ/input/hello.c","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/Oc/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fOc3Gbgez4On6im_5fB63zOkQ/input/Makefile","gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/Oc/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fOc3Gbgez4On6im_5fB63zOkQ/input/.BrokerInfo" }; edg_jobid = "https://emi-demo11.cnaf.infn.it:9000/Oc3Gbgez4On6im_B63zOkQ"; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini"; StdError = "hello.err"; SMPGranularity = 8; InputSandboxPath = "/var/SandboxDir/Oc/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fOc3Gbgez4On6im_5fB63zOkQ/input"; CeApplicationDir = "/storage/local/exp_soft/dteam"; ReallyRunningToken = "gsiftp://emi-demo11.cnaf.infn.it/var/SandboxDir/Oc/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fOc3Gbgez4On6im_5fB63zOkQ/token.txt_0"; rank = -other.GlueCEStateEstimatedResponseTime; WholeNodes = true; CpuNumber = 8; WMSHostname = "emi-demo11.cnaf.infn.it"; lrms_type = "pbs"; MyProxyServer = "myproxy.cnaf.infn.it"; X509UserProxy = "/var/SandboxDir/Oc/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fOc3Gbgez4On6im_5fB63zOkQ/user.proxy"; InputSandboxDestFileName = { "openmpi-wrapper.sh","hello.c","Makefile" }; Hostnumber = 2; requirements = ( ( ( ( ( ( Member("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*",other.GlueCEUniqueID) ) && ( other.GlueCEStateStatus == "Production" ) ) && other.GlueCEInfoTotalCPUs >= 8 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueCEInfoTotalCPUs >= 8 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) : !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://emi-demo11.cnaf.infn.it:2811/var/SandboxDir/Oc/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fOc3Gbgez4On6im_5fB63zOkQ" ]]
--> Test PASSED
Perusal
[capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 6 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 6: Submit a perusal job BEWARE default min perusal interval is 1000 secs, so this phase could take many minutes ===> ===> Test: WMS-job-cycle.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 10:12:07 ===> Ended : 10:46:35 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120120101207.log [capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 6 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 6: Submit a perusal job BEWARE default min perusal interval is 1000 secs, so this phase could take many minutes ===> ===> Test: WMS-job-cycle.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 14:28:05 ===> Ended : 14:55:34 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120120142805.log [capannini@devel15 WMS-service]$ voms-proxy-info -all subject : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy/CN=proxy issuer : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy identity : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy type : proxy strength : 2048 bits path : /tmp/x509up_u504 timeleft : 6:31:39 key usage : Digital Signature, Key Encipherment, Data Encipherment === VO dteam extension information === VO : dteam subject : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini issuer : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr attribute : /dteam/Role=NULL/Capability=NULL attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL timeleft : 7:30:39 uri : voms2.hellasgrid.gr:15004 [capannini@devel15 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -d 3 -l -t 6 ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Set 6: Submit a perusal job BEWARE default min perusal interval is 1000 secs, so this phase could take many minutes ===> ===> Test: WMS-job-cycle.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 14:58:43 ===> Ended : 15:26:03 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120120145843.logSee logs: perusal.tgz: perusal logs
Delegation
[capannini@devel15 WMS-service]$ python WMS-delegation-test.py -c wms-command.conf -V dteam -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test delegation operation ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: Check --autm-delegation option Test 2: Check --delegationid option Test 3: Try to delegate with a short proxy and check the validity Test 4: Works with expired proxy ===> ===> Test: WMS-delegation-test.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 10:22:35 ===> Ended : 10:23:49 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120120102235.log [capannini@devel15 WMS-service]$ python WMS-delegation-test.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test delegation operation ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: Check --autm-delegation option Test 2: Check --delegationid option Test 3: Try to delegate with a short proxy and check the validity Test 4: Works with expired proxy ===> ===> Test: WMS-delegation-test.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 10:25:00 ===> Ended : 10:26:14 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120120102500.log [capannini@devel15 WMS-service]$ myproxy-init -d Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini Enter GRID pass phrase for this identity: Creating proxy ............................... Done Proxy Verify OK Your proxy is valid until: Fri Jan 27 10:27:27 2012 Enter MyProxy pass phrase: Verifying - Enter MyProxy pass phrase: A proxy valid for 168 hours (7.0 days) for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini now exists on myproxy.cnaf.infn.it. [capannini@devel15 WMS-service]$ [capannini@devel15 WMS-service]$ [capannini@devel15 WMS-service]$ myproxy-logon -d --voms dteam Enter MyProxy pass phrase: Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy Contacting voms2.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr] "dteam" Done Creating proxy .................... Done Your proxy is valid until Fri Jan 20 21:27:59 2012 A credential has been received for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini in /tmp/x509up_u504. [capannini@devel15 WMS-service]$ [capannini@devel15 WMS-service]$ [capannini@devel15 WMS-service]$ voms-proxy-info -all subject : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy/CN=proxy issuer : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy identity : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy type : proxy strength : 2048 bits path : /tmp/x509up_u504 timeleft : 10:59:47 key usage : Digital Signature, Key Encipherment, Data Encipherment === VO dteam extension information === VO : dteam subject : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini issuer : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr attribute : /dteam/Role=NULL/Capability=NULL attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL timeleft : 11:58:47 uri : voms2.hellasgrid.gr:15004 [capannini@devel15 WMS-service]$ [capannini@devel15 WMS-service]$ [capannini@devel15 WMS-service]$ python WMS-delegation-test.py -c wms-command.conf -d 3 -l ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test delegation operation ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: Check --autm-delegation option Test 2: Check --delegationid option There are other two tests which require the user proxy password. Use -i option to enable them ===> ===> Test: WMS-delegation-test.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 10:28:43 ===> Ended : 10:28:46 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120120102843.logSee logs: delegation.tgz: delegation logs
Job List-match
Without data
[capannini@devel15 WMS-service]$ python WMS-listmatch-test.py -c wms-command.conf -V dteam -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Job ListMatch Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: check if glite-wms-job-list-match command exists Test 2: test a simple matching Test 3: check --delegationid option Test 4: exclude a CE from the match Test 5: try a failure matching (Requirements == false) Test 6: try a matching with EnableWMSFeedback attribute true ===> ===> Test: WMS-listmatch-test.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 10:49:06 ===> Ended : 10:49:18 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120123104906.logSee log: WMSService-TS_20120123104906.log: list-match log
[capannini@devel15 WMS-service]$ python WMS-listmatch-test.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Job ListMatch Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: check if glite-wms-job-list-match command exists Test 2: test a simple matching Test 3: check --delegationid option Test 4: exclude a CE from the match Test 5: try a failure matching (Requirements == false) Test 6: try a matching with EnableWMSFeedback attribute true ===> ===> Test: WMS-listmatch-test.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 12:10:55 ===> Ended : 12:11:07 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120123121055.logSee logs: WMSService-TS_20120123121055.log: list-match log
[capannini@devel15 WMS-service]$ myproxy-init -d Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini Enter GRID pass phrase for this identity: Creating proxy .................................................................. Done Proxy Verify OK Your proxy is valid until: Mon Jan 30 12:13:56 2012 Enter MyProxy pass phrase: Verifying - Enter MyProxy pass phrase: A proxy valid for 168 hours (7.0 days) for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini now exists on myproxy.cnaf.infn.it. [capannini@devel15 WMS-service]$ myproxy-logon -d --voms dteam Enter MyProxy pass phrase: Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy Contacting voms.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr] "dteam" Done Creating proxy ....................... Done Your proxy is valid until Mon Jan 23 23:14:19 2012 A credential has been received for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini in /tmp/x509up_u504. [capannini@devel15 WMS-service]$ [capannini@devel15 WMS-service]$ voms-proxy-info -all subject : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy/CN=proxy issuer : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy identity : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy type : proxy strength : 2048 bits path : /tmp/x509up_u504 timeleft : 10:59:52 key usage : Digital Signature, Key Encipherment, Data Encipherment === VO dteam extension information === VO : dteam subject : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini issuer : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr attribute : /dteam/Role=NULL/Capability=NULL attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL timeleft : 11:58:52 uri : voms.hellasgrid.gr:15004 [capannini@devel15 WMS-service]$ python WMS-listmatch-test.py -c wms-command.conf -d 3 -l ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Job ListMatch Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: check if glite-wms-job-list-match command exists Test 2: test a simple matching Test 3: check --delegationid option Test 4: exclude a CE from the match Test 5: try a failure matching (Requirements == false) Test 6: try a matching with EnableWMSFeedback attribute true ===> ===> Test: WMS-listmatch-test.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 12:15:09 ===> Ended : 12:15:15 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120123121509.logSee logs: WMSService-TS_20120123121509.log: list-match log
With data
[fabiok@ui ~]$ export LFC_HOST=prod-lfc-shared-central.cern.ch
[fabiok@ui ~]$ lcg-lr --vo dteam lfn:/grid/dteam/fcapanni/e-2M.txt
srm://gridsrm.pi.infn.it/dteam/fcapanni/e-2M.txt
srm://srm-public.cern.ch/castor/cern.ch/grid/dteam/fcapanni/e-2M.txt
The two SEs where the file can be found are gridsrm.pi.infn.it and srm-public.cern.ch.
[capannini@devel15 WMS-service]$ cat JDLs/data-req.jdl
###########################################
# JDL with Data Requirements #
###########################################
Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";
Prologue = "prologue.sh";
FuzzyRank = true;
Environment={"LFC_HOST=prod-lfc-shared-central.cern.ch", "LFC_HOME=prod-lfc-shared-central.cern.ch:/grid/dteam/aleph"};
InputSandbox = {"JDLs/prologue.sh"};
OutputSandbox = {"std.out", "std.err","prologue.out", "PI_1M.txt"};
DataRequirements = {
[
DataCatalogType = "DLI";
DataCatalog ="http://prod-lfc-shared-central.cern.ch:8085/";
InputData = { "lfn:/grid/dteam/fcapanni/PI_1M.txt" };
]
};
DataAccessProtocol = "gsiftp";
AllowZippedISB=true;
RetryCount = 1;
ShallowRetryCount = 2;
[capannini@devel15 WMS-service]$ glite-wms-job-list-match -a -c ~/wms_emi-demo11.conf JDLs/data-req.jdl
Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
==========================================================================
COMPUTING ELEMENT IDs LIST
The following CE(s) matching your job requirements have been found:
*CEId*
- ce130.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam
- ce130.cern.ch:2119/jobmanager-lcglsf-grid_dteam
- ce131.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam
- ce131.cern.ch:2119/jobmanager-lcglsf-grid_dteam
- ce132.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam
- ce132.cern.ch:2119/jobmanager-lcglsf-grid_dteam
- ce133.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam
- ce133.cern.ch:2119/jobmanager-lcglsf-grid_dteam
- ce203.cern.ch:8443/cream-lsf-grid_2nh_dteam
- ce203.cern.ch:8443/cream-lsf-grid_dteam
- ce204.cern.ch:8443/cream-lsf-grid_2nh_dteam
- ce204.cern.ch:8443/cream-lsf-grid_dteam
- ce205.cern.ch:8443/cream-lsf-grid_2nh_dteam
- ce205.cern.ch:8443/cream-lsf-grid_dteam
- ce206.cern.ch:8443/cream-lsf-grid_2nh_dteam
- ce206.cern.ch:8443/cream-lsf-grid_dteam
- ce207.cern.ch:8443/cream-lsf-grid_2nh_dteam
- ce207.cern.ch:8443/cream-lsf-grid_dteam
- ce208.cern.ch:8443/cream-lsf-grid_2nh_dteam
- ce208.cern.ch:8443/cream-lsf-grid_dteam
- cream.sns.it:8443/cream-pbs-cert
- gridce0.pi.infn.it:8443/cream-lsf-cert
- gridce1.pi.infn.it:2119/jobmanager-lcglsf-cert
- gridce2.pi.infn.it:2119/jobmanager-lcglsf-cert
- gridce3.pi.infn.it:8443/cream-lsf-certmpi
- gridce.sns.it:2119/jobmanager-lcgpbs-cert
==========================================================================
In order to verify that these CEs are close to the above SEs:
[fabiok@ui ~]$ lcg-infosites --vo dteam closeSE|grep -A 5 ce130
Name of the CE: ce130.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam
srm-alice.cern.ch
srm-atlas.cern.ch
srm-cms.cern.ch
srm-lhcb.cern.ch
srm-public.cern.ch
--
Name of the CE: ce130.cern.ch:2119/jobmanager-lcglsf-grid_dteam
srm-alice.cern.ch
srm-atlas.cern.ch
srm-cms.cern.ch
srm-lhcb.cern.ch
srm-public.cern.ch
[fabiok@ui ~]$ lcg-infosites --vo dteam closeSE|grep -A 5 ce208.cern.ch
Name of the CE: ce208.cern.ch:8443/cream-lsf-grid_2nh_dteam
srm-alice.cern.ch
srm-atlas.cern.ch
srm-cms.cern.ch
srm-lhcb.cern.ch
srm-public.cern.ch
--
Name of the CE: ce208.cern.ch:8443/cream-lsf-grid_dteam
srm-alice.cern.ch
srm-atlas.cern.ch
srm-cms.cern.ch
srm-lhcb.cern.ch
srm-public.cern.ch
[fabiok@ui ~]$ lcg-infosites --vo dteam closeSE|grep -A 5 cream.sns.it
Name of the CE: cream.sns.it:8443/cream-pbs-cert
gridsrm.pi.infn.it
[fabiok@ui ~]$ lcg-infosites --vo dteam closeSE|grep -A 5 gridce.sns.it
Name of the CE: gridce.sns.it:2119/jobmanager-lcgpbs-cert
gridsrm.pi.infn.it
===> Test PASSED
Gangmatching
[capannini@devel15 certification]$ cat gang.jdl
[
executable="ls.sh";
myproxyserver="";
SErequirements = anyMatch(other.storage.CloseSEs, target.GlueSAStateAvailableSpace > 20);
CErequirements = other.GlueCEStateStatus == "Production" && other.GlueCEInfoTotalCPUs >= 2;
requirements = CErequirements && SErequirements;
inputsandbox={"ls.sh"};
stdoutput="out3.out";
stderror="err2.err";
outputsandbox={"out3.out", "err2.err", "ls.sh", "a"};
]
[capannini@devel15 certification]$ glite-wms-job-list-match -a -c ~/wms_emi-demo11.conf gang.jdl
Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
==========================================================================
COMPUTING ELEMENT IDs LIST
The following CE(s) matching your job requirements have been found:
*CEId*
- cccreamceli05.in2p3.fr:8443/cream-sge-long
- cccreamceli05.in2p3.fr:8443/cream-sge-medium
- cccreamceli05.in2p3.fr:8443/cream-sge-short
- cccreamceli09.in2p3.fr:8443/cream-sge-long
- cccreamceli09.in2p3.fr:8443/cream-sge-medium
- cccreamceli09.in2p3.fr:8443/cream-sge-short
- cmsce01.na.infn.it:2119/jobmanager-lcgpbs-cert
- cmsrm-ce01.roma1.infn.it:2119/jobmanager-lcglsf-cmsgcert
- cmsrm-cream01.roma1.infn.it:8443/cream-lsf-cmsgcert
- grid010.ct.infn.it:2119/jobmanager-lcgpbs-cert
- inaf-ce-01.ct.pi2s2.it:8443/cream-lsf-cert
- unict-dmi-ce-01.ct.pi2s2.it:2119/jobmanager-lcglsf-cert
- virgo-ce.roma1.infn.it:2119/jobmanager-lcgpbs-cert
- ce-01.grid.sissa.it:8443/cream-pbs-blade
- cccreamceli10.in2p3.fr:8443/cream-sge-long
- cccreamceli10.in2p3.fr:8443/cream-sge-medium
- cccreamceli10.in2p3.fr:8443/cream-sge-short
- cmsrm-ce02.roma1.infn.it:2119/jobmanager-lcglsf-cmsgcert
- cccreamceli06.in2p3.fr:8443/cream-sge-long
- cccreamceli06.in2p3.fr:8443/cream-sge-medium
- cccreamceli06.in2p3.fr:8443/cream-sge-short
- cream-ce.pg.infn.it:8443/cream-pbs-cert
==========================================================================
In order to verify that the SErequirements are met:
[capannini@devel15 certification]$ lcg-infosites closeSE|grep -A 5 cccreamceli05.in2p3.fr
Name of the CE: cccreamceli05.in2p3.fr:8443/cream-sge-long
ccsrm.in2p3.fr
[capannini@devel15 certification]$ lcg-infosites se |egrep 'Avail|ccsrm.in2p3.fr'
Avail Space(kB) Used Space(kB) Type SE
11385302023 3647083513 SRM ccsrm.in2p3.fr
[capannini@devel15 certification]$ lcg-infosites closeSE|grep -A 5 cmsce01.na.infn.it
Name of the CE: cmsce01.na.infn.it:2119/jobmanager-lcgpbs-cert
cmsse02.na.infn.it
Name of the CE: cmsrm-ce01.roma1.infn.it:2119/jobmanager-lcglsf-cmsgcert
cmsrm-se01.roma1.infn.it
[capannini@devel15 certification]$ lcg-infosites se |egrep 'Avail|cmsse02.na.infn.it'
Avail Space(kB) Used Space(kB) Type SE
25271855845 690913963 SRM cmsse02.na.infn.it
[capannini@devel15 certification]$ lcg-infosites closeSE|grep -A 5 cmsrm-ce02.roma1.infn.it
Name of the CE: cmsrm-ce02.roma1.infn.it:2119/jobmanager-lcglsf-cmsgcert
cmsrm-se01.roma1.infn.it
Name of the CE: cmsrm-cream01.roma1.infn.it:8443/cream-lsf-cmsgcert
cmsrm-se01.roma1.infn.it
[capannini@devel15 certification]$ lcg-infosites se |egrep 'Avail|cmsrm-se01.roma1.infn.it'
Avail Space(kB) Used Space(kB) Type SE
1000 999999 SRM cmsrm-se01.roma1.infn.it
Moreover, setting SErequirements = true in the jdl, the list of matching CEs is far longer.
===> Test PASSED
Prologue and Epilogue jobs
[capannini@devel15 WMS-service]$ python WMS-prologue-epilogue.py -c wms-command.conf -V dteam -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS test prologue and epilogue attributes ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: Test prologue attribute Test 2: Test epilogue attribute Test 3: Test prologue and epilogue attributes ===> ===> Test: WMS-prologue-epilogue.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 16:05:57 ===> Ended : 16:11:17 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120123160557.log [capannini@devel15 WMS-service]$ python WMS-prologue-epilogue.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS test prologue and epilogue attributes ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: Test prologue attribute Test 2: Test epilogue attribute Test 3: Test prologue and epilogue attributes ===> ===> Test: WMS-prologue-epilogue.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 16:18:49 ===> Ended : 16:25:09 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120123161849.log [capannini@devel15 WMS-service]$ [capannini@devel15 WMS-service]$ [capannini@devel15 WMS-service]$ voms-proxy-info -all subject : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy/CN=proxy issuer : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy identity : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy type : proxy strength : 2048 bits path : /tmp/x509up_u504 timeleft : 6:43:33 key usage : Digital Signature, Key Encipherment, Data Encipherment === VO dteam extension information === VO : dteam subject : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini issuer : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr attribute : /dteam/Role=NULL/Capability=NULL attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL timeleft : 7:42:33 uri : voms.hellasgrid.gr:15004 [capannini@devel15 WMS-service]$ [capannini@devel15 WMS-service]$ python WMS-prologue-epilogue.py -c wms-command.conf -d 3 -l ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS test prologue and epilogue attributes ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: Test prologue attribute Test 2: Test epilogue attribute Test 3: Test prologue and epilogue attributes ===> ===> Test: WMS-prologue-epilogue.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 16:30:56 ===> Ended : 16:37:13 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120123163056.logSee logs: prolepilog.tgz: prologue epilog logs
Shallow and deep resubmission
[capannini@devel15 WMS-service]$ python WMS-job-resubmit.py -c wms-command.conf -V dteam -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Job Resubmission Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: Try a shallow resubmission Test 2: Try a deep resubmission ===> ===> Test: WMS-job-resubmit.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 10:16:38 ===> Ended : 10:28:49 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120124101638.log [capannini@devel15 WMS-service]$ python WMS-job-resubmit.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Job Resubmission Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: Try a shallow resubmission Test 2: Try a deep resubmission ===> ===> Test: WMS-job-resubmit.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 10:32:35 ===> Ended : 10:45:18 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120124103235.log [capannini@devel15 WMS-service]$ myproxy-init -d Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini Enter GRID pass phrase for this identity: Creating proxy ............................................................................................................................................. Done Proxy Verify OK Your proxy is valid until: Tue Jan 31 10:39:47 2012 Enter MyProxy pass phrase: Verifying - Enter MyProxy pass phrase: A proxy valid for 168 hours (7.0 days) for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini now exists on myproxy.cnaf.infn.it. [capannini@devel15 WMS-service]$ [capannini@devel15 WMS-service]$ [capannini@devel15 WMS-service]$ [capannini@devel15 WMS-service]$ myproxy-logon -d --voms dteam Enter MyProxy pass phrase: Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy Contacting voms.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr] "dteam" Done Creating proxy ................................................................................................................................................. Done Your proxy is valid until Tue Jan 24 21:40:11 2012 A credential has been received for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini in /tmp/x509up_u504. [capannini@devel15 WMS-service]$ [capannini@devel15 WMS-service]$ [capannini@devel15 WMS-service]$ voms-proxy-info -all subject : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy/CN=proxy issuer : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy identity : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy/CN=proxy/CN=proxy type : proxy strength : 2048 bits path : /tmp/x509up_u504 timeleft : 10:59:51 key usage : Digital Signature, Key Encipherment, Data Encipherment === VO dteam extension information === VO : dteam subject : /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini issuer : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr attribute : /dteam/Role=NULL/Capability=NULL attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL timeleft : 11:58:50 uri : voms.hellasgrid.gr:15004 [capannini@devel15 WMS-service]$ python WMS-job-resubmit.py -c wms-command.conf -d 3 -l ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Job Resubmission Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: Try a shallow resubmission Test 2: Try a deep resubmission ===> ===> Test: WMS-job-resubmit.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 10:40:39 ===> Ended : 10:53:15 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120124104039.logSee logs: resub.tgz: resubmission logs
Job cancel
[capannini@devel15 WMS-service]$ python WMS-job-cancel.py -c wms-command.conf -V dteam -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Job Cancel Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: Try to cancel a normal job Test 2: Try to cancel a DAG job Test 3: Try to cancel one node of a DAG job Test 4: Try to cancel a collection job Test 5: Try to cancel one node of a collection job Test 6: Try to cancel a parametric job Test 7: Try to cancel one node of a parametric job Test 8: Try to cancel a MPI job ===> ===> Test: WMS-job-cancel.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 10:53:04 ===> Ended : 11:02:03 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120124105304.log [capannini@devel15 WMS-service]$ python WMS-job-cancel.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Job Cancel Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: Try to cancel a normal job Test 2: Try to cancel a DAG job Test 3: Try to cancel one node of a DAG job Test 4: Try to cancel a collection job Test 5: Try to cancel one node of a collection job Test 6: Try to cancel a parametric job Test 7: Try to cancel one node of a parametric job Test 8: Try to cancel a MPI job ===> ===> Test: WMS-job-cancel.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 10:43:32 ===> Ended : 10:52:08 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120124104332.log [capannini@devel15 WMS-service]$ python WMS-job-cancel.py -c wms-command.conf -d 3 -l ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Job Cancel Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: Try to cancel a normal job Test 2: Try to cancel a DAG job Test 3: Try to cancel one node of a DAG job Test 4: Try to cancel a collection job Test 5: Try to cancel one node of a collection job Test 6: Try to cancel a parametric job Test 7: Try to cancel one node of a parametric job Test 8: Try to cancel a MPI job ===> ===> Test: WMS-job-cancel.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 10:52:46 ===> Ended : 10:57:43 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120124105246.logSee logs: cancel.tgz: Job cancel logs
Proxy renewal
Setting MYPROXYSERVER=emitb2.ics.muni.cz in wms-command.conf [capannini@devel15 ~]$ voms-proxy-init --voms dteam Enter GRID pass phrase: Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini Creating temporary proxy .............................................. Done Contacting voms2.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr] "dteam" Done Creating proxy ............................................... Done Your proxy is valid until Thu Jan 26 22:42:50 2012 [capannini@devel15 ~]$ myproxy-init -s emitb2.ics.muni.cz -d -n Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini Enter GRID pass phrase for this identity: Creating proxy .......................................................................................................................................................................................................................................................................... Done Proxy Verify OK Your proxy is valid until: Thu Feb 2 10:44:17 2012 A proxy valid for 168 hours (7.0 days) for user /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini now exists on emitb2.ics.muni.cz. [capannini@devel15 WMS-service]$ python WMS-proxy-renewal.py -c wms-command.conf -V dteam -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS test proxy renewal operation ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: Test proxy renewal with LCG CE Test 2: Test proxy renewal with CREAM CE Test 3: Test proxy renewal with LCG CE (without setting MYPROXYSERVER) Test 4: Test proxy renewal with CREAM CE (without setting MYPROXYSERVER) ===> ===> Test: WMS-proxy-renewal.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 10:47:52 ===> Ended : 11:29:27 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120126104752.logSee log: WMSService-TS_20120126104752.log: proxy renewal log
Configuration file
Verified that the following entries are present in/etc/glite-wms/glite_wms.conf :
ICE section
ICE = [
job_cancellation_threshold_time = 300;
max_logfile_size = 100*1024*1024;
cream_url_prefix = "https://";
start_subscription_updater = true;
max_ice_mem = 2096000;
InputType = "jobdir";
subscription_update_threshold_time = 3600;
ice_log_level = 700;
listener_enable_authn = true;
listener_enable_authz = true;
poller_status_threshold_time = 30*60;
lease_update_frequency = 20*60;
lease_delta_time = 0;
cream_url_postfix = "/ce-cream/services/CREAM2";
subscription_duration = 86400;
start_lease_updater = false;
notification_frequency = 3*60;
creamdelegation_url_postfix = "/ce-cream/services/gridsite-delegation";
ice_host_cert = "/home/glite/.certs/hostcert.pem";
cemon_url_postfix = "/ce-monitor/services/CEMonitor";
ice_topic = "CREAM_JOBS";
start_proxy_renewer = true;
Input = "/var/ice/jobdir";
max_logfile_rotations = 10;
start_poller = true;
listener_port = 7010;
poller_delay = 2*60;
creamdelegation_url_prefix = "https://";
purge_jobs = false;
start_listener = false;
proxy_renewal_frequency = 600;
bulk_query_size = 100;
soap_timeout = 60;
persist_dir = "/var/ice/persist_dir";
log_on_file = true;
logfile = "${WMS_LOCATION_LOG}/ice.log";
ice_host_key = "/home/glite/.certs/hostkey.pem";
max_ice_threads = 10;
log_on_console = false;
cemon_url_prefix = "https://";
ice_empty_threshold = 600;
];
WorkloadManagerProxy:
WorkloadManagerProxy = [
EnableServiceDiscovery = false;
ListMatchRootPath = "/tmp";
OperationLoadScripts = [
jobSubmit = "${WMS_LOCATION_SBIN}/glite_wms_wmproxy_load_monitor --oper jobSubmit --load1 22 --load5 20 --load15 18 --memusage 99
--diskusage 95 --fdnum 1000 --jdnum 1500 --ftpconn 300";
jobRegister = "${WMS_LOCATION_SBIN}/glite_wms_wmproxy_load_monitor --oper jobRegister --load1 22 --load5 20 --load15 18 --memusag
e 99 --diskusage 95 --fdnum 1000 --jdnum 1500 --ftpconn 300";
];
MinPerusalTimeInterval = 1000;
MaxServedRequests = 50;
ServiceDiscoveryInfoValidity = 3600;
LBServer = {"emi-demo11.cnaf.infn.it:9000"};
LBServiceDiscoveryType = "org.glite.lb.server";
LogLevel = 5;
SandboxStagingPath = "${WMS_LOCATION_VAR}/SandboxDir";
LBLocalLogger = "localhost:9002";
WeightsCacheValidity = 86400;
LogFile = "${WMS_LOCATION_LOG}/wmproxy.log";
AsyncJobStart = true;
GridFTPPort = 2811;
MaxInputSandboxSize = 100000000;
];
NetworkServer:
NetworkServer = [
II_Timeout = 100;
EnableQuotaManagement = false;
QuotaInsensibleDiskPortion = 2.0;
SandboxStagingPath = "${WMS_LOCATION_VAR}/SandboxDir";
II_DN = "mds-vo-name=local, o=grid";
MasterThreads = 8;
ConnectionTimeout = 300;
LogFile = "${WMS_LOCATION_LOG}/networkserver_events.log";
LogLevel = 5;
II_Contact = "cert-bdii-04.cnaf.infn.it";
//II_Contact = "egee-bdii.cnaf.infn.it";
II_Port = 2170;
QuotaAdjustmentAmount = 10000;
DispatcherThreads = 10;
Gris_DN = "mds-vo-name=local, o=grid";
Gris_Port = 2170;
DLI_SI_CatalogTimeout = 60;
EnableDynamicQuotaAdjustment = false;
ListMatchParadise = "${WMS_LOCATION_TMP}/MatchArea";
Gris_Timeout = 20;
BacklogSize = 64;
ListeningPort = 7772;
MaxInputSandboxSize = 10000000;
];
WorkloadManager:
WorkloadManager = [
DisablePurchasingFromGris = true;
IsmBlackList = {};
ExpiryPeriod = 7200;
PipeDepth = 200;
JobWrapperTemplateDir = "${WMS_JOBWRAPPER_TEMPLATE}";
IsmDump = "${WMS_LOCATION_VAR}/workload_manager/ismdump.fl";
IsmIiPurchasingRate = 480;
MatchRetryPeriod = 600;
MaxReplansCount = 5;
EnableRecovery = true;
DispatcherType = "jobdir";
LogLevel = 5;
LogFile = "${WMS_LOCATION_LOG}/workload_manager_events.log";
DliServiceName = "data-location-interface";
RuntimeMalloc = "/usr/lib64/libjemalloc.so.1";
IsmUpdateRate = 600;
WmsRequirements = ((ShortDeadlineJob =?= TRUE) ? RegExp(".*sdj$", other.GlueCEUniqueID) : !RegExp(".*sdj$", other.GlueCEUniqueID)) && (other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs) && (EnableWmsFeedback =?= TRUE ? RegExp("cream", other.GlueCEImplementationName, "i") : true);
CeMonitorAsynchPort = 0;
QueueSize = 1000;
EnableBulkMM = true;
MaxRetryCount = 10;
SiServiceName = "org.glite.SEIndex";
IsmThreads = false;
MaxOutputSandboxSize = -1;
CeForwardParameters = {"GlueHostMainMemoryVirtualSize","GlueHostMainMemoryRAMSize","GlueCEPolicyMaxCPUTime"};
CeMonitorServices = {};
IsmIiLDAPCEFilterExt = "(|(GlueCEAccessControlBaseRule=VO:atlas)(GlueCEAccessControlBaseRule=VOMS:/atlas/*)(GlueCEAccessControlBaseRule=VO:alice)(GlueCEAccessControlBaseRule=VOMS:/alice/*)(GlueCEAccessControlBaseRule=VO:lhcb)(GlueCEAccessControlBaseRule=VOMS:/lhcb/*)(GlueCEAccessControlBaseRule=VO:cms)(GlueCEAccessControlBaseRule=VOMS:/cms/*)(GlueCEAccessControlBaseRule=VO:dteam)(GlueCEAccessControlBaseRule=VOMS:/dteam/*)(GlueCEAccessControlBaseRule=VO:biomed)(GlueCEAccessControlBaseRule=VOMS:/biomed/*)(GlueCEAccessControlBaseRule=VO:ops)(GlueCEAccessControlBaseRule=VOMS:/ops/*)(GlueCEAccessControlBaseRule=VO:infngrid)(GlueCEAccessControlBaseRule=VOMS:/infngrid/*)(GlueCEAccessControlBaseRule=VO:testers.eu-emi.eu)(GlueCEAccessControlBaseRule=VOMS:/testers.eu-emi.eu/*))";
ReplanGracePeriod = 3600;
Input = "${WMS_LOCATION_VAR}/workload_manager/jobdir";
WorkerThreads = 5;
];
Common section:
Common = [
HostProxyFile = "${WMS_LOCATION_VAR}/glite/wms.proxy";
LBProxy = true;
DGUser = "${GLITE_WMS_USER}";
];
Logmonitor section:
LogMonitor = [
CondorLogRecycleDir = "${WMS_LOCATION_VAR}/logmonitor/CondorG.log/recycle";
LockFile = "${WMS_LOCATION_VAR}/logmonitor/lock";
AbortedJobsTimeout = 600;
CondorLogDir = "${WMS_LOCATION_VAR}/logmonitor/CondorG.log";
MonitorInternalDir = "${WMS_LOCATION_VAR}/logmonitor/internal";
MainLoopDuration = 5;
GlobusDownTimeout = 7200;
LogLevel = 5;
JobsPerCondorLog = 1000;
LogFile = "${WMS_LOCATION_LOG}/logmonitor_events.log";
ExternalLogFile = "${WMS_LOCATION_LOG}/logmonitor_external.log";
RemoveJobFiles = true;
IdRepositoryName = "irepository.dat";
ForceCancellationRetries = 2;
];
JobController section:
JobController = [
Input = "${WMS_LOCATION_VAR}/jobcontrol/jobdir/";
ContainerRefreshThreshold = 1000;
CondorQuery = "${CONDORG_INSTALL_PATH}/bin/condor_q";
LogFile = "${WMS_LOCATION_LOG}/jobcontoller_events.log";
CondorDagman = "${CONDORG_INSTALL_PATH}/bin/condor_dagman";
MaximumTimeAllowedForCondorMatch = 1800;
CondorRelease = "${CONDORG_INSTALL_PATH}/bin/condor_release";
OutputFileDir = "${WMS_LOCATION_VAR}/jobcontrol/condorio";
InputType = "jobdir";
CondorRemove = "${CONDORG_INSTALL_PATH}/bin/condor_rm";
DagmanMaxPre = 10;
LockFile = "${WMS_LOCATION_VAR}/jobcontrol/lock";
SubmitFileDir = "${WMS_LOCATION_VAR}/jobcontrol/submit";
CondorSubmit = "${CONDORG_INSTALL_PATH}/bin/condor_submit";
LogLevel = 5;
];
Verified that:
- The attribute
II_ContactofNetworkServersection matches the value of the yaim variableBDII_HOST - The attribute
WMExpiryPeriodofWorkloadManagersection matches the value of yaim variableWMS_EXPIRY_PERIOD - The attribute
MatchRetryPeriodofWorkloadManagersection matches the value of yaim variableWMS_MATCH_RETRY_PERIOD - The attribute
IsmIiLDAPCEFilterExtofWorkloadManagersection is(|(GlueCEAccessControlBaseRule=VO:vo1)(GlueCEAccessControlBaseRule=VOMS:/vo1/*)(GlueCEAccessControlBaseRule=VO=vo2... - The attribute
LBServerof theWorkloadManagerProxysection matches the value of yaim variableLB_HOST
WMS feedback
TBDLimiter mechanism
[capannini@devel15 WMS-service]$ python WMS-limiter-mechanism.py -c wms-command.conf -V dteam -d 3 -l -t 1,4,6-10 ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Limiter Mechanism ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: Test option --load1 threshold for load average (1 min) Test 4: Test option --memusage threshold for memory usage (%) Test 6: Test option --ftpconn threshold for number of FTP connections Test 7: Test option --diskusage threshold for disk usage (%) Test 8: Test option --fdnum threshold for used file descriptor Test 9: Test option --jdnum threshold for number of unprocessed jobs (for jobdir) Test 10: Test option --jdsize threshold for input jobdir size (KB) ===> ===> Test: WMS-limiter-mechanism.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 15:17:37 ===> Ended : 15:18:11 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120124151737.logSee log: WMSService-TS_20120124151737.log: limiter log
Purge
Normal job cycle
[capannini@devel15 WMS-service]$ python WMS-check-purger.py -c wms-command.conf -V dteam -d 3 -l ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Purge Mechanism ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: Test purge , normal job cycle ( Submit to LCG CE ) Test 2: Test purge, normal job cycle ( Submit to CREAM CE ) Test 3: Test proxy cache purging ===> ===> Test: WMS-check-purger.py ===> WMS: emi-demo11.cnaf.infn.it ===> Started: 12:07:54 ===> Ended : 12:11:47 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120125120754.logSee log: WMSService-TS_20120125120754.log: purge logs
SandboxDir cron purger
The purge of the SandBox dir is done by a cron job - Submit a job and force its purge before it finishes: [capannini@devel15 WMS-service]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf ~/ls.jdl Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server ====================== glite-wms-job-submit Success ====================== The job has been successfully submitted to the WMProxy Your job identifier is: https://emi-demo11.cnaf.infn.it:9000/I-xR4xvAHsoHfhl7OplqLQ ========================================================================== [capannini@devel15 WMS-service]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/I-xR4xvAHsoHfhl7OplqLQ ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/I-xR4xvAHsoHfhl7OplqLQ Current Status: Scheduled Status Reason: unavailable Destination: ce208.cern.ch:8443/cream-lsf-grid_dteam Submitted: Wed Jan 25 12:26:31 2012 CET ========================================================================== [root@emi-demo11 ~]# su - glite [glite@emi-demo11 ~]$ [glite@emi-demo11 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/I- -s 25 Jan, 12:27:33 -I: [Info] operator()(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:447): https://emi-demo11.cnaf.infn.it:9000/I-xR4xvAHsoHfhl7OplqLQ: removed SCHEDULED job [capannini@devel15 WMS-service]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/I-xR4xvAHsoHfhl7OplqLQ ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/I-xR4xvAHsoHfhl7OplqLQ Current Status: Cleared Status Reason: timed out, resource purge forced Destination: ce208.cern.ch:8443/cream-lsf-grid_dteam Submitted: Wed Jan 25 12:26:31 2012 CET ========================================================================== ===> === test PASSED === - Purge a done job: [capannini@devel15 WMS-service]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/T7uROEeHs0MVTbeiMYDRJg ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/T7uROEeHs0MVTbeiMYDRJg Current Status: Done (Success) Exit code: 0 Status Reason: Job Terminated Successfully Destination: ce208.cern.ch:8443/cream-lsf-grid_dteam Submitted: Wed Jan 25 12:31:26 2012 CET ========================================================================== [glite@emi-demo11 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/T7 -s 25 Jan, 14:09:45 -I: [Info] operator()(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:447): https://emi-demo11.cnaf.infn.it:9000/T7uROEeHs0MVTbeiMYDRJg: removed DONE job [capannini@devel15 WMS-service]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/T7uROEeHs0MVTbeiMYDRJg ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/T7uROEeHs0MVTbeiMYDRJg Current Status: Cleared Status Reason: timed out, resource purge forced Destination: ce208.cern.ch:8443/cream-lsf-grid_dteam Submitted: Wed Jan 25 12:31:26 2012 CET ========================================================================== - Purge only jobs older than 12 hours: Find some directories to be purged with the command: # find /var/SandboxDir/ -maxdepth 2 -type d -cmin +720 /var/SandboxDir/W6/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fW65OfHAN_5fLrcvyoohMGzWQ /var/SandboxDir/ez/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fezKOPmT5HcqCRjmO1ttSZA Try to purge: [glite@emi-demo11 ~]$ date Wed Jan 25 14:15:03 CET 2012 [glite@emi-demo11 ~]$ ls -l /var/SandboxDir/ez total 4 drwxrwx--- 5 dteam045 glite 4096 Jan 24 10:38 https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fezKOPmT5HcqCRjmO1ttSZA [glite@emi-demo11 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/ez -t 43200 25 Jan, 14:20:43 -I: [Info] operator()(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:447): https://emi-demo11.cnaf.infn.it:9000/ezKOPmT5HcqCRjmO1ttSZA: removed ABORTED job [glite@emi-demo11 ~]$ ls -l /var/SandboxDir/ez total 0 [glite@emi-demo11 ~]$ ls -lr /var/SandboxDir/W6 total 4 drwxrwx--- 5 dteam045 glite 4096 Jan 24 14:24 https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fW65OfHAN_5fLrcvyoohMGzWQ [glite@emi-demo11 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/W6 -t 43200 25 Jan, 14:21:02 -I: [Info] operator()(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:447): https://emi-demo11.cnaf.infn.it:9000/W65OfHAN_LrcvyoohMGzWQ: removed DONE job [glite@emi-demo11 ~]$ ls -lr /var/SandboxDir/W6 total 0 Find some directories that should not be purged: # find /var/SandboxDir/ -maxdepth 2 -type d -cmin -240 /var/SandboxDir/St/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fStNSoiVR3S47qnaFFHEOzw /var/SandboxDir/T5/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fT5gK96CZD8DbV8Oi1hSNcA Try to purge: [glite@emi-demo11 ~]$ date Wed Jan 25 14:15:03 CET 2012 [glite@emi-demo11 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/T5 -t 43200 [glite@emi-demo11 ~]$ ls -l /var/SandboxDir/T5 total 4 drwxrwx--- 5 dteam045 glite 4096 Jan 25 10:29 https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fT5gK96CZD8DbV8Oi1hSNcA [glite@emi-demo11 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/St -t 43200 [glite@emi-demo11 ~]$ ls -l /var/SandboxDir/St total 4 drwxrwx--- 5 dteam045 glite 4096 Jan 25 12:59 https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fStNSoiVR3S47qnaFFHEOzw ===> Test PASSED- Purge a job whose proxy is already expired
[glite@emi-demo11 ~]$ openssl x509 -in /var/SandboxDir/z6/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fz6gxfYkbU1h8hZIyOXEOUg/user.proxy -noout -dates
notBefore=Jan 17 15:52:12 2012 GMT
notAfter=Jan 17 21:19:12 2012 GMT
[glite@emi-demo11 ~]$ date
Wed Jan 25 15:18:49 CET 2012
[capannini@devel15 WMS-service]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/z6gxfYkbU1h8hZIyOXEOUg
======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/z6gxfYkbU1h8hZIyOXEOUg
Current Status: Submitted
Submitted: Tue Jan 17 16:57:12 2012 CET
Parent Job: https://emi-demo11.cnaf.infn.it:9000/79cGVuy9XVzQxwvn_Z2Hgw
==========================================================================
[glite@emi-demo11 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/z6 -s
25 Jan, 15:21:10 -I: [Info] operator()(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:447): https://emi-demo11.cnaf.infn.it:9000/z6gxfYkbU1h8hZIyOXEOUg: removed SUBMITTED job
[capannini@devel15 WMS-service]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/z6gxfYkbU1h8hZIyOXEOUg
======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/z6gxfYkbU1h8hZIyOXEOUg
Current Status: Cleared
Status Reason: timed out, resource purge forced
Submitted: Tue Jan 17 16:57:12 2012 CET
Parent Job: https://emi-demo11.cnaf.infn.it:9000/79cGVuy9XVzQxwvn_Z2Hgw
==========================================================================
===> === test PASSED ===
- Purge a dag job
[capannini@devel15 dag]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf dag.jdl
Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
====================== glite-wms-job-submit Success ======================
The job has been successfully submitted to the WMProxy
Your job identifier is:
https://emi-demo11.cnaf.infn.it:9000/Ab_cMtMLDPkCNKsL-5CFaw
==========================================================================
[capannini@devel15 dag]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/Ab_cMtMLDPkCNKsL-5CFaw
======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/Ab_cMtMLDPkCNKsL-5CFaw
Current Status: Running
Status Reason: unavailable
Destination: dagman
Submitted: Wed Jan 25 15:33:46 2012 CET
==========================================================================
- Nodes information for:
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/HnZfPHnH3xgxmhl_NrRr-g
Current Status: Submitted
Submitted: Wed Jan 25 15:33:46 2012 CET
==========================================================================
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/X2-G_Tr9eyWgs88vjfnL2g
Current Status: Submitted
Submitted: Wed Jan 25 15:33:46 2012 CET
==========================================================================
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/Yc4eO097SXSBrXIvhsIqyw
Current Status: Submitted
Submitted: Wed Jan 25 15:33:46 2012 CET
==========================================================================
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/b8bBqrceSFYU1mWEfpH-fQ
Current Status: Submitted
Submitted: Wed Jan 25 15:33:46 2012 CET
==========================================================================
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/hGHl6d1B1D8xPAmNXdLdsw
Current Status: Submitted
Submitted: Wed Jan 25 15:33:46 2012 CET
==========================================================================
[glite@emi-demo11 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/Ab -s25 Jan, 15:34:30 -I: [Info] operator()(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:354): https://emi-demo11.cnaf.infn.it:9000/Ab_cMtMLDPkCNKsL-5CFaw: 5/5 nodes removed
25 Jan, 15:34:30 -I: [Info] operator()(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:366): https://emi-demo11.cnaf.infn.it:9000/Ab_cMtMLDPkCNKsL-5CFaw: removed RUNNING dag
[capannini@devel15 dag]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/Ab_cMtMLDPkCNKsL-5CFaw
======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/Ab_cMtMLDPkCNKsL-5CFaw
Current Status: Cleared
Status Reason: timed out, resource purge forced
Destination: dagman
Submitted: Wed Jan 25 15:33:46 2012 CET
==========================================================================
- Nodes information for:
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/HnZfPHnH3xgxmhl_NrRr-g
Current Status: Cleared
Status Reason: timed out, resource purge forced
Submitted: Wed Jan 25 15:33:46 2012 CET
==========================================================================
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/X2-G_Tr9eyWgs88vjfnL2g
Current Status: Cleared
Status Reason: timed out, resource purge forced
Submitted: Wed Jan 25 15:33:46 2012 CET
==========================================================================
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/Yc4eO097SXSBrXIvhsIqyw
Current Status: Cleared
Status Reason: timed out, resource purge forced
Submitted: Wed Jan 25 15:33:46 2012 CET
==========================================================================
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/b8bBqrceSFYU1mWEfpH-fQ
Current Status: Cleared
Status Reason: timed out, resource purge forced
Submitted: Wed Jan 25 15:33:46 2012 CET
==========================================================================
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/hGHl6d1B1D8xPAmNXdLdsw
Current Status: Cleared
Status Reason: timed out, resource purge forced
Destination: argoce01.na.infn.it:2119/jobmanager-lcgpbs-cert
Submitted: Wed Jan 25 15:33:46 2012 CET
==========================================================================
===> === test PASSED ===
Published information
bdb database information
[capannini@devel15 ~]$ ldapsearch -x -H ldap://emi-demo11.cnaf.infn.it:2170 -b 'o=grid' # extended LDIF # # LDAPv3 # base <o=grid> with scope subtree # filter: (objectclass=*) # requesting: ALL # # grid dn: o=grid objectClass: organization o: grid # local, grid dn: Mds-Vo-name=local,o=grid Mds-Vo-name: local objectClass: Mds # resource, grid dn: Mds-Vo-name=resource,o=grid Mds-Vo-name: resource objectClass: Mds # emi-demo11.cnaf.infn.it_org.glite.lb.Server_2310828617, resource, grid dn: GlueServiceUniqueID=emi-demo11.cnaf.infn.it_org.glite.lb.Server_2310828617 ,Mds-Vo-name=resource,o=grid objectClass: GlueTop objectClass: GlueService objectClass: GlueKey objectClass: GlueSchemaVersion GlueServiceUniqueID: emi-demo11.cnaf.infn.it_org.glite.lb.Server_2310828617 GlueServiceEndpoint: https://emi-demo11.cnaf.infn.it:9003/ GlueServiceVersion: 2.2.10 GlueSchemaVersionMinor: 3 GlueServiceName: INFN-CREAMTEST-Server GlueServiceType: org.glite.lb.Server GlueServiceWSDL: http://egee.cesnet.cz/cms/export/sites/egee/en/WSDL/3.1/LB.ws dl GlueServiceSemantics: https://edms.cern.ch/file/571273/2/LB-guide.pdf GlueForeignKey: GlueSiteUniqueID=INFN-CREAMTEST GlueSchemaVersionMajor: 1 GlueServiceStatus: OK GlueServiceStartTime: 2012-01-19T12:26:49+01:00 GlueServiceStatusInfo: glite-lb-notif-interlogd running as 9582 glite-lb-inter logd for proxy running as 9617 glite-lb-bkserverd running as 9536 glite-jp-im porter not running (disabled) # emi-demo11.cnaf.infn.it_org.glite.wms.WMProxy_841415637, resource, grid dn: GlueServiceUniqueID=emi-demo11.cnaf.infn.it_org.glite.wms.WMProxy_84141563 7,Mds-Vo-name=resource,o=grid GlueServiceAccessControlBaseRule: VO:atlas GlueServiceAccessControlBaseRule: VO:alice GlueServiceAccessControlBaseRule: VO:lhcb GlueServiceAccessControlBaseRule: VO:cms GlueServiceAccessControlBaseRule: VO:dteam GlueServiceAccessControlBaseRule: VO:biomed GlueServiceAccessControlBaseRule: VO:ops GlueServiceAccessControlBaseRule: VO:infngrid GlueServiceAccessControlBaseRule: VO:testers.eu-emi.eu GlueServiceStatus: Other GlueServiceStatusInfo: ?? objectClass: GlueTop objectClass: GlueService objectClass: GlueKey objectClass: GlueSchemaVersion GlueServiceUniqueID: emi-demo11.cnaf.infn.it_org.glite.wms.WMProxy_841415637 GlueServiceAccessControlRule: atlas GlueServiceAccessControlRule: alice GlueServiceAccessControlRule: lhcb GlueServiceAccessControlRule: cms GlueServiceAccessControlRule: dteam GlueServiceAccessControlRule: biomed GlueServiceAccessControlRule: ops GlueServiceAccessControlRule: infngrid GlueServiceAccessControlRule: testers.eu-emi.eu GlueServiceEndpoint: https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_se rver GlueServiceVersion: 3.3.4 GlueSchemaVersionMinor: 3 GlueServiceName: INFN-CREAMTEST-WMProxy GlueServiceType: org.glite.wms.WMProxy GlueServiceWSDL: http://web.infn.it/gLiteWMS/images/WMS/Docs/wmproxy.wsdl GlueServiceSemantics: http://web.infn.it/gLiteWMS/images/WMS/Docs/wmproxy-guid e.pdf GlueForeignKey: GlueSiteUniqueID=INFN-CREAMTEST GlueServiceOwner: atlas GlueServiceOwner: alice GlueServiceOwner: lhcb GlueServiceOwner: cms GlueServiceOwner: dteam GlueServiceOwner: biomed GlueServiceOwner: ops GlueServiceOwner: infngrid GlueServiceOwner: testers.eu-emi.eu GlueSchemaVersionMajor: 1 GlueServiceStartTime: 2012-01-26T00:00:02+01:00 # DN, emi-demo11.cnaf.infn.it_org.glite.wms.WMProxy_841415637, resource, grid dn: GlueServiceDataKey=DN,GlueServiceUniqueID=emi-demo11.cnaf.infn.it_org.glit e.wms.WMProxy_841415637,Mds-Vo-name=resource,o=grid GlueSchemaVersionMajor: 1 objectClass: GlueTop objectClass: GlueServiceData objectClass: GlueKey objectClass: GlueSchemaVersion GlueSchemaVersionMinor: 3 GlueServiceDataKey: DN GlueChunkKey: GlueServiceUniqueID=emi-demo11.cnaf.infn.it_org.glite.wms.WMProx y_841415637 GlueServiceDataValue: /C=IT/O=INFN/OU=Host/L=CNAF/CN=emi-demo11.cnaf.infn.it # glite-info-service_version, emi-demo11.cnaf.infn.it_org.glite.lb.Server_231 0828617, resource, grid dn: GlueServiceDataKey=glite-info-service_version,GlueServiceUniqueID=emi-demo 11.cnaf.infn.it_org.glite.lb.Server_2310828617,Mds-Vo-name=resource,o=grid GlueSchemaVersionMajor: 1 objectClass: GlueTop objectClass: GlueServiceData objectClass: GlueKey objectClass: GlueSchemaVersion GlueSchemaVersionMinor: 3 GlueServiceDataKey: glite-info-service_version GlueChunkKey: GlueServiceUniqueID=emi-demo11.cnaf.infn.it_org.glite.lb.Server_ 2310828617 GlueServiceDataValue: 1.6 # glite-info-service_version, emi-demo11.cnaf.infn.it_org.glite.wms.WMProxy_8 41415637, resource, grid dn: GlueServiceDataKey=glite-info-service_version,GlueServiceUniqueID=emi-demo 11.cnaf.infn.it_org.glite.wms.WMProxy_841415637,Mds-Vo-name=resource,o=grid GlueSchemaVersionMajor: 1 objectClass: GlueTop objectClass: GlueServiceData objectClass: GlueKey objectClass: GlueSchemaVersion GlueSchemaVersionMinor: 3 GlueServiceDataKey: glite-info-service_version GlueChunkKey: GlueServiceUniqueID=emi-demo11.cnaf.infn.it_org.glite.wms.WMProx y_841415637 GlueServiceDataValue: 1.6 # glite-info-service_hostname, emi-demo11.cnaf.infn.it_org.glite.lb.Server_23 10828617, resource, grid dn: GlueServiceDataKey=glite-info-service_hostname,GlueServiceUniqueID=emi-dem o11.cnaf.infn.it_org.glite.lb.Server_2310828617,Mds-Vo-name=resource,o=grid GlueSchemaVersionMajor: 1 objectClass: GlueTop objectClass: GlueServiceData objectClass: GlueKey objectClass: GlueSchemaVersion GlueSchemaVersionMinor: 3 GlueServiceDataKey: glite-info-service_hostname GlueChunkKey: GlueServiceUniqueID=emi-demo11.cnaf.infn.it_org.glite.lb.Server_ 2310828617 GlueServiceDataValue: emi-demo11.cnaf.infn.it # glite-info-service_hostname, emi-demo11.cnaf.infn.it_org.glite.wms.WMProxy_ 841415637, resource, grid dn: GlueServiceDataKey=glite-info-service_hostname,GlueServiceUniqueID=emi-dem o11.cnaf.infn.it_org.glite.wms.WMProxy_841415637,Mds-Vo-name=resource,o=grid GlueSchemaVersionMajor: 1 objectClass: GlueTop objectClass: GlueServiceData objectClass: GlueKey objectClass: GlueSchemaVersion GlueSchemaVersionMinor: 3 GlueServiceDataKey: glite-info-service_hostname GlueChunkKey: GlueServiceUniqueID=emi-demo11.cnaf.infn.it_org.glite.wms.WMProx y_841415637 GlueServiceDataValue: emi-demo11.cnaf.infn.it # search result search: 2 result: 0 Success # numResponses: 11 # numEntries: 10
Stats database definitions
[capannini@devel15 ~]$ ldapsearch -x -H ldap://emi-demo11.cnaf.infn.it:2170 -b 'o=infosys' # extended LDIF # # LDAPv3 # base <o=infosys> with scope subtree # filter: (objectclass=*) # requesting: ALL # # infosys dn: o=infosys objectClass: organization o: infosys # zip, infosys dn: CompressionType=zip,o=infosys objectClass: CompressedContent Hostname: localhost CompressionType: zip Data: file:///var/run/bdii/archive/12-01-26-11-02-24-snapshot.gz # localhost, infosys dn: Hostname=localhost,o=infosys objectClass: UpdateStats Hostname: localhost FailedDeletes: 0 ModifiedEntries: 8 DeletedEntries: 0 UpdateTime: 1 FailedAdds: 0 FailedModifies: 0 TotalEntries: 20 QueryTime: 0 NewEntries: 1 DBUpdateTime: 0 ReadTime: 0 PluginsTime: 0 ProvidersTime: 1 # search result search: 2 result: 0 Success # numResponses: 4 # numEntries: 3
Stats database definitions
[capannini@devel15 ~]$ ldapsearch -x -H ldap://emi-demo11.cnaf.infn.it:2170 -b 'o=glue' # extended LDIF # # LDAPv3 # base <o=glue> with scope subtree # filter: (objectclass=*) # requesting: ALL # # glue dn: o=glue objectClass: organization o: glue # grid, glue dn: GLUE2GroupID=grid,o=glue objectClass: GLUE2Group GLUE2GroupID: grid # resource, glue dn: GLUE2GroupID=resource,o=glue objectClass: GLUE2Group GLUE2GroupID: resource # emi-demo11.cnaf.infn.it_lbserver_201164012, resource, glue dn: GLUE2ServiceID=emi-demo11.cnaf.infn.it_lbserver_201164012,GLUE2GroupID=res ource,o=glue GLUE2EntityValidity: 3600 GLUE2ServiceType: org.glite.lb.Server GLUE2ServiceID: emi-demo11.cnaf.infn.it_lbserver_201164012 objectClass: GLUE2Service GLUE2ServiceQualityLevel: production GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-service GLUE2EntityOtherInfo: InfoProviderVersion=1.1 GLUE2EntityOtherInfo: InfoProviderHost=emi-demo11.cnaf.infn.it GLUE2ServiceComplexity: endpointType=1, share=0, resource=0 GLUE2ServiceCapability: information.logging GLUE2EntityName: INFN-CREAMTEST-Server GLUE2ServiceAdminDomainForeignKey: INFN-CREAMTEST GLUE2EntityCreationTime: 2012-01-26T10:04:25Z # emi-demo11.cnaf.infn.it_wmproxy_2598325561, resource, glue dn: GLUE2ServiceID=emi-demo11.cnaf.infn.it_wmproxy_2598325561,GLUE2GroupID=res ource,o=glue GLUE2EntityValidity: 3600 GLUE2ServiceType: org.glite.wms.WMProxy GLUE2ServiceID: emi-demo11.cnaf.infn.it_wmproxy_2598325561 objectClass: GLUE2Service GLUE2ServiceQualityLevel: production GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-service GLUE2EntityOtherInfo: InfoProviderVersion=1.1 GLUE2EntityOtherInfo: InfoProviderHost=emi-demo11.cnaf.infn.it GLUE2ServiceComplexity: endpointType=1, share=0, resource=0 GLUE2ServiceCapability: executionmanagement.candidatesetgenerator GLUE2ServiceCapability: executionmanagement.jobdescription GLUE2ServiceCapability: executionmanagement.jobmanager GLUE2EntityName: INFN-CREAMTEST-WMProxy GLUE2ServiceAdminDomainForeignKey: INFN-CREAMTEST GLUE2EntityCreationTime: 2012-01-26T10:04:26Z # emi-demo11.cnaf.infn.it_lbserver_201164012_org.glite.lb.Server_201164012, e mi-demo11.cnaf.infn.it_lbserver_201164012, resource, glue dn: GLUE2EndpointID=emi-demo11.cnaf.infn.it_lbserver_201164012_org.glite.lb.Se rver_201164012,GLUE2ServiceID=emi-demo11.cnaf.infn.it_lbserver_201164012,GLUE 2GroupID=resource,o=glue GLUE2EntityValidity: 3600 GLUE2EndpointQualityLevel: production GLUE2EndpointImplementor: gLite GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-endpoint GLUE2EntityOtherInfo: InfoProviderVersion=1.2 GLUE2EntityOtherInfo: InfoProviderHost=emi-demo11.cnaf.infn.it GLUE2EntityOtherInfo: MiddlewareName=EMI GLUE2EntityOtherInfo: MiddlewareVersion=1.7.0-1 GLUE2EndpointCapability: information.logging GLUE2EndpointServiceForeignKey: emi-demo11.cnaf.infn.it_lbserver_201164012 GLUE2EntityName: Server endpoint for Service emi-demo11.cnaf.infn.it_lbserver_ 201164012 GLUE2EndpointTechnology: webservice GLUE2EndpointWSDL: http://egee.cesnet.cz/cms/export/sites/egee/en/WSDL/3.1/LB. wsdl GLUE2EndpointInterfaceName: org.glite.lb.Server GLUE2EndpointURL: https://emi-demo11.cnaf.infn.it:9003/ GLUE2EndpointDowntimeInfo: See the GOC DB for downtimes: https://goc.egi.eu/ GLUE2EndpointImplementationVersion: 2.2.10 GLUE2EndpointSemantics: https://edms.cern.ch/file/571273/2/LB-guide.pdf GLUE2EndpointIssuerCA: /C=IT/O=INFN/CN=INFN CA GLUE2EndpointServingState: production objectClass: GLUE2Endpoint GLUE2EndpointInterfaceVersion: 3.2.4 GLUE2EndpointImplementationName: LB GLUE2EndpointTrustedCA: IGTF GLUE2EndpointID: emi-demo11.cnaf.infn.it_lbserver_201164012_org.glite.lb.Serve r_201164012 GLUE2EndpointStartTime: 2012-01-19T11:26:49Z GLUE2EndpointHealthStateInfo: glite-lb-notif-interlogd running as 9582 glite-l b-interlogd for proxy running as 9617 glite-lb-bkserverd running as 9536 glit e-jp-importer not running (disabled) GLUE2EndpointHealthState: ok GLUE2EntityCreationTime: 2012-01-26T10:04:25Z # emi-demo11.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMProxy_2598325561 , emi-demo11.cnaf.infn.it_wmproxy_2598325561, resource, glue dn: GLUE2EndpointID=emi-demo11.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.W MProxy_2598325561,GLUE2ServiceID=emi-demo11.cnaf.infn.it_wmproxy_2598325561,G LUE2GroupID=resource,o=glue GLUE2EntityValidity: 3600 GLUE2EndpointQualityLevel: production GLUE2EndpointImplementor: gLite GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-endpoint GLUE2EntityOtherInfo: InfoProviderVersion=1.2 GLUE2EntityOtherInfo: InfoProviderHost=emi-demo11.cnaf.infn.it GLUE2EntityOtherInfo: MiddlewareName=EMI GLUE2EntityOtherInfo: MiddlewareVersion=1.7.0-1 GLUE2EndpointCapability: executionmanagement.candidatesetgenerator GLUE2EndpointCapability: executionmanagement.jobdescription GLUE2EndpointCapability: executionmanagement.jobmanager GLUE2EndpointHealthState: ok GLUE2EndpointServiceForeignKey: emi-demo11.cnaf.infn.it_wmproxy_2598325561 GLUE2EntityName: WMProxy endpoint for Service emi-demo11.cnaf.infn.it_wmproxy_ 2598325561 GLUE2EndpointTechnology: webservice GLUE2EndpointWSDL: http://web.infn.it/gLiteWMS/images/WMS/Docs/wmproxy.wsdl GLUE2EndpointInterfaceName: org.glite.wms.WMProxy GLUE2EndpointURL: https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_serve r GLUE2EndpointDowntimeInfo: See the GOC DB for downtimes: https://goc.egi.eu/ GLUE2EndpointImplementationVersion: 3.3.4 GLUE2EndpointSemantics: http://web.infn.it/gLiteWMS/images/WMS/Docs/wmproxy-gu ide.pdf GLUE2EndpointIssuerCA: /C=IT/O=INFN/CN=INFN CA GLUE2EndpointServingState: production objectClass: GLUE2Endpoint GLUE2EndpointInterfaceVersion: 1.0 GLUE2EndpointImplementationName: WMS GLUE2EndpointTrustedCA: IGTF GLUE2EndpointID: emi-demo11.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMPr oxy_2598325561 GLUE2EndpointStartTime: 2012-01-25T23:00:02Z GLUE2EndpointHealthStateInfo: WMProxy httpd listening on port 7443 httpd is ru nning .... === WMProxy Server running instances: UID PID PPID C STIME TTY TIM E CMD glite 13421 30902 0 09:28 ? 00:00:00 /usr/bin/glite_wms_wmproxy_server GLUE2EntityCreationTime: 2012-01-26T10:04:26Z # DN_1, emi-demo11.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMProxy_2598 325561, emi-demo11.cnaf.infn.it_wmproxy_2598325561, resource, glue dn: GLUE2ExtensionLocalID=DN_1,GLUE2EndpointID=emi-demo11.cnaf.infn.it_wmproxy _2598325561_org.glite.wms.WMProxy_2598325561,GLUE2ServiceID=emi-demo11.cnaf.i nfn.it_wmproxy_2598325561,GLUE2GroupID=resource,o=glue GLUE2ExtensionLocalID: DN_1 GLUE2ExtensionKey: DN objectClass: GLUE2Extension GLUE2ExtensionValue: /C=IT/O=INFN/OU=Host/L=CNAF/CN=emi-demo11.cnaf.infn.it GLUE2ExtensionEntityForeignKey: emi-demo11.cnaf.infn.it_wmproxy_2598325561_org .glite.wms.WMProxy_2598325561 # emi-demo11.cnaf.infn.it_lbserver_201164012_org.glite.lb.Server_201164012_Po licy, emi-demo11.cnaf.infn.it_lbserver_201164012_org.glite.lb.Server_2011640 12, emi-demo11.cnaf.infn.it_lbserver_201164012, resource, glue dn: GLUE2PolicyID=emi-demo11.cnaf.infn.it_lbserver_201164012_org.glite.lb.Serv er_201164012_Policy,GLUE2EndpointID=emi-demo11.cnaf.infn.it_lbserver_20116401 2_org.glite.lb.Server_201164012,GLUE2ServiceID=emi-demo11.cnaf.infn.it_lbserv er_201164012,GLUE2GroupID=resource,o=glue GLUE2EntityValidity: 3600 objectClass: GLUE2Policy objectClass: GLUE2AccessPolicy GLUE2PolicyRule: ALL GLUE2PolicyScheme: org.glite.standard GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-endpoint GLUE2EntityOtherInfo: InfoProviderVersion=1.2 GLUE2EntityOtherInfo: InfoProviderHost=emi-demo11.cnaf.infn.it GLUE2AccessPolicyEndpointForeignKey: emi-demo11.cnaf.infn.it_lbserver_20116401 2_org.glite.lb.Server_201164012 GLUE2PolicyID: emi-demo11.cnaf.infn.it_lbserver_201164012_org.glite.lb.Server_ 201164012_Policy GLUE2EntityName: Access control rules for Endpoint emi-demo11.cnaf.infn.it_lbs erver_201164012_org.glite.lb.Server_201164012 GLUE2EntityCreationTime: 2012-01-26T10:04:25Z # emi-demo11.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMProxy_2598325561 _Policy, emi-demo11.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMProxy_25 98325561, emi-demo11.cnaf.infn.it_wmproxy_2598325561, resource, glue dn: GLUE2PolicyID=emi-demo11.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMP roxy_2598325561_Policy,GLUE2EndpointID=emi-demo11.cnaf.infn.it_wmproxy_259832 5561_org.glite.wms.WMProxy_2598325561,GLUE2ServiceID=emi-demo11.cnaf.infn.it_ wmproxy_2598325561,GLUE2GroupID=resource,o=glue GLUE2AccessPolicyEndpointForeignKey: emi-demo11.cnaf.infn.it_wmproxy_259832556 1_org.glite.wms.WMProxy_2598325561 GLUE2EntityValidity: 3600 objectClass: GLUE2Policy objectClass: GLUE2AccessPolicy GLUE2PolicyRule: VO:atlas GLUE2PolicyRule: VO:alice GLUE2PolicyRule: VO:lhcb GLUE2PolicyRule: VO:cms GLUE2PolicyRule: VO:dteam GLUE2PolicyRule: VO:biomed GLUE2PolicyRule: VO:ops GLUE2PolicyRule: VO:infngrid GLUE2PolicyRule: VO:testers.eu-emi.eu GLUE2PolicyScheme: org.glite.standard GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-endpoint GLUE2EntityOtherInfo: InfoProviderVersion=1.2 GLUE2EntityOtherInfo: InfoProviderHost=emi-demo11.cnaf.infn.it GLUE2PolicyUserDomainForeignKey: atlas GLUE2PolicyUserDomainForeignKey: alice GLUE2PolicyUserDomainForeignKey: lhcb GLUE2PolicyUserDomainForeignKey: cms GLUE2PolicyUserDomainForeignKey: dteam GLUE2PolicyUserDomainForeignKey: biomed GLUE2PolicyUserDomainForeignKey: ops GLUE2PolicyUserDomainForeignKey: infngrid GLUE2PolicyUserDomainForeignKey: testers.eu-emi.eu GLUE2PolicyID: emi-demo11.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMProx y_2598325561_Policy GLUE2EntityName: Access control rules for Endpoint emi-demo11.cnaf.infn.it_wmp roxy_2598325561_org.glite.wms.WMProxy_2598325561 GLUE2EntityCreationTime: 2012-01-26T10:04:26Z # search result search: 2 result: 0 Success # numResponses: 11 # numEntries: 10
Performance test
The tests are carried out using emi-demo11, a virtual machine with wms installed, as this has never been tried before. The real certification process should use devel20, a real machine with wms installed, instead.Collection of 1000 nodes
Submission of a collection of 1000 nodes[capannini@cert-17 StressTest]$ ./StressTest.sh -t 1 -n 1000 -c ~/wms_emi-demo11.conf -j ls.jdl Test starts on mer feb 1 11:58:27 CET 2012: submit 1 collections (of 1000 nodes) with a frequency of 60 seconds using 1 users. Submission n. 1: mer feb 1 11:58:27 CET 2012: user 0 is submitting... Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server ====================== glite-wms-job-submit Success ====================== The job has been successfully submitted to the WMProxy Your job identifier is: https://emi-demo11.cnaf.infn.it:9000/Cw4IByQC5DYhcHcEguYxuQ The job identifier has been saved in the following file: /home/capannini/certification/StressTest/Test_1328093897/output/USER0_0.jobid ========================================================================== Submission takes 7 seconds. 1 jobs submitted in 7 seconds: 7/7/7 (min/avg/max) Test finishes at mer feb 1 11:58:34 CET 2012The collection stays in status 'Waiting'. ======> Test NOT PASSED
Stress test
- 2880 collection of 25 nodes each (total: 72000 jobs)
- One collection every 60 seconds
- one user
- Use LCG-CEs and CREAM-CEs (with different batch systems)
- Use automatic-delegation
- The job is a `ls -l`
- Resubmission is enabled
- Enable proxy renewal
[capannini@cert-17 StressTest]$ ./StressTest.sh -t 2880 -n 25 -c ~/wms_emi-demo11.conf -j ls.jdl Test starts on mer feb 1 14:19:49 CET 2012: submit 2880 collections (of 25 nodes) with a frequency of 60 seconds using 1 users.
Results
[capannini@cert-17 output]$ ../../loginfo.sh -e 54 Results: DONE : 47192 NOTDone: 8 Aborted: 1775 ------------------------ Total : 48975 done jobs on CREAM CEs: 29819 (63%) done jobs on LCG CEs: 17373 (37%) The above statistics is obatined in the following way: [capannini@cert-17 output]$ cat *.done|wc -l 47192 [capannini@cert-17 output]$ cat *.done|grep '8443/cream' |wc -l 29819 [capannini@cert-17 output]$ cat *.done|grep '2119/jobmanager'|wc -l 17373====> Test NOT PASSED (Aborted/Total = 3.6% probably too high percentage, but a general rule should be agreed upon)
Regression tests
Bug #81376 (glite-wms-wmproxy.restart.cron MUST support graceful wmp restart) FIXED
[capannini@devel15 regression_tests]$ python execute_regTest.py -d 3 -b 81376 -l +++++++++++++++++++++++++++++++++++++++++++++++++++++ + Regression TestSuite .... + +++++++++++++++++++++++++++++++++++++++++++++++++++++ Testing bug 81376 All tests PASSEDSee log: RegressionTest_20120126111141.log: bug 81376 log
Bug #81568 (some inconsistencies in locations for logs and configuration) FIXED
[capannini@devel15 regression_tests]$ python execute_regTest.py -d 3 -b 81568 -l +++++++++++++++++++++++++++++++++++++++++++++++++++++ + Regression TestSuite .... + +++++++++++++++++++++++++++++++++++++++++++++++++++++ Testing bug 81568 All tests PASSEDSee log: RegressionTest_20120126112001.log: bug 81568 log Verified that all wms logs are in this directory:
[root@emi-demo11 ~]# ls $WMS_LOCATION_LOG glite-wms-purgeStorage.log ice.log logmonitor_events.log workload_manager_events.log.2 glite-wms-wmproxy-purge-proxycache.log ice.log.1 wmproxy.log workload_manager_events.log.3 glite-wms-wmproxy.restart.cron.log ice.log.2 wmproxy_logrotate.log httpd-wmproxy-access.log ice.log.3 workload_manager_events.log httpd-wmproxy-errors.log jobcontoller_events.log workload_manager_events.log.1
Bug #83062 (A different template should be used for glue2 publication) FIXED
[capannini@devel15 regression_tests]$ python execute_regTest.py -d 3 -b 83062 -l +++++++++++++++++++++++++++++++++++++++++++++++++++++ + Regression TestSuite .... + +++++++++++++++++++++++++++++++++++++++++++++++++++++ Testing bug 83062 All tests PASSEDSee log: RegressionTest_20120126112643.log: bug 83062 log
Bug #85327 (glite-wms-wmproxy.restart.cron missing HOSTNAME environment variable) FIXED
[capannini@devel15 regression_tests]$ python execute_regTest.py -d 3 -b 85327 -l +++++++++++++++++++++++++++++++++++++++++++++++++++++ + Regression TestSuite .... + +++++++++++++++++++++++++++++++++++++++++++++++++++++ Testing bug 85327 All tests PASSEDSee log: RegressionTest_20120126113216.log: bug 85327 log
Bug #86485 (ICE doesn't send the iceId to CREAM) FIXED
Submitted a job through the WMS to a CREAM CE in debug mode.
Verified that in the CREAM log file the iceid is reported:
# grep selectToRetrieveJobStatusQuery /var/log/cream/glite-ce-cream.log 20 Oct 2011 07:11:43,186 DEBUG org.glite.ce.cream.jobmanagement.db.table.JobStatusTable (JobStatusTable.java:419) - (TP-Processor25) selectToRetrieveJobStatusQuery = select job_status.type AS type,job_status.exitCode AS exitCode,job_status.failureReason AS failureReason,job_status.description AS description,job_status.time_stamp AS time_stamp,job_status.jobId AS jobId,job_status.id AS id,job.gridJobId AS gridJobId,job.workerNode AS workerNode from job_status, job where job_status.jobId = job.id and job_status.type IN ( '10', '5', '8', '7', '6', '4', '3') and job.userId = '_C_IT_O_INFN_OU_Personal_Certificate_L_Padova_CN_Massimo_Sgaravatto_dteam_Role_NULL_Capability_NULL' and job.iceId = 'C_IT_O_INFN_OU_Host_L_Padova_CN_cream-44.pd.infn.it' and job_status.id >= 0 and time_stamp >= '2011-10-19 15:46:36.0' ORDER BY job_status.id limit 500Verified also that in the CREAMDB the iceid is set for this job:
mysql> use creamdb;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> select iceId from job where id="CREAM806900601"
-> ;
+-----------------------------------------------------+
| iceId |
+-----------------------------------------------------+
| C_IT_O_INFN_OU_Host_L_Padova_CN_cream-44.pd.infn.it |
+-----------------------------------------------------+
1 row in set (0.00 sec)
===> test PASSED
Bug #86490 (EMI WMS stops accepting jobs after 31999th submission for the same DN (on ext3)) FIXED
[capannini@devel15 regression_tests]$ python execute_regTest.py -d 3 -b 86490 -l +++++++++++++++++++++++++++++++++++++++++++++++++++++ + Regression TestSuite .... + +++++++++++++++++++++++++++++++++++++++++++++++++++++ Testing bug 86490 All tests PASSEDSee logs: RegressionTest_20120126120305.log: bug 86490 log
Bug #86682 ( yaim changes ownership of /var) FIXED
[capannini@devel15 regression_tests]$ python execute_regTest.py -d 3 -b 86682 -l +++++++++++++++++++++++++++++++++++++++++++++++++++++ + Regression TestSuite .... + +++++++++++++++++++++++++++++++++++++++++++++++++++++ Testing bug 86682 All tests PASSEDSee log: RegressionTest_20120126120624.log: bug 86682 log
Bug #86769 (EMI WMS Update 6: WMS 'hangs' after several hours of operation) FIX NOT CERTIFIED
It is not easy to verify this bug, but the WMS has passed some stress test of one or two days of submissions. It also has passed a test done by the user who posted the bug.
Bug #86956 (Match-matching of requests with access to catalogues can cause the WM to crash) FIX NOT CERTIFIED
To use the function specified in the bug we need a jdl with DataRequirements. In any case the error is not deterministic, so you can only set this bug as hopefully fixed.
[ale@cream-12 DataReq]$ cat data-req.jdl
###########################################
# JDL with Data Requirements #
###########################################
Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";
Prologue = "prologue.sh";
FuzzyRank = true;
Environment={"LFC_HOST=prod-lfc-shared-central.cern.ch", "LFC_HOME=prod-lfc-shared-central.cern.ch:/grid/dteam/aleph"};
InputSandbox = {"prologue.sh"};
OutputSandbox = {"std.out", "std.err","prologue.out", "file.txt"};
requirements = true;
DataRequirements = {
[
DataCatalogType = "DLI";
DataCatalog ="http://prod-lfc-shared-central.cern.ch:8085/";
InputData = { "guid:17c667b7-9622-4c95-be68-3d15d1eacadb" };
]
};
DataAccessProtocol = "gsiftp";
AllowZippedISB=true;
RetryCount = 1;
ShallowRetryCount = 2;
[ale@cream-12 DataReq]$ glite-wms-job-list-match -a -c /home/ale/UI/etc/wmp_devel19.conf data-req.jdl
Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server
==========================================================================
COMPUTING ELEMENT IDs LIST
The following CE(s) matching your job requirements have been found:
*CEId*
- cert-15.pd.infn.it:8443/cream-lsf-cert
- cert-37.pd.infn.it:8443/cream-lsf-cert
- cream-19.pd.infn.it:8443/cream-lsf-cert
- cream-19.pd.infn.it:8443/cream-lsf-creamcert1
- cream-19.pd.infn.it:8443/cream-lsf-creamcert2
- cream-26.pd.infn.it:2119/jobmanager-lcgpbs-cert
- cream-26.pd.infn.it:2119/jobmanager-lcgpbs-creamtest1
- cream-26.pd.infn.it:2119/jobmanager-lcgpbs-creamtest2
- cream-31.pd.infn.it:8443/cream-pbs-creamtest1
- cream-31.pd.infn.it:8443/cream-pbs-creamtest2
- cream-38.pd.infn.it:8443/cream-pbs-creamtest1
- cream-38.pd.infn.it:8443/cream-pbs-creamtest2
- prod-ce-01.pd.infn.it:8443/cream-lsf-cert
- prod-ce-02.pd.infn.it:2119/jobmanager-lcglsf-cert
- cream-24.pd.infn.it:8443/cream-lsf-cert
- cream-24.pd.infn.it:8443/cream-lsf-testbedB_1
- cream-24.pd.infn.it:8443/cream-lsf-testbedB_2
- cream-41.pd.infn.it:8443/cream-pbs-creamtest1
- cream-32.pd.infn.it:8443/cream-pbs-cert
- cream-32.pd.infn.it:8443/cream-pbs-creamtest1
- cream-32.pd.infn.it:8443/cream-pbs-creamtest2
- ce130.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam
- ce130.cern.ch:2119/jobmanager-lcglsf-grid_dteam
- ce131.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam
- ce131.cern.ch:2119/jobmanager-lcglsf-grid_dteam
- ce132.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam
- ce132.cern.ch:2119/jobmanager-lcglsf-grid_dteam
- ce133.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam
- ce133.cern.ch:2119/jobmanager-lcglsf-grid_dteam
- ce203.cern.ch:8443/cream-lsf-grid_2nh_dteam
- ce203.cern.ch:8443/cream-lsf-grid_dteam
- ce204.cern.ch:8443/cream-lsf-grid_2nh_dteam
- ce204.cern.ch:8443/cream-lsf-grid_dteam
- ce205.cern.ch:8443/cream-lsf-grid_2nh_dteam
- ce205.cern.ch:8443/cream-lsf-grid_dteam
- ce206.cern.ch:8443/cream-lsf-grid_2nh_dteam
- ce206.cern.ch:8443/cream-lsf-grid_dteam
- ce207.cern.ch:8443/cream-lsf-grid_2nh_dteam
- ce207.cern.ch:8443/cream-lsf-grid_dteam
- ce208.cern.ch:8443/cream-lsf-grid_2nh_dteam
- ce208.cern.ch:8443/cream-lsf-grid_dteam
- cream-11.pd.infn.it:2119/jobmanager-lcglsf-cert
- cream-11.pd.infn.it:2119/jobmanager-lcglsf-creamtest1
- cream-11.pd.infn.it:2119/jobmanager-lcglsf-creamtest2
- cream-20.pd.infn.it:8443/cream-lsf-cert
- cream-20.pd.infn.it:8443/cream-lsf-creamcert1
- cream-20.pd.infn.it:8443/cream-lsf-creamcert2
- cream-31.pd.infn.it:8443/cream-pbs-cert
- cream-41.pd.infn.it:8443/cream-pbs-cert
- cream-41.pd.infn.it:8443/cream-pbs-creamtest2
==========================================================================
Bug #87259 (WMS purger errors in jobcontroller/logmonitor) FIXED
For a job submitted to a LCG-CE, verified that in /var/log/glite/logmonitor_events.log the removing of job directory, submit file, classad file, and wrapper file is logged:
28 Dec, 16:39:49 -I- EventAborted::process_event(): Job id = https://emi-demo11.cnaf.infn.it:9000/vHF_dDR46zwWtOPwA3O8Vw
28 Dec, 16:39:49 -I- JobFilePurger::do_purge(...): Removing job directory: /var/jobcontrol/condorio/vH/https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fvHF_5fdDR46zwWtOPwA3O8Vw
28 Dec, 16:39:49 -U- JobFilePurger::do_purge(...): Removed 1 files.
28 Dec, 16:39:49 -I- JobFilePurger::do_purge(...): Removing submit file: /var/jobcontrol/submit/vH/Condor.https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fvHF_5fdDR46zwWtOPwA3O8Vw.submit
28 Dec, 16:39:49 -U- JobFilePurger::do_purge(...): Removed...
28 Dec, 16:39:49 -I- JobFilePurger::do_purge(...): Removing classad file: /var/jobcontrol/submit/vH/ClassAd.https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fvHF_5fdDR46zwWtOPwA3O8Vw
28 Dec, 16:39:49 -U- JobFilePurger::do_purge(...): Removed...
28 Dec, 16:39:49 -I- JobFilePurger::do_purge(...): Removing wrapper file: /var/jobcontrol/submit/vH/JobWrapper.https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2fvHF_5fdDR46zwWtOPwA3O8Vw.sh
28 Dec, 16:39:49 -U- JobFilePurger::do_purge(...): Removed...
Verified that they have been actually removed:
[root@emi-demo11 ~]# ls /var/jobcontrol/condorio/vH/
[root@emi-demo11 ~]# ls /var/jobcontrol/submit/vH/
Submitted a job:
[capannini@devel15 WMS-service]$ glite-wms-job-submit -a -c ~/wms_emi-demo11.conf ~/ls.jdl
Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
====================== glite-wms-job-submit Success ======================
The job has been successfully submitted to the WMProxy
Your job identifier is:
https://emi-demo11.cnaf.infn.it:9000/-iL7RrvLDu1Z9jFultvSLA
==========================================================================
Copied the sandbox dir:
[root@emi-demo11 ~]# cd /var/SandboxDir/-i/
[root@emi-demo11 -i]# ls
https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2f-iL7RrvLDu1Z9jFultvSLA
[root@emi-demo11 -i]# cp -r -p https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2f-iL7RrvLDu1Z9jFultvSLA https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2f-iL7RrvLDu1Z9jFultvSLA.old
[root@emi-demo11 -i]#
[root@emi-demo11 -i]#
[root@emi-demo11 -i]# ls -l
totale 8
drwxrwx--- 5 dteam045 glite 4096 26 gen 12:23 https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2f-iL7RrvLDu1Z9jFultvSLA
drwxrwx--- 5 dteam045 glite 4096 26 gen 12:23 https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2f-iL7RrvLDu1Z9jFultvSLA.old
Retrieved the OSB when the job is done:
[capannini@devel15 WMS-service]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/-iL7RrvLDu1Z9jFultvSLA
======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/-iL7RrvLDu1Z9jFultvSLA
Current Status: Done (Success)
Exit code: 0
Status Reason: Job Terminated Successfully
Destination: atlasce2.lnf.infn.it:8443/cream-pbs-cert
Submitted: Thu Jan 26 12:23:29 2012 CET
==========================================================================
[capannini@devel15 WMS-service]$ glite-wms-job-output https://emi-demo11.cnaf.infn.it:9000/-iL7RrvLDu1Z9jFultvSLA
Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
================================================================================
JOB GET OUTPUT OUTCOME
Output sandbox files for the job:
https://emi-demo11.cnaf.infn.it:9000/-iL7RrvLDu1Z9jFultvSLA
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/capannini_-iL7RrvLDu1Z9jFultvSLA
================================================================================
[capannini@devel15 WMS-service]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/-iL7RrvLDu1Z9jFultvSLA
======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/-iL7RrvLDu1Z9jFultvSLA
Current Status: Cleared
Status Reason: user retrieved output sandbox
Destination: atlasce2.lnf.infn.it:8443/cream-pbs-cert
Submitted: Thu Jan 26 12:23:29 2012 CET
==========================================================================
Restored the sandbox dir in the WMS:
[root@emi-demo11 -i]# mv https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2f-iL7RrvLDu1Z9jFultvSLA.old https_3a_2f_2femi-demo11.cnaf.infn.it_3a9000_2f-iL7RrvLDu1Z9jFultvSLA
Purged the job from LBProxy and from LBServer:
[root@emi-demo11 ~]# echo 'https://emi-demo11.cnaf.infn.it:9000/-iL7RrvLDu1Z9jFultvSLA' >jobid.txt
[root@emi-demo11 ~]#
[root@emi-demo11 ~]# /usr/bin/glite-lb-purge -m emi-demo11.cnaf.infn.it -x -j jobid.txt
The jobs were not dumped.
[root@emi-demo11 ~]# /usr/bin/glite-lb-purge -m emi-demo11.cnaf.infn.it -j jobid.txt
The jobs were not dumped.
Verified that job status returns =Identifier removed= (i.e. error =EIDRM=):
[capannini@devel15 WMS-service]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/-iL7RrvLDu1Z9jFultvSLA
**** Error: API_NATIVE_ERROR ****
Error while calling the "Job:getStatus" native api
Unable to retrieve the status for: https://emi-demo11.cnaf.infn.it:9000/-iL7RrvLDu1Z9jFultvSLA
glite.lb.Exception: edg_wll_JobStatus: Identifier removed: matching job already purged
at glite::lb::Job::status[./src/Job.cpp:87]
On the WMS, purged the sandboxdir, as user glite, invoking the command =glite-wms-purgeStorage.sh= and verifying that =Identifier removed= (i.e. error 43) is returned:
[glite@emi-demo11 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/-i
26 Jan, 15:05:17 -E: [Error] query_job_status(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:134): https://emi-demo11.cnaf.infn.it:9000/-iL7RrvLDu1Z9jFultvSLA: edg_wll_JobStat [43] Identifier removed(matching job already purged)
26 Jan, 15:05:17 -I: [Info] operator()(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:306): https://emi-demo11.cnaf.infn.it:9000/-iL7RrvLDu1Z9jFultvSLA: forced removal, unknown/removed L&B job
Verified that the sandboxdir has been removed:
[glite@emi-demo11 ~]$ ls -l /var/SandboxDir/-i
total 0
===> test PASSED
Bug #87994 yaim-wms doesn't fill the attributes get_acbr and get_owner in the conf file for Glue2 FIXED
Check the attributes get_acbr and get_owner in /etc/glite/info/service/glite-info-glue2-wmproxy.conf
[root@emi-demo11 ~]# grep get_acbr /etc/glite/info/service/glite-info-glue2-wmproxy.conf
get_acbr = echo -e "VO:atlas\nVO:alice\nVO:lhcb\nVO:cms\nVO:dteam\nVO:biomed\nVO:ops\nVO:infngrid\nVO:testers.eu-emi.eu"
[root@emi-demo11 ~]# grep get_owner /etc/glite/info/service/glite-info-glue2-wmproxy.conf
get_owner = echo -e "atlas\nalice\nlhcb\ncms\ndteam\nbiomed\nops\ninfngrid\ntesters.eu-emi.eu"
===> test PASSED
Bug #88072 EMI-1 WMS cannot submit to ARC CEs FIXED
-bash-3.2$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/pl6ZSNy7MNWNSH6x8650DQ
======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:
Status info for the Job : https://devel19.cnaf.infn.it:9000/pl6ZSNy7MNWNSH6x8650DQ
Current Status: Done (Success)
Exit code: 0
Status Reason: Job terminated successfully
Destination: jade-cms.hip.fi:2811/nordugrid-GE-arc
Submitted: Mon Nov 21 16:01:15 2011 CET
==========================================================================
-bash-3.2$ glite-wms-job-output https://devel19.cnaf.infn.it:9000/pl6ZSNy7MNWNSH6x8650DQ
Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server
================================================================================
JOB GET OUTPUT OUTCOME
Output sandbox files for the job:
https://devel19.cnaf.infn.it:9000/pl6ZSNy7MNWNSH6x8650DQ
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/traldi_pl6ZSNy7MNWNSH6x8650DQ
================================================================================
-bash-3.2$ cat /tmp/jobOutput/traldi_pl6ZSNy7MNWNSH6x8650DQ/out.txt
total 32
drwx------ 2 cmsuser43 cmsuser 4096 Nov 21 17:02 .
drwx------ 3 cmsuser43 cmsuser 4096 Nov 21 17:02 ..
-rw------- 1 cmsuser43 cmsuser 118 Nov 21 17:02 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2fpl6ZSNy7MNWNSH6x8650DQ.output
-rw-r--r-- 1 cmsuser43 cmsuser 0 Nov 21 17:02 out.txt
-rw-r--r-- 1 cmsuser43 cmsuser 1359 Nov 21 17:02 pippo.jdl
-rw------- 1 cmsuser43 cmsuser 2 Nov 21 17:02 tmp.bUujW32646
-rw------- 1 cmsuser43 cmsuser 103 Nov 21 17:02 tmp.cNkaG32632
-rw------- 1 cmsuser43 cmsuser 103 Nov 21 17:02 tmp.exhlZ32647
-rw------- 1 cmsuser43 cmsuser 2 Nov 21 17:02 tmp.gZIii32631
-rw------- 1 cmsuser43 cmsuser 0 Nov 21 17:02 tmp.SdwGF32645
[ale@cream-12 WMS-service]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/pl6ZSNy7MNWNSH6x8650DQ
======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:
Status info for the Job : https://devel19.cnaf.infn.it:9000/pl6ZSNy7MNWNSH6x8650DQ
Current Status: Cleared
Status Reason: user retrieved output sandbox
Destination: jade-cms.hip.fi:2811/nordugrid-GE-arc
Submitted: Mon Nov 21 16:01:15 2011 CET
==========================================================================
===> test PASSED
Bug #88096 CA certificate verification failed FIXED NOT CERTIFIED
It is not easy to verify this bug, but the WMS has passed some stress test of one or two days of submissions.
Bug #88097 Collection not terminated FIXED NOT CERTIFIED
It is not easy to verify this bug, but the WMS has passed some stress test of one or two days of submissions.
Bug #88450 Ice configuration section doesn't use environment variable FIXED
[root@cream-44 ~]# cat /etc/glite-wms/glite_wms.conf | grep -i " logfile"
logfile = "${WMS_LOCATION_LOG}/ice.log";
LogFile = "${WMS_LOCATION_LOG}/wmproxy.log";
LogFile = "${WMS_LOCATION_LOG}/networkserver_events.log";
LogFile = "${WMS_LOCATION_LOG}/workload_manager_events.log";
LogFile = "${WMS_LOCATION_LOG}/logmonitor_events.log";
LogFile = "${WMS_LOCATION_LOG}/jobcontoller_events.log";
===> test PASSED
Bug #88558 WM creates submit requests with old sequence code upon a replan FIXED
Event: Resubmission
- Arrived = Wed Nov 23 19:16:16 2011 CET
- Host = devel19.cnaf.infn.it
- Level = SYSTEM
- Priority = synchronous
- Reason = token still exists
- Result = SHALLOW
- Seqcode = UI=000000:NS=0000000004:WM=000007:BH=0000000000:JSS=000002:LM=000000:LRMS=000000:APP=000000:LBS=000000
- Source = WorkloadManager
- Src instance = 13159
- Tag = job will be replanned
- Timestamp = Wed Nov 23 19:16:16 2011 CET
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
[...]
Event: EnQueued
- Arrived = Wed Nov 23 19:16:16 2011 CET
- Host = devel19.cnaf.infn.it
- Level = SYSTEM
- Priority = synchronous
- Queue = /var/workload_manager/jobdir
- Result = OK
- Seqcode = UI=000000:NS=0000000004:WM=000009:BH=0000000000:JSS=000002:LM=000000:LRMS=000000:APP=000000:LBS=000000
- Source = WorkloadManager
- Src instance = 13159
- Timestamp = Wed Nov 23 19:16:16 2011 CET
- User = /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
- Job =
[
version = "1.0.0";
arguments =
[
ad =
[
NodeName = "Node_jdl_953";
FuzzyRank = true;
CEInfoHostName = "cream-31.pd.infn.it";
OutputSandboxPath = "/var/SandboxDir/ji/https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2fjiHL26jdUVDO7Bl6BJlhEQ/output";
StdOutput = "out.log";
GlobusResourceContactString = "cream-31.pd.infn.it:8443/cream-pbs";
QueueName = "creamtest2";
ShallowRetryCount = 3;
RetryCount = 2;
Executable = "/bin/sleep";
Type = "job";
LB_sequence_code = "UI=000000:NS=0000000004:WM=000009:BH=0000000000:JSS=000002:LM=000000:LRMS=000000:APP=000000:LBS=000000";
===> test PASSED
Bug #88569 WMS: LCMAPS should log on file only and not on syslog FIXED
Check that no messages about LCMAPS have been logged by wmproxy on syslog:
[root@cream-44 ~]# grep glite_wms_wmproxy_server /var/log/messages* | grep -i "LCMAPS" [root@cream-44 ~]#And also that there is a log file for lcmaps:
[root@cream-44 ~]# ls -l /var/log/glite/lcmaps.log -rw-r--r-- 1 glite glite 659344 Nov 21 15:43 /var/log/glite/lcmaps.log===> test PASSED
Nagios probes test
Clean installation
Environment
Clean SL5 installation plus an EMI-UI according to EMI guidelines.[root@devel15 ~]# cat /etc/emi-version 1.6.0-1
Repositories
- wget --no-check-certificate https://twiki.cern.ch/twiki/pub/EMI/EMI1Updates/testing.repo -O /etc/yum.repos.d/testing.repo
- wget http://repository.egi.eu/sw/production/sam/1/repofiles/sam.repo -O /etc/yum.repos.d/sam.repo
Install
yum install emi-wms-nagios[root@devel15 ~]# yum install emi-wms-nagios Loaded plugins: downloadonly, kernel-module, priorities, protect-packages, protectbase, security, verify, versionlock Repository EGI-trustanchors is listed more than once in the configuration EMI-1-base | 1.9 kB 00:00 EMI-1-testing-base | 1.9 kB 00:00 EMI-1-third-party | 1.9 kB 00:00 EMI-1-updates | 1.9 kB 00:00 Oracle | 1.9 kB 00:00 core | 1.1 kB 00:00 egi-sam | 1.9 kB 00:00 extras | 3.7 kB 00:00 4168 packages excluded due to repository priority protections 23 packages excluded due to repository protections Reading version lock configuration Setting up Install Process Resolving Dependencies --> Running transaction check ---> Package emi-wms-nagios.noarch 0:1.0.0-1.el5 set to be updated --> Processing Dependency: nagios-submit-conf >= 0.2 for package: emi-wms-nagios --> Processing Dependency: python-GridMon >= 1.1.10 for package: emi-wms-nagios --> Processing Dependency: python-ldap for package: emi-wms-nagios --> Running transaction check ---> Package nagios-submit-conf.noarch 0:0.2-2.el5 set to be updated ---> Package python-GridMon.noarch 0:1.1.12-1.el5 set to be updated ---> Package python-ldap.x86_64 0:2.2.0-2.1 set to be updated --> Finished Dependency Resolution Beginning Kernel Module Plugin Finished Kernel Module Plugin Dependencies Resolved ============================================================================================================================================================= Package Arch Version Repository Size ============================================================================================================================================================= Installing: emi-wms-nagios noarch 1.0.0-1.el5 EMI-1-testing-base 61 k Installing for dependencies: nagios-submit-conf noarch 0.2-2.el5 egi-sam 3.4 k python-GridMon noarch 1.1.12-1.el5 egi-sam 153 k python-ldap x86_64 2.2.0-2.1 sl-base 123 k Transaction Summary ============================================================================================================================================================= Install 4 Package(s) Upgrade 0 Package(s) Total download size: 340 k Is this ok [y/N]: y Downloading Packages: (1/4): nagios-submit-conf-0.2-2.el5.noarch.rpm | 3.4 kB 00:00 (2/4): emi-wms-nagios-1.0.0-1.el5.noarch.rpm | 61 kB 00:00 (3/4): python-ldap-2.2.0-2.1.x86_64.rpm | 123 kB 00:00 (4/4): python-GridMon-1.1.12-1.el5.noarch.rpm | 153 kB 00:00 ------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 85 kB/s | 340 kB 00:04 Running rpm_check_debug Running Transaction Test Finished Transaction Test Transaction Test Succeeded Running Transaction Installing : python-ldap 1/4 Installing : nagios-submit-conf 2/4 Installing : python-GridMon 3/4 Installing : emi-wms-nagios 4/4 Installed: emi-wms-nagios.noarch 0:1.0.0-1.el5 Dependency Installed: nagios-submit-conf.noarch 0:0.2-2.el5 python-GridMon.noarch 0:1.1.12-1.el5 python-ldap.x86_64 0:2.2.0-2.1 Complete!
Upgrade from production
N/A: This is the initial release. No previous version to upgrade from.Unit Tests
N/A: The nagios probes are interpretted python scripts. Unit tests do not apply.Regression Tests
N/A: This is the initial release. No associated bugs exist.Functional Tests
Preparation
Log as a user and create a valid voms proxyWMSProbe
- State + Monit
[capannini@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u504 -H emi-demo11.cnaf.infn.it -m emi.wms.WMS-JobState
OK:
OK:
Testing from: devel15.cnaf.infn.it
DN: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy
VOMS FQANs: /dteam/Role=NULL/Capability=NULL, /dteam/NGI_IT/Role=NULL/Capability=NULL
Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
====================== glite-wms-job-submit Success ======================
The job has been successfully submitted to the WMProxy
Your job identifier is:
https://emi-demo11.cnaf.infn.it:9000/nc4oU4nTtQMspwuhs0CK2A
The job identifier has been saved in the following file:
/var/lib/gridprobes/dteam/emi.wms/WMS/emi-demo11.cnaf.infn.it/jobID
==========================================================================
[capannini@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u504 -H emi-demo11.cnaf.infn.it -m emi.wms.WMS-JobMonit --pass-check-dest active
metric results >>> <emi-demo11.cnaf.infn.it,emi.wms.WMS-JobState-dteam>
OK: [Running] https://emi-demo11.cnaf.infn.it:9000/nc4oU4nTtQMspwuhs0CK2A
OK: [Running] https://emi-demo11.cnaf.infn.it:9000/nc4oU4nTtQMspwuhs0CK2A
glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/nc4oU4nTtQMspwuhs0CK2A
======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:
Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/nc4oU4nTtQMspwuhs0CK2A
Current Status: Running
Status Reason: Job successfully submitted to Globus
Destination: cmsce01.na.infn.it:2119/jobmanager-lcgpbs-cert
Submitted: Thu Jan 26 16:14:33 2012 CET
==========================================================================
OK: Jobs processed - 1
OK: Jobs processed - 1
[Running] : 1|jobs_processed=1;; DONE=0;; RUNNING=1;; SCHEDULED=0;; SUBMITTED=0;; READY=0;; WAITING=0;; ABORTED=0;; CANCELLED=0;; CLEARED=0;; MISSED=0;; UNDETERMINED=0;; unknown=0;1;2
[capannini@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u504 -H emi-demo11.cnaf.infn.it -m emi.wms.WMS-JobState
OK: Active job - Running [2012-01-26T15:16:24Z]
OK: Active job - Running [2012-01-26T15:16:24Z]
Testing from: devel15.cnaf.infn.it
DN: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy
VOMS FQANs: /dteam/Role=NULL/Capability=NULL, /dteam/NGI_IT/Role=NULL/Capability=NULL
Active job - Running [2012-01-26T15:16:24Z]
[capannini@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u504 -H emi-demo11.cnaf.infn.it -m emi.wms.WMS-JobMonit --pass-check-dest active
metric results >>> <emi-demo11.cnaf.infn.it,emi.wms.WMS-JobSubmit-dteam>
OK: success.
glite-wms-job-output --noint --nosubdir --dir /var/lib/gridprobes/dteam/emi.wms/WMS/emi-demo11.cnaf.infn.it/jobOutput https://emi-demo11.cnaf.infn.it:9000/nc4oU4nTtQMspwuhs0CK2A 2>&1
Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
Warning - option --nosubdir specified:
output files with same name will be overridden
Warning - Directory already exists:
/var/lib/gridprobes/dteam/emi.wms/WMS/emi-demo11.cnaf.infn.it/jobOutput
================================================================================
JOB GET OUTPUT OUTCOME
Output sandbox files for the job:
https://emi-demo11.cnaf.infn.it:9000/nc4oU4nTtQMspwuhs0CK2A
have been successfully retrieved and stored in the directory:
/var/lib/gridprobes/dteam/emi.wms/WMS/emi-demo11.cnaf.infn.it/jobOutput
================================================================================
metric results >>> <emi-demo11.cnaf.infn.it,emi.wms.WMS-JobState-dteam>
OK: success.
glite-wms-job-output --noint --nosubdir --dir /var/lib/gridprobes/dteam/emi.wms/WMS/emi-demo11.cnaf.infn.it/jobOutput https://emi-demo11.cnaf.infn.it:9000/nc4oU4nTtQMspwuhs0CK2A 2>&1
Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server
Warning - option --nosubdir specified:
output files with same name will be overridden
Warning - Directory already exists:
/var/lib/gridprobes/dteam/emi.wms/WMS/emi-demo11.cnaf.infn.it/jobOutput
================================================================================
JOB GET OUTPUT OUTCOME
Output sandbox files for the job:
https://emi-demo11.cnaf.infn.it:9000/nc4oU4nTtQMspwuhs0CK2A
have been successfully retrieved and stored in the directory:
/var/lib/gridprobes/dteam/emi.wms/WMS/emi-demo11.cnaf.infn.it/jobOutput
================================================================================
OK: Jobs processed - 1
OK: Jobs processed - 1
Done : 1|jobs_processed=1;; DONE=1;; RUNNING=0;; SCHEDULED=0;; SUBMITTED=0;; READY=0;; WAITING=0;; ABORTED=0;; CANCELLED=0;; CLEARED=0;; MISSED=0;; UNDETERMINED=0;; unknown=0;1;2
[capannini@devel15 ~]$ cat /var/lib/gridprobes/dteam/emi.wms/WMS/emi-demo11.cnaf.infn.it/jobOutput/gridjob.out
cmswn04.na.infn.it
Summary: PASSED
- State + Monit + Cancel
[capannini@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u504 -H emi-demo11.cnaf.infn.it -m emi.wms.WMS-JobState OK: OK: Testing from: devel15.cnaf.infn.it DN: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy VOMS FQANs: /dteam/Role=NULL/Capability=NULL, /dteam/NGI_IT/Role=NULL/Capability=NULL Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server ====================== glite-wms-job-submit Success ====================== The job has been successfully submitted to the WMProxy Your job identifier is: https://emi-demo11.cnaf.infn.it:9000/HnfZcXc--G7TtJ0fzhYGcg The job identifier has been saved in the following file: /var/lib/gridprobes/dteam/emi.wms/WMS/emi-demo11.cnaf.infn.it/jobID ========================================================================== [capannini@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u504 -H emi-demo11.cnaf.infn.it -m emi.wms.WMS-JobMonit --pass-check-dest active metric results >>> <emi-demo11.cnaf.infn.it,emi.wms.WMS-JobState-dteam> OK: [Scheduled] https://emi-demo11.cnaf.infn.it:9000/HnfZcXc--G7TtJ0fzhYGcg OK: [Scheduled] https://emi-demo11.cnaf.infn.it:9000/HnfZcXc--G7TtJ0fzhYGcg glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/HnfZcXc--G7TtJ0fzhYGcg ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/HnfZcXc--G7TtJ0fzhYGcg Current Status: Scheduled Status Reason: unavailable Destination: inaf-ce-01.ct.pi2s2.it:8443/cream-lsf-cert Submitted: Thu Jan 26 16:27:27 2012 CET ========================================================================== OK: Jobs processed - 1 OK: Jobs processed - 1 [Scheduled] : 1|jobs_processed=1;; DONE=0;; RUNNING=0;; SCHEDULED=1;; SUBMITTED=0;; READY=0;; WAITING=0;; ABORTED=0;; CANCELLED=0;; CLEARED=0;; MISSED=0;; UNDETERMINED=0;; unknown=0;1;2 [capannini@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u504 -H emi-demo11.cnaf.infn.it -m emi.wms.WMS-JobCancel OK: job cancelled OK: job cancelled Testing from: devel15.cnaf.infn.it DN: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini/CN=proxy VOMS FQANs: /dteam/Role=NULL/Capability=NULL, /dteam/NGI_IT/Role=NULL/Capability=NULL Job cancellation request sent: glite-wms-job-cancel --noint -i /var/lib/gridprobes/dteam/emi.wms/WMS/emi-demo11.cnaf.infn.it/jobID Job bookkeeping files deleted. [capannini@devel15 ~]$ glite-wms-job-status https://emi-demo11.cnaf.infn.it:9000/HnfZcXc--G7TtJ0fzhYGcg ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://emi-demo11.cnaf.infn.it:9000/HnfZcXc--G7TtJ0fzhYGcg Current Status: Cancelled Destination: inaf-ce-01.ct.pi2s2.it:8443/cream-lsf-cert Submitted: Thu Jan 26 16:27:27 2012 CET ==========================================================================Summary: PASSED -- FabioCapannini - 2011-12-28
| I | Attachment | Action | Size | Date | Who | Comment |
|---|---|---|---|---|---|---|
 log |
RegressionTest_20120126111141.log | manage | 1.6 K | 2012-01-26 - 10:14 | FabioCapannini | bug 81376 log |
| log |
RegressionTest_20120126112001.log | manage | 1.9 K | 2012-01-26 - 10:24 | FabioCapannini | bug 81568 log |
| log |
RegressionTest_20120126112643.log | manage | 2.5 K | 2012-01-26 - 10:28 | FabioCapannini | bug 83062 log |
| log |
RegressionTest_20120126113216.log | manage | 1.9 K | 2012-01-26 - 10:40 | FabioCapannini | bug 85327 log |
| log |
RegressionTest_20120126120305.log | manage | 16.8 K | 2012-01-26 - 11:04 | FabioCapannini | bug 86490 log |
| log |
RegressionTest_20120126120624.log | manage | 1.8 K | 2012-01-26 - 11:07 | FabioCapannini | bug 86682 log |
| log |
WMSService-TS_20120116115816.log | manage | 13.0 K | 2012-01-16 - 11:18 | FabioCapannini | sdj log |
| log |
WMSService-TS_20120116151500.log | manage | 17.8 K | 2012-01-16 - 14:45 | FabioCapannini | bulk collection log |
| log |
WMSService-TS_20120116154624.log | manage | 19.2 K | 2012-01-16 - 15:01 | FabioCapannini | bulk collection log |
| log |
WMSService-TS_20120116163257.log | manage | 16.1 K | 2012-01-16 - 15:42 | FabioCapannini | bulk collection log |
| log |
WMSService-TS_20120123104906.log | manage | 4.1 K | 2012-01-23 - 09:58 | FabioCapannini | list-match log |
| log |
WMSService-TS_20120123121055.log | manage | 4.1 K | 2012-01-23 - 11:12 | FabioCapannini | list-match log |
| log |
WMSService-TS_20120123121509.log | manage | 3.9 K | 2012-01-23 - 11:16 | FabioCapannini | list-match log |
| log |
WMSService-TS_20120124151737.log | manage | 31.1 K | 2012-01-24 - 14:20 | FabioCapannini | limiter log |
| log |
WMSService-TS_20120125120754.log | manage | 46.7 K | 2012-01-25 - 11:18 | FabioCapannini | purge logs |
| log |
WMSService-TS_20120126104752.log | manage | 69.0 K | 2012-01-26 - 10:37 | FabioCapannini | proxy renewal log |
 tgz |
cancel.tgz | manage | 8.8 K | 2012-01-24 - 10:09 | FabioCapannini | Job cancel logs |
| tgz |
coll.tgz | manage | 5.8 K | 2012-01-17 - 10:35 | FabioCapannini | all in a jdl collection logs |
| tgz |
dag.tgz | manage | 2.2 K | 2012-01-18 - 11:17 | FabioCapannini | dag logs |
| tgz |
delegation.tgz | manage | 1.9 K | 2012-01-20 - 09:34 | FabioCapannini | delegation logs |
| log |
install.log | manage | 124.2 K | 2011-12-28 - 11:04 | FabioCapannini | Clean install log |
| tgz |
parallel.tgz | manage | 2.3 K | 2012-01-18 - 15:37 | FabioCapannini | parallel job logs |
| tgz |
parametric.tgz | manage | 3.1 K | 2012-01-18 - 11:03 | FabioCapannini | parametric logs |
| tgz |
perusal.tgz | manage | 4.1 K | 2012-01-20 - 15:16 | FabioCapannini | perusal logs |
| tgz |
prolepilog.tgz | manage | 3.2 K | 2012-01-23 - 15:45 | FabioCapannini | prologue epilog logs |
| tgz |
resub.tgz | manage | 3.8 K | 2012-01-24 - 09:55 | FabioCapannini | resubmission logs |
| tgz |
submit.log.tgz | manage | 5.1 K | 2011-12-29 - 11:06 | FabioCapannini | Normal submission logs |
| log |
update.log | manage | 37.6 K | 2011-12-28 - 14:31 | FabioCapannini | Update log |
| log |
yaim-install.log | manage | 9.0 K | 2011-12-28 - 11:30 | FabioCapannini | yaim install log |
Topic revision: r34 - 2012-03-27 - FabioCapannini
Home 
Site map
CEMon web
Create New Topic
Index
Search
Changes
Notifications
RSS Feed
Statistics
Preferences
Account  |
 |
|
|
|
Copyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors. Ideas, requests, problems regarding TWiki? Send feedback