Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
WMS 3.5 pre-certification report | ||||||||
Line: 77 to 77 | ||||||||
======================================================================
| ||||||||
Changed: | ||||||||
< < | DAG jobs | |||||||
> > | DAG job | |||||||
[mcecchi@devel15 ~]$ cat dag2.jdl | ||||||||
Line: 218 to 218 | ||||||||
======================================================================
| ||||||||
Added: | ||||||||
> > | Parametric | |||||||
Changed: | ||||||||
< < | Job list-match[mcecchi@devel15 ~]$ glite-wms-job-list-match --rank -a --endpoint https://devel09.cnaf.infn.it:7443/glite_wms_wmproxy_server ls.jdl |tail - vgn003.hep.physik.uni-siegen.de:8443/cream-pbs-dteam 1 - vm3.tier2.hep.manchester.ac.uk:8443/cream-pbs-long 1 - vserv11.hep.phy.cam.ac.uk:8443/cream-pbs-express 1 - vserv11.hep.phy.cam.ac.uk:8443/cream-pbs-short 1 - wario.univ-lille1.fr:8443/cream-pbs-dteam 1 - wipp-crm.weizmann.ac.il:8443/cream-pbs-dteam 1 ========================================================================== ISM purchasers | |||||||
> > | Beware of bug #79027. Work with jdl-api-cpp-3.4.1-3 | |||||||
Changed: | ||||||||
< < | [root@devel09 workload_manager]# cat dump [command="ism_dump"] [root@devel09 workload_manager]# cp dump jobdir/new/ [root@devel09 workload_manager]# tail -f /var/log/wms/workload_manager_events.log ": OiBSMngxWlZOcGRHVlZibWx4ZFdWSlJEMUpURjlKVlVORFgwbEhJQT09", "GlueCEUniqueID=ngi-il-ce2.isragrid.org.il:8443/cream-sge-israelvo", "GlueCEUniqueID=ngi-il-ce2.isragrid.org.il:8443/cream-sge-dteam", "GlueCEUniqueID=ngi-il-ce2.isragrid.org.il:8443/cream-sge-ops" }; GlueClusterName = "ngi-il-ce2.isragrid.org.il"; GlueSchemaVersionMajor = 1; GlueInformationServiceURL = "ldap://ngi-il-ce2.isragrid.org.il:2170/mds-vo-name=resource,o=grid" | |||||||
> > | $ cat parametric.jdl [ ParameterStart = 0; ParameterStep = 1; Parameters = 3; JobType = "parametric"; Executable = "/bin/echo"; Arguments = "_PARAM_"; Environment = {"MYPATH_PARAM_=$PATH:/bin:/usr/bin:$HOME"}; StdOutput = "echo_PARAM_.out"; StdError = "echo_PARAM_.err"; OutputSandbox = {"echo_PARAM_.out","echo_PARAM_.err"}; Retrycount=-1; | |||||||
] | ||||||||
Deleted: | ||||||||
< < | 05 Feb, 10:43:47 -I: [Info] operator()(../../src/dispatcher_utils.cpp:292): new ism_dump
[root@devel09 workload_manager]# ll
total 112244
-rw-r--r-- 1 root root 21 Feb 5 10:43 dump
-rw-r--r-- 1 glite glite 114810221 Feb 5 10:43 ismdump.fl
drwxr-xr-x 5 glite glite 4096 Jan 30 13:10 jobdir
Job cancelCancel of a Running job:======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://devel09.cnaf.infn.it:9000/HXnAN9vrdSsSi33YxEZmAQ Current Status: Scheduled Status Reason: unavailable Destination: emi002.grid.sinica.edu.tw:8443/cream-pbs-testers_q Submitted: Mon Feb 4 16:00:08 2013 CET ========================================================================== | |||||||
Changed: | ||||||||
< < | [mcecchi@devel15 ~]$ glite-wms-job-cancel https://devel09.cnaf.infn.it:9000/HXnAN9vrdSsSi33YxEZmAQ![]() | |||||||
> > | $ glite-wms-job-submit -a --endpoint https://devel09.cnaf.infn.it:7443/glite_wms_wmproxy_server![]() | |||||||
Connecting to the service https://devel09.cnaf.infn.it:7443/glite_wms_wmproxy_server![]() | ||||||||
Changed: | ||||||||
< < | ========================= glite-wms-job-cancel Success =========================
The cancellation request has been successfully submitted for the following job(s): | |||||||
> > | ================== glite-wms-job-submit Success ================== | |||||||
Changed: | ||||||||
< < | - https://devel09.cnaf.infn.it:9000/HXnAN9vrdSsSi33YxEZmAQ![]() | |||||||
> > | The job has been successfully submitted to the WMProxy Your job identifier is: | |||||||
Changed: | ||||||||
< < | ==================================================================================== | |||||||
> > | https://devel09.cnaf.infn.it:9000/kf7upvOiNtOd7qAthY70lg![]() | |||||||
Added: | ||||||||
> > | ====================================================================== | |||||||
Changed: | ||||||||
< < | [mcecchi@devel15 ~]$ glite-wms-job-status https://devel09.cnaf.infn.it:9000/HXnAN9vrdSsSi33YxEZmAQ![]() | |||||||
> > | $ glite-wms-job-status https://devel09.cnaf.infn.it:9000/kf7upvOiNtOd7qAthY70lg![]() | |||||||
=================== glite-wms-job-status Success =================
BOOKKEEPING INFORMATION: | ||||||||
Changed: | ||||||||
< < | Status info for the Job : https://devel09.cnaf.infn.it:9000/HXnAN9vrdSsSi33YxEZmAQ![]() | |||||||
> > | Status info for the Job : https://devel09.cnaf.infn.it:9000/kf7upvOiNtOd7qAthY70lg![]() | |||||||
====================================================================== | ||||||||
Changed: | ||||||||
< < |
Cancel of a Done job:
[mcecchi@devel15 ~]$ glite-wms-job-cancel https://devel09.cnaf.infn.it:9000/XbwZJuMcbwQ68XVvHm6ztQ Are you sure you want to remove specified job(s) [y/n]y : y Error - Cancel not allowed Current Job Status is Done Certificate delegation[mcecchi@devel15 ~]$ glite-wms-job-delegate-proxy --endpoint https://devel09.cnaf.infn.it:7443/glite_wms_wmproxy_server -d mc Connecting to the service https://devel09.cnaf.infn.it:7443/glite_wms_wmproxy_server ================== glite-wms-job-delegate-proxy Success ================== Your proxy has been successfully delegated to the WMProxy(s): https://devel09.cnaf.infn.it:7443/glite_wms_wmproxy_server with the delegation identifier: mc | |||||||
> > | - Nodes information for:
Status info for the Job : https://devel09.cnaf.infn.it:9000/9eJkws7-aS5qcUKyZswydg![]() | |||||||
====================================================================== | ||||||||
Changed: | ||||||||
< < |
Check publication in EMIRPut in your siteinfo.def something like:BDII_EMIR_ENABLE=yes BDII_EMIR_SERP_PERIOD=1 BDII_EMIR_SERP_VALIDITY=2 BDII_EMIR_HOST=http://emitbdsr1.cern.ch:9126Then run: [root@devel09 ~]#PATH=$PATH:/sbin /opt/glite/yaim/bin/yaim -r -f config_emir_serp -s siteinfo_/site-info.def WARNING: WARNING: ***************************************************************************** WARNING: Your siteinfo directory is world readable. This is generally a bad idea WARNING: as configuration files may contain passwords and other sensitive information. WARNING: ***************************************************************************** WARNING: INFO: Configuring HOST: devel09.cnaf.infn.it INFO: Using site configuration file: siteinfo_/site-info.def INFO: ################################################################### . /'.-. ') . yA,-"-,( ,m,:/ ) .oo. oo o ooo o. .oo . / .-Y a a Y-. 8. .8' 8'8. 8 8b d'8 . / ~ ~ / 8' .8oo88. 8 8 8' 8 . (_/ '====' 8 .8' 8. 8 8 Y 8 . Y,-''-,Yy,-.,/ o8o o8o o88o o8o o8o o8o . I_))_) I_))_) | |||||||
> > | Status info for the Job : https://devel09.cnaf.infn.it:9000/DQULXLoEiZ81BHomXV9nfw![]() ====================================================================== | |||||||
Added: | ||||||||
> > | Status info for the Job : https://devel09.cnaf.infn.it:9000/LozDHI8Df9uz5GbAMesn_w![]() ====================================================================== | |||||||
Changed: | ||||||||
< < | current working directory: /root site-info.def date: Jan 30 15:39 siteinfo_/site-info.def yaim command: -r -f config_emir_serp -s siteinfo_/site-info.def log file: /opt/glite/yaim/bin/../log/yaimlog Wed Jan 30 15:50:43 CET 2013 : /opt/glite/yaim/bin/yaim | |||||||
> > | Status info for the Job : https://devel09.cnaf.infn.it:9000/varcOAEGkQ0QeIScEVd47w![]() ====================================================================== | |||||||
Changed: | ||||||||
< < | Installed YAIM versions: glite-lb-yaim 4.5.8-1 glite-yaim-bdii 4.3.13-1 glite-yaim-core 5.1.0-1 | |||||||
> > | Status info for the Job : https://devel09.cnaf.infn.it:9000/ykG2-jocfC4vxt3jqPvQAA![]() ====================================================================== | |||||||
Deleted: | ||||||||
< < | ####################################################################
| |||||||
Deleted: | ||||||||
< < | Check that the record has reached its destination
mcecchi@pccecchi ~ $ curl -v -H "Accept:application/xml" -X GET http://emitbdsr1.cern.ch:9126/services?Service_Endpoint_ID=devel09.cnaf.infn.it_wmproxy_2272271975_org.glite.wms.WMProxy_2272271975 * About to connect() to emitbdsr1.cern.ch port 9126 (#0) * Trying 128.142.136.66... connected * Connected to emitbdsr1.cern.ch (128.142.136.66) port 9126 (#0) > GET /services?Service_Endpoint_ID=devel09.cnaf.infn.it_wmproxy_2272271975_org.glite.wms.WMProxy_2272271975 HTTP/1.1 > User-Agent: curl/7.21.0 (i386-redhat-linux-gnu) libcurl/7.21.0 NSS/3.12.10.0 zlib/1.2.5 libidn/1.18 libssh2/1.2.4 > Host: emitbdsr1.cern.ch:9126 > Accept:application/xml > < HTTP/1.1 200 OK < Content-Type: application/xml < Vary: Accept-Encoding < Content-Length: 981 < Server: Jetty(8.1.4.v20120524) < * Connection #0 to host emitbdsr1.cern.ch left intact * Closing connection #0 <?xml version="1.0" encoding="UTF-8" standalone="yes"?><ns2:QueryResult xmlns="http://schemas.ogf.org/glue/2009/03/spec_2.0_r1" xmlns:ns2="http://www.eu-emi.eu/emiregistry" count="1"><Service BaseType="Entity" CreationTime="2013-01-30T15:22:41.000+01:00"><ID>devel09.cnaf.infn.it_wmproxy_2272271975</ID><Name>INFN-CNAF-WMProxy</Name><Extensions/><Type>org.glite.wms.WMProxy</Type><Location/><Endpoint CreationTime="2013-01-30T15:22:41.000+01:00"><ID>devel09.cnaf.infn.it_wmproxy_2272271975_org.glite.wms.WMProxy_2272271975</ID><URL>https://devel09.cnaf.infn.it:7443/glite_wms_wmproxy_server</URL><Capability>executionmanagement.candidatesetgenerator</Capability><Capability>executionmanagement.jobdescription</Capability><Capability>executionmanagement.jobmanager</Capability><Technology>webservice</Technology><InterfaceName>org.glite.wms.WMProxy</InterfaceName><InterfaceVersion>1.0</InterfaceVersion><QualityLevel>production</QualityLevel></Endpoint></Service></ns2:QueryResult>mcecchi@pccecchi ~ $ | |||||||
Added: | ||||||||
> > | Collection | |||||||
Deleted: | ||||||||
< < | JOB SUBMISSION OF A COLLECTION | |||||||
Deleted: | ||||||||
< < | dorigoa@cream-51 13:16:43 ~/JDLs/WMS>glite-wms-job-delegate-proxy -d foo -e https://devel09.cnaf.infn.it:7443/glite_wms_wmproxy_server![]() ![]() ============== glite-wms-job-delegate-proxy Success ==============
Your proxy has been successfully delegated to the WMProxy(s):
https://devel09.cnaf.infn.it:7443/glite_wms_wmproxy_server![]() ====================================================================== | |||||||
dorigoa@cream-51 13:18:59 ~/JDLs/WMS>cat coll_small.jdl [ type = "collection"; | ||||||||
Line: 715 to 598 | ||||||||
======================================================================
| ||||||||
Added: | ||||||||
> > |
Job list-match[mcecchi@devel15 ~]$ glite-wms-job-list-match --rank -a --endpoint https://devel09.cnaf.infn.it:7443/glite_wms_wmproxy_server ls.jdl |tail - vgn003.hep.physik.uni-siegen.de:8443/cream-pbs-dteam 1 - vm3.tier2.hep.manchester.ac.uk:8443/cream-pbs-long 1 - vserv11.hep.phy.cam.ac.uk:8443/cream-pbs-express 1 - vserv11.hep.phy.cam.ac.uk:8443/cream-pbs-short 1 - wario.univ-lille1.fr:8443/cream-pbs-dteam 1 - wipp-crm.weizmann.ac.il:8443/cream-pbs-dteam 1 ========================================================================== ISM purchasers[root@devel09 workload_manager]# cat dump [command="ism_dump"] [root@devel09 workload_manager]# cp dump jobdir/new/ [root@devel09 workload_manager]# tail -f /var/log/wms/workload_manager_events.log ": OiBSMngxWlZOcGRHVlZibWx4ZFdWSlJEMUpURjlKVlVORFgwbEhJQT09", "GlueCEUniqueID=ngi-il-ce2.isragrid.org.il:8443/cream-sge-israelvo", "GlueCEUniqueID=ngi-il-ce2.isragrid.org.il:8443/cream-sge-dteam", "GlueCEUniqueID=ngi-il-ce2.isragrid.org.il:8443/cream-sge-ops" }; GlueClusterName = "ngi-il-ce2.isragrid.org.il"; GlueSchemaVersionMajor = 1; GlueInformationServiceURL = "ldap://ngi-il-ce2.isragrid.org.il:2170/mds-vo-name=resource,o=grid" ] 05 Feb, 10:43:47 -I: [Info] operator()(../../src/dispatcher_utils.cpp:292): new ism_dump [root@devel09 workload_manager]# ll total 112244 -rw-r--r-- 1 root root 21 Feb 5 10:43 dump -rw-r--r-- 1 glite glite 114810221 Feb 5 10:43 ismdump.fl drwxr-xr-x 5 glite glite 4096 Jan 30 13:10 jobdirGlue 2.0 purchasers: 05 Feb, 17:11:45 -D: [Debug] fetch_bdii_ce_info_g2(../../../../src/ism/purchaser/ldap-utils-g2.cpp:1167): #98037 LDAP entries received in 169 seconds Job cancelCancel of a Running job:======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://devel09.cnaf.infn.it:9000/HXnAN9vrdSsSi33YxEZmAQ Current Status: Scheduled Status Reason: unavailable Destination: emi002.grid.sinica.edu.tw:8443/cream-pbs-testers_q Submitted: Mon Feb 4 16:00:08 2013 CET ========================================================================== [mcecchi@devel15 ~]$ glite-wms-job-cancel https://devel09.cnaf.infn.it:9000/HXnAN9vrdSsSi33YxEZmAQ Are you sure you want to remove specified job(s) [y/n]y : y Connecting to the service https://devel09.cnaf.infn.it:7443/glite_wms_wmproxy_server ============================= glite-wms-job-cancel Success ============================= The cancellation request has been successfully submitted for the following job(s): - https://devel09.cnaf.infn.it:9000/HXnAN9vrdSsSi33YxEZmAQ ======================================================================================== [mcecchi@devel15 ~]$ glite-wms-job-status https://devel09.cnaf.infn.it:9000/HXnAN9vrdSsSi33YxEZmAQ ======================= glite-wms-job-status Success ===================== BOOKKEEPING INFORMATION: Status info for the Job : https://devel09.cnaf.infn.it:9000/HXnAN9vrdSsSi33YxEZmAQ Current Status: Cancelled Destination: emi002.grid.sinica.edu.tw:8443/cream-pbs-testers_q Submitted: Mon Feb 4 16:00:08 2013 CET ==========================================================================Cancel of a Done job: [mcecchi@devel15 ~]$ glite-wms-job-cancel https://devel09.cnaf.infn.it:9000/XbwZJuMcbwQ68XVvHm6ztQ Are you sure you want to remove specified job(s) [y/n]y : y Error - Cancel not allowed Current Job Status is Done Certificate delegation[mcecchi@devel15 ~]$ glite-wms-job-delegate-proxy --endpoint https://devel09.cnaf.infn.it:7443/glite_wms_wmproxy_server -d mc Connecting to the service https://devel09.cnaf.infn.it:7443/glite_wms_wmproxy_server ================== glite-wms-job-delegate-proxy Success ================== Your proxy has been successfully delegated to the WMProxy(s): https://devel09.cnaf.infn.it:7443/glite_wms_wmproxy_server with the delegation identifier: mc ========================================================================== Check publication in EMIRPut in your siteinfo.def something like:BDII_EMIR_ENABLE=yes BDII_EMIR_SERP_PERIOD=1 BDII_EMIR_SERP_VALIDITY=2 BDII_EMIR_HOST=http://emitbdsr1.cern.ch:9126Then run: [root@devel09 ~]#PATH=$PATH:/sbin /opt/glite/yaim/bin/yaim -r -f config_emir_serp -s siteinfo_/site-info.def WARNING: WARNING: ***************************************************************************** WARNING: Your siteinfo directory is world readable. This is generally a bad idea WARNING: as configuration files may contain passwords and other sensitive information. WARNING: ***************************************************************************** WARNING: INFO: Configuring HOST: devel09.cnaf.infn.it INFO: Using site configuration file: siteinfo_/site-info.def INFO: ################################################################### . /'.-. ') . yA,-"-,( ,m,:/ ) .oo. oo o ooo o. .oo . / .-Y a a Y-. 8. .8' 8'8. 8 8b d'8 . / ~ ~ / 8' .8oo88. 8 8 8' 8 . (_/ '====' 8 .8' 8. 8 8 Y 8 . Y,-''-,Yy,-.,/ o8o o8o o88o o8o o8o o8o . I_))_) I_))_) current working directory: /root site-info.def date: Jan 30 15:39 siteinfo_/site-info.def yaim command: -r -f config_emir_serp -s siteinfo_/site-info.def log file: /opt/glite/yaim/bin/../log/yaimlog Wed Jan 30 15:50:43 CET 2013 : /opt/glite/yaim/bin/yaim Installed YAIM versions: glite-lb-yaim 4.5.8-1 glite-yaim-bdii 4.3.13-1 glite-yaim-core 5.1.0-1 #################################################################### INFO: The default location of the grid-env.(c)sh files will be: /usr/libexec INFO: Sourcing the utilities in /opt/glite/yaim/functions/utils INFO: Detecting environment INFO: Assuming the node types: WMSLB INFO: Using hostname: devel09.cnaf.infn.it WARNING: config_emir_serp function is not supported by WMSLB! WARNING: Anyway you can force this operation. WARNING: Would you really run config_emir_serp on WMSLB? [y/]: INFO: Executing function: config_emir_serp_check INFO: Executing function: config_emir_serp EMIR-SERP startup [ OK ] INFO: YAIM terminated succesfully.Check that the record has reached its destination mcecchi@pccecchi ~ $ curl -v -H "Accept:application/xml" -X GET http://emitbdsr1.cern.ch:9126/services?Service_Endpoint_ID=devel09.cnaf.infn.it_wmproxy_2272271975_org.glite.wms.WMProxy_2272271975 * About to connect() to emitbdsr1.cern.ch port 9126 (#0) * Trying 128.142.136.66... connected * Connected to emitbdsr1.cern.ch (128.142.136.66) port 9126 (#0) > GET /services?Service_Endpoint_ID=devel09.cnaf.infn.it_wmproxy_2272271975_org.glite.wms.WMProxy_2272271975 HTTP/1.1 > User-Agent: curl/7.21.0 (i386-redhat-linux-gnu) libcurl/7.21.0 NSS/3.12.10.0 zlib/1.2.5 libidn/1.18 libssh2/1.2.4 > Host: emitbdsr1.cern.ch:9126 > Accept:application/xml > < HTTP/1.1 200 OK < Content-Type: application/xml < Vary: Accept-Encoding < Content-Length: 981 < Server: Jetty(8.1.4.v20120524) < * Connection #0 to host emitbdsr1.cern.ch left intact * Closing connection #0 <?xml version="1.0" encoding="UTF-8" standalone="yes"?><ns2:QueryResult xmlns="http://schemas.ogf.org/glue/2009/03/spec_2.0_r1" xmlns:ns2="http://www.eu-emi.eu/emiregistry" count="1"><Service BaseType="Entity" CreationTime="2013-01-30T15:22:41.000+01:00"><ID>devel09.cnaf.infn.it_wmproxy_2272271975</ID><Name>INFN-CNAF-WMProxy</Name><Extensions/><Type>org.glite.wms.WMProxy</Type><Location/><Endpoint CreationTime="2013-01-30T15:22:41.000+01:00"><ID>devel09.cnaf.infn.it_wmproxy_2272271975_org.glite.wms.WMProxy_2272271975</ID><URL>https://devel09.cnaf.infn.it:7443/glite_wms_wmproxy_server</URL><Capability>executionmanagement.candidatesetgenerator</Capability><Capability>executionmanagement.jobdescription</Capability><Capability>executionmanagement.jobmanager</Capability><Technology>webservice</Technology><InterfaceName>org.glite.wms.WMProxy</InterfaceName><InterfaceVersion>1.0</InterfaceVersion><QualityLevel>production</QualityLevel></Endpoint></Service></ns2:QueryResult>mcecchi@pccecchi ~ $ | |||||||
Argus based authZsee bug #99092 test report, below. | ||||||||
Added: | ||||||||
> > | NOTA BENE: access control with Argus is very slow, to the point that it sometimes causes a DOS. NOT suggested for use in production. In these circumstances, the error is the one reported below.[mcecchi@devel15 ~]$ glite-wms-job-list-match --rank -a --endpoint https://devel09.cnaf.infn.it:7443/glite_wms_wmproxy_server ls.jdl | grep lcg Warning - Unable to perform the operation: Argus request on jobListMatch by DN: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi failed for some reason Error code: SOAP-ENV:Server | |||||||
LIST OF BUGS
| ||||||||
Line: 731 to 834 | ||||||||
| ||||||||
Added: | ||||||||
> > |
| |||||||
BUG VERIFICATION | ||||||||
Line: 1205 to 1309 | ||||||||
Added: | ||||||||
> > | BUG: WM memory footprint bloats, especially on 64-bit architectures (bug #97288). HOPEFULLY FIXEDThis bug cannot be checked in non production environments. A substantial change has been brought in the ldap purchasers, by removing legacy data structures and using a cleaner and unified approach. This needs to be tested in production | |||||||
BUG: glite-wms-job-output requires a valid delegated proxy on the WMS to enable output retrieval (bug #98524)To reproduce the bug submit a job producing at least one output sandbox file, to a WMS EMI2 server... | ||||||||
Line: 1323 to 1432 | ||||||||
Deleted: | ||||||||
< < | NOTA BENE: access control with Argus is very slow, to the point that it sometimes causes a DOS. NOT suggested for use in production. In these circumstances, the error is the one reported below.[mcecchi@devel15 ~]$ glite-wms-job-list-match --rank -a --endpoint https://devel09.cnaf.infn.it:7443/glite_wms_wmproxy_server ls.jdl | grep lcg Warning - Unable to perform the operation: Argus request on jobListMatch by DN: /C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Marco Cecchi failed for some reason Error code: SOAP-ENV:Server | |||||||
-- MarcoCecchi - 2013-01-16 |