Certification report for the internal patch 4366

Author(s): Alessio Gianelle

Outcome: CERTIFIED

Information

Patch	https://savannah.cern.ch/patch/index.php?4366
Build reports	http://etics-repository.cern.ch/repository/reports/id/37113781-c2b7-49f1-99b5-22eb63cc918a/sl5_x86_64_gcc412/-/reports/index.html
Repository	http://etics-repository.cern.ch/repository/pm/registered/repomd/name/patch_4388_1/index.html

Upgrade from production

• Upgrade steps:

  wget http://etics-repository.cern.ch/repository/pm/registered/repomd/name/patch_4366_1/etics-registered-build-by-name.repo -O /etc/yum.repos.d/glite-CREAMCLI.repo
  yum update
  

• view the log of YUM for the client upgrade

Automatic tests

• Run the following tests from the testsuite: output for lsf and pbs CE.
  • CREAM-cli-delegation.sh: OK
  • CREAM-cli-delegation-renew.sh: OK
  • CREAM-cli-job-submit: OK
  • CREAM-cli-job-status-simple.sh: OK
  • CREAM-cli-job-status-filtered.sh: OK
  • CREAM-cli-job-cancel.sh: OK
  • CREAM-cli-job-suspend.sh: OK
  • CREAM-cli-job-resume.sh: OK
  • CREAM-cli-job-purge.sh: OK
  • CREAM-cli-job-list.sh: OK
  • CREAM-cli-submission-management.sh: OK
  • CREAM-cli-event-query.sh: OK
  • CREAM-cli-service-info.sh: OK

Checked bugs

Bug #66720 glite-ce-job-output command should be provided FIXED

[ale@cream-03 CREAM-cli]$ cat sbd.jdl
[
JobType = "Normal";
Executable = "/bin/hostname";
outputsandbox={"out.txt", "err.txt"};
outputsandboxbasedesturi="gsiftp://localhost";
StdOutput="out.txt";
StdError="err.txt";
]
[ale@cream-03 CREAM-cli]$ glite-ce-job-submit -a -r cream-32.pd.infn.it:8443/cream-pbs-creamtest1 sbd.jdl
https://cream-32.pd.infn.it:8443/CREAM794123402
[ale@cream-03 CREAM-cli]$ glite-ce-job-status https://cream-32.pd.infn.it:8443/CREAM794123402

******  JobID=[https://cream-32.pd.infn.it:8443/CREAM794123402]
   Status        = [DONE-OK]
   ExitCode      = [0]


[ale@cream-03 CREAM-cli]$ glite-ce-job-output https://cream-32.pd.infn.it:8443/CREAM794123402

2011-02-08 17:42:00,521 INFO - For JobID [https://cream-32.pd.infn.it:8443/CREAM794123402] output will be stored in the dir [./cream-32.pd.infn.it_8443_CREAM794123402
[ale@cream-03 CREAM-cli]$ ls -l ./cream-32.pd.infn.it_8443_CREAM794123402
total 4
-rw-r--r-- 1 ale ale  0 Feb  8 17:42 err.txt
-rw-r--r-- 1 ale ale 27 Feb  8 17:42 out.txt
[ale@cream-03 CREAM-cli]$ cat ./cream-32.pd.infn.it_8443_CREAM794123402/out.txt 
cream-wn-055.pn.pd.infn.it
[ale@cream-03 CREAM-cli]$ glite-ce-job-output -D outdir https://cream-32.pd.infn.it:8443/CREAM794123402

2011-02-08 17:42:40,953 INFO - For JobID [https://cream-32.pd.infn.it:8443/CREAM794123402] output will be stored in the dir [outdir/cream-32.pd.infn.it_8443_CREAM794123402
[ale@cream-03 CREAM-cli]$ ls -l outdir/cream-32.pd.infn.it_8443_CREAM794123402
total 4
-rw-r--r-- 1 ale ale  0 Feb  8 17:42 err.txt
-rw-r--r-- 1 ale ale 27 Feb  8 17:42 out.txt
[ale@cream-03 CREAM-cli]$ cat outdir/cream-32.pd.infn.it_8443_CREAM794123402/out.txt 
cream-wn-055.pn.pd.infn.it
[ale@cream-03 CREAM-cli]$ glite-ce-job-status -L 2 https://cream-32.pd.infn.it:8443/CREAM794123402 | grep "Worker Node"
   Worker Node    = [cream-wn-055.pn.pd.infn.it]

Bug #71140 Annoying warning message on CREAM-CLI should be printed only in debug mode FIXED

[ale@cream-03 CREAM-cli]$ glite-ce-job-submit -a -r cream-32.pd.infn.it:8443/cream-pbs-creamtest1 sbd.jdl
https://cream-32.pd.infn.it:8443/CREAM989268052
[ale@cream-03 CREAM-cli]$ glite-ce-job-status https://cream-32.pd.infn.it:8443/CREAM989268052

******  JobID=[https://cream-32.pd.infn.it:8443/CREAM989268052]
   Status        = [RUNNING]


[ale@cream-03 CREAM-cli]$ glite-ce-job-status -d https://cream-32.pd.infn.it:8443/CREAM989268052
2011-02-08 17:46:45,790 INFO - Using certificate proxy file [/tmp/x509up_u500]
2011-02-08 17:46:45,809 WARN - No configuration file suitable for loading. Using built-in configuration
2011-02-08 17:46:45,809 INFO - Logfile is [/tmp/glite_cream_cli_logs/glite-ce-job-status_CREAM_ale_20110208-174645.log]
2011-02-08 17:46:45,810 DEBUG - Processing job [https://cream-32.pd.infn.it:8443/CREAM989268052]...
2011-02-08 17:46:45,810 INFO - Sending JobStatus request to [https://cream-32.pd.infn.it:8443//ce-cream/services/CREAM2]
2011-02-08 17:46:45,810 DEBUG - Contacting service [https://cream-32.pd.infn.it:8443//ce-cream/services/CREAM2]

******  JobID=[https://cream-32.pd.infn.it:8443/CREAM989268052]
   Status        = [RUNNING]

Bug #72084 Reduce the number of authorization calls in transfers of sandbox files between UI and CREAM FIXED

$ cat argus.jdl
[
executable="ssh1.sh";
inputsandbox={"file:///home/sgaravat/JDLExamples/ssh1.sh", "file:///home/sgaravat/a"};
stdoutput="out3.out";
stderror="err2.err";
outputsandbox={"out3.out", "err2.err", "ssh1.sh", "a"};
outputsandboxbasedesturi="gsiftp://localhost";
] 
$ glite-ce-delegate-proxy -e cream-38 poi
2011-02-09 10:38:46,310 NOTICE - Proxy with delegation id [poi] succesfully delegated to endpoint [https://cream-38:8443//ce-cream/services/gridsite-delegation]
-bash-3.2$ glite-ce-job-submit -D poi -r cream-38.pd.infn.it:8443/cream-pbs-creamtest1 argus.jdl
https://cream-38.pd.infn.it:8443/CREAM128523326 

1297244442473|193.206.210.16|cream-43.pd.infn.it:8154|/authz|
1297244443874|193.206.210.16|cream-43.pd.infn.it:8154|/authz|
1297244444098|193.206.210.16|cream-43.pd.infn.it:8154|/authz| 

$ glite-ce-job-output https://cream-38.pd.infn.it:8443/CREAM128523326

2011-02-09 10:42:51,924 INFO - For JobID [https://cream-38.pd.infn.it:8443/CREAM128523326] output will be stored in the dir [./cream-38.pd.infn.it_8443_CREAM128523326

1297244571865|193.206.210.16|cream-43.pd.infn.it:8154|/authz|
1297244572128|193.206.210.16|cream-43.pd.infn.it:8154|/authz| 

Bug #73746 glite-ce-job-status may abort unnecessarily FIXED

[ale@cream-03 UI]$ egrep -c '^https://.*:8443/CREAM[0-9]+$' test.id
11
[ale@cream-03 UI]$ glite-ce-job-status -i test.id | grep -c 'JobID=' 
2011-02-09 10:54:09,458 ERROR - An error occurred with CREAM URL [https://cream-21.pd.infn.it:8443//ce-cream/services/CREAM2]: Received NULL fault; the error is due to another cause: FaultString=[connection error] - FaultCode=[SOAP-ENV:Client] - FaultSubCode=[SOAP-ENV:Client] - FaultDetail=[Connection refused]
10

Bug #73900 there's a memory leak in cream-client-api-c FIXED not certified

Bug #74699 An ISB file specified as "file:///" is not sent to the CE FIXED

[ale@cream-03 UI]$ cat example.jdl 
[
JobType = "Normal";
Executable = "/bin/ls";
Arguments="-la";
inputsandbox={ "file:///home/ale/UI/example.jdl"};
inputsandboxbaseuri="gsiftp://cream-38.pd.infn.it/etc/";
outputsandboxbasedesturi="gsiftp://localhost"; 
outputsandbox={"out.txt", "example.jdl"};
StdOutput="out.txt";
StdError="err.txt";
]
[ale@cream-03 UI]$ glite-ce-job-submit -a -o test.id -r cream-26.pd.infn.it:8443/cream-lsf-testbedB_1 example.jdl 
https://cream-26.pd.infn.it:8443/CREAM748972290
[ale@cream-03 UI]$ glite-ce-job-status https://cream-26.pd.infn.it:8443/CREAM748972290

******  JobID=[https://cream-26.pd.infn.it:8443/CREAM748972290]
   Status        = [DONE-OK]
   ExitCode      = [0]


[ale@cream-03 UI]$ glite-ce-job-output https://cream-26.pd.infn.it:8443/CREAM748972290

2011-02-09 11:16:33,909 INFO - For JobID [https://cream-26.pd.infn.it:8443/CREAM748972290] output will be stored in the dir [./cream-26.pd.infn.it_8443_CREAM748972290
[ale@cream-03 UI]$ ls -l ./cream-26.pd.infn.it_8443_CREAM748972290/
total 8
-rw-r--r-- 1 ale ale 304 Feb  9 11:16 example.jdl
-rw-r--r-- 1 ale ale 334 Feb  9 11:16 out.txt
[ale@cream-03 UI]$ cat ./cream-26.pd.infn.it_8443_CREAM748972290/out.txt 
total 12
drwxr-xr-x  2 dteam039 dteam 4096 Feb  9 11:16 .
drwxr-xr-x  3 dteam039 dteam 4096 Feb  9 11:16 ..
-rw-r--r--  1 dteam039 dteam    0 Feb  9 11:16 err.txt
-rw-r--r--  1 dteam039 dteam  304 Feb  9 11:16 example.jdl
-rw-r--r--  1 dteam039 dteam    0 Feb  9 11:16 out.txt
-rw-r--r--  1 dteam039 dteam    0 Feb  9 11:16 .tmp_file

Bug #76899 CREAM CLI: Wrong handling of Inputsandbox FIXED

[ale@cream-03 UI]$ cat example.jdl 
[
JobType = "Normal";
Executable = "/bin/ls";
Arguments="-la";
inputsandbox={ "file:///home/ale/UI/example.jdl"};
outputsandboxbasedesturi="gsiftp://localhost"; 
outputsandbox={"out.txt", "example.jdl"};
StdOutput="out.txt";
StdError="err.txt";
]
[ale@cream-03 UI]$ glite-ce-job-submit -a -o test.id -d -r cream-26.pd.infn.it:8443/cream-lsf-testbedB_1 example.jdl 
2011-02-09 11:32:54,976 INFO - *************************************
2011-02-09 11:32:54,976 INFO - CREAM User Interface version 1.2.0 - Starting at Wed Feb  9 11:32:54 2011

2011-02-09 11:32:54,977 DEBUG - Using certificate proxy file [/tmp/x509up_u500]
2011-02-09 11:32:54,994 INFO - VO from certificate=[dteam]
2011-02-09 11:32:54,995 WARN - No configuration file suitable for loading. Using built-in configuration
2011-02-09 11:32:55,109 INFO - Logfile is [/tmp/glite_cream_cli_logs/glite-ce-job-submit_CREAM_ale_20110209-113254.log]
2011-02-09 11:32:55,109 DEBUG - Processing file [file:///home/ale/UI/example.jdl]...
2011-02-09 11:32:55,110 DEBUG - Adding absolute path [/home/ale/UI/example.jdl/file:///home/ale/UI/example.jdl]...
2011-02-09 11:32:55,110 DEBUG - Inserting mangled InputSandbox in JDL: [{"/home/ale/UI/example.jdl"}]...
2011-02-09 11:32:55,113 INFO - Registering to [http://cream-26.pd.infn.it:8443/ce-cream/services/CREAM2] JDL=[ Arguments = "-la"; QueueName = "testbedB_1"; JobType = "Normal"; Executable = "/bin/ls"; VirtualOrganisation = "dteam"; InputSandbox = { "/home/ale/UI/example.jdl" }; StdOutput = "out.txt"; outputsandboxbasedesturi = "gsiftp://localhost"; StdError = "err.txt"; BatchSystem = "lsf"; outputsandbox = { "out.txt","example.jdl" } ]
2011-02-09 11:32:55,113 INFO - certUtil::generateUniqueID() - Generated DelegationID: [24719b560b25eed49b2c03421d872c99aa8edbd7]
2011-02-09 11:32:55,113 INFO - CreamProxy_Delegate::execute() - Calling getProxyReq to remote service [https://cream-26.pd.infn.it:8443/ce-cream/services/gridsite-delegation]
2011-02-09 11:32:57,444 INFO - CreamProxy_Delegate::execute() - Signing returned proxy with our proxy certificate [/tmp/x509up_u500]...
2011-02-09 11:32:57,463 INFO - CreamProxy_Delegate::execute() - Sending signed proxy to [https://cream-26.pd.infn.it:8443/ce-cream/services/gridsite-delegation] with id [24719b560b25eed49b2c03421d872c99aa8edbd7]
2011-02-09 11:32:58,534 INFO - JobID=[https://cream-26.pd.infn.it:8443/CREAM120874597]
2011-02-09 11:32:58,534 INFO - UploadURL=[gsiftp://cream-26.pd.infn.it/opt/glite/var/cream_sandbox/dteam/_C_IT_O_INFN_OU_Personal_Certificate_L_Padova_CN_Alessio_Gianelle_dteam_Role_NULL_Capability_NULL_dteam039/12/CREAM120874597/ISB]
2011-02-09 11:32:58,535 INFO - Sending file [gsiftp://cream-26.pd.infn.it/opt/glite/var/cream_sandbox/dteam/_C_IT_O_INFN_OU_Personal_Certificate_L_Padova_CN_Alessio_Gianelle_dteam_Role_NULL_Capability_NULL_dteam039/12/CREAM120874597/ISB/example.jdl]
2011-02-09 11:32:59,223 INFO - Now invoking JobStart for JobID [https://cream-26.pd.infn.it:8443/CREAM120874597]
https://cream-26.pd.infn.it:8443/CREAM120874597

Bug #77368 Problems detecting job status changes in WMS->CREAM with secondary FQANs FIXED

[ale@cream-03 proxy]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto
type      : proxy
strength  : 1024 bits
path      : /home/ale/UI/proxy/user1.proxy
timeleft  : 3:58:50
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/italy/Role=NULL/Capability=NULL
attribute : /dteam/italy/INFN-PADOVA/Role=NULL/Capability=NULL
timeleft  : 3:58:50
uri       : voms2.hellasgrid.gr:15004

[ale@cream-03 proxy]$ glite-ce-job-submit -a -o test.id -r cream-26.pd.infn.it:8443/cream-lsf-testbedB_1 ../example.jdl 
https://cream-26.pd.infn.it:8443/CREAM710892363

[ale@cream-03 proxy]$ voms-proxy-init -voms dteam -voms dteam:/dteam/italy/INFN-PADOVA -cert usercert_1.pem -key userkey_1.pem
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto
Creating temporary proxy ............................................... Done
Contacting  voms.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr] "dteam" Done
Creating proxy ........................................................................................................... Done
Your proxy is valid until Wed Feb  9 23:48:51 2011
[ale@cream-03 proxy]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto
type      : proxy
strength  : 1024 bits
path      : /home/ale/UI/proxy/user1.proxy
timeleft  : 11:59:55
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/italy/INFN-PADOVA/Role=NULL/Capability=NULL
attribute : /dteam/italy/Role=NULL/Capability=NULL
timeleft  : 11:59:54
uri       : voms.hellasgrid.gr:15004

[ale@cream-03 proxy]$ glite-ce-job-status https://cream-26.pd.infn.it:8443/CREAM710892363

******  JobID=[https://cream-26.pd.infn.it:8443/CREAM710892363]
   Status        = [DONE-OK]
   ExitCode      = [0]

[ale@cream-03 proxy]$ glite-ce-job-output https://cream-26.pd.infn.it:8443/CREAM710892363

2011-02-09 11:52:30,390 INFO - For JobID [https://cream-26.pd.infn.it:8443/CREAM710892363] output will be stored in the dir [./cream-26.pd.infn.it_8443_CREAM710892363
[ale@cream-03 proxy]$ ls -l ./cream-26.pd.infn.it_8443_CREAM710892363
total 8
-rw-r--r-- 1 ale ale 247 Feb  9 11:52 example.jdl
-rw-r--r-- 1 ale ale 334 Feb  9 11:52 out.txt

-- AlessioGianelle - 2011-02-08