CreamCliP4366 < EgeeJra1It

Tags: , view all tags
%TOC%

---+ *Certification report for the internal patch [[https://savannah.cern.ch/patch/index.php?4366][4366]]*

Author(s): Alessio Gianelle

Outcome: %GREEN% *CERTIFIED* %ENDCOLOR%


---++ Information
| Patch | https://savannah.cern.ch/patch/index.php?4366 |
| Build reports | http://etics-repository.cern.ch/repository/reports/id/37113781-c2b7-49f1-99b5-22eb63cc918a/sl5_x86_64_gcc412/-/reports/index.html |
| Repository | http://etics-repository.cern.ch/repository/pm/registered/repomd/name/patch_4388_1/index.html |


---++ Upgrade from production
   * Upgrade steps: <verbatim>
wget http://etics-repository.cern.ch/repository/pm/registered/repomd/name/patch_4366_1/etics-registered-build-by-name.repo -O /etc/yum.repos.d/glite-CREAMCLI.repo
yum update
</verbatim>

   * view the [[%ATTACHURL%/yumlog.txt][log]] of YUM for the client upgrade


---++ Automatic tests
   * Run the following tests from the [[https://twiki.cnaf.infn.it/cgi-bin/twiki/view/EgeeJra1It/CreamCli][testsuite]]: output for [[%ATTACHURL%/test_CREAM_LSF.out][lsf]] and [[%ATTACHURL%/test_CREAM_PBS.out][pbs]] CE.
      * CREAM-cli-delegation.sh: %GREEN% OK%ENDCOLOR%
      * CREAM-cli-delegation-renew.sh: %GREEN% OK%ENDCOLOR%
      * CREAM-cli-job-submit: %GREEN% OK%ENDCOLOR%
      * CREAM-cli-job-status-simple.sh: %GREEN% OK%ENDCOLOR%
      * CREAM-cli-job-status-filtered.sh: %GREEN% OK%ENDCOLOR%
      * CREAM-cli-job-cancel.sh: %GREEN% OK%ENDCOLOR%
      * CREAM-cli-job-suspend.sh: %GREEN% OK%ENDCOLOR% 
      * CREAM-cli-job-resume.sh: %GREEN% OK%ENDCOLOR% 
      * CREAM-cli-job-purge.sh: %GREEN% OK%ENDCOLOR%
      * CREAM-cli-job-list.sh: %GREEN% OK%ENDCOLOR%
      * CREAM-cli-submission-management.sh: %GREEN% OK%ENDCOLOR%
      * CREAM-cli-event-query.sh: %GREEN% OK%ENDCOLOR%
      * CREAM-cli-service-info.sh: %GREEN% OK%ENDCOLOR%

---++ Checked bugs
---+++ Bug [[https://savannah.cern.ch/bugs/?66720][#66720]] glite-ce-job-output command should be provided %GREEN% *FIXED* %ENDCOLOR%
<verbatim>
[ale@cream-03 CREAM-cli]$ cat sbd.jdl
[
JobType = "Normal";
Executable = "/bin/hostname";
outputsandbox={"out.txt", "err.txt"};
outputsandboxbasedesturi="gsiftp://localhost";
StdOutput="out.txt";
StdError="err.txt";
]
[ale@cream-03 CREAM-cli]$ glite-ce-job-submit -a -r cream-32.pd.infn.it:8443/cream-pbs-creamtest1 sbd.jdl
https://cream-32.pd.infn.it:8443/CREAM794123402
[ale@cream-03 CREAM-cli]$ glite-ce-job-status https://cream-32.pd.infn.it:8443/CREAM794123402

******  JobID=[https://cream-32.pd.infn.it:8443/CREAM794123402]
	Status        = [DONE-OK]
	ExitCode      = [0]


[ale@cream-03 CREAM-cli]$ glite-ce-job-output https://cream-32.pd.infn.it:8443/CREAM794123402

2011-02-08 17:42:00,521 INFO - For JobID [https://cream-32.pd.infn.it:8443/CREAM794123402] output will be stored in the dir [./cream-32.pd.infn.it_8443_CREAM794123402
[ale@cream-03 CREAM-cli]$ ls -l ./cream-32.pd.infn.it_8443_CREAM794123402
total 4
-rw-r--r-- 1 ale ale  0 Feb  8 17:42 err.txt
-rw-r--r-- 1 ale ale 27 Feb  8 17:42 out.txt
[ale@cream-03 CREAM-cli]$ cat ./cream-32.pd.infn.it_8443_CREAM794123402/out.txt 
cream-wn-055.pn.pd.infn.it
[ale@cream-03 CREAM-cli]$ glite-ce-job-output -D outdir https://cream-32.pd.infn.it:8443/CREAM794123402

2011-02-08 17:42:40,953 INFO - For JobID [https://cream-32.pd.infn.it:8443/CREAM794123402] output will be stored in the dir [outdir/cream-32.pd.infn.it_8443_CREAM794123402
[ale@cream-03 CREAM-cli]$ ls -l outdir/cream-32.pd.infn.it_8443_CREAM794123402
total 4
-rw-r--r-- 1 ale ale  0 Feb  8 17:42 err.txt
-rw-r--r-- 1 ale ale 27 Feb  8 17:42 out.txt
[ale@cream-03 CREAM-cli]$ cat outdir/cream-32.pd.infn.it_8443_CREAM794123402/out.txt 
cream-wn-055.pn.pd.infn.it
[ale@cream-03 CREAM-cli]$ glite-ce-job-status -L 2 https://cream-32.pd.infn.it:8443/CREAM794123402 | grep "Worker Node"
	Worker Node    = [cream-wn-055.pn.pd.infn.it]
</verbatim>

---+++ Bug [[https://savannah.cern.ch/bugs/?71140][#71140]] Annoying warning message on CREAM-CLI should be printed only in debug mode %GREEN% *FIXED* %ENDCOLOR%
<verbatim>
[ale@cream-03 CREAM-cli]$ glite-ce-job-submit -a -r cream-32.pd.infn.it:8443/cream-pbs-creamtest1 sbd.jdl
https://cream-32.pd.infn.it:8443/CREAM989268052
[ale@cream-03 CREAM-cli]$ glite-ce-job-status https://cream-32.pd.infn.it:8443/CREAM989268052

******  JobID=[https://cream-32.pd.infn.it:8443/CREAM989268052]
	Status        = [RUNNING]


[ale@cream-03 CREAM-cli]$ glite-ce-job-status -d https://cream-32.pd.infn.it:8443/CREAM989268052
2011-02-08 17:46:45,790 INFO - Using certificate proxy file [/tmp/x509up_u500]
2011-02-08 17:46:45,809 WARN - No configuration file suitable for loading. Using built-in configuration
2011-02-08 17:46:45,809 INFO - Logfile is [/tmp/glite_cream_cli_logs/glite-ce-job-status_CREAM_ale_20110208-174645.log]
2011-02-08 17:46:45,810 DEBUG - Processing job [https://cream-32.pd.infn.it:8443/CREAM989268052]...
2011-02-08 17:46:45,810 INFO - Sending JobStatus request to [https://cream-32.pd.infn.it:8443//ce-cream/services/CREAM2]
2011-02-08 17:46:45,810 DEBUG - Contacting service [https://cream-32.pd.infn.it:8443//ce-cream/services/CREAM2]

******  JobID=[https://cream-32.pd.infn.it:8443/CREAM989268052]
	Status        = [RUNNING]
</verbatim>
---+++ Bug [[https://savannah.cern.ch/bugs/?72084][#72084]] Reduce the number of authorization calls in transfers of sandbox files between UI and CREAM %GREEN% *FIXED* %ENDCOLOR%
<verbatim>
$ cat argus.jdl
[
executable="ssh1.sh";
inputsandbox={"file:///home/sgaravat/JDLExamples/ssh1.sh", "file:///home/sgaravat/a"};
stdoutput="out3.out";
stderror="err2.err";
outputsandbox={"out3.out", "err2.err", "ssh1.sh", "a"};
outputsandboxbasedesturi="gsiftp://localhost";
] 
$ glite-ce-delegate-proxy -e cream-38 poi
2011-02-09 10:38:46,310 NOTICE - Proxy with delegation id [poi] succesfully delegated to endpoint [https://cream-38:8443//ce-cream/services/gridsite-delegation]
-bash-3.2$ glite-ce-job-submit -D poi -r cream-38.pd.infn.it:8443/cream-pbs-creamtest1 argus.jdl
https://cream-38.pd.infn.it:8443/CREAM128523326 </verbatim>
Looking into argus log:<verbatim>
1297244442473|193.206.210.16|cream-43.pd.infn.it:8154|/authz|
1297244443874|193.206.210.16|cream-43.pd.infn.it:8154|/authz|
1297244444098|193.206.210.16|cream-43.pd.infn.it:8154|/authz| </verbatim>
When the job arrives to final status (Done OK)<verbatim>
$ glite-ce-job-output https://cream-38.pd.infn.it:8443/CREAM128523326

2011-02-09 10:42:51,924 INFO - For JobID [https://cream-38.pd.infn.it:8443/CREAM128523326] output will be stored in the dir [./cream-38.pd.infn.it_8443_CREAM128523326
</verbatim>
Again fromm argus log file:<verbatim>
1297244571865|193.206.210.16|cream-43.pd.infn.it:8154|/authz|
1297244572128|193.206.210.16|cream-43.pd.infn.it:8154|/authz| </verbatim>

---+++ Bug [[https://savannah.cern.ch/bugs/?73476][#73746]] glite-ce-job-status may abort unnecessarily %GREEN% *FIXED* %ENDCOLOR%
<verbatim>
[ale@cream-03 UI]$ egrep -c '^https://.*:8443/CREAM[0-9]+$' test.id
11
[ale@cream-03 UI]$ glite-ce-job-status -i test.id | grep -c 'JobID=' 
2011-02-09 10:54:09,458 ERROR - An error occurred with CREAM URL [https://cream-21.pd.infn.it:8443//ce-cream/services/CREAM2]: Received NULL fault; the error is due to another cause: FaultString=[connection error] - FaultCode=[SOAP-ENV:Client] - FaultSubCode=[SOAP-ENV:Client] - FaultDetail=[Connection refused]
10
</verbatim>


---+++ Bug [[https://savannah.cern.ch/bugs/?73900][#73900]] there's a memory leak in cream-client-api-c %GREEN% *FIXED not certified* %ENDCOLOR%


---+++ Bug [[https://savannah.cern.ch/bugs/?74699][#74699]] An ISB file specified as "file://<path>/<filename>" is not sent to the CE %GREEN% *FIXED* %ENDCOLOR%
<verbatim>
[ale@cream-03 UI]$ cat example.jdl 
[
JobType = "Normal";
Executable = "/bin/ls";
Arguments="-la";
inputsandbox={ "file:///home/ale/UI/example.jdl"};
inputsandboxbaseuri="gsiftp://cream-38.pd.infn.it/etc/";
outputsandboxbasedesturi="gsiftp://localhost"; 
outputsandbox={"out.txt", "example.jdl"};
StdOutput="out.txt";
StdError="err.txt";
]
[ale@cream-03 UI]$ glite-ce-job-submit -a -o test.id -r cream-26.pd.infn.it:8443/cream-lsf-testbedB_1 example.jdl 
https://cream-26.pd.infn.it:8443/CREAM748972290
[ale@cream-03 UI]$ glite-ce-job-status https://cream-26.pd.infn.it:8443/CREAM748972290

******  JobID=[https://cream-26.pd.infn.it:8443/CREAM748972290]
	Status        = [DONE-OK]
	ExitCode      = [0]


[ale@cream-03 UI]$ glite-ce-job-output https://cream-26.pd.infn.it:8443/CREAM748972290

2011-02-09 11:16:33,909 INFO - For JobID [https://cream-26.pd.infn.it:8443/CREAM748972290] output will be stored in the dir [./cream-26.pd.infn.it_8443_CREAM748972290
[ale@cream-03 UI]$ ls -l ./cream-26.pd.infn.it_8443_CREAM748972290/
total 8
-rw-r--r-- 1 ale ale 304 Feb  9 11:16 example.jdl
-rw-r--r-- 1 ale ale 334 Feb  9 11:16 out.txt
[ale@cream-03 UI]$ cat ./cream-26.pd.infn.it_8443_CREAM748972290/out.txt 
total 12
drwxr-xr-x  2 dteam039 dteam 4096 Feb  9 11:16 .
drwxr-xr-x  3 dteam039 dteam 4096 Feb  9 11:16 ..
-rw-r--r--  1 dteam039 dteam    0 Feb  9 11:16 err.txt
-rw-r--r--  1 dteam039 dteam  304 Feb  9 11:16 example.jdl
-rw-r--r--  1 dteam039 dteam    0 Feb  9 11:16 out.txt
-rw-r--r--  1 dteam039 dteam    0 Feb  9 11:16 .tmp_file
</verbatim>

---+++ Bug [[https://savannah.cern.ch/bugs/?76899][#76899]] CREAM CLI: Wrong handling of Inputsandbox %GREEN% *FIXED* %ENDCOLOR%
<verbatim>[ale@cream-03 UI]$ cat example.jdl 
[
JobType = "Normal";
Executable = "/bin/ls";
Arguments="-la";
inputsandbox={ "file:///home/ale/UI/example.jdl"};
outputsandboxbasedesturi="gsiftp://localhost"; 
outputsandbox={"out.txt", "example.jdl"};
StdOutput="out.txt";
StdError="err.txt";
]
[ale@cream-03 UI]$ glite-ce-job-submit -a -o test.id -d -r cream-26.pd.infn.it:8443/cream-lsf-testbedB_1 example.jdl 
2011-02-09 11:32:54,976 INFO - *************************************
2011-02-09 11:32:54,976 INFO - CREAM User Interface version 1.2.0 - Starting at Wed Feb  9 11:32:54 2011

2011-02-09 11:32:54,977 DEBUG - Using certificate proxy file [/tmp/x509up_u500]
2011-02-09 11:32:54,994 INFO - VO from certificate=[dteam]
2011-02-09 11:32:54,995 WARN - No configuration file suitable for loading. Using built-in configuration
2011-02-09 11:32:55,109 INFO - Logfile is [/tmp/glite_cream_cli_logs/glite-ce-job-submit_CREAM_ale_20110209-113254.log]
2011-02-09 11:32:55,109 DEBUG - Processing file [file:///home/ale/UI/example.jdl]...
2011-02-09 11:32:55,110 DEBUG - Adding absolute path [/home/ale/UI/example.jdl/file:///home/ale/UI/example.jdl]...
2011-02-09 11:32:55,110 DEBUG - Inserting mangled InputSandbox in JDL: [{"/home/ale/UI/example.jdl"}]...
2011-02-09 11:32:55,113 INFO - Registering to [http://cream-26.pd.infn.it:8443/ce-cream/services/CREAM2] JDL=[ Arguments = "-la"; QueueName = "testbedB_1"; JobType = "Normal"; Executable = "/bin/ls"; VirtualOrganisation = "dteam"; InputSandbox = { "/home/ale/UI/example.jdl" }; StdOutput = "out.txt"; outputsandboxbasedesturi = "gsiftp://localhost"; StdError = "err.txt"; BatchSystem = "lsf"; outputsandbox = { "out.txt","example.jdl" } ]
2011-02-09 11:32:55,113 INFO - certUtil::generateUniqueID() - Generated DelegationID: [24719b560b25eed49b2c03421d872c99aa8edbd7]
2011-02-09 11:32:55,113 INFO - CreamProxy_Delegate::execute() - Calling getProxyReq to remote service [https://cream-26.pd.infn.it:8443/ce-cream/services/gridsite-delegation]
2011-02-09 11:32:57,444 INFO - CreamProxy_Delegate::execute() - Signing returned proxy with our proxy certificate [/tmp/x509up_u500]...
2011-02-09 11:32:57,463 INFO - CreamProxy_Delegate::execute() - Sending signed proxy to [https://cream-26.pd.infn.it:8443/ce-cream/services/gridsite-delegation] with id [24719b560b25eed49b2c03421d872c99aa8edbd7]
2011-02-09 11:32:58,534 INFO - JobID=[https://cream-26.pd.infn.it:8443/CREAM120874597]
2011-02-09 11:32:58,534 INFO - UploadURL=[gsiftp://cream-26.pd.infn.it/opt/glite/var/cream_sandbox/dteam/_C_IT_O_INFN_OU_Personal_Certificate_L_Padova_CN_Alessio_Gianelle_dteam_Role_NULL_Capability_NULL_dteam039/12/CREAM120874597/ISB]
2011-02-09 11:32:58,535 INFO - Sending file [gsiftp://cream-26.pd.infn.it/opt/glite/var/cream_sandbox/dteam/_C_IT_O_INFN_OU_Personal_Certificate_L_Padova_CN_Alessio_Gianelle_dteam_Role_NULL_Capability_NULL_dteam039/12/CREAM120874597/ISB/example.jdl]
2011-02-09 11:32:59,223 INFO - Now invoking JobStart for JobID [https://cream-26.pd.infn.it:8443/CREAM120874597]
https://cream-26.pd.infn.it:8443/CREAM120874597
</verbatim>

---+++ Bug [[https://savannah.cern.ch/bugs/?77368][#77368]]  Problems detecting job status changes in WMS->CREAM with secondary FQANs %GREEN% *FIXED* %ENDCOLOR%
<verbatim>
[ale@cream-03 proxy]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto
type      : proxy
strength  : 1024 bits
path      : /home/ale/UI/proxy/user1.proxy
timeleft  : 3:58:50
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/italy/Role=NULL/Capability=NULL
attribute : /dteam/italy/INFN-PADOVA/Role=NULL/Capability=NULL
timeleft  : 3:58:50
uri       : voms2.hellasgrid.gr:15004

[ale@cream-03 proxy]$ glite-ce-job-submit -a -o test.id -r cream-26.pd.infn.it:8443/cream-lsf-testbedB_1 ../example.jdl 
https://cream-26.pd.infn.it:8443/CREAM710892363

[ale@cream-03 proxy]$ voms-proxy-init -voms dteam -voms dteam:/dteam/italy/INFN-PADOVA -cert usercert_1.pem -key userkey_1.pem
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto
Creating temporary proxy ............................................... Done
Contacting  voms.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr] "dteam" Done
Creating proxy ........................................................................................................... Done
Your proxy is valid until Wed Feb  9 23:48:51 2011
[ale@cream-03 proxy]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto
type      : proxy
strength  : 1024 bits
path      : /home/ale/UI/proxy/user1.proxy
timeleft  : 11:59:55
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/italy/INFN-PADOVA/Role=NULL/Capability=NULL
attribute : /dteam/italy/Role=NULL/Capability=NULL
timeleft  : 11:59:54
uri       : voms.hellasgrid.gr:15004

[ale@cream-03 proxy]$ glite-ce-job-status https://cream-26.pd.infn.it:8443/CREAM710892363

******  JobID=[https://cream-26.pd.infn.it:8443/CREAM710892363]
	Status        = [DONE-OK]
	ExitCode      = [0]

[ale@cream-03 proxy]$ glite-ce-job-output https://cream-26.pd.infn.it:8443/CREAM710892363

2011-02-09 11:52:30,390 INFO - For JobID [https://cream-26.pd.infn.it:8443/CREAM710892363] output will be stored in the dir [./cream-26.pd.infn.it_8443_CREAM710892363
[ale@cream-03 proxy]$ ls -l ./cream-26.pd.infn.it_8443_CREAM710892363
total 8
-rw-r--r-- 1 ale ale 247 Feb  9 11:52 example.jdl
-rw-r--r-- 1 ale ale 334 Feb  9 11:52 out.txt
</verbatim>





-- Main.AlessioGianelle - 2011-02-08
Attachments
Topic attachments
I	Attachment	Action	Size	Date	Who	Comment
out	test_CREAM_LSF.out	manage	135.0 K	2011-02-08 - 14:37	AlessioGianelle	LSF test suite output
out	test_CREAM_PBS.out	manage	158.9 K	2011-02-08 - 14:37	AlessioGianelle	PBS test suite output
txt	yumlog.txt	manage	10.6 K	2011-02-08 - 09:43	AlessioGianelle	Yum update log file
Topic revision: r5 - 2011-02-09 - AlessioGianelle
Account
- Log In
Edit
Attach