(r21) CreamTestsP4415 < EgeeJra1It

Tags: , view all tags
%TOC%

---+ *[[https://savannah.cern.ch/patch/index.php?4415][PATCH 4415]]*

---+++ Automatic tests

   * report #1:
      * CREAM UI version: 1.12.1; CREAM testsuite version: 1.0.7
      * Used direct polling for monitoring and BUpdater/BNotifier for status change detection
      * Batch system: TORQUE
      * All tests have been run  (see the [[%ATTACHURL%/reports_patch_4415.tar.gz][report]] from the testsuite) with the following results:
         * test #18 reports one submission failed due to network problems and one execution terminated by the batch system
         * all other tests have been correctly executed


Tested that the BDII is operational

<verbatim>
ldapsearch -x -h cream-40.pd.infn.it -p 2170 -b o=infosys
</verbatim>

Output is [[%ATTACHURL%/infotest1.txt][here]]

Checked the Glue 1.3 root entry:

<verbatim>
ldapsearch -x -h cream-40.pd.infn.it -p 2170 -b mds-vo-name=resource,o=grid
</verbatim>

Output is [[%ATTACHURL%/infotest2.txt][here]]

Checked the Glue 2.0 root entry:

<verbatim>
ldapsearch -x -h cream-40.pd.infn.it -p 2170 -b o=glue
</verbatim>

Output is [[%ATTACHURL%/infotest3.txt][here]]


Information content check:

<verbatim>
$ gstat-validate-ce -H cream-40.pd.infn.it -p 2170 -b o=grid
OK - errors 0, warnings 0, info 0
</verbatim>

---+++ Checked bugs

   * Bug [[https://savannah.cern.ch/bugs/?54755][#54755]]: Could YAIM warn if a site tries to enable whole-VO FQANs? %GREEN%FIXED%ENDCOLOR%
      * Define a VO with the character "/" as a prefix in the variables [queue]_GROUP_ENABLE of the site-info.def
      * Run the YAIM configurator  and verify that in the output the following warning appears: "Queue [vo name] was enabled for one or more FQANs which specify an entire VO"

   * Bug [[https://savannah.cern.ch/bugs/?61396][#61396]]: CREAM and CEMon log4j.properties should take into account the value of GLITE_LOCATION_LOG %GREEN%FIXED%ENDCOLOR%
      * Specify the variable GLITE_LOCATION_LOG in the YAIM configuration (file services-glite-creamce), even with an nonexistent path, and reconfigure with YAIM.
      * Verify that the new path is correctly created and the log files for CREAM and CEMon are written in the new location

   * Bug [[https://savannah.cern.ch/bugs/?62078][#62078]]: Missing glite-pbsparser.log file on standalone Torque-Server %GREEN%FIXED%ENDCOLOR%
      * Act as for bug #61396 and verify that the log file glite-pbsparser.log is correctly written inside the new location

   * Bug [[https://savannah.cern.ch/bugs/?63007][#63007]]: CREAM sandbox dir should be relocatable %GREEN%FIXED%ENDCOLOR%
      * Define the variable CREAM_SANDBOX_PATH in the file services/glite-creamce
      * Reconfigure with YAIM, submit a job and verify that the sandbox of that job is placed into the new sandbox path

   * Bug [[https://savannah.cern.ch/bugs/?63956][#63956]]: config_cream_gip may use wrong VO %GREEN%FIXED%ENDCOLOR%
      * patch verified by the submitter

   * Bug [[https://savannah.cern.ch/bugs/?67238][#67238]]: !BUpdaterLSF problem with bhist failure %RED%TODO%ENDCOLOR%
      * no comment

   * Bug [[https://savannah.cern.ch/bugs/?68225][#68225]]: In some cases the sudoers file created by yaim-cream-ce can be syntactically wrong %GREEN%FIXED%ENDCOLOR%
      * Consider a VO whose name contains a '-' character
      * Run the YAIM configurator and verify that the VO is filtered and does not appear in the sudoers file.

   * Bug [[https://savannah.cern.ch/bugs/?69785][#69785]]: Use #include to modify /etc/sudoer for CREAM with sudo >= 1.7.1 %GREEN%FIXED%ENDCOLOR%
      * With sudo version >= 1.7.1 installed, reconfigure with YAIM 
      * Verify that all CREAM directives are written into the file /etc/sudoers.forcream
      * Update from glite-CREAM version < 3.2.8, reconfigure with YAIM
      * Verify that the CREAM directives have been moved from the original /etc/sudoers file into the new /etc/sudoers.forcream

   * Bug [[https://savannah.cern.ch/bugs/?70285][#70285]]: it should be possible to tell bhist and tracejob how many logs to read %GREEN%FIXED%ENDCOLOR%
      * Submit a bunch of jobs so that all the slots of the CE are busy for more than 24 hours, for example given 200 slots use the following test: <verbatim>
cream-test-monitored-submit -r 30 -n 200 -m 200 -C 50 -j long.jdl -R [ce-resource-url]
</verbatim>, where the jdl file contains:<verbatim>
[executable="/bin/sleep";arguments="86400";]
</verbatim>
      * Submit a new bunch of jobs so that they will be enqueued for more than 24 hours
      * Verify that all the jobs terminate correctly and that all the status changes are correctly logged into the Bnotifier log.

   * Bug [[https://savannah.cern.ch/bugs/?70287][#70287]]: Problem with !AssignFinalState if state is IDLE %GREEN%FIXED%ENDCOLOR%
      * Define a value short enough for the parameter  "alldone_interval" in the blah.config file, for example 1200 seconds.
      * Submit a bunch of jobs so that all the slots of the CE are busy for more than 1200 seconds, for example given 200 slots use the following test: <verbatim>
cream-test-monitored-submit -r 30 -n 200 -m 200 -C 50 -j long.jdl -R [ce-resource-url]
</verbatim>, where the jdl file contains:<verbatim>
[executable="/bin/sleep";arguments="1800";]
</verbatim>
      * Submit a new bunch of jobs whose idle time and effective processing time are larger than 1200 seconds.
      * Verify that all the jobs terminate correctly and that all the status changes are correctly logged into the Bnotifier log.

   * Bug [[https://savannah.cern.ch/bugs/?70808][#70808]]: PBS submission script creates wrong stagein/stageout directives with qsub's -W option %GREEN%FIXED%ENDCOLOR%
      * Save a backup of the BLAH submit wrapper, replacing the statement "rm -f $bls_tmp_file" with "cp $bls_tmp_file /tmp" in the script ${GLITE_LOCATION}/bin/blah_common_submit_functions.sh
      * Submit a job and verify that the -W directive, as specified in the BLAH submit wrapper, contains a list of item "stagein=<value>" and "stageout=<value>"
      * Verify that all the file specified in the stagein or in the stageout directives are correctly moved between the CREAM and the worker node.

   * Bug [[https://savannah.cern.ch/bugs/?71038][#71038]]: CE certificate publication error %GREEN%FIXED%ENDCOLOR%
      * Use a host certificate whose subject DN contains at least one blank character
      * Run the YAIM configurator and verify that the attribute !GlueServiceDataValue of the item <verbatim>
GlueServiceDataKey=DN,GlueServiceUniqueID=[unique ID for the CE],Mds-Vo-name=resource,o=grid
</verbatim> contains the complete DN

   * Bug [[https://savannah.cern.ch/bugs/?71055][#71055]]: !LL_ and !IL_ PIDFILE added more times in /opt/glite/etc/lb.conf  %GREEN%FIXED%ENDCOLOR%
      * Run more times the YAIM configurator and verify that the file /opt/glite/etc/lb.conf does not change.

   * Bug [[https://savannah.cern.ch/bugs/?71967][#71967]]: There should be in blah the option to set the temporary job work dir %GREEN%FIXED%ENDCOLOR%
      * Define the value of blah_wn_temporary_home_dir in the file ${GLITE_LOCATION}/etc/blah.config pointing to '/tmp'
      * Restart all gLite services on the node.
      * Submit the following jdl: <verbatim>
[
executable="/bin/pwd";
stdoutput="out-pwd.txt";
outputsandbox={
    "out-pwd.txt"
};
outputsandboxbasedesturi="gsiftp://xxxxx/tmp";
epilogue="/bin/sleep";
epiloguearguments="30";
]
</verbatim> and verify that the file out-pwd.txt contains something like "/tmp/home_creXX_YYYYYYYYY/CREAMYYYYYYYYY

   * Bug [[https://savannah.cern.ch/bugs/?72097][#72097]]: cream-config.xml and cemonitor-config.xml should use GLITE_LOCATION_LOG and GLITE_LOCATION_VAR  %GREEN%FIXED%ENDCOLOR%
      * Define the variables GLITE_LOCATION_VAR=[new var directory] and CREAM_SANDBOX_PATH=[new sandbox dir] in the file services/glite-creamce
      * Run the YAIM configurator and verify that the new var directory is specified into the following attributes:
         * cream-config.xml: cream_scratch_dir, cream_sandbox_dir, delegation_storage, CREAM_SANDBOX_DIR, JOBS_SCRATCH_DIR
         * cemonitor-config.xml: backendLocation

   * Bug [[https://savannah.cern.ch/bugs/?72508][#72508]]: variable CREAM_DB_PASSWORD missed in services/glite-creamce %GREEN%FIXED%ENDCOLOR%
      * Verify that the file ${GLITE_LOCATION}/yaim/examples/siteinfo/services/glite-creamce contains the variable CREAM_DB_PASSWORD

   * Bug [[https://savannah.cern.ch/bugs/?72756][#72756]]: glite-ce-blahparser missed in gLiteservices %GREEN%FIXED%ENDCOLOR%
      * Stop all the gLite service on the CE
      * Remove the script glite-ce-blahparser from the gLiteservices list
      * Reconfigure with YAIM and verify that the glite-ce-blahparser has been re-inserted into the list

   * Bug [[https://savannah.cern.ch/bugs/?73765][#73765]]: Performance problems with CREAM proxy renewal %GREEN%FIXED%ENDCOLOR%
      * Run the following test: <verbatim>
cream-test-monitored-submit -r 30 -n 1000 -m 200 -C 50 -l log4py.conf --nopurge --sotimeout 60 -j 15m.jdl -R [ce-url]
</verbatim> where the file 15m.jdl contains: <verbatim>
[ executable="/bin/sleep"; arguments="900";]
</verbatim>
      * Run a proxy renewal every 5 minutes specifying the delegation id as reported in log4py.conf
      * Keep the value of pending commands monitored, using the following script: <verbatim>
for ((idx=0; idx < 240; idx++)); do /opt/glite/bin/glite_cream_load_monitor --show | grep pending; sleep 30; done
</verbatim>
      * Verify that all the renewal requests are correctly processed and the number of pending commands does not increase

   * Bug [[https://savannah.cern.ch/bugs/?73961][#73961]]: /tmp of CREAM CE full of glexec* proxies %GREEN%FIXED%ENDCOLOR%
      * Submit a job and verifiy that no proxy has been created in the temporary directory /tmp

   * Bug [[https://savannah.cern.ch/bugs/?74107][#74107]]:  !ClientJobId missing in !BNotifier classad output %GREEN%FIXED%ENDCOLOR%
      * Stop all gLite services on the CREAM node and run only the !BNotifier/BUpdater (/opt/glite/etc/glite-ce-blahparser start)
      * Contact the BNotifier daemon via telnet and send the following set of commands:<verbatim>
CREAMFILTER/[blah_prefix]
PARSERVERSION/
STARTNOTIFYJOBLIST/[blah_prefix][client_job_id]
STARTNOTIFYJOBEND
</verbatim> where 
         * "[blah_prefix]" is the value of the parameter BLAH_JOBID_PREFIX of the cream configuration file
         * "[client_job_id]" is random 9-digits number so that the value "[blah_prefix][client_job_id]" is not contained into the !BNotifier registry, this can be verified  using the following command: <verbatim>
strings $GLITE_LOCATION/var/blah/user_blah_job_registry.bjr/registry | grep [blah_prefix][client_job_id]
</verbatim>
      * Verify that the classad returned by the !BNotifier contains the parameter "ClientJobId"
      * Close the connection with !BNotifier and restart all gLite services on the node

---+++ Clean installation

   * Installation steps:<verbatim>
wget http://etics-repository.cern.ch/repository/pm/registered/repomd/name/patch_4415_2/etics-registered-build-by-name.repo -O /etc/yum.repos.d/glite-CREAM.repo
yum install xml-commons-apis
yum install glite-CREAM
wget http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/3.2/glite-TORQUE_utils.repo -O /etc/yum.repos.d/glite-TORQUE_utils.repo
wget http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/3.2/glite-TORQUE_server.repo -O /etc/yum.repos.d/glite-TORQUE_server.repo
yum install glite-TORQUE_utils glite-TORQUE_server
/opt/glite/yaim/bin/yaim -c -s site-info.def -n creamCE -n TORQUE_server -n TORQUE_utils
</verbatim>
   * View the [[%ATTACHURL%/yum_install_log.txt.gz][log]] of yum for a clean installation 
   * View the [[%ATTACHURL%/yaim_install_log.txt.gz][log]] of yaim for a clean installation (TORQUE is used)

---+++ Upgrade from production
   * Upgrade steps:<verbatim>
wget http://etics-repository.cern.ch/repository/pm/registered/repomd/name/patch_4415_2/etics-registered-build-by-name.repo -O /etc/yum.repos.d/glite-CREAM.repo
yum update
/opt/glite/yaim/bin/yaim -c -s site-info.def -n creamCE -n TORQUE_server -n TORQUE_utils
</verbatim>
   * View the [[%ATTACHURL%/yum_update_log.txt.gz][log]] of yum for an upgrade
   * View the [[%ATTACHURL%/yaim_update_log.txt.gz][log]] of yaim for an upgrade (TORQUE is used)
Attachments
Topic attachments
I	Attachment	Action	Size	Date	Who	Comment
txt	infotest1.txt	manage	0.8 K	2010-10-25 - 07:52	PaoloAndreetto	infotest1
txt	infotest2.txt	manage	64.1 K	2010-10-25 - 07:52	PaoloAndreetto	infotest2
txt	infotest3.txt	manage	6.9 K	2010-10-25 - 07:52	PaoloAndreetto	infotest3
gz	reports_patch_4415.tar.gz	manage	263.8 K	2010-10-28 - 07:20	PaoloAndreetto	Testsuite reports for patch 4415
gz	yaim_install_log.txt.gz	manage	19.7 K	2010-10-19 - 14:17	PaoloAndreetto	YAIM installation log
gz	yaim_update_log.txt.gz	manage	16.4 K	2010-10-19 - 14:18	PaoloAndreetto	YAIM update log
gz	yum_install_log.txt.gz	manage	9.3 K	2010-10-19 - 14:16	PaoloAndreetto	YUM installation log
gz	yum_update_log.txt.gz	manage	1.0 K	2010-10-19 - 14:17	PaoloAndreetto	YUM update log
Topic revision: r21 - 2010-10-28 - PaoloAndreetto
Account
- Log In
~~Edit~~
~~Attach~~