Testing and Certification report savannah task #23312: BLAH v. 1.18.0 (EMI2)
Summary
- Product: BLAH v. 1.18.0
- Release Task: Task #23312
- ETICS Subsystem Configuration Name: **
- VCS Tag:
- EMI Major Release: EMI 2 (Matterhorn)
- Platform: SL5 epel, SL6 epel
- Author: Sara Bertocco
- Testing report:
- Certification report:
- Date: 3 May 2012
- Outcome: In certification
Deployment tests
Clean Installation
- EMI2 BLAH SL5 installation from scratch and configuration log (in CREAM environment) here
- EMI2 BLAH SL6 installation from scratch and configuration log (in CREAM environment) here
.
Upgrade Installation
- EMI2 BLAH SL5 update from EMI1 production version and configuration log (in CREAM environment) here
.
Unit Tests
- SL5 Unit test report available here
.
- SL5 Unit test report available here
.
System tests
Functionality tests
Check Notifications SL5 - LSF New parser. PASSED with bupdater_use_bhist_for_killed="yes" in /etc/blah.config
Trying with default blah configuration file /etc/blah.config the test on cancelled jobs notifications fails:
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_cancelled_jobs | FAIL |
NOTIFICATIONS FAILURE != NOTIFICATIONS OK
------------------------------------------------------------------------------
Tests.Check Notifications For Cancelled Jobs :: Test that notifica... | FAIL |
2 critical tests, 1 passed, 1 failed
2 tests total, 1 passed, 1 failed
==============================================================================
Tests.Check Notifications For Normally Finished Jobs :: Test that notificat...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_normally_finished_jobs | PASS |
------------------------------------------------------------------------------
Tests.Check Notifications For Normally Finished Jobs :: Test that ... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Tests.Check Notifications For Suspended Resumed Jobs :: Test that notificat...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_suspended_resumed_jobs | PASS |
------------------------------------------------------------------------------
Tests.Check Notifications For Suspended Resumed Jobs :: Test that ... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Tests | FAIL |
6 critical tests, 5 passed, 1 failed
6 tests total, 5 passed, 1 failed
==============================================================================
Changing the configuration file /etc/blah.config adding the row
bupdater_use_bhist_for_killed="yes"
all notification tests have success.
The files of this testsuite will be stored under: /tmp/tmp5KGJmM.cream_testing/
Tests.Check Notifications For Cancelled Jobs :: Test that notifications are...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_cancelled_jobs | PASS |
------------------------------------------------------------------------------
Tests.Check Notifications For Cancelled Jobs :: Test that notifica... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Tests.Check Notifications For Normally Finished Jobs :: Test that notificat...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_normally_finished_jobs | PASS |
------------------------------------------------------------------------------
Tests.Check Notifications For Normally Finished Jobs :: Test that ... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Tests.Check Notifications For Suspended Resumed Jobs :: Test that notificat...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_suspended_resumed_jobs | PASS |
------------------------------------------------------------------------------
Tests.Check Notifications For Suspended Resumed Jobs :: Test that ... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Tests | PASS |
6 critical tests, 6 passed, 0 failed
6 tests total, 6 passed, 0 failed
==============================================================================
Check Notifications SL5 - LSF Old parser NOT PASSED
This is because of
bug #90085
already a known issue also for EMI-1
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_cancelled_jobs | PASS |
------------------------------------------------------------------------------
Tests.Check Notifications For Cancelled Jobs :: Test that notifica... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Tests.Check Notifications For Normally Finished Jobs :: Test that notificat...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_normally_finished_jobs | PASS |
------------------------------------------------------------------------------
Tests.Check Notifications For Normally Finished Jobs :: Test that ... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Tests.Check Notifications For Suspended Resumed Jobs :: Test that notificat...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_suspended_resumed_jobs | FAIL |
_error: Expected status should be in ['HELD'] for job https://cream-48.pd.infn.it:8443/CREAM329108161 was actually IDLE
------------------------------------------------------------------------------
Tests.Check Notifications For Suspended Resumed Jobs :: Test that ... | FAIL |
2 critical tests, 1 passed, 1 failed
2 tests total, 1 passed, 1 failed
==============================================================================
Tests | FAIL |
6 critical tests, 5 passed, 1 failed
6 tests total, 5 passed, 1 failed
==============================================================================
Check Notifications SL5 - PBS New parser PASSED
- For normally finished jobs
ON UI:
$ date; glite-ce-job-submit -a -r emi-demo13.cnaf.infn.it:8443/cream-pbs-demo -a /home/bertocco/test1.jdl
Thu May 3 15:26:09 CEST 2012
https://emi-demo13.cnaf.infn.it:8443/CREAM431692435
$ date; glite-ce-job-status https://emi-demo13.cnaf.infn.it:8443/CREAM431692435
Thu May 3 15:27:43 CEST 2012
****** JobID=[https://emi-demo13.cnaf.infn.it:8443/CREAM431692435]
Status = [DONE-OK]
ExitCode = [0]
On CE CREAM:
# grep 431692435 /var/log/cream/glite-ce-bnotifier.log
2012-05-03 15:26:34 Sent for Cream:[BatchJobId="269.emi-demo13.cnaf.infn.it"; JobStatus=2; ChangeTime="2012-05-03 15:26:12"; WorkerNode="emitestbed23.cnaf.infn.it"; ClientJobId="431692435"; BlahJobName="cream_431692435";]
2012-05-03 15:27:14 Sent for Cream:[BatchJobId="269.emi-demo13.cnaf.infn.it"; JobStatus=4; ChangeTime="2012-05-03 15:26:42"; WorkerNode="emitestbed23.cnaf.infn.it"; JwExitCode=0; Reason="reason=0"; ClientJobId="431692435"; BlahJobName="cream_431692435";]
On UI:
date; glite-ce-job-submit -a -r emi-demo13.cnaf.infn.it:8443/cream-pbs-demo -a /home/bertocco/test1.jdl
Thu May 3 15:17:08 CEST 2012
https://emi-demo13.cnaf.infn.it:8443/CREAM585808873
[bertocco@emitestbed08 blah_testing]$ date; glite-ce-job-status https://emi-demo13.cnaf.infn.it:8443/CREAM585808873
Thu May 3 15:17:20 CEST 2012
****** JobID=[https://emi-demo13.cnaf.infn.it:8443/CREAM585808873]
Status = [REALLY-RUNNING]
[bertocco@emitestbed08 blah_testing]$ glite-ce-job-cancel https://emi-demo13.cnaf.infn.it:8443/CREAM585808873
Are you sure you want to cancel specified job(s) [y/n]: y
[bertocco@emitestbed08 blah_testing]$ date; glite-ce-job-status https://emi-demo13.cnaf.infn.it:8443/CREAM585808873
Thu May 3 15:17:43 CEST 2012
****** JobID=[https://emi-demo13.cnaf.infn.it:8443/CREAM585808873]
Status = [CANCELLED]
ExitCode = []
Description = [Cancelled by user]
On CE CREAM:
# grep 585808873 /var/log/cream/glite-ce-bnotifier.log
2012-05-03 15:17:14 Sent for Cream:[BatchJobId="268.emi-demo13.cnaf.infn.it"; JobStatus=2; ChangeTime="2012-05-03 15:17:08"; WorkerNode="emitestbed23.cnaf.infn.it"; ClientJobId="585808873"; BlahJobName="cream_585808873";]
2012-05-03 15:18:14 Sent for Cream:[BatchJobId="268.emi-demo13.cnaf.infn.it"; JobStatus=3; ChangeTime="2012-05-03 15:17:31"; WorkerNode="emitestbed23.cnaf.infn.it"; JwExitCode=-999; Reason="reason=-999"; ClientJobId="585808873"; BlahJobName="cream_585808873";]
- For suspended and resumed jobs:
On UI:
$ glite-ce-job-status https://emi-demo13.cnaf.infn.it:8443/CREAM641304895
****** JobID=[https://emi-demo13.cnaf.infn.it:8443/CREAM641304895]
Status = [IDLE]
$ glite-ce-job-suspend https://emi-demo13.cnaf.infn.it:8443/CREAM641304895
Are you sure you want to suspend specified job(s) [y/n]: y
[bertocco@emitestbed08 blah_testing]$ glite-ce-job-status https://emi-demo13.cnaf.infn.it:8443/CREAM641304895
****** JobID=[https://emi-demo13.cnaf.infn.it:8443/CREAM641304895]
Status = [HELD]
$ glite-ce-job-resume https://emi-demo13.cnaf.infn.it:8443/CREAM641304895
Are you sure you want to resume specified job(s) [y/n]: y
[bertocco@emitestbed08 blah_testing]$ glite-ce-job-status https://emi-demo13.cnaf.infn.it:8443/CREAM641304895
****** JobID=[https://emi-demo13.cnaf.infn.it:8443/CREAM641304895]
Status = [HELD]
$ glite-ce-job-status https://emi-demo13.cnaf.infn.it:8443/CREAM641304895
****** JobID=[https://emi-demo13.cnaf.infn.it:8443/CREAM641304895]
Status = [IDLE]
$ glite-ce-job-status https://emi-demo13.cnaf.infn.it:8443/CREAM641304895
****** JobID=[https://emi-demo13.cnaf.infn.it:8443/CREAM641304895]
Status = [DONE-OK]
ExitCode = [0]
On CE CREAM:
# grep 641304895 /var/log/cream/glite-ce-bnotifier.log
[BatchJobId="291.emi-demo13.cnaf.infn.it"; JobStatus=1; ChangeTime="2012-05-03 15:54:46"; ClientJobId="641304895"; BlahJobName="cream_641304895";]
2012-05-03 15:56:14 Sent for Cream:[BatchJobId="291.emi-demo13.cnaf.infn.it"; JobStatus=5; ChangeTime="2012-05-03 15:56:13"; ClientJobId="641304895"; BlahJobName="cream_641304895";]
[BatchJobId="291.emi-demo13.cnaf.infn.it"; JobStatus=1; ChangeTime="2012-05-03 15:57:16"; ClientJobId="641304895"; BlahJobName="cream_641304895";]
[BatchJobId="291.emi-demo13.cnaf.infn.it"; JobStatus=2; ChangeTime="2012-05-03 16:03:36"; WorkerNode="emitestbed23.cnaf.infn.it"; ClientJobId="641304895"; BlahJobName="cream_641304895";]
[BatchJobId="291.emi-demo13.cnaf.infn.it"; JobStatus=4; ChangeTime="2012-05-03 16:04:36"; WorkerNode="emitestbed23.cnaf.infn.it"; JwExitCode=0; Reason="reason=0"; ClientJobId="641304895"; BlahJobName="cream_641304895";]
Check Notifications SL5 - PBS Old parser Not Tested
Check Notifications SL6 - LSF New parser Not Tested
Check Notifications SL6 - LSF Old parser Not Tested
- Noramally finished jobs Passed ---+++++ Check Notifications SL6 - PBS New parser PASSED
- For normally finished jobs
ON UI:
$ glite-ce-job-status https://emitestbed29.cnaf.infn.it:8443/CREAM400857385
****** JobID=[https://emitestbed29.cnaf.infn.it:8443/CREAM400857385]
Status = [REALLY-RUNNING]
$ glite-ce-job-status https://emitestbed29.cnaf.infn.it:8443/CREAM400857385
****** JobID=[https://emitestbed29.cnaf.infn.it:8443/CREAM400857385]
Status = [DONE-OK]
ExitCode = [0]
ON CE CREAM:
# grep 400857385 /var/log/cream/glite-ce-bnotifier.log
2012-05-03 16:38:46 Sent for Cream:[BatchJobId="36.emitestbed29.cnaf.infn.it"; JobStatus=2; ChangeTime="2012-05-03 16:38:22"; WorkerNode="emitestbed28.cnaf.infn.it"; ClientJobId="400857385"; BlahJobName="cream_400857385";]
2012-05-03 16:39:46 Sent for Cream:[BatchJobId="36.emitestbed29.cnaf.infn.it"; JobStatus=4; ChangeTime="2012-05-03 16:39:22"; WorkerNode="emitestbed28.cnaf.infn.it"; JwExitCode=0; Reason="reason=0"; ClientJobId="400857385"; BlahJobName="cream_400857385";]
ON UI:
date; glite-ce-job-submit -a -r emitestbed29.cnaf.infn.it:8443/cream-pbs-demo -a /home/bertocco/test1.jdl
Thu May 3 16:42:28 CEST 2012
https://emitestbed29.cnaf.infn.it:8443/CREAM060970469
$ glite-ce-job-status https://emitestbed29.cnaf.infn.it:8443/CREAM060970469
****** JobID=[https://emitestbed29.cnaf.infn.it:8443/CREAM060970469]
Status = [REALLY-RUNNING]
$ glite-ce-job-cancel https://emitestbed29.cnaf.infn.it:8443/CREAM060970469
Are you sure you want to cancel specified job(s) [y/n]: y
$ glite-ce-job-status https://emitestbed29.cnaf.infn.it:8443/CREAM060970469
****** JobID=[https://emitestbed29.cnaf.infn.it:8443/CREAM060970469]
Status = [CANCELLED]
ExitCode = []
Description = [Cancelled by user]
On CE CREAM:
# grep 060970469 /var/log/cream/glite-ce-bnotifier.log
2012-05-03 16:42:46 Sent for Cream:[BatchJobId="37.emitestbed29.cnaf.infn.it"; JobStatus=3; ChangeTime="2012-05-03 16:42:42"; WorkerNode="emitestbed28.cnaf.infn.it"; JwExitCode=-999; Reason="reason=-999"; ClientJobId="060970469"; BlahJobName="cream_060970469";]
- For suspended and resumed jobs:
On UI:
# glite-ce-job-submit -a -r emitestbed29.cnaf.infn.it:8443/cream-pbs-demo -a /home/bertocco/test1.jdl
https://emitestbed29.cnaf.infn.it:8443/CREAM535518208
$ glite-ce-job-status https://emitestbed29.cnaf.infn.it:8443/CREAM535518208
****** JobID=[https://emitestbed29.cnaf.infn.it:8443/CREAM535518208]
Status = [IDLE]
$ glite-ce-job-suspend https://emitestbed29.cnaf.infn.it:8443/CREAM535518208
Are you sure you want to suspend specified job(s) [y/n]: y
$ glite-ce-job-status https://emitestbed29.cnaf.infn.it:8443/CREAM535518208
****** JobID=[https://emitestbed29.cnaf.infn.it:8443/CREAM535518208]
Status = [HELD]
$ glite-ce-job-resume https://emitestbed29.cnaf.infn.it:8443/CREAM535518208
Are you sure you want to resume specified job(s) [y/n]: y
$ glite-ce-job-status https://emitestbed29.cnaf.infn.it:8443/CREAM535518208
****** JobID=[https://emitestbed29.cnaf.infn.it:8443/CREAM535518208]
Status = [DONE-OK]
ExitCode = [0]
On CE CREAM:
grep 535518208 /var/log/cream/glite-ce-bnotifier.log
[BatchJobId="40.emitestbed29.cnaf.infn.it"; JobStatus=1; ChangeTime="2012-05-03 17:00:09"; ClientJobId="535518208"; BlahJobName="cream_535518208";]
2012-05-03 17:00:46 Sent for Cream:[BatchJobId="40.emitestbed29.cnaf.infn.it"; JobStatus=5; ChangeTime="2012-05-03 17:00:27"; ClientJobId="535518208"; BlahJobName="cream_535518208";]
2012-05-03 17:01:06 Sent for Cream:[BatchJobId="40.emitestbed29.cnaf.infn.it"; JobStatus=2; ChangeTime="2012-05-03 17:01:05"; WorkerNode="emitestbed28.cnaf.infn.it"; ClientJobId="535518208"; BlahJobName="cream_535518208";]
[BatchJobId="40.emitestbed29.cnaf.infn.it"; JobStatus=2; ChangeTime="2012-05-03 17:01:10"; WorkerNode="emitestbed28.cnaf.infn.it"; ClientJobId="535518208"; BlahJobName="cream_535518208";]
2012-05-03 17:02:46 Sent for Cream:[BatchJobId="40.emitestbed29.cnaf.infn.it"; JobStatus=4; ChangeTime="2012-05-03 17:02:10"; WorkerNode="emitestbed28.cnaf.infn.it"; JwExitCode=0; Reason="reason=0"; ClientJobId="535518208"; BlahJobName="cream_535518208";]
Check Notifications SL6 - PBS Old parser Not Tested
Regression tests
Note: All bug tests are performed on CREAM EMI 2 SL5 old and new parser. Bugs signed with 'NA' are not certified because difficult to reproduce.
Verification attached bugs
==============================================================================
Bug 86238 :: Regression test of bug #86238 blahpd doesn't check the status ...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
bug_86238 | PASS |
------------------------------------------------------------------------------
Bug 86238 :: Regression test of bug #86238 blahpd doesn't check th... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
On CREAM
cat /tmp/filewithenv
MANPATH=/opt/glite/share/man::
ceid=emi-demo13.cnaf.infn.it:8443/cream-pbs-demo
HOSTNAME=emi-demo13.cnaf.infn.it
TERM=vanilla
SHELL=/bin/sh
GRID_ENV_LOCATION=/usr/libexec
HISTSIZE=1000
CATALINA_HOME=/usr/share/tomcat5
BLAHPD_CONFIG_LOCATION=/etc/blah.config
ClientJobId=cream_483887197
X509_CERT_DIR=/etc/grid-security/certificates/
GRIDMAPDIR=/etc/grid-security/gridmapdir
USER=tst22
LS_COLORS=
SUDO_USER=tomcat
SUDO_UID=91
LCG_LOCATION=/usr
USERNAME=tst22
MAIL=/var/spool/mail/tst22
PATH=/bin:/usr/bin:/home/tst22/bin
GLITE_HOST_KEY=/home/glite/.certs/hostkey.pem
uniquejobid=CREAM483887197
INPUTRC=/etc/inputrc
PWD=/var/tmp
LANG=en_US.UTF-8
GLITE_USER=glite
gridType=pbs
x509UserProxySubject=CN=Sara Bertocco,L=Padova,OU=Personal Certificate,O=INFN,C=IT
BLAHPD_LOCATION=/usr
HOME=/home/tst22
SUDO_COMMAND=/usr/libexec/pbs_submit.sh -x /var/cream_sandbox/testers/CN_Sara_Bertocco_L_Padova_OU_Personal_Certificate_O_INFN_C_IT_testers_eu_emi_eu_Role_NULL_Capability_NULL_tst22/proxy/c7b5086871f50a68b630e3eca03499a17a0eec4c_12953230182596 -u CN=Sara Bertocco,L=Padova,OU=Personal Certificate,O=INFN,C=IT -r no -c /var/cream_sandbox/testers/CN_Sara_Bertocco_L_Padova_OU_Personal_Certificate_O_INFN_C_IT_testers_eu_emi_eu_Role_NULL_Capability_NULL_tst22/48/CREAM483887197/CREAM483887197_jobWrapper.sh -T /tmp -C /tmp/ce-req-file-133614658720016 -o /var/cream_sandbox/testers/CN_Sara_Bertocco_L_Padova_OU_Personal_Certificate_O_INFN_C_IT_testers_eu_emi_eu_Role_NULL_Capability_NULL_tst22/48/CREAM483887197/StandardOutput -e /var/cream_sandbox/testers/CN_Sara_Bertocco_L_Padova_OU_Personal_Certificate_O_INFN_C_IT_testers_eu_emi_eu_Role_NULL_Capability_NULL_tst22/48/CREAM483887197/StandardError -q demo -n 1 -j cream_483887197
SHLVL=2
GLITE_LOCATION_VAR=/var
GLOBUS_TCP_PORT_RANGE=20000,25000
VirtualOrganisation=testers.eu-emi.eu
LOGNAME=tst22
queue=demo
LESSOPEN=|/usr/bin/lesspipe.sh %s
x509UserProxyFQAN="userFQAN=/testers.eu-emi.eu/Role=NULL/Capability=NULL"
SUDO_GID=91
GLITE_LOCATION=/usr
GLITE_HOST_CERT=/home/glite/.certs/hostcert.pem
G_BROKEN_FILENAMES=1
_=/bin/env
# cat /tmp/filewithenv |grep x509UserProxySubject
x509UserProxySubject=CN=Sara Bertocco,L=Padova,OU=Personal Certificate,O=INFN,C=IT
[root@emi-demo13 ~]# cat /tmp/filewithenv |grep x509UserProxyFQAN
x509UserProxyFQAN="userFQAN=/testers.eu-emi.eu/Role=NULL/Capability=NULL"
Verifyed, in /usr/libexec/blah_common_submit_functions.sh the requested rows are present:
748 function bls_add_job_wrapper ()
749 {
750 bls_start_job_wrapper >> $bls_tmp_file
751 bls_finish_job_wrapper >> $bls_tmp_file
752 bls_test_working_dir
753 }
==============================================================================
Bug 84261 :: Regression test of bug #84261 BNotifier on CREAM CE seems to n...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
bug_84261 | PASS |
------------------------------------------------------------------------------
Bug 84261 :: Regression test of bug #84261 BNotifier on CREAM CE s... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Verification old bugs
==============================================================================
Bug 81824 :: Regression test of bug #81824 yaim-cream-ce should manage the ...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
bug_81824 | PASS |
------------------------------------------------------------------------------
Bug 81824 :: Regression test of bug #81824 yaim-cream-ce should ma... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
--
SaraBertocco - 2012-05-03