Task23312 < IGIRelease

Tags: , view all tags

Testing and Certification report savannah task #23312: BLAH v. 1.18.0 (EMI2)

Testing and Certification report savannah task #23312: BLAH v. 1.18.0 (EMI2)
Summary

Summary

Product: BLAH v. 1.18.0
Release Task: Task #23312
ETICS Subsystem Configuration Name: **
VCS Tag:
EMI Major Release: EMI 2 (Matterhorn)
Platform: SL5 epel, SL6 epel
Author: Sara Bertocco
Testing report:
Certification report:
Date: 3 May 2012
Outcome: In certification

Deployment tests

Clean Installation

EMI2 BLAH SL5 installation from scratch and configuration log (in CREAM environment) here

EMI2 BLAH SL6 installation from scratch and configuration log (in CREAM environment) here.

Upgrade Installation

EMI2 BLAH SL5 update from EMI1 production version and configuration log (in CREAM environment) here.

Unit Tests

SL5 Unit test report available here.
SL5 Unit test report available here.

System tests

Functionality tests

Check Notifications SL5 - LSF New parser. PASSED with bupdater_use_bhist_for_killed="yes" in /etc/blah.config

Trying with default blah configuration file /etc/blah.config the test on cancelled jobs notifications fails:

 ==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_cancelled_jobs                                | FAIL |
NOTIFICATIONS FAILURE != NOTIFICATIONS OK
------------------------------------------------------------------------------
Tests.Check Notifications For Cancelled Jobs :: Test that notifica... | FAIL |
2 critical tests, 1 passed, 1 failed
2 tests total, 1 passed, 1 failed
==============================================================================
Tests.Check Notifications For Normally Finished Jobs :: Test that notificat...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_normally_finished_jobs                        | PASS |
------------------------------------------------------------------------------
Tests.Check Notifications For Normally Finished Jobs :: Test that ... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Tests.Check Notifications For Suspended Resumed Jobs :: Test that notificat...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_suspended_resumed_jobs                        | PASS |
------------------------------------------------------------------------------
Tests.Check Notifications For Suspended Resumed Jobs :: Test that ... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Tests                                                                 | FAIL |
6 critical tests, 5 passed, 1 failed
6 tests total, 5 passed, 1 failed
==============================================================================

Changing the configuration file /etc/blah.config adding the row

bupdater_use_bhist_for_killed="yes"

all notification tests have success.

The files of this testsuite will be stored under: /tmp/tmp5KGJmM.cream_testing/
Tests.Check Notifications For Cancelled Jobs :: Test that notifications are...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_cancelled_jobs                                | PASS |
------------------------------------------------------------------------------
Tests.Check Notifications For Cancelled Jobs :: Test that notifica... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Tests.Check Notifications For Normally Finished Jobs :: Test that notificat...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_normally_finished_jobs                        | PASS |
------------------------------------------------------------------------------
Tests.Check Notifications For Normally Finished Jobs :: Test that ... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Tests.Check Notifications For Suspended Resumed Jobs :: Test that notificat...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_suspended_resumed_jobs                        | PASS |
------------------------------------------------------------------------------
Tests.Check Notifications For Suspended Resumed Jobs :: Test that ... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Tests                                                                 | PASS |
6 critical tests, 6 passed, 0 failed
6 tests total, 6 passed, 0 failed
==============================================================================

Check Notifications SL5 - LSF Old parser NOT PASSED

This is because of bug #90085 already a known issue also for EMI-1

==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_cancelled_jobs                                | PASS |
------------------------------------------------------------------------------
Tests.Check Notifications For Cancelled Jobs :: Test that notifica... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Tests.Check Notifications For Normally Finished Jobs :: Test that notificat...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_normally_finished_jobs                        | PASS |
------------------------------------------------------------------------------
Tests.Check Notifications For Normally Finished Jobs :: Test that ... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Tests.Check Notifications For Suspended Resumed Jobs :: Test that notificat...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_suspended_resumed_jobs                        | FAIL |
_error: Expected status should be in ['HELD'] for job https://cream-48.pd.infn.it:8443/CREAM329108161 was actually IDLE
------------------------------------------------------------------------------
Tests.Check Notifications For Suspended Resumed Jobs :: Test that ... | FAIL |
2 critical tests, 1 passed, 1 failed
2 tests total, 1 passed, 1 failed
==============================================================================
Tests                                                                 | FAIL |
6 critical tests, 5 passed, 1 failed
6 tests total, 5 passed, 1 failed
==============================================================================

Check Notifications SL5 - PBS New parser PASSED

For normally finished jobs

ON UI:

$ date; glite-ce-job-submit  -a -r emi-demo13.cnaf.infn.it:8443/cream-pbs-demo  -a /home/bertocco/test1.jdl
Thu May  3 15:26:09 CEST 2012
https://emi-demo13.cnaf.infn.it:8443/CREAM431692435
$  date; glite-ce-job-status https://emi-demo13.cnaf.infn.it:8443/CREAM431692435
Thu May  3 15:27:43 CEST 2012

******  JobID=[https://emi-demo13.cnaf.infn.it:8443/CREAM431692435]
   Status        = [DONE-OK]
   ExitCode      = [0]

On CE CREAM:

# grep 431692435 /var/log/cream/glite-ce-bnotifier.log
2012-05-03 15:26:34 Sent for Cream:[BatchJobId="269.emi-demo13.cnaf.infn.it"; JobStatus=2; ChangeTime="2012-05-03 15:26:12"; WorkerNode="emitestbed23.cnaf.infn.it"; ClientJobId="431692435"; BlahJobName="cream_431692435";]
2012-05-03 15:27:14 Sent for Cream:[BatchJobId="269.emi-demo13.cnaf.infn.it"; JobStatus=4; ChangeTime="2012-05-03 15:26:42"; WorkerNode="emitestbed23.cnaf.infn.it"; JwExitCode=0; Reason="reason=0"; ClientJobId="431692435"; BlahJobName="cream_431692435";]

For cancelled jobs

On UI:

date; glite-ce-job-submit  -a -r emi-demo13.cnaf.infn.it:8443/cream-pbs-demo  -a /home/bertocco/test1.jdl
Thu May  3 15:17:08 CEST 2012
https://emi-demo13.cnaf.infn.it:8443/CREAM585808873
[bertocco@emitestbed08 blah_testing]$ date; glite-ce-job-status https://emi-demo13.cnaf.infn.it:8443/CREAM585808873
Thu May  3 15:17:20 CEST 2012

******  JobID=[https://emi-demo13.cnaf.infn.it:8443/CREAM585808873]
   Status        = [REALLY-RUNNING]

[bertocco@emitestbed08 blah_testing]$ glite-ce-job-cancel https://emi-demo13.cnaf.infn.it:8443/CREAM585808873

Are you sure you want to cancel specified job(s) [y/n]: y
[bertocco@emitestbed08 blah_testing]$ date; glite-ce-job-status https://emi-demo13.cnaf.infn.it:8443/CREAM585808873
Thu May  3 15:17:43 CEST 2012

******  JobID=[https://emi-demo13.cnaf.infn.it:8443/CREAM585808873]
   Status        = [CANCELLED]
   ExitCode      = []
   Description   = [Cancelled by user]

On CE CREAM:

# grep 585808873 /var/log/cream/glite-ce-bnotifier.log
2012-05-03 15:17:14 Sent for Cream:[BatchJobId="268.emi-demo13.cnaf.infn.it"; JobStatus=2; ChangeTime="2012-05-03 15:17:08"; WorkerNode="emitestbed23.cnaf.infn.it"; ClientJobId="585808873"; BlahJobName="cream_585808873";]
2012-05-03 15:18:14 Sent for Cream:[BatchJobId="268.emi-demo13.cnaf.infn.it"; JobStatus=3; ChangeTime="2012-05-03 15:17:31"; WorkerNode="emitestbed23.cnaf.infn.it"; JwExitCode=-999; Reason="reason=-999"; ClientJobId="585808873"; BlahJobName="cream_585808873";]

For suspended and resumed jobs:

On UI:

$ glite-ce-job-status https://emi-demo13.cnaf.infn.it:8443/CREAM641304895

******  JobID=[https://emi-demo13.cnaf.infn.it:8443/CREAM641304895]
        Status        = [IDLE]

$ glite-ce-job-suspend https://emi-demo13.cnaf.infn.it:8443/CREAM641304895

Are you sure you want to suspend specified job(s) [y/n]: y
[bertocco@emitestbed08 blah_testing]$ glite-ce-job-status https://emi-demo13.cnaf.infn.it:8443/CREAM641304895

******  JobID=[https://emi-demo13.cnaf.infn.it:8443/CREAM641304895]
        Status        = [HELD]


$ glite-ce-job-resume https://emi-demo13.cnaf.infn.it:8443/CREAM641304895

Are you sure you want to resume specified job(s) [y/n]: y
[bertocco@emitestbed08 blah_testing]$ glite-ce-job-status https://emi-demo13.cnaf.infn.it:8443/CREAM641304895

******  JobID=[https://emi-demo13.cnaf.infn.it:8443/CREAM641304895]
        Status        = [HELD]


$ glite-ce-job-status https://emi-demo13.cnaf.infn.it:8443/CREAM641304895

******  JobID=[https://emi-demo13.cnaf.infn.it:8443/CREAM641304895]
        Status        = [IDLE]

$ glite-ce-job-status https://emi-demo13.cnaf.infn.it:8443/CREAM641304895

******  JobID=[https://emi-demo13.cnaf.infn.it:8443/CREAM641304895]
        Status        = [DONE-OK]
        ExitCode      = [0]

On CE CREAM:

# grep 641304895 /var/log/cream/glite-ce-bnotifier.log
[BatchJobId="291.emi-demo13.cnaf.infn.it"; JobStatus=1; ChangeTime="2012-05-03 15:54:46"; ClientJobId="641304895"; BlahJobName="cream_641304895";]
2012-05-03 15:56:14 Sent for Cream:[BatchJobId="291.emi-demo13.cnaf.infn.it"; JobStatus=5; ChangeTime="2012-05-03 15:56:13"; ClientJobId="641304895"; BlahJobName="cream_641304895";]
[BatchJobId="291.emi-demo13.cnaf.infn.it"; JobStatus=1; ChangeTime="2012-05-03 15:57:16"; ClientJobId="641304895"; BlahJobName="cream_641304895";]
[BatchJobId="291.emi-demo13.cnaf.infn.it"; JobStatus=2; ChangeTime="2012-05-03 16:03:36"; WorkerNode="emitestbed23.cnaf.infn.it"; ClientJobId="641304895"; BlahJobName="cream_641304895";]
[BatchJobId="291.emi-demo13.cnaf.infn.it"; JobStatus=4; ChangeTime="2012-05-03 16:04:36"; WorkerNode="emitestbed23.cnaf.infn.it"; JwExitCode=0; Reason="reason=0"; ClientJobId="641304895"; BlahJobName="cream_641304895";]

Check Notifications SL5 - PBS Old parser Not Tested

Check Notifications SL6 - LSF New parser Not Tested

Check Notifications SL6 - LSF Old parser Not Tested

Noramally finished jobs Passed ---+++++ Check Notifications SL6 - PBS New parser PASSED

For normally finished jobs

ON UI:

$ glite-ce-job-status https://emitestbed29.cnaf.infn.it:8443/CREAM400857385

******  JobID=[https://emitestbed29.cnaf.infn.it:8443/CREAM400857385]
   Status        = [REALLY-RUNNING]

$ glite-ce-job-status https://emitestbed29.cnaf.infn.it:8443/CREAM400857385

******  JobID=[https://emitestbed29.cnaf.infn.it:8443/CREAM400857385]
   Status        = [DONE-OK]
   ExitCode      = [0]

ON CE CREAM:

# grep 400857385 /var/log/cream/glite-ce-bnotifier.log
2012-05-03 16:38:46 Sent for Cream:[BatchJobId="36.emitestbed29.cnaf.infn.it"; JobStatus=2; ChangeTime="2012-05-03 16:38:22"; WorkerNode="emitestbed28.cnaf.infn.it"; ClientJobId="400857385"; BlahJobName="cream_400857385";]
2012-05-03 16:39:46 Sent for Cream:[BatchJobId="36.emitestbed29.cnaf.infn.it"; JobStatus=4; ChangeTime="2012-05-03 16:39:22"; WorkerNode="emitestbed28.cnaf.infn.it"; JwExitCode=0; Reason="reason=0"; ClientJobId="400857385"; BlahJobName="cream_400857385";]

For cancelled jobs:

ON UI:

date; glite-ce-job-submit  -a -r emitestbed29.cnaf.infn.it:8443/cream-pbs-demo  -a /home/bertocco/test1.jdl
Thu May  3 16:42:28 CEST 2012
https://emitestbed29.cnaf.infn.it:8443/CREAM060970469
$ glite-ce-job-status https://emitestbed29.cnaf.infn.it:8443/CREAM060970469

******  JobID=[https://emitestbed29.cnaf.infn.it:8443/CREAM060970469]
   Status        = [REALLY-RUNNING]


$ glite-ce-job-cancel https://emitestbed29.cnaf.infn.it:8443/CREAM060970469

Are you sure you want to cancel specified job(s) [y/n]: y
$ glite-ce-job-status https://emitestbed29.cnaf.infn.it:8443/CREAM060970469

******  JobID=[https://emitestbed29.cnaf.infn.it:8443/CREAM060970469]
   Status        = [CANCELLED]
   ExitCode      = []
   Description   = [Cancelled by user]

On CE CREAM:

# grep 060970469 /var/log/cream/glite-ce-bnotifier.log
2012-05-03 16:42:46 Sent for Cream:[BatchJobId="37.emitestbed29.cnaf.infn.it"; JobStatus=3; ChangeTime="2012-05-03 16:42:42"; WorkerNode="emitestbed28.cnaf.infn.it"; JwExitCode=-999; Reason="reason=-999"; ClientJobId="060970469"; BlahJobName="cream_060970469";]

For suspended and resumed jobs:

On UI:

# glite-ce-job-submit  -a -r emitestbed29.cnaf.infn.it:8443/cream-pbs-demo  -a /home/bertocco/test1.jdl
https://emitestbed29.cnaf.infn.it:8443/CREAM535518208

$ glite-ce-job-status https://emitestbed29.cnaf.infn.it:8443/CREAM535518208

******  JobID=[https://emitestbed29.cnaf.infn.it:8443/CREAM535518208]
   Status        = [IDLE]


$ glite-ce-job-suspend https://emitestbed29.cnaf.infn.it:8443/CREAM535518208

Are you sure you want to suspend specified job(s) [y/n]: y

$ glite-ce-job-status https://emitestbed29.cnaf.infn.it:8443/CREAM535518208

******  JobID=[https://emitestbed29.cnaf.infn.it:8443/CREAM535518208]
   Status        = [HELD]


$ glite-ce-job-resume https://emitestbed29.cnaf.infn.it:8443/CREAM535518208

Are you sure you want to resume specified job(s) [y/n]: y

$ glite-ce-job-status https://emitestbed29.cnaf.infn.it:8443/CREAM535518208

******  JobID=[https://emitestbed29.cnaf.infn.it:8443/CREAM535518208]
   Status        = [DONE-OK]
   ExitCode      = [0]

On CE CREAM:

grep 535518208 /var/log/cream/glite-ce-bnotifier.log
[BatchJobId="40.emitestbed29.cnaf.infn.it"; JobStatus=1; ChangeTime="2012-05-03 17:00:09"; ClientJobId="535518208"; BlahJobName="cream_535518208";]
2012-05-03 17:00:46 Sent for Cream:[BatchJobId="40.emitestbed29.cnaf.infn.it"; JobStatus=5; ChangeTime="2012-05-03 17:00:27"; ClientJobId="535518208"; BlahJobName="cream_535518208";]
2012-05-03 17:01:06 Sent for Cream:[BatchJobId="40.emitestbed29.cnaf.infn.it"; JobStatus=2; ChangeTime="2012-05-03 17:01:05"; WorkerNode="emitestbed28.cnaf.infn.it"; ClientJobId="535518208"; BlahJobName="cream_535518208";]
[BatchJobId="40.emitestbed29.cnaf.infn.it"; JobStatus=2; ChangeTime="2012-05-03 17:01:10"; WorkerNode="emitestbed28.cnaf.infn.it"; ClientJobId="535518208"; BlahJobName="cream_535518208";]
2012-05-03 17:02:46 Sent for Cream:[BatchJobId="40.emitestbed29.cnaf.infn.it"; JobStatus=4; ChangeTime="2012-05-03 17:02:10"; WorkerNode="emitestbed28.cnaf.infn.it"; JwExitCode=0; Reason="reason=0"; ClientJobId="535518208"; BlahJobName="cream_535518208";]

Check Notifications SL6 - PBS Old parser Not Tested

Regression tests

Note: All bug tests are performed on CREAM EMI 2 SL5 old and new parser. Bugs signed with 'NA' are not certified because difficult to reproduce.

Verification attached bugs

bug #86238: blahpd doesn't check the status of its daemons when idling FIXED

==============================================================================
Bug 86238 :: Regression test of bug #86238 blahpd doesn't check the status ...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
bug_86238                                                             | PASS |
------------------------------------------------------------------------------
Bug 86238 :: Regression test of bug #86238 blahpd doesn't check th... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================

bug #86918: Request for passing all submit command attributes to the local configuration script. FIXED

On CREAM

cat /tmp/filewithenv
MANPATH=/opt/glite/share/man::
ceid=emi-demo13.cnaf.infn.it:8443/cream-pbs-demo
HOSTNAME=emi-demo13.cnaf.infn.it
TERM=vanilla
SHELL=/bin/sh
GRID_ENV_LOCATION=/usr/libexec
HISTSIZE=1000
CATALINA_HOME=/usr/share/tomcat5
BLAHPD_CONFIG_LOCATION=/etc/blah.config
ClientJobId=cream_483887197
X509_CERT_DIR=/etc/grid-security/certificates/
GRIDMAPDIR=/etc/grid-security/gridmapdir
USER=tst22
LS_COLORS=
SUDO_USER=tomcat
SUDO_UID=91
LCG_LOCATION=/usr
USERNAME=tst22
MAIL=/var/spool/mail/tst22
PATH=/bin:/usr/bin:/home/tst22/bin
GLITE_HOST_KEY=/home/glite/.certs/hostkey.pem
uniquejobid=CREAM483887197
INPUTRC=/etc/inputrc
PWD=/var/tmp
LANG=en_US.UTF-8
GLITE_USER=glite
gridType=pbs
x509UserProxySubject=CN=Sara Bertocco,L=Padova,OU=Personal Certificate,O=INFN,C=IT
BLAHPD_LOCATION=/usr
HOME=/home/tst22
SUDO_COMMAND=/usr/libexec/pbs_submit.sh -x /var/cream_sandbox/testers/CN_Sara_Bertocco_L_Padova_OU_Personal_Certificate_O_INFN_C_IT_testers_eu_emi_eu_Role_NULL_Capability_NULL_tst22/proxy/c7b5086871f50a68b630e3eca03499a17a0eec4c_12953230182596 -u CN=Sara Bertocco,L=Padova,OU=Personal Certificate,O=INFN,C=IT -r no -c /var/cream_sandbox/testers/CN_Sara_Bertocco_L_Padova_OU_Personal_Certificate_O_INFN_C_IT_testers_eu_emi_eu_Role_NULL_Capability_NULL_tst22/48/CREAM483887197/CREAM483887197_jobWrapper.sh -T /tmp -C /tmp/ce-req-file-133614658720016 -o /var/cream_sandbox/testers/CN_Sara_Bertocco_L_Padova_OU_Personal_Certificate_O_INFN_C_IT_testers_eu_emi_eu_Role_NULL_Capability_NULL_tst22/48/CREAM483887197/StandardOutput -e /var/cream_sandbox/testers/CN_Sara_Bertocco_L_Padova_OU_Personal_Certificate_O_INFN_C_IT_testers_eu_emi_eu_Role_NULL_Capability_NULL_tst22/48/CREAM483887197/StandardError -q demo -n 1 -j cream_483887197
SHLVL=2
GLITE_LOCATION_VAR=/var
GLOBUS_TCP_PORT_RANGE=20000,25000
VirtualOrganisation=testers.eu-emi.eu
LOGNAME=tst22
queue=demo
LESSOPEN=|/usr/bin/lesspipe.sh %s
x509UserProxyFQAN="userFQAN=/testers.eu-emi.eu/Role=NULL/Capability=NULL"
SUDO_GID=91
GLITE_LOCATION=/usr
GLITE_HOST_CERT=/home/glite/.certs/hostcert.pem
G_BROKEN_FILENAMES=1
_=/bin/env

# cat /tmp/filewithenv |grep x509UserProxySubject 
x509UserProxySubject=CN=Sara Bertocco,L=Padova,OU=Personal Certificate,O=INFN,C=IT
[root@emi-demo13 ~]# cat /tmp/filewithenv |grep  x509UserProxyFQAN
x509UserProxyFQAN="userFQAN=/testers.eu-emi.eu/Role=NULL/Capability=NULL"

bug #89504: Repeated notification problem for BLParserLSF NA

bug #90082: BUpdaterPBS workaround if tracejob is in infinite loop NA

bug #90101: Missing 'Iwd' Attribute when trasferring files with the 'TransferInput' attribute may cause thread to loop NA

bug #91318: Request to change functions in blah_common_submit_functions.sh FIXED

Verifyed, in /usr/libexec/blah_common_submit_functions.sh the requested rows are present:

    748 function bls_add_job_wrapper ()
    749 {
    750   bls_start_job_wrapper >> $bls_tmp_file
    751   bls_finish_job_wrapper >> $bls_tmp_file
    752   bls_test_working_dir
    753 }

bug #92554: BNotifier problem can leave connection in CLOSE_WAIT state NA

bug #84261:BNotifier on CREAM CE seems to not restart cleanly FIXED

==============================================================================
Bug 84261 :: Regression test of bug #84261 BNotifier on CREAM CE seems to n...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
bug_84261                                                             | PASS |
------------------------------------------------------------------------------
Bug 84261 :: Regression test of bug #84261 BNotifier on CREAM CE s... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================

Report

Verification old bugs

bug #81824: yaim-cream-ce should manage the attribute bupdater_loop_interval FIXED

==============================================================================
Bug 81824 :: Regression test of bug #81824 yaim-cream-ce should manage the ...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
bug_81824                                                             | PASS |
------------------------------------------------------------------------------
Bug 81824 :: Regression test of bug #81824 yaim-cream-ce should ma... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================