Testing report: IGIRTC-133

Summary

• Product: CREAM v. 1.15.1
• Release Task: Task #34043
• ETICS Subsystem Configuration Name:
• VCS Tag:
• EMI Major Release: EMI 3 (Monte Bianco)
• Platform: SL5 epel
• Author: Sara Bertocco
• Testing report: [[][Testing Report file]]
• Certification report: [[][Certification Report file]]
• Outcome: "In Certification*

Deployment tests

Clean Installation

CREAM PBS PASSED

• Installation log file
• Configuration log file

CREAM LSF PASSED

• Installation log file
• Configuration log file

Upgrade Installation

CREAM PBS PASSED

• Update log file
• Configuration log file

CREAM LSF PASSED

• Update log file
• Configuration log file

Unit Tests

System tests

Functionality tests

CREAM PBS PASSED

The "Verify Job Status Output" test fails because a bug in the UI EMI 2 used to launch the testsuite. Other tests are excluded for configuration issues.

$  pybot  -e bdii -e glue -e job_man -e old_blah  /opt/cream_test/testsuite/cream_test_suite.html
INFO: The files of this testsuite will be stored under: /tmp/tmpMzx6BW.cream_testing/
INFO: Non-mandatory variable "sec_cert" has not been set.
INFO: Non-mandatory variable "sec_key" has not been set.
INFO: Non-mandatory variable "argus_host" has not been set.
INFO: Non-mandatory variable "sec_proxy_pass" has not been set.
==============================================================================
Cream Test Suite :: This is the main testing module,needed for testing the ...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
Check Executables :: Check whether the needed executables for the ... | PASS |
------------------------------------------------------------------------------
Simple Submit :: Execute /bin/uname -a.Submit the jdl and wait for... | PASS |
------------------------------------------------------------------------------
ISB Base URI :: Execute a bash shell script.The script is stored i... | PASS |
------------------------------------------------------------------------------
ISB Client to CE :: Execute a bash shell script.The script is stor... | PASS |
------------------------------------------------------------------------------
ISB gsiftp to CE :: Execute a bash shell script.The script is stor... | PASS |
------------------------------------------------------------------------------
OSB localhost :: Execute /bin/uname -a.Store std out and error str... | PASS |
------------------------------------------------------------------------------
OSB gsiftp BaseDestURI :: Execute /bin/uname -a.Store std out and ... | PASS |
------------------------------------------------------------------------------
OSB gsiftp DestURI :: Execute /bin/uname -a.Store std out and erro... | PASS |
------------------------------------------------------------------------------
Environment :: Execute a bash shell script.The script is stored lo... | PASS |
------------------------------------------------------------------------------
Epilogue :: Execute two bash shell scripts,one for the job and the... | PASS |
------------------------------------------------------------------------------
Prologue :: Execute two bash shell scripts,one for the job and the... | PASS |
------------------------------------------------------------------------------
Explicit Delegation :: Test job submission with explicit delegatio... | PASS |
------------------------------------------------------------------------------
Job Cancel :: Test job cancellation.Two cases are tested:             | PASS |
------------------------------------------------------------------------------
Job List :: Test the job listing cli utility.Store the list of the... | PASS |
------------------------------------------------------------------------------
Job Suspend - Resume :: Submit a job and suspend it.Wait until it'... | PASS |
------------------------------------------------------------------------------
Manually Cancel Job :: Submit a job and then manually cancel it on... | PASS |
------------------------------------------------------------------------------
Verify CE service info :: This test case verifies the output of th... | PASS |
------------------------------------------------------------------------------
Enable - Disable Submission :: This test case tests the proper fun... | PASS |
------------------------------------------------------------------------------
Job Purge :: This test case verifies the proper functionality of t... | PASS |
------------------------------------------------------------------------------
Limiter :: NOTE: This test case is about 25 minutes long!             | PASS |
------------------------------------------------------------------------------
Ban User :: This test case tests if the banning of user works. The... | PASS |
------------------------------------------------------------------------------
Sandbox Staging :: This test case tests the two different Sandbox ... | PASS |
------------------------------------------------------------------------------
Automatic Proxy Purging :: This test case checks the automatic pro... | PASS |
------------------------------------------------------------------------------
Verify Job Status Output :: This test is checking the output of th... | FAIL |
'['https://cert-42.pd.infn.it:8443/CREAM039112575', 'CANCELLED', 'https://cert-42.pd.infn.it:8443/CREAM096873115', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM096922256', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM130270327', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM142219948', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM144042097', 'CANCELLED', 'https://cert-42.pd.infn.it:8443/CREAM144489913', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM151509511', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM216440424', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM242371973', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM351372075', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM371767205', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM377678364', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM411128309', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM514181581', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM526025806', 'REGISTERED', 'https://cert-42.pd.infn.it:8443/CREAM531337500', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM546215002', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM548918653', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM619094711', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM627130670', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM659313536', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM713806828', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM759669997', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM860045672', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM862460658', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM885876820', 'DONE-OK']' does not contain 'https://cert-42.pd.infn.it:8443/CREAM694842335'
------------------------------------------------------------------------------
OutputData :: This test case tests the OutputData jdl attribute. T... | PASS |
------------------------------------------------------------------------------
Automatic Job Purge :: This test case is testing the functionality... | PASS |
------------------------------------------------------------------------------
Proxy Renewal :: Submit a job lasting 20 minutes using a proxy val... | PASS |
------------------------------------------------------------------------------
CPU Allocation 1 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 2 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 3 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 4 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 5 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 6 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
Logging and Bookkeeping :: Check if the logging and bookkeeping se... | PASS |
------------------------------------------------------------------------------
Cream Test Suite :: This is the main testing module,needed for tes... | FAIL |
35 critical tests, 34 passed, 1 failed
35 tests total, 34 passed, 1 failed
==============================================================================
Output:  /home/bertocco/output.xml
Log:     /home/bertocco/log.html
Report:  /home/bertocco/report.html

• Functionality tests log file

CREAM LSF PASSED

• job_man excluded because requires two proxyes which the certifier has not.
• old_blah excluded because the ce is configured with new blah
• OutputData usage check fails for configuration issues
• Logging and Bookkeeping check fails for UI installation issues.

$ /usr/bin/pybot  -e bdii -e glue -e job_man -e old_blah  /opt/cream_test/testsuite/cream_test_suite.html
INFO: The files of this testsuite will be stored under: /tmp/tmp1fA8m_.cream_testing/
INFO: Non-mandatory variable "sec_cert" has not been set.
INFO: Non-mandatory variable "sec_key" has not been set.
INFO: Non-mandatory variable "argus_host" has not been set.
INFO: Non-mandatory variable "sec_proxy_pass" has not been set.
INFO: Non-mandatory variable "argus_root_pass" has not been set.
==============================================================================
Cream Test Suite :: This is the main testing module,needed for testing the ...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
Check Executables :: Check whether the needed executables for the ... | PASS |
------------------------------------------------------------------------------
Simple Submit :: Execute /bin/uname -a.Submit the jdl and wait for... | PASS |
------------------------------------------------------------------------------
ISB Base URI :: Execute a bash shell script.The script is stored i... | PASS |
------------------------------------------------------------------------------
ISB Client to CE :: Execute a bash shell script.The script is stor... | PASS |
------------------------------------------------------------------------------
ISB gsiftp to CE :: Execute a bash shell script.The script is stor... | PASS |
------------------------------------------------------------------------------
OSB localhost :: Execute /bin/uname -a.Store std out and error str... | PASS |
------------------------------------------------------------------------------
OSB gsiftp BaseDestURI :: Execute /bin/uname -a.Store std out and ... | PASS |
------------------------------------------------------------------------------
OSB gsiftp DestURI :: Execute /bin/uname -a.Store std out and erro... | PASS |
------------------------------------------------------------------------------
Environment :: Execute a bash shell script.The script is stored lo... | PASS |
------------------------------------------------------------------------------
Epilogue :: Execute two bash shell scripts,one for the job and the... | PASS |
------------------------------------------------------------------------------
Prologue :: Execute two bash shell scripts,one for the job and the... | PASS |
------------------------------------------------------------------------------
Explicit Delegation :: Test job submission with explicit delegatio... | PASS |
------------------------------------------------------------------------------
Job Cancel :: Test job cancellation.Two cases are tested:             | PASS |
------------------------------------------------------------------------------
Job List :: Test the job listing cli utility.Store the list of the... | PASS |
------------------------------------------------------------------------------
Job Suspend - Resume :: Submit a job and suspend it.Wait until it'... | PASS |
------------------------------------------------------------------------------
Manually Cancel Job :: Submit a job and then manually cancel it on... | PASS |
------------------------------------------------------------------------------
Verify CE service info :: This test case verifies the output of th... | PASS |
------------------------------------------------------------------------------
Enable - Disable Submission :: This test case tests the proper fun... | PASS |
------------------------------------------------------------------------------
Job Purge :: This test case verifies the proper functionality of t... | PASS |
------------------------------------------------------------------------------
Limiter :: NOTE: This test case is about 25 minutes long!             | PASS |
------------------------------------------------------------------------------
Ban User :: This test case tests if the banning of user works. The... | PASS |
------------------------------------------------------------------------------
Sandbox Staging :: This test case tests the two different Sandbox ... | PASS |
------------------------------------------------------------------------------
Automatic Proxy Purging :: This test case checks the automatic pro... | PASS |
------------------------------------------------------------------------------
Verify Job Status Output :: This test is checking the output of th... | PASS |
------------------------------------------------------------------------------
OutputData :: This test case tests the OutputData jdl attribute. T... | FAIL |
'[]' should not be empty
------------------------------------------------------------------------------
Automatic Job Purge :: This test case is testing the functionality... | PASS |
------------------------------------------------------------------------------
Proxy Renewal :: Submit a job lasting 20 minutes using a proxy val... | PASS |
------------------------------------------------------------------------------
CPU Allocation 1 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 2 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 3 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 4 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 5 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 6 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
Logging and Bookkeeping :: Check if the logging and bookkeeping se... | FAIL |
_error: WMS Job status polling failed with return value: 1
Command reported: Traceback (most recent call last):
,  File "/usr/bin/glite-wms-job-status", line 53, in <module>
,    import wmsui_utils
,ImportError: No module named wmsui_utils
------------------------------------------------------------------------------
Cream Test Suite :: This is the main testing module,needed for tes... | FAIL |
35 critical tests, 33 passed, 2 failed
35 tests total, 33 passed, 2 failed
==============================================================================
Output:  /home/bertocco/output.xml
Log:     /home/bertocco/log.html
Report:  /home/bertocco/report.html

• Functionality tests log file

$ /usr/bin/pybot  -i glue  /opt/cream_test/testsuite/cream_test_suite.html
INFO: The files of this testsuite will be stored under: /tmp/tmp48uh_f.cream_testing/
INFO: Non-mandatory variable "sec_cert" has not been set.
INFO: Non-mandatory variable "sec_key" has not been set.
INFO: Non-mandatory variable "argus_host" has not been set.
INFO: Non-mandatory variable "sec_proxy_pass" has not been set.
INFO: Non-mandatory variable "argus_root_pass" has not been set.
==============================================================================
Cream Test Suite :: This is the main testing module,needed for testing the ...
==============================================================================
Validate Glue Schema :: Run the glue validator on the cream host.     | PASS |
------------------------------------------------------------------------------
Cream Test Suite :: This is the main testing module,needed for tes... | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
Output:  /home/bertocco/output.xml
Log:     /home/bertocco/log.html
Report:  /home/bertocco/report.html

• Validate glue schema test log

$ /usr/bin/pybot  -i bdii  /opt/cream_test/testsuite/cream_test_suite.html
INFO: The files of this testsuite will be stored under: /tmp/tmpWmVNrW.cream_testing/
INFO: Non-mandatory variable "sec_cert" has not been set.
INFO: Non-mandatory variable "sec_key" has not been set.
INFO: Non-mandatory variable "argus_host" has not been set.
INFO: Non-mandatory variable "sec_proxy_pass" has not been set.
INFO: Non-mandatory variable "argus_root_pass" has not been set.
==============================================================================
Cream Test Suite :: This is the main testing module,needed for testing the ...
==============================================================================
Resource BDII :: Check if the resource bdii publishes all the rele... | PASS |
------------------------------------------------------------------------------
Cream Test Suite :: This is the main testing module,needed for tes... | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
Output:  /home/bertocco/output.xml
Log:     /home/bertocco/log.html
Report:  /home/bertocco/report.html

• Check resource bdii test log

Standard conformance tests

Glue 1 Compliance

LSF FAILED ?

• glue-validator -t glue1 -h cert-41.pd.infn.it -p 2170 -b o=grid 

  log

PBS FAILED ?

• glue-validator -t glue1 -h cert-42.pd.infn.it -p 2170 -b o=grid

  log

Glue 2 Compliance

LSF PASSED

• glue-validator -t glue2 -h cert-41.pd.infn.it -p 2170 -b o=glue 

  log

PBS PASSED

• glue-validator -t glue2 -h cert-42.pd.infn.it -p 2170 -b o=glue 

  log

Performance and scalability tests

LSF PASSED

• ./bin/cream-test-monitored-submit -r 30 -n 1000 -m 50 -C 50 -l log4py.conf -j jdl/testsandbox.jdl -R cert-41.pd.infn.it:8443/cream-lsf-cert

  log

• ./bin/cream-test-monitored-cancel -r 30 -n 1000 -m 50 -C 50 -l log4py.conf -j jdl/simple.jdl -R cert-41.pd.infn.it:8443/cream-lsf-cert

  log

• ./bin/cream-test-monitored-lease-updated -r 60 -n 50 -m 10 -C 10 -l log4py.conf -W 1200 --vo dteam --valid 00:20 -R cert-41.pd.infn.it:8443/cream-lsf-cert
  

  log

PBS NA

Integration testing

Tests described:

• https://twiki.cern.ch/twiki/bin/view/EMI/EmiJra1TaskForceIntegrationTesting
  • test 2: "CEMon to use Argus for authorization "
    • NA
  • test 3: "CREAM jobs executed under different backends is properly treated by APEL client "
    • PASSED - proven by functionality tests of different CREAM + LSF, TORQUE, SGE modules
  • test 16: "Do authorization filtering based on VOMS attributes and ARGUS authorization" * NA

-- SaraBertocco - 2013-03-04