Tags:
, view all tags

Testing report: IGIRTC-125

Summary

  • Product: CREAM v. 1.14.3
  • Release Task: Task #38531
  • ETICS Subsystem Configuration Name:
  • VCS Tag:
  • EMI Major Release: EMI 2 (Matterhorn)
  • Platform: SL6 epel
  • Author: Sara Bertocco
  • Testing report: [[][Testing Report file]]
  • Certification report: [[][Certification Report file]]
  • Outcome: "In Certification*

Deployment tests

Clean Installation

CREAM PBS

CREAM LSF

Upgrade Installation

CREAM PBS

    • [[][Update log file]]
    • [[][Configuration log file]]

CREAM LSF

    • [[][Update log file]]
    • [[][Configuration log file]]

Unit Tests

System tests

The tests are performed with two different user interfaces:

  • tests of CREAM LSF use UI EMI version 1.11.0-1
  • tests of CREAM PBS use UI EMI version 2.5.1-1
this is the reason why the test "Verify Job Status Output" fails in the pbs version: it is a UI problem.

Functionality tests

  • Test "bdii" and "glue" are covered in compliance tests
  • Test "job_man" is excluded because it requires two proxy certificates and the certifier has one only
  • Test "outputdata" is excluded in the first test suite run because a problem in testbed bdii setup, solved later.

CREAM PBS

$ pybot -e outputdata -e bdii -e glue -e job_man /opt/cream_test/testsuite/cream_test_suite.html
INFO: The files of this testsuite will be stored under: /tmp/tmpMSXELI.cream_testing/
INFO: Non-mandatory variable "sec_cert" has not been set.
INFO: Non-mandatory variable "sec_key" has not been set.
INFO: Non-mandatory variable "argus_host" has not been set.
INFO: Non-mandatory variable "sec_proxy_pass" has not been set.
==============================================================================
Cream Test Suite :: This is the main testing module,needed for testing the ...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
Check Executables :: Check whether the needed executables for the ... | PASS |
------------------------------------------------------------------------------
Simple Submit :: Execute /bin/uname -a.Submit the jdl and wait for... | PASS |
------------------------------------------------------------------------------
ISB Base URI :: Execute a bash shell script.The script is stored i... | PASS |
------------------------------------------------------------------------------
ISB Client to CE :: Execute a bash shell script.The script is stor... | PASS |
------------------------------------------------------------------------------
ISB gsiftp to CE :: Execute a bash shell script.The script is stor... | PASS |
------------------------------------------------------------------------------
OSB localhost :: Execute /bin/uname -a.Store std out and error str... | PASS |
------------------------------------------------------------------------------
OSB gsiftp BaseDestURI :: Execute /bin/uname -a.Store std out and ... | PASS |
------------------------------------------------------------------------------
OSB gsiftp DestURI :: Execute /bin/uname -a.Store std out and erro... | PASS |
------------------------------------------------------------------------------
Environment :: Execute a bash shell script.The script is stored lo... | PASS |
------------------------------------------------------------------------------
Epilogue :: Execute two bash shell scripts,one for the job and the... | PASS |
------------------------------------------------------------------------------
Prologue :: Execute two bash shell scripts,one for the job and the... | PASS |
------------------------------------------------------------------------------
Explicit Delegation :: Test job submission with explicit delegatio... | PASS |
------------------------------------------------------------------------------
Job Cancel :: Test job cancellation.Two cases are tested:             | PASS |
------------------------------------------------------------------------------
Job List :: Test the job listing cli utility.Store the list of the... | PASS |
------------------------------------------------------------------------------
Job Suspend - Resume :: Submit a job and suspend it.Wait until it'... | PASS |
------------------------------------------------------------------------------
Manually Cancel Job :: Submit a job and then manually cancel it on... | PASS |
------------------------------------------------------------------------------
Verify CE service info :: This test case verifies the output of th... | PASS |
------------------------------------------------------------------------------
Enable - Disable Submission :: This test case tests the proper fun... | PASS |
------------------------------------------------------------------------------
Job Purge :: This test case verifies the proper functionality of t... | PASS |
------------------------------------------------------------------------------
Limiter :: NOTE: This test case is about 25 minutes long!             | PASS |
------------------------------------------------------------------------------
Ban User :: This test case tests if the banning of user works. The... | PASS |
------------------------------------------------------------------------------
Sandbox Staging :: This test case tests the two different Sandbox ... | PASS |
------------------------------------------------------------------------------
Automatic Proxy Purging :: This test case checks the automatic pro... | PASS |
------------------------------------------------------------------------------
Verify Job Status Output :: This test is checking the output of th... | FAIL |
'['https://cert-42.pd.infn.it:8443/CREAM008747298', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM092671703', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM124860354', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM190619285', 'IDLE', 'https://cert-42.pd.infn.it:8443/CREAM209313410', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM216599452', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM247509604', 'CANCELLED', 'https://cert-42.pd.infn.it:8443/CREAM298811348', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM338616186', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM397680813', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM409140727', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM418737077', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM438966721', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM478916637', 'IDLE', 'https://cert-42.pd.infn.it:8443/CREAM483549371', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM504522990', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM588399968', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM621620940', 'CANCELLED', 'https://cert-42.pd.infn.it:8443/CREAM658041789', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM722241788', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM803417493', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM805989025', 'CANCELLED', 'https://cert-42.pd.infn.it:8443/CREAM829770983', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM856980366', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM906476021', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM932057863', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM973629738', 'DONE-OK']' does not contain 'https://cert-42.pd.infn.it:8443/CREAM061880031'
------------------------------------------------------------------------------
Automatic Job Purge :: This test case is testing the functionality... | PASS |
------------------------------------------------------------------------------
Proxy Renewal :: Submit a job lasting 20 minutes using a proxy val... | PASS |
------------------------------------------------------------------------------
CPU Allocation 1 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 2 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 3 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 4 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 5 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 6 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
Logging and Bookkeeping :: Check if the logging and bookkeeping se... | PASS |
------------------------------------------------------------------------------
Old BLParser Restart :: Configure a CREAM CE using the old blah bl... | PASS |
------------------------------------------------------------------------------
Cream Test Suite :: This is the main testing module,needed for tes... | FAIL |
35 critical tests, 34 passed, 1 failed
35 tests total, 34 passed, 1 failed
==============================================================================

$ pybot -i outputdata  /opt/cream_test/testsuite/cream_test_suite.html
.......
==============================================================================
Cream Test Suite :: This is the main testing module,needed for testing the ...
==============================================================================
OutputData :: This test case tests the OutputData jdl attribute. T... | PASS |
------------------------------------------------------------------------------
Cream Test Suite :: This is the main testing module,needed for tes... | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================

  • Functionality test suite log
  • Test "outputdata" log

CREAM LSF

$ pybot -e outputdata -e bdii -e glue -e job_man /opt/cream_test/testsuite/cream_test_suite.html
INFO: The files of this testsuite will be stored under: /tmp/tmplKbo0y.cream_testing/
INFO: Non-mandatory variable "sec_cert" has not been set.
INFO: Non-mandatory variable "sec_key" has not been set.
INFO: Non-mandatory variable "argus_host" has not been set.
INFO: Non-mandatory variable "sec_proxy_pass" has not been set.
INFO: Non-mandatory variable "argus_root_pass" has not been set.
==============================================================================
Cream Test Suite :: This is the main testing module,needed for testing the ...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
Check Executables :: Check whether the needed executables for the ... | PASS |
------------------------------------------------------------------------------
Simple Submit :: Execute /bin/uname -a.Submit the jdl and wait for... | PASS |
------------------------------------------------------------------------------
ISB Base URI :: Execute a bash shell script.The script is stored i... | PASS |
------------------------------------------------------------------------------
ISB Client to CE :: Execute a bash shell script.The script is stor... | PASS |
------------------------------------------------------------------------------
ISB gsiftp to CE :: Execute a bash shell script.The script is stor... | PASS |
------------------------------------------------------------------------------
OSB localhost :: Execute /bin/uname -a.Store std out and error str... | PASS |
------------------------------------------------------------------------------
OSB gsiftp BaseDestURI :: Execute /bin/uname -a.Store std out and ... | PASS |
------------------------------------------------------------------------------
OSB gsiftp DestURI :: Execute /bin/uname -a.Store std out and erro... | PASS |
------------------------------------------------------------------------------
Environment :: Execute a bash shell script.The script is stored lo... | PASS |
------------------------------------------------------------------------------
Epilogue :: Execute two bash shell scripts,one for the job and the... | PASS |
------------------------------------------------------------------------------
Prologue :: Execute two bash shell scripts,one for the job and the... | PASS |
------------------------------------------------------------------------------
Explicit Delegation :: Test job submission with explicit delegatio... | PASS |
------------------------------------------------------------------------------
Job Cancel :: Test job cancellation.Two cases are tested:             | PASS |
------------------------------------------------------------------------------
Job List :: Test the job listing cli utility.Store the list of the... | PASS |
------------------------------------------------------------------------------
Job Suspend - Resume :: Submit a job and suspend it.Wait until it'... | PASS |
------------------------------------------------------------------------------
Manually Cancel Job :: Submit a job and then manually cancel it on... | PASS |
------------------------------------------------------------------------------
Verify CE service info :: This test case verifies the output of th... | PASS |
------------------------------------------------------------------------------
Enable - Disable Submission :: This test case tests the proper fun... | PASS |
------------------------------------------------------------------------------
Job Purge :: This test case verifies the proper functionality of t... | PASS |
------------------------------------------------------------------------------
Limiter :: NOTE: This test case is about 25 minutes long!             | PASS |
------------------------------------------------------------------------------
Ban User :: This test case tests if the banning of user works. The... | PASS |
------------------------------------------------------------------------------
Sandbox Staging :: This test case tests the two different Sandbox ... | FAIL |
IOError: [Errno 2] No such file
------------------------------------------------------------------------------
Automatic Proxy Purging :: This test case checks the automatic pro... | PASS |
------------------------------------------------------------------------------
Verify Job Status Output :: This test is checking the output of th... | PASS |
------------------------------------------------------------------------------
Automatic Job Purge :: This test case is testing the functionality... | PASS |
------------------------------------------------------------------------------
Proxy Renewal :: Submit a job lasting 20 minutes using a proxy val... | PASS |
------------------------------------------------------------------------------
CPU Allocation 1 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 2 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 3 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 4 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 5 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 6 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
Logging and Bookkeeping :: Check if the logging and bookkeeping se... | PASS |
------------------------------------------------------------------------------
Old BLParser Restart :: Configure a CREAM CE using the old blah bl... | PASS |
------------------------------------------------------------------------------
Cream Test Suite :: This is the main testing module,needed for tes... | FAIL |
35 critical tests, 34 passed, 1 failed
35 tests total, 34 passed, 1 failed
==============================================================================

$ pybot -i outputdata  /opt/cream_test/testsuite/cream_test_suite.html
........
==============================================================================
Cream Test Suite :: This is the main testing module,needed for testing the ...
==============================================================================
OutputData :: This test case tests the OutputData jdl attribute. T... | PASS |
------------------------------------------------------------------------------
Cream Test Suite :: This is the main testing module,needed for tes... | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================

$ pybot -i sb_stage /opt/cream_test/testsuite/cream_test_suite.html
.....
==============================================================================
Cream Test Suite :: This is the main testing module,needed for testing the ...
==============================================================================
Sandbox Staging :: This test case tests the two different Sandbox ... | PASS |
------------------------------------------------------------------------------
Cream Test Suite :: This is the main testing module,needed for tes... | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================

  • Functionality test suite log
  • Test "outputdata" log
  • Sandbox staging log. This run stand alone is successful. The failure in the previous run is due to some syncronization problem in the testsuite.

Regression tests

Verification attached bugs for CREAM Torque

Bug # FIXED

Verification attached bugs for CREAM LSF

Bug #97441: CREAM: Unwanted auto-updating of the field "creationTime" on the creamdb database FIXED

Standard conformance tests

Glue 1 Compliance

  • glue-validator -t glue1 -h cert-41.pd.infn.it -p 2170 -b o=grid 
    log

Glue 2 Compliance

Performance and scalability tests

Integration testing

Tests described:

-- SaraBertocco - 2013-01-25

Topic attachments
I Attachment Action Size Date Who Comment
Unknown file formatlog functionality_CREAM_SL6_LSF.html.log manage 506.4 K 2013-01-29 - 10:02 SaraBertocco  
Unknown file formatlog functionality_CREAM_SL6_PBS.html.log manage 503.1 K 2013-01-29 - 13:04 SaraBertocco  
Unknown file formatlog outputdata_LSF_SL6.html.log manage 173.9 K 2013-01-29 - 13:26 SaraBertocco  
Unknown file formatlog outputdata_PBS_SL6.html.log manage 177.6 K 2013-01-29 - 13:13 SaraBertocco  
Unknown file formatlog sandbox_staging_CREAM_SL6_LSF.html.log manage 170.4 K 2013-01-29 - 10:02 SaraBertocco  
Edit | Attach | PDF | History: r18 | r7 < r6 < r5 < r4 | Backlinks | Raw View | More topic actions...
Topic revision: r5 - 2013-01-29 - SaraBertocco
 
  • Edit
  • Attach
This site is powered by the TWiki collaboration platformCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback