Tags:
, view all tags

Testing report: IGIRTC-133

Summary

  • Product: CREAM v. 1.15.1
  • Release Task: Task #34043
  • ETICS Subsystem Configuration Name: emi-cream-ce_R_1_15_1_1, emi-cream-ui_R_1_15_1_1
  • VCS Tag: emi-cream-ce-1.2.1-2, dynsched-generic-2.5.1-3, glite-ce-common-java-1.15.1-2, glite-ce-cream-api-java-1.15.1-2, glite-ce-cream-core-1.15.1-2, glite-ce-cream-es-1.15.1-2, glite-ce-cream-utils-1.3.1-4, glite-ce-wsdl-1.15.1-1, glite-ce-yaim-cream-ce-4.4.1-4, glite-jdl-api-java-3.3.1-2,
  • EMI Major Release: EMI 3 (Monte Bianco)
  • Platform: SL5 epel
  • Author: Sara Bertocco
  • Testing report: Testing Report file
  • Certification report: Certification Report file
  • Outcome: "In Certification*

Deployment tests

Clean Installation

CREAM PBS PASSED

CREAM LSF PASSED

Upgrade Installation

CREAM PBS PASSED

CREAM LSF PASSED

Unit Tests

System tests

Functionality tests

CREAM PBS PASSED

The "Verify Job Status Output" test fails because a bug in the UI EMI 2 used to launch the testsuite. Other tests are excluded for configuration issues.
$  pybot  -e bdii -e glue -e job_man -e old_blah  /opt/cream_test/testsuite/cream_test_suite.html
INFO: The files of this testsuite will be stored under: /tmp/tmpMzx6BW.cream_testing/
INFO: Non-mandatory variable "sec_cert" has not been set.
INFO: Non-mandatory variable "sec_key" has not been set.
INFO: Non-mandatory variable "argus_host" has not been set.
INFO: Non-mandatory variable "sec_proxy_pass" has not been set.
==============================================================================
Cream Test Suite :: This is the main testing module,needed for testing the ...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
Check Executables :: Check whether the needed executables for the ... | PASS |
------------------------------------------------------------------------------
Simple Submit :: Execute /bin/uname -a.Submit the jdl and wait for... | PASS |
------------------------------------------------------------------------------
ISB Base URI :: Execute a bash shell script.The script is stored i... | PASS |
------------------------------------------------------------------------------
ISB Client to CE :: Execute a bash shell script.The script is stor... | PASS |
------------------------------------------------------------------------------
ISB gsiftp to CE :: Execute a bash shell script.The script is stor... | PASS |
------------------------------------------------------------------------------
OSB localhost :: Execute /bin/uname -a.Store std out and error str... | PASS |
------------------------------------------------------------------------------
OSB gsiftp BaseDestURI :: Execute /bin/uname -a.Store std out and ... | PASS |
------------------------------------------------------------------------------
OSB gsiftp DestURI :: Execute /bin/uname -a.Store std out and erro... | PASS |
------------------------------------------------------------------------------
Environment :: Execute a bash shell script.The script is stored lo... | PASS |
------------------------------------------------------------------------------
Epilogue :: Execute two bash shell scripts,one for the job and the... | PASS |
------------------------------------------------------------------------------
Prologue :: Execute two bash shell scripts,one for the job and the... | PASS |
------------------------------------------------------------------------------
Explicit Delegation :: Test job submission with explicit delegatio... | PASS |
------------------------------------------------------------------------------
Job Cancel :: Test job cancellation.Two cases are tested:             | PASS |
------------------------------------------------------------------------------
Job List :: Test the job listing cli utility.Store the list of the... | PASS |
------------------------------------------------------------------------------
Job Suspend - Resume :: Submit a job and suspend it.Wait until it'... | PASS |
------------------------------------------------------------------------------
Manually Cancel Job :: Submit a job and then manually cancel it on... | PASS |
------------------------------------------------------------------------------
Verify CE service info :: This test case verifies the output of th... | PASS |
------------------------------------------------------------------------------
Enable - Disable Submission :: This test case tests the proper fun... | PASS |
------------------------------------------------------------------------------
Job Purge :: This test case verifies the proper functionality of t... | PASS |
------------------------------------------------------------------------------
Limiter :: NOTE: This test case is about 25 minutes long!             | PASS |
------------------------------------------------------------------------------
Ban User :: This test case tests if the banning of user works. The... | PASS |
------------------------------------------------------------------------------
Sandbox Staging :: This test case tests the two different Sandbox ... | PASS |
------------------------------------------------------------------------------
Automatic Proxy Purging :: This test case checks the automatic pro... | PASS |
------------------------------------------------------------------------------
Verify Job Status Output :: This test is checking the output of th... | FAIL |
'['https://cert-42.pd.infn.it:8443/CREAM039112575', 'CANCELLED', 'https://cert-42.pd.infn.it:8443/CREAM096873115', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM096922256', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM130270327', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM142219948', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM144042097', 'CANCELLED', 'https://cert-42.pd.infn.it:8443/CREAM144489913', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM151509511', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM216440424', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM242371973', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM351372075', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM371767205', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM377678364', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM411128309', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM514181581', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM526025806', 'REGISTERED', 'https://cert-42.pd.infn.it:8443/CREAM531337500', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM546215002', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM548918653', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM619094711', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM627130670', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM659313536', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM713806828', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM759669997', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM860045672', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM862460658', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM885876820', 'DONE-OK']' does not contain 'https://cert-42.pd.infn.it:8443/CREAM694842335'
------------------------------------------------------------------------------
OutputData :: This test case tests the OutputData jdl attribute. T... | PASS |
------------------------------------------------------------------------------
Automatic Job Purge :: This test case is testing the functionality... | PASS |
------------------------------------------------------------------------------
Proxy Renewal :: Submit a job lasting 20 minutes using a proxy val... | PASS |
------------------------------------------------------------------------------
CPU Allocation 1 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 2 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 3 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 4 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 5 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 6 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
Logging and Bookkeeping :: Check if the logging and bookkeeping se... | PASS |
------------------------------------------------------------------------------
Cream Test Suite :: This is the main testing module,needed for tes... | FAIL |
35 critical tests, 34 passed, 1 failed
35 tests total, 34 passed, 1 failed
==============================================================================
Output:  /home/bertocco/output.xml
Log:     /home/bertocco/log.html
Report:  /home/bertocco/report.html

CREAM LSF PASSED

  • job_man excluded because requires two proxyes which the certifier has not.
  • old_blah excluded because the ce is configured with new blah
  • OutputData usage check fails for configuration issues
  • Logging and Bookkeeping check fails for UI installation issues.
$ /usr/bin/pybot  -e bdii -e glue -e job_man -e old_blah  /opt/cream_test/testsuite/cream_test_suite.html
INFO: The files of this testsuite will be stored under: /tmp/tmp1fA8m_.cream_testing/
INFO: Non-mandatory variable "sec_cert" has not been set.
INFO: Non-mandatory variable "sec_key" has not been set.
INFO: Non-mandatory variable "argus_host" has not been set.
INFO: Non-mandatory variable "sec_proxy_pass" has not been set.
INFO: Non-mandatory variable "argus_root_pass" has not been set.
==============================================================================
Cream Test Suite :: This is the main testing module,needed for testing the ...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
Check Executables :: Check whether the needed executables for the ... | PASS |
------------------------------------------------------------------------------
Simple Submit :: Execute /bin/uname -a.Submit the jdl and wait for... | PASS |
------------------------------------------------------------------------------
ISB Base URI :: Execute a bash shell script.The script is stored i... | PASS |
------------------------------------------------------------------------------
ISB Client to CE :: Execute a bash shell script.The script is stor... | PASS |
------------------------------------------------------------------------------
ISB gsiftp to CE :: Execute a bash shell script.The script is stor... | PASS |
------------------------------------------------------------------------------
OSB localhost :: Execute /bin/uname -a.Store std out and error str... | PASS |
------------------------------------------------------------------------------
OSB gsiftp BaseDestURI :: Execute /bin/uname -a.Store std out and ... | PASS |
------------------------------------------------------------------------------
OSB gsiftp DestURI :: Execute /bin/uname -a.Store std out and erro... | PASS |
------------------------------------------------------------------------------
Environment :: Execute a bash shell script.The script is stored lo... | PASS |
------------------------------------------------------------------------------
Epilogue :: Execute two bash shell scripts,one for the job and the... | PASS |
------------------------------------------------------------------------------
Prologue :: Execute two bash shell scripts,one for the job and the... | PASS |
------------------------------------------------------------------------------
Explicit Delegation :: Test job submission with explicit delegatio... | PASS |
------------------------------------------------------------------------------
Job Cancel :: Test job cancellation.Two cases are tested:             | PASS |
------------------------------------------------------------------------------
Job List :: Test the job listing cli utility.Store the list of the... | PASS |
------------------------------------------------------------------------------
Job Suspend - Resume :: Submit a job and suspend it.Wait until it'... | PASS |
------------------------------------------------------------------------------
Manually Cancel Job :: Submit a job and then manually cancel it on... | PASS |
------------------------------------------------------------------------------
Verify CE service info :: This test case verifies the output of th... | PASS |
------------------------------------------------------------------------------
Enable - Disable Submission :: This test case tests the proper fun... | PASS |
------------------------------------------------------------------------------
Job Purge :: This test case verifies the proper functionality of t... | PASS |
------------------------------------------------------------------------------
Limiter :: NOTE: This test case is about 25 minutes long!             | PASS |
------------------------------------------------------------------------------
Ban User :: This test case tests if the banning of user works. The... | PASS |
------------------------------------------------------------------------------
Sandbox Staging :: This test case tests the two different Sandbox ... | PASS |
------------------------------------------------------------------------------
Automatic Proxy Purging :: This test case checks the automatic pro... | PASS |
------------------------------------------------------------------------------
Verify Job Status Output :: This test is checking the output of th... | PASS |
------------------------------------------------------------------------------
OutputData :: This test case tests the OutputData jdl attribute. T... | FAIL |
'[]' should not be empty
------------------------------------------------------------------------------
Automatic Job Purge :: This test case is testing the functionality... | PASS |
------------------------------------------------------------------------------
Proxy Renewal :: Submit a job lasting 20 minutes using a proxy val... | PASS |
------------------------------------------------------------------------------
CPU Allocation 1 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 2 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 3 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 4 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 5 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 6 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
Logging and Bookkeeping :: Check if the logging and bookkeeping se... | FAIL |
_error: WMS Job status polling failed with return value: 1
Command reported: Traceback (most recent call last):
,  File "/usr/bin/glite-wms-job-status", line 53, in <module>
,    import wmsui_utils
,ImportError: No module named wmsui_utils
------------------------------------------------------------------------------
Cream Test Suite :: This is the main testing module,needed for tes... | FAIL |
35 critical tests, 33 passed, 2 failed
35 tests total, 33 passed, 2 failed
==============================================================================
Output:  /home/bertocco/output.xml
Log:     /home/bertocco/log.html
Report:  /home/bertocco/report.html

$ /usr/bin/pybot  -i glue  /opt/cream_test/testsuite/cream_test_suite.html
INFO: The files of this testsuite will be stored under: /tmp/tmp48uh_f.cream_testing/
INFO: Non-mandatory variable "sec_cert" has not been set.
INFO: Non-mandatory variable "sec_key" has not been set.
INFO: Non-mandatory variable "argus_host" has not been set.
INFO: Non-mandatory variable "sec_proxy_pass" has not been set.
INFO: Non-mandatory variable "argus_root_pass" has not been set.
==============================================================================
Cream Test Suite :: This is the main testing module,needed for testing the ...
==============================================================================
Validate Glue Schema :: Run the glue validator on the cream host.     | PASS |
------------------------------------------------------------------------------
Cream Test Suite :: This is the main testing module,needed for tes... | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
Output:  /home/bertocco/output.xml
Log:     /home/bertocco/log.html
Report:  /home/bertocco/report.html
  • Validate glue schema test log

$ /usr/bin/pybot  -i bdii  /opt/cream_test/testsuite/cream_test_suite.html
INFO: The files of this testsuite will be stored under: /tmp/tmpWmVNrW.cream_testing/
INFO: Non-mandatory variable "sec_cert" has not been set.
INFO: Non-mandatory variable "sec_key" has not been set.
INFO: Non-mandatory variable "argus_host" has not been set.
INFO: Non-mandatory variable "sec_proxy_pass" has not been set.
INFO: Non-mandatory variable "argus_root_pass" has not been set.
==============================================================================
Cream Test Suite :: This is the main testing module,needed for testing the ...
==============================================================================
Resource BDII :: Check if the resource bdii publishes all the rele... | PASS |
------------------------------------------------------------------------------
Cream Test Suite :: This is the main testing module,needed for tes... | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
Output:  /home/bertocco/output.xml
Log:     /home/bertocco/log.html
Report:  /home/bertocco/report.html
  • Check resource bdii test log

Standard conformance tests

Glue 1 Compliance

LSF FAILED ?
  • glue-validator -t glue1 -h cert-41.pd.infn.it -p 2170 -b o=grid 
    log

PBS FAILED ?
  • glue-validator -t glue1 -h cert-42.pd.infn.it -p 2170 -b o=grid
    log

Glue 2 Compliance

LSF PASSED
  • glue-validator -t glue2 -h cert-41.pd.infn.it -p 2170 -b o=glue 
    log

PBS PASSED
  • glue-validator -t glue2 -h cert-42.pd.infn.it -p 2170 -b o=glue 
    log

Performance and scalability tests

LSF PASSED

  • ./bin/cream-test-monitored-submit -r 30 -n 1000 -m 50 -C 50 -l log4py.conf -j jdl/testsandbox.jdl -R cert-41.pd.infn.it:8443/cream-lsf-cert
    log
  • ./bin/cream-test-monitored-cancel -r 30 -n 1000 -m 50 -C 50 -l log4py.conf -j jdl/simple.jdl -R cert-41.pd.infn.it:8443/cream-lsf-cert
    log
  • ./bin/cream-test-monitored-lease-updated -r 60 -n 50 -m 10 -C 10 -l log4py.conf -W 1200 --vo dteam --valid 00:20 -R cert-41.pd.infn.it:8443/cream-lsf-cert
    
    log

PBS NA

Integration testing

Tests described:

-- SaraBertocco - 2013-03-04

Topic attachments
I Attachment Action Size Date Who Comment
Unknown file formatlog bdii.html.log manage 175.3 K 2013-03-06 - 16:49 SaraBertocco  
Unknown file formatlog cream-test-monitored-cancel_cert-41.log manage 983.4 K 2013-03-11 - 16:58 SaraBertocco  
Unknown file formatlog cream-test-monitored-lease-updated_cert-41.log manage 152.1 K 2013-03-12 - 09:53 SaraBertocco  
Unknown file formatlog cream-test-monitored-submit_cert-41.log manage 840.4 K 2013-03-11 - 16:02 SaraBertocco  
Unknown file formatlog glue.html.log manage 163.5 K 2013-03-06 - 16:49 SaraBertocco  
Unknown file formatlog glue1_compliance_sl6_lsf.log manage 6.4 K 2013-03-11 - 16:56 SaraBertocco  
Unknown file formatlog glue1_compliance_sl6_pbs.log manage 7.5 K 2013-03-11 - 16:57 SaraBertocco  
Unknown file formatlog glue2_compliance_sl6_lsf.log manage 9.0 K 2013-03-11 - 16:07 SaraBertocco  
Unknown file formatlog glue2_compliance_sl6_pbs.log manage 9.9 K 2013-03-11 - 17:07 SaraBertocco  
Texttxt task34043_Certification_Report.txt manage 3.0 K 2013-03-12 - 18:00 SaraBertocco  
Texttxt task34043_Test_Report.txt manage 8.2 K 2013-03-12 - 18:00 SaraBertocco  
Unknown file formatlog task34043_conf_install_cream_lsf_sl6.log manage 12.3 K 2013-03-06 - 16:55 SaraBertocco  
Unknown file formatlog task34043_conf_update_cream_pbs_sl6.log manage 15.5 K 2013-03-05 - 14:35 SaraBertocco  
Unknown file formatlog task34043_config_install_cream_pbs_sl6.log manage 15.5 K 2013-03-12 - 10:42 SaraBertocco  
Unknown file formatlog task34043_config_update_cream_lsf_sl6.log manage 12.7 K 2013-03-08 - 17:31 SaraBertocco  
Unknown file formatlog task34043_config_update_cream_pbs_sl6.log manage 15.7 K 2013-03-11 - 13:40 SaraBertocco  
Unknown file formatlog task34043_functionality_lsf_sl6.log manage 519.1 K 2013-03-06 - 16:26 SaraBertocco  
Unknown file formatlog task34043_functionality_pbs_sl6.log manage 532.1 K 2013-03-06 - 11:52 SaraBertocco  
Unknown file formatlog task34043_install_config_cream_lsf_sl5.log manage 12.5 K 2013-03-05 - 10:28 SaraBertocco  
Unknown file formatlog task34043_install_cream_lsf_sl6.log manage 172.8 K 2013-03-06 - 16:55 SaraBertocco  
Unknown file formatlog task34043_install_cream_pbs_sl6.log manage 179.0 K 2013-03-11 - 15:49 SaraBertocco  
Unknown file formatlog task34043_update_cream_lsf_sl6.log manage 41.1 K 2013-03-08 - 17:31 SaraBertocco  
Unknown file formatlog task34043_update_cream_pbs_sl6.log manage 38.9 K 2013-03-11 - 13:40 SaraBertocco  
Edit | Attach | PDF | History: r13 < r12 < r11 < r10 < r9 | Backlinks | Raw View | More topic actions...
Topic revision: r11 - 2013-03-12 - SaraBertocco
 
  • Edit
  • Attach
This site is powered by the TWiki collaboration platformCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback