Tags:
, view all tags

Testing report: IGIRTC-125

Summary

  • Product: CREAM v. 1.14.3
  • Release Task: Task #38531
  • ETICS Subsystem Configuration Name:
  • VCS Tag:
  • EMI Major Release: EMI 2 (Matterhorn)
  • Platform: SL6 epel
  • Author: Sara Bertocco
  • Testing report: [[][Testing Report file]]
  • Certification report: [[][Certification Report file]]
  • Outcome: "In Certification*

Deployment tests

Clean Installation

CREAM PBS

CREAM LSF

Upgrade Installation

CREAM PBS

    • [[][Update log file]]
    • [[][Configuration log file]]

CREAM LSF

    • [[][Update log file]]
    • [[][Configuration log file]]

Unit Tests NA

Not available.

System tests

The tests are performed with two different user interfaces:

  • tests of CREAM LSF use UI EMI version 1.11.0-1
  • tests of CREAM PBS use UI EMI version 2.5.1-1
this is the reason why the test "Verify Job Status Output" fails in the pbs version: it is a UI problem.

Authentication using GJAF

Functionality tests

  • Test "bdii" and "glue" are covered in compliance tests
  • Test "job_man" is excluded because it requires two proxy certificates and the certifier has one only
  • Test "outputdata" is excluded in the first test suite run because a problem in testbed bdii setup, solved later.

CREAM PBS

$ pybot -e outputdata -e bdii -e glue -e job_man /opt/cream_test/testsuite/cream_test_suite.html
INFO: The files of this testsuite will be stored under: /tmp/tmpMSXELI.cream_testing/
INFO: Non-mandatory variable "sec_cert" has not been set.
INFO: Non-mandatory variable "sec_key" has not been set.
INFO: Non-mandatory variable "argus_host" has not been set.
INFO: Non-mandatory variable "sec_proxy_pass" has not been set.
==============================================================================
Cream Test Suite :: This is the main testing module,needed for testing the ...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
Check Executables :: Check whether the needed executables for the ... | PASS |
------------------------------------------------------------------------------
Simple Submit :: Execute /bin/uname -a.Submit the jdl and wait for... | PASS |
------------------------------------------------------------------------------
ISB Base URI :: Execute a bash shell script.The script is stored i... | PASS |
------------------------------------------------------------------------------
ISB Client to CE :: Execute a bash shell script.The script is stor... | PASS |
------------------------------------------------------------------------------
ISB gsiftp to CE :: Execute a bash shell script.The script is stor... | PASS |
------------------------------------------------------------------------------
OSB localhost :: Execute /bin/uname -a.Store std out and error str... | PASS |
------------------------------------------------------------------------------
OSB gsiftp BaseDestURI :: Execute /bin/uname -a.Store std out and ... | PASS |
------------------------------------------------------------------------------
OSB gsiftp DestURI :: Execute /bin/uname -a.Store std out and erro... | PASS |
------------------------------------------------------------------------------
Environment :: Execute a bash shell script.The script is stored lo... | PASS |
------------------------------------------------------------------------------
Epilogue :: Execute two bash shell scripts,one for the job and the... | PASS |
------------------------------------------------------------------------------
Prologue :: Execute two bash shell scripts,one for the job and the... | PASS |
------------------------------------------------------------------------------
Explicit Delegation :: Test job submission with explicit delegatio... | PASS |
------------------------------------------------------------------------------
Job Cancel :: Test job cancellation.Two cases are tested:             | PASS |
------------------------------------------------------------------------------
Job List :: Test the job listing cli utility.Store the list of the... | PASS |
------------------------------------------------------------------------------
Job Suspend - Resume :: Submit a job and suspend it.Wait until it'... | PASS |
------------------------------------------------------------------------------
Manually Cancel Job :: Submit a job and then manually cancel it on... | PASS |
------------------------------------------------------------------------------
Verify CE service info :: This test case verifies the output of th... | PASS |
------------------------------------------------------------------------------
Enable - Disable Submission :: This test case tests the proper fun... | PASS |
------------------------------------------------------------------------------
Job Purge :: This test case verifies the proper functionality of t... | PASS |
------------------------------------------------------------------------------
Limiter :: NOTE: This test case is about 25 minutes long!             | PASS |
------------------------------------------------------------------------------
Ban User :: This test case tests if the banning of user works. The... | PASS |
------------------------------------------------------------------------------
Sandbox Staging :: This test case tests the two different Sandbox ... | PASS |
------------------------------------------------------------------------------
Automatic Proxy Purging :: This test case checks the automatic pro... | PASS |
------------------------------------------------------------------------------
Verify Job Status Output :: This test is checking the output of th... | FAIL |
'['https://cert-42.pd.infn.it:8443/CREAM008747298', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM092671703', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM124860354', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM190619285', 'IDLE', 'https://cert-42.pd.infn.it:8443/CREAM209313410', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM216599452', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM247509604', 'CANCELLED', 'https://cert-42.pd.infn.it:8443/CREAM298811348', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM338616186', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM397680813', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM409140727', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM418737077', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM438966721', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM478916637', 'IDLE', 'https://cert-42.pd.infn.it:8443/CREAM483549371', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM504522990', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM588399968', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM621620940', 'CANCELLED', 'https://cert-42.pd.infn.it:8443/CREAM658041789', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM722241788', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM803417493', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM805989025', 'CANCELLED', 'https://cert-42.pd.infn.it:8443/CREAM829770983', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM856980366', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM906476021', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM932057863', 'DONE-OK', 'https://cert-42.pd.infn.it:8443/CREAM973629738', 'DONE-OK']' does not contain 'https://cert-42.pd.infn.it:8443/CREAM061880031'
------------------------------------------------------------------------------
Automatic Job Purge :: This test case is testing the functionality... | PASS |
------------------------------------------------------------------------------
Proxy Renewal :: Submit a job lasting 20 minutes using a proxy val... | PASS |
------------------------------------------------------------------------------
CPU Allocation 1 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 2 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 3 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 4 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 5 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 6 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
Logging and Bookkeeping :: Check if the logging and bookkeeping se... | PASS |
------------------------------------------------------------------------------
Old BLParser Restart :: Configure a CREAM CE using the old blah bl... | PASS |
------------------------------------------------------------------------------
Cream Test Suite :: This is the main testing module,needed for tes... | FAIL |
35 critical tests, 34 passed, 1 failed
35 tests total, 34 passed, 1 failed
==============================================================================

$ pybot -i outputdata  /opt/cream_test/testsuite/cream_test_suite.html
.......
==============================================================================
Cream Test Suite :: This is the main testing module,needed for testing the ...
==============================================================================
OutputData :: This test case tests the OutputData jdl attribute. T... | PASS |
------------------------------------------------------------------------------
Cream Test Suite :: This is the main testing module,needed for tes... | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================

  • Functionality test suite log
  • Test "outputdata" log

CREAM LSF

$ pybot -e outputdata -e bdii -e glue -e job_man /opt/cream_test/testsuite/cream_test_suite.html
INFO: The files of this testsuite will be stored under: /tmp/tmplKbo0y.cream_testing/
INFO: Non-mandatory variable "sec_cert" has not been set.
INFO: Non-mandatory variable "sec_key" has not been set.
INFO: Non-mandatory variable "argus_host" has not been set.
INFO: Non-mandatory variable "sec_proxy_pass" has not been set.
INFO: Non-mandatory variable "argus_root_pass" has not been set.
==============================================================================
Cream Test Suite :: This is the main testing module,needed for testing the ...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
Check Executables :: Check whether the needed executables for the ... | PASS |
------------------------------------------------------------------------------
Simple Submit :: Execute /bin/uname -a.Submit the jdl and wait for... | PASS |
------------------------------------------------------------------------------
ISB Base URI :: Execute a bash shell script.The script is stored i... | PASS |
------------------------------------------------------------------------------
ISB Client to CE :: Execute a bash shell script.The script is stor... | PASS |
------------------------------------------------------------------------------
ISB gsiftp to CE :: Execute a bash shell script.The script is stor... | PASS |
------------------------------------------------------------------------------
OSB localhost :: Execute /bin/uname -a.Store std out and error str... | PASS |
------------------------------------------------------------------------------
OSB gsiftp BaseDestURI :: Execute /bin/uname -a.Store std out and ... | PASS |
------------------------------------------------------------------------------
OSB gsiftp DestURI :: Execute /bin/uname -a.Store std out and erro... | PASS |
------------------------------------------------------------------------------
Environment :: Execute a bash shell script.The script is stored lo... | PASS |
------------------------------------------------------------------------------
Epilogue :: Execute two bash shell scripts,one for the job and the... | PASS |
------------------------------------------------------------------------------
Prologue :: Execute two bash shell scripts,one for the job and the... | PASS |
------------------------------------------------------------------------------
Explicit Delegation :: Test job submission with explicit delegatio... | PASS |
------------------------------------------------------------------------------
Job Cancel :: Test job cancellation.Two cases are tested:             | PASS |
------------------------------------------------------------------------------
Job List :: Test the job listing cli utility.Store the list of the... | PASS |
------------------------------------------------------------------------------
Job Suspend - Resume :: Submit a job and suspend it.Wait until it'... | PASS |
------------------------------------------------------------------------------
Manually Cancel Job :: Submit a job and then manually cancel it on... | PASS |
------------------------------------------------------------------------------
Verify CE service info :: This test case verifies the output of th... | PASS |
------------------------------------------------------------------------------
Enable - Disable Submission :: This test case tests the proper fun... | PASS |
------------------------------------------------------------------------------
Job Purge :: This test case verifies the proper functionality of t... | PASS |
------------------------------------------------------------------------------
Limiter :: NOTE: This test case is about 25 minutes long!             | PASS |
------------------------------------------------------------------------------
Ban User :: This test case tests if the banning of user works. The... | PASS |
------------------------------------------------------------------------------
Sandbox Staging :: This test case tests the two different Sandbox ... | FAIL |
IOError: [Errno 2] No such file
------------------------------------------------------------------------------
Automatic Proxy Purging :: This test case checks the automatic pro... | PASS |
------------------------------------------------------------------------------
Verify Job Status Output :: This test is checking the output of th... | PASS |
------------------------------------------------------------------------------
Automatic Job Purge :: This test case is testing the functionality... | PASS |
------------------------------------------------------------------------------
Proxy Renewal :: Submit a job lasting 20 minutes using a proxy val... | PASS |
------------------------------------------------------------------------------
CPU Allocation 1 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 2 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 3 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 4 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 5 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
CPU Allocation 6 :: Validate the contents of the batch job submiss... | PASS |
------------------------------------------------------------------------------
Logging and Bookkeeping :: Check if the logging and bookkeeping se... | PASS |
------------------------------------------------------------------------------
Old BLParser Restart :: Configure a CREAM CE using the old blah bl... | PASS |
------------------------------------------------------------------------------
Cream Test Suite :: This is the main testing module,needed for tes... | FAIL |
35 critical tests, 34 passed, 1 failed
35 tests total, 34 passed, 1 failed
==============================================================================

$ pybot -i outputdata  /opt/cream_test/testsuite/cream_test_suite.html
........
==============================================================================
Cream Test Suite :: This is the main testing module,needed for testing the ...
==============================================================================
OutputData :: This test case tests the OutputData jdl attribute. T... | PASS |
------------------------------------------------------------------------------
Cream Test Suite :: This is the main testing module,needed for tes... | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================

$ pybot -i sb_stage /opt/cream_test/testsuite/cream_test_suite.html
.....
==============================================================================
Cream Test Suite :: This is the main testing module,needed for testing the ...
==============================================================================
Sandbox Staging :: This test case tests the two different Sandbox ... | PASS |
------------------------------------------------------------------------------
Cream Test Suite :: This is the main testing module,needed for tes... | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================

  • Functionality test suite log
  • Test "outputdata" log
  • Sandbox staging log. This run stand alone is successful. The failure in the previous run is due to some syncronization problem in the testsuite.

Regression tests

Verification attached bugs for CREAM Torque

Bug #99747: glite-info-dynamic-ce does not update GLUE2ComputingShareServingState FIXED
* From UI
$ glite-ce-enable-submission cert-42.pd.infn.it:8443
Operation for enabling new submissions succeeded
  • On CE
# /sbin/runuser -s /bin/sh ldap -c "/var/lib/bdii/gip/plugin/glite-info-cream-glue2" |grep ServingState
GLUE2EndpointServingState: production
* From UI
$ glite-ce-disable-submission cert-42.pd.infn.it:8443
Operation for disabling new submissions succeeded
  • On CE
# /sbin/runuser -s /bin/sh ldap -c "/var/lib/bdii/gip/plugin/glite-info-cream-glue2" |grep ServingState
GLUE2EndpointServingState: draining

Verification attached bugs for CREAM LSF

Bug #98850 Empty ACBR list in SHARE variable FIXED
# grep FQANVOVIEW cert-41/site-info.def
#FQANVOVIEWS=yes
FQANVOVIEWS=no
# grep CERT_GROUP_ENABLE cert-41/site-info.def
CERT_GROUP_ENABLE="dteam /dteam/ROLE=lcgadmin /dteam/ROLE=production /dteam/ROLE=pilot"

# grep SHARE /etc/glite-ce-glue2/glite-ce-glue2.conf |grep ACBR
SHARE_creamtest1_dteam_ACBRS=( VO:dteam )
SHARE_creamtest2_dteam_ACBRS=( VO:dteam )
SHARE_cert_dteam_ACBRS=( VO:dteam , VOMS:/dteam/Role=lcgadmin , VOMS:/dteam/Role=production , VOMS:/dteam/Role=pilot )

Bug #99823 SHA-1 algorithm for PKCS10 generation in CREAM delegation service
Bug #99747 glite-info-dynamic-ce does not update GLUE2ComputingShareServingState FIXED
* From UI
$ glite-ce-enable-submission cert-41.pd.infn.it:8443
Operation for enabling new submissions succeeded
  • On CE
# /sbin/runuser -s /bin/sh ldap -c "/var/lib/bdii/gip/plugin/glite-info-cream-glue2" |grep ServingState
GLUE2EndpointServingState: production
* From UI
$ glite-ce-disable-submission cert-41.pd.infn.it:8443
Operation for disabling new submissions succeeded
  • On CE
# /sbin/runuser -s /bin/sh ldap -c "/var/lib/bdii/gip/plugin/glite-info-cream-glue2" |grep ServingState
GLUE2EndpointServingState: draining

Bug #99740 updateDelegationProxyInfo error: Rollback executed due to Deadlock
Bug #99738 Under stress conditions due to job submissions, the command queue may accumulate thousand of job purging commands
Bug #99282 Wrong regular expression for group.conf parsing FIXED?
# grep -r DGDTEAM cert-41/site-info.def
VO_DGDTEAM_SW_DIR=$VO_SW_DIR/dteam
VO_DGDTEAM_DEFAULT_SE=$CLASSIC_HOST
VO_DGDTEAM_STORAGE_DIR=$CLASSIC_STORAGE_DIR/dteam
VO_DGDTEAM_VOMS_SERVERS='vomss://voms.hellasgrid.gr:8443/voms/dteam?/dteam/' 
VO_DGDTEAM_VOMSES="'dteam voms.hellasgrid.gr 15004 /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr dteam 24' 'dteam voms2.hellasgrid.gr 15004 /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr dteam 24'" 
VO_DGDTEAM_VOMS_CA_DN="'/C=GR/O=HellasGrid/OU=Certification Authorities/CN=HellasGrid CA 2006' '/C=GR/O=HellasGrid/OU=Certification Authorities/CN=HellasGrid CA 2006'"

# /opt/glite/yaim/bin/yaim -c -s /usr/local/nfs1/jra1_test/cert-41/site-info.def  -n creamCE -n LSF_utils
....
   INFO: YAIM terminated succesfully.
  • ldapsearch -x -H ldap://cert-41.pd.infn.it:2170 -b o=glue output

Bug #99085 Improve parsing of my.cnf FIXED
# grep connections /etc/my.cnf
max_connections = 450

# /opt/glite/yaim/bin/yaim -rr -s /usr/local/nfs1/jra1_test/cert-41/site-info.def  -n creamCE -f config_cream_db 
   WARNING: 
   WARNING: *****************************************************************************
   WARNING: Your siteinfo directory is world readable. This is generally a bad idea      
   WARNING: as configuration files may contain passwords and other sensitive information.
   WARNING: *****************************************************************************
   WARNING: 
   INFO: Configuring HOST: cert-41.pd.infn.it
   INFO: Using site configuration file: /usr/local/nfs1/jra1_test/cert-41/site-info.def
   INFO: Sourcing service specific configuration file: /usr/local/nfs1/jra1_test/cert-41/services/glite-creamce
   INFO: 
         ###################################################################
         
         .             /'.-. ')
         .     yA,-"-,( ,m,:/ )   .oo.     oo    o      ooo  o.     .oo
         .    /      .-Y a  a Y-.     8. .8'    8'8.     8    8b   d'8
         .   /           ~ ~ /         8'    .8oo88.     8    8  8'  8
         . (_/         '===='          8    .8'     8.   8    8  Y   8
         .   Y,-''-,Yy,-.,/           o8o  o8o    o88o  o8o  o8o    o8o
         .    I_))_) I_))_)
         
         
         current working directory: /etc/yum.repos.d
         site-info.def date: Jan 25 10:47 /usr/local/nfs1/jra1_test/cert-41/site-info.def
         yaim command: -rr -s /usr/local/nfs1/jra1_test/cert-41/site-info.def -n creamCE -f config_cream_db
         log file: /opt/glite/yaim/bin/../log/yaimlog
         Mon Feb  4 15:05:35 CET 2013 : /opt/glite/yaim/bin/yaim
         
         Installed YAIM versions:
         glite-yaim-bdii 4.3.11-1
         glite-yaim-core 5.1.0-1
         glite-yaim-cream-ce 4.3.2-1
         glite-yaim-lsf-utils 5.0.0-2
         
         ####################################################################
   INFO: The default location of the grid-env.(c)sh files will be: /usr/libexec
   INFO: Sourcing the utilities in /opt/glite/yaim/functions/utils
   INFO: Detecting environment
   INFO: Assuming the node types: creamCE
   INFO: Using hostname: cert-41.pd.infn.it
   INFO: Executing function: config_cream_db_check 
   INFO: Executing function: config_cream_db_setenv 
   INFO: Executing function: config_cream_db 
creamdb version requested by the service is 2.6
The version retrieved from the creamdb database is 2.6
The creamdb database is synchronized with the one requested by the service
   INFO: creamdb.db_info table updated.
delegationcreamdb version requested by the service is 2.6
The version retrieved from the delegationcreamdb database is 2.6
The delegationcreamdb database is synchronized with the one requested by the service
   INFO: YAIM terminated succesfully.

Bug #99072 Hard-coded reference to tomcat5.pid FIXED
$ pybot bug_99072.html
..........
==============================================================================
Bug 99072 :: Regression test of bug #83338 endpointType (in GLUE2ServiceCom...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
Check Executables :: Check whether the needed executables for the ... | PASS |
------------------------------------------------------------------------------
bug_99072                                                             | PASS |
------------------------------------------------------------------------------
Bug 99072 :: Regression test of bug #83338 endpointType (in GLUE2S... | PASS |
3 critical tests, 3 passed, 0 failed
3 tests total, 3 passed, 0 failed
==============================================================================

Bug #98440 Missing revision number in EndpointImplementationVersion FIXED
$ ldapsearch -x -H ldap://cert-41.pd.infn.it:2170 -b o=glue |grep GLUE2EndpointImplementationVersion
GLUE2EndpointImplementationVersion: 1.14.3
GLUE2EndpointImplementationVersion: 1.0.8

Bug #98144 The switching off of the JobSubmissionManager makes the CREAM service not available for the users FIXED
On CE:
 grep JOB_SUBMISSION_MANAGER_ENABLE /etc/glite-ce-cream/cream-config.xml
      <parameter name="JOB_SUBMISSION_MANAGER_ENABLE" value="false" />
# service tomcat6 restart
Stopping tomcat6:                                          [  OK  ]
Starting tomcat6:                                          [  OK  ]

On UI:
$ glite-ce-job-submit  -r cert-41.pd.infn.it:8443/cream-lsf-cert  -a /home/bertocco/jdl/env.jdl
https://cert-41.pd.infn.it:8443/CREAM854666257
$ glite-ce-job-status https://cert-41.pd.infn.it:8443/CREAM854666257

******  JobID=[https://cert-41.pd.infn.it:8443/CREAM854666257]
   Status        = [DONE-OK]
   ExitCode      = [0]

Bug #95738 glite-ce-job-submit: error message to be improved if JDL file is missing ???
The error error message depends from the UI version and not from CREAM version.
$ ls /home/bertocco/bug_95738.jdl
ls: /home/bertocco/bug_95738.jdl: No such file or directory

$ glite-ce-job-submit  -r cert-41.pd.infn.it:8443/cream-lsf-cert  -a /home/bertocco/bug_95738.jdl
2013-01-31 17:55:27,505 FATAL - JDL File /home/bertocco/bug_95738.jdl missing on disk

Bug #95637 glite-ce-job-submit --help doesn't print out anything FIXED
$ glite-ce-job-submit --help
CREAM User Interface version 1.2.0

glite-ce-job-submit allows the user to submit a job for execution on a CREAM based CE


Usage: glite-ce-job-submit [options] -r <CEID> <JDLFile>

  --resource, -r CEID      Select the CE to send the JDL to. Format must be

           <host>[:<port>]/cream-<lrms-system-name>-<queue-name>

  <JDLFile>         Is the file containing the JDL directives for
                                job submission;

Options:
  --help, -h         prints this help and exits;

  --debug, -d         fires out debug messages and activate the
              writing on logfile;

  --nomsg, -n                   Suppresses any message except fatal
                                errors and output;

  --version, -v         Prints the version and exits;

  --output, -o <result_file>   writes the returned JobID on file result_file;
                                see below for more details about its format

  --autm-delegation, -a      Generates a new delegated proxy with an internal
                                delegation ID;

  --delegationid, -D <ID>   uses a previously delegated proxy certificate
            identified by <ID>;

  --logfile, -l <logfile>   writes any log and/or error message on the file
                                logfile; if the file already exists,
                                the output is appended to it;

  --vo, -V <VO_NAME>            specifies the Virtual Organisation the user
                                belongs to;

  --noint, -N         turns off any user interactivity (assuming
            'yes' for all questions);

  --ftp-streams, -s <number>   specifies the number of streams the gridftp
            client should use when sending InputSandbox.

  --conf, -c <file>           specifies the configuration file

  --donot-verify-ac-sign, -A   Allows the execution of the command even if the VO voms
                             server certificate is not installed.

If the configuration file <file> is not specified the client will try to read the file specified by
the environmental variable GLITE_CREAM_CLIENT_CONFIG; otherwise it will try to
readthe file $HOME/.glite/glite_cream.conf.
Finally if previous tries fail, built-in configurations will be used.

If <result_file> already exists AND its first line is '##CREAMJOBS##' the
output will be appended to it; otherwise you will be asked to overwrite it or
exit.

The specification of the tcpport (:<port>) is optional. If not specified it
will be obtained by the configuration file or the default 8443 will be used


Please report any bug to:
   alvise.dorigo@pd.infn.it

Bug #95041 YAIM could check the format of CE_OTHERDESCR FIXED
1.
# grep CE_OTHER cert-41/services/glite-creamce
CE_OTHERDESCR="Cores=9.75,Benchmark=9.04-HEP-SPEC06"
# /opt/glite/yaim/bin/yaim -c -s /usr/local/nfs1/jra1_test/cert-41/site-info.def  -n creamCE -n LSF_utils
....
   INFO: YAIM terminated succesfully.
# grep -r ERROR /opt/glite/yaim/log/yaimlog 
[root@cert-41 ~]
2.
# grep CE_OTHER cert-41/services/glite-creamce
#CE_OTHERDESCR="Cores=9.75,Benchmark=9.04-HEP-SPEC06"
CE_OTHERDESCR="Cores=10.5"
# /opt/glite/yaim/bin/yaim -c -s /usr/local/nfs1/jra1_test/cert-41/site-info.def  -n creamCE -n LSF_utils
....
   INFO: YAIM terminated succesfully.
# grep -r ERROR /opt/glite/yaim/log/yaimlog 
#
3.
# grep CE_OTHER cert-41/services/glite-creamce
#CE_OTHERDESCR="Cores=9.75,Benchmark=9.04-HEP-SPEC06"
#CE_OTHERDESCR="Cores=10.5"
CE_OTHERDESCR="Cores=10.5,Benchmark=8.0"
# /opt/glite/yaim/bin/yaim -c -s /usr/local/nfs1/jra1_test/cert-41/site-info.def  -n creamCE -n LSF_utils
   ERROR: CE_OTHERDESCR not properly set; format is Cores=<CE_LOGCPU/CE_PHYSICALCPU> [, Benchmark=<value>-HEP-SPEC06]
   ERROR: One of the functions returned with error without specifying its nature !

Bug #88134 JobWrapper doesn't handle correctly the jdl attribute “PerusalListFileURI” FIXED
On CE:
# ls -l /tmp/filelist.txt
ls: cannot access /tmp/filelist.txt: No such file or directory

On UI:

$ cat perusal.jdl
[
Type="Job";
JobType="Normal";
Executable = "perusal.sh";
StdOutput = "stdout.log";
StdError = "stderr.log";
InputSandbox = "perusal.sh";
OutputSandbox = {"stdout.log", "stderr.log", "results.txt"};
PerusalFilesDestURI="gsiftp://cream-05.pd.infn.it/tmp";
PerusalFileEnable = true;
PerusalTimeInterval = 20;
outputsandboxbasedesturi="gsiftp://localhost";
PerusalListFileURI="gsiftp://cream-05.pd.infn.it/tmp/filelist.txt"
]

$ cat perusal.jdl
[
Type="Job";
JobType="Normal";
Executable = "perusal.sh";
StdOutput = "stdout.log";
StdError = "stderr.log";
InputSandbox = "perusal.sh";
OutputSandbox = {"stdout.log", "stderr.log", "results.txt"};
PerusalFilesDestURI="gsiftp://cream-05.pd.infn.it/tmp";
PerusalFileEnable = true;
PerusalTimeInterval = 20;
outputsandboxbasedesturi="gsiftp://localhost";
PerusalListFileURI="gsiftp://cream-05.pd.infn.it/tmp/filelist.txt"
]

$ sleep 120; glite-ce-job-status https://cert-41.pd.infn.it:8443/CREAM895860000

******  JobID=[https://cert-41.pd.infn.it:8443/CREAM895860000]
   Status        = [DONE-OK]
   ExitCode      = [0]

Verification of old bugs for CREAM LSF

Bug #59871 FIXED
$ pybot bug_59871.html
........
==============================================================================
Bug 59871 :: Regression test of bug #59871 lcg-info-dynamic-software must s...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
bug_59871                                                             | PASS |
------------------------------------------------------------------------------
Teardown                                                              | PASS |
------------------------------------------------------------------------------
Bug 59871 :: Regression test of bug #59871 lcg-info-dynamic-softwa... | PASS |
3 critical tests, 3 passed, 0 failed
3 tests total, 3 passed, 0 failed
==============================================================================

Bug #69857 FIXED
$ pybot bug_69857.html
.....
The files of this testsuite will be stored under: /tmp/tmpPFTrFn.cream_testing/
==============================================================================
Bug 69857 :: Regression test of bug #69857: Job submission to CreamCE is en...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
Check Executables :: Check whether the needed executables for the ... | PASS |
------------------------------------------------------------------------------
bug_69857                                                             | PASS |
------------------------------------------------------------------------------
Bug 69857 :: Regression test of bug #69857: Job submission to Crea... | PASS |
3 critical tests, 3 passed, 0 failed
3 tests total, 3 passed, 0 failed
==============================================================================

Bug #81561 FIXED

$ pybot bug_81561.html
.....
The files of this testsuite will be stored under: /tmp/tmpHZ4Xx4.cream_testing/
==============================================================================
Bug 81561 :: Regression test of bug #81561: Make JobDBAdminPurger script co...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
bug_81561                                                             | PASS |
------------------------------------------------------------------------------
Bug 81561 :: Regression test of bug #81561: Make JobDBAdminPurger ... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================

Bug #83238 FIXED
$ pybot bug_83238.html
......
==============================================================================
Bug 83238 :: Regression test of bug #83238 Sometimes CREAM does not update ...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
Check Executables :: Check whether the needed executables for the ... | PASS |
------------------------------------------------------------------------------
bug_83238                                                             | PASS |
------------------------------------------------------------------------------
Bug 83238 :: Regression test of bug #83238 Sometimes CREAM does no... | PASS |
3 critical tests, 3 passed, 0 failed
3 tests total, 3 passed, 0 failed
==============================================================================

Bug #83338 FIXED
$ pybot bug_83338.html
......
==============================================================================
Bug 83338 :: Regression test of bug #83338 endpointType (in GLUE2ServiceCom...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
Check Executables :: Check whether the needed executables for the ... | PASS |
------------------------------------------------------------------------------
bug_83338                                                             | PASS |
------------------------------------------------------------------------------
Bug 83338 :: Regression test of bug #83338 endpointType (in GLUE2S... | PASS |
3 critical tests, 3 passed, 0 failed
3 tests total, 3 passed, 0 failed
==============================================================================

Bug #83592 FIXED
$ pybot bug_83592.html
.........
==============================================================================
Bug 83592 :: Regression test of bug #83592 CREAM client doesn't allow the d...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
Check Executables :: Check whether the needed executables for the ... | PASS |
------------------------------------------------------------------------------
bug_83592                                                             | PASS |
------------------------------------------------------------------------------
Bug 83592 :: Regression test of bug #83592 CREAM client doesn't al... | PASS |
3 critical tests, 3 passed, 0 failed
3 tests total, 3 passed, 0 failed
==============================================================================

Bug #83593 FIXED
$ pybot bug_83593.html
.......
==============================================================================
Bug 83593 :: Regression test of bug #83593 Problems limiting RFC proxies in...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
Check Executables :: Check whether the needed executables for the ... | PASS |
------------------------------------------------------------------------------
bug_83593                                                             | PASS |
------------------------------------------------------------------------------
Bug 83593 :: Regression test of bug #83593 Problems limiting RFC p... | PASS |
3 critical tests, 3 passed, 0 failed
3 tests total, 3 passed, 0 failed
==============================================================================

Bug #83749 FIXED
$ pybot bug_83749.html
==============================================================================
Bug 83749 :: Regression test of bug #83749 yaim-cream-ce should manage the ...
==============================================================================
bug_83749                                                             | PASS |
------------------------------------------------------------------------------
Bug 83749 :: Regression test of bug #83749 yaim-cream-ce should ma... | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================

Bug #84374 FIXED
$ pybot bug_84374.html
..........
==============================================================================
Bug 84374 :: Regression test of Bug #84374 yaim-cream-ce: GlueForeignKey: G...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
bug_84374                                                             | PASS |
------------------------------------------------------------------------------
Bug 84374 :: Regression test of Bug #84374 yaim-cream-ce: GlueFore... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================

Bug #86191 FIXED
$ pybot bug_86191.html
..........
==============================================================================
Bug 86191 :: Regression test of Bug #86191 No info published by the lcg-inf...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
bug_86191                                                             | PASS |
------------------------------------------------------------------------------
Bug 86191 :: Regression test of Bug #86191 No info published by th... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================

Bug #87361 FIXED
$ pybot bug_87361.html
..........
==============================================================================
Bug 87361 :: Regression test of bug #87361 The attribute cream_concurrency_...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
bug_87361                                                             | PASS |
------------------------------------------------------------------------------
Bug 87361 :: Regression test of bug #87361 The attribute cream_con... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================

Bug #87492 FIXED
$ pybot bug_87492.html
..........
==============================================================================
Bug 87492 :: Regression test of bug #87492 CREAM doesn't handle correctly t...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
bug_87492                                                             | PASS |
------------------------------------------------------------------------------
Bug 87492 :: Regression test of bug #87492 CREAM doesn't handle co... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================

Bug #89489 FIXED
$ pybot bug_89489.html
..........
==============================================================================
Bug 89489 :: Regression test of bug #89489 yaim plugin for CREAM CE does no...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
bug_89489                                                             | PASS |
------------------------------------------------------------------------------
Bug 89489 :: Regression test of bug #89489 yaim plugin for CREAM C... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================

Bug #95552 FIXED
$ pybot bug_95552.html
..........
==============================================================================
Bug 95552 :: Regression test of bug #83338 endpointType (in GLUE2ServiceCom...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
Check Executables :: Check whether the needed executables for the ... | PASS |
------------------------------------------------------------------------------
bug_95552                                                             | PASS |
------------------------------------------------------------------------------
Bug 95552 :: Regression test of bug #83338 endpointType (in GLUE2S... | PASS |
3 critical tests, 3 passed, 0 failed
3 tests total, 3 passed, 0 failed
==============================================================================

Bug #96512 FIXED
$ pybot bug_96512.html
..........
==============================================================================
Bug 96512 :: Regression test of bug #81561: Make JobDBAdminPurger script co...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
bug_96512                                                             | PASS |
------------------------------------------------------------------------------
Bug 96512 :: Regression test of bug #81561: Make JobDBAdminPurger ... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================

Bug #97441 FIXED
$ pybot bug_97441.html
..........
==============================================================================
Bug 97441 :: Regression test of bug #83338 endpointType (in GLUE2ServiceCom...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
Check Executables :: Check whether the needed executables for the ... | PASS |
------------------------------------------------------------------------------
bug_97441                                                             | PASS |
------------------------------------------------------------------------------
Bug 97441 :: Regression test of bug #83338 endpointType (in GLUE2S... | PASS |
3 critical tests, 3 passed, 0 failed
3 tests total, 3 passed, 0 failed
==============================================================================

Standard conformance tests

Glue 1 Compliance

  • glue-validator -t glue1 -h cert-41.pd.infn.it -p 2170 -b o=grid 
    log

Glue 2 Compliance

  • glue-validator -t glue2 -h cert-41.pd.infn.it -p 2170 -b o=glue 
    log

Performance and scalability tests

  • Submission with polling and sandbox transfer (LSF)
         ./bin/cream-test-monitored-submit -r 30 -n 1000 -m 50 -C 50 -l log4py.conf -j jdl/testsandbox.jdl -R cert-41.pd.infn.it:8443/cream-lsf-cert
         
    log
  • Submission and cancellation with polling (LSF)
         ./bin/cream-test-monitored-cancel -r 30 -n 1000 -m 50 -C 50 -l log4py.conf -j jdl/simple.jdl -R cert-41.pd.infn.it:8443/cream-lsf-cert
         
    log
  • Simple submission with polling, lease updated and proxy renewal (LSF)
         cream-test-monitored-lease-updated -r 60 -n 50 -m 10 -C 10 -l log4py.conf -W 1200 --vo dteam --valid 00:20 -R cert-41.pd.infn.it:8443/cream-lsf-cert
         
    log

Integration testing

Tests described:

-- SaraBertocco - 2013-01-25

Topic attachments
I AttachmentSorted ascending Action Size Date Who Comment
Unknown file formatlog bug_59871.html.log manage 164.2 K 2013-01-31 - 15:44 SaraBertocco  
Unknown file formatlog bug_69857.html.log manage 167.7 K 2013-01-31 - 15:44 SaraBertocco  
Unknown file formatlog bug_81561.html.log manage 162.7 K 2013-01-31 - 15:44 SaraBertocco  
Unknown file formatlog bug_83238.html.log manage 163.5 K 2013-01-31 - 15:44 SaraBertocco  
Unknown file formatlog bug_83338.html.log manage 165.0 K 2013-01-31 - 15:45 SaraBertocco  
Unknown file formatlog bug_83592.html.log manage 163.4 K 2013-01-31 - 15:47 SaraBertocco  
Unknown file formatlog bug_83593.html.log manage 163.1 K 2013-01-31 - 15:48 SaraBertocco  
Unknown file formatlog bug_83749.html.log manage 189.4 K 2013-01-31 - 15:48 SaraBertocco  
Unknown file formatlog bug_84374.html.log manage 160.0 K 2013-01-31 - 15:48 SaraBertocco  
Unknown file formatlog bug_86191.html.log manage 159.9 K 2013-01-31 - 15:49 SaraBertocco  
Unknown file formatlog bug_87361.html.log manage 183.6 K 2013-01-31 - 15:45 SaraBertocco  
Unknown file formatlog bug_87492.html.log manage 164.8 K 2013-01-31 - 15:45 SaraBertocco  
Unknown file formatlog bug_89489.html.log manage 170.1 K 2013-01-31 - 15:45 SaraBertocco  
Unknown file formatlog bug_95552.html.log manage 161.9 K 2013-01-31 - 15:45 SaraBertocco  
Unknown file formatlog bug_96512.html.log manage 162.6 K 2013-01-31 - 15:45 SaraBertocco  
Unknown file formatlog bug_97441.html.log manage 171.2 K 2013-01-31 - 15:45 SaraBertocco  
Unknown file formatlog bug_99072.html.log manage 161.1 K 2013-01-31 - 15:45 SaraBertocco  
Unknown file formatlog cream-test-monitored-cancel-cert-41.log manage 1046.6 K 2013-02-08 - 16:53 SaraBertocco  
Unknown file formatlog cream-test-monitored-lease-updated-cert-41.log manage 151.1 K 2013-02-10 - 18:26 SaraBertocco  
Unknown file formatlog cream-test-monitored-submit-cert-41.log manage 891.1 K 2013-02-08 - 16:53 SaraBertocco  
Unknown file formatlog functionality_CREAM_SL6_LSF.html.log manage 506.4 K 2013-01-29 - 10:02 SaraBertocco  
Unknown file formatlog functionality_CREAM_SL6_PBS.html.log manage 503.1 K 2013-01-29 - 13:04 SaraBertocco  
Unknown file formatlog glue1_compliance.log manage 6.4 K 2013-02-11 - 11:02 SaraBertocco  
Unknown file formatlog glue2_compliance.log manage 9.2 K 2013-02-11 - 11:02 SaraBertocco  
Unknown file formatout ldapsearch.glue2.out manage 214.6 K 2013-02-04 - 14:25 SaraBertocco  
Unknown file formatlog outputdata_LSF_SL6.html.log manage 173.9 K 2013-01-29 - 13:26 SaraBertocco  
Unknown file formatlog outputdata_PBS_SL6.html.log manage 177.6 K 2013-01-29 - 13:13 SaraBertocco  
Unknown file formatlog sandbox_staging_CREAM_SL6_LSF.html.log manage 170.4 K 2013-01-29 - 10:02 SaraBertocco  
Edit | Attach | PDF | History: r18 | r16 < r15 < r14 < r13 | Backlinks | Raw View | More topic actions...
Topic revision: r14 - 2013-02-11 - SaraBertocco
 
  • Edit
  • Attach
This site is powered by the TWiki collaboration platformCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback