Regression Test Work Plan
BLAH
Bug #75854
Problems related to the growth of the blah registry) Not implemented
Configure a CREAM CE using the new BLparser.
Verify that in /etc/blah.config there is:
job_registry_use_mmap=yes
(default scenario).
Submit 5000 jobs on a CREAM CE using the following JDL:
[
executable="/bin/sleep";
arguments="100";
]
Monitor the BLAH processed. Verify that each of them doesn't use more than 50 MB.
Bug #77776
(BUpdater should have an option to use cached batch system commands) Not implemented
Add:
lsf_batch_caching_enabled=yes
batch_command_caching_filter=/usr/bin/runcmd.p
in
/etc/blah.config
.
Create and fill
/usr/bin/runcmd.pl
with the following content:
#!/usr/bin/perl
#---------------------#
# PROGRAM: argv.pl #
#---------------------#
$numArgs = $#ARGV + 1;
open (MYFILE, '>>/tmp/xyz');
foreach $argnum (0 .. $#ARGV) {
print MYFILE "$ARGV[$argnum]\n";
}
close (MYFILE);
Submit some jobs. Check that in
/tmp/xyz
the queries to the batch system are recorded. E.g. for LSF something like that should be reported:
/opt/lsf/7.0/linux2.6-glibc2.3-x86/bin/bjobs
-u
all
-l
/opt/lsf/7.0/linux2.6-glibc2.3-x86/bin/bjobs
-u
all
-l
...
Bug #80805
(BLAH job registry permissions should be improved) Not implemented
Check permissions and ownership under
/var/blah
. They should be:
/var/blah:
total 12
-rw-r--r-- 1 tomcat tomcat 5 Oct 18 07:32 blah_bnotifier.pid
-rw-r--r-- 1 tomcat tomcat 5 Oct 18 07:32 blah_bupdater.pid
drwxrwx--t 4 tomcat tomcat 4096 Oct 18 07:38 user_blah_job_registry.bjr
/var/blah/user_blah_job_registry.bjr:
total 16
-rw-rw-r-- 1 tomcat tomcat 1712 Oct 18 07:38 registry
-rw-r--r-- 1 tomcat tomcat 260 Oct 18 07:38 registry.by_blah_index
-rw-rw-rw- 1 tomcat tomcat 0 Oct 18 07:38 registry.locktest
drwxrwx-wt 2 tomcat tomcat 4096 Oct 18 07:38 registry.npudir
drwxrwx-wt 2 tomcat tomcat 4096 Oct 18 07:38 registry.proxydir
-rw-rw-r-- 1 tomcat tomcat 0 Oct 18 07:32 registry.subjectlist
/var/blah/user_blah_job_registry.bjr/registry.npudir:
total 0
/var/blah/user_blah_job_registry.bjr/registry.proxydir:
total 0
Bug #81354
(Missing 'Iwd' Attribute when trasferring files with the 'TransferInput' attribute causes thread to loop) Not implemented
Log on a cream ce as user tomcat.
Create a proxy of yours and copy it as
/tmp/proxy
(change the ownership to tomcat.tomcat).
Create the file
/home/dteam001/dir1/fstab
(you can copy /etc/fstab).
Submit a job directly via blah (in the following change pbs and creamtest2 with the relevant batch system and queue names):
$ /usr/bin/blahpd
$GahpVersion: 1.16.2 Mar 31 2008 INFN\ blahpd\ (poly,new_esc_format) $
BLAH_SET_SUDO_ID dteam001
S Sudo\ mode\ on
blah_job_submit 1 [cmd="/bin/cp";Args="fstab\ fstab.out";TransferInput="/home/dteam001/dir1/fstab";TransferOutput="fstab.out";TransferOutputRemaps="fstab.out=/home/dteam001/dir1/fstab.out";gridtype="pbs";queue="creamtest2";x509userproxy="/tmp/proxy"]
S
results
S 1
1 0 No\ error pbs/20111010/304.cream-38.pd.infn.it
Eventually check the content of
/home/dteam001/dir1/
where you see both
fstab
and
fstab.out
:
$ ls /home/dteam001/dir1/
fstab fstab.out
Bug #81824
(yaim-cream-ce should manage the attribute bupdater_loop_interval) Not implemented
Set
BUPDATER_LOOP_INTERVAL
to 30 in siteinfo.def and reconfigure via yaim. Then verify that in
blah.config
there is:
bupdater_loop_interval=30
Bug #82281
(blahp.log records should always contain CREAM job ID) Not implement
Submit a job directly to CREAM using CREAM-CLI.
Then submit a job to CREAM through the WMS.
In the accounting log file (/var/log/cream/accounting/blahp.log-
) in both cases the clientID field should end with the numeric part of the CREAM jobid, e.g.:
"timestamp=2011-10-10 14:37:38" "userDN=/C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto" "userFQAN=/dteam/Role=NULL/Capability=NULL" "userFQAN=/dteam/NGI_IT/Role=NULL/Capability=NULL" "ceID=cream-38.pd.infn.it:8443/cream-pbs-creamtest2" "jobID=CREAM956286045" "lrmsID=300.cream-38.pd.infn.it" "localUser=18757" "clientID=cre38_956286045"
"timestamp=2011-10-10 14:39:57" "userDN=/C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto" "userFQAN=/dteam/Role=NULL/Capability=NULL" "userFQAN=/dteam/NGI_IT/Role=NULL/Capability=NULL" "ceID=cream-38.pd.infn.it:8443/cream-pbs-creamtest2" "jobID=https://devel19.cnaf.infn.it:9000/dLvm84LvD7w7QXtLZK4L0A" "lrmsID=302.cream-38.pd.infn.it" "localUser=18757" "clientID=cre38_315532638"
Bug #82297
(blahp.log rotation period is too short) Not implemented
Check that in /etc/logrotate.d/blahp-logrotate
rotate is equal to 365:
# cat /etc/logrotate.d/blahp-logrotate
/var/log/cream/accounting/blahp.log {
copytruncate
rotate 365
size = 10M
missingok
nomail
}
Bug #83275
(Problem in updater with very short jobs that can cause no notification to cream) Not implemented
Configure a CREAM CE using the old blparser.
Submit a job using the following JDL:
[
executable="/bin/echo";
arguments="ciao";
]
Check in the bnotifier log file (=/var/log/cream/glite-ce-bnotifier.log=_ that a lest a notification is sent for this job, e.g.:
2011-11-04 14:11:11 Sent for Cream:[BatchJobId="927.cream-38.pd.infn.it"; JobStatus=4; ChangeTime="2011-11-04 14:08:55"; JwExitCode=0; Reason="reason=0"; ClientJobId="622028514"; BlahJobName="cre38_622028514";]
Bug #83347
(Incorrect special character handling for BLAH Arguments and Environment attributes) Not implemented
Log on a cream ce as user tomcat.
Create a proxy of yours and copy it as /tmp/proxy
(change the ownership to tomcat.tomcat).
Create the file /home/dteam001/dir1/fstab
(you can copy /etc/fstab).
Submit a job directly via blah (in the following change pbs and creamtest1 with the relevant batch system and queue names):
BLAH_JOB_SUBMIT 1 [Cmd="/bin/echo";Args="$HOSTNAME";Out="/tmp/stdout_l15367";In="/dev/null";GridType="pbs";Queue="creamtest1";x509userproxy="/tmp/proxy";Iwd="/tmp";TransferOutput="output_file";TransferOutputRemaps="output_file=/tmp/stdout_l15367";GridResource="blah"]
Verify that in the output file there is the hostname of the WN.
Bug #87419
(blparser_master add some spurious character in the BLParser command line) Not implemented
Configure a CREAM CE using the old blparser. Check the blparser process using ps. It shouldn't show urious characters:
root 26485 0.0 0.2 155564 5868 ? Sl 07:36 0:00 /usr/bin/BLParserPBS -d 1 -l /var/log/cream/glite-pbsparser.log -s /var/torque -p 33333 -m 56565
CREAM
Bug #69857
Job submission to CreamCE is enabled by restart of service even if it was previously disabled - Implemented
STATUS: Implemented
To test the fix:
- disable the submission on the CE
This can be achieved via the `glite-ce-disable-submission host:port`
command (provided by the CREAM CLI package installed on the UI), that can be issued only by a CREAM CE administrator, that is the DN of this person must be listed in the /etc/grid-security/admin-list
file of the CE.
Output should be: "Operation for disabling new submissions succeeded"
- restart tomcat on the CREAM CE (service tomcat restart - on CE)
- verify if the submission is disabled (glite-ce-allowed-submission)
This can be achieved via the `glite-ce-enable-submission host:port`
command (provided by the CREAM CLI package installed on the UI).
Output should be: "Job submission to this CREAM CE is enabled"
Bug #81561
Make JobDBAdminPurger script compliant with CREAM EMI environment. - Implemented
STATUS: Implemented
To test the fix, simply run on the CREAM CE as root the JobDBAdminPurger.sh. E.g.:
# JobDBAdminPurger.sh -c /etc/glite-ce-cream/cream-config.xml -u <user> -p <passwd> -s DONE-FAILED,0
START jobAdminPurger
It should work without reporting error messages:
-----------------------------------------------------------
Job CREAM595579358 is going to be purged ...
- Job deleted. JobId = CREAM595579358
CREAM595579358 has been purged!
-----------------------------------------------------------
STOP jobAdminPurger
Bug #81824
yaim-cream-ce should manage the attribute bupdater_loop_interval. - Implemented
STATUS: Implemented
To test the fix, check in the CREAM CE if the file /etc/blah.config contains the parameter bupdater_loop_interval.
Bug #83238
Sometimes CREAM does not update the state of a failed job. - Implemented
STATUS: Implemented
To test the fix, try to kill by hand a job.
The status of the job should eventually be:
Status = [DONE-FAILED]
ExitCode = [N/A]
FailureReason = [Job has been terminated (got SIGTERM)]
Bug #83749
JobDBAdminPurger cannot purge jobs if configured sandbox dir has changed. - Not implemented
STATUS: Not implemented
To test the fix, submit some jobs and then reconfigure the service with a different value of CREAM_SANDBOX_PATH
. Then try, with the JobDBAdminPurger.sh
script, to purge some jobs submitted before the switch.
It must be verified:
- that the jobs have been purged from the CREAM DB (i.e. a
glite-ce-job-status
should not find them anymore)
- that the relevant CREAM sandbox directories have been deleted
STATUS: Implemented
To test the fix, query the resource bdii of the CREAM-CE:
ldapsearch -h <CREAM CE host> -x -p 2170 -b "o=grid" | grep -i foreignkey | grep -i glueceuniqueid
Entries such as:
GlueForeignKey: GlueCEUniqueID=cream-35.pd.infn.it:8443/cream-lsf-creamtest1
i.e.:
GlueForeignKey: GlueCEUniqueID=<CREAM CE ID>
should appear.
Bug #86191
No info published by the lcg-info-dynamic-scheduler for one VOView - Implemented
STATUS: Implemented
To test the fix, issue the following ldapsearch query towards the resource bdii of the CREAM-CE:
$ ldapsearch -h cream-35 -x -p 2170 -b "o=grid" | grep -i GlueCEStateWaitingJobs | grep -i 444444
It should not find anything
Bug #87361
The attribute cream_concurrency_level should be configurable via yaim. - Implemented
STATUS: Implemented
To test the fix, set in seiteinfo.def
the variable CREAM_CONCURRENCY_LEVEL
to a certain number (n). After configuration verify that in /etc/glite-ce-cream/cream-config.xml
there is:
cream_concurrency_level="n"
Bug #87492
CREAM doesn't handle correctly the jdl attribute "environment". - Not implemented
STATUS: Not implemented
To test the fix, submit the following JDL using glite-ce-job-submit
:
Environment = {
"GANGA_LCG_VO='camont:/camont/Role=lcgadmin'",
"LFC_HOST='lfc0448.gridpp.rl.ac.uk'",
"GANGA_LOG_HANDLER='WMS'"
};
executable="/bin/env";
stdoutput="out.out";
outputsandbox={"out.out"};
outputsandboxbasedesturi="gsiftp://localhost";
When the job is done, retrieve the output and check that in out.out
the variables GANGA_LCG_VO
, LFC_HOST
and GANGA_LOG_HANDLER
have exactly the values defined in the JDL.
-- MassimoSgaravatto - 2011-11-07