Testing report: IGIRTC-56

Summary

• Product: BLAH 1.16.5
• Release Task: Task #27056
• ETICS Subsystem Configuration Name: emi-cream-ce_R_1_13_8_2
• VCS Tag: emi-blahp_R_1_16_5_2, glite-yaim-cream-ce_R_4_2_4_1
• EMI Major Release: EMI 1 (Kebnekaise)
• Platform: SL5 epel
• Author:
• Testing report:
• Certification report:
• Outcome: "Certified*

Deployment tests

Clean Installation

• Installation log file
• Configuration log file The first configuration attempt fails because pbs server does not start, it must be started manually and then re-configured.

Upgrade Installation

LSF CE

• Installation log file
• Configuration log file (using new BLParser model)
• Configuration log file (using old BLParser model)

PBS CE

• Installation log file
• Configuration log file (using new BLParser model)
• Configuration log file (using old BLParser model)

(S)GE CE

• Installation log file
• Configuration log file (using new BLParser model)

Unit Tests

Not Available. The plan is to provide some unit tests starting with EMI-2.

System tests

Functionality tests

Test submission

• Test result for LSF is available here PASSED
• Test result for PBS is available here PASSED
• Test result for SGE is available here PASSED with WARNING \The Job Cancel Test failed. Anaklysis of problm: it is a delay in state detection by bnotifier, sending a job sleeping for 10min it finishes in DONE-OK:

cat test.jdl
[
executable="/bin/sleep" ;
arguments = "600";
stdoutput="out3.out";
stderror="err3.err";
]

$ glite-ce-job-submit -a -r emitestbed21.cnaf.infn.it:8443/cream-sge-emitesters  -a /home/bertocco/gridka.jdl
https://emitestbed21.cnaf.infn.it:8443/CREAM499803862

$ glite-ce-job-cancel https://emitestbed21.cnaf.infn.it:8443/CREAM499803862

Are you sure you want to cancel specified job(s) [y/n]: y

$ glite-ce-job-status https://emitestbed21.cnaf.infn.it:8443/CREAM499803862

******  JobID=[https://emitestbed21.cnaf.infn.it:8443/CREAM499803862]
   Status        = [CANCELLED]
   ExitCode      = []
   Description   = [Cancelled by user]

Checking the batch system another strange thing: the job seems to remain in the batch system and finish regularly in 10min, after cancellation:

# qstat -u '*'
job-ID  prior   name       user         state submit/start at     queue                          slots ja-task-ID 
-----------------------------------------------------------------------------------------------------------------
    105 0.55500 cream_4998 tst22        r     03/27/2012 17:34:06 emitesters@emitestbed13.cnaf.i     1        

# qstat -j 105
==============================================================
job_number:                 105
exec_file:                  job_scripts/105
submission_time:            Tue Mar 27 17:33:57 2012
owner:                      tst22
uid:                        61022
group:                      testers
gid:                        6100
sge_o_home:                 /home/tst22
sge_o_log_name:             tst22
sge_o_path:                 /opt/SGE/bin/linux-x64:/usr/bin:/bin
sge_o_shell:                /bin/sh
sge_o_workdir:              /var/tmp
sge_o_host:                 emitestbed21
account:                    sge
mail_list:                  tst22@emitestbed21.cnaf.infn.it
notify:                     FALSE
job_name:                   cream_499803862
jobshare:                   0
hard_queue_list:            emitesters
shell_list:                 NONE:/bin/bash
env_list:                   SGE_stagein=CREAM499803862_jobWrapper.sh.61022.6563.1332862437@emitestbed21.cnaf.infn.it:/var/cream_sandbox/testers/_C_IT_O_INFN_OU_Personal_Certificate_L_Padova_CN_Sara_Bertocco_testers_eu_emi_eu_Role_NULL_Capability_NULL_tst22/49/CREAM499803862/CREAM499803862_jobWrapper.sh@@@cream_499803862.proxy@emitestbed21.cnaf.infn.it:/var/cream_sandbox/testers/_C_IT_O_INFN_OU_Personal_Certificate_L_Padova_CN_Sara_Bertocco_testers_eu_emi_eu_Role_NULL_Capability_NULL_tst22/proxy/eef209cdc40518c452fc0ad35b03b9a2ba1d519b11141130366494,SGE_stageout=out_cream_499803862_StandardOutput@emitestbed21.cnaf.infn.it:/var/cream_sandbox/testers/_C_IT_O_INFN_OU_Personal_Certificate_L_Padova_CN_Sara_Bertocco_testers_eu_emi_eu_Role_NULL_Capability_NULL_tst22/49/CREAM499803862/StandardOutput@@@err_cream_499803862_StandardError@emitestbed21.cnaf.infn.it:/var/cream_sandbox/testers/_C_IT_O_INFN_OU_Personal_Certificate_L_Padova_CN_Sara_Bertocco_testers_eu_emi_eu_Role_NULL_Capability_NULL_tst22/49/CREAM499803862/StandardError
script_file:                /tmp/cream_499803862
usage    1:                 cpu=00:00:00, mem=0.00448 GBs, io=0.00110, vmem=404.660M, maxvmem=404.660M
scheduling info:            queue instance "emitesters@emitestbed13.cnaf.infn.it" dropped because it is full

# qacct -j 105
==============================================================
qname        emitesters          
hostname     emitestbed13.cnaf.infn.it
group        testers             
owner        tst22               
project      NONE                
department   defaultdepartment   
jobname      cream_499803862     
jobnumber    105                 
taskid       undefined
account      sge                 
priority     0                   
qsub_time    Tue Mar 27 17:33:57 2012
start_time   Tue Mar 27 17:34:07 2012
end_time     Tue Mar 27 17:44:08 2012
granted_pe   NONE                
slots        1                   
failed       0    
exit_status  0                   
ru_wallclock 601          
ru_utime     0.615        
ru_stime     0.578        
ru_maxrss    2788                
ru_ixrss     0                   
ru_ismrss    0                   
ru_idrss     0                   
ru_isrss     0                   
ru_minflt    43062               
ru_majflt    0                   
ru_nswap     0                   
ru_inblock   0                   
ru_oublock   0                   
ru_msgsnd    0                   
ru_msgrcv    0                   
ru_nsignals  0                   
ru_nvcsw     270                 
ru_nivcsw    344                 
cpu          1.193        
mem          0.004             
io           0.001             
iow          0.000             
maxvmem      404.660M
arid         undefined

BLParser test

Old BLParser

• Test results for LSF PASSED

Tests.Check Notifications For Cancelled Jobs :: Test that notifications are...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_cancelled_jobs                                | PASS |
------------------------------------------------------------------------------
Tests.Check Notifications For Cancelled Jobs :: Test that notifica... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Tests.Check Notifications For Normally Finished Jobs :: Test that notificat...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_normally_finished_jobs                        | PASS |
------------------------------------------------------------------------------
Tests.Check Notifications For Normally Finished Jobs :: Test that ... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Tests.Check Notifications For Suspended Resumed Jobs :: Test that notificat...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_suspended_resumed_jobs                        | FAIL |
_error: Expected status should be in ['HELD'] for job https://cream-29.pd.infn.it:8443/CREAM705156955 was actually IDLE
------------------------------------------------------------------------------
Tests.Check Notifications For Suspended Resumed Jobs :: Test that ... | FAIL |
2 critical tests, 1 passed, 1 failed
2 tests total, 1 passed, 1 failed
==============================================================================
Tests                                                                 | FAIL |
6 critical tests, 5 passed, 1 failed
6 tests total, 5 passed, 1 failed
==============================================================================
Output:  /home/ale/blah/italiangrid-cream_blah_testsuites-09156c3_ver2/output.xml
Log:     /home/ale/blah/italiangrid-cream_blah_testsuites-09156c3_ver2/log.html
Report:  /home/ale/blah/italiangrid-cream_blah_testsuites-09156c3_ver2/report.html

• results for PBS PASSED

Tests.Check Notifications For Cancelled Jobs :: Test that notifications are...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_cancelled_jobs                                | PASS |
------------------------------------------------------------------------------
Tests.Check Notifications For Cancelled Jobs :: Test that notifica... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Tests.Check Notifications For Normally Finished Jobs :: Test that notificat...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_normally_finished_jobs                        | PASS |
------------------------------------------------------------------------------
Tests.Check Notifications For Normally Finished Jobs :: Test that ... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Tests.Check Notifications For Suspended Resumed Jobs :: Test that notificat...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_suspended_resumed_jobs                        | FAIL |
_error: Expected status should be in ['HELD'] for job https://cream-41.pd.infn.it:8443/CREAM324312963 was actually DONE-OK
------------------------------------------------------------------------------
Tests.Check Notifications For Suspended Resumed Jobs :: Test that ... | FAIL |
2 critical tests, 1 passed, 1 failed
2 tests total, 1 passed, 1 failed
==============================================================================
Tests                                                                 | FAIL |
6 critical tests, 5 passed, 1 failed
6 tests total, 5 passed, 1 failed
==============================================================================

• Job which is suspended and then resumed
  • LSF NOT PASSED see Bug #90085
  • PBS NOT PASSED see Bug #90085

New BLParser

• Test results for LSF PASSED

Tests.Check Notifications For Cancelled Jobs :: Test that notifications are...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_cancelled_jobs                                | PASS |
------------------------------------------------------------------------------
Tests.Check Notifications For Cancelled Jobs :: Test that notifica... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Tests.Check Notifications For Normally Finished Jobs :: Test that notificat...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_normally_finished_jobs                        | PASS |
------------------------------------------------------------------------------
Tests.Check Notifications For Normally Finished Jobs :: Test that ... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Tests.Check Notifications For Suspended Resumed Jobs :: Test that notificat...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_suspended_resumed_jobs                        | PASS |
------------------------------------------------------------------------------
Tests.Check Notifications For Suspended Resumed Jobs :: Test that ... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Tests                                                                 | PASS |
6 critical tests, 6 passed, 0 failed
6 tests total, 6 passed, 0 failed
==============================================================================

• Test results for PBS PASSED

Tests.Check Notifications For Cancelled Jobs :: Test that notifications are...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_cancelled_jobs                                | PASS |
------------------------------------------------------------------------------
Tests.Check Notifications For Cancelled Jobs :: Test that notifica... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Tests.Check Notifications For Normally Finished Jobs :: Test that notificat...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_normally_finished_jobs                        | PASS |
------------------------------------------------------------------------------
Tests.Check Notifications For Normally Finished Jobs :: Test that ... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Tests.Check Notifications For Suspended Resumed Jobs :: Test that notificat...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
check_notifications_for_suspended_resumed_jobs                        | PASS |
------------------------------------------------------------------------------
Tests.Check Notifications For Suspended Resumed Jobs :: Test that ... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Tests                                                                 | PASS |
6 critical tests, 6 passed, 0 failed
6 tests total, 6 passed, 0 failed
==============================================================================

Regression tests

Verification attached bugs

Bug #89527: BLAHP produced -W stage(in/out) directives are incompatible with Torque 2.5.8 FIXED

Content of file to check here.

Bug #91037: BUpdaterLSF should use bjobs to detect final job state FIXED

• Change debug_level and restart the services:

[root@cream-29 ~]# sed -i 's/bupdater_debug_level=2/bupdater_debug_level=3/' /etc/blah.config 
[root@cream-29 ~]# mv /var/log/cream/glite-ce-bupdater.log /var/log/cream/glite-ce-bupdater.log.old
[root@cream-29 ~]# service gLite restart
STOPPING SERVICES
*** glite-ce-blahparser:
Shutting down BNotifier:                                   [  OK  ]
Shutting down BUpdaterLSF:                                 [  OK  ]

*** glite-lb-locallogger:
Stopping glite-lb-logd ... done
Stopping glite-lb-interlogd ... done

*** tomcat5:
Stopping tomcat5:                                          [  OK  ]

STARTING SERVICES
*** tomcat5:
Starting tomcat5:                                          [  OK  ]

*** glite-lb-locallogger:
Starting glite-lb-logd ...This is LocalLogger, part of Workload Management System in EU DataGrid & EGEE.
 done
Starting glite-lb-interlogd ... done

*** glite-ce-blahparser:
Starting BNotifier: /usr/bin/BNotifier: Error creating and binding socket: Address already in use
                                                           [FAILED]
Starting BUpdaterLSF:                                      [  OK  ]

• Submit a job and wait for its completation:

[ale@cream-12 UI]$ glite-ce-job-submit -a -r cream-29.pd.infn.it:8443/cream-lsf-cert cream.jdl
https://cream-29.pd.infn.it:8443/CREAM239301025
[ale@cream-12 UI]$ glite-ce-job-status https://cream-29.pd.infn.it:8443/CREAM239301025

******  JobID=[https://cream-29.pd.infn.it:8443/CREAM239301025]
   Status        = [DONE-OK]
   ExitCode      = [0]

[root@cream-29 ~]# grep 239301025 /var/log/cream/glite-ce-bnotifier.log
2012-03-22 17:05:36 Sent for Cream:[BatchJobId="622199"; JobStatus=4; ChangeTime="2012-03-22 17:05:22"; JwExitCode=0; Reason="reason=0"; ClientJobId="239301025"; BlahJobName="cre29_239301025";]

• Verify if bhist has been called:

[root@cream-29 ~]# grep bhist /var/log/cream/glite-ce-bupdater.log
2012-03-22 17:00:43 /usr/bin/BUpdaterLSF: key bupdater_use_bhist_time_constraint not found - using the default:no
2012-03-22 17:00:43 /usr/bin/BUpdaterLSF: key bupdater_use_bhist_for_killed not found - using the default:no
2012-03-22 17:01:26 /usr/bin/BUpdaterLSF: key bupdater_use_bhist_time_constraint not found - using the default:no
2012-03-22 17:01:26 /usr/bin/BUpdaterLSF: key bupdater_use_bhist_for_killed not found - using the default:no
2012-03-22 17:03:54 /usr/bin/BUpdaterLSF: key bupdater_use_bhist_time_constraint not found - using the default:no
2012-03-22 17:03:54 /usr/bin/BUpdaterLSF: key bupdater_use_bhist_for_killed not found - using the default:no
2012-03-22 17:04:51 /usr/bin/BUpdaterLSF: key bupdater_use_bhist_time_constraint not found - using the default:no
2012-03-22 17:04:51 /usr/bin/BUpdaterLSF: key bupdater_use_bhist_for_killed not found - using the default:no

Bug #92281: Purge of registry can cause registry corruption FIXED not certified

It is not possible to replicate the problem sistematically.

Bug #92774: BLParserLSF could crash searching in old logs FIXED not certified

Using the old LSF BLParser it usually crashed, so the fact that the functional tests passed should be a good signal that the bug has been fixed.

Verification old bugs

• Bug #75854 Problems related to the growth of the blah registry)

Submitted 5000 jobs to a CREAM CE configured using the new blparser, and with job_registry_use_mmap=yes.

Monitored the used RSS of the blahpd processes. At the end the maximum value between all the process is 4560.

Test PASSED

• Bug #77776 (BUpdater should have an option to use cached batch system commands)

Configure /etc/blah.config:

[root@cream-29 ~]# tail -4 /etc/blah.config
# Verify fix for bug #77776
lsf_batch_caching_enabled=yes
batch_command_caching_filter=/usr/bin/runcmd.pl

Where runcmd.pl is:

#!/usr/bin/perl
#---------------------#
#  PROGRAM:  argv.pl  #
#---------------------#

$numArgs = $#ARGV + 1;
open (MYFILE, '>>/tmp/xyz');
foreach $argnum (0 .. $#ARGV) {
    print MYFILE "$ARGV[$argnum] ";
}
print MYFILE "\n";
close (MYFILE); 

Restart the services and submit 10 jobs to the CE.

[root@cream-29 cream]# cat /tmp/xyz 
/opt/lsf/7.0/linux2.6-glibc2.3-x86_64/bin/bjobs -u all -l -a 
/opt/lsf/7.0/linux2.6-glibc2.3-x86_64/bin/bhist -u all -d -l -n 10 
/opt/lsf/7.0/linux2.6-glibc2.3-x86_64/bin/bjobs -u all -l -a 
/opt/lsf/7.0/linux2.6-glibc2.3-x86_64/bin/bhist -u all -d -l -n 10 
/opt/lsf/7.0/linux2.6-glibc2.3-x86_64/bin/bjobs -u all -l -a 
/opt/lsf/7.0/linux2.6-glibc2.3-x86_64/bin/bhist -u all -d -l -n 10 
/opt/lsf/7.0/linux2.6-glibc2.3-x86_64/bin/bjobs -u all -l -a 
/opt/lsf/7.0/linux2.6-glibc2.3-x86_64/bin/bhist -u all -d -l -n 10 

Test PASSED

• Bug #80805 (BLAH job registry permissions should be improved)

[root@cream-29 cream]# ls -l /var/blah
total 8
-rw-r--r-- 1 tomcat tomcat    0 Mar 23 15:49 blah_bnotifier.pid
-rw-r--r-- 1 tomcat tomcat    4 Mar 23 15:53 blah_bupdater.pid
drwxrwx--t 4 tomcat tomcat 4096 Mar 23 16:07 user_blah_job_registry.bjr
[root@cream-29 cream]#  ls -l /var/blah/user_blah_job_registry.bjr/
total 14388
-rw-rw-r-- 1 tomcat tomcat 11377096 Mar 23 17:04 registry
-rw-r--r-- 1 tomcat tomcat  3066960 Mar 23 16:07 registry.by_blah_index
-rw-rw-rw- 1 tomcat tomcat        0 Mar 23 17:04 registry.locktest
drwxrwx-wt 2 tomcat tomcat     4096 Mar 23 17:04 registry.npudir
drwxrwx-wt 2 tomcat tomcat   253952 Mar 23 17:04 registry.proxydir
-rw-r--r-- 1 tomcat tomcat       99 Mar 23 15:49 registry.subjectlist
[root@cream-29 cream]# ls -l /var/blah/user_blah_job_registry.bjr/registry.npudir
total 8
-rw-rw-r-- 1 dteam017 dteam 856 Mar 23 17:04 npu_d4X1Ao
-rw-rw-r-- 1 dteam017 dteam 856 Mar 23 17:04 npu_jZXJGF
[root@cream-29 cream]# ls -l /var/blah/user_blah_job_registry.bjr/registry.proxydir/ | head -10
total 2448
lrwxrwxrwx 1 dteam017 dteam 198 Mar 23 16:41 proxy_637551_RUGJ92 -> /var/glite/cream_sandbox/dteam/_C_IT_O_INFN_OU_Personal_Certificate_L_Padova_CN_Alessio_Gianelle_dteam_Role_NULL_Capability_NULL_dteam017/proxy/f9099a3af228b82c323e26c1f9f494aefdd1b43910396930026260
lrwxrwxrwx 1 dteam017 dteam 198 Mar 23 16:41 proxy_637552_LrF5tC -> /var/glite/cream_sandbox/dteam/_C_IT_O_INFN_OU_Personal_Certificate_L_Padova_CN_Alessio_Gianelle_dteam_Role_NULL_Capability_NULL_dteam017/proxy/23a5e3de27a7d82748cabae2d5dccec329853fc610396930026260
lrwxrwxrwx 1 dteam017 dteam 198 Mar 23 16:41 proxy_637553_WvebYm -> /var/glite/cream_sandbox/dteam/_C_IT_O_INFN_OU_Personal_Certificate_L_Padova_CN_Alessio_Gianelle_dteam_Role_NULL_Capability_NULL_dteam017/proxy/751c2ef4ec29986f4bf63a602fbe626d7d1b3cab10396930026260
lrwxrwxrwx 1 dteam017 dteam 198 Mar 23 16:41 proxy_637554_4vdiJs -> /var/glite/cream_sandbox/dteam/_C_IT_O_INFN_OU_Personal_Certificate_L_Padova_CN_Alessio_Gianelle_dteam_Role_NULL_Capability_NULL_dteam017/proxy/4966974580d533d4d42a4635158382566ec6c2f610396930026260
lrwxrwxrwx 1 dteam017 dteam 198 Mar 23 16:41 proxy_637555_QX5Bi6 -> /var/glite/cream_sandbox/dteam/_C_IT_O_INFN_OU_Personal_Certificate_L_Padova_CN_Alessio_Gianelle_dteam_Role_NULL_Capability_NULL_dteam017/proxy/785cbd414da9ee5e79b633477b59a86071dcf50f10396930026260
lrwxrwxrwx 1 dteam017 dteam 198 Mar 23 16:41 proxy_637556_jTHBm1 -> /var/glite/cream_sandbox/dteam/_C_IT_O_INFN_OU_Personal_Certificate_L_Padova_CN_Alessio_Gianelle_dteam_Role_NULL_Capability_NULL_dteam017/proxy/df2c50573906838f8894b361c2a14860c3e4f16210396930026260
lrwxrwxrwx 1 dteam017 dteam 198 Mar 23 16:41 proxy_637557_DbWTbB -> /var/glite/cream_sandbox/dteam/_C_IT_O_INFN_OU_Personal_Certificate_L_Padova_CN_Alessio_Gianelle_dteam_Role_NULL_Capability_NULL_dteam017/proxy/6fd59e85358187345a4635505c9af7fe32ec61a310396930026260
lrwxrwxrwx 1 dteam017 dteam 198 Mar 23 16:41 proxy_637558_hhAHKX -> /var/glite/cream_sandbox/dteam/_C_IT_O_INFN_OU_Personal_Certificate_L_Padova_CN_Alessio_Gianelle_dteam_Role_NULL_Capability_NULL_dteam017/proxy/ced55ce67190ef6b2d8c56907926aed2f1a4ddfd10396930026260
lrwxrwxrwx 1 dteam017 dteam 198 Mar 23 16:41 proxy_637559_WjeEfP -> /var/glite/cream_sandbox/dteam/_C_IT_O_INFN_OU_Personal_Certificate_L_Padova_CN_Alessio_Gianelle_dteam_Role_NULL_Capability_NULL_dteam017/proxy/1d197ae0e6d8681258517525f8fc568b56bdf17810396930026260

Test PASSED

• Bug #81354 (Missing 'Iwd' Attribute when trasferring files with the 'TransferInput' attribute causes thread to loop)

[root@cream-29 cream]# su - tomcat
-sh-3.2$ chown tomcat.tomcat /tmp/proxy 
-sh-3.2$ ls -l /tmp/proxy 
-rw------- 1 tomcat tomcat 6501 Mar 26 10:52 /tmp/proxy
        
-sh-3.2$ /usr/bin/blahpd
$GahpVersion: 1.16.5 Mar 31 2008 INFN\ blahpd\ (poly,new_esc_format) $
BLAH_SET_SUDO_ID dteam001
S Sudo\ mode\ on
blah_job_submit 1 [cmd="/bin/cp";Args="fstab\ fstab.out";TransferInput="/home/dteam001/dir1/fstab";TransferOutput="fstab.out";TransferOutputRemaps="fstab.out=/home/dteam001/dir1/fstab.out";gridtype="lsf";queue="cert";x509userproxy="/tmp/proxy"]
S
results
S 1
1 0 No\ error lsf/20120326/642806
Connection closed by remote host


[root@cream-29 cream]# ls -l /home/dteam001/dir1/ 
total 8
-rw-r--r-- 1 dteam001 dteam 618 Mar 23 17:11 fstab
-rw-r--r-- 1 dteam001 dteam 618 Mar 26 10:54 fstab.out

Test PASSED

• Bug #81824: yaim-cream-ce should manage the attribute bupdater_loop_interval

Content of report file here.

==============================================================================
Bug 81824 :: Regression test of bug #81824 yaim-cream-ce should manage the ...
==============================================================================
Set Log Level :: Set the log level used for the test suite. This c... | PASS |
------------------------------------------------------------------------------
bug_81824                                                             | PASS |
------------------------------------------------------------------------------
Bug 81824 :: Regression test of bug #81824 yaim-cream-ce should ma... | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================

Test PASSED

• Bug #82281 (blahp.log records should always contain CREAM job ID)

Try first with direct submission:

[ale@cream-12 UI]$ glite-ce-job-submit -a -r cream-29.pd.infn.it:8443/cream-lsf-creamtest1 cream.jdl
https://cream-29.pd.infn.it:8443/CREAM499441859

[root@cream-29 cream]# grep CREAM499441859  /var/log/cream/accounting/blahp.log-20120326 
"timestamp=2012-03-26 09:00:36" "userDN=/C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle" "userFQAN=/dteam/Role=NULL/Capability=NULL" "userFQAN=/dteam/NGI_IT/Role=NULL/Capability=NULL" "ceID=cream-29.pd.infn.it:8443/cream-lsf-creamtest1" "jobID=CREAM499441859" "lrmsID=642810" "localUser=18239" "clientID=cre29_499441859"

Try submission through a WMS:

[ale@cream-12 UI]$ glite-wms-job-submit -a -e https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server -r cream-29.pd.infn.it:8443/cream-lsf-creamtest1 test.jdl 

Connecting to the service https://emi-demo11.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://emi-demo11.cnaf.infn.it:9000/kKD1V4dLlSdALEzPA_1qxg

==========================================================================



[ale@cream-12 UI]$ glite-wms-job-logging-info -v 2 --event Transfer  https://emi-demo11.cnaf.infn.it:9000/kKD1V4dLlSdALEzPA_1qxg  | grep "Dest jobid"
- Dest jobid                 =    unavailable
- Dest jobid                 =    https://cream-29.pd.infn.it:8443/CREAM581241790


[root@cream-29 cream]# grep kKD1V4dLlSdALEzPA_1qxg  /var/log/cream/accounting/blahp.log-20120326 
"timestamp=2012-03-26 09:05:37" "userDN=/C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle" "userFQAN=/dteam/Role=NULL/Capability=NULL" "userFQAN=/dteam/NGI_IT/Role=NULL/Capability=NULL" "ceID=cream-29.pd.infn.it:8443/cream-lsf-creamtest1" "jobID=https://emi-demo11.cnaf.infn.it:9000/kKD1V4dLlSdALEzPA_1qxg" "lrmsID=642811" "localUser=18239" "clientID=cre29_581241790"

Test PASSED

• Bug #82297 (blahp.log rotation period is too short)

[root@cream-29 cream]#  cat /etc/logrotate.d/blahp-logrotate | grep rotate
        rotate 365

Test PASSED

• Bug #83275 (Problem in updater with very short jobs that can cause no notification to cream)

[ale@cream-12 UI]$ glite-ce-job-submit -a -r cream-29.pd.infn.it:8443/cream-lsf-creamtest1 cream.jdl
https://cream-29.pd.infn.it:8443/CREAM670118061

[root@cream-29 cream]# grep 670118061  /var/log/cream/glite-ce-bnotifier.log
2012-03-26 11:12:00 Sent for Cream:[BatchJobId="642813"; JobStatus=2; ChangeTime="2012-03-26 11:11:57"; WorkerNode="prod-wn-001"; ClientJobId="670118061"; BlahJobName="cre29_670118061";]
2012-03-26 11:12:20 Sent for Cream:[BatchJobId="642813"; JobStatus=4; ChangeTime="2012-03-26 11:12:08"; JwExitCode=0; Reason="reason=0"; ClientJobId="670118061"; BlahJobName="cre29_670118061";]

Test PASSED

• Bug #83347 (Incorrect special character handling for BLAH Arguments and Environment attributes)

[root@cream-29 cream]# su - tomcat
-sh-3.2$  /usr/bin/blahpd
$GahpVersion: 1.16.5 Mar 31 2008 INFN\ blahpd\ (poly,new_esc_format) $
BLAH_SET_SUDO_ID dteam001
S Sudo\ mode\ on
BLAH_JOB_SUBMIT 1 [Cmd="/bin/echo";Args="$HOSTNAME";Out="/tmp/stdout_l15367";In="/dev/null";GridType="lsf";Queue="creamtest1";x509userproxy="/tmp/proxy";Iwd="/tmp";TransferOutput="output_file";TransferOutputRemaps="output_file=/tmp/stdout_l15367";GridResource="blah"]
S
results
S 1
1 0 No\ error lsf/20120326/642815
Connection closed by remote host
-sh-3.2$  cat /tmp/stdout_l15367 
prod-wn-001
-sh-3.2$ logout

[root@cream-29 cream]# bhist -w -l 642815 | grep Dispatched
Mon Mar 26 11:18:42: Dispatched to <prod-wn-001>;

Test PASSED

• Bug #87419 (blparser_master add some spurious character in the BLParser command line)

[root@cream-29 ~]# ps ax | grep BLParserLSF
  754 pts/1    S+     0:00 grep BLParserLSF
31468 ?        Sl     0:00 /usr/bin/BLParserLSF -d 1 -l /var/log/cream/glite-lsfparser.log -b /opt/lsf/7.0/linux2.6-glibc2.3-x86_64/bin -c /opt/lsf/conf -p 33333 -m 56565

Test PASSED

• Bug #89859: There is a memory leak in the updater for LSF, PBS and Condor

Submit 1000 jobs, one every 3 seconds monitoring the Used RSS memory of the /usr/bin/BUpdaterLSF process:

Submit 1000 jobs, one every 3 seconds monitoring the Used RSS memory of the /usr/bin/BUpdaterPBS process:

Test PASSED