(r4) WmsBugs3dot1dot100 < EgeeJra1It

Tags: , view all tags
%TOC%

---++ glite_wms_R_3_1_100
---+++ WMPROXY

   * Submitting a "parametric" jobs it always failed with this error:
<verbatim>
*************************************************************
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel17.cnaf.infn.it:9000/wzOz1QuyFAEJ5JnwXHUhng
Current Status:     Aborted
Status Reason:      requirements: unable to complete the operation: the attribute has not been initialised yet
Submitted:          Wed Jun 25 16:25:03 2008 CEST
*************************************************************
</verbatim>

   * The status of a collection (or a dag) is corretly set to CLEARED after a "glite-wms-get-output", but its nodes remain in "Done  (Success)" status.
<verbatim>
*************************************************************
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel17.cnaf.infn.it:9000/xe-J0mjeFjU86p2mstn8-A
Current Status:     Cleared
Status Reason:      user retrieved output sandbox
Destination:        dagman
Submitted:          Wed Jun 25 16:08:43 2008 CEST
*************************************************************

- Nodes information for:
    Status info for the Job : https://devel17.cnaf.infn.it:9000/6NeMTtefAmTCRI0N1waKzA
    Current Status:     Done (Success)
    Logged Reason(s):
        - Job terminated successfully
    Exit code:          0
    Status Reason:      Job terminated successfully
    Destination:        ce-01.roma3.infn.it:2119/jobmanager-lcgpbs-cert
    Submitted:          Wed Jun 25 16:08:43 2008 CEST
*************************************************************

    Status info for the Job : https://devel17.cnaf.infn.it:9000/OpPpNrh1EYNwPhjQQaVazQ
    Current Status:     Done (Success)
    Logged Reason(s):
        - Job terminated successfully
    Exit code:          0
    Status Reason:      Job terminated successfully
    Destination:        atlasce01.na.infn.it:2119/jobmanager-lcgpbs-cert
    Submitted:          Wed Jun 25 16:08:43 2008 CEST
*************************************************************
</verbatim>

   * One collection over 60 submissions remains in "Waiting" status due to a LB problem (its nodes are in status "Submitted"); this are the wmproxy logs:
<verbatim> 
23 Jun, 18:58:18 -D- PID: 24834 - "WMPEventlogger::isAborted": Quering LB Proxy...
23 Jun, 18:58:18 -D- PID: 24834 - "WMPEventlogger::isAborted": LBProxy is enabled
Unable to query LB and LBProxy
edg_wll_QueryEvents[Proxy]
Exit code: 4
LB[Proxy] Error: Interrupted system call
(edg_wll_plain_read())
23 Jun, 18:58:18 -D- PID: 24834 - "wmpcoreoperations::jobStart": Logging LOG_ENQUEUE_FAIL
23 Jun, 18:58:18 -D- PID: 24834 - "WMPEventlogger::logEvent": Logging to LB Proxy...
23 Jun, 18:58:18 -D- PID: 24834 - "WMPEventlogger::logEvent": Logging Enqueue FAIL event...
23 Jun, 18:58:19 -D- PID: 24834 - "WMPEventlogger::testAndLog": LB call succeeded
23 Jun, 18:58:19 -D- PID: 24834 - "wmpcoreoperations::jobStart": Removing lock...
23 Jun, 18:58:19 -I- PID: 24834 - "wmpgsoapoperations::ns1__jobStart": jobStart operation completed

23 Jun, 18:58:19 -I- PID: 24834 - "wmproxy::main": -------- Exiting Server Instance -------
23 Jun, 18:58:19 -I- PID: 24834 - "wmproxy::main": Signal code received: 15
23 Jun, 18:58:19 -I- PID: 24834 - "wmproxy::main": ----------------------------------------
</verbatim>

   * There is this warning in wmproxy submitting collections:
<verbatim>
19 Jun, 14:58:32 -W- PID: 2746 - "wmpcoreoperations::submit": Unable to find SDJRequirements in configuration file
</verbatim>

   * Fond a problem in the wmproxy. After some collections submissions it stops working and with the top command you can see that the glite-wms-proxy processes are using all the CPU (It seems that the problem is related with the porting to gsoap 2.7.10): 
<verbatim>
[Thu Jun 19 15:43:50 2008] [warn] FastCGI: (dynamic) server "/opt/glite/bin/glite_wms_wmproxy_server" (pid 7938) terminated due to uncaught signal '11' (Segmentation fault)
</verbatim>


---+++ WM
   * Recovery doesn't work  with a list-match request:
<verbatim>
25 Jun, 15:31:50 -I: [Info] operator()(dispatcher.cpp:754): Dispatcher: starting
25 Jun, 15:31:50 -I: [Info] operator()(dispatcher.cpp:757): Dispatcher: doing recovery
25 Jun, 15:31:50 -I: [Info] operator()(RequestHandler.cpp:389): RequestHandler: starting
25 Jun, 15:31:50 -I: [Info] operator()(recovery.cpp:613): recovering https://localhost:6000/4xvi9qZqMbrNgQeuzOIGbA
25 Jun, 15:31:50 -I: [Info] operator()(RequestHandler.cpp:389): RequestHandler: starting
25 Jun, 15:31:50 -I: [Info] operator()(RequestHandler.cpp:389): RequestHandler: starting
25 Jun, 15:31:50 -I: [Info] operator()(RequestHandler.cpp:389): RequestHandler: starting
25 Jun, 15:31:50 -I: [Info] operator()(RequestHandler.cpp:389): RequestHandler: starting
25 Jun, 15:31:50 -I: [Info] main(main.cpp:292): running...
25 Jun, 15:31:50 -W: [Warning] single_request_recovery(recovery.cpp:317): cannot create LB context (2)
25 Jun, 15:31:50 -E: [Error] operator()(dispatcher.cpp:779): Dispatcher: cannot create LB context (2). Exiting...
25 Jun, 15:31:50 -I: [Info] operator()(RequestHandler.cpp:467): RequestHandler: End of input. Exiting...
25 Jun, 15:31:50 -I: [Info] operator()(RequestHandler.cpp:467): RequestHandler: End of input. Exiting...
25 Jun, 15:31:50 -I: [Info] operator()(RequestHandler.cpp:467): RequestHandler: End of input. Exiting...
25 Jun, 15:31:50 -I: [Info] operator()(RequestHandler.cpp:467): RequestHandler: End of input. Exiting...
25 Jun, 15:31:50 -I: [Info] operator()(RequestHandler.cpp:467): RequestHandler: End of input. Exiting...
25 Jun, 15:31:50 -I: [Info] main(main.cpp:295): TaskQueue destroyed
</verbatim>

---+++ JC/LM

---+++ ICE

---+++ LB
   * The status of the collection is not correctly computed
   * glite-lb-interlogd doesn't stop (you need a "kill -9")

---+++ Configuration

   * Add glite-wms-ice at the metapackage dependencies
   * Remove "rgma" fron the enviroment variable GLITE_SD_PLUGIN
   * Add "globus gridftp" at the gliteservices list (=gLiteservices=)
   * Remove from the metapackage these rpms:
      * checkpointing
      * partitioner
      * interactive (and  thirdparty-bypass)
      * rgma-* (and  service-discovery-rgma-c)
   * glite-lb-interlogd doesn't stop (you need a "kill -9")
   * Modify the cron purger file
<verbatim>
# Execute the 'purger' command at every day except on Sunday with a frequency of six hours
3 */6 * * mon-sat glite . /opt/glite/etc/profile.d/grid-env.sh ; /opt/glite/sbin/glite-wms-purgeStorage.sh -l /var/log/glite/glite-wms-purgeStorage.log -p /var/glite/SandboxDir -t 604800 > /dev/null 2>&1

# Execute the 'purger' command on each Sunday (sun) forcing removal of dag nodes,
# orphan dag nodes without performing any status checking (threshold of 2 weeks).
0 1 * * sun glite . /opt/glite/etc/profile.d/grid-env.sh ; /opt/glite/sbin/glite-wms-purgeStorage.sh -l /var/log/glite/glite-wms-purgeStorage.log -p /var/glite/SandboxDir -o -s -n -t 1296000 > /dev/null 2>&1
</verbatim>
   * Set the ICE configuration section
   * Remove =IsmBlackList= parameter
   * Set =jobdir= as the default  for the DispatcherType parameter of wm
   * Add the parameter =EnableRecovery = true;= ti the wm configuration section
   * Add this cron to create the host proxy:
<verbatim>
0 */6 * * * glite . /opt/glite/etc/profile.d/grid-env.sh ; /opt/glite/sbin/glite-wms-create-proxy.sh /var/glite/wms.proxy /var/log/glite/create_proxy.log
</verbatim>
   * Rotate the logs of the purgeStorage
   * Check bugs [[https://savannah.cern.ch/bugs/?35244][#35244]]

---+++ Other










-- Main.AlessioGianelle - 20 Jun 2008
Topic revision: r4 - 2008-06-25 - AlessioGianelle
Account
- Log In
~~Edit~~
~~Attach~~