(r29) WebHome < WMS

Tags: , view all tags
---+ Welcome to the WMS page

%TOC%

---++ What is the WMS

The Workload Management System (WMS) is a software service of the gLite/EMI middleware which is responsible for distributing and managing tasks across computing and storage resources available on a Grid. WMS assigns user jobs to CEs and SEs belonging to a Grid environment in a convenient fashion, so that:

   * jobs are always executed on resources that match the job requirements
   * Grid-wide load balance is maintained, i.e. jobs are evenly and efficiently distributed across the entire Grid.
The WMS basically receives requests of job execution from a client, finds the required appropriate resources, then dispatches and follows them until completion. This is done handling failure in between and whenever possible. Other than single batch-like jobs, compound job types handled by the WMS are Directed Acyclic Graphs (a set of jobs where the input/output/execution of one of more jobs may depend on one or more other jobs), Parametric Jobs (multiple jobs with one parametrized description), and Collections (multiple jobs with a common description). Jobs are described via a flexible, high-level Job Definition Language (JDL).

As a global Grid resource broker, the WMS represents a reliable and efficient entry point to high-end services available on a Grid, providing a common front-end for user job submission.

---+++ gLite WMS software
   * [[getWMS][Get gLite WMS]]
   * [[gLiteWMSLicence][gLite WMS license information]]

---+++ EMI WMS Software
   * [[http://www.eu-emi.eu/emi-1-kebnekaise][EMI 1 (Kebnekaise) software repository]] 
      * [[http://www.eu-emi.eu/kebnekaise-products/-/asset_publisher/4BKc/content/wms][WMS 3.3.0]]
      * [[http://www.eu-emi.eu/emi-1-kebnekaise-updates/-/asset_publisher/Ir6q/content/emi-1-kebnekaise-update-1-17-06-2011][WMS 3.3.1]] (Update 1 - 17.06.2011 )
      * [[http://www.eu-emi.eu/emi-1-kebnekaise-updates/-/asset_publisher/Ir6q/content/emi-1-kebnekaise-update-4-21-07-2011][WMS 3.3.1-1]] (Update 4 - 21.07.2011)
      * [[http://www.eu-emi.eu/emi-1-kebnekaise-updates/-/asset_publisher/Ir6q/content/update-5-04-08-2011#WMS_v_3_3_2_task_20812][WMS 3.3.2]] (Update 5 - 04.08.2011)
      * [[http://www.eu-emi.eu/emi-1-kebnekaise-updates/-/asset_publisher/Ir6q/content/update-6-01-09-2011#WMS_v_3_3_3_task_21777][WMS 3.3.3]] (Update 6 - 01.09.2011)
      * [[http://www.eu-emi.eu/emi-1-kebnekaise-updates/-/asset_publisher/Ir6q/content/update-11-15-12-2011#WMS_v_3_3_4_task_22847][WMS 3.3.4]] (Update 11 - 15.12.2011)

---++ Documentation

---+++ Documentation for EMI versions

   * [[https://twiki.cern.ch/twiki/pub/EMI/EMIui/EMI_UI_v1_0_0_0.pdf][Client Installation and Configuration (pdf)]]
   * [[http://wiki.italiangrid.org/twiki/bin/view/WMS/WMSSystemAdministratorGuide][System Administrator Guide]]
   * [[EMIWMSServiceRefCard][Service Reference card]]
   * [[http://web.infn.it/gLiteWMS/images/WMS/Docs/wmproxy-guide.pdf][User Guide (pdf)]]
   * [[http://edms.cern.ch/document/590869][Job Description Language (JDL) documentation]]

---+++ Documentation for gLite versions

---++++ Documentation for administrators

   * [[https://twiki.cnaf.infn.it/twiki/bin/view/EgeeJra1It/WMS_guide][WMS installation and configuration guide]]
   * Information concerning configuration 
      * [[https://twiki.cnaf.infn.it/twiki/bin/view/EgeeJra1It/WMS_guide#Configuration_Guide][Configuration guide]]
      * [[https://twiki.cnaf.infn.it/cgi-bin/twiki/view/EgeeJra1It/WMSConfFile][Details of the WMS Configuration file]]
   * [[https://twiki.cern.ch/twiki/bin/view/EGEE/GliteWMS][gLite WMS service reference card]]
   * [[WMS System Administrator Guide]]
   * [[WMProxyDocumentation]]

---++++ Documentation for users

   * [[https://edms.cern.ch/document/674643][WMS User's guide]]
   * [[http://edms.cern.ch/document/590869][Job Description Language (JDL) documentation]]
   * [[WMSClientInstallConfig][WMS Client Installation and Configuration]]

---+++ Other documentation concerning the WMS

   * [[https://twiki.cnaf.infn.it/twiki/bin/view/EgeeJra1It/WMS_limiter][Information concerning the WMS limiter mechanism]]
   * [[https://twiki.cnaf.infn.it/twiki/bin/view/EgeeJra1It/JobWrapperMaxOSB][How the limit on the maximum output sandbox size works]]
   * [[https://twiki.cnaf.infn.it/twiki/bin/view/EgeeJra1It/AuthZinMM][How the authorization information is used in the WMS matchmaking process]]
   * [[HowToSetupBuildEnv][How to setup a build environment]]
   * Information concerning the WMS !JobWrapper 
      * [[https://twiki.cnaf.infn.it/twiki/bin/view/EgeeJra1It/JobWrapperGeneration][How the Job wrapper is generated]]
      * [[https://twiki.cnaf.infn.it/twiki/bin/view/EgeeJra1It/CustomizationPoints][Job Wrapper customization points]]

---++ Known issues
Version of ICE 3.3.5-3 has a bug that is triggered in some particular circumstances. Before explaining it, some details on the bug: there's a piece of code that matches the myproxy server address for correcteness; this check is performed by mean of a (quite) complicated regex that matches the address to be compliant with the FQDN format. The complexity of this regex triggers an high usage of boost's internal memory buffer that can run out of memory matching some particular addresses. 

Now, the bug: the addresses that we found triggering this problem are like "ed8ac012f7da92dd487bc8d3edc4a49b" (or even shorter; not any alphanumeric combination is problematic though). Some VOs use addresse like that (LHCb for example)... We also noted that just adding the domain name to those addresses would by pass the boost's memory exhaustion.

To check if ICE keeps crashing for this problem you can follow these steps:

   * login as root in the WMS node
   * stop the daemon the automatically restart WMS services (if any)
   * stop ICE
   * su - glite
   * execute (as glite) /usr/bin/glite-wms-ice --conf glite_wms.conf 

After a little while you should receive on the console an error like this:

<verbatim>
/usr/bin/glite-wms-ice --conf glite_wms.conf
Logfile is [/var/log/wms/ice.log]
terminate called after throwing an instance of 'std::runtime_error'
what(): Memory exhausted
Aborted
</verbatim>

If this is the case you should proceed to modify the myproxy addresses stored in the ICE's database (thanks to Alessandro Paolini and Sergio Traldi):

<verbatim>
1) come back to root user and change dir to ICE's persist directory:

cd /var/ice/persist_dir

2) put all the myproxy-url without the "dot" in the file "file.txt" by executing:

sqlite3 ice.db "select myproxyurl from delegation where myproxyurl not like '%.%';" > file.txt

3) grep -v ^$ file.txt > file1.txt 

4) generate the instructions to update the ICE DB, and put them in a script

cat file1.txt | gawk '{print "sqlite3 /var/ice/persist_dir/ice.db \"update delegation set myproxyurl=\x27"$0".desy.de\x27 where myproxyurl=\x27"$0"\x27;\""}' > script 

(Substitute the example desy.de domain name with yours).

6) execute the script:

chmod +x script
./script

</verbatim>

---++ Papers and Presentations

   * [[WmsPapers][Papers]]
   * [[WMSPresentations][Presentations]]

---++ Testing and Certification
---+++ Documentation
   * [[TestPlan][Test Plan]] 
      * [[RegressionTest][Regression tests (fixed bugs)]]
   * [[WmsTestSuite][Test Suite]] 
      * [[WmsCliTestSuite][Command Line Interface Test Suite]]
      * [[WmsServiceTestSuite][Service Test Suite]]
      * [[WmsRegressionTestSuite][Regression Test Suite]]

---+++ Reports
   * [[Task21572][Task #21572: WMS v. 3.3.1-1]]
   * [[Task20812][Task #20812: WMS v. 3.3.2]]
   * [[Task21777][Task #21777: WMS v. 3.3.3]]
   * [[Task22847][Task #22847: WMS v. 3.3.4]]


---++ Monitoring
   * [[WmsProbe][Nagios WMS probe]]

---++ Contacts and Support

   * [[HowToReportProblems][How to report problems]]
   * [[HowToContactWMSSupport][How to contact the WMS support team]]
   * [[WMSUsersML][WMS users mailing list]]
Topic revision: r29 - 2012-08-09 - AlviseDorigo
Account
- Log In
~~Edit~~
~~Attach~~