WMSMonitor WMS instances Load Balancing Arbiter (Available from release 2.1)

The Load Balancing Arbiter is a python script implementing a DNS-based WMS instances load balancing.
When periodically executed as a cron it updates the WMS IPs set behind predefined WMS alias on the base of a load and functionality metric.
In particular, once a set of WMS alias is defined in the local DNS, and a list of WMSs instances is associated to each alias in the wmsmon_site-info.def, the load balancing arbiter for each alias:

  • collects load metrics for each WMS instance associated to the alias. This metric takes into account cpu load, queues of WM, JC and LB components of the WMS, memory usage and disk space occupation to produce a positive number increasing mnonotonically in the load of the WMS instance. Also daemons status are considered resulting in a negative metric in case of not running daemons.
  • collects (if available at a specified url) the result of a submission test for each WMS instance. If the submission test fails for a given instance, the instance's metric is set to negative.
  • all WMS instances with negative metric associated are excluded from the IP set behind the alias till next evaluation. Others instances with positive metric are ordered according to increasing metric value and all but one of them form the set of IP to put behind the alias, and the alias is accordingly updated in the DNS. Note that at least one WMS IP is always left in the alias set, even when all WMS considered at a given time have negative metrics. This is to always have an answer from a WMS service for all user requests reaching the alias.

Each time a user request reaches the alias, one among the WMS IP in the associated set takes in charge the request in a round robin rotation.

Also an utility for regional loadbalancing is provided, to collect metrics from WMS instances monitored by alternative WMSMonitor server.

Configuration of LOAD BALANCING Arbiter

Preconfiguration STEPS: the Arbiter assumes that:

  • A DNS server and a DNS zone has been configured to get updates from the arbiter
  • The Arbiter has the grants to perform alias IP set updates in the specific DNS. This has the form of a couple KEYNAME:KEY set in the DNS for the WMSMon Arbiter
  • A list of alias has been defined in the local DNS the arbiter will update. As an example at CNAF we abilitated an alias for each HEP VO.

The following parameters must be set in the wmsmon_site-info.def file

LOAD_BALANCING_SITE_CONTACT = your-support-mail-address
This is used to notify administrators about problems


DNS_KEYSTRING = YOUR-KEYNAME:YOUR-KEY
DNS_SERVER = your-DNS-server
DNS_ZONE = your-DNS-ZONE
These are mandatory variables to fill with DNS coordinates and the KEYNAME:KEY couple for arbiter authentication on the DNS.

ALIAS_LIST = wms-alice, wms-atlas, wms-cms-analysis, wms-cms-prod, wms-lhcb, wms-cdf, wms-multi, wms-test (These are Examples) Put here the list of aliases defined in your DNS (you must skip the DNS_ZONE postfix string)


Let's now consider the variable you must set for each defined alias:

WMS-ALICE_FLAG = 0 Set this Flag to 0 to disable the Load Balancing Procedure for this alias


WMS-ALICE_WMSLIST = wmshostname1, wmshostname2, wmshostname3
Number of wms instances to leave out of the alias each new nsupdate N.B. This number must be 0 < NUM_OUT < (number of wms in the alias list)

WMS-ALICE_NUM_OUT = 1


Set this flag to 1 if you want to integrate SUBMISSION TEST in loadbalancing metric, 0 otherwise
WMS-ALICE_SUBMISSION_TEST_ENABLE = 0
Put the url of SUBMISSION TEST result page for considered alias
WMS-ALICE_SUBMISSION_TEST_PATH = http://your-url-publishing-page-with-submission-test-result

Note that this url page is expected to list submission test results for considered WMS instances in the form

timestamp ERRORFLAG WMSINSTANCE_HOSTNAME1
timestamp ERRORFLAG WMSINSTANCE_HOSTNAME2
timestamp ERRORFLAG WMSINSTANCE_HOSTNAME3
'''''

-- DaniloDongiovanni - 12 Jun 2009

Topic revision: r2 - 2009-06-12 - DanieleCesini
 
TWIKI.NET
This site is powered by the TWiki collaboration platformCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback