You are here:
TWiki
>
WMSMonitor Web
>
InstallationProcedureV2_1
(revision 14) (raw view)
---++ *Introduction to v2.1 Installation and Configuration* This installation guide is divided as follows: * <b>Installation of the sensors on the machine to be monitored (WMS/LB/WMSLB)<span style="color: red;"></span></b> * *Installation of the data_collector server* *For the installation and configuration of other WMSMonitor modules (i.e. [[WMSLoadBalancingArbiter][LoadBalancingArbiter]] and [[WMSMonDBAnalyzer][DBAnalyzer]]) please refer to the proper twiki session using the left bar.* This version of the server and of the sensor requires SL4 and snmp (installed by the installation scripts) on all the nodes. Python, http, php and other packages are also needed on the collector machine - they will be installed automatically when the WMSMonitor packages are installed. If sensors and/or data_collector are already running at your site, instruction on how to upgrade are given highlighted in <span style="color: green;">green</span>. This version is installed via rpm and yum.<br /> It is advisable, even if not required that the sensors are installed before the collector. In case of problems during installation please contact wms-support<at>cnaf.infn.it <b><span style="color: red;">NOTE: In release 2.0 the database schema has changed! if you are running an update from a version older that 2.0 it will be automatically ported to the new schema by the configuration script.</span> </b> <b><span style="color: red;">If you run a version earlier to 2.0 and want to port the old data to the new VO statistics page please contact wms-support<at>cnaf.infn.it after the installation/configuration completes. If you are upgrading from 2.0 VO statistics will be automatically ported.</span></b> ---++ *WMSMON Sensors installation on WMS/LB/WMSLB nodes* *(do all the following as root on all your WMS, LB, WMSLB nodes - the procedure is independent on the node type)* <span style="color: green;"></span> <b><span style="color: green;"></span></b> * *cd /etc/yum.repos.d* * *Download this repo file containing the WMSMonitor repository contact string:* wget http://grid-it.cnaf.infn.it/certification/downloads/WMSMonitor.repo <b> * cd /root<br /> * yum clean all * yum install WMSMonitor.sensors </b><span style="color: green;">In release 2.1 the WMSMonitor rpms installs everything under /opt/WMSMonitor (in previous releases it was, by default, under /root/wmsmon).</span> <b> * cd /opt/WMSMonitor/common </b>In /opt/WMSMonitor/common are installed all the files needed by all WMSMonitor components. One of this files is the wmsmon_site-info.def, it is now much easier to edit since many default parameters have been moved to the wmsmon_default.conf file. The defaults file should usually not be changed. <b> * Edit the wmsmon_site-info.def <span style="color: green;"></span></b>The wmsmon_site-info.def contains all the information needed to set up correctly all the monitor components. <b><span style="color: red;">It is important to edit it carefully using a key = value notation.</span></b> Comments can be made using # at the beginning of the line. Inline comments help you during the editing. <b><span style="color: green;">If this is an upgrade please re-edit the file in the new location, now it is much easier to edit. Of course use the same values used in the wmsmon_site-info.def used up to now</span></b>. The important variable for sensors installation are: - WMSMON_HOST = 'Set here the name of the wmsmon data collector host'<br />- SNMPPASSWD = 'The password you choose for snmp communication between data collector and wms instance !! NOTE: it must be the same on every wmsmon_site-info.def <span style="color: red;">- SERVER_MYSQL_PASSWORD = 'your_pass' *(This is FUNDAMENTAL ON LB NODES)* </span> - <span style="color: red;">LB_PARA_HOST = Host publishing the LB_PARAMETER FILE that initializes lb queries. It is the procol://host:port of the http service running on the collector instances. </span> <span style="color: red;"> *!!!(THIS IS FUNDAMENTAL ON EVERY HOST PLEASE READ CAREFULLY WHAT FOLLOWS)!!!* </span> If you follow the instruction below for the server installation without modifying the httpd configuration it is [[http://%3cwmsmmon_host%3e/][http://<WMSMMON_HOST>]] # Other examples in case of modified httpd conf #Examples:<br /># [[http://host.domain/][http://host.domain]] # [[https://host.domain/][https://host.domain]] # [[https://host.domain:8443/][https://host.domain:8443]] <b><span style="color: red;">In the post installation steps you'll find some basic instruction on how to configure httpd - if you change the httpd configuration (in particular http to https or the port number) remember to change the LB_PARA_HOST on all instances)</span></b> If the parameters mentioned above are not set an error will be raised a t configuration time (only a warning for the mail address) When setting a password please read the comments in the wmsmon_site-info.def file to know if the password must be enclosed between ' ' You can ignore the LOAD BALANCING PARAMETERS section if you are not installing a load balancing arbiter. * *cd /opt/WMSMonitor/sensors/etc* <br /> * *Run the configuration script* *./WMSLB_wmsmon_configuration.py* _(Alternatively if python is not present on the machine or you experience problems in modules importing you can run the python frozen code in setup_bins - cd setup_bins; ./WMSLB_wmsmon_configuration - )_ This script will configure the /etc/snmp/snmpd.conf file in order to accept requests from the data collector. It is important that the snmp password in wmsmon_site-info.def is set correctly. It willl not destroy lines not belonging to WMSMonitor in that file, in any cse a backup copy will be created in local directory.<br />If the node is an LB the script will add grants to for wmsmon user on DB lbserver20. It's important that the MYSQL root passwd is set corrrectly in wmsmon_site-info.def file. <span style="color: green;">The script will not remove old WMSMonitor path /root/wmsmon in order not to destroy user manual changes. That directory can be removed manually without creating problems to the sensors.</span> *If at the end of the script you see the string: "WMSMONITOR SENSORS SUCCESSFULLY CONFIGURED!" it means that everything went fine* *.<br /><br />In case of problems running the script please contact wms-support<at>cnaf.infn.it* *.* <b><span style="color: red;"></span></b> ---++ *WMSMON Data Collector installation* *(do all the following as root)* * <b> Install an SL4 machine (<span style="color: green;">Not to be done if this is an upgrade</span>)<not to="" be="" update="" is="" an="" this="" if=""></not></b> <span style="color: green;"></span> * <b> <span style="color: green;">If it is an upgrade it is safer to create a backup copy of the wmsmon database using an utility such as mysqldump (i.e. mysqldump -p wmsmon > wmsmon-dump.sql)</span></b> * <b><span style="color: green;">IMPORTANT: If you manually changed files of the web interface under /var/www/html/wmsmon/ backup them because that directory will be overwritten.<br /></span></b> * *cd /etc/yum.repos.d* * *Download this repo file containing the WMSMonitor repository contact string:* wget http://grid-it.cnaf.infn.it/certification/downloads/WMSMonitor.repo <b> * cd /root<br /> * yum clean all * yum install WMSMonitor.collector </b> <span style="color: green;">In release 2.1 the WMSMonitor rpms installs everything under /opt/WMSMonitor (in previous releases it was, by default, under /root/wmsmon).</span> <span style="color: green;"> </span> <b> * cd /opt/WMSMonitor/common </b>In /opt/WMSMonitor/common are installed all the files needed by all WMSMonitor components. One of these files is the wmsmon_site-info.def, which is now much easier to edit since many default parameters have been moved to the wmsmon_default.conf file. The defaults file should usually not be changed. <b> * Edit the wmsmon_site-info.def <span style="color: green;"></span></b>The wmsmon_site-info.def contains all the information needed to set up correctly all the monitor components. <b><span style="color: red;">It is important to edit it carefully using a key = value notation.</span></b> Comments can be made using # at the beginning of the line. Inline comments help you during the editing. <b><span style="color: green;">If this is an upgrade please re-edit the file in the new location, now it is much easier to edit. Of course use the same values used in the wmsmon_site-info.def used up to now</span></b>. The important variable for a collector installation are: - <span style="color: red;">WMSMON_HOST = 'Set here the name of the wmsmon data collector host' </span> - WMSMON_DB_PWD = Chose a root - password for mysql - if already set use it.<br />- LEMONFLAG = 1 'If you do not have a lemon tool on the wms/lb instance set to 0'<br />- LEMONURL = 'Set to the machine lemon url if any'<br />- SNMPPASSWD = 'The password you choose for snmp communication between data collector and wms instance !! NOTE: it must be the same on every wmsmon_site-info.def - WMSMON_SEVER_CONTACT_EMAIL = MAIL CONTACT that will appear in web pages error messages. Users will be invited to send email to this address. If the parameters mentioned above are not set an error will be raised a t configuration time (only a warning for the mail address) When setting a password please read the comments in the wmsmon_site-info.def file to know if the password must be enclosed between ' ' You can ignore the LOAD BALANCING PARAMETERS section if you are not installing a load balancing arbiter. <b> * cd */opt/WMSMonitor/collector/etc* * *Edit the wmslist.conf* </b><b><span style="color: green;">If this is an upgrade you can maintain the same wmslist.conf file used by the previous release, but note that it is no more called wmsmonlist.con (R1.4) but wmslist.conf (since R2.0).</span></b> This file contains the list of WMS present in your cluster (those were the wmsmon sensors are or will be installed).<br />It must be edited as shown in the template file in the following way: _wms1.your_domain lb1.your_domain vo1_ _...._ _wmsn.your_domain lbn.your_domain von_ You should insert the wms/lb pairs monitored and a vo served by the pair. The vo indication will be used only to group the wms in the wmsmon web pages, and not for the job counting per vo.<br />If a pair serve more than one VO you can choose a word like multi or multiVO. You can also use this tag to group the wms by their role: PROD, DEVEL etc... *IMPORTANT: THE FILE CANNOT BE COMMENTED!* <b> * <b>Check that MySQL server is running (service mysqld status). If not, please start it (service mysqld start)<br /></b> * <b>Run the configuration script<br /></b> *./wmsmon_server_setup.py* </b> <b><span style="color: green;">The script will look for older wmsmon installations.</span></b> <span style="color: green;">If none is found a brand new installation will start otherwise an upgrade process will be attempted.</span> *The old cron files that trigger the collector execution are moved to /tmp directory and new crons will be installed.* <b><span style="color: green;">The old $INSTALL_PATH/wmsmon directory (by default /root/wmsmon) will not be removed or moved in order not to destroy manual changes - if you want you can remove manually or leave untouched, it will not cause any problems.</span></b> Errors will be reported, please pay attention to them. When the script completes the _/opt/WMSMonitor_ directory should have been moved to the _INSTALL_PATH_ defined and the _/var/www/html/wmsmon_ directory should now be in _/var/www/html/WEBDIR_, where _WEBDIR_ is defined in the _wmsmon_default.conf_ file. Two crons should be present in _/etc/cron.d_: _wmsmon.cron_ and _wmsmon_logrotate.conf_. The _wmsmon.cron_ is the cron that launches the main data collector script. By default it is launched every 15 mins, if you want to change this frequency you should change the cron file by hand, and in this case in order to optimize performance you are advised to change also the _STEPDATE_ value in the _wmsmon_default.conf_ file to 2x(cron frequency). By default the data collector script logs to the _/var/log/WMSMONITOR.log_ file, but you can configure this on the default.def file. By default the wmsmon_logrotate cron creates 10 files 100MB big, to modify this parameter you should change by hand the _/etc/wmsmon_logrotate.cron_ file. The standard output of the data collector script is maintained in the _/var/log/data_collector_main.log_ for debugging reasons, this file is rotated as the main log file, please keep it. *Look for errors. Configuration is complete only if a string like "WMSMonitor Successfully Configured" is displayed at the end.* In case of problems please contact wms-support<at>cnaf.infn.it <b><b> ---++++ *Post installation STEPS* </b></b> <ul><li> ---++++++ <a name="Increased_php_available_memory"></a><a name="_Increased_php_available_memory_"></a> *Increased php available memory* </li></ul> In order to increase php performance it is advisable that you increase the php allocable memory. <br />Modify the _/etc/php.ini_ to have the folloving line:<br /> _memory_limit = 56M_ <ul><li> ---++++++ <a name="Optional_http_port_change"></a><a name="_Optional_http_port_change_"></a> *Optional http port change* </li></ul> The WMSMON web configuration does not modify the default port (80) used by httpd. To modify the port number edit _/etc/httpd/conf/httpd.conf_ and edit the line: <pre> Listen <port_number> </pre> Then restart httpd ( _service httpd restart_) <ul><li> ---++++++ <a name="Optional_secure_http_enabled"></a><a name="_Optional_secure_http_enabled_"></a> *Optional secure http enabled* </li></ul> The WMSMON server needs a valid host certificate stored in a HOST_CERTIFICATE_DIR (i.e. _/etc/grid-security_)<br />Install mod_ssl package:<br /> -Run: _yum install mod_ssl_ Install the accepted ca packages, i.e. you can execute the following:<br /> - Create the /etc/yum.repos.d/lcg-ca.repo file containing: <pre> [CA] </pre> <pre> name=CAs </pre> <pre> baseurl=http://linuxsoft.cern.ch/LCG-CAs/current </pre> <pre> protect=1 </pre> - Run : _yum install lcg_CA_ Edit the _/etc/httpd/conf/httpd.con_ f and add the following inside the _<Directory /var/www/html>_ section the following lines _:_ <pre> SSLRequireSSL </pre> <pre> SSLVerifyClient require </pre> <pre> SSLVerifyDepth 10 </pre> Edit the _/etc/httpd/conf.d/ssl.conf_ and: - set the _SSLCertificateFile_ variable to HOST_CERTIFICATE_DIR/hostcert.pem and comment any other line that set this variable. - set the _SSLCertificateKeyFile_ variable to HOST_CERTIFICATE_DIR/hostkey.pem and comment any other line that set this variable. _ _ - set the _SSLCACertificatePath_ variable to the name of the directory containing the CA file (i.e. _/etc/grid-security/certificates_ if you installed the _lcg_CA_ metapackage) and comment any other line that set this variable. If you want to change the default https port (443) you should change in the _/etc/httpd/conf.d/ssl.conf_ file the line: <br /> Listen <port_number> (i.e. Listen 8443) _Oprtional redirect - If you want to automatically redirect http requests to https pages you should add to the /etc/httpd/conf/httpd.conf file the following section (using the proper values for the variables_ <span style="font-family: monospace;">SERVER_HOST_IP,SERVER_HOST_NAME and YOUR_DOMAIN): <pre> <VirtualHost SERVER_HOST_IP:80> </pre> <pre> DocumentRoot /var/www/html </pre> <pre> ServerName SERVER_HOST_NAME.YOUR_DOMAIN </pre> <pre> RedirectMatch (/.*)$ https://SERVER_HOST_NAME.YOUR_DOMAIN/$1 </pre> <pre> </VirtualHost> </pre> </span> Restart httpd <ul><li> ---++++++ <a name="Unlock_protected_pages_to_specif"></a><a name="_Unlock_protected_pages_to_speci"></a> *Unlock protected pages to specific certificate DNs* </li></ul> WMSMONitor reports a section with users activity on each WMS. Name and surname of each user is reported on some pages and these pages for privacy reasons are not exposed to all certificates If you want to unlock those pages to some certificate DN you should enable https protocol as descrbed in the previous paragraph(Optional secure http enabled) and do the following: - Edit the _/var/www/html/wmsmon/common/config.php_ file and modify the last line adding the list of DNs in the line <span style="font-family: monospace;"> <pre> $config->dnEnabledList=array('DN1','DN2'....'DNn'); </pre> </span> If you want to unlock the pages with sensible data to everyone (using either http or https) you have to change the value of the _$config->protectedPage_ variable in _/var/www/html/WEBDIR/common/config.php_ file (WEBDIR is defined in the site-info.def file) and set it to 0: <br /> <span style="font-family: monospace;"> <pre> $config->protectedPages=0; </pre> </span> <ul><li> ---++++++ <a name="Enable_high_port_range_communica"></a><a name="_Enable_high_port_range_communic"></a> *Enable high port range communication for inter cluster monitoring* </li></ul> WMSMON uses the snmp standard port (in general the 183) for sensor-collector communications. If a firewall blocks the snmp port it is possible to enable the communication on a not-standard port. This can be useful when the data collector and the WMS cluster are not in the same computing centre. <br />If you are not in this case you can skip this section.<br />NOTE: this feature is not well tested, please report any problem and bug found to wms-support<at>cnaf.infn.it<br />To enable high port support you should modify the _wmslist.conf_ file adding a fourth column indicating which is the port number to be used on that particular wms/lb pair: _wms1.your_domain lb1.your_domain vo1 port1_ NOTE: it is not possible to specify 2 different port numbers for WMS and LB. On WMS/LB sensor side you should set snmp in order to listen for request on the port you choose. This is accomplished by adding in the /etc/snmp/snmpd.conf file the following line: <pre wrap=""> agentaddress <port_number> </pre> and restarting snmp (service snmpd restart) <ul><li> ---++++++ <a name="Configuring WMS load balancing Arbiter"></a><a name="_Configuring WMS load balancing Arbiter"></a> *Configuring WMS load balancing Arbiter* </li></ul> To configure the Arbiter parameters in the site_info.def file refer to:<br /> [[WMSLoadBalancingArbiter][https://twiki.cnaf.infn.it/cgi-bin/twiki/view/WMSMonitor/WMSLoadBalancingArbiter]] The WMS instances load balancing arbiter is a python script wms_balancing_arbiter.py which must be executed periodically in a cron. It should be executed in cascade to the data_collector script in order to take advantage of latest available metric collected by the WMSMonitor from each WMS instance. <br /> Note that also an utility is provided, external_wmsmon_metrics_collector.py, to allow the inclusion of WMS instances monitored by other WMSMonitor servers in the list of defined aliases. This is useful for ROC level definition of some WMS aliases pointing to WMS instances geographycally distributed and momitored by different WMSMonitor instances. To make this work you must edit the external_wmsmon_metrics_collector.py script directly adding the WMSMonitor urls of instances to monitor as in the example below:<br /> urllist=['https://wmsmon.ct.infn.it:8443/wmsmon/details/details.php?wms=prod-wms-01.ct.infn.it', 'https://wmsmon.ct.infn.it:8443/wmsmon/details/details.php?wms=prod-wms-02.ct.infn.it']<br /> and add its execution in the load balancing cron, which is something like: 5,20,35,50 * * * * root /PATH/external_wmsmon_metrics_collector.py; /PATH/wms_balancing_arbiter.py >> /var/www/html/loadbalancing_test.log 2>&1 Note that /var/www/html/loadbalancing_test.log file readable through the web displays the WMS IP list in the alias for all defined aliases. It is advisable that you configure a logrotate for the arbiter log file. This is possiblle ading the following lines to the /etc/wmsmon_logrotate.conf file: /var/www/html/loadbalancing_test.log { copytruncate<br /> rotate 100<br /> size = 1M<br /> missingok<br /> nomail }
Edit
|
Attach
|
P
rint version
|
H
istory
:
r16
<
r15
<
r14
<
r13
<
r12
|
B
acklinks
|
V
iew topic
|
More topic actions...
Topic revision: r14 - 2010-03-10
-
DaniloDongiovanni
Home
Documentation
WebMainPage
WebDetailsPage
WebCustomPlot
WebUsersStat
WebResUsage
WebLoadBal
WebVOStatsPage
WMS Load Balancing Arbiter
DBAnalyzer
Download/Install version 2.1
Download/Install version 3.0
Publications
Screenshots
VideoTour
Credits & Contacts
TWIKI.NET
WMSMonitor
Edit
Attach
Copyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki?
Send feedback