Difference: InstallationProcedure (18 vs. 19)

Revision 192009-02-16 - DanieleCesini

Line: 1 to 1
 
META TOPICPARENT name="WebDocumentation"
Deleted:
<
<

Introduction

This installation guide is divided as follows:

  • Installation of the sensor on the machine to be monitored (WMS/LB/WMSLB)
  • Installation of the data_collector server
This version of the server and of the sensor requires SL4 and snmp (installed by the installation scripts) on all the nodes.

Python, http, php are also needed on the collector machine.

If sensors or/and data_collector are already running at your site, instruction on how to upgrade are given highlighted in green.

It is advisable, even if not required that the sensors are installed before the collector.

In case of problems during installation please contact wms-support<at>cnaf.infn.it

WMS/LB sensors installation:

Sensors MUST be installed on every machine that needs to be monitored, being it a WMS, an LB or a coupled WMSLB. The following procedure is identical for every kind of nodes, since all the sensors are installed on every node. This waste a little bit of h/d space but simplifies the following installation procedure.

(done all the following as root)

  • Download the following tar.gz in /root:
cd /root

wget --no-check-certificate https://grid-it.cnaf.infn.it/certification/downloads/wmsmon_sensors_v2.0.tgz

  • If this is an upgrade remove the /root/wmsmon directory, possibly backup the wmsmon_site-info.def file 
  • Untar the file in /root

tar -xvzf wmsmon_sensors_v2.0.tgz

  • cd /root/wmsmon
  • Edit the  wmsmon_site-info.def (after having read the next few notes)

The wmsmon_site-info.def contains all the information needed to set up correctly all the monitor components. It is important that it is edited carefully using a key = value notation.
Comments can be made using # at the beginning of the line.

You can leave the defaults if you do not see the need for a change.  For sure on the WMS/LB instances you need to change the following keys:

-  WMSMON_HOST = 'Set here the name of the wmsmon data collector host'
-  LEMONFLAG = 1 'If you do not have a lemon tool on the wms/lb instance set to 0'
-  LEMONURL = 'Set to the machine lemon url if any'
-  SERVER_MYSQL_PASSWORD = 'If the node is an LB set the root mysql passwd here'   !!! IT IS IMPORTANT TO HAVE THE RIGHT PASSWORD ON LB nodes info.def, it is not important on WMS. If the node was installed using the gLite yaim tool this password can be grabbed from the  site_info.def file used at configuration time.

- SNMPPASSWD = 'The password you choose for snmp communication between data collector and wms instance !! NOTE: it must be the same on every wmsmon_site-info.def

When setting a password please read the comments in the wmsmon_site-info.def file to know if the password must be enclosed between ' '

- LB_PARA_HOST = Host publishing the  LB_PARAMETER FILE that initializes lb queries. It is the procol://host:port of the http service running on the collector instances.

If you follow the instruction below for the server installation without modifying the httpd configuration it s

http://<WMSMMON_HOST>

# Other examples in case of modified httpd conf

#Examples:
# http://host.domain
# https://host.domain
# https://host.domain:8443

  • Run the setup.sh file on /root/wmsmon
At the end you should have the following rpm installed on the machine:
net-snmp-devel-5.1.2-11.el4_6.11.2
net-snmp-5.1.2-11.el4_6.11.2
net-snmp-utils-5.1.2-11.el4_6.11.2
net-snmp-libs-5.1.2-11.el4_6.11.2
net-snmp-perl-5.1.2-11.el4_6.11.2

(version numbers may vary)

Finally the file /etc/rc.local should contain a line about snmp

Please check all the above statements.

  • Run in /root/wmsmon "python WMSLB_wmsmon_configuration.py"
If python is not present on the machine try running the executable WMSLB_wmsmon_configuration on /root/wmsmon/snmpconf/

At the end check that the following lines are present in /etc/snmp/snmp.conf file

exec .1.3.6.1.4.1.10403.98 /bin/sh /root/wmsmon/bin/send_ce_stats.sh
exec .1.3.6.1.4.1.10403.97 /bin/sh /root/wmsmon/bin/send_users_stats.sh
exec .1.3.6.1.4.1.10403.96 /bin/sh /root/wmsmon/bin/sendLongFile/send_long_file /root/wmsmon/tmp/USERSMAPPING.txt 5
exec .1.3.6.1.4.1.10403.95 /bin/sh /root/wmsmon/bin/sendLongFile/send_long_file /root/wmsmon/tmp/CE_MM.txt 50
exec .1.3.6.1.4.1.10403.94 /bin/sh /root/wmsmon/bin/CE_MM.sh
exec .1.3.6.1.4.1.10403.60 /bin/sh /root/wmsmon/bin/wms/wms-sensor-wrapper
exec .1.3.6.1.4.1.10403.70 /bin/sh /root/wmsmon/bin/lb/lb-sensor-wrapper
exec .1.3.6.1.4.1.10403.75 /bin/sh /root/wmsmon/bin/lb-refill/lb-refill-sensor-wrapper

OIDs are those set in the wmsmon_site-info.def file.

and the snmp daemon should be running

#service snmpd status
#snmpd (pid 21770) is running...

  • The sensors installation on the node is now completed.
 

WMS/LB sensors installation:

Sensors MUST be installed on every machine that needs to be monitored, being it a WMS, an LB or a coupled WMSLB. The following procedure is identical for every kind of nodes, since all the sensors are installed on every node. This waste a little bit of h/d space but simplifies the following installation procedure.

Line: 154 to 63
 #service snmpd status
#snmpd (pid 21770) is running...

  • The sensors installation on the node is now completed.
Deleted:
<
<
 

WMSMON data collector installation

Changed:
<
<
(done all the following as root)
>
>
(done all the following as root)
 
Changed:
<
<
  • Prepare an SL4 machine
  • Install the following packages
>
>
  • Prepare an SL4 machine
  • Install the following packages
 
Changed:
<
<
yum install httpd
>
>
yum install httpd_
 (tested with httpd-2.0.52-38.sl4.2)
Changed:
<
<
yum install php  php-mysql
>
>
yum install php  php-mysql
 
Changed:
<
<
(tested with php-4.3.9-3.22.9, php-mysql-4.3.9-3.22.9)
>
>
(tested with php-4.3.9-3.22.9, php-mysql-4.3.9-3.22.9
 
Changed:
<
<
yum install python
>
>
yum install python_
 (tested with PYTHON-2.3.4-14.4.EL4_6.1)    #PYTHON 2.3 is REQUIRED
Changed:
<
<
yum install mysql mysql-server
>
>
yum install mysql mysql-server
 
Changed:
<
<
(tested with mysql-4.1.20-3.RHEL4.1.el4_6.1, mysql-server-4.1.20-3.RHEL4.1.el4_6.1)
>
>
(tested with mysql-4.1.20-3.RHEL4.1.el4_6.1, mysql-server-4.1.20-3.RHEL4.1.el4_6.1)
 
Changed:
<
<
yum install MySQL-python python-devel python-sqlite
>
>
yum install MySQL-python python-devel python-sqlite
 
Changed:
<
<
(tested with python-sqlite-1.1.6-1, mysql-python-1.2.1_p2-1.el4.1, python-devel-2.3.4-14.4.el4_6.1 )
>
>
(tested with python-sqlite-1.1.6-1, mysql-python-1.2.1_p2-1.el4.1, python-devel-2.3.4-14.4.el4_6.1 )
 
Changed:
<
<
yum install net-snmp net-snmp-devel net-snmp-perl net-snmp-libs net-snmp-utils
>
>
yum install net-snmp net-snmp-devel net-snmp-perl net-snmp-libs net-snmp-utils
 
Changed:
<
<
Tested with:
>
>
Tested with:
 
Changed:
<
<
net-snmp-utils-5.1.2-11.el4_6.11.2
>
>
net-snmp-utils-5.1.2-11.el4_6.11.2
 net-snmp-5.1.2-11.el4_6.11.2
net-snmp-libs-5.1.2-11.el4_6.11.2
net-snmp-perl-5.1.2-11.el4_6.11.2
net-snmp-devel-5.1.2-11.el4_6.11.2
Added:
>
>
 
Changed:
<
<
>
>
 
  • Download this tar.gz file in /root : 
wget --no-check-certificate https://grid-it.cnaf.infn.it/certification/downloads/wmsmon_server-v1.4-1.tgz
Added:
>
>
 
Changed:
<
<
>
>
 
  • Untar the downloaded  file
tar -xvzf wmsmon_server-v1.4-1.tgz
Added:
>
>
 
Changed:
<
<
>
>
 
  • cd /root/wmsmon
  • Edit the wmsmon_site-info.def
Added:
>
>
 
Changed:
<
<
You must pay attention in editing the wmsmon_site-info.def file because it contains all the information needed by executables to run properly.
It is important that it is edited carefully using a key = value notation.
Comments can be made using # at the beginning of the line.
>
>
You must pay attention in editing the wmsmon_site-info.def file because it contains all the information needed by executables to run properly.
It is important that it is edited carefully using a key = value notation.
Comments can be made using # at the beginning of the line.
  You can leave the defaults if you do not see the need for a change.  For sure on the collector instance you need to change the following keys:
Changed:
<
<
-  WMSMON_HOST = 'Set here the name of the wmsmon data collector host'
-  WMSMON_DB_PWD = 'set here the collector root mysql passwd'
>
>
-  WMSMON_HOST = 'Set here the name of the wmsmon data collector host'
-  WMSMON_DB_PWD = 'set here the collector root mysql passwd'
  - SNMPPASSWD = 'The password you choose for snmp communication between data collector and wms instance !! NOTE: it must be the same on
Line: 218 to 130
 wmsmon_site-info.def file to know if the password must be enclosed between ' '
Changed:
<
<
  • Start the mysql server
>
>
Start the mysql server

  service mysqld start
Added:
>
>
 
Changed:
<
<
>
>
 
  • Create the db
./createdb.sh pass1 pass2 (read the following comment)
Added:
>
>
 
Changed:
<
<
createdb.sh needs 2 arguments, the first one is the current mysql_root@localhost passwd
>
>
createdb.sh needs 2 arguments, the first one is the current mysql_root@localhost passwd
 
Changed:
<
<
If it is not defined define it using:
mysql -e "SET PASSWORD FOR root@localhost = PASSWORD('pwd')"
or
mysql -e "SET PASSWORD FOR root = PASSWORD('pwd')"
>
>
If it is not defined define it using:
mysql -e "SET PASSWORD FOR root@localhost = PASSWORD('pwd')"
or
mysql -e "SET PASSWORD FOR root = PASSWORD('pwd')"*
 
Changed:
<
<
The second one is the password that will be used in wmsmonitor db connection and that was set in wmsmon_site-info.def file with WMSMON_DB_PWD
>
>
The second one is the password that will be used in wmsmonitor db connection and that was set in wmsmon_site-info.def file with* WMSMON_DB_PWD
 This script needs on the running directory the wmsmon.sql file, that should be included in distribution tar
Added:
>
>
 
Changed:
<
<
Please note if errors are issued during the run of the createdb.sh script
>
>
Please note if errors are issued during the run of the createdb.sh script*
 
Changed:
<
<
>
>
 
  • Configure snmp
python snmpdconf_script.py Please note if errors are issued during the run
Add /etc/init.d/snmpd start to the /etc/rc.local file in order to be sure that snmp start at next reboot
Added:
>
>
 
Changed:
<
<
>
>
 
  • Configure the cron jobs
cp /root/wmsmon/cron.d/*.cron /etc/cron.d
Added:
>
>
 
Changed:
<
<
cp /root/wmsmon/cron.d/wmsmon_logrotate.conf /etc/
>
>
cp /root/wmsmon/cron.d/wmsmon_logrotate.conf /etc/
 
Changed:
<
<
(Pay attention that if you change the log file name in wmsmon_site-info.def you need to change it also in the wmsmon_logrotate.conf file)  
>
>
(Pay attention that if you change the log file name in wmsmon_site-info.def you need to change it also in the wmsmon_logrotate.conf file) 
 
Changed:
<
<
The wmsmon.cron and the wmsmon_daily.cron  cronjobs trigger the execution of the two python executables that collect the data. The first one is executed every 15 minutes and collects data from the wms/lb instances, while the second one creates some daily aggregated statistics and runs every two hours.
>
>
The wmsmon.cron and the wmsmon_daily.cron  cronjobs trigger the execution of the two python executables that collect the data. The first one is executed every 15 minutes and collects data from the wms/lb instances, while the second one creates some daily aggregated statistics and runs every two hours.
 
Changed:
<
<
If you want to increase the rate of data collection you need to increase the running frequency of the first cronjob. Note the the tool was not widely tested at acquisition rates greater than 900 seconds.
>
>
If you want to increase the rate of data collection you need to increase the running frequency of the first cronjob. Note the the tool was not widely tested at acquisition rates greater than 900 seconds.
 
Changed:
<
<
>
>
 
  • Edit the wmsmonlist.conf file
Put here the WMS/LB instances you want to monitor as described in the file.
Every line contains a WMS(first string), its associated LB (second string) and a word indicating which VO is served by the WMS (you can put just one word) (third string)
Please do not use comments in this file 
Added:
>
>
 
Changed:
<
<
>
>
 
  • Test the snmp communication
Run the following command:
Added:
>
>
 
Changed:
<
<
snmpwalk -v 3 -u 'wmsmon' -l authNoPriv -a MD5 -A 'snmp_passwd' <a_wmshost_in_wmsmonlist_file> .1.3.6.1.4.1.10403.60 -t 180
>
>
snmpwalk -v 3 -u 'wmsmon' -l authNoPriv -a MD5 -A 'snmp_passwd' <a_wmshost_in_wmsmonlist_file> .1.3.6.1.4.1.10403.60 -t 180
 
Changed:
<
<
Execute the cron executables:
>
>
Execute the cron executables:
 
Changed:
<
<
 /root/wmsmon/bin/data_collector_main_autoup.py /root/wmsmon/wmsmonlist.conf 900
>
>
  /root/wmsmon/bin/data_collector_main_autoup.py /root/wmsmon/wmsmonlist.conf 900
 
Changed:
<
<
Check if errors are reported
>
>
Check if errors are reported
WEB INSTALLATION AND CONFIGURATION
 
Changed:
<
<
WEB INSTALLATION AND CONFIGURATION 
>
>
  • Download and untar the web tar.gz
cd /var/www/html/
wget --no-check-certificate_ https://grid-it.cnaf.infn.it/certification/downloads/wmsmon_web-v1.4-1.tgz
 
Changed:
<
<
  • Download and untar the web tar.gz  
cd /var/www/html/
wget --no-check-certificate
https://grid-it.cnaf.infn.it/certification/downloads/wmsmon_web-v1.4-1.tgz
>
>
tar -xvzf wmsmon_web-v1.4-1.tgz
 
Changed:
<
<
tar -xvzf wmsmon_web-v1.4-1.tgz
>
>
  • Configure the WEB
cd /root/wmsmon yum install  php-gd gd gd-devel
python phpconf_script.py /root/wmsmon/wmsmonlist.conf
 
Changed:
<
<
  • Configure the WEB  
cd /root/wmsmon yum install  php-gd gd gd-devel python phpconf_script.py /root/wmsmon/wmsmonlist.conf
>
>
In /etc/php.ini set : memory_limit = 24M (Default should be 8M)
 
Changed:
<
<
In /etc/php.ini set : memory_limit = 24M (Default should be 8M)
>
>
  • Restart httpd
 
Changed:
<
<
  • Restart httpd
>
>
service httpd restart
 
Changed:
<
<
service httpd start

>
>
  If you need an https connection  you need to install the mod_ssl rpm (yum install mod_ssl) and restart the httpd service
service httpd restart
The web interface will be reachable at:
https://<your_wmsmon_host>/wmsmon/main
Deleted:
<
<
Moreover if you would like to use a server certificate, or to change the standard port you need to edit the /etc/httpd/conf.d/ssl.conf file and restart the httd service.
 \ No newline at end of file
Added:
>
>
Moreover if you would like to use a server certificate, or to change the standard port you need to edit the /etc/httpd/conf.d/ssl.conf file and restart the httd service.
 
TWIKI.NET
This site is powered by the TWiki collaboration platformCopyright © 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback