(r5) NotesAboutInstallationAndConfigurationOfWNTORQUEGLEXEC < SiteAdminCorner

Tags: , view all tags
---+!! Notes about Installation and Configuration of WN using Torque and GLEXEC
   * *These notes are provided by site admins on a best effort base as a contribution to the IGI communities and MUST not be considered as a subsitute of the [[http://wiki.italiangrid.it/twiki/bin/view/IGIRelease/IgiEmi][Official IGI documentation]].*
   * This document is addressed to site administrators responsible for middleware installation and configuration.
   * The goal of this page is to provide some hints and examples on how to install and configure an *EMI WN + gLExec* service based on EMI middleware using *TORQUE* as batch system

%TOC%

---++ References
   1. [[http://www.italiangrid.it/][About IGI - Italian Grid infrastructure]]
   1. [[http://wiki.italiangrid.it/twiki/bin/view/IGIRelease/WebHome][About IGI Release]]
   1. [[http://wiki.italiangrid.it/twiki/bin/view/IGIRelease/IgiEmi][IGI Official Installation and Configuration guide]]
   1. [[https://twiki.cern.ch/twiki/bin/view/EMI/GenericInstallationConfigurationEMI1][Generic Installation & Configuration for EMI 1]]
   1. [[https://twiki.cern.ch/twiki/bin/view/LCG/YaimGuide400][ Yaim Guide]]
   1. [[https://twiki.cern.ch/twiki/bin/view/LCG/Site-info_configuration_variables#site_info_def][site-info.def yaim variables]]
   1. [[https://twiki.cern.ch/twiki/bin/view/LCG/Site-info_configuration_variables#GLEXEC_wn][GLEXEC_wn yaim variables]]
   1. [[https://twiki.cern.ch/twiki/bin/view/LCG/Site-info_configuration_variables#WN][WN yaim variables]]
   1. [[https://twiki.cern.ch/twiki/bin/view/LCG/Site-info_configuration_variables#TORQUE][TORQUE Yaim variables]]
   1. [[http://www.eu-emi.eu/products/-/asset_publisher/z2MT/content/emi-wn][EMI-WN v.1.0.0]]
   1. [[http://www.eu-emi.eu/products/-/asset_publisher/z2MT/content/glexec][gLExec v.0.8.10]]
   1. [[https://wiki.egi.eu/wiki/Tools/Manuals/SiteProblemsFollowUp][Troubleshooting Guide for Operational Errors on EGI Sites]]
   1. [[https://wiki.egi.eu/wiki/Tools/Manuals/AdministrationFaq][Grid Administration FAQs page]]

---++ Service installation
%TWISTY{
mode="div"
showlink="   *O.S. and Repos* "
hidelink="     *O.S. and Repos* " 
remember="off" firststart="hide"
showimgright="%ICONURLPATH{toggleopen}%"
hideimgright="%ICONURLPATH{toggleclose}%"
}%

---+++ O.S. and Repos 
   * Starts from a fresh installation of Scientific Linux 5.x (x86_64). 
<verbatim>
# cat /etc/redhat-release 
Scientific Linux SL release 5.7 (Boron) 
</verbatim>
  
  * Install the additional repositories: EPEL, Certification Authority, UMD

<verbatim>
# yum install yum-priorities yum-protectbase
# cd /etc/yum.repos.d/
# rpm -ivh http://mirror.switch.ch/ftp/mirror/epel//5/x86_64/epel-release-5-4.noarch.rpm
# wget http://repo-pd.italiangrid.it/mrepo/repos/egi-trustanchors.repo
# rpm -ivh http://repo-pd.italiangrid.it/mrepo/EMI/1/sl5/x86_64/updates/emi-release-1.0.1-1.sl5.noarch.rpm
</verbatim>

   * Be sure that SELINUX is disabled (or permissive). Details on how to disable SELINUX are [[http://fedoraproject.org/wiki/SELinux/setenforce][here]]:

<verbatim>
# getenforce 
Disabled
</verbatim>

   * Check the repos list (sl-*.repo are the repos of the O.S. and they should be present by default).

<verbatim>
# ls /etc/yum.repos.d/
egi-trustanchors.repo  
emi1-third-party.repo emi1-base.repo emi1-updates.repo
epel.repo epel-testing.repo  
sl-contrib.repo sl-fastbugs.repo sl-security.repo sl-testing.repo sl-debuginfo.repo sl.repo sl-srpms.repo
</verbatim>
*IMPORTANT*: remove the dag repository if present
%ENDTWISTY%

%TWISTY{
mode="div"
showlink="   *yum install* "
hidelink="     *yum install* " 
remember="off" firststart="hide"
showimgright="%ICONURLPATH{toggleopen}%"
hideimgright="%ICONURLPATH{toggleclose}%"
}%
---+++ yum install
<verbatim>
# yum clean all
Loaded plugins: downloadonly, kernel-module, priorities, protect-packages, protectbase, security, verify, versionlock
Cleaning up Everything

# yum install ca-policy-egi-core
# yum install emi-wn emi-torque-client emi-glexec_wn
# yum install tcsh openldap-clients
# yum install nfs-utils
</verbatim>
see [[YumWNtorqueGLEXEC][here]] for details
%ENDTWISTY%

---++ Service configuration
You have to copy the configuration files in another path, for example root, and set them properly (see later):
<verbatim>
# ls /opt/glite/yaim/examples/siteinfo/
services  site-info.def

# ls /opt/glite/yaim/examples/siteinfo/services/
glite-glexec_wn  glite-vobox  glite-wn  glite-wn_tar

# cp -r /opt/glite/yaim/examples/siteinfo/* .
</verbatim>

%TWISTY{
mode="div"
showlink="   *vo.d directory* "
hidelink="     *vo.d directory* " 
remember="off" firststart="hide"
showimgright="%ICONURLPATH{toggleopen}%"
hideimgright="%ICONURLPATH{toggleclose}%"
}%

---+++ vo.d directory
Create the vo.d directory for the VO configuration file (you can decide if keep the VO information in the site.def or putting them in the vo.d directory)
<verbatim>
# mkdir vo.d
</verbatim>
[[VoDirContent][here]] an example for some VOs.

Information about the several VOs are available at the [[http://operations-portal.in2p3.fr/vo][CENTRAL OPERATIONS PORTAL]].
%ENDTWISTY%

%TWISTY{
mode="div"
showlink="   *users and groups configuration* "
hidelink="     *users and groups configuration* " 
remember="off" firststart="hide"
showimgright="%ICONURLPATH{toggleopen}%"
hideimgright="%ICONURLPATH{toggleclose}%"
}%
---+++ users and groups configuration
here an example on how to define pool accounts ([[https://forge.cnaf.infn.it/plugins/scmsvn/viewcvs.php/*checkout*/branches/BRANCH-4_0_X/ig-yaim/examples/ig-users.conf?rev=6066&root=igrelease][ig-users.conf]]) and groups ([[https://forge.cnaf.infn.it/plugins/scmsvn/viewcvs.php/*checkout*/branches/BRANCH-4_0_X/ig-yaim/examples/ig-groups.conf?rev=6075&root=igrelease][ig-groups.conf]]) for several VOs
%ENDTWISTY%

%TWISTY{
mode="div"
showlink="   *site-info.def* "
hidelink="     *site-info.def* " 
remember="off" firststart="hide"
showimgright="%ICONURLPATH{toggleopen}%"
hideimgright="%ICONURLPATH{toggleclose}%"
}%
---+++ site-info.def
SUGGESTION: use the same [[SiteDefCreamWN][site-info.def]] for CREAM and WNs: for this reason in this example file there are yaim variable used by CREAM, TORQUE or emi-WN.

It is also included the settings of some VOs

For your convenience there is an explanation of each yaim variable. For more details look at [6, 7, 8, 9]
%ENDTWISTY%

%TWISTY{
mode="div"
showlink="   *services/glite-glexec_wn* "
hidelink="     *services/glite-glexec_wn* " 
remember="off" firststart="hide"
showimgright="%ICONURLPATH{toggleopen}%"
hideimgright="%ICONURLPATH{toggleclose}%"
}%
---+++ services/glite-glexec_wn
<verbatim>
##############################################################################
# Copyright (c) Members of the EGEE Collaboration. 2004.
# See http://www.eu-egee.org/partners/ for details on the copyright
# holders.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS
# OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
##############################################################################
#
# NAME :        glite-glexec_wn
#
# DESCRIPTION : This configuration file contains the variables needed to configure the
#               glexec for the WN. Sys admins must define these variables.                
#
# AUTHORS :     yaim-contact@cern.ch
#
# NOTES :
#
# YAIM MODULE:  glite-yaim-clients
#
##############################################################################

# Define this variable to configure glexec to work with SCAS.
# - yes : means you want to use a SCAS server and therefore you need to define:
# SCAS_ENDPOINTS="https://scas1.site.com:8443 https://scas2.site.com:8443"
# Alternatively, the old style variables can be used as well:
#         - SCAS_HOST="scas server hostname"
#         - SCAS_PORT="scas server port"
# - no  : means you don't want to use any SCAS server.
GLEXEC_WN_SCAS_ENABLED="no"

# Define this variable to configure glexec to use the ARGUS authorization framework.
# - yes : means you want to use ARGUS and therefore you need to define:
# ARGUS_PEPD_ENDPOINTS="http://argus1.site.com:8154/authz http://argus2.site.com:8154/authz"
# A list of endpoints for the pepc plugin to try.
# - no  : means you don't want to use ARGUS.
GLEXEC_WN_ARGUS_ENABLED="yes"

# Note that if both GLEXEC_WN_SCAS_ENABLED = yes and GLEXEC_WN_ARGUS_ENABLED = yes then
# the policy is to use ARGUS first, then SCAS. This may be useful if e.g. you use
# ARGUS for global banning and SCAS for account mapping, but typically you want just
# one or the other, not both.

# Define this variable to configure the operation mode of glexec in your WN.
# The possibilities are:
# - setuid   : it will actually enable glexec to do the identity change
# - log-only : it won't do any identity change. If you select log-only, it
#              doesn't matter whether SCAS is enabled or not. It isn't used.
GLEXEC_WN_OPMODE="setuid"           

# Optional variable to tell glexec where to send the glexec logging information.
# There are two values: 'syslog' and 'file'. The default is 'syslog'
# The value 'syslog' puts all messages in the syslog
# and 'file' puts the messages in a file.
# Define this variable if you want to specify a file.
# For value 'file' the variable GLEXEC_WN_LOG_FILE defines the location
# of the log file.
# REMEMBER that for log-only mode, 'syslog' should be used !
# GLEXEC_WN_LOG_DESTINATION=file

# Optional variable to add additional users to the glexec white list,
# e.g. for local testing purposes. Syntax is comma separated user and/or
# pool names.
# GLEXEC_EXTRA_WHITELIST="john,fred,.pvier"
GLEXEC_EXTRA_WHITELIST="misva"

# Glexec user home dir; this optional variable is passed to the adduser
# call only if it is set.
# GLEXEC_USER_HOME=/var/lib/glexec

# Variables to set the locking mechanism used by glexec, for
# the input proxies and the target proxy
# Allowed values are flock, fcntl, disabled.
# GLEXEC_WN_INPUT_LOCK=flock
# GLEXEC_WN_TARGET_LOCK=flock
</verbatim>

%ENDTWISTY%

%TWISTY{
mode="div"
showlink="   *munge configuration* "
hidelink="     *munge configuration* " 
remember="off" firststart="hide"
showimgright="%ICONURLPATH{toggleopen}%"
hideimgright="%ICONURLPATH{toggleclose}%"
}%
---+++ munge configuration
*IMPORTANT*: The updated EPEL5 build of torque-2.5.7-1 as compared to previous versions enables munge as an inter node authentication method.

   * verify that munge is correctly installed:
<verbatim>
# rpm -qa | grep munge
munge-libs-0.5.8-8.el5
munge-0.5.8-8.el5
</verbatim>
   * On one host (for example the batch server) generate a key by launching:
<verbatim>
# /usr/sbin/create-munge-key

# ls -ltr /etc/munge/
total 4
-r-------- 1 munge munge 1024 Jan 13 14:32 munge.key
</verbatim>
   * Copy the key, /etc/munge/munge.key to every host of your cluster, adjusting the permissions:
<verbatim>
# chown munge:munge /etc/munge/munge.key
</verbatim>
   * Start the munge daemon on each node:
<verbatim>
# service munge start
Starting MUNGE:                                            [  OK  ]

# chkconfig munge on
</verbatim>

%ENDTWISTY%

%TWISTY{
mode="div"
showlink="   *software area settings* "
hidelink="     *software area settings* " 
remember="off" firststart="hide"
showimgright="%ICONURLPATH{toggleopen}%"
hideimgright="%ICONURLPATH{toggleclose}%"
}%
---+++ software area settings
you have to import the software area from CE.
   * Edit the file /etc/fstab by adding a line like the following:
<verbatim>
cremino.cnaf.infn.it:/opt/exp_soft/ /opt/exp_soft/ nfs rw,defaults 0 0
</verbatim>
   * check nfs and portmap status
<verbatim>
# service nfs status
rpc.mountd is stopped
nfsd is stopped

# service portmap status
portmap is stopped

# service portmap start
Starting portmap:                                          [  OK  ]

# service nfs start
Starting NFS services:                                     [  OK  ]
Starting NFS daemon:                                       [  OK  ]
Starting NFS mountd:                                       [  OK  ]
Starting RPC idmapd:                                       [  OK  ]

# chkconfig nfs on
# chkconfig portmap on
</verbatim>
   * after any modification in /etc/fstab launch
<verbatim>
mount -a
</verbatim>
   * verify the mount:
<verbatim>
# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3              65G  1.9G   59G   4% /
/dev/sda1              99M   18M   76M  19% /boot
tmpfs                 2.0G     0  2.0G   0% /dev/shm
cremino.cnaf.infn.it:/opt/exp_soft/
                       65G  4.4G   57G   8% /opt/exp_soft
</verbatim>

%ENDTWISTY%

%TWISTY{
mode="div"
showlink="   *yaim check* "
hidelink="     *yaim check* " 
remember="off" firststart="hide"
showimgright="%ICONURLPATH{toggleopen}%"
hideimgright="%ICONURLPATH{toggleclose}%"
}%
---+++ yaim check
Verify to have set all the yaim variables by launching:
<verbatim>
# /opt/glite/yaim/bin/yaim -v -s site-info_cremino.def -n WN -n TORQUE_client -n GLEXEC_wn
</verbatim>

see [[WNTorqueGLEXECYaimVerConf#YAIM_Verification][details]]
%ENDTWISTY%

%TWISTY{
mode="div"
showlink="   *yaim config* "
hidelink="     *yaim config* " 
remember="off" firststart="hide"
showimgright="%ICONURLPATH{toggleopen}%"
hideimgright="%ICONURLPATH{toggleclose}%"
}%
---+++ yaim config
<verbatim>
# /opt/glite/yaim/bin/yaim -c -s site-info_cremino.def -n WN -n TORQUE_client -n GLEXEC_wn
</verbatim>

see [[WNTorqueGLEXECYaimVerConf#YAIM_Configuration][details]]

%ENDTWISTY%

---++ Service checks

%TWISTY{
mode="div"
showlink="   *software area* "
hidelink="     *software area* " 
remember="off" firststart="hide"
showimgright="%ICONURLPATH{toggleopen}%"
hideimgright="%ICONURLPATH{toggleclose}%"
}%
verify that the ownership and permissions of the software area are properly set
<verbatim>
# ls -ltr /opt/exp_soft/
total 28
drwxrwxr-x 2 sgmops001      sgmops      4096 Jan 16 10:26 ops
drwxrwxr-x 2 sgminfngrid001 sgminfngrid 4096 Jan 16 10:26 infngrid
drwxrwxr-x 2 sgmgridit001   sgmgridit   4096 Jan 16 10:26 gridit
drwxr-xr-x 2 sgmglast       glast       4096 Jan 16 10:26 glast
drwxrwxr-x 2 sgmenmr001     sgmenmr     4096 Jan 16 10:26 enmr
drwxrwxr-x 2 sgmdteam001    sgmdteam    4096 Jan 16 10:26 dteam
drwxrwxr-x 2 sgmcomputer001 sgmcomputer 4096 Jan 16 10:26 computer
</verbatim>

%ENDTWISTY%

%TWISTY{
mode="div"
showlink="   *ssh communication* "
hidelink="     *ssh communication* " 
remember="off" firststart="hide"
showimgright="%ICONURLPATH{toggleopen}%"
hideimgright="%ICONURLPATH{toggleclose}%"
}%
ssh has to work without using password from WN to CE:
   * on a WN:
<verbatim>
[root@wn01 ~]# su - dteam002
[dteam002@wn01 ~]$ ssh cremino
Last login: Tue Jan 17 20:50:02 2012 from vwn-tf-24.cnaf.infn.it
 ___ _   _ _____ _   _        ____ _   _    _    _____
|_ _| \ | |  ___| \ | |      / ___| \ | |  / \  |  ___|
 | ||  \| | |_  |  \| |_____| |   |  \| | / _ \ | |_
 | || |\  |  _| | |\  |_____| |___| |\  |/ ___ \|  _|
|___|_| \_|_|   |_| \_|      \____|_| \_/_/   \_\_|
[dteam002@cremino ~]$
</verbatim>
%ENDTWISTY%

---++ Revisions
| *Date* | *Comment* | *By* |
|2012-01-25 | installation notes completed | Alessandro Paolini |
| 2012-01-20 | First draft   | Alessandro Paolini |

-- Main.AlessandroPaolini - 2012-01-25
Topic revision: r5 - 2012-02-03 - AlessandroPaolini
Account
- Log In
~~Edit~~
~~Attach~~