System Administrator Guide for CREAM for EMI-1 release
1 Installation and Configuration
1.1 Prerequisites
1.1.1 Operating system
A standard 64 bit SL(C)5 distribution is supposed to be properly installed.
1.1.2 Node synchronization
A general requirement for the Grid nodes is that they are synchronized. This requirement may be fulfilled in several ways. One of the most common one is using the
NTP
protocol with a time server.
1.1.3 Cron and logrotate
Many components deployed on the CREAM CE rely on the presence of
cron
(including support for
/etc/cron.*
directories) and
logrotate
. You should make sure these utils are available on your system.
1.1.4 Batch system
If you plan to use LSF as batch system for your CREAM CE, you have to install and configure it before installing and configuring the CREAM software. Since LSF is a commercial software it can't be distributed together with the middleware.
1.2 Plan how to deploy the CREAM CE
1.2.1 CREAM CE and gLite-cluster
glite-CLUSTER is a node type that can publish information about clusters and subclusters in a site, referenced by any number of compute elements.
glite-CLUSTER can be deployed in the same host of the CREAM-CE or in a different one.
The following deployment models are possible:
- CREAM-CE can be configured without worrying about the glite-CLUSTER node. This can be useful for small sites who don't want to worry about cluster/subcluster configurations because they have a very simple setup. In this case CREAM-CE will publish a single cluster/subcluster. This is called no cluster mode. This is done as described below by defining the yaim setting
CREAMCE_CLUSTER_MODE=no
(or by no defining at all that variable).
- CREAM-CE can work on cluster mode using the glite-CLUSTER node type. This is done as described below by defining the yaim setting
CREAMCE_CLUSTER_MODE=yes
. The CREAM-CE can be in the same host or in a different host from the glite-CLUSTER node.
More information about glite-CLUSTER can be found at
https://twiki.cern.ch/twiki/bin/view/LCG/CLUSTER
1.2.2 Choose the authorization model
The CREAM CE can be configured to use as authorization system:
- the ARGUS authorization framework
OR
- the grid Java Authorization Framework (gJAF)
In the former case a ARGUS box (usually at site level) where to define policies for the CREAM CE box is needed.
To use ARGUS as authorization system, yaim variable
USE_ARGUS
must be set in the following way:
USE_ARGUS=yes
In this case it is also necessary to set the following yaim variables:
-
ARGUS_PEPD_ENDPOINTS
The endpoint of the ARGUS box (e.g."https://cream-43.pd.infn.it:8154/authz")
-
CREAM_PEPC_RESOURCEID
The id of the CREAM CE in the ARGUS box (e.g. "http://pd.infn.it/cream-18")
If instead gJAF should be used as authorization system, yaim variable
USE_ARGUS
must be set in the following way:
USE_ARGUS=no
1.2.3 Choose the BLAH BLparser deployment model
The BLAH Blparser is the component of the CREAM CE responsible to notify CREAM about job status changes.
For LSF and PBS/Torque it is possible to configure the BLAH blparser in two possible ways:
- The new BLAH BLparser, which relies on the status/history batch system commands
- The old BLAH BLparser, which parses the batch system log files
For SGE and Condor, only the configuration with the new BLAH blparser is possible
1.2.3.1 New BLAH Blparser
The new Blparser runs on the CREAM CE machine and it is automatically installed when installing the CREAM CE. The configuration of the new BLAH Blparser is done when configuring the CREAM CE (i.e. it is not necessary to configure the Blparser separately from the CREAM CE).
To use the new BLAH blparser, it is just necessary to set:
BLPARSER_WITH_UPDATER_NOTIFIER=true
in the siteinfo.def. This is the default value.
The new BLParser doesn't parse the log files. However the
bhist
(for LSF) and
tracejob
(for Torque) commands (used by the new BLParser) require the batch system log files, which therefore must be available (in case e.g. via NFS in the CREAM CE node.
Actually for Torque the blparser uses
tracejob
(which requires the log files) only when
qstat
can't find anymore the job. And this can happen if the job has been completed more than
keep_completed
seconds ago and the blparser was not able to detect before that the job completed/was cancelled/whatever. This can happen e.g. if keep_completed is too short or if the BLAH blparser for whatever reason didn't run for a while. If the log files are not available and the
tracejob
command is issued (for the reasons specified above), the BLAH blparser will not be able to find the job, which will considered "lost" (DONE-FAILED wrt CREAM).
The init script of the new Blparser is
/etc/init.d/glite-ce-blahparser
.
Please note that it is not needed to explicitly start the new blparser: when CREAM is started, it starts also this new BLAH Blparser if it is not already running.
When the new Blparser is running, you should see the following two processes on the CREAM CE node:
-
/usr/bin/BUpdaterxxx
-
/usr/bin/BNotifier
Please note that the user
tomcat
on the CREAM CE should be allowed to issue the relevant status/history commands (for Torque:
qstat
,
tracejob
, for LSF:
bhist
,
bjobs
). Some sites configure the batch system so that users can only see their own jobs (e.g. in torque:
set server query_other_jobs = False
). If this is done at the site, then the tomcat user will need a special privilege in order to be exempt from this setting (in torque:
set server operators += tomcat@creamce.yoursite.domain
).
1.3 Installation
This section explains how to install:
- a CREAM CE in no cluster mode
- a CREAM CE in cluster mode
- a glite-CLUSTER node
For all these scenarios, the setting of the repositories is the same.
1.3.1 Repositories
For a successful installation, you will need to configure your package manager to reference a number of repositories (in addition to your OS);
- the EPEL repository
- the EMI middleware repository
- the CA repository
and to
REMOVE (!!!) or
DEACTIVATE (!!!)
1.3.1.1 The EPEL repository
You can install the EPEL repository, issuing:
rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-4.noarch.rpm
1.3.1.2 The EMI middleware repository
The EMI-1 RC3 repository can be found under:
http://emisoft.web.cern.ch/emisoft/dist/EMI/1/RC3/sl5/x86_64
To use yum, the yum repo to be installed in
/etc/yum.repos.d
can be found at
https://twiki.cern.ch/twiki/pub/EMI/EMI-1/rc3.repo
1.3.1.3 The Certification Authority repository
The most up-to-date version of the list of trusted Certification Authorities (CA) is needed on your node. The relevant yum repo can be installed issuing:
wget http://repository.egi.eu/sw/production/cas/1/current/repo-files/egi-trustanchors.repo -O /etc/yum.repos.d/egi-trustanchors.repo
1.3.1.4 Important note on automatic updates
An update of an RPM not followed by configuration can cause problems. Therefore
WE STRONGLY RECOMMEND NOT TO USE AUTOMATIC UPDATE PROCEDURE OF ANY KIND.
Running the script available at
http://forge.cnaf.infn.it/frs/download.php/101/disable_yum.sh (implemented by Giuseppe Platania (INFN Catania) yum autoupdate will be disabled
1.3.2 Installation of a CREAM CE node in no cluster mode
First of all, install the
yum-protectbase
rpm:
yum install yum-protectbase.noarch
Then proceed with the installation of the CA certificates.
1.3.2.1 Installation of the CA certificates
The CA certificate can be installed issuing:
yum install ca-policy-egi-core
1.3.2.2 Installation of the CREAM CE software
To install the middleware software needed for the CREAM CE, install first of all Sun JDK (
jdk
) or openjdk (
java-1.6.0-openjdk
)
Then install
xml-commons-apis
:
yum install xml-commons-apis
This is due to a dependency problem within the Tomcat distribution
Then install the CREAM-CE metapackage:
yum install emi-cream-ce
1.3.2.3 Installation of the batch system specific software
After the installation of the CREAM CE metapackage it is necessary to install the batch system specific metapackage(s).
If you are running LSF, install the
emi-lsf-utils
metapackage:
yum install emi-lsf-utils
1.3.3 Installation of a CREAM CE node in cluster mode
First of all, install the
yum-protectbase
rpm:
yum install yum-protectbase.noarch
Then proceed with the installation of the CA certificates.
1.3.3.1 Installation of the CA certificates
The CA certificate can be installed issuing:
yum install ca-policy-egi-core
1.3.3.2 Installation of the CREAM CE software
To install the middleware software needed for the CREAM CE, install first of all Sun JDK (
jdk
) or openjdk (
java-1.6.0-openjdk
)
Then install
xml-commons-apis
:
yum install xml-commons-apis
This is due to a dependency problem within the Tomcat distribution
Then install the CREAM-CE metapackage:
yum install emi-cream-ce
1.3.3.3 Installation of the batch system specific software
After the installation of the CREAM CE metapackage it is necessary to install the batch system specific metapackage(s).
If you are running LSF, install the
emi-lsf-utils
metapackage:
yum install emi-lsf-utils
1.3.3.4 Installation of the cluster metapackage
If the CREAM CE node has to host also the
glite-cluster
, install also this metapackage:
yum install emi-cluster
1.3.4 Installation of a glite-cluster node
First of all, install the
yum-protectbase
rpm:
yum install yum-protectbase.noarch
Then proceed with the installation of the CA certificates.
1.3.4.1 Installation of the CA certificates
The CA certificate can be installed issuing:
yum install ca-policy-egi-core
1.3.4.2 Installation of the cluster metapackage
Install the glite-CLUSTER metapackage:
yum install emi-cluster
1.3.5 Installation of the CREAM CLI
The CREAM CLI is part of the EMI-UI. To install it please refer to xxx.
1.4 Configuration
1.4.1 Using the YAIM configuration tool
For a detailed description on how to configure the middleware with YAIM, please check the
YAIM guide.
The necessary YAIM modules needed to configure a certain node type are automatically installed with the middleware.
1.4.2 Configuration of a CREAM CE node in no cluster mode
1.4.2.1 Install host certificate
The CREAM CE node requires the host certificate/key files to be installed. Contact your national Certification Authority (CA) to understand how to obtain a host certificate if you do not have one already.
Once you have obtained a valid certificate:
- hostcert.pem - containing the machine public key
- hostkey.pem - containing the machine private key
make sure to place the two files in the target node into the
/etc/grid-security
directory.
Then set the proper mode and ownerships doing:
chown root.root /etc/grid-security/hostcert.pem
chown root.root /etc/grid-security/hostkey.pem
chmod 600 /etc/grid-security/hostcert.pem
chmod 400 /etc/grid-security/hostkey.pem
1.4.2.2 Configure the siteinfo.def file
Set your
siteinfo.def
file, which is the input file used by yaim. Documentation about yaim variables relevant for CREAM CE is available at
https://twiki.cern.ch/twiki/bin/view/LCG/Site-info_configuration_variables#cream_CE
Be sure that
CREAMCE_CLUSTER_MODE
is set to
no
(or not set at all).
1.4.2.3 Run yaim
After having filled the
siteinfo.def
file, run yaim:
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n <LRMSnode>
Examples:
- Configuration of a CREAM CE in no cluster mode using LSF as batch system
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n LSF_utils
1.4.3 Configuration of a CREAM CE node in cluster mode
1.4.3.1 Install host certificate
The CREAM CE node requires the host certificate/key files to be installed. Contact your national Certification Authority (CA) to understand how to obtain a host certificate if you do not have one already.
Once you have obtained a valid certificate:
- hostcert.pem - containing the machine public key
- hostkey.pem - containing the machine private key
make sure to place the two files in the target node into the
/etc/grid-security
directory.
Then set the proper mode and ownerships doing:
chown root.root /etc/grid-security/hostcert.pem
chown root.root /etc/grid-security/hostkey.pem
chmod 600 /etc/grid-security/hostcert.pem
chmod 400 /etc/grid-security/hostkey.pem
1.4.3.2 Configure the siteinfo.def file
Set your
siteinfo.def
file, which is the input file used by yaim. Documentation about yaim variables relevant for CREAM CE is available at
https://twiki.cern.ch/twiki/bin/view/LCG/Site-info_configuration_variables#cream_CE
Be sure that
CREAMCE_CLUSTER_MODE
is set to
yes
1.4.3.3 Run yaim
After having filled the
siteinfo.def
file, run yaim:
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n <LRMSnode> [-n glite-CLUSTER]
-n glite-CLUSTER
must be specified only if the glite-CLUSTER is deployed in the same node of the CREAM-CE
Examples:
- Configuration of a CREAM CE in cluster mode (with glite-CLUSTER deployed on a different node) using LSF as batch system
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n LSF_utils
- Configuration of a CREAM CE in cluster mode (with glite-CLUSTER deployed on the same node of the CREAM-CE) using LSF as batch system
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n LSF_utils -n glite-CLUSTER
1.4.4 Configuration of a glite-CLUSTER node
1.4.4.1 Install host certificate
The glite-CLUSTER node requires the host certificate/key files to be installed. Contact your national Certification Authority (CA) to understand how to obtain a host certificate if you do not have one already.
Once you have obtained a valid certificate:
- hostcert.pem - containing the machine public key
- hostkey.pem - containing the machine private key
make sure to place the two files in the target node into the
/etc/grid-security
directory.
Then set the proper mode and ownerships doing:
chown root.root /etc/grid-security/hostcert.pem
chown root.root /etc/grid-security/hostkey.pem
chmod 600 /etc/grid-security/hostcert.pem
chmod 400 /etc/grid-security/hostkey.pem
1.4.4.2 Configure the siteinfo.def file
Set your
siteinfo.def
file, which is the input file used by yaim. Documentation about yaim variables relevant for glite-CLUSTER is available at
https://twiki.cern.ch/twiki/bin/view/LCG/Site-info_configuration_variables#CLUSTER
1.4.4.3 Run yaim
After having filled the
siteinfo.def
file, run yaim:
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n glite-CLUSTER
1.4.5 Configuration of the CREAM CLI
The CREAM CLI is part of the EMI-UI. To configure it please refer to xxx.
--
MassimoSgaravatto - 2011-04-07