Difference: SystemAdministratorGuideForEMI1 (8 vs. 9)
Revision 92011-04-19 - MassimoSgaravatto
| Line: 1 to 1 | |||||||||
|---|---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | |||||||||
| Line: 10 to 9 | |||||||||
0.1 Prerequisites0.1.1 Operating system | |||||||||
| Deleted: | |||||||||
| < < | A standard 64 bit SL(C)5 distribution is supposed to be properly installed. | ||||||||
| Added: | |||||||||
| > > | A standard 64 bit SL(C)5 distribution is supposed to be properly installed. | ||||||||
0.0.1 Node synchronization | |||||||||
| Deleted: | |||||||||
| < < | A general requirement for the Grid nodes is that they are synchronized. This requirement may be fulfilled in several ways. One of the most common one is using the NTP protocol with a time server. | ||||||||
| Added: | |||||||||
| > > | A general requirement for the Grid nodes is that they are synchronized. This requirement may be fulfilled in several ways. One of the most common one is using the NTP protocol with a time server. | ||||||||
0.0.1 Cron and logrotate | |||||||||
| Deleted: | |||||||||
| < < | Many components deployed on the CREAM CE rely on the presence of cron (including support for /etc/cron.* directories) and logrotate. You should make sure these utils are available on your system. | ||||||||
| Added: | |||||||||
| > > | Many components deployed on the CREAM CE rely on the presence of cron (including support for /etc/cron.* directories) and logrotate. You should make sure these utils are available on your system. | ||||||||
0.0.1 Batch system | |||||||||
| Deleted: | |||||||||
| < < | If you plan to use LSF as batch system for your CREAM CE, you have to install and configure it before installing and configuring the CREAM software. Since LSF is a commercial software it can't be distributed together with the middleware. | ||||||||
| Added: | |||||||||
| > > | If you plan to use LSF as batch system for your CREAM CE, you have to install and configure it before installing and configuring the CREAM software. Since LSF is a commercial software it can't be distributed together with the middleware. | ||||||||
0.1 Plan how to deploy the CREAM CE | |||||||||
| Line: 92 to 86 | |||||||||
| BLPARSER_WITH_UPDATER_NOTIFIER=true | |||||||||
| Changed: | |||||||||
| < < | in the siteinfo.def. This is the default value.
The new BLParser doesn't parse the log files. However the bhist (for LSF) and tracejob (for Torque) commands (used by the new BLParser) require the batch system log files, which therefore must be available (in case e.g. via NFS in the CREAM CE node.
Actually for Torque the blparser uses tracejob (which requires the log files) only when qstat can't find anymore the job. And this can happen if the job has been completed more than keep_completed seconds ago and the blparser was not able to detect before that the job completed/was cancelled/whatever. This can happen e.g. if keep_completed is too short or if the BLAH blparser for whatever reason didn't run for a while. If the log files are not available and the tracejob command is issued (for the reasons specified above), the BLAH blparser will not be able to find the job, which will considered "lost" (DONE-FAILED wrt CREAM). | ||||||||
| > > | in the siteinfo.def and then configure the CREAM CE. This is the default value. | ||||||||
| Added: | |||||||||
| > > | The new BLParser doesn't parse the log files. However the bhist (for LSF) and tracejob (for Torque) commands (used by the new BLParser) require the batch system log files, which therefore must be available (in case e.g. via NFS in the CREAM CE node. Actually for Torque the blparser uses tracejob (which requires the log files) only when qstat can't find anymore the job. And this can happen if the job has been completed more than keep_completed seconds ago and the blparser was not able to detect before that the job completed/was cancelled/whatever. This can happen e.g. if keep_completed is too short or if the BLAH blparser for whatever reason didn't run for a while. If the log files are not available and the tracejob command is issued (for the reasons specified above), the BLAH blparser will not be able to find the job, which will considered "lost" (DONE-FAILED wrt CREAM). | ||||||||
| Changed: | |||||||||
| < < | The init script of the new Blparser is /etc/init.d/glite-ce-blahparser.
Please note that it is not needed to explicitly start the new blparser: when CREAM is started, it starts also this new BLAH Blparser if it is not already running. | ||||||||
| > > | The init script of the new Blparser is /etc/init.d/glite-ce-blahparser. Please note that it is not needed to explicitly start the new blparser: when CREAM is started, it starts also this new BLAH Blparser if it is not already running. | ||||||||
| When the new Blparser is running, you should see the following two processes on the CREAM CE node: | |||||||||
| Line: 106 to 97 | |||||||||
| |||||||||
| Deleted: | |||||||||
| < < | |||||||||
Please note that the user tomcat on the CREAM CE should be allowed to issue the relevant status/history commands (for Torque: qstat, tracejob, for LSF: bhist, bjobs). Some sites configure the batch system so that users can only see their own jobs (e.g. in torque: | |||||||||
| Changed: | |||||||||
| < < | set server query_other_jobs = False). If this is done at the site, then the tomcat user will need a special privilege in order to be exempt from this setting (in torque: set server operators += tomcat@creamce.yoursite.domain). | ||||||||
| > > | set server query_other_jobs = False). If this is done at the site, then the tomcat user will need a special privilege in order to be exempt from this setting (in torque: set server operators += tomcat@creamce.yoursite.domain). 0.0.0.1 Old BLAH BlparserThe old BLAH blparser must be installed on a machine where the batch system log files are available (let's call this hostBLPARSER_HOST. So the BLPARSER_HOST can be the batch system master or a different machine where the log files are available (e.g. they have been exported via NFS). There are two possible layouts:
BLPARSER_WITH_UPDATER_NOTIFIER=truein the siteinfo.def before configuring via yaim. | ||||||||
0.1 Installation | |||||||||
| Line: 129 to 145 | |||||||||
| |||||||||
| Deleted: | |||||||||
| < < | |||||||||
and to REMOVE (!!!) or DEACTIVATE (!!!)
| |||||||||
| Line: 305 to 307 | |||||||||
| yum install emi-cluster | |||||||||
| Added: | |||||||||
| > > | 0.0.1 Installation of the BLAH BLparserIf the new BLAH Blparser must be used, there isn't anything to be installed for the BLAH Blparser (i.e. the installation of the CREAM-CE is enough). This is also the case when the old BLAH Blparser must be used AND the BLPARSER_HOST is the CREAM-CE. Only when the old BLAH Blparser must be used AND the BLPARSER_HOST is different than the CREAM-CE, it is necessary to install the BLParser software on this BLPARSER_HOST. This is done in the following way:yum install glite-ce-blahp yum install glite-yaim-cream-ce | ||||||||
0.0.1 Installation of the CREAM CLI | |||||||||
| Line: 329 to 342 | |||||||||
| |||||||||
| Changed: | |||||||||
| < < |
make sure to place the two files in the target node into the /etc/grid-security directory.
Then set the proper mode and ownerships doing: | ||||||||
| > > | make sure to place the two files in the target node into the /etc/grid-security directory. Then set the proper mode and ownerships doing: | ||||||||
chown root.root /etc/grid-security/hostcert.pem | |||||||||
| Line: 374 to 383 | |||||||||
| |||||||||
| Changed: | |||||||||
| < < |
make sure to place the two files in the target node into the /etc/grid-security directory.
Then set the proper mode and ownerships doing: | ||||||||
| > > | make sure to place the two files in the target node into the /etc/grid-security directory. Then set the proper mode and ownerships doing: | ||||||||
chown root.root /etc/grid-security/hostcert.pem | |||||||||
| Line: 427 to 432 | |||||||||
| |||||||||
| Changed: | |||||||||
| < < |
make sure to place the two files in the target node into the /etc/grid-security directory.
Then set the proper mode and ownerships doing: | ||||||||
| > > | make sure to place the two files in the target node into the /etc/grid-security directory. Then set the proper mode and ownerships doing: | ||||||||
chown root.root /etc/grid-security/hostcert.pem | |||||||||
| Line: 438 to 441 | |||||||||
| chmod 400 /etc/grid-security/hostkey.pem | |||||||||
| Deleted: | |||||||||
| < < | |||||||||
0.0.0.1 Configure the siteinfo.def file | |||||||||
| Deleted: | |||||||||
| < < | Set your siteinfo.def file, which is the input file used by yaim. Documentation about yaim variables relevant for glite-CLUSTER is available at https://twiki.cern.ch/twiki/bin/view/LCG/Site-info_configuration_variables#CLUSTER | ||||||||
| Added: | |||||||||
| > > | Set your siteinfo.def file, which is the input file used by yaim. Documentation about yaim variables relevant for glite-CLUSTER is available at https://twiki.cern.ch/twiki/bin/view/LCG/Site-info_configuration_variables#CLUSTER | ||||||||
0.0.0.1 Run yaim | |||||||||
| Line: 452 to 453 | |||||||||
| /opt/glite/yaim/bin/yaim -c -s <site-info.def> -n glite-CLUSTER | |||||||||
| Added: | |||||||||
| > > | 0.0.1 Configuration of the BLAH BlparserIf the new BLAH Blparser must be used, there isn't anything to be configured for the BLAH Blparser (i.e. the configuration of the CREAM-CE is enough). If the old BLparser must be used, it is necessary to configure it on the BLPARSER_HOST (which, as said above, can be the CREAM-CE node or on a different host). This is done in the following way:/opt/glite/yaim/bin/yaim -r -s <site-info.def> -n creamCE -f config_cream_blparserThen it is necessary to restart tomcat on the CREAM-CE node: service tomcat5 restart 0.0.1.1 Configuration of the old BLAH Blparser to serve multiple CREAM CEsThe configuration instructions reported above explains how to configure a CREAM CE and the BLAH blparser (old model) considering the scenario where the BLAH blparser has to "serve" a single CREAM CE. Considering that the blparser (old model) has to run where the batch system log files are available, let's consider a scenario where there are 2 CREAM CEs (ce1.mydomain and ce2.mydomain) that must be configured. Let's suppose that the batch system log files are not available on these 2 CREAM CEs machine. Let's assume they are available in another machine ( blhost.mydomain), where the old blparser has to be installed.
The following summarizes what must be done:
BLPARSER_HOST=blhost.mydomain BLAH_JOBID_PREFIX=cre01_ BLP_PORT=33333and configure ce1.mydomain via yaim:
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n <LRMSnode> [-n glite-CLUSTER]
BLPARSER_HOST=blhost.mydomain BLAH_JOBID_PREFIX=cre02_ BLP_PORT=33334and configure ce2.mydomain via yaim:
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n <LRMSnode> [-n glite-CLUSTER]
CREAM_PORT=56565and configure blhost.mydomain via yaim:
/opt/glite/yaim/bin/yaim -r -s <site-info.def> -n creamCE -f config_cream_blparser
GLITE_CE_BLPARSERPBS_NUM=2 # ce01.mydomain GLITE_CE_BLPARSERPBS_PORT1=33333 GLITE_CE_BLPARSERPBS_CREAMPORT1=56565 # ce02.mydomain GLITE_CE_BLPARSERPBS_PORT2=33334 GLITE_CE_BLPARSERPBS_CREAMPORT2=56566
/etc/init.d/glite-ce-blparser restart
| ||||||||
0.0.1 Configuration of the CREAM CLIThe CREAM CLI is part of the EMI-UI. To configure it please refer to xxx. | |||||||||
| Deleted: | |||||||||
| < < | -- MassimoSgaravatto - 2011-04-07 | ||||||||
| \ No newline at end of file | |||||||||
| Added: | |||||||||
| > > | 1 Operating the system1.1 Tomcat configuration guidelinesIn/etc/tomcat5/tomcat5.conf, there are some settings related to heap. They are in the JAVA_OPTS setting (see -Xms and -Xmx).
It is suggested to customize such settings taking into account how much physical memory is available, as indicated in the following table (which refers to 64bit architectures):
1.2 How to start the CREAM serviceA site admin can start the CREAM service just starting the CREAM container:/etc/init.d/tomcat5 startIn case the new BLAH blparser is used, this will also start it (if not already running). If for some reason it necessary to explicitly start the new BLAH blparser, the following command can be used: /etc/init.d/glite-ce-blahparser startIf instead the old BLAH blparser is used, before starting tomcat it is necessary to start it on the BLPARSER_HOST using the command: /etc/init.d/glite-ce-blparser startTo stop the CREAM service, it is just necessary to stop the CREAM container: /etc/init.d/tomcat5 stop 1.3 DaemonsInformation about daemons running in the CREAM CE is available in http://wiki.italiangrid.org/twiki/bin/view/CREAM/ServiceReferenceCard#Daemons_running
1.4 Init scriptsInformation about init scripts in the CREAM CE is available in the http://wiki.italiangrid.org/twiki/bin/view/CREAM/ServiceReferenceCard#Init_scripts_and_options_start_s
1.5 Configuration filesInformation about configuration files in the CREAM CE is available in http://wiki.italiangrid.org/twiki/bin/view/CREAM/ServiceReferenceCard#Configuration_files_location_wit
1.6 Log filesInformation about log files in the CREAM CE is available in http://wiki.italiangrid.org/twiki/bin/view/CREAM/ServiceReferenceCard#Logfile_locations_and_management
1.7 Network portsInformation about ports used in the CREAM CE is available in http://wiki.italiangrid.org/twiki/bin/view/CREAM/ServiceReferenceCard#Open_ports
1.8 Cron jobsInformation about cron jobs used in the CREAM CE is available in http://wiki.italiangrid.org/twiki/bin/view/CREAM/ServiceReferenceCard#Cron_jobs
1.9 Security related operations1.9.1 Security recommendationsSecurity recommendations relevant for the CREAM CE is available in http://wiki.italiangrid.org/twiki/bin/view/CREAM/ServiceReferenceCard#Security_recommendations
1.9.2 How to block/ban a userInformation about how to ban users is available in http://wiki.italiangrid.org/twiki/bin/view/CREAM/ServiceReferenceCard#How_to_block_ban_a_user
1.9.3 How to block/ban a VOTo ban a VO, it is suggested to reconfigure the service via yaim without that VO in thesiteinfo.def
1.9.4 How to define a CREAM administratorA CREAM administrator (aka super-user) can manage (e.g. cancel, check the status, etc.) also the jobs submitted by other people. Moreover he/she can issue some privileged operations, in particular the ones to disable the new job submissions (glite-ce-disable-submission) and then to re-enable them (glite-ce-disable-submission)
To define a CREAM CE administrator for a specific CREAM CE, the DN of this person must be specified in the /etc/grid-security/admin-list of this CREAM CE node, e.g.:
"/C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto"Please note that including the DN between " is important http://wiki.italiangrid.org/twiki/bin/view/CREAM/ServiceReferenceCard#Cron_jobs -- MassimoSgaravatto - 2011-04-07 | ||||||||
View topic | History: r69 < r68 < r67 < r66 | More topic actions...
Copyright © 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors. Ideas, requests, problems regarding TWiki? Send feedback