Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 264 to 264 | ||||||||
Then install xml-commons-apis : | ||||||||
Changed: | ||||||||
< < | yum install xml-commons-apis | |||||||
> > | yum install xml-commons-apis java-1.6.0-openjdk-devel | |||||||
This is due to a dependency problem within the Tomcat distribution | ||||||||
Line: 327 to 327 | ||||||||
Then install xml-commons-apis : | ||||||||
Changed: | ||||||||
< < | yum install xml-commons-apis | |||||||
> > | yum install xml-commons-apis java-1.6.0-openjdk-devel | |||||||
This is due to a dependency problem within the Tomcat distribution |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 416 to 418 | ||||||||
The CREAM CLI is part of the EMI-UI. To install it please refer to https://twiki.cern.ch/twiki/bin/view/EMI/EMIui#Client_Installation_Configuratio . | ||||||||
Added: | ||||||||
> > |
0.1 CREAM CE updateTo update the CREAM CE from EMI-1 Update x to EMI-1 Update y:
| |||||||
0.1 CREAM CE configuration0.1.1 Using the YAIM configuration tool |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 1505 to 1505 | ||||||||
mysql> select jstd.name, count(*) from job, job_status_type_description jstd, job_status AS status LEFT OUTER JOIN job_status AS latest ON latest.jobId=status.jobId AND status.id < latest.id WHERE latest.id IS null and job.id=status.jobId and jstd.type=status.type group by jstd.name; | ||||||||
Added: | ||||||||
> > | 0.1 How to clean the CREAM databases, sandboxes, log files?[1] stop the serviceservice tomcat5 stop[2] delete cream log files rm -rf /var/log/cream/*[3] delete tomcat log files rm -rf /var/log/tomcat5/*[4] drop databases DROP DATABASE creamdb; DROP DATABASE delegationdb;[5] delete sandboxes (see the value of the "CREAM_SANDBOX_DIR" parameter in the /etc/glite-ce-cream/cream-config.xml file) rm -rf "CREAM_SANDBOX_DIR"[6] configure via yaim see: https://wiki.italiangrid.it/twiki/bin/view/CREAM/SystemAdministratorGuideForEMI1#1_4_CREAM_CE_configuration | |||||||
-- MassimoSgaravatto - 2011-04-07 |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 409 to 409 | ||||||||
yum install glite-ce-blahp | ||||||||
Changed: | ||||||||
< < | yum install glite-yaim-cream-ce | |||||||
> > | yum install glite-ce-yaim-cream-ce | |||||||
0.0.1 Installation of the CREAM CLI | ||||||||
Line: 1339 to 1339 | ||||||||
0.1 How to check if you are using the old or the new blparser | ||||||||
Changed: | ||||||||
< < | If you want to quickly check if you are using the old or the new BLAH Blparser, do a grep registry blah.config . If you see something like: | |||||||
> > | If you want to quickly check if you are using the old or the new BLAH Blparser, do a grep registry /etc/blah.config . If you see something like: | |||||||
Changed: | ||||||||
< < | # grep registry blah.config | |||||||
> > | # grep registry /etc/blah.config | |||||||
job_registry=/var/blah/user_blah_job_registry.bjr |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 213 to 213 | ||||||||
You can install the EPEL repository, issuing: | ||||||||
Changed: | ||||||||
< < | rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-4.noarch.rpm | |||||||
> > | rpm -Uvh http://dl.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm | |||||||
0.0.0.1 The EMI middleware repository |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 984 to 984 | ||||||||
- system( 'cp', $remotefile, $localfile ); + system( 'scp', "$remotemachine:$remotefile", $localfile ); } else { | ||||||||
Changed: | ||||||||
< < | - system( 'cp', $localfile, $remotefile" ); | |||||||
> > | - system( 'cp', $localfile, $remotefile ); | |||||||
+ system( 'scp', $localfile, "$remotemachine:$remotefile" ); } } |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 244 to 244 | ||||||||
First of all, install the yum-protectbase rpm: | ||||||||
Changed: | ||||||||
< < | yum install yum-protectbase.noarch | |||||||
> > | yum install yum-protectbase | |||||||
Then proceed with the installation of the CA certificates. | ||||||||
Line: 307 to 307 | ||||||||
First of all, install the yum-protectbase rpm: | ||||||||
Changed: | ||||||||
< < | yum install yum-protectbase.noarch | |||||||
> > | yum install yum-protectbase | |||||||
Then proceed with the installation of the CA certificates. | ||||||||
Line: 378 to 378 | ||||||||
First of all, install the yum-protectbase rpm: | ||||||||
Changed: | ||||||||
< < | yum install yum-protectbase.noarch | |||||||
> > | yum install yum-protectbase | |||||||
Then proceed with the installation of the CA certificates. |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 91 to 91 | ||||||||
| ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
0.0.1 Choose the authorization model |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 91 to 91 | ||||||||
| ||||||||
Added: | ||||||||
> > | * The use of alias is not supported for submissions through the gLite-WMS | |||||||
0.0.1 Choose the authorization model | ||||||||
Line: 162 to 163 | ||||||||
| ||||||||
Deleted: | ||||||||
< < | If the BLPARSER_HOST is the CREAM CE host, after having installed and configured the CREAM CE, it is necessary to configure the old BLAH Blparser as explained below. | |||||||
Changed: | ||||||||
< < | lasIf the BLPARSER_HOST is different than the CREAM CE host, after having installed and configured the CREAM CE it is necessary: | |||||||
> > | If the BLPARSER_HOST is the CREAM CE host, after having installed and configured the CREAM CE, it is necessary to configure the old BLAH Blparser (as explained below) and then to restart tomcat. If the BLPARSER_HOST is different than the CREAM CE host, after having installed and configured the CREAM CE it is necessary: | |||||||
| ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
After having configured CREAM, it is necessary to also configure the BLAH Blparser as explained below. On the CREAM CE, to use the old BLAH blparser, it is necessary to set: |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 840 to 840 | ||||||||
0.0.0.1 Tune the number of concurrent BLAH instancesTo allow a parallelism when interacting with the batch system, in particular to have a good throughput when submitting jobs to the batch system, CREAM can run multiple BLAH instances. A new instance is created whenever needed, till the maximum number defined in theetc/glite-ce-cream/cream-config.xml configuration file is reached. The relevant attribute is cream_concurrency_level . The default value is 50. | ||||||||
Added: | ||||||||
> > | This value should be usually fine. You might need to decrease it if you notice an overload of the batch system and many jobs aborted because the submission to the batch system failed with "send command timeout" error message.
Please note that with BLAH version < 1.16.3, there are some memory issues: each BLAH instance uses too much memory and therefore a high value of this parameter can cause memory problems. The problem is fixed with BLAH version 1.16.3. Besides this version of BLAH, it is also needed to have in blah.config :
job_registry_use_mmap=yeswhich is automatically added by yaim (starting with yaim-cream-ce version >= 4.2.2-1). Starting with yaim-cream-ce version >= 4.2.3-1, this value for the number of concurrent BLAH instances is configurable via yaim. The name of the relevant yaim variable is CREAM_CONCURRENCY_LEVEL . | |||||||
0.0.0.1 Tune the BLAH BUpdater polling frequency |
Line: 1 to 1 | |||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | |||||||||||||||||||||||||||||
Line: 497 to 497 | |||||||||||||||||||||||||||||
0.0.0.1 Configure the siteinfo.def file | |||||||||||||||||||||||||||||
Changed: | |||||||||||||||||||||||||||||
< < | Set your siteinfo.def file, which is the input file used by yaim. Documentation about yaim variables relevant for CREAM CE is available at https://twiki.cern.ch/twiki/bin/view/LCG/Site-info_configuration_variables#cream_CE | ||||||||||||||||||||||||||||
> > | Set your siteinfo.def file, which is the input file used by yaim.
Variables which are required in cluster mode are described below.
When the CREAM CE is configured in cluster mode it will stop publishing information about clusters and subclusters. That information should be published by the glite-CLUSTER node type instead. A specific set of yaim variables has been defined for configuring the information which is still required by the CREAM CE in cluster mode. The names of these variables follow this syntax:
| ||||||||||||||||||||||||||||
Be sure that CREAMCE_CLUSTER_MODE is set to yes |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 27 to 27 | ||||||||
If you plan to use LSF as batch system for your CREAM CE, you have to install and configure it before installing and configuring the CREAM software. Since LSF is a commercial software it can't be distributed together with the middleware. If you plan to use GE as batch system for your CREAM CE, you have to install and configure it before installing and configuring the CREAM software. The CREAM CE integration was tested with GE 6.2u5 but it should work with any forked version of the original GE software. The support of the GE batch system software (or any of its forked versions) is out of the scope of this activity. | ||||||||
Added: | ||||||||
> > | More information abut batch system integration is available in the relevant section. | |||||||
0.1 Plan how to deploy the CREAM CE0.1.1 CREAM CE and gLite-cluster | ||||||||
Line: 832 to 834 | ||||||||
| ||||||||
Changed: | ||||||||
< < | Batch system integrationGrid EngineRequirements | |||||||
> > | 0.1 Batch system integration0.1.1 Torque0.1.1.1 InstallationIf the CREAM-CE has to be also the torque server, install theemi-torque-server metapackage: yum install emi-torque-server
In all cases (Torque server in the CREAM-CE or in a different host) then install the emi-torque-utils metapackage: yum install emi-torque-utils
0.1.1.2 ConfigurationSet yoursiteinfo.def file, which is the input file used by yaim. Documentation about yaim variables relevant for CREAM CE is available at CREAM CE: https://twiki.cern.ch/twiki/bin/view/LCG/Site-info_configuration_variables#cream_CE
The CREAM CE Torque integration is then configured running YAIM:
0.1.2 LSF0.1.2.1 RequirementsYou have to install and configure the LSF batch system software before installing and configuring the CREAM software.0.1.2.2 InstallationIf you are running LSF, install theemi-lsf-utils metapackage: yum install emi-lsf-utils
0.1.2.3 ConfigurationSet yoursiteinfo.def file, which is the input file used by yaim. Documentation about yaim variables relevant for CREAM CE is available at CREAM CE: https://twiki.cern.ch/twiki/bin/view/LCG/Site-info_configuration_variables#cream_CE
The CREAM CE LSF integration is then configured running YAIM:
0.1.3 Grid Engine0.1.3.1 Requirements | |||||||
You have to install and configure the GE batch system software before installing and configuring the CREAM software. The CREAM CE integration was tested with GE 6.2u5 but it should work with any forked version of the original GE software. The support of the GE batch system software (or any of its forked versions) is out of the scope of this activity. | ||||||||
Line: 842 to 890 | ||||||||
| ||||||||
Changed: | ||||||||
< < | Integration plugins | |||||||
> > | 0.0.0.1 Integration plugins | |||||||
The GE integration with CREAM CE consists in deploying specific BLAH plugins and configure them to properly interoperate with Grid Engine batch system. The following GE BLAH plugins are deployed with CREAM CE installation: BUpdaterSGE, sge_hold.sh, sge_submit.sh, sge_resume.sh, sge_status.sh and sge_cancel. | ||||||||
Changed: | ||||||||
< < | Instalation | |||||||
> > | 0.0.0.1 Installation | |||||||
Changed: | ||||||||
< < | If you are running GE, install the emi-ge-utils metapackage: yum install emi-ge-utils | |||||||
> > | If you are running GE, install the emi-ge-utils metapackage: yum install emi-ge-utils | |||||||
Changed: | ||||||||
< < | Configuration | |||||||
> > | 0.0.0.1 Configuration | |||||||
Set your siteinfo.def file, which is the input file used by yaim. Documentation about yaim variables relevant for CREAM CE and GE is available at | ||||||||
Line: 870 to 918 | ||||||||
SGE_SHARED_INSTALL=yes in your site-info.def , otherwise YAIM may change your setup according to the definitions in your site-info.def . | ||||||||
Changed: | ||||||||
< < | The CREAM CE GE integration in then configured running YAIM: | |||||||
> > | The CREAM CE GE integration is then configured running YAIM: | |||||||
| ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
0.0.0.1 Important notes | |||||||
Changed: | ||||||||
< < | Important notesFile transfers | |||||||
> > | 0.0.0.0.1 File transfers | |||||||
Besides the input/output sandbox files (transfered via GFTP) there are some other files that need to be transferred from/to the CREAM sandbox directory on the CE node to/from the Worker Node, namely:
| ||||||||
Line: 901 to 950 | ||||||||
+ system( 'scp', $localfile, "$remotemachine:$remotefile" ); } } | ||||||||
Changed: | ||||||||
< < | GE accounting file | |||||||
> > |
0.0.0.0.1 GE accounting file | |||||||
BUpdaterSGE needs to consult the GE accounting file to determine how did a given job ended. Therefore, the GE accounting file must be shared between the GE SERVER / QMASTER and the CREAM CE. | ||||||||
Changed: | ||||||||
< < | Moreover, to guarantee that the accounting file is updated on the fly, the GE configuration should be tunned (using qconf -mconf) in order to add under the reporting_params the following definitions: accounting=true accounting_flush_time=00:00:00 | |||||||
> > | Moreover, to guarantee that the accounting file is updated on the fly, the GE configuration should be tunned (using qconf -mconf) in order to add under the reporting_params the following definitions: accounting=true accounting_flush_time=00:00:00 | |||||||
Changed: | ||||||||
< < | GE SERVER (QMASTER) tunning | |||||||
> > | 0.0.0.0.1 GE SERVER (QMASTER) tuning | |||||||
The following suggestions should be implemented to achieve better performance when integrating with CREAM CE:
|
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 26 to 26 | ||||||||
If you plan to use LSF as batch system for your CREAM CE, you have to install and configure it before installing and configuring the CREAM software. Since LSF is a commercial software it can't be distributed together with the middleware. | ||||||||
Changed: | ||||||||
< < | If you plan to use SGE as batch system for your CREAM CE, it will be installed and configured along with the middleware (i.e. you don't have to install and configure it in advance) | |||||||
> > | If you plan to use GE as batch system for your CREAM CE, you have to install and configure it before installing and configuring the CREAM software. The CREAM CE integration was tested with GE 6.2u5 but it should work with any forked version of the original GE software. The support of the GE batch system software (or any of its forked versions) is out of the scope of this activity. | |||||||
0.1 Plan how to deploy the CREAM CE0.1.1 CREAM CE and gLite-cluster | ||||||||
Line: 124 to 124 | ||||||||
| ||||||||
Changed: | ||||||||
< < | For SGE and Condor, only the configuration with the new BLAH blparser is possible | |||||||
> > | For GE and Condor, only the configuration with the new BLAH blparser is possible | |||||||
0.0.0.1 New BLAH Blparser | ||||||||
Line: 182 to 182 | ||||||||
Click here for information how to deploy the databases on a machine different wrt the CREAM-CE. | ||||||||
Changed: | ||||||||
< < | 0.1 Installation | |||||||
> > | 0.1 CREAM CE Installation | |||||||
This section explains how to install: | ||||||||
Line: 292 to 292 | ||||||||
yum install emi-lsf-utils | ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
yum install emi-ge-utils | ||||||||
Line: 355 to 355 | ||||||||
yum install emi-lsf-utils | ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
yum install emi-ge-utils | ||||||||
Line: 410 to 410 | ||||||||
The CREAM CLI is part of the EMI-UI. To install it please refer to https://twiki.cern.ch/twiki/bin/view/EMI/EMIui#Client_Installation_Configuratio . | ||||||||
Changed: | ||||||||
< < | 0.1 Configuration | |||||||
> > | 0.1 CREAM CE configuration | |||||||
0.0.1 Using the YAIM configuration tool | ||||||||
Line: 471 to 471 | ||||||||
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n LSF_utils | ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n SGE_utils 0.0.1 Configuration of a CREAM CE node in cluster mode | ||||||||
Line: 516 to 516 | ||||||||
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n LSF_utils | ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n SGE_utils
| ||||||||
Line: 537 to 537 | ||||||||
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n LSF_utils -n glite-CLUSTER | ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n SGE_utils -n glite-CLUSTER
| ||||||||
Line: 671 to 670 | ||||||||
| ||||||||
Deleted: | ||||||||
< < | ||||||||
0.0.0.1 Configuration of the new BLAH Blparser to to use cached batch system commands | ||||||||
Changed: | ||||||||
< < | With BLAH version >= 1.16-3, the new BLAH blparser can be configured in order to not interact directly with the batch system, but through a program (to be implemented by the site admin) which can implement some caching functionality. This is the case for example of CommandProxyTools , implement at Cern | |||||||
> > | With BLAH version >= 1.16-3, the new BLAH blparser can be configured in order to not interact directly with the batch system, but through a program (to be implemented by the site admin) which can implement some caching functionality. This is the case for example of CommandProxyTools , implement at Cern | |||||||
To enable this feature, add in /etc/blah.config (the example below is for lsf, with /usr/bin/runcmd.pl as name of the "caching" program): | ||||||||
Line: 684 to 681 | ||||||||
batch_command_caching_filter=/usr/bin/runcmd.pl | ||||||||
Changed: | ||||||||
< < | So the blparser, insead of issuing bjobs -u .... , will issue /usr/bin/runcmd.pl bjobs -u .. ."
| |||||||
> > | So the blparser, insead of issuing bjobs -u .... , will issue /usr/bin/runcmd.pl bjobs -u .. ." </verbatim> | |||||||
0.0.1 Configuration of the CREAM databases on a host different than the CREAM-CE | ||||||||
Line: 793 to 784 | ||||||||
mysqlshow --password="$MYSQL_PASSWORD" | grep "creamdatabase" > /dev/null 2>&1 | ||||||||
Deleted: | ||||||||
< < | ||||||||
Changed: | ||||||||
< < | ||||||||
> > | ||||||||
mysql -u root --password="$MYSQL_PASSWORD" -e "DROP DATABASE creamdatabase" | ||||||||
Deleted: | ||||||||
< < | ||||||||
Changed: | ||||||||
< < | ||||||||
> > | ||||||||
mysqlshow --password="$MYSQL_PASSWORD" | grep "delegationdatabase" > /dev/null 2>&1 | ||||||||
Deleted: | ||||||||
< < | ||||||||
Changed: | ||||||||
< < | ||||||||
> > | ||||||||
mysql -u root --password="$MYSQL_PASSWORD" -e "DROP DATABASE delegationdatabase" | ||||||||
Line: 842 to 832 | ||||||||
| ||||||||
Added: | ||||||||
> > | Batch system integrationGrid EngineRequirementsYou have to install and configure the GE batch system software before installing and configuring the CREAM software. The CREAM CE integration was tested with GE 6.2u5 but it should work with any forked version of the original GE software. The support of the GE batch system software (or any of its forked versions) is out of the scope of this activity. Before proceeding, please take note of the following remarks:
Integration pluginsThe GE integration with CREAM CE consists in deploying specific BLAH plugins and configure them to properly interoperate with Grid Engine batch system. The following GE BLAH plugins are deployed with CREAM CE installation: BUpdaterSGE, sge_hold.sh, sge_submit.sh, sge_resume.sh, sge_status.sh and sge_cancel.InstalationIf you are running GE, install the emi-ge-utils metapackage:yum install emi-ge-utils
ConfigurationSet yoursiteinfo.def file, which is the input file used by yaim. Documentation about yaim variables relevant for CREAM CE and GE is available at
SGE_SHARED_INSTALL=yes in your site-info.def , otherwise YAIM may change your setup according to the definitions in your site-info.def .
The CREAM CE GE integration in then configured running YAIM:
Important notesFile transfersBesides the input/output sandbox files (transfered via GFTP) there are some other files that need to be transferred from/to the CREAM sandbox directory on the CE node to/from the Worker Node, namely:
# diff -Nua sge_filestaging.modified sge_filestaging.orig --- sge_filestaging.modified 2010-03-25 19:38:11.000000000 +0000 +++ sge_filestaging.orig 2010-03-25 19:05:43.000000000 +0000 | |||||||
Line: 21 to 21 | ||||||||
Added: | ||||||||
> > | my $remotefile = $3;
if ( $STAGEIN ) {
- system( 'cp', $remotefile, $localfile );
+ system( 'scp', "$remotemachine:$remotefile", $localfile );
} else {
- system( 'cp', $localfile, $remotefile" );
+ system( 'scp', $localfile, "$remotemachine:$remotefile" );
}
}
GE accounting fileBUpdaterSGE needs to consult the GE accounting file to determine how did a given job ended. Therefore, the GE accounting file must be shared between the GE SERVER / QMASTER and the CREAM CE. Moreover, to guarantee that the accounting file is updated on the fly, the GE configuration should be tunned (using qconf -mconf) in order to add under the reporting_params the following definitions:accounting=true accounting_flush_time=00:00:00
GE SERVER (QMASTER) tunningThe following suggestions should be implemented to achieve better performance when integrating with CREAM CE:
| |||||||
1 PostconfigurationHave a look at the Known issue page. |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 670 to 671 | ||||||||
| ||||||||
Added: | ||||||||
> > |
0.0.0.1 Configuration of the new BLAH Blparser to to use cached batch system commandsWith BLAH version >= 1.16-3, the new BLAH blparser can be configured in order to not interact directly with the batch system, but through a program (to be implemented by the site admin) which can implement some caching functionality. This is the case for example ofCommandProxyTools , implement at Cern
To enable this feature, add in /etc/blah.config (the example below is for lsf, with /usr/bin/runcmd.pl as name of the "caching" program):
lsf_batch_caching_enabled=yes batch_command_caching_filter=/usr/bin/runcmd.plSo the blparser, insead of issuing bjobs -u .... , will issue /usr/bin/runcmd.pl bjobs -u .. ."
| |||||||
0.0.1 Configuration of the CREAM databases on a host different than the CREAM-CETo configure the CREAM databases on a host different than the CREAM-CE: |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 897 to 897 | ||||||||
service mysqld stop | ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
[mysqld] innodb_log_file_size=64M |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 874 to 874 | ||||||||
0.1 MySQL database: How to resize Innodb log files | ||||||||
Changed: | ||||||||
< < | If the following error occurs | |||||||
> > | If the following error occurs (see the mysql log file: /var/log/mysqld.log) | |||||||
InnoDB: ERROR: the age of the last checkpoint is , |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 680 to 680 | ||||||||
creamdb and delegationdb . This means that some manual hacks are needed if the same mysql server host should be used to host the databases of multiple CREAM CEs. | ||||||||
Changed: | ||||||||
< < | The files where the database names are specified are: | |||||||
> > | The files where the database names are specified and should be changed before configuring via yaim are: | |||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
url="jdbc:mysql://localhost:3306/creamdb" | ||||||||
Line: 694 to 694 | ||||||||
url="jdbc:mysql://localhost:3306/delegationdb?autoReconnect=true" | ||||||||
Changed: | ||||||||
< < | It must NOT be changed in the following line: | |||||||
> > | You must NOT change in the following line: | |||||||
<Resource name="jdbc/creamdb" | ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
if [ $3 == "creamdb" -a -d /opt/glite/var/cream_sandbox ] ; then | ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
/************ Drop: Database ***************/ | ||||||||
Line: 719 to 720 | ||||||||
USE creamdb; | ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
/************ Drop: Database ***************/ | ||||||||
Line: 733 to 734 | ||||||||
use delegationdb; | ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
my $querycmd= "mysql -B --skip-column-names -u" . $userdb . " --password=\"" . $passworddb . "\" -e \"use creamdb;select count(*) from job_status AS status LEFT OUTER JOIN job_status AS latest ON latest.jobId=status.jobId AND status.id < latest.id WHERE latest.id IS null AND status.type IN ('0','1','2','3','4','6');\""; | ||||||||
Added: | ||||||||
> > |
<cream db name>_database_version="2.4" <delegation db name>_database_version="2.4"just after: creamdb_database_version="2.4" delegationdb_database_version="2.4"
create_mysql_db creamdatabase ${CREAM_DB_USER} ${CREAM_DB_PASSWORD} "${GLITE_CREAM_LOCATION_ETC}/glite-ce-cream/populate_creamdb_mysql.sql" ${CREAM_DB_HOST} \ && create_mysql_db delegationdatabase ${CREAM_DB_USER} ${CREAM_DB_PASSWORD} "${GLITE_CREAM_LOCATION_ETC}/glite-ce-cream/populate_delegationdb.sql" ${CREAM_DB_HOST} \
-e "s/url=\"jdbc:mysql:\/\/localhost:3306\/delegationdb?autoReconnect=true/url=\"jdbc:mysql:\/\/${CREAM_DB_HOST}:3306\/delegationdb\?autoReconnect=true/" \ -e "s/url=\"jdbc:mysql:\/\/localhost:3306\/creamdb\"/url=\"jdbc:mysql:\/\/${CREAM_DB_HOST}:3306\/creamdb\"/" \
mysqlshow --password="$MYSQL_PASSWORD" | grep "creamdatabase" > /dev/null 2>&1 mysql -u root --password="$MYSQL_PASSWORD" -e "DROP DATABASE creamdatabase" mysqlshow --password="$MYSQL_PASSWORD" | grep "delegationdatabase" > /dev/null 2>&1 mysql -u root --password="$MYSQL_PASSWORD" -e "DROP DATABASE delegationdatabase" | |||||||
0.0.1 Configuration of the CREAM CLIThe CREAM CLI is part of the EMI-UI. To configure it please refer to https://twiki.cern.ch/twiki/bin/view/EMI/EMIui#Client_Installation_Configuratio. |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 293 to 293 | ||||||||
| ||||||||
Changed: | ||||||||
< < | ||||||||
> > | ||||||||
yum install emi-ge-utils | ||||||||
Deleted: | ||||||||
< < | ||||||||
0.0.1 Installation of a CREAM CE node in cluster mode | ||||||||
Line: 357 to 356 | ||||||||
| ||||||||
Changed: | ||||||||
< < | ||||||||
> > | ||||||||
yum install emi-ge-utils | ||||||||
Deleted: | ||||||||
< < | ||||||||
0.0.0.1 Installation of the cluster metapackage | ||||||||
Line: 475 to 472 | ||||||||
| ||||||||
Changed: | ||||||||
< < | /opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n SGE_utils | |||||||
> > | /opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n SGE_utils | |||||||
0.0.1 Configuration of a CREAM CE node in cluster mode | ||||||||
Line: 520 to 515 | ||||||||
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n LSF_utils | ||||||||
Added: | ||||||||
> > | ||||||||
| ||||||||
Changed: | ||||||||
< < | /opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n SGE_utils | |||||||
> > | /opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n SGE_utils | |||||||
| ||||||||
Line: 544 to 538 | ||||||||
| ||||||||
Changed: | ||||||||
< < | ||||||||
> > | ||||||||
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n SGE_utils -n glite-CLUSTER | ||||||||
Deleted: | ||||||||
< < | ||||||||
|
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 689 to 689 | ||||||||
The files where the database names are specified are: | ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
url="jdbc:mysql://localhost:3306/creamdb" | ||||||||
Line: 707 to 707 | ||||||||
<Resource name="jdbc/creamdb" | ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
if [ $3 == "creamdb" -a -d /opt/glite/var/cream_sandbox ] ; then | ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
Added: | ||||||||
> > | /************ Drop: Database ***************/ DROP DATABASE IF EXISTS creamdb; /************ Create: Database ***************/ | |||||||
CREATE DATABASE creamdb; | ||||||||
Added: | ||||||||
> > | /************ Use: Database ***************/ USE creamdb; | |||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
Added: | ||||||||
> > | /************ Drop: Database ***************/ | |||||||
DROP DATABASE IF EXISTS delegationdb; | ||||||||
Added: | ||||||||
> > | /************ Create: Database ***************/ CREATE DATABASE delegationdb; /************ Use Delegation DB ***************/ use delegationdb; | |||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
my $querycmd= "mysql -B --skip-column-names -u" . $userdb . " --password=\"" . $passworddb . "\" -e \"use creamdb;select count(*) from job_status AS status LEFT OUTER JOIN job_status AS latest ON latest.jobId=status.jobId AND status.id < latest.id WHERE latest.id IS null AND status.type IN ('0','1','2','3','4','6');\""; |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 293 to 293 | ||||||||
| ||||||||
Changed: | ||||||||
< < | ||||||||
> > | ||||||||
yum install emi-ge-utils | ||||||||
Line: 476 to 475 | ||||||||
| ||||||||
Changed: | ||||||||
< < | <-- pre.cjk { font-family: "DejaVu Sans",monospace; }p { margin-bottom: 0.21cm; } --> /opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n SGE_utils | |||||||
> > | /opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n SGE_utils | |||||||
Line: 521 to 519 | ||||||||
| ||||||||
Changed: | ||||||||
< < | /opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n LSF_utils | |||||||
> > | /opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n LSF_utils
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n SGE_utils | |||||||
Changed: | ||||||||
< < | ||||||||
> > | ||||||||
| ||||||||
Line: 542 to 543 | ||||||||
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n LSF_utils -n glite-CLUSTER | ||||||||
Added: | ||||||||
> > |
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n SGE_utils -n glite-CLUSTER | |||||||
| ||||||||
Line: 710 to 713 | ||||||||
if [ $3 == "creamdb" -a -d /opt/glite/var/cream_sandbox ] ; then | ||||||||
Deleted: | ||||||||
< < | ||||||||
| ||||||||
Line: 733 to 731 | ||||||||
my $querycmd= "mysql -B --skip-column-names -u" . $userdb . " --password=\"" . $passworddb . "\" -e \"use creamdb;select count(*) from job_status AS status LEFT OUTER JOIN job_status AS latest ON latest.jobId=status.jobId AND status.id < latest.id WHERE latest.id IS null AND status.type IN ('0','1','2','3','4','6');\""; | ||||||||
Deleted: | ||||||||
< < | ||||||||
0.0.1 Configuration of the CREAM CLIThe CREAM CLI is part of the EMI-UI. To configure it please refer to https://twiki.cern.ch/twiki/bin/view/EMI/EMIui#Client_Installation_Configuratio. |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 680 to 680 | ||||||||
| ||||||||
Changed: | ||||||||
< < | Please note that, because of bug #88078, the name of the databases are hardwired in some files to creamdb and delegationdb . This means there are problems if the same mysql server host should be used to host the databases of multiple CREAM CEs. | |||||||
> > | Please note that, because of bug #88078, the name of the databases are hardwired in some files to creamdb and delegationdb . This means that some manual hacks are needed if the same mysql server host should be used to host the databases of multiple CREAM CEs.
The files where the database names are specified are:
url="jdbc:mysql://localhost:3306/creamdb"and: url="jdbc:mysql://localhost:3306/delegationdb?autoReconnect=true"It must NOT be changed in the following line: <Resource name="jdbc/creamdb"
if [ $3 == "creamdb" -a -d /opt/glite/var/cream_sandbox ] ; then
CREATE DATABASE creamdb;
DROP DATABASE IF EXISTS delegationdb;
my $querycmd= "mysql -B --skip-column-names -u" . $userdb . " --password=\"" . $passworddb . "\" -e \"use creamdb;select count(*) from job_status AS status LEFT OUTER JOIN job_status AS latest ON latest.jobId=status.jobId AND status.id < latest.id WHERE latest.id IS null AND status.type IN ('0','1','2','3','4','6');\""; | |||||||
0.0.1 Configuration of the CREAM CLI |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release
| ||||||||
Changed: | ||||||||
< < | 1 Installation and Configurationf | |||||||
> > | 1 Installation and Configuration | |||||||
0.1 Prerequisites | ||||||||
Line: 26 to 26 | ||||||||
If you plan to use LSF as batch system for your CREAM CE, you have to install and configure it before installing and configuring the CREAM software. Since LSF is a commercial software it can't be distributed together with the middleware. | ||||||||
Added: | ||||||||
> > | If you plan to use SGE as batch system for your CREAM CE, it will be installed and configured along with the middleware (i.e. you don't have to install and configure it in advance) | |||||||
0.1 Plan how to deploy the CREAM CE0.1.1 CREAM CE and gLite-cluster | ||||||||
Line: 77 to 77 | ||||||||
| ||||||||
Changed: | ||||||||
< < | It is then also possible to define a DNS alias to refer to the set of CREAM head nodes: after the initial contact from outside clients to the CREAM-CE alias name for job submission, all further actions on that job are based on the jobid which contains the physical hostname of the CREAM-CE to which the job was submitted. This allows to switch the DNS alias in order to distribute load. | |||||||
> > | It is then also possible to define a DNS alias to refer to the set of CREAM head nodes: after the initial contact from outside clients to the CREAM-CE alias name for job submission, all further actions on that job are based on the jobid which contains the physical hostname of the CREAM-CE to which the job was submitted. This allows to switch the DNS alias in order to distribute load. | |||||||
The alias shouldn't be published in the information service, but should be simply communicated to the relevant users. | ||||||||
Changed: | ||||||||
< < | There are various techniques to change an alias entry in the DNS. The choice depends strongly on the way the network is set up and managed.
For example at DESY a self-written service called POISE is used; using metrics (which take into account in particular the load and the sandbox size ) it decides the physical instance
the alias should point to. Another possibility to define aliases is to use commercial network techniques such as F5. | |||||||
> > | There are various techniques to change an alias entry in the DNS. The choice depends strongly on the way the network is set up and managed. For example at DESY a self-written service called POISE is used; using metrics (which take into account in particular the load and the sandbox size ) it decides the physical instance the alias should point to. Another possibility to define aliases is to use commercial network techniques such as F5. | |||||||
It must be noted that, as observed by Desy sysadmins, the proliferation of the aliases (C-records) is not well defined among DNS'. Therefore changes of an alias sometimes can take hours to be propagated to other sites. | ||||||||
Line: 300 to 292 | ||||||||
yum install emi-lsf-utils | ||||||||
Added: | ||||||||
> > |
yum install emi-ge-utils | |||||||
0.0.1 Installation of a CREAM CE node in cluster modeFirst of all, install theyum-protectbase rpm: | ||||||||
Line: 359 to 357 | ||||||||
yum install emi-lsf-utils | ||||||||
Added: | ||||||||
> > |
yum install emi-ge-utils | |||||||
0.0.0.1 Installation of the cluster metapackageIf the CREAM CE node has to host also theglite-cluster , install also this metapackage: | ||||||||
Line: 471 to 475 | ||||||||
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n LSF_utils | ||||||||
Added: | ||||||||
> > |
<-- pre.cjk { font-family: "DejaVu Sans",monospace; }p { margin-bottom: 0.21cm; } --> /opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n SGE_utils | |||||||
0.0.1 Configuration of a CREAM CE node in cluster mode0.0.1.1 Install host certificate |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 70 to 70 | ||||||||
in the current implementation when configuring via yaim, all queues of a given CE head node gets mapped to a single cluster (i.e. it not possible to map different queues to different clusters). Waiting for the fixes for these bugs, the ldif files produced by yaim must be then manually edited. | ||||||||
Added: | ||||||||
> > |
0.0.1 Define a DNS alias to refer to set of CREAM CEsIn order to distribute load for job submissions, it is possible to deploy multiple CREAM CEs head nodes referring to the same set of resources. As explained in the previous section, this should be implemented with:
POISE is used; using metrics (which take into account in particular the load and the sandbox size ) it decides the physical instance
the alias should point to. Another possibility to define aliases is to use commercial network techniques such as F5.
It must be noted that, as observed by Desy sysadmins, the proliferation of the aliases (C-records) is not well defined among DNS'. Therefore changes of an alias sometimes can take hours to be propagated to other sites.
The use of alias for job submission is a good solution to improve load balancing and availability of the service (the unavailability of a physical CREAM CE would be hidden by the use of the alias). It must however be noted that:
| |||||||
0.0.1 Choose the authorization modelThe CREAM CE can be configured to use as authorization system: | ||||||||
Line: 142 to 169 | ||||||||
| ||||||||
Changed: | ||||||||
< < | If the BLPARSER_HOST is different than the CREAM CE host, after having installed and configured the CREAM CE it is necessary: | |||||||
> > | lasIf the BLPARSER_HOST is different than the CREAM CE host, after having installed and configured the CREAM CE it is necessary: | |||||||
|
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 156 to 156 | ||||||||
in the siteinfo.def before configuring via yaim. | ||||||||
Added: | ||||||||
> > | 0.0.1 Deployment models for CREAM databasesThe databases used by CREAM can be deployed in the CREAM CE host (which is the default scenario) or on a different machine. Click here for information how to deploy the databases on a machine different wrt the CREAM-CE. | |||||||
0.1 InstallationThis section explains how to install: | ||||||||
Line: 628 to 635 | ||||||||
| ||||||||
Added: | ||||||||
> > |
0.0.1 Configuration of the CREAM databases on a host different than the CREAM-CETo configure the CREAM databases on a host different than the CREAM-CE:
creamdb and delegationdb . This means there are problems if the same mysql server host should be used to host the databases of multiple CREAM CEs. | |||||||
0.0.1 Configuration of the CREAM CLIThe CREAM CLI is part of the EMI-UI. To configure it please refer to https://twiki.cern.ch/twiki/bin/view/EMI/EMIui#Client_Installation_Configuratio. | ||||||||
Line: 709 to 730 | ||||||||
SHOW VARIABLES like 'innodb_buffer_pool_size'; | ||||||||
Changed: | ||||||||
< < | 0.1 MySQL database: How to resize Innodb log files? | |||||||
> > | 0.1 MySQL database: How to resize Innodb log files | |||||||
If the following error occurs | ||||||||
Changed: | ||||||||
< < | ||||||||
> > | ||||||||
InnoDB: ERROR: the age of the last checkpoint is , | ||||||||
Deleted: | ||||||||
< < | ||||||||
InnoDB: which exceeds the log group capacity . | ||||||||
Deleted: | ||||||||
< < | ||||||||
InnoDB: If you are using big BLOB or TEXT rows,you must set the | ||||||||
Deleted: | ||||||||
< < | ||||||||
InnoDB: combined size of log files at least 10 times bigger than the | ||||||||
Deleted: | ||||||||
< < | ||||||||
InnoDB: largest such row. | ||||||||
Changed: | ||||||||
< < | ||||||||
> > | ||||||||
Changed: | ||||||||
< < | then you must resize innodb log files. | |||||||
> > | then you must resize the innodb log files. | |||||||
Follow these steps:
|
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 32 to 32 | ||||||||
glite-CLUSTER is a node type that can publish information about clusters and subclusters in a site, referenced by any number of compute elements. In particular it allows to deal with sites having multiple CREAM CE nodes and/or multiple subclusters (i.e. disjoint sets of worker nodes, each set having sufficiently homogeneous properties). | ||||||||
Changed: | ||||||||
< < | In Glue1, batch system queues are represented through GlueCE objectclasses. Each GlueCE refers to a Cluster, which can be composed by one or more SubClusters. However the gLite WMS requires the publication of exactly one SubCluster per Cluster (and hence per batch queue). | |||||||
> > | In Glue1, batch system queues are represented through GlueCE objectclasses. Each GlueCE refers to a Cluster, which can be composed by one or more SubClusters. However the gLite WMS requires the publication of exactly one SubCluster per Cluster (and hence per batch queue). | |||||||
Thus sites with heterogeneous hardware have two possible choices:
| ||||||||
Line: 38 to 37 | ||||||||
Thus sites with heterogeneous hardware have two possible choices:
| ||||||||
Deleted: | ||||||||
< < | ||||||||
However, without the use of the gLite-cluster, YAIM allows configuring a single Cluster per CREAM head node. | ||||||||
Changed: | ||||||||
< < | A second problem, addressed by gLite-cluster arises for larger sites which install multiple CE headnodes submitting to the same batch queues for redundancy or load-balancing. Without the use of gLite-cluster, YAIM generates a separate Cluster/SubCluster pair for each head node even though they all describe the same hardware. This causes no problems for job submission, but by default would overcount the installed capacity at the site by a multiple of the number of SubClusters. The workaround, before introducing the gLite-cluster, was to publish zero values for the installed capacity from all but one of the nodes (but this is clearly far from being an ideal solution). | |||||||
> > | A second problem, addressed by gLite-cluster arises for larger sites which install multiple CE headnodes submitting to the same batch queues for redundancy or load-balancing. Without the use of gLite-cluster, YAIM generates a separate Cluster/SubCluster pair for each head node even though they all describe the same hardware. This causes no problems for job submission, but by default would overcount the installed capacity at the site by a multiple of the number of SubClusters. The workaround, before introducing the gLite-cluster, was to publish zero values for the installed capacity from all but one of the nodes (but this is clearly far from being an ideal solution). | |||||||
The glite-CLUSTER node addresses this issue. It contains a subset of the functionality incorporated in the CREAM node types: the publication of the Glue1 GlueCluster and its dependent objects, the publication of the Glue1 GlueService object for the GridFTP endpoint, and the directories which store the RunTimeEnvironment tags, together with the YAIM functions which configure them. | ||||||||
Line: 80 to 68 | ||||||||
http://savannah.cern.ch/bugs/?87691 | ||||||||
Changed: | ||||||||
< < | in the current implementation when configuring via yaim, all queues of a given CE head node gets mapped to a single cluster (i.e. it not possible to map different queues to different clusters). Waiting for the fixes for these bugs, the ldif files produced by yaim must be then manually edited. | |||||||
> > | in the current implementation when configuring via yaim, all queues of a given CE head node gets mapped to a single cluster (i.e. it not possible to map different queues to different clusters). Waiting for the fixes for these bugs, the ldif files produced by yaim must be then manually edited. | |||||||
0.0.1 Choose the authorization model | ||||||||
Line: 241 to 227 | ||||||||
0.0.0.1 Installation of the CREAM CE software | ||||||||
Changed: | ||||||||
< < | The CREAM software itself can work with both Sun jdk and openjdk. However, the apel-core package, deployed in the CREAM CE node, requires mm-mysql , which explicitly requires Sun jdk. So to install the middleware software needed for the CREAM CE, install first of all Sun JDK ( jdk ). This is actually not needed in a standard SL5 box, since in this case the Sun JDK rpm is available in the OS repo.
Please note that this dependency on mm-mysql is being fixed by APEL developers. | |||||||
> > | The CREAM software itself can work with both Sun jdk and openjdk. However, the apel-core package, deployed in the CREAM CE node, requires mm-mysql , which explicitly requires Sun jdk. So to install the middleware software needed for the CREAM CE, install first of all Sun JDK ( jdk ). This is actually not needed in a standard SL5 box, since in this case the Sun JDK rpm is available in the OS repo. Please note that this dependency on mm-mysql is being fixed by APEL developers. | |||||||
Then install xml-commons-apis : | ||||||||
Line: 301 to 286 | ||||||||
0.0.0.1 Installation of the CREAM CE software | ||||||||
Changed: | ||||||||
< < | The CREAM software itself can work with both Sun jdk and openjdk. However, the apel-core package, deployed in the CREAM CE node, requires mm-mysql , which explicitly requires Sun jdk. So to install the middleware software needed for the CREAM CE, install first of all Sun JDK ( jdk ). This is actually not needed in a standard SL5 box, since in this case the Sun JDK rpm is available in the OS repo.
Please note that this dependency on mm-mysql is being fixed by APEL developers. | |||||||
> > | The CREAM software itself can work with both Sun jdk and openjdk. However, the apel-core package, deployed in the CREAM CE node, requires mm-mysql , which explicitly requires Sun jdk. So to install the middleware software needed for the CREAM CE, install first of all Sun JDK ( jdk ). This is actually not needed in a standard SL5 box, since in this case the Sun JDK rpm is available in the OS repo. Please note that this dependency on mm-mysql is being fixed by APEL developers. | |||||||
Then install xml-commons-apis : | ||||||||
Line: 669 to 651 | ||||||||
If the new BLAH Blparser is used (click here for instructions to check if you are using the old or the new blparser) the parameter bupdater_loop_interval attribute in /etc/blah.config defines how often the batch system is queried to check the status of the jobs. If a low value is used, job status changes are detected promptly, but this also means that several batch system queries are done, and this can cause a high load. | ||||||||
Changed: | ||||||||
< < | With yaim-cream-ce v >= 4.2.2-1, this parameter is configurable via yaim: the relevant yaim variable is BUPDATER_LOOP_INTERVAL which has 30 (secs) as default value. | |||||||
> > | With yaim-cream-ce v >= 4.2.2-1, this parameter is configurable via yaim: the relevant yaim variable is BUPDATER_LOOP_INTERVAL which has 30 (secs) as default value. | |||||||
Changed: | ||||||||
< < | With yaim-cream-ce v. < 4.2.2-1, this parameter is not configurable via yaim, and therefore it is needed to manually edit the blah configuration file, otherwise the default value (5 s.) is used. After having set this value in the blah configuration file it is necessary: | |||||||
> > | With yaim-cream-ce v. < 4.2.2-1, this parameter is not configurable via yaim, and therefore it is needed to manually edit the blah configuration file, otherwise the default value (5 s.) is used. After having set this value in the blah configuration file it is necessary: | |||||||
| ||||||||
Line: 732 to 709 | ||||||||
SHOW VARIABLES like 'innodb_buffer_pool_size'; | ||||||||
Added: | ||||||||
> > | 0.1 MySQL database: How to resize Innodb log files?If the following error occursInnoDB: ERROR: the age of the last checkpoint is , InnoDB: which exceeds the log group capacity . InnoDB: If you are using big BLOB or TEXT rows,you must set the InnoDB: combined size of log files at least 10 times bigger than the InnoDB: largest such row. then you must resize innodb log files. Follow these steps:
SHOW VARIABLES like "innodb_log_file_size";
service mysqld stop
[mysqld] innodb_log_file_size=64M
mv /var/lib/mysql/ib_logfile* /tmp
service mysqld start
ls -lrth /var/lib/mysql/ib_logfile* | |||||||
0.1 How to start the CREAM serviceA site admin can start the CREAM service just starting the CREAM container: |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release
| ||||||||
Changed: | ||||||||
< < | 1 Installation and Configuration | |||||||
> > | 1 Installation and Configurationf | |||||||
0.1 Prerequisites | ||||||||
Line: 665 to 665 | ||||||||
To allow a parallelism when interacting with the batch system, in particular to have a good throughput when submitting jobs to the batch system, CREAM can run multiple BLAH instances. A new instance is created whenever needed, till the maximum number defined in the etc/glite-ce-cream/cream-config.xml configuration file is reached. The relevant attribute is cream_concurrency_level . The default value is 50. | ||||||||
Added: | ||||||||
> > | 0.0.0.1 Tune the BLAH BUpdater polling frequencyIf the new BLAH Blparser is used (click here for instructions to check if you are using the old or the new blparser) the parameterbupdater_loop_interval attribute in /etc/blah.config defines how often the batch system is queried to check the status of the jobs. If a low value is used, job status changes are detected promptly, but this also means that several batch system queries are done, and this can cause a high load.
With yaim-cream-ce v >= 4.2.2-1, this parameter is configurable via yaim: the relevant yaim variable is BUPDATER_LOOP_INTERVAL which has 30 (secs) as default value.
With yaim-cream-ce v. < 4.2.2-1, this parameter is not configurable via yaim, and therefore it is needed to manually edit the blah configuration file, otherwise the default value (5 s.) is used.
After having set this value in the blah configuration file it is necessary:
bhist -u all -d -l -n 1 ).
| |||||||
1 Postconfiguration |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release
| ||||||||
Changed: | ||||||||
< < | 1 Operations/Installation and Configuration | |||||||
> > | 1 Installation and Configuration | |||||||
0.1 Prerequisites | ||||||||
Line: 221 to 221 | ||||||||
Running the script available at http://forge.cnaf.infn.it/frs/download.php/101/disable_yum.sh (implemented by Giuseppe Platania (INFN Catania) yum autoupdate will be disabled | ||||||||
Changed: | ||||||||
< < | 0.0.1 Operations/Installation of a CREAM CE node in no cluster mode | |||||||
> > | 0.0.1 Installation of a CREAM CE node in no cluster mode | |||||||
First of all, install the yum-protectbase rpm: | ||||||||
Line: 231 to 231 | ||||||||
Then proceed with the installation of the CA certificates. | ||||||||
Changed: | ||||||||
< < | 0.0.0.1 Operations/Installation of the CA certificates | |||||||
> > | 0.0.0.1 Installation of the CA certificates | |||||||
The CA certificate can be installed issuing: | ||||||||
Line: 239 to 239 | ||||||||
yum install ca-policy-egi-core | ||||||||
Changed: | ||||||||
< < | 0.0.0.1 Operations/Installation of the CREAM CE software | |||||||
> > | 0.0.0.1 Installation of the CREAM CE software | |||||||
The CREAM software itself can work with both Sun jdk and openjdk. However, the apel-core package, deployed in the CREAM CE node, requires mm-mysql , which explicitly requires Sun jdk. So to install the middleware software needed for the CREAM CE, install first of all Sun JDK ( jdk ). This is actually not needed in a standard SL5 box, since in this case the Sun JDK rpm is available in the OS repo.
Please note that this dependency on mm-mysql is being fixed by APEL developers. | ||||||||
Line: 258 to 258 | ||||||||
yum install emi-cream-ce | ||||||||
Changed: | ||||||||
< < | 0.0.0.1 Operations/Installation of the batch system specific software | |||||||
> > | 0.0.0.1 Installation of the batch system specific software | |||||||
After the installation of the CREAM CE metapackage it is necessary to install the batch system specific metapackage(s): | ||||||||
Line: 281 to 281 | ||||||||
yum install emi-lsf-utils | ||||||||
Changed: | ||||||||
< < | 0.0.1 Operations/Installation of a CREAM CE node in cluster mode | |||||||
> > | 0.0.1 Installation of a CREAM CE node in cluster mode | |||||||
First of all, install the yum-protectbase rpm: | ||||||||
Line: 291 to 291 | ||||||||
Then proceed with the installation of the CA certificates. | ||||||||
Changed: | ||||||||
< < | 0.0.0.1 Operations/Installation of the CA certificates | |||||||
> > | 0.0.0.1 Installation of the CA certificates | |||||||
The CA certificate can be installed issuing: | ||||||||
Line: 299 to 299 | ||||||||
yum install ca-policy-egi-core | ||||||||
Changed: | ||||||||
< < | 0.0.0.1 Operations/Installation of the CREAM CE software | |||||||
> > | 0.0.0.1 Installation of the CREAM CE software | |||||||
The CREAM software itself can work with both Sun jdk and openjdk. However, the apel-core package, deployed in the CREAM CE node, requires mm-mysql , which explicitly requires Sun jdk. So to install the middleware software needed for the CREAM CE, install first of all Sun JDK ( jdk ). This is actually not needed in a standard SL5 box, since in this case the Sun JDK rpm is available in the OS repo.
Please note that this dependency on mm-mysql is being fixed by APEL developers. | ||||||||
Line: 318 to 318 | ||||||||
yum install emi-cream-ce | ||||||||
Changed: | ||||||||
< < | 0.0.0.1 Operations/Installation of the batch system specific software | |||||||
> > | 0.0.0.1 Installation of the batch system specific software | |||||||
After the installation of the CREAM CE metapackage it is necessary to install the batch system specific metapackage(s). | ||||||||
Line: 341 to 341 | ||||||||
yum install emi-lsf-utils | ||||||||
Changed: | ||||||||
< < | 0.0.0.1 Operations/Installation of the cluster metapackage | |||||||
> > | 0.0.0.1 Installation of the cluster metapackage | |||||||
If the CREAM CE node has to host also the glite-cluster , install also this metapackage: | ||||||||
Line: 349 to 349 | ||||||||
yum install emi-cluster | ||||||||
Changed: | ||||||||
< < | 0.0.1 Operations/Installation of a glite-cluster node | |||||||
> > | 0.0.1 Installation of a glite-cluster node | |||||||
First of all, install the yum-protectbase rpm: | ||||||||
Line: 359 to 359 | ||||||||
Then proceed with the installation of the CA certificates. | ||||||||
Changed: | ||||||||
< < | 0.0.0.1 Operations/Installation of the CA certificates | |||||||
> > | 0.0.0.1 Installation of the CA certificates | |||||||
The CA certificate can be installed issuing: | ||||||||
Line: 367 to 367 | ||||||||
yum install ca-policy-egi-core | ||||||||
Changed: | ||||||||
< < | 0.0.0.1 Operations/Installation of the cluster metapackage | |||||||
> > | 0.0.0.1 Installation of the cluster metapackage | |||||||
Install the glite-CLUSTER metapackage: | ||||||||
Line: 375 to 375 | ||||||||
yum install emi-cluster | ||||||||
Changed: | ||||||||
< < | 0.0.1 Operations/Installation of the BLAH BLparser | |||||||
> > | 0.0.1 Installation of the BLAH BLparser | |||||||
If the new BLAH Blparser must be used, there isn't anything to be installed for the BLAH Blparser (i.e. the installation of the CREAM-CE is enough). | ||||||||
Line: 388 to 388 | ||||||||
yum install glite-yaim-cream-ce | ||||||||
Changed: | ||||||||
< < | 0.0.1 Operations/Installation of the CREAM CLI | |||||||
> > | 0.0.1 Installation of the CREAM CLI | |||||||
The CREAM CLI is part of the EMI-UI. To install it please refer to https://twiki.cern.ch/twiki/bin/view/EMI/EMIui#Client_Installation_Configuratio . |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 92 to 92 | ||||||||
OR
| ||||||||
Changed: | ||||||||
< < | In the former case a ARGUS box (usually at site level) where to define policies for the CREAM CE box is needed. | |||||||
> > | In the former case a ARGUS box (recommended to be at site level: it can of course serve multiple CEs of that site) where to define policies for the CREAM CE box is needed. | |||||||
To use ARGUS as authorization system, yaim variable USE_ARGUS must be set in the following way: |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 165 to 165 | ||||||||
On the CREAM CE, to use the old BLAH blparser, it is necessary to set: | ||||||||
Changed: | ||||||||
< < | BLPARSER_WITH_UPDATER_NOTIFIER=true | |||||||
> > | BLPARSER_WITH_UPDATER_NOTIFIER=false | |||||||
in the siteinfo.def before configuring via yaim. |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 203 to 203 | ||||||||
You can install the EMI-1 yum repository, issuing: | ||||||||
Changed: | ||||||||
< < | wget http://emisoft.web.cern.ch/emisoft/dist/EMI/1/sl5/x86_64/base/emi-release-1.0.0-1.sl5.noarch.rpm yum install ./emi-release-1.0.0-1.sl5.noarch.rpm | |||||||
> > | wget http://emisoft.web.cern.ch/emisoft/dist/EMI/1/sl5/x86_64/updates/emi-release-1.0.1-1.sl5.noarch.rpm yum install ./emi-release-1.0.1-1.sl5.noarch.rpm | |||||||
0.0.0.1 The Certification Authority repository |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release
| ||||||||
Changed: | ||||||||
< < | 1 Installation and Configuration | |||||||
> > | 1 Operations/Installation and Configuration | |||||||
0.1 Prerequisites | ||||||||
Line: 221 to 221 | ||||||||
Running the script available at http://forge.cnaf.infn.it/frs/download.php/101/disable_yum.sh (implemented by Giuseppe Platania (INFN Catania) yum autoupdate will be disabled | ||||||||
Changed: | ||||||||
< < | 0.0.1 Installation of a CREAM CE node in no cluster mode | |||||||
> > | 0.0.1 Operations/Installation of a CREAM CE node in no cluster mode | |||||||
First of all, install the yum-protectbase rpm: | ||||||||
Line: 231 to 231 | ||||||||
Then proceed with the installation of the CA certificates. | ||||||||
Changed: | ||||||||
< < | 0.0.0.1 Installation of the CA certificates | |||||||
> > | 0.0.0.1 Operations/Installation of the CA certificates | |||||||
The CA certificate can be installed issuing: | ||||||||
Line: 239 to 239 | ||||||||
yum install ca-policy-egi-core | ||||||||
Changed: | ||||||||
< < | 0.0.0.1 Installation of the CREAM CE software | |||||||
> > | 0.0.0.1 Operations/Installation of the CREAM CE software | |||||||
The CREAM software itself can work with both Sun jdk and openjdk. However, the apel-core package, deployed in the CREAM CE node, requires mm-mysql , which explicitly requires Sun jdk. So to install the middleware software needed for the CREAM CE, install first of all Sun JDK ( jdk ). This is actually not needed in a standard SL5 box, since in this case the Sun JDK rpm is available in the OS repo.
Please note that this dependency on mm-mysql is being fixed by APEL developers. | ||||||||
Line: 258 to 258 | ||||||||
yum install emi-cream-ce | ||||||||
Changed: | ||||||||
< < | 0.0.0.1 Installation of the batch system specific software | |||||||
> > | 0.0.0.1 Operations/Installation of the batch system specific software | |||||||
After the installation of the CREAM CE metapackage it is necessary to install the batch system specific metapackage(s): | ||||||||
Line: 281 to 281 | ||||||||
yum install emi-lsf-utils | ||||||||
Changed: | ||||||||
< < | 0.0.1 Installation of a CREAM CE node in cluster mode | |||||||
> > | 0.0.1 Operations/Installation of a CREAM CE node in cluster mode | |||||||
First of all, install the yum-protectbase rpm: | ||||||||
Line: 291 to 291 | ||||||||
Then proceed with the installation of the CA certificates. | ||||||||
Changed: | ||||||||
< < | 0.0.0.1 Installation of the CA certificates | |||||||
> > | 0.0.0.1 Operations/Installation of the CA certificates | |||||||
The CA certificate can be installed issuing: | ||||||||
Line: 299 to 299 | ||||||||
yum install ca-policy-egi-core | ||||||||
Changed: | ||||||||
< < | 0.0.0.1 Installation of the CREAM CE software | |||||||
> > | 0.0.0.1 Operations/Installation of the CREAM CE software | |||||||
The CREAM software itself can work with both Sun jdk and openjdk. However, the apel-core package, deployed in the CREAM CE node, requires mm-mysql , which explicitly requires Sun jdk. So to install the middleware software needed for the CREAM CE, install first of all Sun JDK ( jdk ). This is actually not needed in a standard SL5 box, since in this case the Sun JDK rpm is available in the OS repo.
Please note that this dependency on mm-mysql is being fixed by APEL developers. | ||||||||
Line: 318 to 318 | ||||||||
yum install emi-cream-ce | ||||||||
Changed: | ||||||||
< < | 0.0.0.1 Installation of the batch system specific software | |||||||
> > | 0.0.0.1 Operations/Installation of the batch system specific software | |||||||
After the installation of the CREAM CE metapackage it is necessary to install the batch system specific metapackage(s). | ||||||||
Line: 341 to 341 | ||||||||
yum install emi-lsf-utils | ||||||||
Changed: | ||||||||
< < | 0.0.0.1 Installation of the cluster metapackage | |||||||
> > | 0.0.0.1 Operations/Installation of the cluster metapackage | |||||||
If the CREAM CE node has to host also the glite-cluster , install also this metapackage: | ||||||||
Line: 349 to 349 | ||||||||
yum install emi-cluster | ||||||||
Changed: | ||||||||
< < | 0.0.1 Installation of a glite-cluster node | |||||||
> > | 0.0.1 Operations/Installation of a glite-cluster node | |||||||
First of all, install the yum-protectbase rpm: | ||||||||
Line: 359 to 359 | ||||||||
Then proceed with the installation of the CA certificates. | ||||||||
Changed: | ||||||||
< < | 0.0.0.1 Installation of the CA certificates | |||||||
> > | 0.0.0.1 Operations/Installation of the CA certificates | |||||||
The CA certificate can be installed issuing: | ||||||||
Line: 367 to 367 | ||||||||
yum install ca-policy-egi-core | ||||||||
Changed: | ||||||||
< < | 0.0.0.1 Installation of the cluster metapackage | |||||||
> > | 0.0.0.1 Operations/Installation of the cluster metapackage | |||||||
Install the glite-CLUSTER metapackage: | ||||||||
Line: 375 to 375 | ||||||||
yum install emi-cluster | ||||||||
Changed: | ||||||||
< < | 0.0.1 Installation of the BLAH BLparser | |||||||
> > | 0.0.1 Operations/Installation of the BLAH BLparser | |||||||
If the new BLAH Blparser must be used, there isn't anything to be installed for the BLAH Blparser (i.e. the installation of the CREAM-CE is enough). | ||||||||
Line: 388 to 388 | ||||||||
yum install glite-yaim-cream-ce | ||||||||
Changed: | ||||||||
< < | 0.0.1 Installation of the CREAM CLI | |||||||
> > | 0.0.1 Operations/Installation of the CREAM CLI | |||||||
The CREAM CLI is part of the EMI-UI. To install it please refer to https://twiki.cern.ch/twiki/bin/view/EMI/EMIui#Client_Installation_Configuratio . |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 74 to 74 | ||||||||
Information concerning glue2 publication is available here. | ||||||||
Added: | ||||||||
> > | Please also note that, because of these bugs: http://savannah.cern.ch/bugs/?87690 http://savannah.cern.ch/bugs/?87691 in the current implementation when configuring via yaim, all queues of a given CE head node gets mapped to a single cluster (i.e. it not possible to map different queues to different clusters). Waiting for the fixes for these bugs, the ldif files produced by yaim must be then manually edited. | |||||||
0.0.1 Choose the authorization modelThe CREAM CE can be configured to use as authorization system: |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 32 to 32 | ||||||||
glite-CLUSTER is a node type that can publish information about clusters and subclusters in a site, referenced by any number of compute elements. In particular it allows to deal with sites having multiple CREAM CE nodes and/or multiple subclusters (i.e. disjoint sets of worker nodes, each set having sufficiently homogeneous properties). | ||||||||
Changed: | ||||||||
< < | In Glue1, batch system queues are represented through GlueCE objectclasses. Each GlueCE refers to a Cluster, which can be composed by one or more SubClusters. However the gLite WMS requires the publication of exactly one SubCluster per Cluster (and hence per batch queue). | |||||||
> > | In Glue1, batch system queues are represented through GlueCE objectclasses. Each GlueCE refers to a Cluster, which can be composed by one or more SubClusters. However the gLite WMS requires the publication of exactly one SubCluster per Cluster (and hence per batch queue). | |||||||
Thus sites with heterogeneous hardware have two possible choices:
| ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
However, without the use of the gLite-cluster, YAIM allows configuring a single Cluster per CREAM head node. A second problem, addressed by gLite-cluster arises for larger sites which install multiple CE headnodes submitting to the same | ||||||||
Changed: | ||||||||
< < | batch queues for redundancy or load-balancing. Without the use of gLite-cluster, YAIM generates a separate Cluster/SubCluster pair for each head node even though they all describe the same hardware. This causes no problems for job submission, but by default would overcount the installed capacity at the site by a multiple of the number of SubClusters. The workaround, before introducing the gLite-cluster, was to publish zero values for the installed capacity from all but one of the nodes (but this is clearly far from being an ideal solution). | |||||||
> > | batch queues for redundancy or load-balancing. Without the use of gLite-cluster, YAIM generates a separate Cluster/SubCluster pair for each head node even though they all describe the same hardware. This causes no problems for job submission, but by default would overcount the installed capacity at the site by a multiple of the number of SubClusters. The workaround, before introducing the gLite-cluster, was to publish zero values for the installed capacity from all but one of the nodes (but this is clearly far from being an ideal solution). | |||||||
Changed: | ||||||||
< < | The glite-CLUSTER node addresses this issue. It contains a subset of the functionality incorporated in the CREAM node types: the publication of the Glue1 GlueCluster and its dependent objects, the publication of the Glue1 GlueService object for the GridFTP endpoint, and the directories which store the RunTimeEnvironment tags, together with the YAIM functions which configure them. | |||||||
> > | The glite-CLUSTER node addresses this issue. It contains a subset of the functionality incorporated in the CREAM node types: the publication of the Glue1 GlueCluster and its dependent objects, the publication of the Glue1 GlueService object for the GridFTP endpoint, and the directories which store the RunTimeEnvironment tags, together with the YAIM functions which configure them. | |||||||
So, gLite-CLUSTER should be considered: | ||||||||
Line: 54 to 54 | ||||||||
When configuring a gLite-cluster, please consider that there should be: | ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
glite-CLUSTER can be deployed in the same host of a CREAM-CE or in a different one. | ||||||||
Changed: | ||||||||
< < | The following deployment models are therefore possible for a CREAM-CE: | |||||||
> > | The following deployment models are possible for a CREAM-CE: | |||||||
| ||||||||
Line: 377 to 380 | ||||||||
0.0.1 Installation of the CREAM CLI | ||||||||
Changed: | ||||||||
< < | The CREAM CLI is part of the EMI-UI. To install it please refer to xxx. | |||||||
> > | The CREAM CLI is part of the EMI-UI. To install it please refer to https://twiki.cern.ch/twiki/bin/view/EMI/EMIui#Client_Installation_Configuratio . | |||||||
0.1 Configuration | ||||||||
Line: 536 to 539 | ||||||||
Set your siteinfo.def file, which is the input file used by yaim. Documentation about yaim variables relevant for glite-CLUSTER is available at https://twiki.cern.ch/twiki/bin/view/LCG/Site-info_configuration_variables#CLUSTER | ||||||||
Deleted: | ||||||||
< < | You should have:
| |||||||
0.0.0.1 Run yaim | ||||||||
Line: 646 to 638 | ||||||||
0.0.1 Configuration of the CREAM CLI | ||||||||
Changed: | ||||||||
< < | The CREAM CLI is part of the EMI-UI. To configure it please refer to xxx. | |||||||
> > | The CREAM CLI is part of the EMI-UI. To configure it please refer to https://twiki.cern.ch/twiki/bin/view/EMI/EMIui#Client_Installation_Configuratio. | |||||||
0.0.1 Manual configuration |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 33 to 33 | ||||||||
glite-CLUSTER is a node type that can publish information about clusters and subclusters in a site, referenced by any number of compute elements. In particular it allows to deal with sites having multiple CREAM CE nodes and/or multiple subclusters (i.e. disjoint sets of worker nodes, each set having sufficiently homogeneous properties). In Glue1, batch system queues are represented through GlueCE objectclasses. Each GlueCE refers to a Cluster, which can be composed by one or more SubClusters. | ||||||||
Changed: | ||||||||
< < | However the gLite WMS requires the the publication of exactly one SubCluster per Cluster (and hence per batch queue). | |||||||
> > | However the gLite WMS requires the publication of exactly one SubCluster per Cluster (and hence per batch queue). | |||||||
Changed: | ||||||||
< < | Thus sites with heterogeneous hardware have two choices:
| |||||||
> > | Thus sites with heterogeneous hardware have two possible choices:
| |||||||
However, without the use of the gLite-cluster, YAIM allows configuring a single Cluster per CREAM head node. A second problem, addressed by gLite-cluster arises for larger sites which install multiple CE headnodes submitting to the same | ||||||||
Changed: | ||||||||
< < | batch queues for redundancy and load-balancing. Without the use of gLite-cluster, YAIM generates a separate Cluster/SubCluster pair for each head node even though they all describe the same hardware. This causes no problems for job submission, but by default would overcount the installed capacity at the site by a multiple of the number of SubClusters. The workaround, before introducing the gLite-cluster, was to publish zero values for the installed capacity from all but one of the nodes (but this was clearly far from being an ideal solution). | |||||||
> > | batch queues for redundancy or load-balancing. Without the use of gLite-cluster, YAIM generates a separate Cluster/SubCluster pair for each head node even though they all describe the same hardware. This causes no problems for job submission, but by default would overcount the installed capacity at the site by a multiple of the number of SubClusters. The workaround, before introducing the gLite-cluster, was to publish zero values for the installed capacity from all but one of the nodes (but this is clearly far from being an ideal solution). | |||||||
The glite-CLUSTER node addresses this issue. It contains a subset of the functionality incorporated in the CREAM node types: the publication of the Glue1 GlueCluster and its dependent objects, the publication of the Glue1 GlueService object for the GridFTP endpoint, and the directories which store the RunTimeEnvironment tags, together with the YAIM functions which configure them. | ||||||||
Changed: | ||||||||
< < | So, gLite-CLUSTER should be consider
| |||||||
> > | So, gLite-CLUSTER should be considered:
| |||||||
| ||||||||
Added: | ||||||||
> > | When configuring a gLite-cluster, please consider that there should be:
| |||||||
glite-CLUSTER can be deployed in the same host of a CREAM-CE or in a different one. | ||||||||
Changed: | ||||||||
< < | The following deployment models are possible: | |||||||
> > | The following deployment models are therefore possible for a CREAM-CE: | |||||||
| ||||||||
Changed: | ||||||||
< < | More information about glite-CLUSTER can be found at https://twiki.cern.ch/twiki/bin/view/LCG/CLUSTER and in this note tanti cluster quanti sono questi set disgiunti 1 subcluster per cluster 1 coda deve essere configurata per usare solo i WNs di un subcluster | |||||||
> > | More information about glite-CLUSTER can be found at https://twiki.cern.ch/twiki/bin/view/LCG/CLUSTER and in this note. | |||||||
Added: | ||||||||
> > | Information concerning glue2 publication is available here. | |||||||
0.0.1 Choose the authorization model | ||||||||
Line: 538 to 536 | ||||||||
Set your siteinfo.def file, which is the input file used by yaim. Documentation about yaim variables relevant for glite-CLUSTER is available at https://twiki.cern.ch/twiki/bin/view/LCG/Site-info_configuration_variables#CLUSTER | ||||||||
Added: | ||||||||
> > | You should have:
| |||||||
0.0.0.1 Run yaimAfter having filled thesiteinfo.def file, run yaim: |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 30 to 30 | ||||||||
0.0.1 CREAM CE and gLite-cluster | ||||||||
Changed: | ||||||||
< < | glite-CLUSTER is a node type that can publish information about clusters and subclusters in a site, referenced by any number of compute elements. | |||||||
> > | glite-CLUSTER is a node type that can publish information about clusters and subclusters in a site, referenced by any number of compute elements. In particular it allows to deal with sites having multiple CREAM CE nodes and/or multiple subclusters (i.e. disjoint sets of worker nodes, each set having sufficiently homogeneous properties). | |||||||
Changed: | ||||||||
< < | glite-CLUSTER can be deployed in the same host of the CREAM-CE or in a different one. | |||||||
> > | In Glue1, batch system queues are represented through GlueCE objectclasses. Each GlueCE refers to a Cluster, which can be composed by one or more SubClusters.
However the gLite WMS requires the the publication of exactly one SubCluster per Cluster (and hence per batch queue).
Thus sites with heterogeneous hardware have two choices:
| |||||||
The following deployment models are possible:
| ||||||||
Added: | ||||||||
> > |
| |||||||
Deleted: | ||||||||
< < |
| |||||||
0.0.1 Choose the authorization model |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 728 to 728 | ||||||||
Let's consider that a certain CREAM CE has been configured to use ARGUS as authorization system. | ||||||||
Changed: | ||||||||
< < | Let's suppose that the id of the CREAM CE in the ARGUS box (yaim variable CREAM_PEPC_RESOURCEID is http://pd.infn.it/cream-18). | |||||||
> > | Let's suppose that we chose http://pd.infn.it/cream-18 as the id of the CREAM CE (i.e. yaim variable CREAM_PEPC_RESOURCEID is http://pd.infn.it/cream-18 ). | |||||||
On the ARGUS box (identified by the yaim variable ARGUS_PEPD_ENDPOINTS ) to enable the VO XYZ, it is necessary to define the following policy: |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 813 to 813 | ||||||||
CREAM is able to protect itself if the load, memory usage, etc. is too high. This happens disabling new job submissions, while the other commands are still allowed. | ||||||||
Changed: | ||||||||
< < | The whole stuff is implemented via a limiter script (@@/usr/bin/glite_cream_load_monitor@@) very similar to the one used in the WMS. | |||||||
> > | The whole stuff is implemented via a limiter script (/usr/bin/glite_cream_load_monitor ) very similar to the one used in the WMS. | |||||||
Basically this limiter script check the values for some system and CREAM specific parameters, and compare them against some thresholds. If one or more threshold is exceeded, new job submissions get disabled. If a new submission is attempted when submissions are disabled, an error message is returned, e.g.: |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 199 to 199 | ||||||||
0.0.0.1 Installation of the CREAM CE softwareThe CREAM software itself can work with both Sun jdk and openjdk. However, the apel-core package, deployed in the CREAM CE node, requiresmm-mysql , which explicitly requires Sun jdk. So to install the middleware software needed for the CREAM CE, install first of all Sun JDK ( jdk ). This is actually not needed in a standard SL5 box, since in this case the Sun JDK rpm is available in the OS repo. | ||||||||
Added: | ||||||||
> > | Please note that this dependency on mm-mysql is being fixed by APEL developers. | |||||||
Then install xml-commons-apis : | ||||||||
Line: 258 to 259 | ||||||||
0.0.0.1 Installation of the CREAM CE softwareThe CREAM software itself can work with both Sun jdk and openjdk. However, the apel-core package, deployed in the CREAM CE node, requiresmm-mysql , which explicitly requires Sun jdk. So to install the middleware software needed for the CREAM CE, install first of all Sun JDK ( jdk ). This is actually not needed in a standard SL5 box, since in this case the Sun JDK rpm is available in the OS repo. | ||||||||
Added: | ||||||||
> > | Please note that this dependency on mm-mysql is being fixed by APEL developers. | |||||||
Then install xml-commons-apis : |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 653 to 638 | ||||||||
| ||||||||
Added: | ||||||||
> > | 0.1 MySQL database configuration guidelinesDefault values of some MySQL settings are likely to be suboptimal especially for large machines. In particular some parameters could improve the overall performance if carefully tuned.In this context one relevant parameter to be set is the innodb_buffer_poll_size which specifies the size of the buffer pool (the default value is 8MB).
The benefits obtained by using a proper value for this parameter are principally: an appreciable performance improving and the reduced amount of disk I/O needed for accessing the data in the tables. The optimal value depends on the amount of physical memory and the CPU architecture available in the host machine.
The maximum value depends on the CPU architecture, 32-bit or 64-bit. For 32-bit systems, the CPU architecture and operating system sometimes impose a lower practical maximum size. Larger this value is set, less disk I/O is needed to access data in tables. On a dedicated database server, it is possible to set this to up to 80% of the machine physical memory size. Scale back this value whether one of the following issues occur:
/etc/my.cnf , in particular within the [mysqld] section, it is suggested to customize the innodb_buffer_pool_size parameter taking into account how much physical memory is available.
Example:
[mysqld] innodb_buffer_pool_size=512MBAfter that, it's necessary to restart the mysql service for applying the change: /etc/init.d/mysqld restartFinally, the following sql command (root rights are needed) could be used for checking if the new value was applied successfully: SHOW VARIABLES like 'innodb_buffer_pool_size'; | |||||||
0.1 How to start the CREAM service | ||||||||
Line: 726 to 739 | ||||||||
} | ||||||||
Deleted: | ||||||||
< < | ||||||||
0.0.1 Security recommendationsSecurity recommendations relevant for the CREAM CE is available in http://wiki.italiangrid.org/twiki/bin/view/CREAM/ServiceReferenceCard#Security_recommendations | ||||||||
Line: 900 to 911 | ||||||||
0.1 How to check if you are using the old or the new blparser | ||||||||
Changed: | ||||||||
< < | If you want to quickly check if you are using the old or the new BLAH Blparser, do a grep registry blah.config .
If you see something like: | |||||||
> > | If you want to quickly check if you are using the old or the new BLAH Blparser, do a grep registry blah.config . If you see something like: | |||||||
# grep registry blah.config |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 633 to 633 | ||||||||
| ||||||||
Added: | ||||||||
> > |
1 PostconfigurationHave a look at the Known issue page. In particular consider the workaround needed for this problem. | |||||||
1 Operating the system1.1 Tomcat configuration guidelines |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 616 to 616 | ||||||||
The CREAM CLI is part of the EMI-UI. To configure it please refer to xxx. | ||||||||
Added: | ||||||||
> > | 0.0.1 Manual configurationyaim allows to choose the most important parameters (via yaim variables) related to the CREAM-CE. It is then possible to tune some other attributes manually editing the relevant configuration files. The following subsections describe some of the parameters that can be manually configured. Please note that:
0.0.1.1 Tune the number of concurrent BLAH instancesTo allow a parallelism when interacting with the batch system, in particular to have a good throughput when submitting jobs to the batch system, CREAM can run multiple BLAH instances. A new instance is created whenever needed, till the maximum number defined in theetc/glite-ce-cream/cream-config.xml configuration file is reached. The relevant attribute is cream_concurrency_level . The default value is 50.
| |||||||
1 Operating the system1.1 Tomcat configuration guidelines | ||||||||
Line: 873 to 890 | ||||||||
This will return all the information relevant for this job | ||||||||
Added: | ||||||||
> > | 0.1 How to check if you are using the old or the new blparserIf you want to quickly check if you are using the old or the new BLAH Blparser, do agrep registry blah.config .
If you see something like:
# grep registry blah.config job_registry=/var/blah/user_blah_job_registry.bjryou are using the new BLAH blparser. Otherwise you are using the old one. | |||||||
0.1 Job purgingPurging a CREAM job means removing it from the CREAM database and removing from the CREAM CE any information relevant for that job (e.g. the job sandbox area). |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 170 to 170 | ||||||||
The most up-to-date version of the list of trusted Certification Authorities (CA) is needed on your node. The relevant yum repo can be installed issuing: | ||||||||
Changed: | ||||||||
< < | wget http://repository.egi.eu/sw/production/cas/1/current/repo-files/egi-trustanchors.repo -O /etc/yum.repos.d/egi-trustanchors.repo | |||||||
> > | wget http://repository.egi.eu/sw/production/cas/1/current/repo-files/EGI-trustanchors.repo -O /etc/yum.repos.d/EGI-trustanchors.repo | |||||||
0.0.0.1 Important note on automatic updates |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 683 to 683 | ||||||||
0.1 Security related operations | ||||||||
Added: | ||||||||
> > |
0.0.1 How to enable a certain VO for a certain CREAM CE in ArgusLet's consider that a certain CREAM CE has been configured to use ARGUS as authorization system. Let's suppose that the id of the CREAM CE in the ARGUS box (yaim variableCREAM_PEPC_RESOURCEID is http://pd.infn.it/cream-18).
On the ARGUS box (identified by the yaim variable ARGUS_PEPD_ENDPOINTS ) to enable the VO XYZ, it is necessary to define the following policy:
resource "http://pd.infn.it/cream-18" { obligation "http://glite.org/xacml/obligation/local-environment-map" {} action ".*" { rule permit { vo = "XYZ" } } } | |||||||
0.0.1 Security recommendationsSecurity recommendations relevant for the CREAM CE is available in http://wiki.italiangrid.org/twiki/bin/view/CREAM/ServiceReferenceCard#Security_recommendations |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 199 to 199 | ||||||||
0.0.0.1 Installation of the CREAM CE software | ||||||||
Changed: | ||||||||
< < | The CREAM software itself can work with both Sun jdk and openjdk. However, the apel-core package, deployed in the CREAM CE node, requires mm-mysql , which explicitly requires Sun jdk. So to install the middleware software needed for the CREAM CE, install first of all Sun JDK ( jdk ). | |||||||
> > | The CREAM software itself can work with both Sun jdk and openjdk. However, the apel-core package, deployed in the CREAM CE node, requires mm-mysql , which explicitly requires Sun jdk. So to install the middleware software needed for the CREAM CE, install first of all Sun JDK ( jdk ). This is actually not needed in a standard SL5 box, since in this case the Sun JDK rpm is available in the OS repo. | |||||||
Line: 261 to 261 | ||||||||
0.0.0.1 Installation of the CREAM CE software | ||||||||
Changed: | ||||||||
< < | The CREAM software itself can work with both Sun jdk and openjdk. However, the apel-core package, deployed in the CREAM CE node, requires mm-mysql , which explicitly requires Sun jdk. So to install the middleware software needed for the CREAM CE, install first of all Sun JDK ( jdk ). | |||||||
> > | The CREAM software itself can work with both Sun jdk and openjdk. However, the apel-core package, deployed in the CREAM CE node, requires mm-mysql , which explicitly requires Sun jdk. So to install the middleware software needed for the CREAM CE, install first of all Sun JDK ( jdk ). This is actually not needed in a standard SL5 box, since in this case the Sun JDK rpm is available in the OS repo. | |||||||
Then install xml-commons-apis : |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 199 to 199 | ||||||||
0.0.0.1 Installation of the CREAM CE software | ||||||||
Changed: | ||||||||
< < | To install the middleware software needed for the CREAM CE, install first of all Sun JDK ( jdk ) or openjdk ( java-1.6.0-openjdk ) | |||||||
> > | The CREAM software itself can work with both Sun jdk and openjdk. However, the apel-core package, deployed in the CREAM CE node, requires mm-mysql , which explicitly requires Sun jdk. So to install the middleware software needed for the CREAM CE, install first of all Sun JDK ( jdk ). | |||||||
Then install xml-commons-apis : | ||||||||
Line: 259 to 261 | ||||||||
0.0.0.1 Installation of the CREAM CE software | ||||||||
Changed: | ||||||||
< < | To install the middleware software needed for the CREAM CE, install first of all Sun JDK ( jdk ) or openjdk ( java-1.6.0-openjdk ) | |||||||
> > | The CREAM software itself can work with both Sun jdk and openjdk. However, the apel-core package, deployed in the CREAM CE node, requires mm-mysql , which explicitly requires Sun jdk. So to install the middleware software needed for the CREAM CE, install first of all Sun JDK ( jdk ). | |||||||
Then install xml-commons-apis : |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 160 to 160 | ||||||||
You can install the EMI-1 yum repository, issuing: | ||||||||
Changed: | ||||||||
< < | rpm -Uvh http://emisoft.web.cern.ch/emisoft/dist/EMI/1/sl5/x86_64/base/emi-release-1.0.0-1.sl5.noarch.rpm | |||||||
> > | wget http://emisoft.web.cern.ch/emisoft/dist/EMI/1/sl5/x86_64/base/emi-release-1.0.0-1.sl5.noarch.rpm yum install ./emi-release-1.0.0-1.sl5.noarch.rpm | |||||||
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 157 to 157 | ||||||||
0.0.0.1 The EMI middleware repository | ||||||||
Changed: | ||||||||
< < | The EMI-1 RC4 repository can be found under: | |||||||
> > | You can install the EMI-1 yum repository, issuing: | |||||||
Changed: | ||||||||
< < | http://emisoft.web.cern.ch/emisoft/dist/EMI/1/RC4/sl5/x86_64 | |||||||
> > | rpm -Uvh http://emisoft.web.cern.ch/emisoft/dist/EMI/1/sl5/x86_64/base/emi-release-1.0.0-1.sl5.noarch.rpm | |||||||
Deleted: | ||||||||
< < | To use yum, the yum repo to be installed in /etc/yum.repos.d can be found at https://twiki.cern.ch/twiki/pub/EMI/EMI-1/rc4.repo | |||||||
0.0.0.1 The Certification Authority repository |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 998 to 998 | ||||||||
The relevant documentation is available at http://wiki.italiangrid.org/twiki/bin/view/CREAM/UserGuide#1_Forward_of_requirements_to_the | ||||||||
Added: | ||||||||
> > | 0.1 Querying the CREAM Database0.1.1 Check how many jobs are stored in the CREAM databaseThe following mysql query can be used to check how many jobs (along with their status) are reported in the CREAM database:mysql> select jstd.name, count(*) from job, job_status_type_description jstd, job_status AS status LEFT OUTER JOIN job_status AS latest ON latest.jobId=status.jobId AND status.id < latest.id WHERE latest.id IS null and job.id=status.jobId and jstd.type=status.type group by jstd.name; | |||||||
-- MassimoSgaravatto - 2011-04-07 \ No newline at end of file |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 279 to 279 | ||||||||
After the installation of the CREAM CE metapackage it is necessary to install the batch system specific metapackage(s). | ||||||||
Changed: | ||||||||
< < | If you are running LSF, install the emi-lsf-utils metapackage: | |||||||
> > |
yum install emi-torque-server yum install emi-torque-utils
yum install emi-torque-utils
| |||||||
yum install emi-lsf-utils | ||||||||
Line: 379 to 393 | ||||||||
Examples: | ||||||||
Added: | ||||||||
> > |
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n TORQUE_server -n TORQUE_utils
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n TORQUE_utils | |||||||
| ||||||||
Line: 428 to 459 | ||||||||
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n LSF_utils | ||||||||
Added: | ||||||||
> > |
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n TORQUE_server -n TORQUE_utils
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n TORQUE_utils | |||||||
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n LSF_utils -n glite-CLUSTER | ||||||||
Added: | ||||||||
> > |
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n TORQUE_server -n TORQUE_utils -n glite-CLUSTER
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n TORQUE_utils -n glite-CLUSTER | |||||||
0.0.1 Configuration of a glite-CLUSTER node0.0.1.1 Install host certificate |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 217 to 217 | ||||||||
0.0.0.1 Installation of the batch system specific software | ||||||||
Changed: | ||||||||
< < | After the installation of the CREAM CE metapackage it is necessary to install the batch system specific metapackage(s). | |||||||
> > | After the installation of the CREAM CE metapackage it is necessary to install the batch system specific metapackage(s): | |||||||
Changed: | ||||||||
< < | If you are running LSF, install the emi-lsf-utils metapackage: | |||||||
> > |
yum install emi-torque-server yum install emi-torque-utils
yum install emi-torque-utils
| |||||||
yum install emi-lsf-utils |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 22 to 22 | ||||||||
0.0.1 Batch system | ||||||||
Added: | ||||||||
> > | If you plan to use Torque as batch system for your CREAM CE, it will be installed and configured along with the middleware (i.e. you don't have to install and configure it in advance) | |||||||
If you plan to use LSF as batch system for your CREAM CE, you have to install and configure it before installing and configuring the CREAM software. Since LSF is a commercial software it can't be distributed together with the middleware.
0.1 Plan how to deploy the CREAM CE |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 642 to 641 | ||||||||
| ||||||||
Changed: | ||||||||
< < |
The choice is done at configuration time setting the yaim variable SANDBOX_TRANSFER_METHOD_BETWEEN_CE_WN .
Possible values are: | |||||||
> > | The choice is done at configuration time setting the yaim variable SANDBOX_TRANSFER_METHOD_BETWEEN_CE_WN . Possible values are: | |||||||
| ||||||||
Line: 913 to 897 | ||||||||
The CREAM job wrapper, besides running the user payload, is also responsible for other operations, such as the transfer of the input and output sandbox files from/to remote gridftp servers. | ||||||||
Changed: | ||||||||
< < | If in such transfers there is a failure, the operation is retried after a while. The time between two attempts is xxx | |||||||
> > | If in such transfers there is a failure, the operation is retried after a while. The sleep time between the first attempt and the second one is the “initial wait time” specified in the CREAM configuration file. In every next attempt the sleep time is doubled. | |||||||
In the CREAM configuration file (/etc/glite-ce-cream/cream-config.xml ) it is possible to set: |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 909 to 909 | ||||||||
When done, tomcat must be restarted. | ||||||||
Added: | ||||||||
> > | 0.0.1 Customization of the Input/Output Sandbox file transfers | |||||||
Added: | ||||||||
> > | The CREAM job wrapper, besides running the user payload, is also responsible for other operations, such as the transfer of the input and output sandbox files from/to remote gridftp servers. | |||||||
Added: | ||||||||
> > | If in such transfers there is a failure, the operation is retried after a while. The time between two attempts is xxx
In the CREAM configuration file (/etc/glite-ce-cream/cream-config.xml ) it is possible to set:
<parameter name="JOB_WRAPPER_COPY_RETRY_COUNT_ISB" value="2" /> <parameter name="JOB_WRAPPER_COPY_RETRY_FIRST_WAIT_ISB" value="60" /> <!-- sec. --> <parameter name="JOB_WRAPPER_COPY_RETRY_COUNT_OSB" value="6" /> <parameter name="JOB_WRAPPER_COPY_RETRY_FIRST_WAIT_OSB" value="300" /> <!-- sec. -->If one or more of these values are changed, it is then necessary to restart tomcat. 0.1 Managing the forwarding of requirements to the batch systemThe CREAM CE allows to forward, via tha BLAH component, requirements to the batch system. From a site administrator point of view, this requires creating and properly filling some scripts (/usr/bin/xxx_local_submit_attributes.sh ).
The relevant documentation is available at http://wiki.italiangrid.org/twiki/bin/view/CREAM/UserGuide#1_Forward_of_requirements_to_the | |||||||
-- MassimoSgaravatto - 2011-04-07 |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 643 to 633 | ||||||||
Please note that including the DN between " is important | ||||||||
Added: | ||||||||
> > |
0.1 Input and Output Sandbox files transfer between the CREAM CE and the WNThe input and output sandbox files (unless they have to be copied from/to remote servers) are copied between the CREAM CE node and the Worker Node. These files transfers can be done in two possible ways:
SANDBOX_TRANSFER_METHOD_BETWEEN_CE_WN .
Possible values are:
0.2 Sharing of the CREAM sandbox area between the CREAM CE and the WNBesides the input/output sandbox files there are some other files that need to be transferred from/to the CREAM sandbox directory on the CE node to/from the Worker Node:
0.2.1 Sharing of the CREAM sandbox area between the CREAM CE and the WN for TorqueWhen Torque is used as batch system, to share the CREAM sandbox area between the CREAM CE node and the WNs:
$usecp <CE node>://var/cream_sandbox /cream_sandboxThis $usecp line means that every time Torque will have to copy a file from/t the cream_sandbox directory on the CE (which is the case during the stage in/stage out phase), it will have to use a cp from /cream_sandbox instead.
| |||||||
0.1 Self-limiting CREAM behaviorCREAM is able to protect itself if the load, memory usage, etc. is too high. This happens disabling new job submissions, while the other commands are still allowed. | ||||||||
Line: 792 to 825 | ||||||||
0.0.1 Purging jobs in a non terminal status | ||||||||
Changed: | ||||||||
< < | TBD | |||||||
> > | The (manual or automatic) purge operation can be issued only for jobs which are in a terminal status. If it is necessary to purge a job which has been terminated but which is for CREAM in a non terminal status (e.g. RUNNING, REALLY_RUNNING) because of some bugs/problems/..., a specific utility (JobAdminPurger ) provided with the glite-ce-cream package can be used.
JobAdminPurger allows to purge jobs based on their CREAM jobids and/or their status (considering how long the job is in that status).
Usage:
JobDBAdminPurger.sh [-c|--conf CREAMConfPath] -u|--userDB userDB -p|--pswDB pswDB [-j|--jobids jobId1:jobId2:...] | [-f|--filejobIds filenameJobIds] | [-s|--status statusType0,deltaTime:statusType1:...] [-h|--help]Options:
JobDBAdminPurger.sh -u xyz -p abc -j CREAM217901296:CREAM324901232 JobDBAdminPurger.sh -u xyz -p abc -s registered:pending:idle JobDBAdminPurger.sh u xyz -p abc -s registered,3:pending:idle,5 JobDBAdminPurger.sh u xyz -p abc -f /tmp/jobIdsToPurge.txtPlease note that this script should be run just to clean the CREAM DB in case of problems (i.e. jobs reported in a non terminal status while this is not the case) Please also note that this script purges jobs from the CREAM DB. The relevant job sandbox directories are also deleted. 0.1 Proxy purgingExpired delegation proxies are automatically purged:
/etc/glite-ce-cream/cream-config.xml ) there is a property called delegation_purge_rate which defines how often the proxy purger is run. The default value is 720 (720 minutes, that is 12 hours).
If the value is changed, it is then necessary to restart tomcat.
Setting that value to -1 means disabling the proxy purger.
0.2 Job wrapper management0.2.1 Customization pointsThe CREAM JobWrapper running on the WN execute some scripts (to be provided by the local administrators) if they exist. These are calledcustomization points .
There are 3 customization points:
0.2.2 Customization of the CREAM Job wrapperTo customize the CREAM job wrapper it is just necessary to edit as appropriate the template file/usr/share/tomcat5/webapps/ce-cream/WEB-INF/jobwrapper.tpl .
When done, tomcat must be restarted. | |||||||
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 159 to 159 | ||||||||
0.0.0.1 The EMI middleware repository | ||||||||
Changed: | ||||||||
< < | The EMI-1 RC3 repository can be found under: | |||||||
> > | The EMI-1 RC4 repository can be found under: | |||||||
Changed: | ||||||||
< < | http://emisoft.web.cern.ch/emisoft/dist/EMI/1/RC3/sl5/x86_64 | |||||||
> > | http://emisoft.web.cern.ch/emisoft/dist/EMI/1/RC4/sl5/x86_64 | |||||||
Changed: | ||||||||
< < | To use yum, the yum repo to be installed in /etc/yum.repos.d can be found at https://twiki.cern.ch/twiki/pub/EMI/EMI-1/rc3.repo | |||||||
> > | To use yum, the yum repo to be installed in /etc/yum.repos.d can be found at https://twiki.cern.ch/twiki/pub/EMI/EMI-1/rc4.repo | |||||||
0.0.0.1 The Certification Authority repository |
Line: 1 to 1 | |||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | |||||||||||||||||||||||
Line: 642 to 642 | |||||||||||||||||||||||
Please note that including the DN between " is important | |||||||||||||||||||||||
Added: | |||||||||||||||||||||||
> > | 0.1 Self-limiting CREAM behavior | ||||||||||||||||||||||
Added: | |||||||||||||||||||||||
> > | CREAM is able to protect itself if the load, memory usage, etc. is too high. This happens disabling new job submissions, while the other commands are still allowed. | ||||||||||||||||||||||
Added: | |||||||||||||||||||||||
> > | The whole stuff is implemented via a limiter script (@@/usr/bin/glite_cream_load_monitor@@) very similar to the one used in the WMS. | ||||||||||||||||||||||
Added: | |||||||||||||||||||||||
> > | Basically this limiter script check the values for some system and CREAM specific parameters, and compare them against some thresholds. If one or more threshold is exceeded, new job submissions get disabled. If a new submission is attempted when submissions are disabled, an error message is returned, e.g.: | ||||||||||||||||||||||
Changed: | |||||||||||||||||||||||
< < | http://wiki.italiangrid.org/twiki/bin/view/CREAM/ServiceReferenceCard#Cron_jobs -- MassimoSgaravatto - 2011-04-07 | ||||||||||||||||||||||
> > | $ glite-ce-job-submit -a -r cream-35.pd.infn.it:8443/cream-lsf-creamtest2 myjob.jdl MethodName=[jobRegister] ErrorCode=[0] Description=[The CREAM service cannot accept jobs at the moment] FaultCause=[Threshold for Memory Usage: 95 => Detected value for Memory Usage: 96.71%] Timestamp=[Mon 02 Nov 2009 21:36:04]The limiter script is run every 10 minutes. To disable the limiter, it is necessary to edit the CREAM configuration file /etc/glite-ce-cream/cream-config.xml setting JOB_SUBMISSION_MANAGER_ENABLE to false and restarting tomcat.
The values that are currently taken into account are the following:
# Default ValuesIf needed, the limiter script can be easily augmented to take into account some other parameters. 0.1 How to drain a CREAM CEThe administrator of a CREAM CE can decide to drain a CREAM CE, that is disabling new job submissions while allowing the other commands. This can be useful for example because of scheduled shutdown of the CREAM CE. This can be achieved via theglite-ce-disable-submission command (provided by the CREAM CLI package installed on the UI), that can be issued only by a CREAM CE administrator, that is the DN of this person must be listed in the /etc/grid-security/admin-list file of the CE.
If newer job submissions are attempted, users will get an error message such as:
> glite-ce-job-submit -a -r grid006.pd.infn.it:8443/cream-lsf-grid02 lnl_test.j\dl MethodName=[jobRegister] ErrorCode=[0] Description=[The CREAM2 service cannot accept jobs anymore] FaultCause=[The CREAM2 service cannot accept jobs anymore]\ Timestamp=[Tue 22 Jan 2008 16:28:47]It is possible to then resume new job submissions calling the glite-ce-enable-submission command.
To check if job submissions on a specific CREAM CE are allowed, the command glite-ce-allowed-submission can be used.
It is possible to resume the job submission calling the proper operation (glite-ce-enable-submission ).
E.g.:
> glite-ce-disable-submission grid006.pd.infn.it:8443 Operation for disabling new submissions succeeded > > glite-ce-allowed-submission grid006.pd.infn.it:8443 Job Submission to this CREAM CE is disabled > > glite-ce-enable-submission grid006.pd.infn.it:8443 Operation for enabling new submissions succeeded > > glite-ce-allowed-submission grid006.pd.infn.it:8443 Job Submission to this CREAM CE is enabled 0.2 How to trace a specific jobTo trace a specific job, first of all get the CREAMjobid. If the job was submitted through the WMS, you can get its CREAMjobdid in the following way:glite-wms-job-logging-info -v 2 <gridjobdid> | grep "Dest jobid"If the job is not yours and you are not LB admin, you can get the CREAMjobid of that gridjobid if you have access to the CREAM logs doing: grep <gridjobid> /var/log/cream.glite-ce-cream.log*Grep the "last part" of the CREAMjobid in the CREAM log file (e.g. if the CREAMjobid is https://cream-07.pd.infn.it:8443/CREAM383606450 considers CREAM383606450): grep CREAM383606450 /var/log/cream/glite-ce-cream.log*This will return all the information relevant for this job 0.3 Job purgingPurging a CREAM job means removing it from the CREAM database and removing from the CREAM CE any information relevant for that job (e.g. the job sandbox area). When a job has been purged, it is not possible to manage it anymore (e.g. it is not possible to check anymore its status). A job can be purged:
/etc/glite_wms.conf ) the attribute purge_jobs in the ICE section is set to false .
0.3.1 Automatic job purgingThe automatic CREAM job purger is responsible to purge old - forgotten jobs, according to a policy specified in the CREAM configuration file (/etc/glite-ce-cream/cream-config.xml ).
This policy is specified by the attribute JOB_PURGE_POLICY .
For example, if JOB_PURGE_POLICY is the following:
<parameter name="JOB_PURGE_POLICY" value="ABORTED 1 days; CANCELLED 2 days; DONE-OK 3 days; DONE-FAILED 4 days; REGISTERED 5 days;" />then the job purger will purge jobs which are:
0.3.2 Purging jobs in a non terminal statusTBD -- MassimoSgaravatto - 2011-04-07 |
Line: 1 to 1 | |||||||||
---|---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | |||||||||
Line: 10 to 9 | |||||||||
0.1 Prerequisites0.1.1 Operating system | |||||||||
Deleted: | |||||||||
< < | A standard 64 bit SL(C)5 distribution is supposed to be properly installed. | ||||||||
Added: | |||||||||
> > | A standard 64 bit SL(C)5 distribution is supposed to be properly installed. | ||||||||
0.0.1 Node synchronization | |||||||||
Deleted: | |||||||||
< < | A general requirement for the Grid nodes is that they are synchronized. This requirement may be fulfilled in several ways. One of the most common one is using the NTP protocol with a time server. | ||||||||
Added: | |||||||||
> > | A general requirement for the Grid nodes is that they are synchronized. This requirement may be fulfilled in several ways. One of the most common one is using the NTP protocol with a time server. | ||||||||
0.0.1 Cron and logrotate | |||||||||
Deleted: | |||||||||
< < | Many components deployed on the CREAM CE rely on the presence of cron (including support for /etc/cron.* directories) and logrotate . You should make sure these utils are available on your system. | ||||||||
Added: | |||||||||
> > | Many components deployed on the CREAM CE rely on the presence of cron (including support for /etc/cron.* directories) and logrotate . You should make sure these utils are available on your system. | ||||||||
0.0.1 Batch system | |||||||||
Deleted: | |||||||||
< < | If you plan to use LSF as batch system for your CREAM CE, you have to install and configure it before installing and configuring the CREAM software. Since LSF is a commercial software it can't be distributed together with the middleware. | ||||||||
Added: | |||||||||
> > | If you plan to use LSF as batch system for your CREAM CE, you have to install and configure it before installing and configuring the CREAM software. Since LSF is a commercial software it can't be distributed together with the middleware. | ||||||||
0.1 Plan how to deploy the CREAM CE | |||||||||
Line: 92 to 86 | |||||||||
BLPARSER_WITH_UPDATER_NOTIFIER=true | |||||||||
Changed: | |||||||||
< < | in the siteinfo.def. This is the default value.
The new BLParser doesn't parse the log files. However the bhist (for LSF) and tracejob (for Torque) commands (used by the new BLParser) require the batch system log files, which therefore must be available (in case e.g. via NFS in the CREAM CE node.
Actually for Torque the blparser uses tracejob (which requires the log files) only when qstat can't find anymore the job. And this can happen if the job has been completed more than keep_completed seconds ago and the blparser was not able to detect before that the job completed/was cancelled/whatever. This can happen e.g. if keep_completed is too short or if the BLAH blparser for whatever reason didn't run for a while. If the log files are not available and the tracejob command is issued (for the reasons specified above), the BLAH blparser will not be able to find the job, which will considered "lost" (DONE-FAILED wrt CREAM). | ||||||||
> > | in the siteinfo.def and then configure the CREAM CE. This is the default value. | ||||||||
Added: | |||||||||
> > | The new BLParser doesn't parse the log files. However the bhist (for LSF) and tracejob (for Torque) commands (used by the new BLParser) require the batch system log files, which therefore must be available (in case e.g. via NFS in the CREAM CE node. Actually for Torque the blparser uses tracejob (which requires the log files) only when qstat can't find anymore the job. And this can happen if the job has been completed more than keep_completed seconds ago and the blparser was not able to detect before that the job completed/was cancelled/whatever. This can happen e.g. if keep_completed is too short or if the BLAH blparser for whatever reason didn't run for a while. If the log files are not available and the tracejob command is issued (for the reasons specified above), the BLAH blparser will not be able to find the job, which will considered "lost" (DONE-FAILED wrt CREAM). | ||||||||
Changed: | |||||||||
< < | The init script of the new Blparser is /etc/init.d/glite-ce-blahparser .
Please note that it is not needed to explicitly start the new blparser: when CREAM is started, it starts also this new BLAH Blparser if it is not already running. | ||||||||
> > | The init script of the new Blparser is /etc/init.d/glite-ce-blahparser . Please note that it is not needed to explicitly start the new blparser: when CREAM is started, it starts also this new BLAH Blparser if it is not already running. | ||||||||
When the new Blparser is running, you should see the following two processes on the CREAM CE node: | |||||||||
Line: 106 to 97 | |||||||||
| |||||||||
Deleted: | |||||||||
< < | |||||||||
Please note that the user tomcat on the CREAM CE should be allowed to issue the relevant status/history commands (for Torque: qstat , tracejob , for LSF: bhist , bjobs ). Some sites configure the batch system so that users can only see their own jobs (e.g. in torque: | |||||||||
Changed: | |||||||||
< < | set server query_other_jobs = False). If this is done at the site, then the tomcat user will need a special privilege in order to be exempt from this setting (in torque: set server operators += tomcat@creamce.yoursite.domain). | ||||||||
> > | set server query_other_jobs = False). If this is done at the site, then the tomcat user will need a special privilege in order to be exempt from this setting (in torque: set server operators += tomcat@creamce.yoursite.domain). 0.0.0.1 Old BLAH BlparserThe old BLAH blparser must be installed on a machine where the batch system log files are available (let's call this hostBLPARSER_HOST . So the BLPARSER_HOST can be the batch system master or a different machine where the log files are available (e.g. they have been exported via NFS). There are two possible layouts:
BLPARSER_WITH_UPDATER_NOTIFIER=truein the siteinfo.def before configuring via yaim. | ||||||||
0.1 Installation | |||||||||
Line: 129 to 145 | |||||||||
| |||||||||
Deleted: | |||||||||
< < | |||||||||
and to REMOVE (!!!) or DEACTIVATE (!!!)
| |||||||||
Line: 305 to 307 | |||||||||
yum install emi-cluster | |||||||||
Added: | |||||||||
> > | 0.0.1 Installation of the BLAH BLparserIf the new BLAH Blparser must be used, there isn't anything to be installed for the BLAH Blparser (i.e. the installation of the CREAM-CE is enough). This is also the case when the old BLAH Blparser must be used AND the BLPARSER_HOST is the CREAM-CE. Only when the old BLAH Blparser must be used AND the BLPARSER_HOST is different than the CREAM-CE, it is necessary to install the BLParser software on this BLPARSER_HOST. This is done in the following way:yum install glite-ce-blahp yum install glite-yaim-cream-ce | ||||||||
0.0.1 Installation of the CREAM CLI | |||||||||
Line: 329 to 342 | |||||||||
| |||||||||
Changed: | |||||||||
< < |
make sure to place the two files in the target node into the /etc/grid-security directory.
Then set the proper mode and ownerships doing: | ||||||||
> > | make sure to place the two files in the target node into the /etc/grid-security directory. Then set the proper mode and ownerships doing: | ||||||||
chown root.root /etc/grid-security/hostcert.pem | |||||||||
Line: 374 to 383 | |||||||||
| |||||||||
Changed: | |||||||||
< < |
make sure to place the two files in the target node into the /etc/grid-security directory.
Then set the proper mode and ownerships doing: | ||||||||
> > | make sure to place the two files in the target node into the /etc/grid-security directory. Then set the proper mode and ownerships doing: | ||||||||
chown root.root /etc/grid-security/hostcert.pem | |||||||||
Line: 427 to 432 | |||||||||
| |||||||||
Changed: | |||||||||
< < |
make sure to place the two files in the target node into the /etc/grid-security directory.
Then set the proper mode and ownerships doing: | ||||||||
> > | make sure to place the two files in the target node into the /etc/grid-security directory. Then set the proper mode and ownerships doing: | ||||||||
chown root.root /etc/grid-security/hostcert.pem | |||||||||
Line: 438 to 441 | |||||||||
chmod 400 /etc/grid-security/hostkey.pem | |||||||||
Deleted: | |||||||||
< < | |||||||||
0.0.0.1 Configure the siteinfo.def file | |||||||||
Deleted: | |||||||||
< < | Set your siteinfo.def file, which is the input file used by yaim. Documentation about yaim variables relevant for glite-CLUSTER is available at https://twiki.cern.ch/twiki/bin/view/LCG/Site-info_configuration_variables#CLUSTER | ||||||||
Added: | |||||||||
> > | Set your siteinfo.def file, which is the input file used by yaim. Documentation about yaim variables relevant for glite-CLUSTER is available at https://twiki.cern.ch/twiki/bin/view/LCG/Site-info_configuration_variables#CLUSTER | ||||||||
0.0.0.1 Run yaim | |||||||||
Line: 452 to 453 | |||||||||
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n glite-CLUSTER | |||||||||
Added: | |||||||||
> > | 0.0.1 Configuration of the BLAH BlparserIf the new BLAH Blparser must be used, there isn't anything to be configured for the BLAH Blparser (i.e. the configuration of the CREAM-CE is enough). If the old BLparser must be used, it is necessary to configure it on the BLPARSER_HOST (which, as said above, can be the CREAM-CE node or on a different host). This is done in the following way:/opt/glite/yaim/bin/yaim -r -s <site-info.def> -n creamCE -f config_cream_blparserThen it is necessary to restart tomcat on the CREAM-CE node: service tomcat5 restart 0.0.1.1 Configuration of the old BLAH Blparser to serve multiple CREAM CEsThe configuration instructions reported above explains how to configure a CREAM CE and the BLAH blparser (old model) considering the scenario where the BLAH blparser has to "serve" a single CREAM CE. Considering that the blparser (old model) has to run where the batch system log files are available, let's consider a scenario where there are 2 CREAM CEs (ce1.mydomain and ce2.mydomain ) that must be configured. Let's suppose that the batch system log files are not available on these 2 CREAM CEs machine. Let's assume they are available in another machine ( blhost.mydomain ), where the old blparser has to be installed.
The following summarizes what must be done:
BLPARSER_HOST=blhost.mydomain BLAH_JOBID_PREFIX=cre01_ BLP_PORT=33333and configure ce1.mydomain via yaim:
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n <LRMSnode> [-n glite-CLUSTER]
BLPARSER_HOST=blhost.mydomain BLAH_JOBID_PREFIX=cre02_ BLP_PORT=33334and configure ce2.mydomain via yaim:
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n <LRMSnode> [-n glite-CLUSTER]
CREAM_PORT=56565and configure blhost.mydomain via yaim:
/opt/glite/yaim/bin/yaim -r -s <site-info.def> -n creamCE -f config_cream_blparser
GLITE_CE_BLPARSERPBS_NUM=2 # ce01.mydomain GLITE_CE_BLPARSERPBS_PORT1=33333 GLITE_CE_BLPARSERPBS_CREAMPORT1=56565 # ce02.mydomain GLITE_CE_BLPARSERPBS_PORT2=33334 GLITE_CE_BLPARSERPBS_CREAMPORT2=56566
/etc/init.d/glite-ce-blparser restart
| ||||||||
0.0.1 Configuration of the CREAM CLIThe CREAM CLI is part of the EMI-UI. To configure it please refer to xxx. | |||||||||
Deleted: | |||||||||
< < | -- MassimoSgaravatto - 2011-04-07 | ||||||||
\ No newline at end of file | |||||||||
Added: | |||||||||
> > | 1 Operating the system1.1 Tomcat configuration guidelinesIn/etc/tomcat5/tomcat5.conf , there are some settings related to heap. They are in the JAVA_OPTS setting (see -Xms and -Xmx ).
It is suggested to customize such settings taking into account how much physical memory is available, as indicated in the following table (which refers to 64bit architectures):
1.2 How to start the CREAM serviceA site admin can start the CREAM service just starting the CREAM container:/etc/init.d/tomcat5 startIn case the new BLAH blparser is used, this will also start it (if not already running). If for some reason it necessary to explicitly start the new BLAH blparser, the following command can be used: /etc/init.d/glite-ce-blahparser startIf instead the old BLAH blparser is used, before starting tomcat it is necessary to start it on the BLPARSER_HOST using the command: /etc/init.d/glite-ce-blparser startTo stop the CREAM service, it is just necessary to stop the CREAM container: /etc/init.d/tomcat5 stop 1.3 DaemonsInformation about daemons running in the CREAM CE is available in http://wiki.italiangrid.org/twiki/bin/view/CREAM/ServiceReferenceCard#Daemons_running1.4 Init scriptsInformation about init scripts in the CREAM CE is available in the http://wiki.italiangrid.org/twiki/bin/view/CREAM/ServiceReferenceCard#Init_scripts_and_options_start_s1.5 Configuration filesInformation about configuration files in the CREAM CE is available in http://wiki.italiangrid.org/twiki/bin/view/CREAM/ServiceReferenceCard#Configuration_files_location_wit1.6 Log filesInformation about log files in the CREAM CE is available in http://wiki.italiangrid.org/twiki/bin/view/CREAM/ServiceReferenceCard#Logfile_locations_and_management1.7 Network portsInformation about ports used in the CREAM CE is available in http://wiki.italiangrid.org/twiki/bin/view/CREAM/ServiceReferenceCard#Open_ports1.8 Cron jobsInformation about cron jobs used in the CREAM CE is available in http://wiki.italiangrid.org/twiki/bin/view/CREAM/ServiceReferenceCard#Cron_jobs1.9 Security related operations1.9.1 Security recommendationsSecurity recommendations relevant for the CREAM CE is available in http://wiki.italiangrid.org/twiki/bin/view/CREAM/ServiceReferenceCard#Security_recommendations1.9.2 How to block/ban a userInformation about how to ban users is available in http://wiki.italiangrid.org/twiki/bin/view/CREAM/ServiceReferenceCard#How_to_block_ban_a_user1.9.3 How to block/ban a VOTo ban a VO, it is suggested to reconfigure the service via yaim without that VO in thesiteinfo.def
1.9.4 How to define a CREAM administratorA CREAM administrator (aka super-user) can manage (e.g. cancel, check the status, etc.) also the jobs submitted by other people. Moreover he/she can issue some privileged operations, in particular the ones to disable the new job submissions (glite-ce-disable-submission ) and then to re-enable them (glite-ce-disable-submission )
To define a CREAM CE administrator for a specific CREAM CE, the DN of this person must be specified in the /etc/grid-security/admin-list of this CREAM CE node, e.g.:
"/C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Massimo Sgaravatto"Please note that including the DN between " is important http://wiki.italiangrid.org/twiki/bin/view/CREAM/ServiceReferenceCard#Cron_jobs -- MassimoSgaravatto - 2011-04-07 |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 306 to 306 | ||||||||
Added: | ||||||||
> > | 0.0.1 Installation of the CREAM CLIThe CREAM CLI is part of the EMI-UI. To install it please refer to xxx. | |||||||
0.1 Configuration0.1.1 Using the YAIM configuration tool | ||||||||
Line: 448 to 453 | ||||||||
Added: | ||||||||
> > | 0.0.1 Configuration of the CREAM CLI | |||||||
Added: | ||||||||
> > | The CREAM CLI is part of the EMI-UI. To configure it please refer to xxx. | |||||||
-- MassimoSgaravatto - 2011-04-07 |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 72 to 72 | ||||||||
0.0.1 Choose the BLAH BLparser deployment model | ||||||||
Changed: | ||||||||
< < | The BLAH Blparser is .... | |||||||
> > | The BLAH Blparser is the component of the CREAM CE responsible to notify CREAM about job status changes. | |||||||
Added: | ||||||||
> > | For LSF and PBS/Torque it is possible to configure the BLAH blparser in two possible ways:
0.0.0.1 New BLAH BlparserThe new Blparser runs on the CREAM CE machine and it is automatically installed when installing the CREAM CE. The configuration of the new BLAH Blparser is done when configuring the CREAM CE (i.e. it is not necessary to configure the Blparser separately from the CREAM CE). To use the new BLAH blparser, it is just necessary to set:BLPARSER_WITH_UPDATER_NOTIFIER=truein the siteinfo.def. This is the default value. The new BLParser doesn't parse the log files. However the bhist (for LSF) and tracejob (for Torque) commands (used by the new BLParser) require the batch system log files, which therefore must be available (in case e.g. via NFS in the CREAM CE node.
Actually for Torque the blparser uses tracejob (which requires the log files) only when qstat can't find anymore the job. And this can happen if the job has been completed more than keep_completed seconds ago and the blparser was not able to detect before that the job completed/was cancelled/whatever. This can happen e.g. if keep_completed is too short or if the BLAH blparser for whatever reason didn't run for a while. If the log files are not available and the tracejob command is issued (for the reasons specified above), the BLAH blparser will not be able to find the job, which will considered "lost" (DONE-FAILED wrt CREAM).
The init script of the new Blparser is /etc/init.d/glite-ce-blahparser .
Please note that it is not needed to explicitly start the new blparser: when CREAM is started, it starts also this new BLAH Blparser if it is not already running.
When the new Blparser is running, you should see the following two processes on the CREAM CE node:
tomcat on the CREAM CE should be allowed to issue the relevant status/history commands (for Torque: qstat , tracejob , for LSF: bhist , bjobs ). Some sites configure the batch system so that users can only see their own jobs (e.g. in torque:
set server query_other_jobs = False). If this is done at the site, then the tomcat user will need a special privilege in order to be exempt from this setting (in torque: set server operators += tomcat@creamce.yoursite.domain). | |||||||
0.1 Installation |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 70 to 70 | ||||||||
USE_ARGUS=no | ||||||||
Added: | ||||||||
> > | 0.0.1 Choose the BLAH BLparser deployment model | |||||||
Changed: | ||||||||
< < | ||||||||
> > | The BLAH Blparser is .... | |||||||
0.1 Installation |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release | ||||||||
Line: 41 to 41 | ||||||||
More information about glite-CLUSTER can be found at https://twiki.cern.ch/twiki/bin/view/LCG/CLUSTER | ||||||||
Added: | ||||||||
> > | 0.0.1 Choose the authorization modelThe CREAM CE can be configured to use as authorization system:
USE_ARGUS must be set in the following way:
USE_ARGUS=yesIn this case it is also necessary to set the following yaim variables:
USE_ARGUS must be set in the following way:
USE_ARGUS=no | |||||||
0.1 InstallationThis section explains how to install: | ||||||||
Line: 122 to 155 | ||||||||
0.0.0.1 Installation of the CREAM CE software | ||||||||
Changed: | ||||||||
< < | To install the middleware software needed for the CREAM CE, install first of all xml-commons-apis :
, install first of all xml-commons-apis : | |||||||
> > | To install the middleware software needed for the CREAM CE, install first of all Sun JDK (jdk ) or openjdk (java-1.6.0-openjdk )
Then install xml-commons-apis : | |||||||
yum install xml-commons-apis | ||||||||
Line: 170 to 204 | ||||||||
0.0.0.1 Installation of the CREAM CE software | ||||||||
Changed: | ||||||||
< < | To install the middleware software needed for the CREAM CE, install first of all xml-commons-apis :
, install first of all xml-commons-apis : | |||||||
> > |
To install the middleware software needed for the CREAM CE, install first of all Sun JDK (jdk ) or openjdk (java-1.6.0-openjdk )
Then install xml-commons-apis : | |||||||
yum install xml-commons-apis |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
System Administrator Guide for CREAM for EMI-1 release
| ||||||||
Deleted: | ||||||||
< < | Installation and Configuration | |||||||
Added: | ||||||||
> > | 1 Installation and Configuration | |||||||
Changed: | ||||||||
< < | Prerequisites | |||||||
> > | 0.1 Prerequisites | |||||||
Changed: | ||||||||
< < | Operating system | |||||||
> > | 0.0.1 Operating system | |||||||
A standard 64 bit SL(C)5 distribution is supposed to be properly installed. | ||||||||
Changed: | ||||||||
< < | Node synchronization | |||||||
> > | 0.0.1 Node synchronization | |||||||
A general requirement for the Grid nodes is that they are synchronized. This requirement may be fulfilled in several ways. One of the most common one is using the NTP protocol with a time server. | ||||||||
Changed: | ||||||||
< < | Cron and logrotate | |||||||
> > | 0.0.1 Cron and logrotate | |||||||
Many components deployed on the CREAM CE rely on the presence of cron (including support for /etc/cron.* directories) and logrotate . You should make sure these utils are available on your system. | ||||||||
Changed: | ||||||||
< < | Batch system | |||||||
> > | 0.0.1 Batch system | |||||||
If you plan to use LSF as batch system for your CREAM CE, you have to install and configure it before installing and configuring the CREAM software. Since LSF is a commercial software it can't be distributed together with the middleware. | ||||||||
Changed: | ||||||||
< < | Plan how to deploy the CREAM CE | |||||||
> > | 0.1 Plan how to deploy the CREAM CE | |||||||
Changed: | ||||||||
< < | CREAM CE and gLite-cluster | |||||||
> > | 0.0.1 CREAM CE and gLite-cluster | |||||||
glite-CLUSTER is a node type that can publish information about clusters and subclusters in a site, referenced by any number of compute elements. glite-CLUSTER can be deployed in the same host of the CREAM-CE or in a different one. | ||||||||
Changed: | ||||||||
< < | The following deployment model are possible: | |||||||
> > | The following deployment models are possible: | |||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
More information about glite-CLUSTER can be found at https://twiki.cern.ch/twiki/bin/view/LCG/CLUSTER | ||||||||
Changed: | ||||||||
< < | Installation | |||||||
> > | 0.1 Installation | |||||||
Added: | ||||||||
> > | This section explains how to install: | |||||||
Changed: | ||||||||
< < | First of all, install the yum-protectbase rpm: | |||||||
> > |
| |||||||
Changed: | ||||||||
< < | yum install yum-protectbase.noarch | |||||||
> > | For all these scenarios, the setting of the repositories is the same. | |||||||
Deleted: | ||||||||
< < | Then proceed with the setting of the software repositories | |||||||
Changed: | ||||||||
< < | Repositories | |||||||
> > | 0.0.1 Repositories | |||||||
For a successful installation, you will need to configure your package manager to reference a number of repositories (in addition to your OS); | ||||||||
Line: 65 to 65 | ||||||||
| ||||||||
Changed: | ||||||||
< < | The EPEL repository | |||||||
> > | 0.0.0.1 The EPEL repository | |||||||
You can install the EPEL repository, issuing: | ||||||||
Line: 73 to 73 | ||||||||
rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-4.noarch.rpm | ||||||||
Changed: | ||||||||
< < | The EMI middleware repository | |||||||
> > |
0.0.0.1 The EMI middleware repository | |||||||
The EMI-1 RC3 repository can be found under: | ||||||||
Line: 83 to 84 | ||||||||
To use yum, the yum repo to be installed in /etc/yum.repos.d can be found at https://twiki.cern.ch/twiki/pub/EMI/EMI-1/rc3.repo | ||||||||
Changed: | ||||||||
< < | The Certification Authority repository | |||||||
> > |
0.0.0.1 The Certification Authority repository | |||||||
The most up-to-date version of the list of trusted Certification Authorities (CA) is needed on your node. The relevant yum repo can be installed issuing: | ||||||||
Line: 92 to 94 | ||||||||
Changed: | ||||||||
< < | Important note on automatic updates | |||||||
> > | 0.0.0.1 Important note on automatic updates | |||||||
An update of an RPM not followed by configuration can cause problems. Therefore WE STRONGLY RECOMMEND NOT TO USE AUTOMATIC UPDATE PROCEDURE OF ANY KIND. Running the script available at http://forge.cnaf.infn.it/frs/download.php/101/disable_yum.sh (implemented by Giuseppe Platania (INFN Catania) yum autoupdate will be disabled | ||||||||
Changed: | ||||||||
< < | Installation of the CA certificates | |||||||
> > | 0.0.1 Installation of a CREAM CE node in no cluster modeFirst of all, install theyum-protectbase rpm:
yum install yum-protectbase.noarchThen proceed with the installation of the CA certificates. 0.0.1.1 Installation of the CA certificates | |||||||
The CA certificate can be installed issuing: | ||||||||
Line: 108 to 120 | ||||||||
Changed: | ||||||||
< < | Installation of the CREAM CE software | |||||||
> > | 0.0.0.1 Installation of the CREAM CE software | |||||||
To install the middleware software needed for the CREAM CE, install first of all xml-commons-apis :
, install first of all xml-commons-apis : | ||||||||
Line: 126 to 138 | ||||||||
yum install emi-cream-ce | ||||||||
Changed: | ||||||||
< < | Installation of the batch system specific software | |||||||
> > | 0.0.0.1 Installation of the batch system specific software | |||||||
After the installation of the CREAM CE metapackage it is necessary to install the batch system specific metapackage(s). | ||||||||
Line: 136 to 148 | ||||||||
yum install emi-lsf-utils | ||||||||
Changed: | ||||||||
< < | Installation of the cluster metapackage | |||||||
> > |
0.0.1 Installation of a CREAM CE node in cluster modeFirst of all, install theyum-protectbase rpm:
yum install yum-protectbase.noarchThen proceed with the installation of the CA certificates. 0.0.1.1 Installation of the CA certificatesThe CA certificate can be installed issuing:yum install ca-policy-egi-core 0.0.1.2 Installation of the CREAM CE softwareTo install the middleware software needed for the CREAM CE, install first of allxml-commons-apis :
, install first of all xml-commons-apis :
yum install xml-commons-apisThis is due to a dependency problem within the Tomcat distribution Then install the CREAM-CE metapackage: yum install emi-cream-ce 0.0.1.3 Installation of the batch system specific softwareAfter the installation of the CREAM CE metapackage it is necessary to install the batch system specific metapackage(s). If you are running LSF, install theemi-lsf-utils metapackage:
yum install emi-lsf-utils 0.0.1.4 Installation of the cluster metapackage | |||||||
If the CREAM CE node has to host also the glite-cluster , install also this metapackage: | ||||||||
Line: 145 to 206 | ||||||||
Changed: | ||||||||
< < | Configuration | |||||||
> > | 0.0.1 Installation of a glite-cluster nodeFirst of all, install theyum-protectbase rpm:
yum install yum-protectbase.noarchThen proceed with the installation of the CA certificates. 0.0.1.1 Installation of the CA certificatesThe CA certificate can be installed issuing:yum install ca-policy-egi-core 0.0.1.2 Installation of the cluster metapackageInstall the glite-CLUSTER metapackage:yum install emi-cluster | |||||||
Changed: | ||||||||
< < | Install host certificate | |||||||
> > | 0.1 Configuration0.1.1 Using the YAIM configuration toolFor a detailed description on how to configure the middleware with YAIM, please check the YAIM guide. The necessary YAIM modules needed to configure a certain node type are automatically installed with the middleware.0.1.2 Configuration of a CREAM CE node in no cluster mode0.1.2.1 Install host certificate | |||||||
The CREAM CE node requires the host certificate/key files to be installed. Contact your national Certification Authority (CA) to understand how to obtain a host certificate if you do not have one already. | ||||||||
Line: 166 to 262 | ||||||||
chmod 400 /etc/grid-security/hostkey.pem | ||||||||
Added: | ||||||||
> > | 0.0.0.1 Configure the siteinfo.def fileSet yoursiteinfo.def file, which is the input file used by yaim. Documentation about yaim variables relevant for CREAM CE is available at https://twiki.cern.ch/twiki/bin/view/LCG/Site-info_configuration_variables#cream_CE | |||||||
Changed: | ||||||||
< < | Using the YAIM configuration tool | |||||||
> > | Be sure that CREAMCE_CLUSTER_MODE is set to no (or not set at all). | |||||||
Changed: | ||||||||
< < | For a detailed description on how to configure the middleware with YAIM, please check the YAIM guide. | |||||||
> > | 0.0.0.1 Run yaimAfter having filled thesiteinfo.def file, run yaim:
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n <LRMSnode>Examples:
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n LSF_utils 0.0.1 Configuration of a CREAM CE node in cluster mode0.0.1.1 Install host certificateThe CREAM CE node requires the host certificate/key files to be installed. Contact your national Certification Authority (CA) to understand how to obtain a host certificate if you do not have one already. Once you have obtained a valid certificate:
/etc/grid-security directory.
Then set the proper mode and ownerships doing:
chown root.root /etc/grid-security/hostcert.pem chown root.root /etc/grid-security/hostkey.pem chmod 600 /etc/grid-security/hostcert.pem chmod 400 /etc/grid-security/hostkey.pem | |||||||
Deleted: | ||||||||
< < | The necessary YAIM modules needed to configure a certain node type are automatically installed with the middleware. | |||||||
Changed: | ||||||||
< < | Configure the siteinfo.def file | |||||||
> > | 0.0.0.1 Configure the siteinfo.def file | |||||||
Set your siteinfo.def file, which is the input file used by yaim. Documentation about yaim variables relevant for CREAM CE is available at https://twiki.cern.ch/twiki/bin/view/LCG/Site-info_configuration_variables#cream_CE | ||||||||
Changed: | ||||||||
< < | Run yaim | |||||||
> > | Be sure that CREAMCE_CLUSTER_MODE is set to yes
0.0.0.1 Run yaim | |||||||
After having filled the siteinfo.def file, run yaim: | ||||||||
Line: 184 to 321 | ||||||||
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n | ||||||||
Changed: | ||||||||
< < | -n glite-CLUSTER must be specified only if the CREAM CE is in cluster mode AND the glite-CLUSTER is deployed in the same node of the CREAM-CE | |||||||
> > | -n glite-CLUSTER must be specified only if the glite-CLUSTER is deployed in the same node of the CREAM-CE | |||||||
Examples: | ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n LSF_utils | ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
Changed: | ||||||||
< < | /opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n LSF_utils | |||||||
> > | /opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n LSF_utils -n glite-CLUSTER | |||||||
Deleted: | ||||||||
< < |
| |||||||
Added: | ||||||||
> > | 0.0.1 Configuration of a glite-CLUSTER node0.0.1.1 Install host certificateThe glite-CLUSTER node requires the host certificate/key files to be installed. Contact your national Certification Authority (CA) to understand how to obtain a host certificate if you do not have one already. Once you have obtained a valid certificate:
/etc/grid-security directory.
Then set the proper mode and ownerships doing: | |||||||
Changed: | ||||||||
< < | /opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n LSF_utils -n glite-CLUSTER | |||||||
> > | chown root.root /etc/grid-security/hostcert.pem chown root.root /etc/grid-security/hostkey.pem chmod 600 /etc/grid-security/hostcert.pem chmod 400 /etc/grid-security/hostkey.pem | |||||||
Added: | ||||||||
> > | 0.0.0.1 Configure the siteinfo.def fileSet yoursiteinfo.def file, which is the input file used by yaim. Documentation about yaim variables relevant for glite-CLUSTER is available at https://twiki.cern.ch/twiki/bin/view/LCG/Site-info_configuration_variables#CLUSTER
0.0.0.2 Run yaimAfter having filled thesiteinfo.def file, run yaim:
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n glite-CLUSTER | |||||||
-- MassimoSgaravatto - 2011-04-07 |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
Changed: | ||||||||
< < | System Administrator Guide for CREAM for EMI-1 release | |||||||
> > | System Administrator Guide for CREAM for EMI-1 release | |||||||
| ||||||||
Line: 22 to 22 | ||||||||
Batch system | ||||||||
Changed: | ||||||||
< < | If you plan to use LSF as batch system for your CREAM CE, you have to install and configure it before installing and configuring the CREAM software | |||||||
> > | If you plan to use LSF as batch system for your CREAM CE, you have to install and configure it before installing and configuring the CREAM software. Since LSF is a commercial software it can't be distributed together with the middleware. | |||||||
Added: | ||||||||
> > | Plan how to deploy the CREAM CECREAM CE and gLite-clusterglite-CLUSTER is a node type that can publish information about clusters and subclusters in a site, referenced by any number of compute elements. glite-CLUSTER can be deployed in the same host of the CREAM-CE or in a different one. The following deployment model are possible:
| |||||||
Installation | ||||||||
Line: 40 to 56 | ||||||||
For a successful installation, you will need to configure your package manager to reference a number of repositories (in addition to your OS); | ||||||||
Added: | ||||||||
> > |
| |||||||
| ||||||||
Deleted: | ||||||||
< < |
| |||||||
and to REMOVE (!!!) or DEACTIVATE (!!!)
| ||||||||
Added: | ||||||||
> > | The EPEL repositoryYou can install the EPEL repository, issuing:rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-4.noarch.rpm | |||||||
The EMI middleware repository | ||||||||
Added: | ||||||||
> > | The EMI-1 RC3 repository can be found under:
http://emisoft.web.cern.ch/emisoft/dist/EMI/1/RC3/sl5/x86_64To use yum, the yum repo to be installed in /etc/yum.repos.d can be found at https://twiki.cern.ch/twiki/pub/EMI/EMI-1/rc3.repo
The Certification Authority repositoryThe most up-to-date version of the list of trusted Certification Authorities (CA) is needed on your node. The relevant yum repo can be installed issuing:wget http://repository.egi.eu/sw/production/cas/1/current/repo-files/egi-trustanchors.repo -O /etc/yum.repos.d/egi-trustanchors.repo Important note on automatic updatesAn update of an RPM not followed by configuration can cause problems. Therefore WE STRONGLY RECOMMEND NOT TO USE AUTOMATIC UPDATE PROCEDURE OF ANY KIND. Running the script available at http://forge.cnaf.infn.it/frs/download.php/101/disable_yum.sh (implemented by Giuseppe Platania (INFN Catania) yum autoupdate will be disabledInstallation of the CA certificatesThe CA certificate can be installed issuing:yum install ca-policy-egi-core Installation of the CREAM CE softwareTo install the middleware software needed for the CREAM CE, install first of allxml-commons-apis :
, install first of all xml-commons-apis :
yum install xml-commons-apisThis is due to a dependency problem within the Tomcat distribution Then install the CREAM-CE metapackage: yum install emi-cream-ce Installation of the batch system specific softwareAfter the installation of the CREAM CE metapackage it is necessary to install the batch system specific metapackage(s). If you are running LSF, install theemi-lsf-utils metapackage:
yum install emi-lsf-utils Installation of the cluster metapackageIf the CREAM CE node has to host also theglite-cluster , install also this metapackage:
yum install emi-cluster ConfigurationInstall host certificateThe CREAM CE node requires the host certificate/key files to be installed. Contact your national Certification Authority (CA) to understand how to obtain a host certificate if you do not have one already. Once you have obtained a valid certificate:
/etc/grid-security directory.
Then set the proper mode and ownerships doing:
chown root.root /etc/grid-security/hostcert.pem chown root.root /etc/grid-security/hostkey.pem chmod 600 /etc/grid-security/hostcert.pem chmod 400 /etc/grid-security/hostkey.pem Using the YAIM configuration toolFor a detailed description on how to configure the middleware with YAIM, please check the YAIM guide. The necessary YAIM modules needed to configure a certain node type are automatically installed with the middleware.Configure the siteinfo.def fileSet yoursiteinfo.def file, which is the input file used by yaim. Documentation about yaim variables relevant for CREAM CE is available at https://twiki.cern.ch/twiki/bin/view/LCG/Site-info_configuration_variables#cream_CE
Run yaimAfter having filled thesiteinfo.def file, run yaim:
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n <LRMSnode> [-n glite-CLUSTER] -n glite-CLUSTER must be specified only if the CREAM CE is in cluster mode AND the glite-CLUSTER is deployed in the same node of the CREAM-CE
Examples:
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n LSF_utils
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n LSF_utils
/opt/glite/yaim/bin/yaim -c -s <site-info.def> -n creamCE -n LSF_utils -n glite-CLUSTER | |||||||
-- MassimoSgaravatto - 2011-04-07 |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
Line: 6 to 6 | ||||||||
Installation and Configuration | ||||||||
Added: | ||||||||
> > |
PrerequisitesOperating systemA standard 64 bit SL(C)5 distribution is supposed to be properly installed.Node synchronizationA general requirement for the Grid nodes is that they are synchronized. This requirement may be fulfilled in several ways. One of the most common one is using theNTP protocol with a time server.
Cron and logrotateMany components deployed on the CREAM CE rely on the presence ofcron (including support for /etc/cron.* directories) and logrotate . You should make sure these utils are available on your system.
Batch systemIf you plan to use LSF as batch system for your CREAM CE, you have to install and configure it before installing and configuring the CREAM softwareInstallationFirst of all, install theyum-protectbase rpm:
yum install yum-protectbase.noarchThen proceed with the setting of the software repositories RepositoriesFor a successful installation, you will need to configure your package manager to reference a number of repositories (in addition to your OS);
The EMI middleware repository | |||||||
-- MassimoSgaravatto - 2011-04-07 |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
Added: | ||||||||
> > |
Installation and Configuration-- MassimoSgaravatto - 2011-04-07 |