Difference: IgiEmi (34 vs. 35)

Revision 352012-02-17 - CristinaAiftimiei

Line: 1 to 1
 
META TOPICPARENT name="IGIGuides"

IGI (based on EMI) Installation and Configuration

Line: 597 to 597
 Before starting the installation procedure remember to clean all yum cache and headers:


Changed:
<
<
yum clean all
>
>
# yum clean all
 

WN Prerequisites

Line: 611 to 611
 

CAs installation:

  • Install CAs on ALL profiles:
Changed:
<
<
yum install ca-policy-egi-core

>
>
# yum install ca-policy-egi-core

 

Service installation

Line: 619 to 619
 
  • Have a look to the WN documentation before starting the installationl: WN Guide.
  • Install the WN metapackages, containing all packages needed:

Changed:
<
<
yum install <metapackage>
>
>
# yum install <metapackage>
 
  • IGI provides 5 custom profiles (metapackages) one for each specific batch system used in your cluster. Please use one of these (see table bellow)
    • IGI customizations:
Line: 659 to 659
 
  • Before starting the configuration PLEASE TEST that you have defined all the mandatory variables for all the WN profiles (WN_torque, WN_torque_noafs, WN_LSF, WN_LSF_noafs)

For Torque:

Changed:
<
<
 /opt/glite/yaim/bin/yaim -v -s <site-info.def> -n  WN_torque 
>
>
# /opt/glite/yaim/bin/yaim -v -s <site-info.def> -n  WN_torque 
 or
Changed:
<
<
 /glite/yaim/bin/yaim -v -s <site-info.def> -n  WN_torque_noafs 
>
>
# /glite/yaim/bin/yaim -v -s <site-info.def> -n  WN_torque_noafs 
  For LSF:
Changed:
<
<
 /opt/glite/yaim/bin/yaim -v -s <site-info.def> -n  WN_LSF
>
>
# /opt/glite/yaim/bin/yaim -v -s <site-info.def> -n  WN_LSF
 or
Changed:
<
<
 /opt/glite/yaim/bin/yaim -v -s <site-info.def> -n  WN_LSF_noafs
>
>
# /opt/glite/yaim/bin/yaim -v -s <site-info.def> -n  WN_LSF_noafs
  For Other Batch System:
Changed:
<
<
 /opt/glite/yaim/bin/yaim -v -s <site-info.def> -n  WN
>
>
# /opt/glite/yaim/bin/yaim -v -s <site-info.def> -n  WN
  You can find in this documentation: YAIM WN Variables all mandatory variables.
Line: 687 to 687
 

  • For Torque:
Changed:
<
<
 /opt/glite/yaim/bin/yaim -c -d 6 -s  <site-info.def>  -n  WN_torque 2>&1 | tee /root/conf_WN_torque.`hostname -s`.`date +%Y-%m-%d-%H-%M-%S`.log
>
>
 # /opt/glite/yaim/bin/yaim -c -d 6 -s  <site-info.def>  -n  WN_torque 2>&1 | tee /root/conf_WN_torque.`hostname -s`.`date +%Y-%m-%d-%H-%M-%S`.log
 or
Changed:
<
<
 /opt/glite/yaim/bin/yaim -c -d 6 -s  <site-info.def>  -n  WN_torque_noafs 2>&1 | tee /root/conf_WN_torque_noafs.`hostname -s`.`date +%Y-%m-%d-%H-%M-%S`.log
>
>
 # /opt/glite/yaim/bin/yaim -c -d 6 -s  <site-info.def>  -n  WN_torque_noafs 2>&1 | tee /root/conf_WN_torque_noafs.`hostname -s`.`date +%Y-%m-%d-%H-%M-%S`.log
 
  • For LSF:
Changed:
<
<
 /opt/glite/yaim/bin/yaim -c -d 6 -s  <site-info.def>  -n  WN_LSF 2>&1 | tee /root/conf_WN_LSF`.hostname -s`.`date +%Y-%m-%d-%H-%M-%S`.log
>
>
 # /opt/glite/yaim/bin/yaim -c -d 6 -s  <site-info.def>  -n  WN_LSF 2>&1 | tee /root/conf_WN_LSF`.hostname -s`.`date +%Y-%m-%d-%H-%M-%S`.log
 or
Changed:
<
<
 /opt/glite/yaim/bin/yaim -c -d 6 -s  <site-info.def>  -n  WN_LSF_noafs 2>&1 | tee /root/conf_WN_LSF_noafs.`hostname -s`.`date +%Y-%m-%d-%H-%M-%S`.log
>
>
 # /opt/glite/yaim/bin/yaim -c -d 6 -s  <site-info.def>  -n  WN_LSF_noafs 2>&1 | tee /root/conf_WN_LSF_noafs.`hostname -s`.`date +%Y-%m-%d-%H-%M-%S`.log
 
  • For Other Batch System:
 /opt/glite/yaim/bin/yaim -c -d 6 -s  <site-info.def>  -n  WN 2>&1 | tee /root/conf_WN.`hostname -s`.`date +%Y-%m-%d-%H-%M-%S`.log
Line: 715 to 715
 
  • Have a look to the section Repository Settings and ensure that you have the common repo files
  • Before starting the installation procedure remember to clean all yum cache and headers:

Changed:
<
<
yum clean all
>
>
# yum clean all
 

CAs installation:

  • Install CAs on ALL profiles:
Changed:
<
<
yum install ca-policy-egi-core

>
>
# yum install ca-policy-egi-core

 

Service installation

Line: 729 to 729
 
  • Install the UI metapackages, containing all packages needed.


Changed:
<
<
yum install emi-ui
>
>
# yum install emi-ui
 

Service Configuration

Line: 738 to 738
 
  • Before starting the configuration PLEASE TEST that you have defined all the mandatory variables for the UI profile
Changed:
<
<
 /opt/glite/yaim/bin/yaim -v -s <site-info.def> -n  UI 
>
>
 # /opt/glite/yaim/bin/yaim -v -s <site-info.def> -n  UI 
  You can find in this documentation: YAIM UI Variables all mandatory variables.
Line: 769 to 769
 
  • Have a look to the section Repository Settings and ensure that you have the common repo files
  • Before starting the installation procedure remember to clean all yum cache and headers:
Changed:
<
<
yum clean all
>
>
# yum clean all 
 

MPI Service installation on CE-CREAM

Changed:
<
<
Follow the CE-CREAM installation guide before starting to follow this steps below
>
>
  • Follow the CE-CREAM installation guide to install a creamCE
  • Install the MPI metapackage. It provides mpi_start and glite-yaim-mpi for configuration
 
Changed:
<
<
yum install glite-mpi
>
>
# yum install glite-mpi 
 
IMPORTANT NOTE:
Changed:
<
<
At the moment the metapackage name has not already change from glite-mpi to emi-mpi. Please install glite-mpi also in emi cream-ce and emi or igi worker nodes
>
>
Please pay attention to the metapackage name! It is glite-mpi, NOT emi-mpi or the previous one glite-MPI_utils.
Please install glite-mpi both in the creamCE and WNs nodes (IGI/EMI flavour)
 

MPI Service installation on WN

Changed:
<
<
Follow the WN installation guide before starting to follow this steps below
>
>
  • Follow the WN installation guide
  • Install the MPI metapackage. It provides mpi_start and glite-yaim-mpi for configuration
 
Changed:
<
<
yum install glite-mpi
>
>
# yum install glite-mpi 
  • WNs also require a working MPI implementation, Open MPI and MPICH-2 are recommended. The devel packages should also be installed in order to allow user to compile their applications. Refer to your OS repositories for the exact packages.
    • If you would like to use the OPEN MPI flavour in your site please install:
      #yum install openmpi openmpi-devel 
    • If you would like to use the MPICH2 flavour in your site please install:
      # yum install mpich2 
 
IMPORTANT NOTE:
Changed:
<
<
At the moment the metapackage name has not already change from glite-mpi to emi-mpi. Please install glite-mpi also in emi cream-ce and emi or igi worker nodes
>
>
If you are using Torque remember to create the munge key, and copy it to all cluster hosts (CE, Batch Master, WNs): Munge configuration
 
Changed:
<
<
If you would like to use the OPEN MPI flavour in your site please install:
yum install openmpi openmpi-devel 

If you would like to use the MPICH2 flavour in your site please install:

yum install mpich2 

  • Have a look to the MPI documentation before starting to install: MPI Guides.

IMPORTANT NOTE:
Remember if you are using Torque to create the munge key, and copy it to all cluster hosts (CE, Batch Master, WNs): Munge configuration

Service Configuration

>
>

Service Configuration

 

Useful Variables

Changed:
<
<
Remember to copy these three files in your service directory:
  • glite-mpi
  • glite-mpi_ce
  • glite-mpi_wn

Please set properly the variables in the files in service directory. In particular customize this important values:

>
>
  • Remember to copy these three files in your services/ directory:
    • /opt/glite/yaim/examples/siteinfo/services/glite-mpi
    • /opt/glite/yaim/examples/siteinfo/services/glite-mpi_ce
    • /opt/glite/yaim/examples/siteinfo/services/glite-mpi_wn
  • Set properly the variables in the above files in services/ directory. In particular customize these important values:
 
Line: 850 to 830
 

YAIM Verification

  • Before starting the configuration PLEASE TEST that you have defined all the mandatory variables for the MPI profile
Changed:
<
<
    • On CE-CREAM MPI Verification
 /opt/glite/yaim/bin/yaim -v -s <site-info.def> -n MPI_CE -n creamCE -n TORQUE_server -n TORQUE_utils 

    • On WN MPI Verification
 /opt/glite/yaim/bin/yaim -v -s <site-info.def> -n MPI_WN -n WN_torque_noafs 

You can find in this documentation: YAIM MPI Variables all mandatory variables.

If no errors are reported during the verification you can proceed to the configuration, otherwise correct them before continuing.

>
>
    • On CE-CREAM MPI Verification:
      # /opt/glite/yaim/bin/yaim -v -s <site-info.def> -n MPI_CE -n creamCE -n TORQUE_server -n TORQUE_utils 
    • On WN MPI Verification:
      # /opt/glite/yaim/bin/yaim -v -s <site-info.def> -n MPI_WN -n WN_torque_noafs 
  • You can find in the "YAIM MPI Variables" documentation information about all mandatory variables.
    • If no errors are reported during the verification you can proceed to the configuration, otherwise correct them.
 

YAIM Configuration

Line: 869 to 843
 
IMPORTANT NOTE:
Changed:
<
<
Remember to put at first position the nodetype MPI_CE or MPI_WN
>
>
When configuring with yaim remember to put first the nodetype MPI_CE or MPI_WN
 
Added:
>
>
 
    • On CE-CREAM
Changed:
<
<
 /opt/glite/yaim/bin/yaim -c -d 6 -s <site-info.def>  -n MPI_CE -n creamCE -n TORQUE_server -n TORQUE_utils 2>&1 | tee /root/conf_EMI_CREAM_Torque_MPI.`hostname -s`.`date +%Y-%m-%d-%H-%M-%S`.log 
>
>
# /opt/glite/yaim/bin/yaim -c -d 6 -s <site-info.def>  -n MPI_CE -n creamCE -n TORQUE_server -n TORQUE_utils 2>&1 | tee /root/conf_EMI_CREAM_Torque_MPI.`hostname -s`.`date +%Y-%m-%d-%H-%M-%S`.log 
 
    • On WN
Changed:
<
<
  /opt/glite/yaim/bin/yaim -c -d 6 -s <site-info.def> -n MPI_WN -n WN_torque_noafs  2>&1 | tee /root/conf_WN_Torque_MPI.`hostname -s`.`date +%Y-%m-%d-%H-%M-%S`.log 
>
>
#  /opt/glite/yaim/bin/yaim -c -d 6 -s <site-info.def> -n MPI_WN -n WN_torque_noafs  2>&1 | tee /root/conf_WN_Torque_MPI.`hostname -s`.`date +%Y-%m-%d-%H-%M-%S`.log 
 

Check files in WN

Changed:
<
<
If in YAIM Configuration you are chose to use MPI_SSH_HOST_BASED_AUTH (well-advised) check these files:
  • /etc/ssh/sshd_config
  • /etc/ssh/shosts.equiv
>
>
  • If in YAIM Configuration step you choose to use MPI_SSH_HOST_BASED_AUTH (recommended) check these files:
    • /etc/ssh/sshd_config
    • /etc/ssh/shosts.equiv
 
Changed:
<
<
The first one sshd_config should have the follow variables set like below
>
>
  • The first one, sshd_config, should have the follow variables set like below
 
HostbasedAuthentication yes 
IgnoreUserKnownHosts yes
IgnoreRhosts yes
Changed:
<
<
The second file shosts.equiv should contains: the CE hostname, the default SE host name and all the WNs hostname. May be in the CE it was created in configuration process, copy it form the CE.

Restart sshd services after file modification:

service sshd restart
>
>
  • The second file, shosts.equiv, should contain: the CE hostname, the default SE host name and all the WNs hostname. Maybe in the CE it was created in configuration process, copy it form there to all WNs
  • Restart sshd services after file modification:
 service sshd restart
 

Service Testing - Reference Card

Changed:
<
<
After service installation to have a look if all were installed in a proper way, you could have a look to Service Reference Card and User & Service Troubleshooting Guide. In this page you can find some common errors and other useful service information.
>
>
 

MPI Documentation References:

Changed:
<
<
>
>
 

META FILEATTACHMENT attachment="Upgrade_Instructions_to_StoRM_v18.pdf" attr="" comment="Upgrade StoRM istructions. Useful to the data stored partition" date="1321956018" name="Upgrade_Instructions_to_StoRM_v18.pdf" path="Upgrade_Instructions_to_StoRM_v18.pdf" size="55433" user="SergioTraldi" version="1"
 
This site is powered by the TWiki collaboration platformCopyright © 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback