You are here: TWiki> Cloud Web>WNoDeSPBSInstallation (revision 8)
  • Hardware Prerequisite:
    • 1 dedicated host (it could be both virtual or physical) to install Wnodes Nameserver
    • A repository for the Virtual Machine images
      • It could be a shared file system (NFS, Lustre, GPFS, etc) or a Web Server
    • At least one physical machine where to install bait virtual host and where the job will be executed
  • Software Prerequisite:
    • Download the WNoDeS RPM from here: https://web.infn.it/wnodes/index.php/all-wnodes-packages
    • A Torque/Maui batch server
    • A DHCP server
    • With the possibility to modify its configuration
    • The capability to add new DNS entry for each service/node
    • Install KVM on all the real nodes
      • yum groupinstall KVM
      • yum install pyOpenSSL
  • YAIM related configuration and operations:
    • Create and publish the new queue dedicated to WNoDeS
      • On the CE edit the YAIM site.def file putting:
        • "VOS" to the list of VOs that you want to support:
        • "QUEUES" to the list of queues that you want to expose to the grid
        • and define "QUEUENAME_GROUP_ENABLE" equal to the VO that will be enable to the specific queue (where "QUEUENAME" is the name of the queue)
        • Reconfigure the CE with YAIM
  • Step Needed:
    • Create a IP/DNS for each virtual node that will be created (1bait + n Virtual WN, per real Node)
      • the hostname for the bait host should contain "bait" (e.g.: bait-wnodes-XX)
      • the hostname for the virtual node should contain "vm" (e.g.: vm-wnodes-XX)
    • Install a single nameserver:
      • yum install wnodes_manager-1.0.0-2.noarch.rpm wnodes_nameserver-1.2.2-3.noarch.rpm wnodes_utils-1.1.5-3.noarch.rpm
      • configure file: /var/log/wnodes/nameserver/wnodes_ns_state and /etc/wnodes/nameserver/wnodes_hv_config.ini /etc/wnodes/nameserver/wnodes_bait_config.ini
    • Download the images for the bait and for the WN
    • Configure the new nodes on the torque server editing the file /var/spool/pbs/server_priv/nodes like the following (where the first node i a standard node farm, the sencond is an example of a bait and the third is a virtual node that will run the jobs):
      • wn01.cnaf.infn.it np=2 lcgpro
      • bait-tf-01.cnaf.infn.it np=2 cloudtf bait
      • vn-tf-01.cnaf.infn.it np=1 vn-tf-01
        • the properties are really important and should be configured as in the example
    • edit di configuration files and configure the images for both using YAIM in order to customize the images for you site
    • Create a dedicated queue for wnodes jobs:
      • For examples using: qmgr -c < queue_command :
        • cat queue_command:
          • create queue cloudtf
          • set queue cloudtf queue_type = Execution
          • set queue cloudtf Priority = 1000000
          • set queue cloudtf max_running = 80
          • set queue cloudtf resources_max.cput = 100:00:00
          • set queue cloudtf resources_max.walltime = 100:00:00
          • set queue cloudtf resources_default.neednodes = cloudtf
          • set queue cloudtf enabled = True
          • set queue cloudtf started = True
    • Configure the torque server in order to allow job management from WNoDeS hosts (real machine and baits):
    • Configure maui in order to create a partition (editing maui.cfg):
      • NODECFG[vm01-bait.ba.infn.it] PARTITION=virtual
      • CLASSCFG[cloudtf] PLIST=virtual PDEF=virtual
    • Configure the torque server to let the "real" job to use only the others queue. Execute on each of the others queue something like:
      • qmgr -c "set queue prod resources_default.neednodes = lcgpro"
      • qmgr -c "set queue cert resources_default.neednodes = lcgpro"
    • Configure the CEs, WNs and submission nodes in general to allow ssh without password:
      • each WN could login without password to the CE and among WNs
      • it is suggested to put the same ssh host keys in all the nodes (both bait and WN) in order to easy the managing of the host list
    • Configure the prologue in all bait nodes:
    • Configure the epilogue in all WN nodes:
  • Configure WNoDeS:
    • On the nameserver configure the file: /etc/wnodes/nameserver/mac_list.ini
      • and then start the service:
        • service wnodes_nameserver start
    • On the real nodes hosting bait and virtual WN node configure this file: /etc/wnodes/nameserver/wnodes_hv_config.ini
      • And then start the service:
        • service wnodes_hypervisor start
    • On the virtual nodes installed as bait configure this file: /etc/wnodes/nameserver/wnodes_bait_config.ini
      • And then start the service:
        • service wnodes_bait start

-- GiacintoDonvito - 2011-11-05

Topic attachments
I Attachment Action Size Date Who CommentSorted ascending
Unknown file formatEXT epilogue manage 0.1 K 2011-11-08 - 22:20 GiacintoDonvito  
Textini mac_list.ini manage 0.8 K 2011-11-07 - 23:27 GiacintoDonvito  
Unknown file formatEXT prologue manage 0.1 K 2011-11-05 - 11:33 GiacintoDonvito  
Textini wnodes_bait_config.ini manage 0.7 K 2011-11-07 - 23:30 GiacintoDonvito  
Textini wnodes_hv_config.ini manage 1.6 K 2011-11-07 - 23:36 GiacintoDonvito  
Unknown file formatEXT wnodes_postexec manage 3.4 K 2011-11-08 - 22:21 GiacintoDonvito  
Unknown file formatEXT wnodes_preexec manage 10.4 K 2011-11-08 - 22:19 GiacintoDonvito  
PNGpng wnodes_schema.png manage 109.0 K 2011-12-23 - 09:59 GiacintoDonvito WNoDeS schema
Topic revision: r8 - 2012-02-20 - GiacintoDonvito
 
TWIKI.NET
This site is powered by the TWiki collaboration platformCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback