• Hardware Prerequisite:
  • 1 dedicated host (it could be both virtual or physical) to install Wnodes Nameserver
  • A repository for the Virtual Machine images
    • It could be a shared file system (NFS, Lustre, GPFS, etc) or a Web Server
  • At least one physical machine where to install bait virtual host and where the job will be executed
• Software Prerequisite:
  • Download the WNoDeS RPM from here: https://web.infn.it/wnodes/index.php/all-wnodes-packages
  • A Torque/Maui batch server
  • A DHCP server
  • With the possibility to modify its configuration
  • The capability to add new DNS entry for each service/node
  • Install KVM on all the real nodes
    
    • yum groupinstall KVM
    • yum install pyOpenSSL
• YAIM related configuration and operations:
  • Create and publish the new queue dedicated to WNoDeS
    • On the CE edit the YAIM site.def file putting:
      • "VOS" to the list of VOs that you want to support:
      • "QUEUES" to the list of queues that you want to expose to the grid
      • and define "QUEUENAME_GROUP_ENABLE" equal to the VO that will be enable to the specific queue (where "QUEUENAME" is the name of the queue)
      • Reconfigure the CE with YAIM
• Step Needed:
  • Create a IP/DNS for each virtual node that will be created (1bait + n Virtual WN, per real Node)
    • the hostname for the bait host should contain "bait" (e.g.: bait-wnodes-XX)
    • the hostname for the virtual node should contain "vm" (e.g.: vm-wnodes-XX)
  • Install a single nameserver:
    • yum install wnodes_manager-1.0.0-2.noarch.rpm wnodes_nameserver-1.2.2-3.noarch.rpm wnodes_utils-1.1.5-3.noarch.rpm
    • configure file: /var/log/wnodes/nameserver/wnodes_ns_state and /etc/wnodes/nameserver/wnodes_hv_config.ini /etc/wnodes/nameserver/wnodes_bait_config.ini
  • Download the images for the bait and for the WN
  • Configure the new nodes on the torque server editing the file /var/spool/pbs/server_priv/nodes like the following (where the first node i a standard node farm, the sencond is an example of a bait and the third is a virtual node that will run the jobs):
    • wn01.cnaf.infn.it np=2 lcgpro
    • bait-tf-01.cnaf.infn.it np=2 cloudtf bait
    • vn-tf-01.cnaf.infn.it np=1 vn-tf-01
      • the properties are really important and should be configured as in the example
  • edit di configuration files and configure the images for both using YAIM in order to customize the images for you site
  • Create a dedicated queue for wnodes jobs:
    • For examples using: qmgr -c < queue_command :
      • cat queue_command:
        • create queue cloudtf
        • set queue cloudtf queue_type = Execution
        • set queue cloudtf Priority = 1000000
        • set queue cloudtf max_running = 80
        • set queue cloudtf resources_max.cput = 100:00:00
        • set queue cloudtf resources_max.walltime = 100:00:00
        • set queue cloudtf resources_default.neednodes = cloudtf
        • set queue cloudtf enabled = True
        • set queue cloudtf started = True
  • Configure the torque server in order to allow job management from WNoDeS hosts (real machine and baits):
    • qmgr -c "set server managers += root@gridlab02.cnaf.infn.it"
    • qmgr -c "set server managers += root@gridlab03.cnaf.infn.it"
    • qmgr -c "set server managers += root@gridlab10.cnaf.infn.it"
    • qmgr -c "set server managers += root@gridlab20.cnaf.infn.it"
    • qmgr -c "set server managers += root@bait-tf-01.cnaf.infn.it"
    • qmgr -c "set server managers += root@bait-tf-02.cnaf.infn.it"
    • qmgr -c "set server managers += root@bait-tf-03.cnaf.infn.it"
    • qmgr -c "set server managers += root@bait-tf-04.cnaf.infn.it"
  • Configure maui in order to create a partition (editing maui.cfg):
    • NODECFG[vm01-bait.ba.infn.it] PARTITION=virtual
    • CLASSCFG[cloudtf] PLIST=virtual PDEF=virtual
  • Configure the torque server to let the "real" job to use only the others queue. Execute on each of the others queue something like:
    • qmgr -c "set queue prod resources_default.neednodes = lcgpro"
    • qmgr -c "set queue cert resources_default.neednodes = lcgpro"
  • Configure the CEs, WNs and submission nodes in general to allow ssh without password:
    • each WN could login without password to the CE and among WNs
    • it is suggested to put the same ssh host keys in all the nodes (both bait and WN) in order to easy the managing of the host list
  • Configure the prologue in all bait nodes:
    • chmod 500 /var/torque/mom_priv/prologue
    • chmod 500 /root/ wnodes_preexec
  • Configure the epilogue in all WN nodes:
    
    • chmod 500 /var/torque/mom_priv/ epilogue
    • chmod 500 /root/ wnodes_postexec
      
• Configure WNoDeS:
  • On the nameserver configure the file: /etc/wnodes/nameserver/mac_list.ini
    • and then start the service:
      • service wnodes_nameserver start
  • On the real nodes hosting bait and virtual WN node configure this file: /etc/wnodes/nameserver/wnodes_hv_config.ini
    • And then start the service:
      • service wnodes_hypervisor start
  • On the virtual nodes installed as bait configure this file: /etc/wnodes/nameserver/wnodes_bait_config.ini
    • And then start the service:
      • service wnodes_bait start

-- GiacintoDonvito - 2011-11-05

• WNoDeS schema: