IGI CERTIFICATION TESTBED
This testbed is supposed to be used by the "Testing, Certification and Release" group people; it should be used mainly for functionality tests and for stress tests.
HOWTO, POLICIES & PROCEDURES
HOWTO Request a new instance
To request a new instance for certification please email grid-release mailing list specifying:
- When the instance is needed for (>request day + 3 working days)
- What is the instance needed for, and its reference in the inventory before (or whether it needs to be added)
- HW requirements (DISK, RAM)
- Virtual Machine deployment is default, please specify whether you need a physical server
- Host certificate requirements
- OS platform
- Grid service to install
- Life Time Estimate
Reasonable requests will be taken in charge and tracked as sub-task on Jira task TESTBED MANAGEMENT number 7. The requester will be added as "watcher" in the relative sub-task.
HOWTO Install Debian 6.0 on virtual KVM
- Requirements:
- dom0 server running KVM with Intel VIrtualization technology flag enabled (check with "cat /proc/cpuinfo |grep vmx")
- host public IP + MAC address number
- connect to your dom0: $ ssh -X root@dom0
- Installation from scratch
- Download iso image: $wget http://linux.studenti.polito.it/linuxstudenti/debian-6.0.2.1-amd64-netinst.iso
- NEW Virtual Image:
- Start virt image, click on dom0 hostname, select NEW and enter following parameters: $ virt-image&
- fully virtualized, x64, kvm
- local install ISO, OS linux, Generic 2.6
- select from browser the iso previously downloaded
- select storage (file or partition depending on your choice) and size (>= 10GB)
- shared physical bridge, (eth0/1), fixed MAC (choose a free one from NewEntropy)
- from virt-image console enter parameters for installation
- NETWORK: IP, gateway, nameserver
- users (root passwd, one user)
- US keyboard, ...
- ssh server + graphycs if needed
- Once finished, enter machine from console and run:
- Installation from previously created template image:
- Shutdown template image to clone (ex. cert-31-Debian is the template image for which we have both .img file and xml conf file)
- Clone image:
- $virt-clone --original-xml=XMLFILEOFTEMPLATE --name=NEWmachineNAMEinVIRTMANAGER --mac=NEWMAC_address --file=DestinationImageFILE
- Example: $virt-clone --original-xml=/etc/libvirt/qemu/cert-31-Debian.xml --name=cert-32-Debian --mac=00:16:3e:08:01:9b --file=/virtdisk1/cert-32-Debian.img
- New image post configuration
- NOTE The original image must be NOT running
- start new machine from virt-manager and enter machine console:
- CHANGE HOSTNAME: $sed -i 's/OLD_HOSTNAME/NEW_HOSTNAME/g' /etc/hostname
- CHANGE NETWORK CONF:
- $ifconfing > check which eth is used
- $ifdown ethXX (from above)
- editing files: /etc/resolv.con , /etc/network/interfaces (new IP there)
- restart network: $ifup ethXX *$ ifconfig ----> to check it is fine.
- restart both original and new machine.
- remove puppet files and rerun the puppet configuration commands as in https://yam-server.cnaf.infn.it/EMIinstall/emi-post-install.sh script *$ apt-get remove puppet *$ rm /root/puppetfile?HOSTNAME *execute the following:
wget -q http://doctorwho.cnaf.infn.it/cgi-bin/hostinfo.cgi?certname=$HOSTNAME
apt-get install -y puppet
touch /etc/puppet/namespaceauth.conf
TEST=`grep doctorwho /etc/puppet/puppet.conf|wc -l`
if [ $TEST -eq 0 ] ; then
cat >> /etc/puppet/puppet.conf << EOF
server = doctorwho.cnaf.infn.it
report = true
listen = true
runinterval = 360
EOF
update-rc.d puppet enable 5
sed -i 's/START=no/START=yes/g' /etc/default/puppet
/etc/init.d/puppet restart
fi
HOWTO Restore clean OS image disk in XEN
- Requirements and checks:
- Daemons stopped (or in @chkconfig on@) in server hosting your service
- Backup File Image with clean OS
- Connect to dom0 server, as root
- get help on xen manager with:
xm
- List running machines:
$xm list
- Shutdown your machine with: =$xm shutdown NAME_OF_MACHINE_AS_FROM_XM_LIST_OUTPUT (ex.. emitestbed09-SL6X-64)
- Restore clean os image file from backup image:
cp emitestbed18-SL5X-64_CLEANOS.img emitestbed18-SL5X-64.img
- N.B. The image file used for a given virtual server is stored in the xen configuration file. ex:
[root@emitest-dom01 ~]# cat /etc/xen/emitestbed18-SL5X-64
name = "emitestbed18-SL5X-64"
builder = "hvm"
memory = "1024"
disk = [ 'file:/domU/emitestbed18-SL5X-64.img,hda,w' ] <<<<<<<<<<<<<<<<<<<<<<<<<<< THE FILE<<<<<<<<<<<<<<<<
boot = "c"
vif = [ 'mac=00:16:3e:08:01:76, bridge=xenbr0' ]
vfb = ["type=vnc,vncunused=1"]
device_model = "/usr/lib64/xen/bin/qemu-dm"
kernel = "/usr/lib/xen/boot/hvmloader"
vcpus=1
on_reboot = 'restart'
on_crash = 'restart'
- START THE NEW MACHINE WITH: $>xm create emitestbed18-SL5X-64 ----------> same name as the conf file without path /etc/xen/emitestbed18-SL5X-64
PUPPET SETUP
AVAILABLE MODULES
- BASENODE: deployed on all IGI servers (production + R&D)
- fail2ban -> bans users aftern X wrong password attempts
- lcg-ca -> egi trustanchor repo + CA
- lemon -> lemon monitoring setup
- nrpe -> nagios probes
- nsca -> nagios probes
- ntp -> ntp server setup
- pakiti-> pakiti security monitoring setup
- puppet
- repo-updates
- resolv
- security-checks -> monitor existence of dangerous files
- security-updates -> monitor existence of dangerous files
- ssh -> ssh setup + some ssh keys
- yum -> repos
- EMI AND TESTBEDS: ON ALL EMITESTBED + IGI CERTIFICATION/RELEASE nodes
- emi-cerntb -> install cern utilities repo + BitFAce CA for Nagios tests
- emi-testers-vo -> creates following files /root/siteinfo/ + /root/siteinfo/groups.conf+users.conf + /root/siteinfo/vo.d/testers2.eu-emi.eu+testers.eu-emi.eu
- emi-generic-vo -> creates following files /root/siteinfo/ + /root/siteinfo/groups.conf+users.conf + /root/siteinfo/vo.d/testers2.eu-emi.eu+testers.eu-emi.eu+cms+dteam (other VO can be added if needed)
- emi-users -> creates users with password for pre-registered bastion users in IGI Middleware Unit
- emi-testwrapperscript -> creates file /root/siteinfo/script-wrapper.py
- EMI AND TESTBEDS: REPOSITORIES AVAILABLE ON DEMAND
- emi1-prodrepo -> install emi1 release package (on emi1 emitesbed servers)
- emi1-releaseupdatesrepo -> install emi1-deployment repo (on emi1 emitesbed servers)
- emi2-prodrepo -> install emi2 release package
- emi2-rcrepo -> install emi2 RC repo
- EMI TESTBEDS: AUTOMATED DEPLOYMENT MODULES
- ARGUS
- SCENARIO files to deploy an argus with emitestbed policy emi2-argus-deploy -> Provides template files for automated deployment test with /root/siteinfo/script-wrapper.py |-- files | |-- EMI2_SL5-64_ARGUS_deployement_command_list.txt | |-- EMI2_SL6-64_ARGUS_deployement_command_list.txt | |-- emitestbed_policy.txt | `-- site-info.def
- WMS
- SCENARIO files to deploy a WMS from testing repo and emitestbed setup. emi1-wmstesting-deploy -> Provides template files for automated deployment test with /root/siteinfo/script-wrapper.py |-- files |-- EMI1_TESTING_SL5-64_WMS_deployement_command_list.txt | `-- site-info.def
- SCENARIO files to deploy a WMS from production then update it to testing repo and emitestbed setup. emi1-wmsupdate-deploy -> Provides template files for automated deployment test with /root/siteinfo/script-wrapper.py |-- files |-- |-- EMI1_UPDATE_SL5-64_WMS_deployement_command_list.txt `-- site-info.def
- IGI CERTIFICATION: AUTOMATED DEPLOYMENT MODULES
- WMS
- SCENARIO: install a wms from testing repo with multi VO enabled and certification bdii etc. : emi1-wms-cert-install -> Provides template files for automated deployment test with /root/siteinfo/script-wrapper.py |-- files | |-- EMI1_CERT_INSTALL_SL5-64_WMS_deployement_command_list.txt | -- site-info.def
- SCENARIO: install a wms from production repo with multi VO enabled and certification bdii etc. then update it to testing: emi1-wms-cert-update -> Provides template files for automated deployment test with /root/siteinfo/script-wrapper.py |-- files | |-- | |-- EMI1_CERT_UPDATE_SL5-64_WMS_deployement_command_list.txt | -- site-info.def
TESTBED INVENTORY
Other services
| HOSTNAME |
SO |
SERVICE |
State |
Note |
| emitestbed53.cnaf.infn.it |
SL6/x86_64 |
BDII_top 1.1.3 (EMI 3) |
 |
BDII_top for certification testbed (Cris) |
| emitestbed58.cnaf.infn.it |
SL5/x86_64 |
BDII_site 1.2.1 (EMI 3) |
|
BDII_site for certification testbed (Cris) |
| cream-46.pd.infn.it |
SL 5.5 Epel |
EMI Argus server 1.3.0-4 |
|
|
| cream-45.pd.infn.it |
SL 6.5 Epel |
worklog |
|
reinstalled on 22 Nov, empty |
| cream-32.pd.infn.it |
SL 6.4 Epel |
WMS EMI 3 |
|
WMS v. 3.6.0 |
User Interface
HLR Server
Computing elements: GlueCEStateStatus: Testing
EMI 2 CEs
LSF (7.0.5.125007) CEs
Torque (2.5.7-2) CEs
SPARE Virtual machine for deployment tests
| HOSTNAME |
Master host |
Assigned to |
Note |
| cert-18.pd.infn.it |
cert-14.pd.infn.it |
Cristina |
Storm 1.8.3 |
| cert-12.pd.infn.it |
cert-03 |
Sergio |
old Worklog |
Installation of RC3 fails (scratched) |
| cert-40.pd.infn.it |
cert-38 |
Sergio |
old Worklog |
old host used to certify the sensors (scratched) |
| cream-06.pd.infn.it |
cream-mstr-021.pd.infn.it |
Sara Bertocco |
--- |
| cream-48.pd.infn.it |
cream-mstr-020.pd.infn.it |
Sara Bertocco |
--- |
| cream-wn-036.pd.infn.it |
cream-mstr-024.pn.pd.infn.it |
Sara Bertocco |
--- |
| cream-wn-016.pd.infn.it |
cream-master-008.pn.pd.infn.it |
Sara Bertocco |
--- |
| cream-wn-035.pd.infn.it |
prod-wn-005.pn.pd.infn.it |
Sara Bertocco |
--- |
| cream-wn-042.pd.infn.it |
cream-mstr-027.pn.pd.infn.it |
Sara Bertocco |
--- |
| cream-wn-043.pd.infn.it |
cream-mstr-027.pn.pd.infn.it |
Sara Bertocco |
--- |
EMI Integration Testbed INFN
EMI Platform Release Testing
https://twiki.cern.ch/twiki/bin/view/EMI/EMITestbedInventory#INFN
EMI pre EMI Production
EMI-1 Production
Logbook at page:
https://twiki.cern.ch/twiki/bin/view/EMI/EMITestbedInventory#INFN_AN2
EMI-2 Production + RC (SL6 + DEBIAN)
Logbooks at page:
https://twiki.cern.ch/twiki/bin/view/EMI/EMITestbedInventory#INFN_AN1
EMI 3 RC INFN
The following VMs belong to domus13.cnaf.infn.it dom0 physical machine.
| |
Service Deployed |
Platform |
Server Hostname |
Service Status |
PT |
Additional information |
| 5 |
EMI 3 RC WMS |
SL5/x86_64 |
emitestbed41.cnaf.infn.it |
- |
WMS |
V. 3.6.0-1 |
| 6 |
EMI 3 RC WMS |
SL6/x86_64 |
emitestbed44.cnaf.infn.it |
- |
WMS |
V. 3.6.2-1 |
| 7 |
EMI 3 RC ARGUS |
SL5/x86_64 |
emitestbed42.cnaf.infn.it |
- |
ARGUS |
|
| 8 |
EMI 3 RC ARGUS |
SL6/x86_64 |
emitestbed45.cnaf.infn.it |
- |
ARGUS |
|
| 9 |
EMI 3 RC WN |
SL6/x86_64 |
emitestbed46.cnaf.infn.it |
- |
EMI Common, CREAM - emitestbed59 |
|
| 10 |
EMI 3 RC UI |
SL6/x86_64 |
emitestbed47.cnaf.infn.it |
- |
EMI Common |
|
| 12 |
EMI 3 RC CREAM Torque |
SL6/x86_64 |
emitestbed49.cnaf.infn.it |
- |
CREAM |
WN: emitestbed55 |
| 13 |
EMI 3 RC CREAM LSF |
SL6/x86_64 |
emitestbed50.cnaf.infn.it |
- |
CREAM |
LSF server: cert-36 |
| 14 |
EMI 3 RC CREAM SGE |
SL6/x86_64 |
emitestbed51.cnaf.infn.it |
- |
CREAM |
SGE master: emitestbed22 TO BE DISMISSED |
| 15 |
JENKINS BUILD SLAVE |
SL6/x86_64 |
emitestbed52.cnaf.infn.it |
- |
|
|
| 18 |
EMI 3 RC WN |
SL5/x86_64 |
emitestbed55.cnaf.infn.it |
- |
EMI Common, CREAM - emitestbed49 |
|
| 22 |
EMI 3 RC CREAM Torque |
SL5/x86_64 |
emitestbed59.cnaf.infn.it |
- |
CREAM |
WN: emitestbed46 |
| 23 |
EMI 3 RC CREAM LSF |
SL5/x86_64 |
emitestbed60.cnaf.infn.it |
- |
CREAM |
LSF server: cert-36 |
| 24 |
EMI 3 RC CREAM SGE |
SL5/x86_64 |
emitestbed61.cnaf.infn.it |
- |
CREAM |
SGE master: emitestbed22 TO BE DISMISSED |
| 25 |
JENKINS BUILD SLAVE |
SL5/x86_64 |
emitestbed62.cnaf.infn.it |
- |
|
|
Useful Links
*
Nagios monitoring page
ods 
Home 
Site map
CEMon web
Create New Topic
Index
Search
Changes
Notifications
RSS Feed
Statistics
Preferences
Account 
Copyright © 2008-2023 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.