• Release 4.0.13-0

• Release 4.0.13-0

Certification

Certification stage

• Release 4.0.13-0
• Release 4.0.15-0

• Release 4.0.13-0
• Release 4.0.15-0

Second Functionality Test (Legacy mode):

Copy from a preproduction CE many accounting jobs info to allow the HLR to work with real load:

Copy the accounting data:

[root@cert-37 ~]# rsync -axv /var/log/cream/accounting/* cert-34:/var/log/cream/accounting/

[root@cert-37 ~]# rsync -axv /opt/lsf_7.0/work/pd_lsf70_sl4/logdir/* cert-34:/opt/lsf_7.0/work/pd_lsf70_sl4/logdir/

Change the dgas sensors file:

[root@cert-34 ~]# cat /etc/dgas/dgas_sensors.conf | egrep -v '^#|^$'
siteName="INFN-PADOVA"
lrmsType="lsf"
lsfAcctLogDir = "/opt/lsf_7.0/work/pd_lsf70_sl4/logdir/"
res_acct_bank_id="cert-43.pd.infn.it:56568:"
ignoreJobsLoggedBefore="2009-04-01"
useCEJobMap = "yes"
ceJobMapLog="/var/log/cream/accounting/blahp.log"
collectorLockFileName="/var/lock/dgas/dgas_urcollector.lock"
collectorHADLockFileName="/var/lock/dgas/dgas_urcollector_had.lock"
collectorLogFileName="/var/log/dgas/dgas_urcollector.log"
collectorBufferFileName = "/var/dgasCollectorBuffer"
mainPollInterval="5"
jobPerTimeInterval="100"
timeInterval="1"
systemLogLevel="7"
limiterWaitFor="5"
maxNumRecords="10000"
localHostNameAsCEHostName = "no"
jobsToProcess="grid"
keyList="GlueHostBenchmarkSF00,GlueHostBenchmarkSI00,GlueHostApplicationSoftwareRunTimeEnvironment: SI00MeanPerCPU,GlueHostApplicationSoftwareRunTimeEnvironment: SF00MeanPerCPU"
glueLdifFile="/var/lib/bdii/gip/ldif/static-file-Cluster.ldif"
pushdLockFileName="/var/lock/dgas/dgas_pushd.lock"
pushdLogFileName="/var/log/dgas/dgas_pushd.log"
localUserGroup2VOMap="/etc/dgas/dgas_localUserGroup2VOMap.conf"
gipDynamicTmpCEFiles = "ldap://`hostname`:2170"
qMult="1"
queuePollInterval="2"
queuePollInterval="2"
voToProcess=""
useCEHostName="cert-34.pd.infn.it"

Restart the dgas sensors process:

[root@cert-34 ~]# /etc/init.d/dgas-urcollector restart
Stopping urCollector HAD...                                [  OK  ]
Stopping dgas-urcollector-SQLite.pl:                       [  OK  ]
Starting dgas-urcollector-SQLite.pl:                       [  OK  ]
Starting urCollector HAD...                                [  OK  ]

[root@cert-34 ~]# /etc/init.d/dgas-pushd restart
Stopping dgas-pushd-SQLite.pl:/var/lock/dgas/dgas_pushd.loc[  OK  ]1286)
Starting dgas-pushd-SQLite.pl:                             [  OK  ]

In HLR Server host :

[root@cert-43 ~]# mysql -u root -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 11513
Server version: 5.1.61 Source distribution

Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

Have a look into the database:

mysql> use hlr;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

mysql> select count(*) from trans_in;
+----------+
| count(*) |
+----------+
|      2629 |
+----------+
1 row in set (0.00 sec)

• Service Guide 0.6
• Service Guide 4.0.13_1.1

Testing reports SL5

Testing Report SL6

Installation & Configuration

Following the above instruction, you can found:

• installation log SL6 here
• configuration log SL6 here

RPM checks

The rpms installed:

[root@cert-43 ~]# rpm -qa | grep dgas | sort
glite-dgas-common-4.0.13-0.centos6.x86_64
glite-dgas-hlr-clients-4.0.13-0.centos6.x86_64
glite-dgas-hlr-service-4.0.13-3.centos6.x86_64
glite-dgas-hlr-service-consumers-4.0.13-2.centos6.x86_64
yaim-dgas-5.0.0-4.noarch

Check the rpm installed put the executable, libraries and configuration file in the right palce:

[root@cert-43 ~]# rpm -qil glite-dgas-common-4.0.13-0.centos6.x86_64
Name        : glite-dgas-common            Relocations: (not relocatable)
Version     : 4.0.13                            Vendor: (none)
Release     : 0.centos6                     Build Date: Thu 11 Oct 2012 10:44:50 AM CEST
Install Date: Mon 22 Oct 2012 02:55:16 PM CEST      Build Host: dgas-dev-27.to.infn.it
Group       : System Environment/Libraries   Source RPM: glite-dgas-common-4.0.13-0.centos6.src.rpm
Size        : 1505950                          License: Apache Software License
Signature   : (none)
Packager    : ETICS
Summary     : Common set of utility libraries needed by all DGAS modules
Description :
Common set of utility libraries needed by all DGAS modules
/usr/include/glite
/usr/include/glite/dgas
/usr/include/glite/dgas/tls
/usr/include/glite/dgas/tls/socket++
/usr/include/glite/dgas/tls/socket++/errors.h
/usr/include/glite/dgas/tls/socket++/tokens.h
/usr/lib64/libglite_dgas_config.a
/usr/lib64/libglite_dgas_config.so
/usr/lib64/libglite_dgas_config.so.0
/usr/lib64/libglite_dgas_config.so.0.0.0
/usr/lib64/libglite_dgas_localSecurity.a
/usr/lib64/libglite_dgas_localSecurity.so
/usr/lib64/libglite_dgas_localSecurity.so.0
/usr/lib64/libglite_dgas_localSecurity.so.0.0.0
/usr/lib64/libglite_dgas_lock.a
/usr/lib64/libglite_dgas_lock.so
/usr/lib64/libglite_dgas_lock.so.0
/usr/lib64/libglite_dgas_lock.so.0.0.0
/usr/lib64/libglite_dgas_log.a
/usr/lib64/libglite_dgas_log.so
/usr/lib64/libglite_dgas_log.so.0
/usr/lib64/libglite_dgas_log.so.0.0.0
/usr/lib64/libglite_dgas_tls_gsisocket_pp.a
/usr/lib64/libglite_dgas_tls_gsisocket_pp.so
/usr/lib64/libglite_dgas_tls_gsisocket_pp.so.0
/usr/lib64/libglite_dgas_tls_gsisocket_pp.so.0.0.0
/usr/lib64/libglite_dgas_tls_socket_pp.a
/usr/lib64/libglite_dgas_tls_socket_pp.so
/usr/lib64/libglite_dgas_tls_socket_pp.so.0
/usr/lib64/libglite_dgas_tls_socket_pp.so.0.0.0
/usr/lib64/libglite_dgas_xmlutil.a
/usr/lib64/libglite_dgas_xmlutil.so
/usr/lib64/libglite_dgas_xmlutil.so.0
/usr/lib64/libglite_dgas_xmlutil.so.0.0.0


[root@cert-43 ~]# rpm -qil glite-dgas-hlr-clients-4.0.13-0.centos6.x86_64
Name        : glite-dgas-hlr-clients       Relocations: (not relocatable)
Version     : 4.0.13                            Vendor: (none)
Release     : 0.centos6                     Build Date: Thu 11 Oct 2012 10:45:15 AM CEST
Install Date: Mon 22 Oct 2012 02:55:44 PM CEST      Build Host: dgas-dev-27.to.infn.it
Group       : System Environment/Libraries   Source RPM: glite-dgas-hlr-clients-4.0.13-0.centos6.src.rpm
Size        : 422626                           License: Apache Software License
Signature   : (none)
Packager    : ETICS
Summary     : Emi.dgas.hlr-clients
Description :
emi.dgas.hlr-clients
/usr/bin/dgas-hlr-query
/usr/bin/dgas-ping
/usr/lib64/libglite_dgas_pingClient.a
/usr/lib64/libglite_dgas_pingClient.so
/usr/lib64/libglite_dgas_pingClient.so.0
/usr/lib64/libglite_dgas_pingClient.so.0.0.0
/usr/share/man/man1/dgas-hlr-query.1.gz
/usr/share/man/man1/dgas-ping.1.gz


[root@cert-43 ~]# rpm -qil glite-dgas-hlr-service-4.0.13-3.centos6.x86_64
Name        : glite-dgas-hlr-service       Relocations: (not relocatable)
Version     : 4.0.13                            Vendor: (none)
Release     : 3.centos6                     Build Date: Thu 11 Oct 2012 10:44:16 AM CEST
Install Date: Mon 22 Oct 2012 02:56:16 PM CEST      Build Host: dgas-dev-27.to.infn.it
Group       : System Environment/Libraries   Source RPM: glite-dgas-hlr-service-4.0.13-3.centos6.src.rpm
Size        : 7116837                          License: Apache Software License
Signature   : (none)
Packager    : ETICS
Summary     : The module containing the server backend for te Distributed Grid Accounting System
Description :
emi.dgas.hlr-service
/etc/dgas
/etc/dgas/dgas_hlr.conf.template
/etc/dgas/dgas_hlr.sql
/etc/dgas/dgas_hlr_tmp.sql
/etc/dgas/dgas_sumrecords2goc.conf.template
/etc/dgas/glite-dgas-sumrecords2goc-SumCPU.sql
/usr/lib64/libdgasAtmResourceEngine.a
/usr/lib64/libdgasAtmResourceEngine.so
/usr/lib64/libdgasAtmResourceEngine.so.0
/usr/lib64/libdgasAtmResourceEngine.so.0.0.0
/usr/lib64/libdgasAtmResourceEngine2.a
/usr/lib64/libdgasAtmResourceEngine2.so
/usr/lib64/libdgasAtmResourceEngine2.so.0
/usr/lib64/libdgasAtmResourceEngine2.so.0.0.0
/usr/lib64/libdgasEngineCmnUtl.a
/usr/lib64/libdgasEngineCmnUtl.so
/usr/lib64/libdgasEngineCmnUtl.so.0
/usr/lib64/libdgasEngineCmnUtl.so.0.0.0
/usr/lib64/libglite_dgas_atmResBankClient.a
/usr/lib64/libglite_dgas_atmResBankClient.so
/usr/lib64/libglite_dgas_atmResBankClient.so.0
/usr/lib64/libglite_dgas_atmResBankClient.so.0.0.0
/usr/lib64/libglite_dgas_atmResBankClient2.a
/usr/lib64/libglite_dgas_atmResBankClient2.so
/usr/lib64/libglite_dgas_atmResBankClient2.so.0
/usr/lib64/libglite_dgas_atmResBankClient2.so.0.0.0
/usr/lib64/libglite_dgas_dbhelper.a
/usr/lib64/libglite_dgas_dbhelper.so
/usr/lib64/libglite_dgas_dbhelper.so.0
/usr/lib64/libglite_dgas_dbhelper.so.0.0.0
/usr/lib64/libglite_dgas_hlr.a
/usr/lib64/libglite_dgas_hlr.so
/usr/lib64/libglite_dgas_hlr.so.0
/usr/lib64/libglite_dgas_hlr.so.0.0.0
/usr/lib64/libglite_dgas_hlrGenericQuery.a
/usr/lib64/libglite_dgas_hlrGenericQuery.so
/usr/lib64/libglite_dgas_hlrGenericQuery.so.0
/usr/lib64/libglite_dgas_hlrGenericQuery.so.0.0.0
/usr/lib64/libglite_dgas_hlrQTransaction.a
/usr/lib64/libglite_dgas_hlrQTransaction.so
/usr/lib64/libglite_dgas_hlrQTransaction.so.0
/usr/lib64/libglite_dgas_hlrQTransaction.so.0.0.0
/usr/lib64/libglite_dgas_hlrTransLog.a
/usr/lib64/libglite_dgas_hlrTransLog.so
/usr/lib64/libglite_dgas_hlrTransLog.so.0
/usr/lib64/libglite_dgas_hlrTransLog.so.0.0.0
/usr/lib64/libglite_dgas_serviceCommonUtils.a
/usr/lib64/libglite_dgas_serviceCommonUtils.so
/usr/lib64/libglite_dgas_serviceCommonUtils.so.0
/usr/lib64/libglite_dgas_serviceCommonUtils.so.0.0.0
/usr/libexec
/usr/libexec/dgas-hlr-checkHostCertProxy.sh
/usr/libexec/dgas-hlr-urforward
/usr/libexec/dgas-ml-client
/usr/libexec/dgasmonitor
/usr/libexec/dgasmonitor/glite-dgas-monitor-run.sh
/usr/libexec/dgasmonitor/plugins
/usr/libexec/dgasmonitor/plugins/ping_2
/usr/libexec/dgasmonitor/plugins/ping_3
/usr/libexec/dgasmonitor/plugins/ping_4
/usr/libexec/dgastests
/usr/libexec/dgastests/full.test
/usr/libexec/dgastests/glite-dgas-testrun.pl
/usr/libexec/dgastests/joinCover.pl
/usr/libexec/dgastests/level1.test
/usr/libexec/dgastests/level2.test
/usr/libexec/dgastests/sensors_full.test
/usr/libexec/dgastests/storage.test
/usr/libexec/dgastests/test.conf
/usr/libexec/dgastests/tests
/usr/libexec/dgastests/tests/UR_LRMS_lsf.records
/usr/libexec/dgastests/tests/UR_LRMS_pbs.records
/usr/libexec/dgastests/tests/addResourceAuth.sh
/usr/libexec/dgastests/tests/atmClient.sh
/usr/libexec/dgastests/tests/benchmarksTest.sh
/usr/libexec/dgastests/tests/checkTable.sh
/usr/libexec/dgastests/tests/cleanResubJob.sh
/usr/libexec/dgastests/tests/cleanUpJob.sh
/usr/libexec/dgastests/tests/createLSFRecords.sh
/usr/libexec/dgastests/tests/delResourceAuth.sh
/usr/libexec/dgastests/tests/dgas_sensors_test.conf
/usr/libexec/dgastests/tests/dgas_sensors_test.conf.lsf.ldif.sed
/usr/libexec/dgastests/tests/dgas_sensors_test.conf.lsf.sed
/usr/libexec/dgastests/tests/dgas_sensors_test.conf.template
/usr/libexec/dgastests/tests/dgas_sensors_test_poolPatterns.conf
/usr/libexec/dgastests/tests/generateJobId.sh
/usr/libexec/dgastests/tests/glideInTest.sh
/usr/libexec/dgastests/tests/glite-info-dynamic-ce.ldif.4482
/usr/libexec/dgastests/tests/grid-jobmap.template
/usr/libexec/dgastests/tests/grid-jobmap.template.lsf
/usr/libexec/dgastests/tests/gridJobFQAN.lsf.sed
/usr/libexec/dgastests/tests/gridJobFQAN.sed
/usr/libexec/dgastests/tests/gridJobGlideIn.sed
/usr/libexec/dgastests/tests/gridJobpool.sed
/usr/libexec/dgastests/tests/hlrDBExists.sh
/usr/libexec/dgastests/tests/hlrHADIsAlive.sh
/usr/libexec/dgastests/tests/hlrQmgrIsAlive.sh
/usr/libexec/dgastests/tests/hlrServerdIsAlive.sh
/usr/libexec/dgastests/tests/hlr_tmpDBExists.sh
/usr/libexec/dgastests/tests/localJobmap.sed
/usr/libexec/dgastests/tests/missingStartQtimeJob.sh
/usr/libexec/dgastests/tests/multiplePing.sh
/usr/libexec/dgastests/tests/multipleRecords.sh
/usr/libexec/dgastests/tests/mysqldPing.sh
/usr/libexec/dgastests/tests/mysqldStatus.sh
/usr/libexec/dgastests/tests/negativeWCTJob.sh
/usr/libexec/dgastests/tests/poolAccountPattern.conf.template
/usr/libexec/dgastests/tests/putLSFURindgasURBox.sh
/usr/libexec/dgastests/tests/putURindgasURBox.SQL.sh
/usr/libexec/dgastests/tests/putURindgasURBox.sh
/usr/libexec/dgastests/tests/static-test.ldif
/usr/libexec/dgastests/tests/test.lsf.UR
/usr/libexec/dgastests/tests/test.pbs.UR
/usr/libexec/dgastests/tests/test.pbs.UR.SQL
/usr/libexec/dgastests/tests/testJobMap.conf
/usr/libexec/dgastests/tests/testMinus_a.sh
/usr/libexec/dgastests/tests/testMinus_o.sh
/usr/libexec/dgastests/tests/testResubmission.sh
/usr/libexec/dgastests/tests/voOriginTest.sh
/usr/libexec/dgastests/tests/voproblemjob.sh
/usr/libexec/dgastests/tests/waitForJob.sh
/usr/libexec/dgastests/tests/wrongcputjob.sh
/usr/sbin/dgas-hlr-addadmin
/usr/sbin/dgas-hlr-addresource
/usr/sbin/dgas-hlr-archive
/usr/sbin/dgas-hlr-bdiiresimport.pl
/usr/sbin/dgas-hlr-checkqueue
/usr/sbin/dgas-hlr-dbcreate
/usr/sbin/dgas-hlr-deladmin
/usr/sbin/dgas-hlr-delresource
/usr/sbin/dgas-hlr-had.pl
/usr/sbin/dgas-hlr-listener
/usr/sbin/dgas-hlr-populateJobTransSummary
/usr/sbin/dgas-hlr-qmgr
/usr/sbin/dgas-hlr-queryadmin
/usr/sbin/dgas-hlr-queryresource
/usr/sbin/dgas-hlr-sqlexec
/usr/sbin/dgas-hlr-translatedb
/usr/sbin/dgas-hlrd
/usr/sbin/dgas-sumrecords2goc.pl



[root@cert-43 ~]# rpm -qil glite-dgas-hlr-service-consumers-4.0.13-2.centos6.x86_64
Name        : glite-dgas-hlr-service-consumers  Relocations: (not relocatable)
Version     : 4.0.13                            Vendor: (none)
Release     : 2.centos6                     Build Date: Thu 11 Oct 2012 10:42:36 AM CEST
Install Date: Mon 22 Oct 2012 02:56:19 PM CEST      Build Host: dgas-dev-27.to.infn.it
Group       : System Environment/Libraries   Source RPM: glite-dgas-hlr-service-consumers-4.0.13-2.centos6.src.rpm
Size        : 1152949                          License: Apache Software License
Signature   : (none)
Packager    : ETICS
Summary     : Emi.dgas.hlr-service-consumers
Description :
emi.dgas.hlr-service-consumers
/etc/dgas
/etc/dgas/car_aggregated_v1.0.xsd
/etc/dgas/car_v1.0.xsd
/etc/dgas/dgas_amq_consumer.conf.template
/etc/dgas/ogfur_v1.xsd.xml
/usr/lib64/libglite_dgas_AMQConsumer.a
/usr/lib64/libglite_dgas_AMQConsumer.so
/usr/lib64/libglite_dgas_AMQConsumer.so.0
/usr/lib64/libglite_dgas_AMQConsumer.so.0.0.0
/usr/lib64/libglite_dgas_legacyRecordManager.a
/usr/lib64/libglite_dgas_legacyRecordManager.so
/usr/lib64/libglite_dgas_legacyRecordManager.so.0
/usr/lib64/libglite_dgas_legacyRecordManager.so.0.0.0
/usr/libexec
/usr/libexec/dgas-AMQConsumer
/usr/libexec/dgas-legacyRecordManager
/usr/libexec/glite-dgas-ogfurParser
/usr/sbin/dgas-hlr-amq-consumer


[root@cert-43 ~]# rpm -qil yaim-dgas-5.0.0-4.noarch
Name        : yaim-dgas                    Relocations: /opt/glite 
Version     : 5.0.0                             Vendor: INFN
Release     : 4                             Build Date: Thu 14 Jun 2012 12:12:25 PM CEST
Install Date: Mon 22 Oct 2012 02:55:53 PM CEST      Build Host: igi-01.pd.infn.it
Group       : Custom/INFN-GRID              Source RPM: yaim-dgas-5.0.0-4.src.rpm
Size        : 43660                            License: LCG
Signature   : (none)
Packager    : grid-release@lists.infn.it
URL         : http://repo-pd.italiangrid.it
Summary     : DGAS YAIM module
Description :
YAIM is a generic configuration for Grid Middleware developed by EGEE.
This package provides additional customizations for DGAS profile of INFN-GRID release.
/opt/glite/yaim/defaults
/opt/glite/yaim/defaults/hlr.pre
/opt/glite/yaim/examples/siteinfo/services
/opt/glite/yaim/examples/siteinfo/services/dgas_sensors
/opt/glite/yaim/examples/siteinfo/services/hlr
/opt/glite/yaim/examples/siteinfo/services/igi-hlr
/opt/glite/yaim/functions
/opt/glite/yaim/functions/config_dgas_ce
/opt/glite/yaim/functions/config_dgas_hlr
/opt/glite/yaim/functions/config_dgas_mysql
/opt/glite/yaim/functions/utils
/opt/glite/yaim/functions/utils/manage_conf
/opt/glite/yaim/node-info.d
/opt/glite/yaim/node-info.d/dgas_sensors
/opt/glite/yaim/node-info.d/hlr
/opt/glite/yaim/node-info.d/igi-hlr

Only yaim files are in /opt/glite all other HLR server files are in EMI or linux standard places.

First Functionality Test (Legacy mode):

Try to submit some jobs to a CE and see if the records arrive to HLR :

From HLR Server : Check the dgas-hlrd status and add the resource (CE DN is needed) from dgas:

[root@cert-43 ~]# service dgas-hlrd status
/var/lock/dgas/dgas_hlrd.lock
Process HLR listener: 28239
root     28239  0.6  1.0 476772 31096 pts/0    SN   10:30   0:31 /usr//sbin/dgas-hlr-listener -c /etc/dgas/dgas_hlr.conf
The process is running.
/var/lock/dgas/dgas_hlr_qmgrd.lock
Process Batch Transaction manager: 28240
root     28240  0.0  0.0 111724  2876 pts/0    SN   10:30   0:00 /usr//sbin/dgas-hlr-qmgr -l /var/log/dgas/dgas_hlr_qmgrd.log -c /etc/dgas/dgas_hlr.conf
The process is running.
Server alive.
listener answers to ping
/var/lock/dgas/dgas_hlr_urforward.lock
Process urForward: 28241
root     28241  0.0  0.1 101560  4312 pts/0    SN   10:30   0:00 /usr//libexec/dgas-hlr-urforward -c /etc/dgas/dgas_hlr.conf -d
The process is running.


[root@cert-43 ~]# dgas-hlr-addadmin -Sa "/C=IT/O=INFN/OU=Host/L=Padova/CN=cert-34.pd.infn.it"
Record added!

From CE CREAM Host: The dgas sensors into the CE CREAM host cert-34.pd.infn.it are the production sensors release:

 
[root@cert-34 ~]# rpm -qa | grep -i dgas | sort
glite-dgas-common-4.0.13-0.centos5
glite-dgas-hlr-clients-4.0.13-0.centos5
glite-dgas-hlr-sensors-4.0.13-1.centos5
glite-dgas-hlr-sensors-producers-4.0.13-2.centos5
yaim-dgas-5.0.0-3

From UI :

[traldi@cert-25 ~]$ glite-ce-job-submit -r cert-34.pd.infn.it:8443/cream-pbs-cert -a testCreamSleep.jdl 
https://cert-34.pd.infn.it:8443/CREAM996637620
[traldi@cert-25 ~]$ glite-ce-job-status https://cert-34.pd.infn.it:8443/CREAM996637620

******  JobID=[https://cert-34.pd.infn.it:8443/CREAM996637620]
        Status        = [REALLY-RUNNING]


[traldi@cert-25 ~]$ glite-ce-job-status https://cert-34.pd.infn.it:8443/CREAM996637620

******  JobID=[https://cert-34.pd.infn.it:8443/CREAM996637620]
        Status        = [DONE-OK]
        ExitCode      = [0]

From HLR Server : Check the data

mysql> select * from trans_in;
+-----+-------------------------+------+-------------------------------------------------------------------------------+--------+------------+----------------------------------------------------+----------------------------------+---------------------+
| tid | rid                     | gid  | from_dn                                                                       | amount | tr_stamp   | dgJobId                                            | uniqueChecksum                   | accountingProcedure |
+-----+-------------------------+------+-------------------------------------------------------------------------------+--------+------------+----------------------------------------------------+----------------------------------+---------------------+
...
|  10 | cert-34.pd.infn.it:cert | NONE | /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Sergio Traldi                |      0 | 1351068262 | CREAM996637620                                     | 8d5b27b5eb5bffb531b84cb6072476f3 | grid                |
+-----+-------------------------+------+-------------------------------------------------------------------------------+--------+------------+----------------------------------------------------+----------------------------------+---------------------+

After 15 submission:

 
mysql> select count(*) from trans_in;
+----------+
| count(*) |
+----------+
|       15 |
+----------+
1 row in set (0.00 sec)

Hardware Requirements

The HLR server host should be an real or virtual host having an optimal disk access.

The suggested requirements are:

• processor with 4 or 8 cores
• 8GB RAM or better 16GB
• minimum 200 GB of space for a first level HLR. (check your database actual dimension)

If you will use a virtual host ensure you are not using virtio to access to the mysql storage DB. Please use a physic partition.

Simple test with Sensors in CREAM SL6

From CE

[root@cert-42 ~]# cat /etc/redhat-release 
Scientific Linux release 6.3 (Carbon)

[root@cert-42 ~]# rpm -qa | grep -i dgas
yaim-dgas-5.0.0-4.noarch
glite-dgas-hlr-sensors-producers-4.0.13-2.centos6.x86_64
glite-dgas-common-4.0.13-0.centos6.x86_64
glite-dgas-hlr-clients-4.0.13-0.centos6.x86_64
glite-dgas-hlr-sensors-4.0.13-0.centos6.x86_64


[root@cert-42 ~]# /opt/glite/yaim/bin/yaim -c -d 6 -s /root/siteinfo/site-info.def -n creamCE -n TORQUE_server -n TORQUE_utils -n DGAS_sensors  2>&1 | tee /root/conf_EMI1-CE_Torque.`hostname -s`.`date +%Y%m%d-%H%M%S`.log

[root@cert-42 ~]# /etc/init.d/dgas-urcollector status
root      2374  0.9  0.3 151828  8976 pts/0    S    Oct08  12:21 /usr/bin/perl -w /usr//sbin/dgas-urcollector-SQLite.pl /etc/dgas/dgas_sensors.conf
The process is running.

[root@cert-42 ~]# /etc/init.d/dgas-pushd status
Process dgas-pushd-SQLite.pl: 2388
root      2388  0.0  0.2 150528  7740 pts/0    S    Oct08   1:15 /usr/bin/perl -w /usr//sbin/dgas-pushd-SQLite.pl /etc/dgas/dgas_sensors.conf
The process is running.

From HLR server:

mysql> select * from trans_in order by tid desc limit 5;
+----------+--------------------------------------+------+--------------------------------------------------------------------------------+--------+------------+-------------------------------------------------------------+----------------------------------+---------------------+
| tid      | rid                                  | gid  | from_dn                                                                        | amount | tr_stamp   | dgJobId                                                     | uniqueChecksum                   | accountingProcedure |
+----------+--------------------------------------+------+--------------------------------------------------------------------------------+--------+------------+-------------------------------------------------------------+----------------------------------+---------------------+
| 24265834 | cert-42.pd.infn.it:cert              | NONE | CN=Sergio Traldi,L=Padova,OU=Personal Certificate,O=INFN,C=IT                  |      0 | 1349707293 | CREAM927406209                                              | 59bc75e528f390a260a5c9a14005b9b2 | grid                | 
| 24265833 | cert-42.pd.infn.it:cert              | NONE | CN=Sergio Traldi,L=Padova,OU=Personal Certificate,O=INFN,C=IT                  |      0 | 1349707293 | CREAM942376304                                              | f9d425fb01e6c41c70c3dbb445ea1ee2 | grid                | 
| 24265832 | cert-42.pd.infn.it:cert              | NONE | CN=Sergio Traldi,L=Padova,OU=Personal Certificate,O=INFN,C=IT                  |      0 | 1349707293 | CREAM947154007                                              | d97b252633482d4b6c54b7cf37d91bc7 | grid                | 
| 24265831 | cert-42.pd.infn.it:cert              | NONE | CN=Sergio Traldi,L=Padova,OU=Personal Certificate,O=INFN,C=IT                  |      0 | 1349707293 | CREAM798661272                                              | aad635177c496c63c09a79ba54b8718f | grid                | 
| 24265830 | cert-42.pd.infn.it:cert              | NONE | CN=Sergio Traldi,L=Padova,OU=Personal Certificate,O=INFN,C=IT                  |      0 | 1349707293 | CREAM250809907                                              | f652cbb16cf14bd0e6181bcc8fdbf3e5 | grid                |

Installation & Configuration

Following the above instruction, you can found:

• installation log here
• configuration log here

RPM checks

The rpms installed:

[root@cert-13 ~]# rpm -qa | grep -i dgas | sort
glite-dgas-common-4.0.13-0.sl5
glite-dgas-hlr-clients-4.0.13-0.sl5
glite-dgas-hlr-service-4.0.13-0.sl5
glite-dgas-hlr-service-consumers-4.0.13-0.sl5
glite-dgas-pa-clients-4.0.8-0.sl5
yaim-dgas-5.0.0-4

Check the rpm installed put the executable, libraries and configuration file in the right palce:

[root@cert-13 ~]# rpm -ql glite-dgas-common-4.0.13-0.sl5
/usr/include/glite
/usr/include/glite/dgas
/usr/include/glite/dgas/tls
/usr/include/glite/dgas/tls/socket++
/usr/include/glite/dgas/tls/socket++/errors.h
/usr/include/glite/dgas/tls/socket++/tokens.h
/usr/lib64/libglite_dgas_config.a
/usr/lib64/libglite_dgas_config.so
/usr/lib64/libglite_dgas_config.so.0
/usr/lib64/libglite_dgas_config.so.0.0.0
/usr/lib64/libglite_dgas_localSecurity.a
/usr/lib64/libglite_dgas_localSecurity.so
/usr/lib64/libglite_dgas_localSecurity.so.0
/usr/lib64/libglite_dgas_localSecurity.so.0.0.0
/usr/lib64/libglite_dgas_lock.a
/usr/lib64/libglite_dgas_lock.so
/usr/lib64/libglite_dgas_lock.so.0
/usr/lib64/libglite_dgas_lock.so.0.0.0
/usr/lib64/libglite_dgas_log.a
/usr/lib64/libglite_dgas_log.so
/usr/lib64/libglite_dgas_log.so.0
/usr/lib64/libglite_dgas_log.so.0.0.0
/usr/lib64/libglite_dgas_tls_gsisocket_pp.a
/usr/lib64/libglite_dgas_tls_gsisocket_pp.so
/usr/lib64/libglite_dgas_tls_gsisocket_pp.so.0
/usr/lib64/libglite_dgas_tls_gsisocket_pp.so.0.0.0
/usr/lib64/libglite_dgas_tls_socket_pp.a
/usr/lib64/libglite_dgas_tls_socket_pp.so
/usr/lib64/libglite_dgas_tls_socket_pp.so.0
/usr/lib64/libglite_dgas_tls_socket_pp.so.0.0.0
/usr/lib64/libglite_dgas_xmlutil.a
/usr/lib64/libglite_dgas_xmlutil.so
/usr/lib64/libglite_dgas_xmlutil.so.0
/usr/lib64/libglite_dgas_xmlutil.so.0.0.0


[root@cert-13 ~]# rpm -ql glite-dgas-hlr-clients-4.0.13-0.sl5
/usr/bin/dgas-hlr-query
/usr/bin/dgas-ping
/usr/lib64/libglite_dgas_pingClient.a
/usr/lib64/libglite_dgas_pingClient.so
/usr/lib64/libglite_dgas_pingClient.so.0
/usr/lib64/libglite_dgas_pingClient.so.0.0.0
/usr/share/man/man1/dgas-hlr-query.1.gz
/usr/share/man/man1/dgas-ping.1.gz


[root@cert-13 ~]# rpm -ql glite-dgas-hlr-service-4.0.13-0.sl5
/etc/dgas
/etc/dgas/dgas_hlr.conf.template
/etc/dgas/dgas_hlr.sql
/etc/dgas/dgas_hlr_tmp.sql
/etc/dgas/dgas_sumrecords2goc.conf.template
/etc/dgas/glite-dgas-sumrecords2goc-SumCPU.sql
/usr/lib64/libdgasAtmResourceEngine.a
/usr/lib64/libdgasAtmResourceEngine.so
/usr/lib64/libdgasAtmResourceEngine.so.0
/usr/lib64/libdgasAtmResourceEngine.so.0.0.0
/usr/lib64/libdgasAtmResourceEngine2.a
/usr/lib64/libdgasAtmResourceEngine2.so
/usr/lib64/libdgasAtmResourceEngine2.so.0
/usr/lib64/libdgasAtmResourceEngine2.so.0.0.0
/usr/lib64/libdgasEngineCmnUtl.a
/usr/lib64/libdgasEngineCmnUtl.so
/usr/lib64/libdgasEngineCmnUtl.so.0
/usr/lib64/libdgasEngineCmnUtl.so.0.0.0
/usr/lib64/libglite_dgas_atmResBankClient.a
/usr/lib64/libglite_dgas_atmResBankClient.so
/usr/lib64/libglite_dgas_atmResBankClient.so.0
/usr/lib64/libglite_dgas_atmResBankClient.so.0.0.0
/usr/lib64/libglite_dgas_atmResBankClient2.a
/usr/lib64/libglite_dgas_atmResBankClient2.so
/usr/lib64/libglite_dgas_atmResBankClient2.so.0
/usr/lib64/libglite_dgas_atmResBankClient2.so.0.0.0
/usr/lib64/libglite_dgas_dbhelper.a
/usr/lib64/libglite_dgas_dbhelper.so
/usr/lib64/libglite_dgas_dbhelper.so.0
/usr/lib64/libglite_dgas_dbhelper.so.0.0.0
/usr/lib64/libglite_dgas_hlr.a
/usr/lib64/libglite_dgas_hlr.so
/usr/lib64/libglite_dgas_hlr.so.0
/usr/lib64/libglite_dgas_hlr.so.0.0.0
/usr/lib64/libglite_dgas_hlrGenericQuery.a
/usr/lib64/libglite_dgas_hlrGenericQuery.so
/usr/lib64/libglite_dgas_hlrGenericQuery.so.0
/usr/lib64/libglite_dgas_hlrGenericQuery.so.0.0.0
/usr/lib64/libglite_dgas_hlrQTransaction.a
/usr/lib64/libglite_dgas_hlrQTransaction.so
/usr/lib64/libglite_dgas_hlrQTransaction.so.0
/usr/lib64/libglite_dgas_hlrQTransaction.so.0.0.0
/usr/lib64/libglite_dgas_hlrTransLog.a
/usr/lib64/libglite_dgas_hlrTransLog.so
/usr/lib64/libglite_dgas_hlrTransLog.so.0
/usr/lib64/libglite_dgas_hlrTransLog.so.0.0.0
/usr/lib64/libglite_dgas_serviceCommonUtils.a
/usr/lib64/libglite_dgas_serviceCommonUtils.so
/usr/lib64/libglite_dgas_serviceCommonUtils.so.0
/usr/lib64/libglite_dgas_serviceCommonUtils.so.0.0.0
/usr/libexec
/usr/libexec/dgas-hlr-checkHostCertProxy.sh
/usr/libexec/dgas-hlr-urforward
/usr/libexec/dgas-ml-client
/usr/libexec/dgasmonitor
/usr/libexec/dgasmonitor/glite-dgas-monitor-run.sh
/usr/libexec/dgasmonitor/plugins
/usr/libexec/dgasmonitor/plugins/ping_2
/usr/libexec/dgasmonitor/plugins/ping_3
/usr/libexec/dgasmonitor/plugins/ping_4
/usr/libexec/dgastests
/usr/libexec/dgastests/full.test
/usr/libexec/dgastests/glite-dgas-testrun.pl
/usr/libexec/dgastests/joinCover.pl
/usr/libexec/dgastests/level1.test
/usr/libexec/dgastests/level2.test
/usr/libexec/dgastests/sensors_full.test
/usr/libexec/dgastests/storage.test
/usr/libexec/dgastests/test.conf
/usr/libexec/dgastests/tests
/usr/libexec/dgastests/tests/UR_LRMS_lsf.records
/usr/libexec/dgastests/tests/UR_LRMS_pbs.records
/usr/libexec/dgastests/tests/addResourceAuth.sh
/usr/libexec/dgastests/tests/atmClient.sh
/usr/libexec/dgastests/tests/benchmarksTest.sh
/usr/libexec/dgastests/tests/checkTable.sh
/usr/libexec/dgastests/tests/cleanResubJob.sh
/usr/libexec/dgastests/tests/cleanUpJob.sh
/usr/libexec/dgastests/tests/createLSFRecords.sh
/usr/libexec/dgastests/tests/delResourceAuth.sh
/usr/libexec/dgastests/tests/dgas_sensors_test.conf
/usr/libexec/dgastests/tests/dgas_sensors_test.conf.lsf.ldif.sed
/usr/libexec/dgastests/tests/dgas_sensors_test.conf.lsf.sed
/usr/libexec/dgastests/tests/dgas_sensors_test.conf.template
/usr/libexec/dgastests/tests/dgas_sensors_test_poolPatterns.conf
/usr/libexec/dgastests/tests/generateJobId.sh
/usr/libexec/dgastests/tests/glideInTest.sh
/usr/libexec/dgastests/tests/glite-info-dynamic-ce.ldif.4482
/usr/libexec/dgastests/tests/grid-jobmap.template
/usr/libexec/dgastests/tests/grid-jobmap.template.lsf
/usr/libexec/dgastests/tests/gridJobFQAN.lsf.sed
/usr/libexec/dgastests/tests/gridJobFQAN.sed
/usr/libexec/dgastests/tests/gridJobGlideIn.sed
/usr/libexec/dgastests/tests/gridJobpool.sed
/usr/libexec/dgastests/tests/hlrDBExists.sh
/usr/libexec/dgastests/tests/hlrHADIsAlive.sh
/usr/libexec/dgastests/tests/hlrQmgrIsAlive.sh
/usr/libexec/dgastests/tests/hlrServerdIsAlive.sh
/usr/libexec/dgastests/tests/hlr_tmpDBExists.sh
/usr/libexec/dgastests/tests/localJobmap.sed
/usr/libexec/dgastests/tests/missingStartQtimeJob.sh
/usr/libexec/dgastests/tests/multiplePing.sh
/usr/libexec/dgastests/tests/multipleRecords.sh
/usr/libexec/dgastests/tests/mysqldPing.sh
/usr/libexec/dgastests/tests/mysqldStatus.sh
/usr/libexec/dgastests/tests/negativeWCTJob.sh
/usr/libexec/dgastests/tests/poolAccountPattern.conf.template
/usr/libexec/dgastests/tests/putLSFURindgasURBox.sh
/usr/libexec/dgastests/tests/putURindgasURBox.SQL.sh
/usr/libexec/dgastests/tests/putURindgasURBox.sh
/usr/libexec/dgastests/tests/static-test.ldif
/usr/libexec/dgastests/tests/test.lsf.UR
/usr/libexec/dgastests/tests/test.pbs.UR
/usr/libexec/dgastests/tests/test.pbs.UR.SQL
/usr/libexec/dgastests/tests/testJobMap.conf
/usr/libexec/dgastests/tests/testMinus_a.sh
/usr/libexec/dgastests/tests/testMinus_o.sh
/usr/libexec/dgastests/tests/testResubmission.sh
/usr/libexec/dgastests/tests/voOriginTest.sh
/usr/libexec/dgastests/tests/voproblemjob.sh
/usr/libexec/dgastests/tests/waitForJob.sh
/usr/libexec/dgastests/tests/wrongcputjob.sh
/usr/sbin/dgas-hlr-addadmin
/usr/sbin/dgas-hlr-addresource
/usr/sbin/dgas-hlr-archive
/usr/sbin/dgas-hlr-bdiiresimport.pl
/usr/sbin/dgas-hlr-checkqueue
/usr/sbin/dgas-hlr-dbcreate
/usr/sbin/dgas-hlr-deladmin
/usr/sbin/dgas-hlr-delresource
/usr/sbin/dgas-hlr-had.pl
/usr/sbin/dgas-hlr-listener
/usr/sbin/dgas-hlr-populateJobTransSummary
/usr/sbin/dgas-hlr-qmgr
/usr/sbin/dgas-hlr-queryadmin
/usr/sbin/dgas-hlr-queryresource
/usr/sbin/dgas-hlr-sqlexec
/usr/sbin/dgas-hlr-translatedb
/usr/sbin/dgas-hlrd
/usr/sbin/dgas-sumrecords2goc.pl



[root@cert-13 ~]# rpm -ql glite-dgas-hlr-service-consumers-4.0.13-0.sl5
/etc/dgas
/etc/dgas/car_aggregated_v1.0.xsd
/etc/dgas/car_v1.0.xsd
/etc/dgas/dgas_amq_consumer.conf.template
/etc/dgas/ogfur_v1.xsd.xml
/usr/lib64/libglite_dgas_AMQConsumer.a
/usr/lib64/libglite_dgas_AMQConsumer.so
/usr/lib64/libglite_dgas_AMQConsumer.so.0
/usr/lib64/libglite_dgas_AMQConsumer.so.0.0.0
/usr/lib64/libglite_dgas_legacyRecordManager.a
/usr/lib64/libglite_dgas_legacyRecordManager.so
/usr/lib64/libglite_dgas_legacyRecordManager.so.0
/usr/lib64/libglite_dgas_legacyRecordManager.so.0.0.0
/usr/libexec
/usr/libexec/dgas-AMQConsumer
/usr/libexec/dgas-legacyRecordManager
/usr/libexec/glite-dgas-ogfurParser
/usr/sbin/dgas-hlr-amq-consumer


[root@cert-13 ~]# rpm -ql glite-dgas-pa-clients-4.0.8-0.sl5
/usr/bin/dgas-pa-info
/usr/lib64/libglite_dgas_paClient.a
/usr/lib64/libglite_dgas_paClient.so
/usr/lib64/libglite_dgas_paClient.so.0
/usr/lib64/libglite_dgas_paClient.so.0.0.0



[root@cert-13 ~]# rpm -ql yaim-dgas-5.0.0-4
/opt/glite/yaim/defaults
/opt/glite/yaim/defaults/hlr.pre
/opt/glite/yaim/examples/siteinfo/services
/opt/glite/yaim/examples/siteinfo/services/dgas_sensors
/opt/glite/yaim/examples/siteinfo/services/hlr
/opt/glite/yaim/examples/siteinfo/services/igi-hlr
/opt/glite/yaim/functions
/opt/glite/yaim/functions/config_dgas_ce
/opt/glite/yaim/functions/config_dgas_hlr
/opt/glite/yaim/functions/config_dgas_mysql
/opt/glite/yaim/functions/utils
/opt/glite/yaim/functions/utils/manage_conf
/opt/glite/yaim/node-info.d
/opt/glite/yaim/node-info.d/dgas_sensors
/opt/glite/yaim/node-info.d/hlr
/opt/glite/yaim/node-info.d/igi-hlr

Only yaim files are in /opt/glite all other HLR server files are in EMI or linux standard places.

First Functionality Test (Legacy mode):

Try to submit some jobs to a CE and see if the records arrive to HLR :

From HLR Server : Check the dgas-hlrd status and add the resource (CE DN is needed) from dgas:

[root@cert-13 ~]# /etc/init.d/dgas-hlrd status
/var/lock/dgas/dgas_hlrd.lock
Process HLR listener: 13711
root     13711  0.2  1.5 245492 49420 ?        S    May28   9:32 /usr//sbin/dgas-hlr-listener -c /etc/dgas/dgas_hlr.conf
The process is running.
/var/lock/dgas/dgas_hlr_qmgrd.lock
Process Batch Transaction manager: 13712
root     13712  0.0  0.0 112424  2672 ?        S    May28   1:11 /usr//sbin/dgas-hlr-qmgr -l /var/log/dgas/dgas_hlr_qmgrd.log -c /etc/dgas/dgas_hlr.conf
The process is running.
Server alive.
listener answers to ping
/var/lock/dgas/dgas_hlr_urforward.lock
Process urForward: 13713
root     13713  0.0  0.1 100460  4228 ?        S    May28   0:00 /usr//libexec/dgas-hlr-urforward -c /etc/dgas/dgas_hlr.conf -d
The process is running.


[root@cert-13 ~]# dgas-hlr-addadmin -Sa "/C=IT/O=INFN/OU=Host/L=Padova/CN=cert-34.pd.infn.it"
Record added!

From CE CREAM Host: The dgas sensors into the CE CREAM host cert-34.pd.infn.it are the production sensors release:

 
[root@cert-34 ~]# rpm -qa | grep -i dgas | sort
glite-dgas-common-4.0.6-0.sl5
glite-dgas-hlr-clients-4.0.6-0.sl5
glite-dgas-hlr-sensors-4.0.6-2.sl5
glite-dgas-hlr-sensors-producers-4.0.6-1.sl5
yaim-dgas-5.0.0-3

From UI :

-bash-3.2$ glite-ce-job-submit -r cert-34.pd.infn.it:8443/cream-pbs-cert -a testCream1.jdl
https://cert-34.pd.infn.it:8443/CREAM262009988


-bash-3.2$ glite-ce-job-status https://cert-34.pd.infn.it:8443/CREAM262009988

******  JobID=[https://cert-34.pd.infn.it:8443/CREAM262009988]
        Status        = [REALLY-RUNNING]


-bash-3.2$ glite-ce-job-status https://cert-34.pd.infn.it:8443/CREAM262009988

******  JobID=[https://cert-34.pd.infn.it:8443/CREAM262009988]
        Status        = [DONE-OK]
        ExitCode      = [0]

From HLR Server : Check the data

mysql> select * from trans_in;
+-----+-------------------------+------+----------------------------------------------------------------+--------+------------+----------------+----------------------------------+---------------------+
| tid | rid                     | gid  | from_dn                                                        | amount | tr_stamp   | dgJobId        | uniqueChecksum                   | accountingProcedure |
+-----+-------------------------+------+----------------------------------------------------------------+--------+------------+----------------+----------------------------------+---------------------+
|   1 | cert-34.pd.infn.it:cert | NONE | /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Sergio Traldi |      0 | 1337759000 | CREAM262009988 | 900706bf5be9e7156c56cd5c27a17d3a | grid                |
+-----+-------------------------+------+----------------------------------------------------------------+--------+------------+----------------+----------------------------------+---------------------+
1 row in set (0.00 sec)

Second Functionality Test (Legacy mode):

Copy from a preproduction CE many accounting jobs info to allow the HLR to work with real load:

Copy the accounting data:

[root@cert-37 ~]# rsync -axv /var/log/cream/accounting/* cert-34:/var/log/cream/accounting/

[root@cert-37 ~]# rsync -axv /opt/lsf_7.0/work/pd_lsf70_sl4/logdir/* cert-34:/opt/lsf_7.0/work/pd_lsf70_sl4/logdir/

Change the dgas sensors file:

[root@cert-34 ~]# cat /etc/dgas/dgas_sensors.conf | egrep -v '^#|^$'
siteName="INFN-PADOVA"
lrmsType="lsf"
lsfAcctLogDir = "/opt/lsf_7.0/work/pd_lsf70_sl4/logdir/"
res_acct_bank_id="cert-13.pd.infn.it:56568:"
ignoreJobsLoggedBefore="2009-04-01"
useCEJobMap = "yes"
ceJobMapLog="/var/log/cream/accounting/blahp.log"
collectorLockFileName="/var/lock/dgas/dgas_urcollector.lock"
collectorHADLockFileName="/var/lock/dgas/dgas_urcollector_had.lock"
collectorLogFileName="/var/log/dgas/dgas_urcollector.log"
collectorBufferFileName = "/var/dgasCollectorBuffer"
mainPollInterval="5"
jobPerTimeInterval="100"
timeInterval="1"
systemLogLevel="7"
limiterWaitFor="5"
maxNumRecords="10000"
localHostNameAsCEHostName = "no"
jobsToProcess="grid"
keyList="GlueHostBenchmarkSF00,GlueHostBenchmarkSI00,GlueHostApplicationSoftwareRunTimeEnvironment: SI00MeanPerCPU,GlueHostApplicationSoftwareRunTimeEnvironment: SF00MeanPerCPU"
glueLdifFile="/var/lib/bdii/gip/ldif/static-file-Cluster.ldif"
pushdLockFileName="/var/lock/dgas/dgas_pushd.lock"
pushdLogFileName="/var/log/dgas/dgas_pushd.log"
localUserGroup2VOMap="/etc/dgas/dgas_localUserGroup2VOMap.conf"
gipDynamicTmpCEFiles = "ldap://`hostname`:2170"
qMult="1"
queuePollInterval="2"
queuePollInterval="2"
voToProcess=""
useCEHostName="cert-34.pd.infn.it"

Restart the dgas sensors process:

[root@cert-34 ~]# /etc/init.d/dgas-urcollector restart
Stopping urCollector HAD...                                [  OK  ]
Stopping dgas-urcollector-SQLite.pl:                       [  OK  ]
Starting dgas-urcollector-SQLite.pl:                       [  OK  ]
Starting urCollector HAD...                                [  OK  ]

[root@cert-34 ~]# /etc/init.d/dgas-pushd restart
Stopping dgas-pushd-SQLite.pl:/var/lock/dgas/dgas_pushd.loc[  OK  ]1286)
Starting dgas-pushd-SQLite.pl:                             [  OK  ]

In HLR Server host : Have a look into the database:

mysql> select count(*) from trans_in;
+----------+
| count(*) |
+----------+
|      4376|
+----------+
1 row in set (0.00 sec)


mysql> select count(*) from trans_in;
+----------+
| count(*) |
+----------+
|   193376 |
+----------+
1 row in set (0.00 sec)


mysql> select * from trans_in order by tid desc limit 10;
+--------+-------------------------+------+--------------------------------------------------------------------+--------+------------+-----------------------------------------------------------+----------------------------------+---------------------+
| tid    | rid                     | gid  | from_dn
| amount | tr_stamp   | dgJobId
| uniqueChecksum                   | accountingProcedure |
+--------+-------------------------+------+--------------------------------------------------------------------+--------+------------+-----------------------------------------------------------+----------------------------------+---------------------+
| 193376 | cert-34.pd.infn.it:grid | NONE |
/O=dutchgrid/O=users/O=universiteit-utrecht/OU=chem/CN=Tobias Madl |      0 |
1337967306 | https://lb-enmr.science.uu.nl:9000/Zj_4hAmYDB1oz8xIUXb_6g |
b6efae1ad1dd78e72ddd082497aac157 | grid                |
| 193375 | cert-34.pd.infn.it:grid | NONE |
/O=dutchgrid/O=users/O=universiteit-utrecht/OU=chem/CN=Tobias Madl |      0 |
1337967306 | https://lb-enmr.science.uu.nl:9000/eMb7bbYFkg0_OKy0-yZXhg |
1a76f03292ac1f77a3603a5034174ecd | grid                |
| 193374 | cert-34.pd.infn.it:grid | NONE |
/O=dutchgrid/O=users/O=universiteit-utrecht/OU=chem/CN=Tobias Madl |      0 |
1337967305 | https://lb-enmr.science.uu.nl:9000/jQdyFL_gB-jvG5J37q0N3g |
f66541cd5ddfbb50213556c15f327e68 | grid                |
| 193373 | cert-34.pd.infn.it:grid | NONE |
/O=dutchgrid/O=users/O=universiteit-utrecht/OU=chem/CN=Tobias Madl |      0 |
1337967306 | https://lb-enmr.science.uu.nl:9000/0MP62id1sjtThxsvBmhIgg |
a71a28e4dcdffefc9b98341f08e69bec | grid                |
| 193372 | cert-34.pd.infn.it:grid | NONE |
/O=dutchgrid/O=users/O=universiteit-utrecht/OU=chem/CN=Tobias Madl |      0 |
1337967306 | https://lb-enmr.science.uu.nl:9000/Xgu6gjzpQzmhyPXxASQ8sg |
03fa418be9cbc708c13019fb37ae1be2 | grid                |
| 193371 | cert-34.pd.infn.it:grid | NONE |
/O=dutchgrid/O=users/O=universiteit-utrecht/OU=chem/CN=Tobias Madl |      0 |
1337967306 | https://lb-enmr.science.uu.nl:9000/R6KN9pKnFLiTT3FVWPI9Uw |
b1df5764c854cd5912c62cdaa3778608 | grid                |
| 193370 | cert-34.pd.infn.it:grid | NONE |
/O=dutchgrid/O=users/O=universiteit-utrecht/OU=chem/CN=Tobias Madl |      0 |
1337967306 | https://lb-enmr.science.uu.nl:9000/SnvtupymmS6OvUM0M5hx9g |
2231f6d83953b933f8a6bc4117eb5e47 | grid                |
| 193369 | cert-34.pd.infn.it:grid | NONE |
/O=dutchgrid/O=users/O=universiteit-utrecht/OU=chem/CN=Tobias Madl |      0 |
1337967306 | https://lb-enmr.science.uu.nl:9000/YVtkXTignrfq0x2EwwRCVg |
31a3efda407af8307b5f05c64ff07a07 | grid                |
| 193368 | cert-34.pd.infn.it:grid | NONE |
/O=dutchgrid/O=users/O=universiteit-utrecht/OU=chem/CN=Tobias Madl |      0 |
1337967306 | https://lb-enmr.science.uu.nl:9000/fbRhTJoMDmhOfg-hx49v7Q |
4a3631c980804674f97f6a25c55db164 | grid                |
| 193367 | cert-34.pd.infn.it:grid | NONE |
/O=dutchgrid/O=users/O=universiteit-utrecht/OU=chem/CN=Tobias Madl |      0 |
1337967306 | https://lb-enmr.science.uu.nl:9000/6vlZ5r8XWhuUdSWzvLPgFQ |
a3ef4800428662ee5dff027efc323be3 | grid                |
+--------+-------------------------+------+--------------------------------------------------------------------+--------+------------+-----------------------------------------------------------+----------------------------------+---------------------+
10 rows in set (0.00 sec)

Update Test:

Back up the old databases :

Form previous HLR Server stop the dgas services and make a dump of the databases:

Stop the services :

[root@prod-hlr-01 ~]# /etc/init.d/glite-dgas-hlrd stop
Stopping HAD...                                            [  OK  ]
Stopping HLR listener:/opt/glite/var/dgas/hlr.lock (pid 172[  OK  ]
Stopping Batch Transaction manager:/opt/glite/var/dgas/hlr_[  OK  ]ck (pid 17204)
Stopping urForward:/opt/glite/var/dgas/glite-dgas-ur-forwar[  OK  ]pid 17206)

[root@prod-hlr-01 ~]# ps -ef | grep -i dgas
root      2006 31449  0 11:22 pts/2    00:00:00 grep -i dgas

Check disk space Ensure you have enough space in the disk:

[root@prod-hlr-01 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/md0               29G  3.5G   24G  13% /
/dev/md1              494M   17M  452M   4% /boot
none                  1.5G     0  1.5G   0% /dev/shm
/dev/md3               81G   73G  7.8G  91% /var/lib/mysql
prod-mon-01:/data
                      228G   2G  227G 1% /data

Make the dump :

mysqldump --user=root --password hlr_Padova > /data/hlr.sql
mysqldump --user=root --password hlr_tmp_Padova > /data/hlr_tmp.sql

Installation : Install again the host with SL5 x86_64 SL distribution or install a new host where the HLR Server will be installed.

Following the above instructions install the HLR Server and configure using yaim.

Stop the HLR server process:

[root@cert-13 ~]# /etc/init.d/dgas-hlrd stop
chown: `dgas:dgas': invalid user
Stopping HAD...                                            [  OK  ]
Stopping HLR listener:/var/lock/dgas/dgas_hlrd.lock (pid 23[  OK  ]
Stopping Batch Transaction manager:/var/lock/dgas/dgas_hlr_qmgrd.lock (pid 23264)                                                          [  OK  ]
Stopping urForward:/var/lock/dgas/dgas_hlr_urforward.lock ([  OK  ]5)

[root@cert-13 ~]# ps -ef | grep -i dgas
root     23431 22263  0 11:34 pts/0    00:00:00 grep -i dgas

Restore the dump :

[root@cert-13 ~]# mysql -u root -p hlr_tmp < /data/hlr_tmp.sql
Enter password:

[root@cert-13 ~]# mysql -u root -p hlr < /data/hlr.sql
Enter password: 

check dgas services

[root@cert-13 ~]# ps -ef | grep dgas
root     25194  2324  0 11:43 pts/0    00:00:00 grep dgas

execute translate DB

[root@cert-13 ~]# nohup /usr/sbin/dgas-hlr-translatedb --D &
[1] 25195
[root@cert-13 ~]# nohup: appending output to `nohup.out'
[1]+  Done                    nohup /usr/sbin/dgas-hlr-translatedb --D

Start dgas services :

[root@cert-13 ~]# /etc/init.d/dgas-hlrd start
chown: `dgas:dgas': invalid user
Initializing DGAS HLR....Locking other instances out.
Initializing database, this operation can take minutes to hours to complete depending on the size of your DB. DO NOT KILL THE COMMAND PLEASE.
Starting HLR listener:                                     [  OK  ]
Starting Batch Transaction manager:                        [  OK  ]
Starting urForward:                                        [  OK  ]
Starting DGAS HAD...                                       [  OK  ]

Check dgas services :

[root@cert-13 ~]# ps -ef | grep dgas
root     13910     1  0 09:46 pts/0    00:00:00 /usr//sbin/dgas-hlr-listener -c /etc/dgas/dgas_hlr.conf
root     13911     1  0 09:46 pts/0    00:00:00 /usr//sbin/dgas-hlr-qmgr -l /var/log/dgas/dgas_hlr_qmgrd.log -c /etc/dgas/dgas_hlr.conf
root     13912     1  0 09:46 pts/0    00:00:00 /usr//libexec/dgas-hlr-urforward -c /etc/dgas/dgas_hlr.conf -d
root     13913     1  0 09:46 pts/0    00:00:00 /usr/bin/perl -w /usr//sbin/dgas-hlr-had.pl /etc/rc.d/init.d/dgas-hlrd /usr//libexec/dgas-hlr-checkHostCertProxy.sh
root     13922  2324  0 09:46 pts/0    00:00:00 grep dgas

Check data :

[root@cert-13 ~]# mysql -u root -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 272404
Server version: 5.0.95 Source distribution

Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| hlr                |
| hlr_tmp            |
| mysql              |
| test               |
+--------------------+
5 rows in set (0.07 sec)

mysql> use hlr;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select count(*) from jobTransSummary;
+----------+
| count(*) |
+----------+
|        0 |
+----------+
1 row in set (0.00 sec)

mysql> select count(*) from trans_in;
+----------+
| count(*) |
+----------+
| 22257636 |
+----------+
1 row in set (0.00 sec)

mysql> exit
Bye

Execute populateJobTransSummary:

[root@cert-13 ~]# /usr/sbin/dgas-hlr-populateJobTransSummary
Locking other instances out.
 writing records: ######################################## [  0%] E:905 sec ETA:0 min
 writing records: ######################################## [  0%] E:617 sec ETA:0 min
 writing records: ######################################## [  1%] E:769 sec ETA:1268 min
 writing records: ######################################## [  1%] E:835 sec ETA:1268 min
 writing records: ######################################## [  2%] E:663 sec ETA:1082 min
 writing records: ######################################## [  2%] E:559 sec ETA:1082 min
 writing records: ######################################## [  3%] E:427 sec ETA:690 min
 writing records: ######################################## [  3%] E:609 sec ETA:690 min
 writing records: ######################################## [  4%] E:844 sec ETA:1350 min
 writing records: ########EE#EEEEEEE#EE#######EEE######### [  4%] E:692 sec ETA:1350 min
 writing records: ###############EEE##EEE################# [  4%] E:657 sec ETA:1350 min
 writing records: ######################################## [  5%] E:612 sec ETA:969 min
 writing records: ######################################## [  5%] E:306 sec ETA:969 min
 writing records: #################################EEEE#EE [  6%] E:494 sec ETA:773 min
 writing records: EE###E#E#######E######################## [  6%] E:482 sec ETA:773 min
 writing records: ######################################## [  7%] E:536 sec ETA:830 min
 writing records: ######################################## [  7%] E:906 sec ETA:830 min
 writing records: ######################################## [  8%] E:1972 sec ETA:3023 min
 writing records: ######################################## [  8%] E:1504 sec ETA:3023 min
 writing records: ######################################## [  9%] E:1261 sec ETA:1912 min
 writing records: ######################################## [  9%] E:387 sec ETA:1912 min
 writing records: ######################################## [  9%] E:265 sec ETA:1912 min
 writing records: ######################################## [ 10%] E:381 sec ETA:571 min
 writing records: ######################################## [ 10%] E:531 sec ETA:571 min
 writing records: ######################################## [ 11%] E:1197 sec ETA:1775 min
 writing records: ######################################## [ 11%] E:605 sec ETA:1775 min
 writing records: ######################################## [ 12%] E:666 sec ETA:976 min
 writing records: ######################################## [ 12%] E:509 sec ETA:976 min
 writing records: ######################################## [ 13%] E:1272 sec ETA:1844 min
 writing records: ######################################## [ 13%] E:1006 sec ETA:1844 min
 writing records: ######################################## [ 13%] E:1223 sec ETA:1844 min
 writing records: ######################################## [ 14%] E:1239 sec ETA:1775 min
 writing records: ######################################## [ 14%] E:748 sec ETA:1775 min
 writing records: ######################################## [ 15%] E:1107 sec ETA:1568 min
 writing records: ######################################## [ 15%] E:817 sec ETA:1568 min
 writing records: ######################################## [ 16%] E:790 sec ETA:1106 min
 writing records: ######################################## [ 16%] E:959 sec ETA:1106 min
 writing records: ######################################## [ 17%] E:1071 sec ETA:1481 min
 writing records: ######################################## [ 17%] E:1608 sec ETA:1481 min
 writing records: ######################################## [ 18%] E:1534 sec ETA:2096 min
 writing records: ######################################## [ 18%] E:1609 sec ETA:2096 min
 writing records: ######################################## [ 18%] E:2089 sec ETA:2096 min
 writing records: ######################################## [ 19%] E:2449 sec ETA:3306 min
 writing records: ######################################## [ 19%] E:2100 sec ETA:3306 min
 writing records: ######################################## [ 20%] E:1890 sec ETA:2520 min
 writing records: ######################################## [ 20%] E:1075 sec ETA:2520 min
 writing records: ######################################## [ 21%] E:685 sec ETA:901 min
 writing records: ######################################## [ 21%] E:4486 sec ETA:901 min
 writing records: ######################################## [ 22%] E:1509 sec ETA:1961 min
 writing records: ######################################## [ 22%] E:1192 sec ETA:1961 min
 writing records: ######################################## [ 22%] E:1836 sec ETA:1961 min
 writing records: ######################################## [ 23%] E:2283 sec ETA:2929 min
 writing records: ######################################## [ 23%] E:2029 sec ETA:2929 min
 writing records: ######################################## [ 24%] E:3468 sec ETA:4392 min
 writing records: ######################################## [ 24%] E:1370 sec ETA:4392 min
 writing records: ######################################## [ 25%] E:1687 sec ETA:2108 min
 writing records: ######################################## [ 25%] E:2564 sec ETA:2108 min
 writing records: ######################################## [ 26%] E:1066 sec ETA:1314 min
 writing records: ######################################## [ 26%] E:1204 sec ETA:1314 min
 writing records: ######################################## [ 27%] E:2223 sec ETA:2704 min
 writing records: ######################################## [ 27%] E:1339 sec ETA:2704 min
 writing records: ######################################## [ 27%] E:387 sec ETA:2704 min
 writing records: ######################################## [ 28%] E:989 sec ETA:1186 min
 writing records: ######################################## [ 28%] E:961 sec ETA:1186 min
 writing records: ######################################## [ 29%] E:1470 sec ETA:1739 min
 writing records: ######################################## [ 29%] E:1535 sec ETA:1739 min
 writing records: ######################################## [ 30%] E:1394 sec ETA:1626 min
 writing records: ######################################## [ 30%] E:1632 sec ETA:1626 min
 writing records: ######################################## [ 31%] E:1855 sec ETA:2133 min
 writing records: ######################################## [ 31%] E:1613 sec ETA:2133 min
 writing records: ######################################## [ 31%] E:1903 sec ETA:2133 min
 writing records: ######################################## [ 32%] E:1264 sec ETA:1432 min
 writing records: ######################################## [ 32%] E:1637 sec ETA:1432 min
 writing records: ######################################## [ 33%] E:1628 sec ETA:1817 min
 writing records: ######################################## [ 33%] E:1463 sec ETA:1817 min
 writing records: ######################################## [ 34%] E:1030 sec ETA:1133 min
 writing records: ######################################## [ 34%] E:1115 sec ETA:1133 min
 writing records: ######################################## [ 35%] E:1263 sec ETA:1368 min
 writing records: ######################################## [ 35%] E:1362 sec ETA:1368 min
 writing records: ######################################## [ 36%] E:1223 sec ETA:1304 min
 writing records: ######################################## [ 36%] E:614 sec ETA:1304 min
 writing records: ######################################## [ 36%] E:522 sec ETA:1304 min
 writing records: ######################################## [ 37%] E:695 sec ETA:729 min
 writing records: ######################################## [ 37%] E:519 sec ETA:729 min
 writing records: ######################################## [ 38%] E:545 sec ETA:563 min
 writing records: ######################################## [ 38%] E:729 sec ETA:563 min
 writing records: ######################################## [ 39%] E:442 sec ETA:449 min
 writing records: ######################################## [ 39%] E:366 sec ETA:449 min
 writing records: ######################################## [ 40%] E:412 sec ETA:412 min
 writing records: ######################################## [ 40%] E:406 sec ETA:412 min
 writing records: ######################################## [ 40%] E:340 sec ETA:412 min
 writing records: ######################################## [ 41%] E:369 sec ETA:362 min
 writing records: ######################################## [ 41%] E:374 sec ETA:362 min
 writing records: ######################################## [ 42%] E:582 sec ETA:562 min
 writing records: ######################################## [ 42%] E:846 sec ETA:562 min
 writing records: ######################################## [ 43%] E:876 sec ETA:832 min
 writing records: ######################################## [ 43%] E:1983 sec ETA:832 min
 writing records: ######################################## [ 44%] E:1616 sec ETA:1508 min
 writing records: ######################################## [ 44%] E:1516 sec ETA:1508 min
 writing records: ######################################## [ 45%] E:1985 sec ETA:1819 min
 writing records: ######################################## [ 45%] E:1580 sec ETA:1819 min
 writing records: ######################################## [ 45%] E:1593 sec ETA:1819 min
 writing records: ######################################## [ 46%] E:1641 sec ETA:1476 min
 writing records: ######################################## [ 46%] E:1937 sec ETA:1476 min
 writing records: ######################################## [ 47%] E:1736 sec ETA:1533 min
 writing records: ######################################## [ 47%] E:1714 sec ETA:1533 min
 writing records: ######################################## [ 48%] E:1652 sec ETA:1431 min
 writing records: ######################################## [ 48%] E:1377 sec ETA:1431 min
 writing records: ######################################## [ 49%] E:1129 sec ETA:959 min
 writing records: ######################################## [ 49%] E:894 sec ETA:959 min
 writing records: ######################################## [ 50%] E:1122 sec ETA:935 min
 writing records: ######################################## [ 50%] E:783 sec ETA:935 min
 writing records: ######################################## [ 50%] E:1018 sec ETA:935 min
 writing records: ######################################## [ 51%] E:759 sec ETA:619 min
 writing records: ######################################## [ 51%] E:562 sec ETA:619 min
 writing records: ######################################## [ 52%] E:634 sec ETA:507 min
 writing records: ######################################## [ 52%] E:633 sec ETA:507 min
 writing records: ######################################## [ 53%] E:633 sec ETA:495 min
 writing records: ######################################## [ 53%] E:637 sec ETA:495 min
 writing records: ######################################## [ 54%] E:793 sec ETA:607 min
 writing records: ######################################## [ 54%] E:793 sec ETA:607 min
 writing records: ######################################## [ 54%] E:708 sec ETA:607 min
 writing records: ######################################## [ 55%] E:569 sec ETA:426 min
 writing records: ######################################## [ 55%] E:577 sec ETA:426 min
 writing records: ######################################## [ 56%] E:1979 sec ETA:1451 min
 writing records: ######################################## [ 56%] E:1839 sec ETA:1451 min
 writing records: ######################################## [ 57%] E:1822 sec ETA:1305 min
 writing records: ######################################## [ 57%] E:1635 sec ETA:1305 min
 writing records: ######################################## [ 58%] E:1637 sec ETA:1145 min
 writing records: ######################################## [ 58%] E:1738 sec ETA:1145 min
 writing records: ######################################## [ 59%] E:2376 sec ETA:1623 min
 writing records: ######################################## [ 59%] E:2497 sec ETA:1623 min
 writing records: ######################################## [ 59%] E:1929 sec ETA:1623 min
 writing records: ######################################## [ 60%] E:1642 sec ETA:1094 min
 writing records: ######################################## [ 60%] E:1536 sec ETA:1094 min
 writing records: ######################################## [ 61%] E:2099 sec ETA:1364 min
 writing records: ######################################## [ 61%] E:2539 sec ETA:1364 min
 writing records: ######################################## [ 62%] E:2272 sec ETA:1438 min
 writing records: ######################################## [ 62%] E:2033 sec ETA:1438 min
 writing records: ######################################## [ 63%] E:1723 sec ETA:1062 min
 writing records: ######################################## [ 63%] E:1434 sec ETA:1062 min
 writing records: ######################################## [ 63%] E:1758 sec ETA:1062 min
 writing records: ######################################## [ 64%] E:1438 sec ETA:862 min
 writing records: ######################################## [ 64%] E:1150 sec ETA:862 min
 writing records: ######################################## [ 65%] E:1011 sec ETA:589 min
 writing records: ######################################## [ 65%] E:1008 sec ETA:589 min
 writing records: ######################################## [ 66%] E:978 sec ETA:554 min
 writing records: ######################################## [ 66%] E:911 sec ETA:554 min
 writing records: ######################################## [ 67%] E:946 sec ETA:520 min
 writing records: ######################################## [ 67%] E:1560 sec ETA:520 min
 writing records: #######E################################ [ 68%] E:2546 sec ETA:1357 min
 writing records: ######################################## [ 68%] E:1591 sec ETA:1357 min
 writing records: ######################################## [ 68%] E:1051 sec ETA:1357 min
 writing records: ######################################## [ 69%] E:929 sec ETA:479 min
 writing records: ######################################## [ 69%] E:914 sec ETA:479 min
 writing records: ######################################## [ 70%] E:816 sec ETA:408 min
 writing records: ######################################## [ 70%] E:857 sec ETA:408 min
 writing records: ######################################## [ 71%] E:820 sec ETA:396 min
 writing records: ######################################## [ 71%] E:884 sec ETA:396 min
 writing records: ######################################## [ 72%] E:868 sec ETA:405 min
 writing records: ######################################## [ 72%] E:1524 sec ETA:405 min
 writing records: ######################################## [ 72%] E:1123 sec ETA:405 min
 writing records: ######################################## [ 73%] E:1168 sec ETA:525 min
 writing records: ######################################## [ 73%] E:1068 sec ETA:525 min
 writing records: ######################################## [ 74%] E:866 sec ETA:375 min
 writing records: ######################################## [ 74%] E:1088 sec ETA:375 min
 writing records: ######################################## [ 75%] E:748 sec ETA:311 min
 writing records: ######################################## [ 75%] E:977 sec ETA:311 min
 writing records: ######################################## [ 76%] E:1100 sec ETA:440 min
 writing records: ######################################## [ 76%] E:1083 sec ETA:440 min
 writing records: ######################################## [ 77%] E:1690 sec ETA:647 min
 writing records: ######################################## [ 77%] E:1377 sec ETA:647 min
 writing records: ######################################## [ 77%] E:1082 sec ETA:647 min
 writing records: ######################################## [ 78%] E:1368 sec ETA:501 min
 writing records: ######################################## [ 78%] E:946 sec ETA:501 min
 writing records: ######################################## [ 79%] E:1195 sec ETA:418 min
 writing records: ######################################## [ 79%] E:923 sec ETA:418 min
 writing records: ######################################## [ 80%] E:1196 sec ETA:398 min
 writing records: ######################################## [ 80%] E:1865 sec ETA:398 min
 writing records: ######################################## [ 81%] E:2092 sec ETA:662 min
 writing records: ######################################## [ 81%] E:1465 sec ETA:662 min
 writing records: ######################################## [ 81%] E:1029 sec ETA:662 min
 writing records: ######################################## [ 82%] E:1093 sec ETA:327 min
 writing records: ######################################## [ 82%] E:929 sec ETA:327 min
 writing records: ######################################## [ 83%] E:970 sec ETA:274 min
 writing records: ######################################## [ 83%] E:822 sec ETA:274 min
 writing records: ######################################## [ 84%] E:973 sec ETA:259 min
 writing records: ######################################## [ 84%] E:1296 sec ETA:259 min
 writing records: ######################################## [ 85%] E:2126 sec ETA:531 min
 writing records: ######################################## [ 85%] E:1262 sec ETA:531 min
 writing records: ######################################## [ 86%] E:1120 sec ETA:261 min
 writing records: ######################################## [ 86%] E:966 sec ETA:261 min
 writing records: ######################################## [ 86%] E:985 sec ETA:261 min
 writing records: ######################################## [ 87%] E:1125 sec ETA:243 min
 writing records: ######################################## [ 87%] E:1009 sec ETA:243 min
 writing records: ######################################## [ 88%] E:1046 sec ETA:209 min
 writing records: ######################################## [ 88%] E:1436 sec ETA:209 min
 writing records: ######################################## [ 89%] E:2160 sec ETA:396 min
 writing records: ######################################## [ 89%] E:1371 sec ETA:396 min
 writing records: ######################################## [ 90%] E:1135 sec ETA:189 min
 writing records: ######################################## [ 90%] E:1027 sec ETA:189 min
 writing records: ######################################## [ 90%] E:1084 sec ETA:189 min
 writing records: ######################################## [ 91%] E:1187 sec ETA:178 min
 writing records: ######################################## [ 91%] E:1058 sec ETA:178 min
 writing records: ######################################## [ 92%] E:940 sec ETA:125 min
 writing records: ######################################## [ 92%] E:1480 sec ETA:125 min
 writing records: ######################################## [ 93%] E:1514 sec ETA:176 min
 writing records: ######################################## [ 93%] E:1778 sec ETA:176 min
 writing records: ######################################## [ 94%] E:1583 sec ETA:158 min
 writing records: ######################################## [ 94%] E:1494 sec ETA:158 min
 writing records: ######################################## [ 95%] E:1531 sec ETA:127 min
 writing records: ######################################## [ 95%] E:1003 sec ETA:127 min
 writing records: ######################################## [ 95%] E:964 sec ETA:127 min
 writing records: ######################################## [ 96%] E:1715 sec ETA:114 min
 writing records: ######################################## [ 96%] E:1425 sec ETA:114 min
 writing records: ######################################## [ 97%] E:1340 sec ETA:67 min
 writing records: ######################################## [ 97%] E:1059 sec ETA:67 min
 writing records: ######################################## [ 98%] E:1181 sec ETA:39 min
 writing records: ######################################## [ 98%] E:1113 sec ETA:39 min
 writing records: ######################################## [ 99%] E:1104 sec ETA:18 min
 writing records: ######################################## [ 99%] E:1089 sec ETA:18 min
 writing records: ######################################## [100%] E:1116 sec ETA:0 min
Found 22258856 raw records, inserted in jobTransSummary:22214183
Elapsed time:272079 Rec/sec:81.6461
Done.

Restart dgas services :

[root@cert-13 ~]# /etc/init.d/dgas-hlrd restart
chown: `dgas:dgas': invalid user
Stopping HAD...                                            [  OK  ]
Stopping HLR listener:/var/lock/dgas/dgas_hlrd.lock (pid 13[  OK  ]
Stopping Batch Transaction manager:/var/lock/dgas/dgas_hlr_[  OK  ]ck (pid 13911)
Stopping urForward:/var/lock/dgas/dgas_hlr_urforward.lock ([  OK  ]2)
Initializing DGAS HLR....Locking other instances out.
Initializing database, this operation can take minutes to hours to complete depending on the size of your DB. DO NOT KILL THE COMMAND PLEASE.
Starting HLR listener:                                     [  OK  ]
Starting Batch Transaction manager:                        [  OK  ]
Starting urForward:                                        [  OK  ]
Starting DGAS HAD...                                       [  OK  ]

Check databases:

mysql> use hlr;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select count(*) from trans_in;
+----------+
| count(*) |
+----------+
| 23462895 |
+----------+
1 row in set (0.00 sec)

mysql> select count(*) from jobTransSummary;
+----------+
| count(*) |
+----------+
| 22214260 |
+----------+
1 row in set (0.00 sec) 

Forward Test:

Install a Second Level HLR For this test a second level HLR server has been installed in Bologna the name of the host is:

dgas-test-vm02.cnaf.infn.it

Change the conf file in First Level HLR : Ensure you have the variables sendRecordsStartDate and sendRecordsEndDate correctly set

[root@cert-13 ~]# cat /etc/dgas/dgas_hlr.conf | egrep -v '^$|^#'
hlr_sql_server = "localhost"
hlr_sql_user="dgas"
hlr_sql_password="omit"
hlr_sql_dbname = "hlr"
hlr_tmp_sql_dbname = "hlr_tmp"
hostProxyFile = "/tmp/hlrHostProxyFile"
hlr_user = "root"
hlr_def_port = "56568"
dgas_var_dir = "/var/"
hlr_def_log="/var/log/dgas/dgas_hlrd.log"
hlr_qmgr_def_log="/var/log/dgas/dgas_hlr_qmgrd.log"
hlr_def_lock="/var/lock/dgas/dgas_hlrd.lock"
hlr_qmgr_def_lock="/var/lock/dgas/dgas_hlr_qmgrd.lock"
hlr_had_def_lock="/var/lock/dgas/dgas_hlr_had.lock"
systemLogLevel = "6"
accountCheckPolicy = "lazy"
hlr_qmgr_expPeriod = "600"
hlr_qmgr_tPerIter="20"
hlr_qmgr_pollPeriod = "30"
thread_number = "20"
maxItemsPerQuery = "5000"
authUserSqlQueries = "false"
jtsCommandLock = "/var/lock/dgas/hlrTranslateDb.lock"
sendRecordsStartDate = "2006-01-01"
sendRecordsEndDate = "2013-01-01"
recordsPerConnection="6000"
urForwardLock="/var/lock/dgas/dgas_hlr_urforward.lock"
urForwardLog="/var/log/dgas/dgas_hlr_urforward.log"
forwardPeriod="10800"
serversFile="/etc/dgas/dgas_2lservers.conf"
translateQueryLenght="1"



[root@cert-13 ~]# cat /etc/dgas/dgas_2lservers.conf 
dgas-test-vm02.cnaf.infn.it

From the first Level HLRrestart the hlrd services:

[root@cert-13 ~]# /etc/init.d/dgas-hlrd restart
chown: `dgas:dgas': invalid user
Stopping HAD...                                            [  OK  ]
Stopping HLR listener:/var/lock/dgas/dgas_hlrd.lock (pid 23[  OK  ]
Stopping Batch Transaction manager:/var/lock/dgas/dgas_hlr_[  OK  ]ck (pid 23535)
Stopping urForward:/var/lock/dgas/dgas_hlr_urforward.lock ([  OK  ]6)
Initializing DGAS HLR....Locking other instances out.
Initializing database, this operation can take minutes to hours to complete depending on the size of your DB. DO NOT KILL THE COMMAND PLEASE.
Starting HLR listener:                                     [  OK  ]
Starting Batch Transaction manager:                        [  OK  ]
Starting urForward:                                        [  OK  ]
Starting DGAS HAD...                                       [  OK  ]

Send a reset form First Level HLR

/usr/libexec/dgas-hlr-urforward -r

Have a look in log file:

[root@cert-13 ~]# tail -f  /var/log/dgas/dgas_hlr_urforward.log
2012 Aug 02 13:58:19 (5):Log level:6
2012 Aug 02 13:58:19 (4):Start sending records (sendRecordsStartDate) from:2006-01-01
2012 Aug 02 13:58:19 (4):Sending until (sendRecordsEndDate):2013-01-01
2012 Aug 02 13:58:19 (4):Number of record sent per iteration (recordsPerConnection):6000
2012 Aug 02 13:58:19 (6):serviceVersion():_hlrDb.errNo=0,
2012 Aug 02 13:58:19 (4):Contacting:dgas-test-vm02.cnaf.infn.it
2012 Aug 02 13:58:19 (3):Entering getInfo()
2012 Aug 02 13:58:19 (6):query:SELECT max(id) FROM jobTransSummary
2012 Aug 02 13:58:19 (4):Entering contactServer(dgas-test-vm02.cnaf.infn.it)
2012 Aug 02 13:58:19 (5):server parameters:
2012 Aug 02 13:58:19 (5):serverVersion:4.0.13
2012 Aug 02 13:58:19 (6):acceptRecordsStartDate:2006-01-01
2012 Aug 02 13:58:19 (6):acceptRecordsEndDate:2017-01-01
2012 Aug 02 13:58:19 (6):recordsPerConnection:6000
2012 Aug 02 13:58:19 (5):urSourceServer:
2012 Aug 02 13:58:19 (5):remoteRecordId:
2012 Aug 02 13:58:19 (5):recordDate:
2012 Aug 02 13:58:19 (5):recordInsertDate:
2012 Aug 02 13:58:19 (5):lastInsertedUniqueChecksum:
2012 Aug 02 13:58:19 (4):Server answer:dgas-test-vm02.cnaf.infn.it
2012 Aug 02 13:58:19 (6):Entering sendUsegeRecords
2012 Aug 02 13:58:19 (6):query:SELECT max(id) FROM jobTransSummary
2012 Aug 02 13:58:19 (6):query:SELECT min(id),max(id) FROM jobTransSummary WHERE date >='2006-01-01' AND date <='2017-01-01' AND id >0
2012 Aug 02 13:59:54 (6):Performing query:SELECT * FROM jobTransSummary WHERE id >=1 AND id < 6001 AND date >='2006-01-01' AND date <='2017-01-01' AND id <= 22214260
2012 Aug 02 13:59:54 (4):Entering contactServer(dgas-test-vm02.cnaf.infn.it)
2012 Aug 02 14:00:25 (5):Percentage of sent records:0.0270124,ETA:114731 secs
2012 Aug 02 14:00:25 (6):Performing query:SELECT * FROM jobTransSummary WHERE id >=6001 AND id < 12001 AND date >='2006-01-01' AND date <='2017-01-01' AND id <= 22214260
2012 Aug 02 14:00:26 (4):Entering contactServer(dgas-test-vm02.cnaf.infn.it)
...

warning: rpmts_HdrFromFdno: Header V3 DSA signature: NOKEY, key ID df9e12ef

Public key for emi-release-2.0.0-1.sl5.noarch.rpm is not installed

You have to pu the variable gpgcheck=0 in /etc/yum.conf

[~]# cat /etc/yum.conf 

[main]
cachedir=/var/cache/yum
keepcache=0
debuglevel=2
logfile=/var/log/yum.log
distroverpkg=redhat-release
tolerant=1
exactarch=1
obsoletes=1
gpgcheck=0
plugins=1

metadata_expire=2h
installonly_limit=4

# PUT YOUR REPOS HERE OR IN separate files named file.repo
# in /etc/yum.repos.d

• Test Plan

HLR Server

Documentation

• Sensors Guide
• Service Guide
• Test Plan

IGI Documentation

Prerequisites

Check the FQDN hostname

Ensure that the hostnames of your machines are correctly set. Run the command:

hostname -f

Disabling SELinux

# disabled - No SELinux policy is loaded.
SELINUX=disabled

Repository Settings

IMPORTANT NOTE:

• If present remember to disable the dag.repo if it is enabled.

cd /etc/yum.repos.d/
mv dag.repo dag.repo.disabled

Get the repos file:

cd /root
wget http://download.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm
wget http://emisoft.web.cern.ch/emisoft/dist/EMI/1/sl5/x86_64/updates/emi-release-1.0.1-1.sl5.noarch.rpm
yum localinstall *.rpm
wget http://repo-pd.italiangrid.it/mrepo/repos/igi/sl5/x86_64/igi-cert-emi.repo -O /etc/yum.repos.d/igi-cert-emi.repo

Installation:

CAs installation:

• Install CAs on ALL profiles:

yum install ca-policy-egi-core

Service installation

• Install the HLR Server metapackage, containing all packages needed by this service:

yum install igi-hlr

Configure the service:

YAIM Variables:

Hi the Mandatory Variables for the HLR Server are:

Other useful variables can be found in /opt/glite/yaim/examples/siteinfo/services/hlr file.

Configuring HLR

/opt/glite/yaim/bin/yaim -c -d 6 -s <site-info.def> -n hlr 2>&1 | tee /root/conf_HLR.`hostname -s`.`date +%Y%m%d-%H%M%S`.log

Testing reports

-- SergioTraldi - 2012-04-23