Difference: WMSTask27731 (15 vs. 16)
Revision 162012-07-02 - FabioCapannini
| Line: 1 to 1 | ||||||||
|---|---|---|---|---|---|---|---|---|
WMS test report | ||||||||
| Line: 1113 to 1113 | ||||||||
===> Test log file is WMSService-TS_20120702105949.log
| ||||||||
| Added: | ||||||||
| > > | Testing forwarding parameters for parallel jobsIn order to test the passing of mpi parameters (wholenodes,smpgranularity,cpunumber,hostnumber) to the ce, an mpi jdl is submitted with varying values for the parameters, specifying a cream CE as destination. Then the cream jobid is retrieved via a glite-wms-job-logging-info -v 2 ("Dest jobid" field of a Transfer event).Finally the cream jdl is found via glite-ce-job-status -L 2 <cream-jobid>. The following jdls were submitted: 1)
Executable = "openmpi-wrapper.sh";
Arguments="hello";
StdOutput = "hello.out";
StdError = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
InputSandbox = {"/home/capannini/certification/mpi/openmpi-wrapper.sh","/home/capannini/certification/mpi/hello.c","/home/capannini/certification/mpi/Makefile"};
Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID);
WholeNodes = true;
SMPGranularity = 2;
CpuNumber=2;
Hostnumber = 1;
Resulting cream JDL:
JDL = [[ CEInfoHostName = "cert-ce-01.cnaf.infn.it"; OutputSandboxPath = "/var/SandboxDir/xt/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fxtXTZHqgFsO-U5-LktmARA/output"; StdOutput = "hello.out"; BatchSystem = "pbs"; GlobusResourceContactString = "cert-ce-01.cnaf.infn.it:8443/cream-pbs"; QueueName = "cert"; ShallowRetryCount = 10; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 3; Executable = "openmpi-wrapper.sh"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000005:BH=0000000000:JSS=000002:LM=000002:LRMS=000000:APP=000000:LBS=000000"; Arguments = "hello"; AllowZippedISB = true; maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank = -other.GlueCEStateEstimatedResponseTime; ce_id = "cert-ce-01.cnaf.infn.it:8443/cream-pbs-cert"; ZippedISB = { "ISBfiles_sKVKJJgGMUeK6H6RwhcWBQ_0.tar.gz" }; OutputSandboxDestURI = { "gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/xt/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fxtXTZHqgFsO-U5-LktmARA/output/hello.out","gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/xt/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fxtXTZHqgFsO-U5-LktmARA/output/hello.err" }; OutputSandbox = { "hello.out","hello.err" }; CeRequirements = "( true && ( true && ( true && ( true && ( true && ( true && Member(\"OPENMPI\",other.GlueHostApplicationSoftwareRunTimeEnvironment) ) ) ) ) ) ) && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )"; InputSandbox = { "gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/xt/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fxtXTZHqgFsO-U5-LktmARA/input/openmpi-wrapper.sh","gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/xt/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fxtXTZHqgFsO-U5-LktmARA/input/hello.c","gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/xt/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fxtXTZHqgFsO-U5-LktmARA/input/Makefile","gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/xt/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fxtXTZHqgFsO-U5-LktmARA/input/.BrokerInfo" }; edg_jobid = "https://emi-demo12.cnaf.infn.it:9000/xtXTZHqgFsO-U5-LktmARA"; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini"; StdError = "hello.err"; SMPGranularity = 2; InputSandboxPath = "/var/SandboxDir/xt/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fxtXTZHqgFsO-U5-LktmARA/input"; CeApplicationDir = "/opt/exp_soft/dteam"; ReallyRunningToken = "gsiftp://emi-demo12.cnaf.infn.it/var/SandboxDir/xt/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fxtXTZHqgFsO-U5-LktmARA/token.txt_0"; rank = -other.GlueCEStateEstimatedResponseTime; WholeNodes = true; CpuNumber = 2; WMSHostname = "emi-demo12.cnaf.infn.it"; lrms_type = "pbs"; MyProxyServer = "myproxy.cnaf.infn.it"; X509UserProxy = "/var/SandboxDir/xt/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fxtXTZHqgFsO-U5-LktmARA/user.proxy"; InputSandboxDestFileName = { "openmpi-wrapper.sh","hello.c","Makefile" }; Hostnumber = 1; requirements = ( ( ( ( ( ( Member("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*",other.GlueCEUniqueID) ) && ( true ) ) && other.GlueCEInfoTotalCPUs >= 2 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueCEInfoTotalCPUs >= 2 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) : !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/xt/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fxtXTZHqgFsO-U5-LktmARA" ]]
2)
Executable = "openmpi-wrapper.sh";
Arguments="hello";
StdOutput = "hello.out";
StdError = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
InputSandbox = {"/home/capannini/certification/mpi/openmpi-wrapper.sh","/home/capannini/certification/mpi/hello.c","/home/capannini/certification/mpi/Makefile"};
Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID);
WholeNodes = true;
SMPGranularity = 2;
Resulting ce JDL:
JDL = [[ CEInfoHostName = "cex.grid.unipg.it"; OutputSandboxPath = "/var/SandboxDir/lY/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2flYvOaWtj5auRglvI-uSZpw/output"; StdOutput = "hello.out"; BatchSystem = "pbs"; GlobusResourceContactString = "cex.grid.unipg.it:8443/cream-pbs"; QueueName = "cert"; ShallowRetryCount = 10; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 3; Executable = "openmpi-wrapper.sh"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000017:BH=0000000000:JSS=000006:LM=000022:LRMS=000000:APP=000000:LBS=000000"; Arguments = "hello"; AllowZippedISB = true; maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank = -other.GlueCEStateEstimatedResponseTime; ce_id = "cex.grid.unipg.it:8443/cream-pbs-cert"; ZippedISB = { "ISBfiles_4dXYGVKTKVRagB4f-hex8Q_0.tar.gz" }; OutputSandboxDestURI = { "gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/lY/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2flYvOaWtj5auRglvI-uSZpw/output/hello.out","gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/lY/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2flYvOaWtj5auRglvI-uSZpw/output/hello.err" }; OutputSandbox = { "hello.out","hello.err" }; CeRequirements = "( true && ( true && ( true && ( true && Member(\"OPENMPI\",other.GlueHostApplicationSoftwareRunTimeEnvironment) ) ) ) ) && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )"; InputSandbox = { "gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/lY/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2flYvOaWtj5auRglvI-uSZpw/input/openmpi-wrapper.sh","gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/lY/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2flYvOaWtj5auRglvI-uSZpw/input/hello.c","gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/lY/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2flYvOaWtj5auRglvI-uSZpw/input/Makefile","gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/lY/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2flYvOaWtj5auRglvI-uSZpw/input/.BrokerInfo" }; edg_jobid = "https://emi-demo12.cnaf.infn.it:9000/lYvOaWtj5auRglvI-uSZpw"; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini"; StdError = "hello.err"; SMPGranularity = 2; InputSandboxPath = "/var/SandboxDir/lY/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2flYvOaWtj5auRglvI-uSZpw/input"; CeApplicationDir = "/opt/exp_soft/dteam"; ReallyRunningToken = "gsiftp://emi-demo12.cnaf.infn.it/var/SandboxDir/lY/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2flYvOaWtj5auRglvI-uSZpw/token.txt_0"; rank = -other.GlueCEStateEstimatedResponseTime; WholeNodes = true; edg_previous_matches = { "grisuce.scope.unina.it","emi-ce01.scope.unina.it" }; ReplansCount = 0; WMSHostname = "emi-demo12.cnaf.infn.it"; lrms_type = "torque"; MyProxyServer = "myproxy.cnaf.infn.it"; edg_previous_matches_ex = { [ timestamp = 1341236596; ce_id = "grisuce.scope.unina.it" ],[ timestamp = 1341236597; ce_id = "emi-ce01.scope.unina.it" ] }; X509UserProxy = "/var/SandboxDir/lY/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2flYvOaWtj5auRglvI-uSZpw/user.proxy"; InputSandboxDestFileName = { "openmpi-wrapper.sh","hello.c","Makefile" }; requirements = ( ( ( ( Member("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*",other.GlueCEUniqueID) ) && ( true ) ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) : !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/lY/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2flYvOaWtj5auRglvI-uSZpw" ]]
3)
Executable = "openmpi-wrapper.sh";
Arguments="hello";
StdOutput = "hello.out";
StdError = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
InputSandbox = {"/home/capannini/certification/mpi/openmpi-wrapper.sh","/home/capannini/certification/mpi/hello.c","/home/capannini/certification/mpi/Makefile"};
Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID);
WholeNodes = false;
SMPGranularity = 3;
CpuNumber=3;
Hostnumber = 1;
Submission result:
[capannini@cert-17 mpi]$ glite-wms-job-submit -a -c ~/wms_emi-demo12.conf mpi.jdl
Connecting to the service https://emi-demo12.cnaf.infn.it:7443/glite_wms_wmproxy_server
Error -
SMPGranularity and HostNumber are mutually exclusive when WholeNodes allocation is not requested: wrong combination of values
4)
Executable = "openmpi-wrapper.sh";
Arguments="hello";
StdOutput = "hello.out";
StdError = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
InputSandbox = {"/home/capannini/certification/mpi/openmpi-wrapper.sh","/home/capannini/certification/mpi/hello.c","/home/capannini/certification/mpi/Makefile"};
Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID);
WholeNodes = false;
SMPGranularity = 3;
CpuNumber=3;
Resulting ce JDL:
JDL = [[ CEInfoHostName = "ng-ce.grid.unipg.it"; OutputSandboxPath = "/var/SandboxDir/nM/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fnM8Uj4qxVn2csciKW9eh2g/output"; StdOutput = "hello.out"; BatchSystem = "pbs"; GlobusResourceContactString = "ng-ce.grid.unipg.it:8443/cream-pbs"; QueueName = "cert"; ShallowRetryCount = 10; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 3; Executable = "openmpi-wrapper.sh"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000011:BH=0000000000:JSS=000004:LM=000012:LRMS=000000:APP=000000:LBS=000000"; Arguments = "hello"; AllowZippedISB = true; maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank = -other.GlueCEStateEstimatedResponseTime; ce_id = "ng-ce.grid.unipg.it:8443/cream-pbs-cert"; ZippedISB = { "ISBfiles_IC2yIMsun757SF2rrFtzqA_0.tar.gz" }; OutputSandboxDestURI = { "gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/nM/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fnM8Uj4qxVn2csciKW9eh2g/output/hello.out","gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/nM/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fnM8Uj4qxVn2csciKW9eh2g/output/hello.err" }; OutputSandbox = { "hello.out","hello.err" }; CeRequirements = "( true && ( true && ( true && ( true && ( true && ( true && Member(\"OPENMPI\",other.GlueHostApplicationSoftwareRunTimeEnvironment) ) ) ) ) ) ) && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )"; InputSandbox = { "gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/nM/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fnM8Uj4qxVn2csciKW9eh2g/input/openmpi-wrapper.sh","gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/nM/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fnM8Uj4qxVn2csciKW9eh2g/input/hello.c","gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/nM/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fnM8Uj4qxVn2csciKW9eh2g/input/Makefile","gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/nM/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fnM8Uj4qxVn2csciKW9eh2g/input/.BrokerInfo" }; edg_jobid = "https://emi-demo12.cnaf.infn.it:9000/nM8Uj4qxVn2csciKW9eh2g"; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini"; StdError = "hello.err"; SMPGranularity = 3; InputSandboxPath = "/var/SandboxDir/nM/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fnM8Uj4qxVn2csciKW9eh2g/input"; CeApplicationDir = "/opt/exp_soft/dteam"; ReallyRunningToken = "gsiftp://emi-demo12.cnaf.infn.it/var/SandboxDir/nM/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fnM8Uj4qxVn2csciKW9eh2g/token.txt_0"; rank = -other.GlueCEStateEstimatedResponseTime; WholeNodes = false; CpuNumber = 3; edg_previous_matches = { "ce.scope.unina.it" }; ReplansCount = 0; WMSHostname = "emi-demo12.cnaf.infn.it"; lrms_type = "torque"; MyProxyServer = "myproxy.cnaf.infn.it"; edg_previous_matches_ex = { [ timestamp = 1341237438; ce_id = "ce.scope.unina.it" ] }; X509UserProxy = "/var/SandboxDir/nM/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fnM8Uj4qxVn2csciKW9eh2g/user.proxy"; InputSandboxDestFileName = { "openmpi-wrapper.sh","hello.c","Makefile" }; requirements = ( ( ( ( ( ( Member("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*",other.GlueCEUniqueID) ) && ( true ) ) && other.GlueCEInfoTotalCPUs >= 3 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueCEInfoTotalCPUs >= 3 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) : !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/nM/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fnM8Uj4qxVn2csciKW9eh2g" ]]
5)
Executable = "openmpi-wrapper.sh";
Arguments="hello";
StdOutput = "hello.out";
StdError = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
InputSandbox = {"/home/capannini/certification/mpi/openmpi-wrapper.sh","/home/capannini/certification/mpi/hello.c","/home/capannini/certification/mpi/Makefile"};
Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID);
WholeNodes = true;
SMPGranularity = 8;
CpuNumber=8;
Hostnumber = 2;
Resulting ce JDL:
JDL = [[ CEInfoHostName = "gridsrv2-4.dir.garr.it"; OutputSandboxPath = "/var/SandboxDir/XH/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fXH4DRPKTlCxGt5uL3NyVjA/output"; StdOutput = "hello.out"; BatchSystem = "pbs"; GlobusResourceContactString = "gridsrv2-4.dir.garr.it:8443/cream-pbs"; QueueName = "cert"; ShallowRetryCount = 10; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 3; Executable = "openmpi-wrapper.sh"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000017:BH=0000000000:JSS=000006:LM=000022:LRMS=000000:APP=000000:LBS=000000"; Arguments = "hello"; AllowZippedISB = true; maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank = -other.GlueCEStateEstimatedResponseTime; ce_id = "gridsrv2-4.dir.garr.it:8443/cream-pbs-cert"; ZippedISB = { "ISBfiles_vh0tUb1uOBxl3Nj8-svtgw_0.tar.gz" }; OutputSandboxDestURI = { "gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/XH/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fXH4DRPKTlCxGt5uL3NyVjA/output/hello.out","gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/XH/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fXH4DRPKTlCxGt5uL3NyVjA/output/hello.err" }; OutputSandbox = { "hello.out","hello.err" }; CeRequirements = "( true && ( true && ( true && ( true && ( true && ( true && Member(\"OPENMPI\",other.GlueHostApplicationSoftwareRunTimeEnvironment) ) ) ) ) ) ) && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )"; InputSandbox = { "gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/XH/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fXH4DRPKTlCxGt5uL3NyVjA/input/openmpi-wrapper.sh","gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/XH/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fXH4DRPKTlCxGt5uL3NyVjA/input/hello.c","gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/XH/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fXH4DRPKTlCxGt5uL3NyVjA/input/Makefile","gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/XH/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fXH4DRPKTlCxGt5uL3NyVjA/input/.BrokerInfo" }; edg_jobid = "https://emi-demo12.cnaf.infn.it:9000/XH4DRPKTlCxGt5uL3NyVjA"; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini"; StdError = "hello.err"; SMPGranularity = 8; InputSandboxPath = "/var/SandboxDir/XH/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fXH4DRPKTlCxGt5uL3NyVjA/input"; CeApplicationDir = "/opt/exp_soft/dteam"; ReallyRunningToken = "gsiftp://emi-demo12.cnaf.infn.it/var/SandboxDir/XH/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fXH4DRPKTlCxGt5uL3NyVjA/token.txt_0"; rank = -other.GlueCEStateEstimatedResponseTime; WholeNodes = true; CpuNumber = 8; edg_previous_matches = { "ce.scope.unina.it","grisuce.scope.unina.it" }; ReplansCount = 0; WMSHostname = "emi-demo12.cnaf.infn.it"; lrms_type = "pbs"; MyProxyServer = "myproxy.cnaf.infn.it"; edg_previous_matches_ex = { [ timestamp = 1341238279; ce_id = "ce.scope.unina.it" ],[ timestamp = 1341238280; ce_id = "grisuce.scope.unina.it" ] }; X509UserProxy = "/var/SandboxDir/XH/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fXH4DRPKTlCxGt5uL3NyVjA/user.proxy"; InputSandboxDestFileName = { "openmpi-wrapper.sh","hello.c","Makefile" }; Hostnumber = 2; requirements = ( ( ( ( ( ( Member("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*",other.GlueCEUniqueID) ) && ( true ) ) && other.GlueCEInfoTotalCPUs >= 8 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueCEInfoTotalCPUs >= 8 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) : !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://emi-demo12.cnaf.infn.it:2811/var/SandboxDir/XH/https_3a_2f_2femi-demo12.cnaf.infn.it_3a9000_2fXH4DRPKTlCxGt5uL3NyVjA" ]]
===> Test PASSED
Perusal[capannini@cert-17 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 6 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 6: Submit a perusal job BEWARE default min perusal interval is 1000 secs, so this phase could take many minutes ===> ===> Test: WMS-job-cycle.py ===> WMS: emi-demo12.cnaf.infn.it ===> Started: 16:19:19 ===> Ended : 16:48:50 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120702161919.log [capannini@cert-17 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 6 -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test a complete job cycle: from submission to get output ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Set 6: Submit a perusal job BEWARE default min perusal interval is 1000 secs, so this phase could take many minutes ===> ===> Test: WMS-job-cycle.py ===> WMS: emi-demo12.cnaf.infn.it ===> Started: 16:19:50 ===> Ended : 17:14:07 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120702161950.log
Delegation[capannini@cert-17 WMS-service]$ python WMS-delegation-test.py -c wms-command.conf -V dteam -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test delegation operation ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: Check --autm-delegation option Test 2: Check --delegationid option Test 3: Try to delegate with a short proxy and check the validity Test 4: Works with expired proxy ===> ===> Test: WMS-delegation-test.py ===> WMS: emi-demo12.cnaf.infn.it ===> Started: 16:25:04 ===> Ended : 16:26:19 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120702162504.log [capannini@cert-17 WMS-service]$ python WMS-delegation-test.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test delegation operation ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: Check --autm-delegation option Test 2: Check --delegationid option Test 3: Try to delegate with a short proxy and check the validity Test 4: Works with expired proxy ===> ===> Test: WMS-delegation-test.py ===> WMS: emi-demo12.cnaf.infn.it ===> Started: 16:25:25 ===> Ended : 16:26:37 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120702162525.log [capannini@cert-17 WMS-service]$ python WMS-delegation-test.py -c wms-command.conf -d 3 -l ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: Test delegation operation ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: Check --autm-delegation option Test 2: Check --delegationid option There are other two tests which require the user proxy password. Use -i option to enable them ===> ===> Test: WMS-delegation-test.py ===> WMS: emi-demo12.cnaf.infn.it ===> Started: 16:26:03 ===> Ended : 16:26:08 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120702162603.log
Job list-matchwithout data[capannini@cert-17 WMS-service]$ python WMS-listmatch-test.py -c wms-command.conf -V dteam -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Job ListMatch Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: check if glite-wms-job-list-match command exists Test 2: test a simple matching Test 3: check --delegationid option Test 4: exclude a CE from the match Test 5: try a failure matching (Requirements == false) Test 6: try a matching with EnableWMSFeedback attribute true ===> ===> Test: WMS-listmatch-test.py ===> WMS: emi-demo12.cnaf.infn.it ===> Started: 16:37:45 ===> Ended : 16:37:54 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120702163745.log [capannini@cert-17 WMS-service]$ python WMS-listmatch-test.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -i ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Job ListMatch Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Enter the user proxy password: Test 1: check if glite-wms-job-list-match command exists Test 2: test a simple matching Test 3: check --delegationid option Test 4: exclude a CE from the match Test 5: try a failure matching (Requirements == false) Test 6: try a matching with EnableWMSFeedback attribute true ===> ===> Test: WMS-listmatch-test.py ===> WMS: emi-demo12.cnaf.infn.it ===> Started: 16:38:08 ===> Ended : 16:38:20 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120702163808.log [capannini@cert-17 WMS-service]$ [capannini@cert-17 WMS-service]$ python WMS-listmatch-test.py -c wms-command.conf -d 3 -l ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ ===> + TestSuite of the WMS Service ===> + Description: WMS Job ListMatch Testing ===> +++++++++++++++++++++++++++++++++++++++++++++++++++++ Test 1: check if glite-wms-job-list-match command exists Test 2: test a simple matching Test 3: check --delegationid option Test 4: exclude a CE from the match Test 5: try a failure matching (Requirements == false) Test 6: try a matching with EnableWMSFeedback attribute true ===> ===> Test: WMS-listmatch-test.py ===> WMS: emi-demo12.cnaf.infn.it ===> Started: 16:38:29 ===> Ended : 16:38:36 ===> ===> === test PASSED === ===> ===> Test log file is WMSService-TS_20120702163829.log
with data
[capannini@cert-17 WMS-service]$ export LFC_HOST=prod-lfc-shared-central.cern.ch
[capannini@cert-17 WMS-service]$ lcg-lr --vo dteam lfn:/grid/dteam/fcapanni/PI_1M.txt
srm://gridsrm.pi.infn.it/dteam/fcapanni/PI_1M.txt
srm://srm-public.cern.ch/castor/cern.ch/grid/dteam/fcapanni/PI_1M.txt
The two SEs where the file can be found are gridsrm.pi.infn.it and srm-public.cern.ch.
[capannini@cert-17 WMS-service]$ cat JDLs/data-req.jdl
###########################################
# JDL with Data Requirements #
###########################################
Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";
Prologue = "prologue.sh";
FuzzyRank = true;
Environment={"LFC_HOST=prod-lfc-shared-central.cern.ch", "LFC_HOME=prod-lfc-shared-central.cern.ch:/grid/dteam/aleph"};
InputSandbox = {"JDLs/prologue.sh"};
OutputSandbox = {"std.out", "std.err","prologue.out", "PI_1M.txt"};
DataRequirements = {
[
DataCatalogType = "DLI";
DataCatalog ="http://prod-lfc-shared-central.cern.ch:8085/";
InputData = { "lfn:/grid/dteam/fcapanni/PI_1M.txt" };
]
};
DataAccessProtocol = "gsiftp";
AllowZippedISB=true;
RetryCount = 1;
ShallowRetryCount = 2;
[capannini@cert-17 WMS-service]$ glite-wms-job-list-match -a -c ~/wms_emi-demo12.conf JDLs/data-req.jdl
Connecting to the service https://emi-demo12.cnaf.infn.it:7443/glite_wms_wmproxy_server
==========================================================================
COMPUTING ELEMENT IDs LIST
The following CE(s) matching your job requirements have been found:
*CEId*
- ce204.cern.ch:8443/cream-lsf-grid_2nh_dteam
- ce204.cern.ch:8443/cream-lsf-grid_dteam
- ce205.cern.ch:8443/cream-lsf-grid_2nh_dteam
- ce205.cern.ch:8443/cream-lsf-grid_dteam
- ce207.cern.ch:8443/cream-lsf-grid_2nh_dteam
- ce207.cern.ch:8443/cream-lsf-grid_dteam
- gridce4.pi.infn.it:8443/cream-lsf-certmpi
- gridce0.pi.infn.it:8443/cream-lsf-cert
- gridce1.pi.infn.it:2119/jobmanager-lcglsf-cert
- gridce2.pi.infn.it:2119/jobmanager-lcglsf-cert
- gridce3.pi.infn.it:8443/cream-lsf-certmpi
==========================================================================
In order to verify that these CEs are close to the above SEs:
[capannini@cert-17 WMS-service]$ lcg-infosites --vo dteam closeSE|grep -A 5 ce204
Name of the CE: ce204.cern.ch:8443/cream-lsf-grid_2nh_dteam
srm-atlas.cern.ch
srm-cms.cern.ch
srm-lhcb.cern.ch
srm-public.cern.ch
Name of the CE: ce204.cern.ch:8443/cream-lsf-grid_dteam
srm-atlas.cern.ch
srm-cms.cern.ch
srm-lhcb.cern.ch
srm-public.cern.ch
[capannini@cert-17 WMS-service]$ lcg-infosites --vo dteam closeSE|grep -A 5 ce205
Name of the CE: ce205.cern.ch:8443/cream-lsf-grid_2nh_dteam
srm-atlas.cern.ch
srm-cms.cern.ch
srm-lhcb.cern.ch
srm-public.cern.ch
Name of the CE: ce205.cern.ch:8443/cream-lsf-grid_dteam
srm-atlas.cern.ch
srm-cms.cern.ch
srm-lhcb.cern.ch
srm-public.cern.ch
[capannini@cert-17 WMS-service]$ lcg-infosites --vo dteam closeSE|grep -A 5 ce207
Name of the CE: ce207.cern.ch:8443/cream-lsf-grid_2nh_dteam
srm-atlas.cern.ch
srm-cms.cern.ch
srm-lhcb.cern.ch
srm-public.cern.ch
Name of the CE: ce207.cern.ch:8443/cream-lsf-grid_dteam
srm-atlas.cern.ch
srm-cms.cern.ch
srm-lhcb.cern.ch
srm-public.cern.ch
[capannini@cert-17 WMS-service]$ lcg-infosites --vo dteam closeSE|grep -A 5 gridce4.pi.infn.it
Name of the CE: gridce4.pi.infn.it:8443/cream-lsf-certmpi
gridsrm.pi.infn.it
Name of the CE: gridgate.cp.dias.ie:2119/jobmanager-lcgpbs-test
gridstore.cp.dias.ie
[capannini@cert-17 WMS-service]$ lcg-infosites --vo dteam closeSE|grep -A 5 gridce0.pi.infn.it
Name of the CE: gridce0.pi.infn.it:8443/cream-lsf-cert
gridsrm.pi.infn.it
Name of the CE: gridce01.ifca.es:8443/cream-sge-sgm
griddpm01.ifca.es
storm.ifca.es
[capannini@cert-17 WMS-service]$ lcg-infosites --vo dteam closeSE|grep -A 5 gridce1.pi.infn.it
Name of the CE: gridce1.pi.infn.it:2119/jobmanager-lcglsf-cert
gridsrm.pi.infn.it
Name of the CE: gridce2.pi.infn.it:2119/jobmanager-lcglsf-cert
gridsrm.pi.infn.it
[capannini@cert-17 WMS-service]$ lcg-infosites --vo dteam closeSE|grep -A 5 gridce2.pi.infn.it
Name of the CE: gridce2.pi.infn.it:2119/jobmanager-lcglsf-cert
gridsrm.pi.infn.it
Name of the CE: gridce3.pi.infn.it:8443/cream-lsf-certmpi
gridsrm.pi.infn.it
[capannini@cert-17 WMS-service]$ lcg-infosites --vo dteam closeSE|grep -A 5 gridce3.pi.infn.it
Name of the CE: gridce3.pi.infn.it:8443/cream-lsf-certmpi
gridsrm.pi.infn.it
Name of the CE: gridce4.pi.infn.it:8443/cream-lsf-certmpi
gridsrm.pi.infn.it
===> Test PASSED
gangmatching
[capannini@cert-17 certification]$ cat gang.jdl
[
executable="ls.sh";
myproxyserver="";
SErequirements = anyMatch(other.storage.CloseSEs, target.GlueSAStateAvailableSpace > 20);
//SErequirements = true;
CErequirements = other.GlueCEStateStatus == "Production" && other.GlueCEInfoTotalCPUs >= 2;
requirements = CErequirements && SErequirements;
inputsandbox={"ls.sh"};
stdoutput="out3.out";
stderror="err2.err";
outputsandbox={"out3.out", "err2.err", "ls.sh", "a"};
]
[capannini@cert-17 certification]$ glite-wms-job-list-match -a -c ~/wms_emi-demo12.conf gang.jdl
Connecting to the service https://emi-demo12.cnaf.infn.it:7443/glite_wms_wmproxy_server
==========================================================================
COMPUTING ELEMENT IDs LIST
The following CE(s) matching your job requirements have been found:
*CEId*
- cccreamceli05.in2p3.fr:8443/cream-sge-long
- cccreamceli05.in2p3.fr:8443/cream-sge-medium
- cccreamceli05.in2p3.fr:8443/cream-sge-short
- cccreamceli06.in2p3.fr:8443/cream-sge-long
- cccreamceli06.in2p3.fr:8443/cream-sge-medium
- cccreamceli06.in2p3.fr:8443/cream-sge-short
- cccreamceli09.in2p3.fr:8443/cream-sge-long
- cccreamceli09.in2p3.fr:8443/cream-sge-medium
- cccreamceli09.in2p3.fr:8443/cream-sge-short
- cmsce01.na.infn.it:2119/jobmanager-lcgpbs-cert
- cmsrm-cream02.roma1.infn.it:8443/cream-lsf-cmsgcert
- unict-dmi-ce-01.ct.pi2s2.it:2119/jobmanager-lcglsf-cert
- unime-ce-01.me.pi2s2.it:2119/jobmanager-lcglsf-cert
- virgo-ce.roma1.infn.it:2119/jobmanager-lcgpbs-cert
- cmsrm-ce01.roma1.infn.it:2119/jobmanager-lcglsf-cmsgcert
- cmsrm-ce02.roma1.infn.it:2119/jobmanager-lcglsf-cmsgcert
- cmsrm-cream01.roma1.infn.it:8443/cream-lsf-cmsgcert
- cccreamceli10.in2p3.fr:8443/cream-sge-long
- cccreamceli10.in2p3.fr:8443/cream-sge-medium
- cccreamceli10.in2p3.fr:8443/cream-sge-short
- inaf-ce-01.ct.pi2s2.it:8443/cream-lsf-cert
==========================================================================
[capannini@cert-17 certification]$ lcg-infosites closeSE|grep -A 5 cccreamceli05
Name of the CE: cccreamceli05.in2p3.fr:8443/cream-sge-long
ccsrm.in2p3.fr
Name of the CE: cccreamceli05.in2p3.fr:8443/cream-sge-medium
ccsrm.in2p3.fr
Name of the CE: cccreamceli05.in2p3.fr:8443/cream-sge-short
ccsrm.in2p3.fr
Name of the CE: cccreamceli06.in2p3.fr:8443/cream-sge-long
ccsrm.in2p3.fr
[capannini@cert-17 certification]$
[capannini@cert-17 certification]$ lcg-infosites se |egrep 'Avail|ccsrm.in2p3.fr'
Avail Space(kB) Used Space(kB) Type SE
12730125753 2302259783 SRM ccsrm.in2p3.fr
[capannini@cert-17 certification]$ lcg-infosites closeSE|grep -A 5 cmsrm-ce01
Name of the CE: cmsrm-ce01.roma1.infn.it:2119/jobmanager-lcglsf-cmsgcert
cmsrm-se01.roma1.infn.it
Name of the CE: cmsrm-ce02.roma1.infn.it:2119/jobmanager-lcglsf-cmsgcert
cmsrm-se01.roma1.infn.it
[capannini@cert-17 certification]$
[capannini@cert-17 certification]$ lcg-infosites se |egrep 'Avail|cmsrm-se01.roma1.infn.it'
Avail Space(kB) Used Space(kB) Type SE
1000 999999 SRM cmsrm-se01.roma1.infn.it
[capannini@cert-17 certification]$ lcg-infosites se |egrep 'Avail|cmsrm-se01.roma1.infn.it'
Avail Space(kB) Used Space(kB) Type SE
1000 999999 SRM cmsrm-se01.roma1.infn.it
[capannini@cert-17 certification]$
[capannini@cert-17 certification]$ lcg-infosites closeSE|grep -A 5 unict-dmi-ce-01
Name of the CE: unict-dmi-ce-01.ct.pi2s2.it:2119/jobmanager-lcglsf-cert
unict-dmi-se-01.ct.pi2s2.it
Name of the CE: unime-ce-01.me.pi2s2.it:2119/jobmanager-lcglsf-cert
unime-se-01.me.pi2s2.it
[capannini@cert-17 certification]$ lcg-infosites se |egrep 'Avail|unict-dmi-se-01.ct.pi2s2.it'
Avail Space(kB) Used Space(kB) Type SE
12512874596 6164874393 SRM unict-dmi-se-01.ct.pi2s2.it
===> Test PASSED | |||||||
Regression testsBugs attached to the task | ||||||||
| Line: 1233 to 1809 | ||||||||
| ||||||||
| Added: | ||||||||
| > > |
| |||||||
View topic | History: r27 < r26 < r25 < r24 | More topic actions...
Copyright © 2008-2023 by the contributing authors. All material on this collaboration platform is the property of the contributing authors. Ideas, requests, problems regarding TWiki? Send feedback