Submit this jdl: [ale@cream-15 UI]$ cat jdl/prove/mpi.jdl [ requirements = Member("MPI-START", other.GlueHostApplicationSoftwareRunTimeEnvironment) && Member("OPENMPI", other.GlueHostApplicationSoftwareRunTimeEnvironment) ; NodeNumber = 8; Executable = "MPItest.sh"; Arguments = "cpi 8"; StdOutput = "test.out"; StdError = "test.err"; InputSandbox = {"exe/MPItest.sh","exe/cpi"}; OutputSandbox = {"test.err","test.out","executable.out"}; ] Where: [ale@cream-15 UI]$ cat exe/MPItest.sh EXE=$1 CPU_NEEDED=$2 HOST_NODEFILE=$PBS_NODEFILE for i in `cat $HOST_NODEFILE` ; do ssh $i mkdir -p `pwd` /usr/bin/scp -rp ./* $i:`pwd` ssh $i chmod 755 `pwd`/$EXE done mod 755 $EXE mpiexec -np $CPU_NEEDED `pwd`/$EXE >& executable.out First to a cream CE: [ale@cream-15 UI]$ glite-wms-job-submit --config etc/wmp_devel18.conf -a jdl/prove/mpi.jdl Connecting to the service https://devel18.cnaf.infn.it:7443/glite_wms_wmproxy_server ====================== glite-wms-job-submit Success ====================== The job has been successfully submitted to the WMProxy Your job identifier is: https://devel15.cnaf.infn.it:9000/nseGexj8DrrO95L5HP-IfQ ========================================================================== [ale@cream-15 UI]$ glite-wms-job-status https://devel15.cnaf.infn.it:9000/nseGexj8DrrO95L5HP-IfQ ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : https://devel15.cnaf.infn.it:9000/nseGexj8DrrO95L5HP-IfQ Current Status: Done (Success) Logged Reason(s): - job completed - Job Terminated Successfully Exit code: 0 Status Reason: Job Terminated Successfully Destination: stremsel.nikhef.nl:8443/cream-pbs-infra Submitted: Wed Jun 9 13:06:40 2010 CEST ************************************************************* verify the output: [ale@cream-15 UI]$ glite-wms-job-output https://devel15.cnaf.infn.it:9000/nseGexj8DrrO95L5HP-IfQ Connecting to the service https://devel18.cnaf.infn.it:7443/glite_wms_wmproxy_server ================================================================================ JOB GET OUTPUT OUTCOME Output sandbox files for the job: https://devel15.cnaf.infn.it:9000/nseGexj8DrrO95L5HP-IfQ have been successfully retrieved and stored in the directory: /tmp/ale_nseGexj8DrrO95L5HP-IfQ ================================================================================ [ale@cream-15 UI]$ cat /tmp/ale_nseGexj8DrrO95L5HP-IfQ/executable.out Process 0 of 1 on wn-smrt-130.farm.nikhef.nl Process 0 of 1 on wn-smrt-130.farm.nikhef.nl Process 0 of 1 on wn-smrt-130.farm.nikhef.nl Process 0 of 1 on wn-smrt-130.farm.nikhef.nl Process 0 of 1 on wn-smrt-130.farm.nikhef.nl Process 0 of 1 on wn-smrt-130.farm.nikhef.nl Process 0 of 1 on wn-smrt-130.farm.nikhef.nl Process 0 of 1 on wn-smrt-130.farm.nikhef.nl pi is approximately 3.1415926544231341, Error is 0.0000000008333410 wall clock time = 10.002291 pi is approximately 3.1415926544231341, Error is 0.0000000008333410 wall clock time = 10.002404 pi is approximately 3.1415926544231341, Error is 0.0000000008333410 wall clock time = 10.002424 pi is approximately 3.1415926544231341, Error is 0.0000000008333410 wall clock time = 10.002564 pi is approximately 3.1415926544231341, Error is 0.0000000008333410 wall clock time = 10.003227 pi is approximately 3.1415926544231341, Error is 0.0000000008333410 wall clock time = 10.002411 pi is approximately 3.1415926544231341, Error is 0.0000000008333410 wall clock time = 10.003075 pi is approximately 3.1415926544231341, Error is 0.0000000008333410 wall clock time = 10.002921 ------ Try with an LCG-CE: [ale@cream-15 UI]$ glite-wms-job-submit --config etc/wmp_devel18.conf -a jdl/prove/mpi.jdl Connecting to the service https://devel18.cnaf.infn.it:7443/glite_wms_wmproxy_server ====================== glite-wms-job-submit Success ====================== The job has been successfully submitted to the WMProxy Your job identifier is: https://devel15.cnaf.infn.it:9000/wCJ29MWeqJnXUlsKTJEbVA ========================================================================== [ale@cream-15 UI]$ glite-wms-job-status https://devel15.cnaf.infn.it:9000/wCJ29MWeqJnXUlsKTJEbVA ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : https://devel15.cnaf.infn.it:9000/wCJ29MWeqJnXUlsKTJEbVA Current Status: Done (Success) Exit code: 0 Status Reason: Job terminated successfully Destination: cedric.scai.fraunhofer.de:2119/jobmanager-pbs-egdteam Submitted: Wed Jun 9 12:25:12 2010 CEST ************************************************************* verify the output: [ale@cream-15 UI]$ glite-wms-job-output https://devel15.cnaf.infn.it:9000/wCJ29MWeqJnXUlsKTJEbVA Connecting to the service https://devel18.cnaf.infn.it:7443/glite_wms_wmproxy_server ================================================================================ JOB GET OUTPUT OUTCOME Output sandbox files for the job: https://devel15.cnaf.infn.it:9000/wCJ29MWeqJnXUlsKTJEbVA have been successfully retrieved and stored in the directory: /tmp/ale_wCJ29MWeqJnXUlsKTJEbVA ================================================================================ [ale@cream-15 UI]$ cat /tmp/ale_wCJ29MWeqJnXUlsKTJEbVA/executable.out Process 0 of 1 on wn05.d-grid.scai.fraunhofer.de Process 0 of 1 on wn02.d-grid.scai.fraunhofer.de Process 0 of 1 on wn07.d-grid.scai.fraunhofer.de Process 0 of 1 on wn04.d-grid.scai.fraunhofer.de Process 0 of 1 on wn06.d-grid.scai.fraunhofer.de Process 0 of 1 on wn08.d-grid.scai.fraunhofer.de Process 0 of 1 on wn33.d-grid.scai.fraunhofer.de Process 0 of 1 on wn03.d-grid.scai.fraunhofer.de pi is approximately 3.1415926544231341, Error is 0.0000000008333410 wall clock time = 10.001729 pi is approximately 3.1415926544231341, Error is 0.0000000008333410 wall clock time = 10.001912 pi is approximately 3.1415926544231341, Error is 0.0000000008333410 wall clock time = 10.001483 pi is approximately 3.1415926544231341, Error is 0.0000000008333410 wall clock time = 10.001141 pi is approximately 3.1415926544231341, Error is 0.0000000008333410 wall clock time = 10.001550 pi is approximately 3.1415926544231341, Error is 0.0000000008333410 wall clock time = 10.002116 pi is approximately 3.1415926544231341, Error is 0.0000000008333410 pi is approximately 3.1415926544231341, Error is 0.0000000008333410 wall clock time = 10.001968 wall clock time = 10.003710