(r8) ClientApiCppDoc < CREAM

Tags: , view all tags
---+ [ WORK IN PROGRESS ]

---++ Preliminaries: build machine configuration

Before developing a CREAM's CLI you must configure properly a SL5_64bit machine with EPEL repositories and basic EMI1 software. Instructions are here:

<a target="_blank" href="SystemAdministratorGuideForEMI1#1_3_Installation" id="internal-source-marker_0.3138147739227861" title="System Administrator Guide for EMI1">http://wiki.italiangrid.org/twiki/bin/view/CREAM/SystemAdministratorGuideForEMI1#1_3_Installation</a> .

Here are summarized all the steps described in the link above plus some more rpm to install and other settings:

<verbatim style="padding-left: 30px;">\rm /etc/yum.repos.d/dag.repo
rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-4.noarch.rpm
wget http://emisoft.web.cern.ch/emisoft/dist/EMI/1/sl5/x86_64/base/emi-release-1.0.0-1.sl5.noarch.rpm
yum -y install ./emi-release-1.0.0-1.sl5.noarch.rpm
wget http://repository.egi.eu/sw/production/cas/1/current/repo-files/egi-trustanchors.repo -O /etc/yum.repos.d/egi-trustanchors.repo
wget http://forge.cnaf.infn.it/frs/download.php/101/disable_yum.sh
chmod +x disable_yum.sh
./ disable_yum.sh
yum clean all; yum update all
yum -y install yum-protectbase.noarch
yum -y install ca-policy-egi-core
yum -y install glite-ce-cream-client-api-c.x86_64
yum -y install glite-lbjp-common-gsoap-plugin
yum -y install boost-devel
yum -y install voms-devel voms-clients</verbatim>

Then, download, build and install gSOAP 2.7.6b from this URL:

<a target="_blank" href="https://etics-repository.cern.ch/repository/download/registered/externals/gsoap/2.7.6b/src/" title="gSOAP 2.7.6b Source code">https://etics-repository.cern.ch/repository/download/registered/externals/gsoap/2.7.6b/src/</a>

Now create these symlinks:

<verbatim style="padding-left: 30px;">ln -s /usr/lib64/liblog4cpp.so.4 /usr/lib64/liblog4cpp.so
ln -s /usr/lib64/libclassad.so.1 /usr/lib64/libclassad.so
ln -s /usr/lib64/libclassad_ns.so.1 /usr/lib64/libclassad_ns.so
ln -s /usr/lib64/libcares.so.2 /usr/lib64/libcares.so
ln -s /usr/lib64/libglobus_common.so.0 /usr/lib64/libglobus_common.so
ln -s /usr/lib64/libglobus_gsi_callback.so.0 /usr/lib64/libglobus_gsi_callback.so
ln -s /usr/lib64/libglobus_gssapi_gsi.so.4 /usr/lib64/libglobus_gssapi_gsi.so
ln -s /usr/lib64/libglobus_gsi_credential.so.1 /usr/lib64/libglobus_gsi_credential.so
ln -s /usr/lib64/libltdl.so.3 /usr/lib64/libltdl.so
ln -s /usr/lib64/libglobus_oldgaa.so.0 /usr/lib64/libglobus_oldgaa.so
ln -s /usr/lib64/libglobus_proxy_ssl.so.1 /usr/lib64/libglobus_proxy_ssl.so
ln -s /usr/lib64/libglobus_openssl_error.so.0 /usr/lib64/libglobus_openssl_error.so
ln -s /usr/lib64/libglobus_gsi_proxy_core.so.0 /usr/lib64/libglobus_gsi_proxy_core.so
ln -s /usr/lib64/libglobus_gsi_cert_utils.so.0 /usr/lib64/libglobus_gsi_cert_utils.so
ln -s /usr/lib64/libglobus_gsi_sysconfig.so.1 /usr/lib64/libglobus_gsi_sysconfig.so
ln -s /usr/lib64/libglobus_openssl.so.0 /usr/lib64/libglobus_openssl.so

</verbatim>

---+ CLI Development
---++ Building of the example source code (see attachments)

In the attachments you can find small examples that explain how to use the CREAM Client API C++. You can build them by issuing the commands:
---++++++ COMPILATION:
<pre><verbatim style="padding-left: 30px;">
g++ -c -I/usr/local/include <EXAMPLE>.cpp</verbatim>
</pre>
---++++++ LINKING:
<pre><verbatim style="padding-left: 30px;">
g++ -o <EXAMPLE> <EXAMPLE>.o -lglite_security_gsoap_plugin_276b \
	-lglite_ce_cream_client_soap -lglite_ce_cream_client_util \
	/usr/local/lib/libgsoap++.a -lboost_thread -llog4cpp \
	-lvomsapi -lclassad_ns -lgridsite -lboost_regex</verbatim>
</pre>

---++ Quick overview of the API

Currently, the C++ API implements the following operations:

   * JobRegister
   * JobStart
   * JobCancel
   * JobSuspend
   * JobResume
   * JobPurge
   * JobList
   * JobInfo
   * JobStatus
   * ProxyDelegation
   * ProxyRenew
   * ServiceInfo
   * Enable/Disable JobSubmission
   * Query JobSubmission enable status
The API architecture is founded on a super and abstract class <a target="_blank" href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1AbsCreamProxy.html" title="AbsCreamProxy">AbsCreamProxy</a> that exposes 3 relevant public methods: <span style="font-family: 'Courier New', courier, monaco, monospace; font-size: 16px;"><a target="_blank" href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1AbsCreamProxy.html#1701485b30deeb2b0b6444286d1d6126" title="AbsCreamProxy::setCredential(...)">AbsCreamProxy::setCredential(...)</a></span>, <span style="font-family: 'Courier New', courier, monaco, monospace; font-size: 16px;"><a target="_blank" href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1AbsCreamProxy.html#4dd4f8490789fed0bce2b7b971c3aa3f" title="AbsCreamProxy::setConnectionTimeout(...)">AbsCreamProxy::setConnectionTimeout(...)</a></span>, <span style="font-family: 'Courier New', courier, monaco, monospace; font-size: 16px;"><a target="_blank" href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1AbsCreamProxy.html#8cf7a83ee8e32da416a5481879e68a6f" title="AbsCreamProxy::execute(...)">AbsCreamProxy::execute(...)</a></span>. The method <span style="font-family: 'Courier New', courier, monaco, monospace; font-size: 16px;">execute(...)</span> is pure virtual.

There're several <a target="_blank" href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1AbsCreamProxy.html" title="AbsCreamProxy">AbsCreamProxy</a>'s subclasses, one for each operation mentioned above: CreamProxy _Start, CreamProxy _Register, CreamProxy _Cancel, and so on. Each of them implements its own execute(...) method, specific for the kind of operation the subclass itself represents; this implementation is responsible for the connection to the remote Web Service (CREAM), sending the SOAP request, receiving the SOAP response and unserializing it and hides the user from any SOAP communication and authentication detail. All the subclasses have protected constructors, so the developer cannot directly create a subclass, but she/he must use a special factory named <a target="_blank" href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1CreamProxyFactory.html" title="CreamProxyFactory">CreamProxyFactory</a>. For each operation the factory has a make_&lt;OPERATION_NAME&gt;(...) method. The factory's 'make' methods return a pointer to an <a target="_blank" href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1AbsCreamProxy.html" title="AbsCreamProxy">AbsCreamProxy</a> object (polimorphic object) that allows the developer to handle more instances of different subclasses in the same way (for example to put all of them in a container). The 'make' methods take different parameters depending on the kind of operation is performed by the returned object.

When the user got a pointer to <a target="_blank" href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1AbsCreamProxy.html" title="AbsCreamProxy">AbsCreamProxy</a> she/he has to invoke the setCredential(...) method in order to set the authentication credentials, optionally invoke the setConnectionTimeout(...) in order to set a maximum socket timeout for the connections with the CREAM service, and then must invoke execute(...) to actually communicate with the service (send SOAP request, receive and parse SOAP response).

It is clear that the developer has to know nothing about subclasses: she/he just uses factory's make methods and the two methods setCredential(...) and execute(...); then the subclasses are not documented at all. The API documentation only describes the <a target="_blank" href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1AbsCreamProxy.html" title="AbsCreamProxy">AbsCreamProxy</a>, the factory and the data structures to pass as argument of the 'make' methods of the factory.

At the end, the user must delete the pointer to <a target="_blank" href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1AbsCreamProxy.html" title="AbsCreamProxy">AbsCreamProxy</a> in order to free the heap-memory that the factory allocated.

---++ Step by step Job Submission explanation
---+++ Proxy Delegation

In order to register a job, the user must previously delegate a proxy into the remote CREAM CE service. The procedure is a matter of a few steps:
   * define an arbitrary string containing a delegation identifier (it will be used later as job parameter for registration of one or more jobs); *please consider the importance of re-use a single delegation identifier for multiple job submissions; in fact the delegation process takes some time and can be a large overhead for the submission process*
   * create an instance of a subclass of <a href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1AbsCreamProxy.html">AbsCreamProxy</a> with the invocation of the static method <a href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1CreamProxyFactory.html#0fa3fa0e5cadce88ecdfedda6fc9f67e">CreamProxyFactory::make_CreamProxyDelegate(...)</a>
   * invoke the setCredential(...) method on the instance created above
   * invoke the execute(...) method on the instance created above
   * delete the AbsCreamProxy instance
Download the example code in attach ( <a href="http://wiki.italiangrid.org/twiki/pub/CREAM/ClientApiCppDoc/tryDelegate.cpp">tryDelegate.cpp</a> ) and build it with the instructions described above.

---+++ Job Registration

The steps for the JobRegister operation are:
   * Prepare one or more strings containing the JDL descriptions of one or more jobs to submit
   * Obtain a delegation identifier of a pre-delegated proxy (see Proxy Delegation)
   * Prepare one <a href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1JobDescriptionWrapper.html">JobDescriptionWrapper</a> object for each job to submit to the CREAM CE
   * Put all the JobDescriptionWrapper objects in a list (see below)
   * Create an instance of a subclass of AbsCreamProxy with the invocation of the static method <a href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1CreamProxyFactory.html#81cc3d718ca6b6a7424a7e9ea219bd5d">CreamProxyFactory::make_CreamProxyRegister(...)</a>
   * Invoke the setCredential(...) method on the instance created above
   * Invoke the execute(...) method on the instance created above
   * Process the output
   * Delete the AbsCreamProxy instance
The input and output arguments of the <a href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1CreamProxyFactory.html#81cc3d718ca6b6a7424a7e9ea219bd5d">CreamProxyFactory::make_CreamProxyRegister(...)</a> are a bit more complicated than in the case of the Proxy Delegation. As described in the API reference, the input and output arguments are pointers to:
   * <a href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1AbsCreamProxy.html#0f3f03bfda3502162afe54395a11ece7">AbsCreamProxy::RegisterArrayRequest </a>
   * <a href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1AbsCreamProxy.html#c04571bf86adcdcc0a77d2825e843b64">AbsCreamProxy::RegisterArrayResult </a>
They are typedef for (respectively):
   * std::list&lt; JobDescriptionWrapper* &gt;
   * std::map&lt; std::string, boost::tuple&lt; JobIdWrapper , std::string&gt; &gt;
The first one is simply a C++ STL list where the user has to insert pointers to JobDescriptionWrapper objects; the second one is a complex structure based on the <a href="http://www.boost.org/doc/libs/1_32_0/libs/tuple/doc/tuple_users_guide.html">boost::tuple</a> library. Each JobDescriptionWrapper object is built with a JobDescription identifier (an arbitrary string chosen by user) and other parameters that can been seen in the example source code <a href="http://wiki.italiangrid.org/twiki/pub/CREAM/ClientApiCppDoc/tryRegister.cpp">tryRegister.cpp</a>. After the invocation of the execute(...) method, the second argument (passed to the CreamProxyFactory::make_CreamProxyRegister(...) function as output parameter) will be filled as follows:
   * The key of the map is the JobDescription identifier (as defined by the user)
   * The value corresponding to the key is a boost::tuple
this tuple groups three elements:
   * The result of the operation, JobIdWrapper::RESULT
   * The Job (represented by a JobIdWrapper object, use the getCreamJobID method to obtain the Cream Job ID that you will need to start later)
   * A string representing an error message (non empty if the first element is different than JobIdWrapper::OK)
As shown in the example source code <a href="http://wiki.italiangrid.org/twiki/pub/CREAM/ClientApiCppDoc/tryRegister.cpp">tryRegister.cpp</a>, a mandatory argument is autostart. In the example it is set to false; this means that the job is ONLY REGISTERED and NOT STARTED. Below it is explained the usefulness of autostart set to false. If she/he needs to start the job immediately after registration she/he can set to true the autostart parameter. An array of properties (implemented as a std::map &lt; string, string &gt;, i.e. couples key -&gt; value ) is embedded in the JobIdWrapper object returned by JobRegister operation. At the moment the only relevant property returned by CREAM is the remote path in the CE the user can send its InputSandbox to, and name 'CREAMInputSandboxURI' (see again the example source code).

   * Example code for *single job submission* is <a href="http://wiki.italiangrid.org/twiki/pub/CREAM/ClientApiCppDoc/tryRegister.cpp">here</a>
   * Example code for *multiple job registration* with a single remote call is <a href="http://wiki.italiangrid.org/twiki/pub/CREAM/ClientApiCppDoc/tryMultipleRegister.cpp">here</a>

---+++ Job start

In the previous example, jobs are simply registered in the CREAM CE (autostart is set to false). This is useful if the user needs to do something between job registration and job start (e.g.: sending an input sandbox in the remote path specified by the CREAM CE). To explicitly start a job the user must:
   * Obtain the Cream Job IDs of the jobs to start (as result of JobRegister operation)
   * Build the JobIdWrapper object representing the jobs to start (one for each Cream Job ID to start)
   * Define a time range (specified by two variables fromDate and toDate) which will allow her/him to only start the jobs that were registered in this time range [fromDate, toDate]
   * Build a <a href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1JobFilterWrapper.html">JobFilterWrapper</a> with all the JobIdWrapper objects representing the jobs to start, and the time range
   * Create the proper subclass of AbsCreamProxy by invoking the static factory method <a href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1CreamProxyFactory.html#1f514ff84dafcc08b2387c5e95d21772">CreamProxyFactory::make_CreamProxyStart(...)</a>
   * Invoke, as usual, the setCredential(...) and execute(...) methods on the previously created AbsCreamProxy's instance
To start all her/his jobs, the user has to use an empty vector of JobIdWrapper objects as argument of the JobFilterWrapper's costructor.

---+++ Cancelling, Suspending, Resuming and Purging jobs

The code for these four operations is basically the same. The user has to
   * prepare a JobFilterWrapper object (the same for the JobStart operation) and pass a pointer to it as first argument of the static method <a href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1CreamProxyFactory.html#b120422fa7a45dfab12df648bdc32df4">CreamProxyFactory::make_CreamProxyCancel(...)</a>/<a href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1CreamProxyFactory.html#80c1b540febb05e8ffa867b923d51b04">CreamProxyFactory::make_CreamProxySuspend(...)</a>/<a href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1CreamProxyFactory.html#7e15a44cfa386a3902c9dffb26f3f4a8">CreamProxyFactory::make_CreamProxyResume(...)</a>/<a href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1CreamProxyFactory.html#b14bb8b6739a69bab22a9879750b27a4">CreamProxyFactory::make_CreamProxyPurge(...)</a>.
The second argument of the factory methods is a pointer to a <a href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1ResultWrapper.html">ResultWrapper</a> object that will be filled-in with the results sent back by CREAM.

Like in the JobStart case, to cancel/suspend/resume/purge all her/his jobs, the user has to use an empty vector of JobIdWrapper objects as argument of the JobFilterWrapper's costructor.

The source code example is <a href="http://wiki.italiangrid.org/twiki/pub/CREAM/ClientApiCppDoc/tryCancel.cpp">here</a>. Please note that it is only for JobCancel operation; adapting it for the other three operations is straightforward.

---+++ Listing jobs in a CREAM CE

The code for listing all the jobs submitted to a CREAM CE is very simple; a few simple parameters are needed for the factory <a href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1CreamProxyFactory.html#55c63ef933f274bbcfa7f29067043b7e">CreamProxyFactory_makeCreamProxyList(...)</a>. A look at the <a href="http://wiki.italiangrid.org/twiki/pub/CREAM/ClientApiCppDoc/tryList.cpp">source code</a> should be enough to understand what the user has to do.

---+++ Getting Information about jobs submitted to a CREAM CE

A user can invoke two operations to get information on one or more jobs submitted to a CREAM CE: JobInfo and JobStatus. The latter is quicker but provides less information than the former. In both cases the user must prepare a JobFilterWrapper object that collects all the identifier strings of the jobs to query and the conditions the jobs must satisfy in order to be included in the result. For example a user might need the query status of all jobs submitted between 8:00am and 15:00am of a particular day (if they were not purged out yet). Or she/he might need information on all the jobs that are in the "RUNNING" OR "DONE-OK" status... and so on. Please see the documentation of JobFilterWrapper to see what filters a user can define.

---++++ Fast info query (JobStatus)

With the invocation of the JobStatus remote operation CREAM will return a minimal set of information about the jobs: the current status of the job, the timestamp of the last status change, the exit code of the job and the failure reason (if the job finished or aborted). To query the states of some jobs, the user must prepare a JobFilterWrapper object and fill it with JobIdWrapper objects (one for each job to query) and with other constraints to select a particular set of jobs (see the JobFilterWrapper documentation).

As usual a user can query for the states of all her/his jobs; this can be achieved by using an empty vector of JobIdWrapper objects as argument of the JobFilterWrapper's costructor. See the <a href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1CreamProxyFactory.html#10f1cbc006fd1a9eaba6beace0a46c19">CreamProxyFactory::make_CreamProxyStatus(...)</a> and <a href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1JobStatusWrapper.html">JobStatusWrapper</a> documentation for more details about the structure of the information returned by CREAM.

As usual the <a href="http://wiki.italiangrid.org/twiki/pub/CREAM/ClientApiCppDoc/tryStatus.cpp">example code</a> is the best way to explain how it works.

---++++ Slow info query (JobInfo)

To get complete information about jobs the user must invoke the remote call JobInfo. The procedure is much similar to the fast call JobStatus: the user has to prepare a JobFilterWrapper object to select jobs and filters; then the result will be put in a <a href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1JobInfoWrapper.html">JobInfoWrapper</a> structure instead of a JobStatusWrapper. The data structure containing the output of the JobInfo operation is very similar to that one for the JobStatus operation. See the <a href="http://www.italiangrid.org/CREAM-CLIENT-API/classglite_1_1ce_1_1cream__client__api_1_1soap__proxy_1_1CreamProxyFactory.html#c89548fcca3ded7dd48c2c31745cf8eb">CreamProxyFactory::make_CreamProxyInfo(...)</a> and JobInfoWrapper documentation for more details about the structure of the information returned by CREAM.

Example <a href="http://wiki.italiangrid.org/twiki/pub/CREAM/ClientApiCppDoc/tryInfo.cpp">source code</a>.

[ TO BE CONTINUED ... ]

-- Main.AlviseDorigo - 2011-06-14
Attachments
Topic attachments
I	Attachment	Action	Size	Date	Who
cpp	tryCancel.cpp	manage	4.3 K	2011-06-20 - 08:29	AlviseDorigo
cpp	tryDelegate.cpp	manage	1.3 K	2011-06-15 - 07:03	AlviseDorigo
cpp	tryInfo.cpp	manage	4.6 K	2011-06-20 - 08:29	AlviseDorigo
cpp	tryList.cpp	manage	1.2 K	2011-06-20 - 08:29	AlviseDorigo
cpp	tryMultipleRegister.cpp	manage	4.7 K	2011-07-06 - 12:55	AlviseDorigo
cpp	tryRegister.cpp	manage	3.6 K	2011-06-15 - 07:03	AlviseDorigo
cpp	tryServiceInfo.cpp	manage	1.6 K	2011-06-20 - 08:29	AlviseDorigo
cpp	tryStart.cpp	manage	4.6 K	2011-06-15 - 07:03	AlviseDorigo
cpp	tryStatus.cpp	manage	4.5 K	2011-06-20 - 08:29	AlviseDorigo
Topic revision: r8 - 2011-07-11 - AlviseDorigo
Account
- Log In
~~Edit~~
~~Attach~~