0.1 Provided Functionality

Version:

• getVersion - returns server internal version

• jobListMatch - finds resources which match job requirements
• jobSubmit - one shot job submission (job registration + job start)
• jobRegister - registers a job for submission
• jobStart - starts a previously registered job
• jobCancel - cancels a job from WMS
• jobPurge - clean-up of job local disk space
• getOutputFileList – returns the URIs of job output files

• getSandboxDestURI - returns the input sandbox destination URI location for a job
• getSandboxBulkDestURI - returns the input sandbox destination URIs location for a group of jobs
• getMaxInputSandboxSize - returns the max size available for input sandbox files
• getFreeQuota – gets the user available local disk space
• getTotalQuota - gets the user total local disk assigned space

• getJDL - returns the JDL of a submitted job (original or registered to LB after pre-processing)

• getProxyReq - gets a request for a Proxy to delegate
• putProxy - put the delegated Proxy into the server machine
• getDelegatedProxyInfo - gets information of user delegated proxy
• getJobProxyInfo - gets information of the delegated proxy used to register/submit a job

• enableFilePerusal - enable Job's File Perusal functionality setting the file name list
• getPerusalFiles - get perusal files previously inserted in the list for perusal functionality (allows inspection of job’s files while the job is running)

• addACLItems – adds jobs GACL file entries
• getACLItems – gets jobs GACL file entries
• removeACLItem – removes jobs GACL file entry

• getJobTemplate - gets a template for a simple job
• getDAGTemplate - gets a template for a DAG
• getCollectionTemplate - gets a template for a collection
• getIntParametricJobTemplate - gets a template for a parametric job with integer parameters
• getStringParametricJobTemplate - gets a template for a parametric job with string parameters

• getTransferProtocols - get the server currently available file transfer protocols

0.2 Job Submission

0.3 Service Interface

http://web.infn.it/gLiteWMS/index.php/techdoc/howtosandguides

The most important purpose of SOA is to achieve loose coupling among interacting components allowing implementation of autonomous services. The loose coupling is achieved through a small set of simple and ubiquitous interfaces, and a set of descriptive messages. In the interfaces only the generic semantics are described: the necessary semantic needed to anticipate possible server behaviour. These interfaces are theoretically available to the global Internet community. The messages represent the way of communication between server and client, and must be descriptive and non-instructive: the server is the only one responsible for the specific design and implementation of the provided service. The messages must be also extensible to allow changes in the service, if needed. To adhere to the SOA model during design and implementation of the WMProxy, the previous guidelines were adopted trying at the same time to provide good service performances to fulfil the user needs for Quality of Service (QoS). Since the WMProxy is a SOAP Web service, the interface is described through the WSDL. A WSDL file represents the available interface needed by the client to access the service. In this file all the operations (services) provided by the WMProxy are described with their argument list, the return value (or structure) and the fault messages. The WSDL file was written following the WS-I Basic Profile. This profile defines a set of Web service specifications that promote interoperability. WS-I Basic Profile addresses the most common problems that could generally affect interoperability, although conformance to it does not completely guarantee the interoperability of a particular service. The profile defines a basic level of conformance based on artifacts. There are three kinds of artifacts, these are:

• MESSAGE
• DESCRIPTION
• REGDATA

0.1 Architecture

• The Fast CGI module provides Common Gateway Interface (CGI) functionality with some other specific features. The most important advantage of Fast CGI is its performance and persistence. Fast CGI can be seen as a new implementation of CGI, designed to overcome CGI's performance problems. The major implementation differences are:
  • Fast CGI processes are persistent: Fast CGI applications are able to serve multiple requests
  • Application instances are spawned/killed dynamically according to demand
  • Fast CGI protocol multiplexes the environment information, standard input, output, and error over a single full-duplex connection.

WMProxy in its default configuration is run as a dynamic Fast CGI application in order to take full advantage of its capacity to accommodate request loads. However, it can be run in any of the available Fast CGI modalities. WMProxy is implemented as a single-threaded application thus ensuring more reliability and robustness than multi-threaded applications; thanks to the Fast CGI process manager the web server maintains a pool of processes that allow improving performances as generally done through multi-threaded application.

• Grid Site provides a module extending the Apache webserver for use within Grid frameworks by adding support for Grid security credentials such as Grid Security Infrastructure (GSI) and Virtual Organization Membership Service (VOMS), and file transfer over HTTPS. It also provides a library for handling Grid Access Control Lists (GACL).

The integration of the WMProxy within the WMS is shown in Figure 1. WMProxy provides a core module performing validation, conversion, environment preparation and information logging for each incoming request, before delivering it to the Workload Manager (WM). WM is the core component of the WMS taking the appropriate actions to satisfy incoming requests. Communication between the WMProxy and the WM occurs through a thread-safe, file-system based queue. The LBProxy, which is used as a state storage of active jobs, is the only external service with which the WMProxy interacts. The LBProxy service provides an optimized access to the Logging and Bookkeeping Service (LB). Other relevant information about processed jobs/requests are stored in the local file system in a reserved area managed by the WMS.

0.2 Security, AuthN & AuthZ

Message exchange between client and server is performed with SOAP/HTTPS.

Authentication of the requesting user is done through the X.509 proxy certificate signed by a trusted Certification Authority (CA), which guarantees that the exposed public key is really owned by the user. The authentication process is handled by the Apache HTTP server by means of the SSL and Grid Site modules. The authenticated request together with information about the checked credential (e.g. expiration time, VOMS extensions), are then passed to WMProxy within the CGI environment. WMProxy does not need to directly manipulate Grid credentials in this phase.

Authorization is implemented through the Grid Access Control List (GACL) library, which is provided by Grid Site for manipulating Access Control List (ACL) files. Authorization can be either Fully Qualified Attribute Name (FQAN) (coarse-grained) or Distinguished Name (DN) based (fine-grained) according to the type of proxy presented by the client.

There are two authorization steps that are performed for an incoming request within the WMProxy:

• Determine whether the requesting user is authorized to use the WMS services.
• Determine whether the requesting user is either the owner of the job or has permissions to manage it. (This step is only performed for operations directly related to a given job)

0.3 Credentials Delegation

0.3.1 Delegation Process

0.3.2 Credentials Renewal

1. For each supported Virtual Organisation a directory must be created, with the following name:

   /etc/glite-wms/<vo name-lowercase>

1. Files installation:
   org.glite.wms.client (version > 3.1.9) installs a template file in the following location:
   

   /etc/vo_template/glite_wmsui.conf.template

   This file should be copied with the name "glite_wmsui.conf" in each directory created in step 1)
   
   for backward compatibility (version <= 3.1.9) old configuration file name approach is still accepted
   

   /etc/vo_template/glite_wms.conf

   This file should be copied with the name "glite_wms.conf" in each directory created in step 1)
2. Edit the configuration files, e.g.:
   /etc/glite-wms/dteam/glite_wms.conf
   and check if desired parameters are correctly set, e.g.:

   [
         WMProxyEndPoints = {"https://ghemon.cnaf.infn.it:7443/glite_wms_wmproxy_server"};
         ErrorStorage = "/var/tmp"; 
         OutputStorage= "/tmp";
         ListenerStorage = "/tmp";
         VirtualOrganisation = "dteam";
         JdlDefaultAttributes=[
                 RetryCount = 3;
                 ShallowRetryCount = 3;
                 rank =  -other.GlueCEStateEstimatedResponseTime;
                 requirements = other.GlueCEStateStatus == "Production" ;
                 myCustomizedAttribute = "foo";
                 SignificantAttributes={"rank","requirements","fuzzyrank"};
                 ]
         ]
    

   For backward compatibility (version <= 3.1.9), WmsClient section of the configuration file will contain above attributes
3. Configuration file attributes description
   A brief summary of all supported attributes for the configuration file follows.
WMProxyEndPoints: list of endpoints URL of the WMProxy to contact.
   ErrorStorage: path of the directory where the UI creates log files.
   OutputStorage: path of the directory where the job OutputSandbox files are stored if not specified by the user through commands options.
ListenerStorage: path of the directory where are created the pipes where the glite_wms_console_shadow process saves the job standard streams for interactive jobs.
VirtualOrganisation: the virtual organisation used to perform the operation
   JdlDefaultAttributes: (version >= 3.1.13) This is a classad attribute and allows the user specifying a set of attributes/values that will be automatically inserted in the submitting JDL
   For backward compatibility (since version < 3.1.13, JdlDefaultAttributes section not present) following attriubtes will be taken in consideration:
   Rank: default value for the ranking expression in the JDL.
   Requirements: default value for the requirements expression in the JDL (a further condition on the submitting requirements will be added)
   RetryCount: default value for the RetryCount attribute in the JDL.
   MyProxyServer: MyProxy server address. To be set only if proxy renewal has to be enabled for all requests.
   LBAddress: this attribute forces the WMProxy server to register jobs and/or log events to the specified LB.
4. Configuration File Priority
   Attributes listed configuration file described in step 2) may be overridden by (in order of priority):
   
   --config option of WMProxy Client commands
   $HOME/.glite/<vo name>/glite_wmsclient.conf
   $GLITE_WMS_CLIENT_CONFIG
   
   Please note that the attributes not present in the customized file, yet listed in file described in step 2), are inherited by final configuration attributes. If the user does not want to inherit a specific attribute, he may assign a null/empty value (depending on the type).
   Please notice that default configuration files will be parsed anyway: default attributes will be taken in consideration when not found on customized configuration file.
5. Other client configuration steps:
   • Make sure $GLOBUS_LOCATION/lib and $GLITE_LOCATION/lib are included in the LD_LIBRARY_PATH
   • Add $GLITE_LOCATION/bin to your PATH

0.1 Client configuration

C++ WMProxy client commands behavior may be configured by editing a proper configuration file. Here follow the steps needed to properly configure file location on client side:

1 For each supported Virtual Organisation a directory must be created, with the following name:

/etc/glite-wms/<vo name-lowercase>
(e.g. for dteam /etc/glite-wms/dteam)

1. Files installation:

• org.glite.wms.client (version > 3.1.9) installs a template file in the following location:

    /etc/vo_template/glite_wmsui.conf.template          

This file should be copied with the name "glite_wmsui.conf" in each directory created in step 1)

• for backward compatibility (version <= 3.1.9) old configuration file name approach is still accepted

    /etc/vo_template/glite_wms.conf          

This file should be copied with the name "glite_wms.conf" in each directory created in step 1)

1 WMProxy Service

1.1 About WMProxy

WMProxy is a new component to access the EMI Workload Management System (WMS).

The need of such a component, besides adhering to the emerging design methodology called Service Oriented Architecture (SOA), is to improve the performance of existing similar components, to handle more efficiently a large number of requests for job submission and control and also to provide additional features.

The WMProxy is implemented as a Web service. The Web is considered a successful technology mostly because of its simplicity and ubiquity. A Web service allows us to take advantage of the benefits of the Web, not only to provide information, but also to offer services to a greater community of possible users.

Since the European Middleware Initiative (EMI) Middleware is a composition of Grid services provided by different vendors and/or operated by different organizations, the Web services technology is a powerful means to achieve service interoperability and allow easier compliance with emerging standards such as Open Grid Services Architecture (OGSA) and Web Services Resource Framework (WSRF).

-- FabioCapannini - 2011-11-04