News from the High Energy Physics Research Computing Group at the University of Victoria

The Dynamic Federation project (DynaFed)  being developed at CERN is intended to federate any number of different types of storage endpoints allowing users to read or write data transparently and efficiently. How it works in a nutshell: A client sends a COPY, GET, PUT, DELETE HTTP request to the DynaFed instance's URL. DynaFed decides which amongst the storage endpoints it federates is the "best" one to handle the request, sorted according to geographic location and available free space. The client receives a 302 redirect link pointing to this storage endpoint with the protocol and authentication tokens necessary for the transaction. The client re-sends the HTTP request directly to this storage endpoint. As we can see, after the redirect link is provided, DynaFed is out of the loop and as such is unaware of whether the client-storage endpoint transaction is successful or not. So while we cannot get this valuable information from Dynafed, we instead are interested i

The Dynamic Federation project (DynaFed)  is a good way to federate existing Grid storage as well as making S3 based object stores available for Grid storage. To access DyanFed, the webdav/https protocol is used. The default authentication for accessing the storage backend is based on X.509 certificates and voms proxies. In that case, the user has to use grid tools to contact a voms server to authenticate and create a X.509 proxy locally on the client machine. This proxy can then be used by tools that support it to list and access files behind DynaFed. However, that kind of authentication must be supported by the tool used. It will for example not work using a web browser. In addition to the build-in authentication, DynaFed also supports Python based scripts to handle the authentication. One of the standard Grid authentication method is to authenticate against a so called grid-mapfile which is usually created by tools like  edg-mkgridmap . We implemented this grid-mapfile lookup i