Glint Version 2 Enters Production

After several months of prototyping and development Glint version 2.0 (Glint v2) has entered production. Glint v2 is a standalone web service inspired by Colin Leavett-Brown and Ron Desmarais' original glint service (Glint v1). The idea of glint was to allow for image replication across multiple openstack clouds using a simple interface instead of manually downloading and uploading images to new locations. Version 2 differs from the original in that it is a dedicated web service instead of an extension of the Openstack Horizon dashboard. Unfortunately the Openstack developers had a different philosophy regarding image and repository management and decided not to accept glint v1 as a proprietary module.

The Openstack 6 month development cycle made it unreasonable for a small group like UVic's HEPRC group to maintain Glint v1 as an openstack plugin. Instead a new version of the service was conceived, a version that is independent of Openstack and one day could even be expanded to include different types of clouds. Glint v2 eventually introduced several features not available in the original version:

 - x509 certificate based authentication
 - Image conflict detection
 - Direct Image upload/download

An admin of Glint v2 is able to add users based on their x509 certificate DN allowing for secure SSL authentication and communication with the service negating any need for password management. Version 2 also can help provide some support for an Openstack cloud admin by detecting duplicate image names and identical images with differing names that may have an effect on batch production using a service such as cloud scheduler. A user is also able to upload an image directly (or from a web link) to any number of openstack projects which have been made available in glint substantially reducing the work required to propagate a brand new image across several clouds. Finally, the images in glintv2 are available for direct download if you'd like a copy of an image to move to another type cloud. We hope that one day glint will be compatible with many cloud types such as Amazon's EC2, Google's Compute Engine, and Microsoft's Azure allowing easy image management across all popular cloud services.

For more information about Glint v2 please see the Glint v2 github page or the Glint v2 User's Guide.

Comments

Post a Comment

Popular posts from this blog

Grid-mapfile based authentication for DynaFed

The Dynamic Federation project (DynaFed)  is a good way to federate existing Grid storage as well as making S3 based object stores available for Grid storage. To access DyanFed, the webdav/https protocol is used. The default authentication for accessing the storage backend is based on X.509 certificates and voms proxies. In that case, the user has to use grid tools to contact a voms server to authenticate and create a X.509 proxy locally on the client machine. This proxy can then be used by tools that support it to list and access files behind DynaFed. However, that kind of authentication must be supported by the tool used. It will for example not work using a web browser. In addition to the build-in authentication, DynaFed also supports Python based scripts to handle the authentication. One of the standard Grid authentication method is to authenticate against a so called grid-mapfile which is usually created by tools like  edg-mkgridmap . We implemented this grid-mapfile...
Read more

Monitoring Dynafed with ELK

Image
The Dynamic Federation project (DynaFed)  being developed at CERN is intended to federate any number of different types of storage endpoints allowing users to read or write data transparently and efficiently. How it works in a nutshell: A client sends a COPY, GET, PUT, DELETE HTTP request to the DynaFed instance's URL. DynaFed decides which amongst the storage endpoints it federates is the "best" one to handle the request, sorted according to geographic location and available free space. The client receives a 302 redirect link pointing to this storage endpoint with the protocol and authentication tokens necessary for the transaction. The client re-sends the HTTP request directly to this storage endpoint. As we can see, after the redirect link is provided, DynaFed is out of the loop and as such is unaware of whether the client-storage endpoint transaction is successful or not. So while we cannot get this valuable information from Dynafed, we instead are interested i...
Read more

ACAT 2017

The 18th International Workshop on Advanced Computing and Analysis Techniques in Physics Research ( ACAT ) took place in Seattle this week. We presented our work on integrating the dynamic web federation into HEP computing as a poster . The conference focused on the use of machine learning algorithm in physics research with contributions from industry offering effective computing technology to execute workflows employing deep neural nets. These technologies offer solutions to the computing issues the field is facing in light of a great increase in data with a constant computing budget. When the LHC experiments were planned it was assumed that Dennard Scaling would solve this problem for us, it has become clear that this is not the case. It was shown that generative adversarial neural nets may be used to do do simulation, and that supervised learning may provide options for triggering and reconstruction. In some places these technologies are already used. nVidia, Microsoft, and DW...
Read more