Monitoring Dynafed with ELK
The Dynamic Federation project (DynaFed) being developed at CERN is intended to federate any number of different types of storage endpoints allowing users to read or write data transparently and efficiently. How it works in a nutshell: A client sends a COPY, GET, PUT, DELETE HTTP request to the DynaFed instance's URL. DynaFed decides which amongst the storage endpoints it federates is the "best" one to handle the request, sorted according to geographic location and available free space. The client receives a 302 redirect link pointing to this storage endpoint with the protocol and authentication tokens necessary for the transaction. The client re-sends the HTTP request directly to this storage endpoint. As we can see, after the redirect link is provided, DynaFed is out of the loop and as such is unaware of whether the client-storage endpoint transaction is successful or not. So while we cannot get this valuable information from Dynafed, we instead are interested i