FTS3 / WebFTS – A Powerful File Transfer Service for Scientific Communities

Abstract FTS3, the service responsible for globally distributing the majority of the LHC data across the WLCG infrastructure, is now available to everybody. Already integrated into LHC experiment frameworks, a new web interface now makes the FTS3's transfer technology directly available to end users. In this article we describe this intuitive new interface, “WebFTS”, which allows users to easily schedule and manage large data transfers right from the browser, profiting from a service which has been proven at the scale of petabytes per month. We will shed light on new development activities to extend FTS3 transfers capabilities outside Grid boundaries with support of non-Grid endpoints like Dropbox and S3. We also describe the latest changes integrated into the transfer engine itself, such as new data management operations like deletions and staging files from archive, all of which may be accessed through our standards-compliant REST API. For the Service Managers, we explain such features as the service's horizontal scalability, advanced monitoring and its “zero configuration” approach to deployment made possible by specialised transfer optimisation logic. For the Data Managers, we will present new tools for management of FTS3 transfer parameters like limits for bandwidth and max active file transfers per endpoint and VO, user and endpoint banning and powerful command line tools. We finish by describing our effort to extend WebFTS's captivating graphical interface with support of Federated Identity technologies, thus demonstrating the use of grid resources without the burden of certificate management. In this manner we show how FTS3 can cover the needs of wide range of parties from casual users to high-load services. The evolution of FTS3 is addressing technical and performance requirements and challenges for LHC Run 2, moreover, its simplicity, generic design, web portal and REST interface makes it an ideal file transfer scheduler both inside and outside of HEP community.