A distributed data management system for data-intensive radio astronomy

The next generation of telescopes, such as the Square Kilometre Array (SKA), will generate orders of magnitude more data than previous instruments, far in excess of current storage and networking system handling abilities. To address this problem, we propose an architecture where data is distributed over several archive sites, each holding only a portion of the overall data, that provides efficient and transparent access to the archive as a whole. This paper describes that architecture in detail and the design and implementation of a prototype system, based on the Integrated Rule-Oriented Data System (iRODS) software.