Coordination of Access to Large-scale Datasets in Distributed Environments

The data needs of scientific as well as commercial applications has increased drastically over the recent years. This increase in the demand for large-scale data processing has necessitated collaboration and sharing of data collections among the world’s leading education, research, and industrial institutions and use of distributed resources owned by collaborating parties. In a collaborative distributed computing environment, data is often not locally accessible and has thus to be remotely retrieved, processed, and stored. While traditional distributed system solutions work well for computation that requires limited data handling, they may fail in unexpected ways when the computation accesses, creates, and moves large amounts of data especially over wide-area networks. In this chapter, we provide some state of the art solutions for handling and coordination of access to large-scale datasets in widely distributed computing environments.