Poster: Scalable Cooperative Caching with RDMA-Based Directory Management for Large-Scale Data Processing
2012
Cooperative caching provides an extensive virtual file cache by combining file caches on all nodes. We propose a novel cooperative caching method that addresses two problems of existing methods: lack of utilization of high-throughput, low-latency remote direct memory access (RDMA) and low scalability against concentrated requests for a particular cache block. The proposed method uses only RDMA to provide cooperative caching, namely, to lookup a cache block location in the cache directory, transfer a cache block between nodes, etc. In addition, nodes are partitioned into groups and managed semi-independently to achieve high scalability through mitigating access concentration on a particular node and to reduce data transfer for cache management. We implemented the proposed method as a library and evaluated it on the K computer and Fujitsu PRIMEHPC FX10. Results showed speedups for sequential, random and strided accesses.
Keywords:
- Correction
- Source
- Cite
- Save
- Machine Reading By IdeaReader
8
References
0
Citations
NaN
KQI