CooLSM: Distributed and Cooperative Indexing Across Edge and Cloud Machines

Author(s): Mittal, Natasha | Advisor(s): Nawab, Faisal | Abstract: Emerging edge applications such as the Internet of Things (IoT) and Industry 4.0require stringent real-time guarantees. This makes it infeasible to rely solely on farawaycloud nodes for these applications. Similarly, edge machines are less capable andreliable than cloud machines. To balance the trade-off of edge and cloud computing,we propose a data infrastructure that spans both edge and cloud machines, that we calledge-cloud computing. We tackle one of the fundamental data management challengesin edge-cloud computing, the problem of data indexing. We propose Cooperative LSM[3] (CooLSM), a distributed Log-Structured Merge Tree that is designed to overcomethe unique challenges of edge-cloud indexing such as machine and workloadheterogeneity and the communication latency asymmetry between the edge and thecloud. To tackle these challenges, CooLSM deconstructs the structure of LSM into itsbasic parts. This deconstruction allows a better distribution and placement of resourcesacross edge and cloud devices. For example, append-specific functionality is managedat the edge to ensure appending and serving data in real-time, whereas resource-intensiveoperations such as compaction are managed at the cloud where morecomputing resources are available.