SPARCLE: Stream Processing Applications over Dispersed Computing Networks

In this paper, we propose SPARCLE, a novel scheduling system offering network-aware polynomial-time task assignment and resource allocation algorithms for stream processing applications in dispersed computing networks. In particular, we address two major challenges. The first one concerns the assignment of both computation and transport tasks comprising a stream processing application to computing nodes and communication links of the network, respectively, to maximize the application’s processing rate. The second one concerns the resource allocation of multiple stream processing applications to satisfy their requested QoS. Our experimental results on a real image stream processing application and extensive simulations show that SPARCLE can increase the application’s processing rate by 9 times and 3 times, compared to the cloud computing case and state-of-the-art algorithms, respectively.