Software-addressable optical accelerators for data-intensive applications in cluster-computing platforms

We present a control plane architecture to enable software-addressable optical acceleration from the application layer. The architecture is experimentally examined on a cluster-computing test-bed by enabling physical layer optical multicasting on-demand for the application layer to achieve non-blocking performance. Introduction With the growing interest in Big Data analysis and cloud computing, new challenges and opportunities arise in the design and control of the interconnection networks that supports these cluster-computing platforms. Software-defined networking (SDN) paradigms that provide central network management and dynamic configuration of the architecture are considered as potential future directions to address these challenges. Furthermore, optical interconnect technologies that can provide high bandwidths and lower energy consumption are becoming a part of the future interconnection networks efforts. Hybrid network architectures that combine Ethernet and optical circuit switching has been proposed for point-to-point optical connectivity in data center networks [1, 2]. In these architectures, optical space switches (OSS) are used to connect the Top-of-Rack (ToR) switches and a SDN-based control plane to manage the traffic between the two networks. The optical links connected by the OSS are primarily utilized for large packet transmission (Elephant flows) since the 25ms switching speed does not allow for switching individual or small groups of packets (Mouse flows). Data centers support a wide variety of applications that generate flows of all sizes, thus further investigations are required to improve such hybrid architectures. Big data analysis generally involves parallel data processing techniques such as Hadoop, which uses a distributed file system and a MapReduce [3] type algorithm for data analysis. These operations require large data transfers with richer traffic patterns (Multi/In-cast) between the clusters (i.e. ToR switches) that introduce heavy traffic to the interconnection network. The shuffle stage of MapReduce and the data storage in the distributed file system require Incast (many-to-one) traffic delivery. Multicast (one-to-many) traffic delivery is also required in various applications including data replication (for improving the reliability of distributed file systems), parallel database join operation, data dissemination in virtual machine (VM) provisioning and in-cluster software updating, as well as data analysis in broadcast phase of Orchestra [4] (a control architecture to manage intraand intertransfer activities). In current platforms, these patterns are managed either by sequence of unicast transmissions or more advanced software solutions such as peer-to-peer method. These methods are naturally inefficient since multiple copies of the same data are transferred on the network. For instance, Orchestra system transmits 12 copies of the same data. Physical layer transparent data replication using passive optical splitters by setting up a multicast tree between the source and the destinations [5] can provide a significantly more efficient multicasting operation since only one copy of data is sent. Optical modules can clearly provide functionalities beyond point-to-point data transmission. However, due to the complexities in configuring optical devices and the circuit-based nature of optical switching, integration of optics with current network architectures is challenging. Cross-layer architectures can potentially overcome these complexities and provide optical functionalities more seamlessly to the application layer. In this work, we present a control plane architecture for a hybrid interconnection networks that can accelerate data delivery for data-intensive applications. The control plane architecture is experimentally examined on our 10G Hybrid Cluster-Computing Test-bed with a demonstration of physical layer optical multicasting. The implemented control plane employs a messaging system, a resource allocation algorithm, and APIs to control the optical resources and manage the network. Architecture Fig. 1 demonstrates the network layers diagram consisting of the Application, Control Plane and Data Plane layers. The Control Plane has a central network controller that manages the control plane and includes a resource allocation algorithm. The network controller communicates with the data plane layer via southbound APIs that are Floodlight for the OpenFlow switches, OSS API, that is a python-based API developed in-house for the OSS and other APIs for configuration of active optical resources. For the northbound A plane, we have developed a pub system using Redis [6]. Byte siz transmitted on the Ethernet networ and controllers. Compared to typically used for the northbound has much lower latency and only traffic. We measured the latency messages of 20 bytes among hos average latency of 300 s. Requests for the optical resource directly by the application layer. O providing the optical resource to t an explicit request to the controlle running the application. Howe architecture is mainly designed fo applications that run parallel proce clusters, we assume that a ce controller provides the traffic matri controller. We believe future applica central application controller ( manages different processes r clusters. This approach is also c SDN concept, which perform management of the network a performance of data-intensive app on cluster-computing platforms improve through an intelligent ma network architecture by combin knowledge of the interconnection traffic matrix of the applications. The resource allocation algorith network controller. Its objective is optical splitters across the multic maximizing the obtained throughpu the cascading of two splitters if cannot serve the request. We mode an Integer Program (IP). We deno of multicast requests and H the nu hosts. Each request i is ass transmission size si and a numbe optical splitters. The binary variable i is active as a result of the co constant mij is 1 if the i th request req sender or a receiver. The num splitters is denoted by S and we splitters have an identical numb model can trivially be extended to number of ports). Finally, the va request i is allocated to host j. formulated as following: