MergeSFL: Split Federated Learning with Feature Merging and Batch Size Regulation
0
Citation
0
Reference
10
Related Paper
Abstract:
Recently, federated learning (FL) has emerged as a popular technique for edge AI to mine valuable knowledge in edge computing (EC) systems. To mitigate the computing/communication burden on resource-constrained workers and protect model privacy, split federated learning (SFL) has been released by integrating both data and model parallelism. Despite resource limitations, SFL still faces two other critical challenges in EC, i.e., statistical heterogeneity and system heterogeneity. To address these challenges, we propose a novel SFL framework, termed MergeSFL, by incorporating feature merging and batch size regulation in SFL. Concretely, feature merging aims to merge the features from workers into a mixed feature sequence, which is approximately equivalent to the features derived from IID data and is employed to promote model accuracy. While batch size regulation aims to assign diverse and suitable batch sizes for heterogeneous workers to improve training efficiency. Moreover, MergeSFL explores to jointly optimize these two strategies upon their coupled relationship to better enhance the performance of SFL. Extensive experiments are conducted on a physical platform with 80 NVIDIA Jetson edge devices, and the experimental results show that MergeSFL can improve the final model accuracy by 5.82% to 26.22%, with a speedup by about 1.74x to 4.14x, compared to the baselines.Keywords:
Merge (version control)
Speedup
Federated Learning
Feature (linguistics)
Edge device
In a future IoT-dominated environment the majority of
data will be produced at the edge, which may be moved
to the network core. We argue that this reverses today’s
“core-to-edge” data flow to an “edge-to-core” model
and puts severe stress on edge access/cellular links. In
this paper, we propose a data-centric communication approach
which treats storage and wire the same as far as
their ability to supply the requested data is concerned.
Given that storage is cheaper to provide and scales better
than wires, we argue for enhancing network connectivity
with local storage services (e.g., in WiFi Access Points,
or similar) at the edge of the network. Such local storage
services can be used to buffer IoT and user-generated
data at the edge, prior to data-cloud synchronization.
Edge device
Core network
Data access
Cite
Citations (27)
Most edge AI focuses on prediction tasks on resource-limited edge devices while the training is done at server machines. However, retraining or customizing a model is required at edge devices as the model is becoming outdated due to environmental changes over time. To follow such a concept drift, a neural-network based on-device learning approach is recently proposed, so that edge devices train incoming data at runtime to update their model. In this case, since a training is done at distributed edge devices, the issue is that only a limited amount of training data can be used for each edge device. To address this issue, one approach is a cooperative learning or federated learning, where edge devices exchange their trained results and update their model by using those collected from the other devices. In this paper, as an on-device learning algorithm, we focus on OS-ELM (Online Sequential Extreme Learning Machine) to sequentially train a model based on recent samples and combine it with autoencoder for anomaly detection. We extend it for an on-device federated learning so that edge devices can exchange their trained results and update their model by using those collected from the other edge devices. This cooperative model update is one-shot while it can be repeatedly applied to synchronize their model. Our approach is evaluated with anomaly detection tasks generated from a driving dataset of cars, a human activity dataset, and MNIST dataset. The results demonstrate that the proposed on-device federated learning can produce a merged model by integrating trained results from multiple edge devices as accurately as traditional backpropagation based neural networks and a traditional federated learning approach with lower computation or communication cost.
Autoencoder
Edge device
MNIST database
Retraining
Cite
Citations (11)
Edge device
Transfer of learning
Cite
Citations (0)
Edge device
Transfer of learning
Cite
Citations (16)
Measuring reliability of edge networks in the Internet is difficult due to the size and heterogeneity of networks, the rarity of outages, and the difficulty of finding vantage points that can accurately capture such events at scale. In this paper, we use logs from a major CDN, detailing hourly request counts from address blocks. We discovered that in many edge address blocks, devices, collectively, contact the CDN every hour over weeks and months. We establish that a sudden temporary absence of these requests indicates a loss of Internet connectivity of those address blocks, events we call disruptions.
Edge device
Cite
Citations (29)
Summary In this contribution, we define the concept of transferring intelligence from the cloud to the edge of the network and highlight its importance in modern IoT architectures. It leads to edge‐enabled IoT solutions, where intelligence is distributed to the end devices, ie, edge of the network. The IoT gateway is a middle‐ware between devices and cloud and facilitates computations and communication. We have encountered a research gap and decided to propose our solution on creating an edge‐enabled IoT gateway and evaluating its rating by functionality. In the beginning, we describe the general architecture where we demonstrate the location of the edge‐enabled IoT gateway in edge‐enabled solution. Then, we discuss a selection of major criteria, which could be implemented to any edge‐enabled solution. In the end, we evaluate our proposed edge‐enabled IoT gateway and also compare several scientific works focused on IoT gateway design according to our criteria.
Gateway (web page)
Edge device
Gateway address
Cite
Citations (23)
In the last few years, Internet of Things (IoT) devices are multiplicating their presence in our daily life. This means that the data generated in our houses, offices, and common places is starting to be too big to be elaborated in a limited number of places. In this scenario, the advent of Edge Computing, in general, and Edge Intelligence, in particular, is favoring the scalability and the efficiency of IoT systems. Such paradigms allow, by using devices placed at the edge of the network, the distributed elaboration of data created by IoT devices so permitting to transmit to the cloud only synthetic information. Edge Intelligence supports the so-called Federated Learning (FL), which is a novel paradigm that allows the distributed training of neural network models. Such models are initially distributed from the cloud to edge nodes and, on such edge nodes, they are refined based on data gathered from IoT nodes. Such refined models are sent back to the cloud and merged with other models elaborated on different edge nodes. This paper presents a novel architecture for Federated Learning enabling a Multi-Layer Hierarchical Federated Learning (MLH-FL) that allows to execute the traditional FL with model aggregation at different layers. The proposed approach will be also evaluated with some simulations and the final accuracy and loss of the obtained models will be compared with the traditional FL approach.
Edge device
Federated Learning
Cite
Citations (5)
Federated learning (FL) is a new distributed machine learning paradigm that enables machine learning on edge devices. One unique feature of FL is that edge devices belong to individuals; and since they are not "owned" by the FL coordinator, but can be "federated" instead, there can potentially be a huge number of edge devices. In the current distributed ML architecture, the parameter server (PS) architecture, model aggregation is centralized. When facing a large number of edge devices, the centralized model aggregation becomes the bottleneck and fundamentally restricts system scalability.
Federated Learning
Edge device
Distributed learning
Feature (linguistics)
Cite
Citations (4)
In edge computing, there are two fundamental targets that are in conflict with each other: ensuring a minimum level of Quality of Service (QoS) for an edge application, and at the same time, maximising the utilisation of the edge infrastructure for all applications requesting edge services. In this paper, an edge resource allocation model is proposed. The proposed model provides a solution for both targets and their inherent contradiction. To estimate the edge application QoS, End-to-End Latency (E2EL) as measured by the application is used. Throughout the paper, it is shown that this is a viable approach for many different real-world scenarios, as well as for a broad set of edge application categories. Furthermore, E2EL measurement is a simple concept for application developers to understand and implement. To maximise edge utilisation, a score-based approach is suggested. The approach dynamically determines the best edge node and network path combination for a given edge application. The two solutions are interconnected such that the model maximises the overall edge utilisation while maintaining an acceptable QoS level for each edge application. The proposed edge resource allocation model is implemented and tested on an emulated edge infrastructure with challenging edge scenarios. The experiments show that the model delivers good and robust results for both fundamental edge targets even in real-world edge environments with many uncertainties.
Edge device
Cite
Citations (4)
To fully exploit the enormous data generated by the devices in edge computing, edge federated learning (EFL) is envisioned as a promising solution. The distributed collaborative training in EFL deals with the delay and privacy issues as compared to the traditional model training. However, the existence of straggling devices degrades the model performance. The stragglers are manifested due to the data or system heterogeneity. In this paper, we introduce elastic optimized edge federated learning (FedEN) approach to mitigate the straggling-effect due to the data heterogeneity. This issue can be alleviated by the reinforced device selection by the edge server which can solve device heterogeneity to some extent. But, the statistical heterogeneity remains unsolved. Specifically, we define the problem of stragglers in EFL. Then, we formulate the optimization problem to be solved at the edge devices. We experimented on the MNIST and CIFAR-10 datasets for the proposed model. Simulated experiments demonstrates that the proposed approach improves the training performance. The results confirm the improved performance of FedEN approach over the baselines.
MNIST database
Edge device
Federated Learning
Cite
Citations (1)