A new outlier rejection methodology for supporting load forecasting in smart grids based on big data

Internet of things (IoT) enables smart electrical grids (SEGs) to solve its problems and to support a lot of tasks. These tasks include power monitoring, demand-side energy management coordination of distributed storage, and the integration of renewable energy generators. Sending the complete captured data directly to the cloud would lead to resource wastage. Hence, 2-tier architecture is replaced by 3-tier one in order to include a fog computing tier. Fog tier acts as a bridge in the middle between IoT devices embedded in SEG and cloud tier to overcome the cloud challenges. The main actions of the added fog tier are collecting, computing, and storing smart meters data before transmitting it to the cloud. In this paper, a new electrical load forecasting (ELF) strategy has been proposed based on the pre-mentioned 3-tier architecture. ELF consists of two main phases, which are; (i) data pre-processing phase (DP2) and (ii) load prediction phase (LP2). Both phases are executed at cloud servers (CSs) on the collected data, which is received from all fogs connected to the entire cloud. The main objective of DP2 is to; (i) select the meaningful features and (ii) eliminate outlier items from the collected data. The main contribution of this paper lied on outlier rejection phase. The paper introduces a new outlier rejection methodology called hybrid outlier rejection methodology (HORM). HORM try to eliminate all outliers from the training dataset before start learning the prediction model during LP2. HORM involves two stages which are; (i) a new statistical based outlier rejection stage, which is called fast outlier rejection (FOR) and (ii) an accurate outlier rejection (AOR) stage using genetic algorithm (GA). Then, the filtered data is used to give fast and accurate load prediction decisions. Experimental results have shown that the proposed HORM outperforms recent outlier rejection methods in terms of accuracy, precision, recall, and F1-measure.