Research on Preprocessing Technology of Power Mobile Data Collection Based on Distributed Agents

There are a wide range of monitoring contents in power mobile terminals, including large data scale, various fields, and different formats, which bring huge challenges to subsequent security analysis. At the same time, data collection requires high real-time performance, otherwise it is difficult to meet the needs of timely detection of safety hazards. Therefore, it is necessary to design a reasonable distributed real-time data preprocessing framework to ensure the realtime performance and stability in the data collection and processing process. In view of the shortcomings of current power terminal distributed agent data collection and preprocessing, this paper proposes a distributed real-time data preprocessing solution for power terminal monitoring logs, builds a distributed real-time data preprocessing framework for power terminals, and proposes a solution based on an automated method for data attribute generalization.