A Real-time Receiving and Distributed Processing System for Large-scale Burst Data

According to the data characteristics and requirements of receiving and processing in large automatic weather monitoring network, the design and key technology on receiving and processing system of large burst data are given, and an in-depth discussion is made. Making use of multi-thread technology and large-scale communication event response technology, the data receiving thread pool is constructed and realized. Based on the theory of "data partition management", a concurrent processing data buffer is planned and excogitated. Through performance testing and application of the system, the results show that the system is real time, efficient and reliable and meets the current needs of the weather operation and is adaptable to the development of future weather operation.