Optimal Control of Wireless Sensor Networks for Structural Health Monitoring

Optimal control of large sensor networks for structural health monitoring requires efficient management of limited energy, network capacity, and computational power. These netorks must accommodate “typical” changes in the host structure and the sensor network, while detecting and responding to anomalous conditions. We treat wireless sensor networks as distributed agents that observe the current state of their environment, choose an action, and receive a vector of reward data encoding the designer’s goals for the network. In this setting, the control problem becomes a Markov decision process (MDP) in which the agent acts to maximize the discounted sum of rewards it receives over time, and derives an optimal controller by iteratively improving its policy. The data structures, update rule, and decision procedure that we present exploit the generalized distributive law (GDL), an algebraic formalism that implements tree-structured computations by passing messages between nodes in a tree-structured network. Each node in the network receives a reward signal for its actions, and the coordination procedures specified by the GDL insure that each node chooses an optimal action given its neighbors’ decisions. We describe an update procedure for distributed agents so that the network’s distributed control law converges by reinforcement learning to a globally optimal solution using only local estimators. We also describe several modifications to the basic junction tree reinforcement learning algorithm that make it more practical to implement in low-power, limited-memory devices.