A Neural-Q-Learning Based Approach for Delay Tolerant Underwater Acoustic Wireless Sensor Networks

Delay tolerance is a major design concern for supporting underwater acoustic wireless sensor networks (UA-WSNs) to carry out tasks in harsh subsea environments. Due to the great difference between the underwater acoustic channel and the radio frequency channel, most of the existing routing protocols developed for terrestrial wireless sensor networks perform poorly in underwater acoustic communications. In this work, we present a Neural-Q-Learning algorithm based delay tolerant protocol for UA-WSNs. Due to the advantages of the artificial neural network along with the Q-Learning algorithm, the ferry node is capable of determining an optimal route in a two-dimensional continuous space to relay packets effectively and efficiently between sensors. Simulation results show that the delivery delay and delivery cost of the network significantly decrease by maximizing the meeting probability between the ferry node and sensors.