A Novel Fuzzy Sarsa Learning Incorporated with Ant Colony Optimization

Fuzzy sarsa learning (FSL) is one of fuzzy reinforcement learning al- gorithms based on sarsa architecture. FSL approximates the action value function and is an on-policy method. In each fuzzy rules, actions are selected according to the proposed modified softmax formula. Because it is difficult for FSL to bal- ance exploration vs. exploitation, an novel ant colony optimization FSL (NACO- FSL) is offered by integrating the proposed ant colony optimization into FSL. In NACO-FSL, the tour of an ant is regarded as a combination of consequent actions selected from every rule based on pheromone trail. Simulation results in moun- tain car problem show that NACO-FSL well manager balance, and outperforms FSL in terms of learning speed and action quality.