A NEW METHOD FOR MARSHALING PLAN USING A REINFORCEMENT LEARNING CONSIDERING DESIRED LAYOUT OF CONTAINERS IN PORT TERMINALS

Abstract In container yard terminals, containers brought by trucks in the random order. Containers have to be loaded into the ship in a certain order, since each container has its own shipping destination and it cannot be rearranged after loading. Therefore, containers have to be rearranged from the initial arrangement into the desired arrangement before shipping. In the problem, the number of container-arrangements increases by the exponential rate with increase of total count of containers, and the rearrangement process occupies large part of total run time of material handling operation at the terminal. Moreover, conventional methods require enormous time and cost to derive an admissible result for rearrangement process. In this paper, a Q-Learning algorithm considering the desired position of containers for a marshaling in the container yard terminal is proposed. In the proposed method, the learning process consists of two parts: rearrangement plan assuring explicit transfer of container to the desired position , and, removal plan for preparing the rearrange operation. Using the proposed method, the learning performance can be improved as compared to the conventional method. In order to show effectiveness of the proposed method, computer simulations for several examples are conducted.