A physics-guided reinforcement learning framework for an autonomous manufacturing system with expensive data

Making intelligent decisions is the biggest challenge in building an autonomous manufacturing system that can build artifacts with desired properties without human intervention. Although reinforcement learning (RL) has favorable characteristics for such a task, the sample efficiency of RL is poor, which makes it difficult to implement on a manufacturing system due to the expense of producing parts to collect reward and action data. This paper focuses on building a framework for implementation of RL on manufacturing systems with expensive data and presents the framework for autonomous manufacturing of Phononic Crystals (PnCs), a type of acoustic metamaterial. Leveraging knowledge from physics-guided models and temporal abstraction ideas, we detail a framework that reduces the task of finding optimal manufacturing parameters from thousands of manufacturing samples to the order of 50 samples. The method is applied in simulation to a stochastic model of PnC production. Critically, we show that by using a long temporal abstraction horizon and order of 50 sample budget, the RL algorithm finds the optimal region greater than 95% of the time.