Autonomous Inventory Monitoring through Multi-Modal Sensing (AIM3S) for Cashier-Less Stores

Auto-checkout technology could revolutionize physical retail by bringing down operating costs and enabling automated stores. The first step towards fully autonomous stores is automated live inventory monitoring. Existing automated approaches tend to focus on vision only and are cost prohibitive, slow, or inaccurate for practical real-world applications. To overcome these challenges we propose a novel sensor fusion framework through cameras, weight sensors and prior knowledge of item arrangement. Namely we focus on the tasks of event detection (i.e. when did customers pick up or return an item?), and robust item classification (i.e. what product was it?). In this demo we show the system can be implemented to accurately predict inventory in real-time.