AIM3S: Autonomous Inventory Monitoring through Multi-Modal Sensing for Cashier-Less Convenience Stores

A common pain point for physical retail stores is live inventory monitoring, i.e. knowing how many items of each product are left on the shelves. About 4% of sales are lost due to an average 5-10% out-of-shelf stockout rate, while additional supplies existed in the warehouse. Traditional techniques rely on manual inspection, per-item tagging using RFIDs, or human-in-the-loop systems such as Amazon Go. These approaches, while effective, either have poor accuracy, long delays between results or are cost prohibitive. In this paper, we present AIM3S (Autonomous Inventory Monitoring through Multi-Modal Sensing) for cashier-less stores. To the best of our knowledge, this is the first fully autonomous system that fuses multiple sensing modalities. Utilizing weight difference on a shelf, visual item recognition in customers' hands and prior knowledge of item layout AIM3S monitors products picked up or returned without human-in-the-loop. We present results from a real-world setup with 85 items (33 unique products) replicating the layout of a local 7-Eleven store. Our results show that the fused approach provides up to 93.2% item identification accuracy, better than reported self-checkout stations (86%).