Robotics Simulation, Digital Twins & Training
Complete Automated Retail
An automated retail checkout for box products: five deterministic manipulation skills plus a tipping skill trained with reinforcement learning in Isaac Lab.
The problem
A checkout robot fails on the cases nobody planned for. Products arrive in different positions and orientations, with the barcode anywhere — including on a face the scanner cannot see. The design goal was to remove those logical blind spots rather than demonstrate one clean pick.
What we solved
The register and assets were modelled in Blender, then simulated and controlled in NVIDIA Isaac Sim. CuMotion provides collision-aware motion, and a custom adaptive gripper handles the products. Five deterministic skills cover the barcode positions — underneath, facing up and requiring reorientation, on either large side face, and on a side face of a flat product. A learned tipping skill trained with PPO in NVIDIA Isaac Lab handles the remaining cases, with custom observations, rewards, contact logic and task constraints. One main controller coordinates the skills, the RL policy, conveyor logic, adaptive grasping, inspection, scanner behaviour and LED states.
The result
A system that reads product orientation before it commits to a strategy. A five-camera inspection setup — four diagonal and one overhead — checks orientation and barcode location, and when no barcode is visible on an exposed face the product is treated as occluded on the support face and routed accordingly. Tipping direction is chosen by where the product sits in the grasp zone, so difficult rail-edge configurations are avoided.
This project sets the direction we are developing further: digital twins, robotics, simulation and reinforcement learning combined, while keeping CGI presentation quality.