MIT’s CSAIL department has unveiled RoboGrocery, a soft robotic system designed to bag groceries using computer vision and a soft robotic gripper.
This system is a notable advancement in the ongoing automation trend in grocery stores, which started with the installation of the first self-checkout system in a Kroger grocery store near Atlanta in 1986. Over the decades, self-checkout technology has become widespread across the United States, paving the way for further innovations like robotic bagging.
How Does RoboGrocery Work?
RoboGrocery integrates multiple technologies, including RGB-D cameras, closed-loop control servo motors, and soft tactile sensors, to accurately assess and handle a variety of grocery items based on their weight, size, and shape. To test the system, researchers placed ten different objects, ranging from delicate products like grapes, bread, kale, muffins, and crackers to sturdier items like soup cans, meal boxes, and ice cream containers.
The system begins by using computer vision to detect the objects and determine their size and orientation on the conveyor belt. Once the gripper touches an item, pressure sensors in its fingers assess the item’s fragility.
For example, when the gripper handles grapes, the sensors detect their delicacy and ensure they are not placed at the bottom of the bag. Conversely, the gripper identifies the soup can as a rigid object and positions it at the bottom of the bag to avoid crushing other items.
“This is a significant first step towards having robots pack groceries and other items in real-world settings,” said Annan Zhang, one of the study’s lead authors. While RoboGrocery is not yet ready for commercial deployment, the research demonstrates the potential of integrating multiple sensing modalities in soft robotic systems.
Daniela Rus, the Director of CSAIL and Erna Viterbi Professor of Electrical Engineering and Computer Science, emphasized that integrating soft-handed robots with multi-modal algorithms represents more than just automation—it’s a paradigm shift that enhances precision, reduces waste, and adapts seamlessly to modern retail logistics.
Allied Market Research projects the global retail logistics market to be worth $809.7 billion by 2032, up from $238.5 billion in 2022.
RoboGrocery’s combination of technologies allows it to outperform traditional methods, cutting down on actions that could damage items by up to nine times compared to sensorless system and 4.5 times compared to vision-only approaches. The system’s microprocessor chip handles all sensory data and executes packing decisions in real-time, ensuring efficient and safe packing.
Researchers noted that there is still room for improvement, including upgrades to the gripper and the imaging system to better determine how and in what order to pack items. As the system becomes more robust, it may also be adapted for use in more industrial environments, such as recycling plants.
Featured Image courtesy of MIT CSAIL