New model offers robots precise pick-and-place solutions

Researchers at Massachusetts Institute of Technology (MIT) Department of Mechanical Engineering have developed SimPLE — an approach to object manipulation that aims to allow robots to learn to pick and place objects without any prior experience or encounters with the specific objects.

Pick-and-place machines are a type of automated equipment used to place objects into structured, organised locations. These machines are used for a variety of applications — from electronics assembly to packaging, bin picking and even inspection — but many current pick-and-place solutions are limited. Current solutions lack ‘precise generalisation’, or the ability to solve many tasks without compromising on accuracy.

“In industry, you often see that manufacturers end up with very tailored solutions to the particular problem that they have, so a lot of engineering and not so much flexibility in terms of the solution,” said Maria Bauza Villalonga, a senior research scientist at Google DeepMind where she works on robotics and robotic manipulation. “SimPLE solves this problem and provides a solution to pick and place that is flexible and still provides the needed precision.”

A new paper published in the journal Science Robotics explores pick-and-place solutions with more precision. In precise pick and place, also known as kitting, the robot transforms an unstructured arrangement of objects into an organised arrangement. The approach, dubbed SimPLE (Simulation to Pick Localize and placE), learns to pick, regrasp and place objects using the object’s computer-aided design (CAD) model, and all without any prior experience or encounters with the specific objects.

“The promise of SimPLE is that we can solve many different tasks with the same hardware and software using simulation to learn models that adapt to each specific task,” said Alberto Rodriguez, an MIT visiting scientist and associate director of manipulation research for Boston Dynamics. SimPLE was developed by members of the Manipulation and Mechanisms Lab at MIT (MCube) under Rodriguez’s direction.

“In this work we show that it is possible to achieve the levels of positional accuracy that are required for many industrial pick-and-place tasks without any other specialisation,” he said.

Using a dual-arm robot equipped with visuotactile sensing, the SimPLE solution employs three main components: task-aware grasping, perception by sight and touch (visuotactile perception), and regrasp planning. Real observations are matched against a set of simulated observations through supervised learning so that a distribution of likely object poses can be estimated, and placement accomplished.

In experiments, SimPLE successfully demonstrated the ability to pick and place diverse objects spanning a wide range of shapes, achieving successful placements over 90% of the time for six objects, and over 80% of the time for 11 objects.

Pick and place with precision: MIT doctoral student Antonia Delores Bronars describes the new SimPLE system. Credit: John Freidah/MIT Department of Mechanical Engineering.

“There’s an intuitive understanding in the robotics community that vision and touch are both useful, but until now there haven’t been many systematic demonstrations of how it can be useful for complex robotics tasks,” said mechanical engineering doctoral student Antonia Delores Bronars.

Bauza and Bronars said this work was informed by several generations of collaboration.

“In order to really demonstrate how vision and touch can be useful together, it’s necessary to build a full robotic system, which is something that’s very difficult to do as one person over a short horizon of time,” Bronars said. “Collaboration, with each other… and across many generations and labs really allowed us to build an end-to-end system.”

Top image credit: John Freidah/MIT Department of Mechanical Engineering