A Robotic Arm That Picks Up Unfamiliar Objects With Great Precision and Places Them Where They Belong

Researchers at the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) have built a highly intelligent robotic arm that is able to gently pick up objects that it has never seen before and with great precision, place them exactly where and how they belong. The team used 3-dimensional “keypoints” to visually represent the objects. The robot senses the keypoints within each object and learns to act in accordance with “Keypoint Affordance Manipulation” (KPAM).

In the case of the mug, all the system needs are three keypoints, which consist of the center of the mug’s side, bottom, and handle, respectively. For the shoe, KPAM needed just six keypoints to be able to pick up more than 20 different pairs of shoes ranging from slippers to boots.

MIT professor Russ Tedrake, who is the senior author on the project paper stated that it’s the successful placement of the unfamiliar items that is the real accomplishment here.

Whenever you see a robot video on YouTube, you should watch carefully for what the robot is NOT doing…Robots can pick almost anything up, but if it’s an object they haven’t seen before, they can’t actually put it down in any meaningful way.

WATCH: MIT robot precisely moves objects it's never seen, like placing a small mug onto a tiny hook. https://t.co/Jn8eFQpwWV #robotics pic.twitter.com/iWa0x9aCHI

— MIT CSAIL (@MIT_CSAIL) March 18, 2019