Researchers at the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) have built a highly intelligent robotic arm that is able to gently pick up objects that it has never seen before and with great precision, place them exactly where and how they belong. The team used 3-dimensional “keypoints” to visually represent the objects. The robot senses the keypoints within each object and learns to act in accordance with “Keypoint Affordance Manipulation” (KPAM).
In the case of the mug, all the system needs are three keypoints, which consist of the center of the mug’s side, bottom, and handle, respectively. For the shoe, KPAM needed just six keypoints to be able to pick up more than 20 different pairs of shoes ranging from slippers to boots.
Whenever you see a robot video on YouTube, you should watch carefully for what the robot is NOT doing…Robots can pick almost anything up, but if it’s an object they haven’t seen before, they can’t actually put it down in any meaningful way.
— MIT CSAIL (@MIT_CSAIL) March 18, 2019