Robobarista: Object Part based Transfer of Manipulation Motion from Crowd-sourcing in 3D Pointclouds
Jaeyong Sung, Seok Hyun Jin, Ashutosh Saxena
Cornell University
Robobarista Project:
In order for robots to interact within household environments, robots should be able to manipulate a large variety of objects and appliances in human environments, such as stoves, coffee dispensers, juice extractors, and so on. Consider the espresso machine above — even without having seen the machine before, a person can prepare a cup of latte by visually observing the machine and by reading the instruction manual. This is possible because humans have vast prior experience of manipulating differently-shaped objects. In this project, our goal is to enable robots to generalize to different objects and tasks. As shown in this video, our deep learning model (Deep Multimodal Embedding) even allows our robot to make a cup of latte without having seen the espresso machine or coffee grinder before.
