Do robots like videos of cute puppies or daring skateboard stunts? Researchers at the Defense Advanced Research Projects Agency might find out, but it will be as a byproduct of the Mathematics of Sensing, Exploitation and Execution (MSEE) program, which is teaching robots to respond to visual information by having them watch YouTube videos.
"Robots can learn to recognize objects and patterns fairly well, but to interpret and be able to act on visual input is much more difficult," noted a DARPA news release. So, DARPA-funded University of Maryland researchers taught robots to process visual data from Youtube cooking videos. The robots were able to recognize, grab and manipulate the correct kitchen utensil or object and perform the tasks demonstrated in the videos without any additional human input or programming, according to DARPA.
"The MSEE program initially focused on sensing, which involves perception and understanding of what's happening in a visual scene, not simply recognizing and identifying objects," said DARPA program manager Reza Ghanadan. "We've now taken the next step to execution, where a robot processes visual cues through a manipulation action-grammar module and translates them into actions."