Guanya ShiAssistant ProfessorRobotics Institute,Carnegie Mellon UniversityOctober 11, 2024Building Generalist Robots with Agility via Learning and Control: Humanoids and Beyond
Abstract: Recent breathtaking advances in AI and robotics have brought us closer to building general-purpose robots in the real world, e.g., humanoids capable of performing a wide range of human tasks in complex environments. Two key challenges in realizing such general-purpose robots are: (1) achieving “breadth” in task/environment diversity, i.e., the generalist aspect, and (2) achieving “depth” in task execution, i.e., the agility aspect. In this talk, I will present recent works that aim to achieve both generalist-level adaptability and specialist-level agility, demonstrated across various real-world robots, including full-size humanoids, quadrupeds, aerial robots, and ground vehicles. The first part of the talk focuses on learning agile and general-purpose humanoid whole-body control using sim2real reinforcement learning. The second part will discuss the limitations of such end2end sim2real pipelines and how combining learning with control can enhance safety, efficiency, and adaptability.
