3/24/2023 0 Comments Are automatrons realUnlike prior setups that define all the skills up front and then collect curated demonstrations for each skill, we continuously collect data across multiple robots without scene resets or any low-level skill segmentation. Key to our approach is a scalable recipe for creating large, diverse language-conditioned robot demonstration datasets. We are also excited to announce the release of Language-Table, the largest available language-annotated robot dataset, which we hope will drive further research focused on real-time language-controllable robots. After training with our approach, we find that an individual policy is capable of addressing over 87,000 unique instructions (an order of magnitude larger than prior works), with an estimated average success rate of 93.5%. In Interactive Language, we present a large scale imitation learning framework for producing real-time, open vocabulary language-conditionable robots. Thus, a critical open question in the open vocabulary setting is: how can we scale the collection of robot data to include not dozens, but hundreds of thousands of behaviors in an environment, and how can we connect all these behaviors to the natural language an end user might actually provide? Existing multitask learning setups make use of curated imitation learning datasets or complex reinforcement learning (RL) reward functions to drive the learning of each task, and this significant per-task effort is difficult to scale beyond a small predefined set. This is a setting with an inherently large number of tasks, including many small corrective behaviors. However, getting robots to follow open vocabulary language poses a significant challenge from a ML perspective. To be successfully guided through a long horizon task like "put all the blocks in a vertical line", a robot must respond precisely to a wide variety of commands, including small corrective behaviors like "nudge the red circle right a bit". The challenges of open-vocabulary language following. Furthermore, real-time language could make it easier for people and robots to collaborate on complex, long-horizon tasks, with people iteratively and interactively guiding robot manipulation with occasional language feedback. Particularly in open human environments, it may be important for end users to customize robot behavior as it is happening, offering quick corrections ("stop, move your arm up a bit") or specifying constraints ("nudge that slowly to the right"). Ideally, robots of the future would react in real time to any relevant task a user could describe in natural language. Despite this progress, an important missing property of current "language in, actions out" robot learning systems is real time interaction with humans. Code as Policies has shown that code-generating language models combined with pre-trained perception systems can produce language conditioned policies for zero shot robot manipulation. Recent Palm-Sa圜an work has produced robots that leverage language models to plan long-horizon behaviors and reason about abstract goals. Over the last few years, there have been significant advances in the application of machine learning (ML) for instruction following, both in simulation and in real world systems. Posted by Corey Lynch, Research Scientist, and Ayzaan Wahid, Research Engineer, Robotics at GoogleĪ grand vision in robot learning, going back to the SHRDLU experiments in the late 1960s, is that of helpful robots that inhabit human spaces and follow a wide variety of natural language commands.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |