The open-source system, known as Dobb-E, has made significant progress in teaching robots simple household tasks within a short timeframe. By using data collected from real homes, Dobb-E can effectively instruct a robot on actions such as opening an air fryer, closing a door, or straightening a cushion. This development addresses the challenges faced in training robots, as the required data cannot be easily obtained from the internet, and laboratory conditions don’t simulate the complexities of a real home.
To overcome these obstacles, the team behind Dobb-E devised a straightforward and replicable method to collect the necessary training data. They utilized an iPhone attached to a reacher-grabber stick, commonly used for picking up trash, to record videos of household tasks performed by volunteers in 22 different homes in New York. The iPhone’s lidar systems, motion sensors, and gyroscopes captured crucial data related to movement, depth, and rotation – vital information for training a robot to replicate these actions independently.
With just 13 hours’ worth of recordings, the team employed the collected data to train an AI model that provides instructions to a robot on how to carry out various tasks. The model utilized self-supervised learning techniques, enabling neural networks to identify patterns in the datasets without explicit labeling.
The next phase involved testing the reliability of Stretch, a commercially available robot consisting of a wheeled unit, a tall pole, and a retractable arm, in executing the tasks instructed by the AI system. To replicate the setup used with the stick and iPhone, an iPhone was mounted on Stretch’s arm using a 3D-printed mount.
Over a period of 30 days, researchers assessed the performance of the robot in 10 homes throughout New York. Remarkably, Stretch successfully completed 81% of the 109 household tasks assigned to it. Typically, Dobb-E required approximately 20 minutes to learn each task, which entailed five minutes of initial demonstration from a human using the stick and attached iPhone, followed by 15 minutes of fine-tuning as the system compared prior training with the new demonstration.
The application of Dobb-E in training robots to accomplish everyday household tasks holds great promise for transforming the capabilities of robots in a practical and efficient manner. By utilizing data collected directly from real homes, this system offers unique advantages over other AI models reliant on internet scraping. As researchers continue to refine and expand upon this breakthrough, the future integration of robots into our daily lives becomes increasingly tangible.
In conclusion, the development of Dobb-E, an open-source system that can teach robots simple household tasks within 20 minutes, has made significant progress. By utilizing data collected from real homes, this system holds promise in training robots to perform diverse tasks, paving the way for greater integration of robots in our everyday lives. The ability to collect physical data and replicate real home conditions provides valuable insights for training robots effectively and scaling training databases to meet future needs. As this technology progresses, it has the potential to revolutionize the role of robots in various aspects of our daily routines, simplifying household chores and enhancing overall convenience.
