Figure AI and Brookfield Team Up to Build the World's Largest Humanoid Pretraining Dataset

---

In a significant move for the robotics industry, Figure AI has announced Project Go-Big, an ambitious initiative aimed at creating the world’s largest pretraining dataset for humanoid robots. Unlike language models (LLMs), which can leverage vast amounts of internet data, robotic datasets are scarce and often lack the diversity needed to train sophisticated AI systems. This is where Project Go-Big steps in.

The Challenge of Robotic Data

One of the primary challenges in robotics is the scarcity of high-quality, diverse training data. While LLMs like GPT-3 can scrape vast amounts of text from the internet, robots require a different kind of data-specifically, real-world interactions and movements. This data is crucial for training robots to navigate complex environments and perform tasks with precision.

Project Go-Big: Accelerated by Brookfield

To address this challenge, Figure AI has partnered with Brookfield, a global leader in real estate with over 100,000 residential units. This partnership provides access to a wide range of real-world settings where the Helix robot can learn and improve.

• Diverse Environments: Brookfield’s extensive portfolio includes apartments, houses, and mixed-use developments, offering a rich variety of environments for data collection.
• Real-World Data: The dataset will include video footage from these residential units, capturing human activities and movements that the Helix robot can learn from.

Helix: Learning Directly from Human Video

One of the key innovations in Project Go-Big is the use of human video data to train the Helix robot. This approach allows the robot to learn navigation and manipulation skills directly from how humans interact with their environments.

• End-to-End Training: A single neural network now outputs both manipulation and navigation tasks, processing inputs like language and pixel data.
• First of Its Kind: To our knowledge, this is the first instance of a humanoid robot learning navigation end-to-end using only human video data.

Technical Details

The technical implementation of Project Go-Big involves several key components:

• Data Collection: High-resolution cameras are installed in Brookfield’s residential units to capture detailed video footage.
• Preprocessing: The collected data is preprocessed to enhance quality and ensure privacy. This includes anonymization techniques to protect the identities of individuals captured on camera.
• Training Pipeline: The Helix robot uses a deep learning framework that integrates both visual and linguistic inputs. The neural network is trained using a combination of supervised and unsupervised learning methods.

Results and Future Plans

Figure AI has already seen promising results from this approach. The Helix robot has demonstrated improved navigation and manipulation skills, particularly in complex and dynamic environments. The company plans to continue expanding the dataset and refining the training process.

• Scalability: Figure AI is actively seeking more partners to contribute data and help scale the project.
• Career Opportunities: If you are interested in contributing to this groundbreaking initiative, Figure AI is hiring for various roles. Check out their careers page for more information: figure.ai/careers

Conclusion

Project Go-Big represents a significant step forward in the field of robotics. By leveraging real-world data from diverse environments, Figure AI and Brookfield are paving the way for more advanced and capable humanoid robots. This initiative not only addresses the challenge of data scarcity but also sets a new standard for how robots can learn and adapt to human-like behaviors.