Encord is testing whether brain activity can provide better training data for robots learning complex physical tasks. At its San Leandro facility, human operators perform activities such as dismantling Jenga towers while wearing camera-equipped headsets that also measure their brain waves.
The trial is being conducted with Zander Labs, a German-Dutch neuroscience company developing passive brain-computer interfaces. Encord plans to create an initial brain-wave-tagged dataset, test it with customers’ robotics models and determine whether the additional signals improve performance.
Brain Signals Could Identify Difficult Moments
The headsets are designed to detect mental states such as intent, surprise and recognition of an error. These signals could help developers identify parts of a task that require greater attention or more capable AI models.
Zander Labs says passive brain-computer interfaces can interpret naturally occurring brain activity without requiring users to issue deliberate mental commands. The technology is intended to give computer systems more context about a person’s cognitive state while they complete a task.
Encord’s experiment remains at an early stage. The company has not yet established whether brain-wave data produces enough improvement to justify collecting it at a larger scale.
Robotics Companies Need More Real-World Data
Encord originally developed software for annotating data and evaluating computer-vision models. It has since expanded into producing real-world datasets because robotics companies cannot find enough high-quality examples of physical manipulation tasks.
Its physical AI data service collects video and sensor information from human demonstrations and remotely operated robots. Human operators, called pilots, use paired leader-follower robotic arms to perform tasks such as pouring coffee, stacking poker chips and connecting cables to servers.
The company also collects first-person video from workers wearing cameras in factories and other environments. Additional sensors can measure muscle activity in a person’s arm, helping reconstruct hand positions that may not be fully visible in video.
Physical Data Is Expensive to Produce
Robotics datasets require detailed descriptions linking movements to language, such as identifying when a right hand tightens a bolt. Encord argues that densely annotated examples are more useful for training specific skills than large quantities of unstructured video.
Producing this information is also more expensive than collecting the text used to train large language models. Internet data could be gathered at enormous scale, while physical AI data requires people, equipment, controlled environments and repeated demonstrations.
Encord is betting that generating and managing these datasets will become a separate industry as humanoid and warehouse robots improve. Its brain-wave trial represents one attempt to determine which additional human signals can help robots learn difficult tasks more efficiently.
Featured image credits: Google Gemini
For more stories like it, click the +Follow button at the top of this page to follow us.