Scientists from MIT have created a robot memory mechanism that allows a robot to recognize when something is wrong with the description of an object and correct the information stored.
A new memory mechanism that was created by scientists at the Massachusetts Institute of Technology may help robots to be more effective and work properly in real-life environments. UQ-DAAAM, which stands for Uncertainty Quantification for Describe Anything, Anywhere, at Any Moment, is a memory mechanism that will allow robots to recognize any mistakes in the description of the object and to correct the information stored in memory.
The UQ-DAAAM algorithm is an improvement on the previously developed DAAAM memory architecture at the Massachusetts Institute of Technology. This architecture employs a vision-language model (VLM), which generates descriptions for objects identified through the cameras on board the robot. These descriptions are then stored in form of captions in the 3D map of the environment, enabling robots to access information about the object, location, and past experience. However, incomplete views, insufficient lighting, distance from the camera, and blurring may result in generation of incorrect descriptions using the VLMs. For example, the robot may confuse a chimney for a squirrel.
Measuring Uncertainty To Correct Memory
However, UQ-DAAAM overcomes this problem by integrating two uncertainty estimates, namely the level of certainty of the description produced by the VLM and the agreement between the produced description and descriptions produced from alternative viewpoints of the same object.
In cases when the uncertainty becomes too high, more images are selected in order to have clearer views, after which a large language model (LLM) merges the existing descriptions and takes into account the latest ones, whose observations are not ambiguous.
The most important thing about this method is that it can pick out helpful viewpoints without having to run the computationally expensive VLM once again. A mathematical heuristic was created in order to find out the most informative frames.
In the evaluation phase, the algorithm was able to resolve the ambiguity for approximately 92% of the objects by using merely two extra captions for each object. Moreover, it enhanced the object description accuracy by 5% than the previous techniques and provided more precise answers related to the objects, their positions, and the time at which they were last seen. The future research of the scientists will include implementing the model on mobile robotic manipulators and exploring the role of uncertainty measurement in the process of determining the required revisit areas for robots.
