Waymo recently announcedA technologyBy connecting large natural language models in series, the autonomous driving system of a self-driving car can "explain" how it drives.
According to the instructions, Waymo uses a model called LINGO-1 to enable the self-driving system of a self-driving car to describe the road conditions ahead as "seen" by computer vision, and also to explain the decisions made while the vehicle is driving, allowing people to better understand the actual operation of the self-driving system and to make corrections for erroneous judgments.
LINGO-1 operates based on Vision-Language-Action Models (VLAMs). This allows the autonomous driving system to analyze and understand the images captured by the camera, and express them in natural language, even explaining the reasons for their decisions, allowing people to confirm whether the autonomous driving system's judgments are correct.
Compared to the past, when various analyses were required to determine why an autonomous driving system made an incorrect decision, artificial intelligence can now allow the autonomous driving system to actually explain the reasons for its decision-making, while also making it easier for people to measure the operational capabilities of the autonomous driving system.
In addition to allowing autonomous driving systems to explain their operating modes, the LINGO-1 model can also be used to train autonomous driving systems through natural language. For example, natural conversations can be used to teach autonomous driving systems which driving methods to avoid, allowing the autonomous driving system to understand the conversation and build learning experience.
Based on current Waymo test results, natural language training has increased the accuracy of autonomous driving systems from 30% below human performance to nearly 60% in just one month. This technology can be used not only for autonomous driving system training but also for training systems in robotics and other devices, thereby improving training efficiency and accuracy.
