Before Google I/O 2024 started, Google announcedApplication functions similar to Google Lens, confirmed as a development project called "Project Astra", will be able to combine voice interaction, image recognition, environmental perception and other methods to enable artificial intelligence to instantly describe and interpret the content of the picture taken by the mobile phone camera, and even seamlessly transfer it to glasses with camera lens functions for continued use.
"Project Astra" was conceived and created by the Google DeepMind team, and aims to be a general-purpose artificial intelligence digital assistant, allowing users to improve work efficiency, simplify work processes, and solve difficult problems in daily life through this service.
However, "Project Astra" is still in a very early stage of development and has not yet been specifically planned to become an official product or service. However, Demis Hassabis, head of the Google DeepMind team, said that the related functions of "Project Astra" will be applied to mobile phones or smart wearable devices such as glasses, and some functions will even be first applied to Google services such as Gemini this year.
Judging from the actual demonstration at Google I/O 2024, "Project Astra" will be able to capture images through a mobile phone camera, and artificial intelligence will continuously describe the details of the image in real time. For example, it can describe the sound that a loudspeaker can make, and even point out the amplification components that can produce high-frequency sounds. It can also describe creative reduplication of crayons, recognize program coding content, and even associate "Schrödinger's Cat" with a cat and cardboard box drawn on a whiteboard.
Furthermore, the user in the video also asks if they remember the location of their glasses, and the AI quickly answers, indicating its ability to memorize information. Furthermore, the AI can follow the user's switch from a phone to using the glasses with a camera, indicating that Project Astra has certain data migration features, allowing users to continue using it on different devices.
Demis Hassabis said that Project Astra's artificial intelligence can now process a lot of information through multi-modal design, but shortening the response time to that of a normal conversation is actually a very difficult challenge.
