Google Mulling Gemini AI-Powered Chatbot to Tell Personal Life Stories Using Photos, Search Activity: Report

Google is considering the development of a new chatbot that is capable of telling the story of a user’s life based on their photos and search history, according to a recent CNBC report. The search giant could use large language models (LLMs), such as the recently unveiled multimodal Gemini model, as part of a new AI project. Gemini is touted to compete with OpenAI’s GPT-4 model, and Google claims that its top-of-the-line model outperforms its closest competitor on some benchmarks.

A CNBC report that cites internal documents states that one of Google’s AI teams has suggested that the company develop an AI-based technology to use data from users’ smartphones — including photos and their search activity — that will be consumed by an AI-powered chatbot. The project, dubbed Project Ellman, can then use the information to provide answers to “previously impossible questions”, according to the report.

Instead of simply relying on “just pixels with labels and metadata”, Project Ellman would try to spot patterns in a user’s photos, studying photos and memories before and after the image to gain context, according to the report. The company’s internal document also envisions “Ellman Chat” becoming “Your Life Story Teller.”

Google currently collects user’s photos that are stored on the company’s servers as part of its Google Photos backup and sync feature. The company did not specify whether the data source would be from Google Photos synced to the cloud, or whether the images would be processed on the user’s device.

“This was an early internal exploration and, as always, should we decide to roll out new features, we would take the time needed to ensure they were helpful to people, and designed to protect users’ privacy and safety as our top priority”, a company spokesperson told the publication.

It is unclear whether Google is actively working on adding support for such a personalised AI chatbot that relies on its new Gemini AI models, which were unveiled by the company last week. Google’s most powerful model — Gemini Ultra — won’t be available until next year and is capable of outperforming OpenAI’s GPT-4 model in some tests, according to Google.