I tried 8 of our newest AI products and updates

Jun 05, 2024

[[read-time]] min read

Chaim Gartenberg

Keyword Contributor

At I/O, we don’t conscionable denote a clump of news, similar new Gemini models, AI agents and Android updates — we fto developers, reporters and partners acquisition immoderate of that newness successful enactment for the precise archetypal clip done merchandise demos.

This year, I was fortunate capable to walk the time astatine Shoreline Amphitheatre, wherever I/O happens, and dive into each of the demos. Here’s the wrong scoop connected a fewer of them.

For my archetypal demo of the day, I watched Gemini Advanced parse done a 20-plus leafage spot lease, afloat of analyzable ineligible phrasing and gotchas. I could past inquire questions astir the lease, similar whether my landlord would let maine to person a favored dog, oregon whether I’d person to wage immoderate other fees. (I’m personally looking guardant to being capable to usage the diagnostic to decipher my adjacent convoluted lease erstwhile my flat is up for renewal.)

The adjacent demo took things up a notch: Two Googlers fed Gemini the PDF of an full economics textbook that was hundreds of pages long. It would person taken maine hours to work the book, but Gemini was capable to summarize and item important topics to survey successful seconds. It besides made a aggregate prime quiz — creating not conscionable the azygous close prime but besides 3 incorrect answers to effort and travel maine up — to assistance maine hole for a theoretical upcoming test.

Both these demos took vantage of Gemini 1.5 Pro — which added the longest discourse model of immoderate large-scale instauration exemplary erstwhile we debuted it earlier this year. We’re opening up aboriginal Gemini 1.5 Pro entree to Gemini Advanced subscribers and giving them the quality to upload documents to the instrumentality close from Drive, truthful they tin usage Gemini to summarize oregon analyse documents up to 1,500 pages long.

Gemini 1.5 Pro is besides coming to the broadside sheet of Workspace apps similar Gmail, Docs, Sheets, Slides, and Drive. To spot this successful action, I utilized Gemini successful Gmail to summarize a illustration email of a play schoolhouse report, and propulsion retired circumstantial details similar which activities were for 7th-grade students, oregon what the packing database for an overnight travel was.

The improved agelong discourse model tin adjacent propulsion accusation from aggregate documents erstwhile responding to a azygous prompt. In the broadside sheet successful Docs, I asked for assistance penning a illustration missive to a imaginable occupation campaigner — successful the punctual I linked to the occupation statement papers and the applicant’s PDF portfolio, some of which were successful my Drive — and instantly received a email draft, which factored successful applicable details from some documents.

Gemini 1.5 Pro isn’t our lone shiny caller model, though: I besides got to effort the freshly-announced Imagen 3, our highest-quality text-to-image exemplary yet. One of the caller abilities I was excited astir was its quality to make decorative substance and letters, truthful I enactment it done its paces. I started by asking for a stylized alphabet — similar letters spelled retired successful jam connected toast, oregon with metallic balloons floating successful the sky. Imagen 3 generated a afloat alphabet of letters, which I could past usage to benignant retired my ain (delicious) menus.

After my Imagen 3 interlude, I continued with much Gemini demos. In 1 of them, I could propulsion up Gemini’s overlay connected an Android telephone and inquire questions astir thing connected the screen. This truly showed however we’re not lone expanding what you tin inquire Gemini, but we’re besides making Gemini discourse aware, truthful it tin expect your needs and supply adjuvant suggestions.

The usage lawsuit present was a lengthy oven manual. Whether it's a demo oregon existent life, that's not thing I'd beryllium excited astir reading. Instead of skimming done the document, I pulled up Gemini and instantly got an "Ask this PDF" suggestion. I tested questions similar "how bash I update the clock" and rapidly got close answers. It worked conscionable arsenic good with YouTube videos. Instead of watching a 20-minute workout video, I asked a speedy question astir however to modify planks, got an answer, and was connected my mode onto the adjacent demo, wherever I tested a caller speech mode called Gemini Live that lets you speech with Gemini successful the app, nary typing required.

Speaking with Gemini was a antithetic acquisition than the accepted chatbot interface: Gemini’s answers are a batch much conversational than the paragraphs of texts and bullet-pointed lists you mightiness usually get. In my demo, I learned you could adjacent chopped disconnected Gemini successful the mediate of an answer. After asking for a database of kid’s activities for a summertime vacation, I was capable to interrupt a database of suggestions to dive successful deeper connected what materials I’d request for tie-dying a shirt.

The Project Astra — oregon “advanced seeing and talking responsive agent” — demo took things a measurement further to amusement the cutting borderline of wherever our conversational AI projects are heading.

Instead of lone moving with immoderate is connected your screen, oregon the accusation that you’ve typed into a chat box, Astra’s multimodal capabilities tin recognize conversational code prompts and unrecorded video feeds astatine the aforesaid clip to unlock caller kinds of AI experiences.

Astra’s alliteration demo started retired simply: I showed the camera — an overhead camera successful this setup, but Astra tin besides usage a telephone camera oregon a camera connected a wearable instrumentality — an object, similar a banana oregon a portion of bread, and Gemini riffed connected it with an alliterative sentence. I added much objects, and Gemini kept the speech going, from “Bright bananas bask beautifully connected the board” with a azygous effect to “Culinary creations tin drawback the eye” erstwhile presented with a full buffet board.