Gemini Now Has Ears
And a Host of New Features for Gemini 1.5 Pro
In case you missed the announcement on Tuesday, Google's Gemini 1.5 Pro has leveled up, and it's packed with groundbreaking features - most notably the ability to understand audio! This opens doors to a whole new realm of possibilities for developers and everyday users alike. Here’s a summary of what we’ve just announced:
Broader Availability: Gemini 1.5 Pro empowers developers across 180+ countries to build groundbreaking AI applications - including the recently launched support in Canada
Audio Understanding: Experience the breakthrough native audio comprehension capabilities of Gemini, unlocking a host of new use cases
Video Intelligence: Delve deeper into video content with Gemini's ability to analyze both audio and visual elements, extracting even deeper insights.
System Instructions: Guide the model's behavior with unparalleled accuracy using system instructions, tailoring its responses to your specific needs.
JSON Mode: Structured data is now at Your fingertips with this new mode that can effortlessly extract and utilize it
Better Function Calling: Experience enhanced control and reliability with refined function calling capabilities.
Next-Generation Text Embeddings: Leverage the power of our cutting-edge text embedding model, with superior performance against comparable models.
You can read all the details in the Google Developer Blog post:
The Power of Audio Understanding
One of the most exciting aspects of Gemini 1.5 Pro is its audio comprehension ability. Imagine taking voice memos and having them automatically transcribed and analyzed. This is now a reality.
For instance, at a recent birthday party for my twins, I recorded voice memos as we unwrapped gifts, mentioning the gift and giver. Later, I uploaded these recordings to Gemini and asked it to identify who gave each present. By enabling JSON mode, I even received the output in a structured format.
What would have been a chaotic note-taking process became effortless, thanks to Gemini's ability to understand and process audio.
Gemini 1.5 Pro marks a significant step in AI development. With its advanced features, particularly audio understanding, it empowers us to interact with technology in more intuitive and efficient ways.
Now, if only it could help me write those thank-you notes!
…oh wait, it definitely can ;)