Google is integrating its Gemini family of AI models across all its products, including the Android operating system and Pixel phones. At the recent Made by Google event, the company showcased various applications for Gemini, even taking a jab at OpenAI for not releasing new products. Google introduced Gemini Live, similar to ChatGPT Voice, but it’s available only with a paid subscription.
Gemini Advanced, Google’s counterpart to ChatGPT Plus or Claude Pro, costs about $20 per month and comes with other Google One perks, granting access to the most advanced model in the Gemini lineup, Gemini Pro 1.5. Owners of the Pixel 9-series will get this service for free for the first year, but others will need to pay the monthly fee to use Gemini Live. The question arises: is it worth the cost? Besides the voice assistant, the primary selling point is the superior model, but is the improvement noticeable?
Testing between Gemini and Gemini Advanced
Testing between Gemini and Gemini Advanced reveals some differences. Although Gemini Flash 1.5, available for free across all versions, is one of the best small models on the market—fast, capable, and sufficient for most tasks—Gemini Pro 1.5 excels in reasoning, generating longer outputs, and handling complex queries. For example, during testing, Gemini Pro 1.5 was able to analyze a 402-page transcript from Apollo 11, providing insights into conversations, events, and details across the document.
Google has made both image generation and analysis accessible to free and paid users alike, and all Google plugins are available regardless of subscription. The key differences come down to how the models respond to the same queries.
To compare, seven diverse prompts were used, with each model’s responses evaluated based on responsiveness, accuracy, creativity, overall quality, and depth of analysis. This comprehensive assessment highlighted the capabilities of both models.
Explain it like I’m 10:
The prompt asked for a simple explanation of quantum entanglement suitable for a 10-year-old. While both models performed well, Gemini (base) provided a more accurate and deeper explanation, including the concept of superposition.
A short story of hope
The task was to write a 200-word story incorporating themes of climate change, hope, and technology. Gemini (base) won with a more vivid and emotionally resonant narrative, although neither model reached the word limit.
A driverless revolution:
When analyzing the economic impacts of widespread autonomous vehicle adoption, both models delivered detailed responses. However, Advanced won due to better citation of sources, despite both models offering similar quality in argumentation.
A math problem:
For a math problem involving the volume of a sphere, Advanced provided the correct solution, while Gemini (base) did not respond, giving Advanced an easy win despite a minor rounding error.
A leap into code:
The prompt required describing three ethical concerns related to the development of artificial general intelligence (AGI) and suggesting mitigation strategies. Gemini (base) was deemed the winner for introducing a broader range of concerns and offering a more creative approach to solutions.
Generating product names
Prompt: “Generate five creative names for a new line of eco-friendly cleaning products.”
Advanced: Suggests names emphasizing environmental responsibility and effectiveness, such as “Verdant Virtue” and “EcoGleam.”
Gemini: Proposes names like “Purely Clean” and “Squeaky Green,” focusing on simplicity and environmental friendliness.
Winner: Gemini (base) for more creative and diverse options.
Ethical concerns in AGI
Prompt: “Describe three potential ethical concerns related to AGI development and suggest mitigation strategies. Conclude with a summary.”
Advanced: Identifies key ethical concerns and suggests proactive measures, emphasizing the importance of education, fairness, and safety.
Gemini: Covers a broader range of ethical issues and suggests a multidisciplinary approach for effective mitigation.
Overall, the test results were surprising. While more technical tasks might favor Gemini Advanced, the evaluation of general and human-centered use cases demonstrated the strengths of the free Gemini model. For most users, especially those who don’t need or can’t access Gemini Live, the free version is sufficient and offers excellent performance without the additional cost.