Google is taking on OpenAI’s GPT 4 Turbo model with a new offering of its own called Gemini 1.5, which boasts a much bigger input context length than before.

Google DeepMind CEO Demis Hassabis claims that Gemini 1.5 Pro, which is the first of its kind in its new generation of AI models, boasts the same capabilities as Gemini 1.0 Ultra, but can run with less computing power. This was made possible by a new Mixture-of-Experts (MoE) architecture that makes the AI model more efficient to train and deploy.

Much like GPT 4 Turbo, Gemini 1.5 Pro also boasts a standard context window of 128,000 tokens. which is a defining feature of this new offering. However, a limited amount of users, including some developers and enterprise customers, will even be able to use a mind-boggling 1 million context-length window, making it an industry first.

Google says that this feature will make it easy for professionals to handle significantly large datasets, for instance,  an hour of video, 11 hours of audio, codebases with more than 30,000 lines, or documents with more than 700,000 words. Anthropic’s latest Claude 2.1 model boasts an input limit of 200,000 tokens, which is higher than GPT 4 Turbo’s 128,000 tokens.

Google’s researchers have even tested Gemini 1.5 Pro with 10 million tokens, making it fit for massive data troves.

Gemini 1.5 Pro also went through the “Needle In A Haystack” test, which analyzes an AI model’s ability to find text in large datasets. Google’s new model was able to find the right text 99% of the time among 1 million tokens in this test. According to Google Gemini’s team, the new model was also able to find hidden keywords in an audio clip that was nearly a day long and retrieved information from random frames in a 3-hour video.

Other than that, Gemini Pro boasts improvements in various domains, including Math, Science, Reasoning, Multilinguality, Video Understanding, and Code.

Jeff Dean, the Chief Scientist at Google DeepMind said: “We’ll also introduce 1.5 Pro with a standard 128,000 token context window when the model is ready for a wider release. Coming soon, we plan to introduce pricing tiers that start at the standard 128,000 context window and scale up to 1 million tokens, as we improve the model.”

Google is offering a limited preview of Gemini 1.5 which will be accessible to developers and enterprise customers through the AI Studio and Vertex AI.

Via: The Decoder