The race is on at Google Deepmind, it is already coming out with Gemini 1.5 within months of the launch of Gemini 1.0. It sounds like a new version of an existing model, but nothing could be further from the truth. Google is opting for a Mixture-of-Experts architecture that is much more intelligent and advanced than previous models. Gemini 1.5 is available in preview only for now.
The pace at which Google is developing its AI models is fast, as is the innovation involved. Today, Google presents Gemini 1.5, a new model that works differently from previous models.
Gemini 1.5 architecture
Google opts for a so-called Mixture-of-Experts architecture. This is fundamentally different from previous foundation models. In Gemini 1.5, the foundation model consists of a set of smaller neural networks, whereas normally, a foundation model consists of one large neural network. We can say that Gemini 1.5 consists of hundreds of small foundation models that work together and act as one large model. Based on the input context or query from the user, the model determines which submodel is most relevant, and that will then generate the response.
Model is more efficient and faster
It may sound simple, but Google has realised a new, highly advanced model with Gemini 1.5. This model also offers several advantages. For example, it is a lot faster than its predecessor, is much more efficient and therefore more economical to use. Both for training and serving (inferencing) AI data.
As a result, Google now also sees the possibility of making the context window significantly larger. In Gemini 1.0 the limit was still 32,000 tokens (a token can be a letter or a word), in Gemini 1.5 Google is now testing with a limit of 1 million tokens. Enough for 30,000 lines of code, more than 700,000 words, 1 hour of video content or 11 hours of audio. This makes Gemini 1.5 a lot more powerful than its predecessors.
Much better prompt processing
Google further states that learning skills and processing prompts are now much better. Previously, you needed multiple prompts to adjust a model or learn exactly what kind of output you wanted. With Gemini 1.5 Pro, you can immediately put all requirements on the table with one long prompt. This is without ignoring any requirements, something that is common with other models.
Gemini 1.5 Pro is better in 87% of benchmarks
Google also unleashed Gemini 1.5 Pro on the necessary benchmarks for AI models and claimed that the new version improved by a whopping 87% of these benchmarks. Compared to 1.0 Ultra, the quality level is about the same, resulting in better efficiency.
19 hours ago
