Google has introduced a new artificial intelligence (AI) model called Gemini that surpasses the performance of ChatGPT in most tests and exhibits advanced reasoning in various formats. Gemini is capable of reviewing and providing feedback on a student’s physics homework, among other tasks. This marks the first AI model announcement since the global AI safety summit last month, where tech companies pledged to collaborate with governments on testing advanced systems. Google is currently in discussions with the newly established UK AI Safety Institute to conduct tests on Gemini’s most powerful version, set to be released in 2023.
According to Google, Gemini outperforms leading AI models, including ChatGPT’s powerful version GPT-4, in 30 out of 32 benchmark tests, particularly in reasoning and image understanding. It also surpasses GPT-3.5, which underlies the free-access version of ChatGPT, in six out of eight tests. The Gemini model encompasses three versions and is described as multimodal, meaning it can comprehend and process text, audio, images, video, and computer code concurrently.
Google plans to integrate Gemini into its products, including its search engine, and will initially release it in more than 170 countries, including the United States, as an upgrade to its chatbot, Bard. However, the Bard upgrade will not be available in the UK and Europe until regulatory clearance is obtained.
DeepMind, the Google subsidiary responsible for developing Gemini, stated that this project presented its most complex undertaking to date. The model comprises two smaller iterations, Pro and Nano, which will be released alongside the Bard chatbot on Wednesday. While Pro can be accessed through Bard, Nano will be compatible with mobile phones running Google’s Android system.
Google is currently conducting external testing on the most powerful version of Gemini, Ultra, which is expected to be released in early 2024. Ultra has already outperformed human experts in multitasking tests, achieving a score of 90% on MMLU, which covers subjects such as mathematics, physics, law, medicine, and ethics. This version will be integrated into a code-writing tool called AlphaCode2, which Google claims can outperform 85% of computer programmers at competition level.
Google’s DeepMind stated that Gemini had yet to overcome the challenge of hallucinations, or providing incorrect answers. While Gemini understands prompts from multiple formats, including text and code, the publicly released Pro and Nano versions can only respond in text or code format.
Gemini’s capabilities were showcased in promotional videos, including demonstrations of the Ultra model correctly understanding and providing detailed explanations for a student’s handwritten physics homework. Additionally, Gemini’s Pro version correctly identified a drawing of a duck and determined the film depicted in a smartphone video. Demis Hassabis, CEO of DeepMind, highlighted that Gemini displayed advanced reasoning and possessed novel capabilities not previously seen in AI models.
Gemini’s release has raised concerns about AI development, ranging from the dissemination of mass-produced misinformation to the creation of superintelligent systems that may evade human control. While Gemini represents a step towards artificial general intelligence (AGI), researchers at DeepMind noted that there are still aspects being researched and improved upon to achieve the final form of AGI.
DeepMind gathered training data for Gemini from various sources, including the open web. However, the use of copyrighted content available online by AI companies to build models has sparked protests from the publishing and creative industries.
With Gemini, Google aims to revolutionize AI capabilities and provide enhanced functionalities to its users. The model’s integration into the company’s existing products signifies the beginning of a new era for Google’s AI advancements. As Gemini undergoes further testing and refinement, it is expected to set new benchmarks for AI performance and pave the way for future developments in the field.
