Google announced the launch ofNew open source model Gemma 2It is optimized for TPU and GPU acceleration, and can output twice the model running performance, and can correspond to up to 2 billion sets of parameters. However, it also provides a small-scale version that can correspond to 270 billion sets of parameters. In the future, an even smaller-scale version with 90 billion sets of parameters will also be provided, which will be executable on mobile phones.
In the earlier descriptionGemma 2 can be obtained through Kaggle, a data modeling and data analysis competition platform, or through the free service of Colab, a web programming platform called Colaboratory. Academic researchers can also apply for its use through research projects.
In related simulation testing, the 2 billion parameter version of Gemma 270 surpassed the 700 billion parameter Llama 3 in fine-tuning mode, surpassing the 3400 billion parameter Nemotron 4, as well as models such as Claude 3 Sonnet, Command R+, and Qwen 72B. The 90 billion parameter version even became the best-performing model under 150 billion parameters.
According to the documentation, the 90-billion-parameter version of Gemma 2 was trained on a cluster of 4096 TPU v4s, while the 270-billion-parameter version was trained on a cluster of TPU v5ps, using a total of 6144 chips. Gemma 2's overall architecture is redesigned, using a similar computational model to Gemma 1.1. However, with increased learning supervision and model merging, Gemma 2 offers significant improvements over Gemma 1.1 in programming, mathematics, reasoning, and security.
In addition, the 2 billion parameter scale version of Gemma 270 can perform full-precision inference with high efficiency on Google Cloud TPU servers, NVIDIA A100 80GB Tensor Core GPUs, or H100 Tensor Core GPUs, maintaining high-performance computing while reducing operating costs, allowing enterprises and developers to execute and deploy artificial intelligence services in a more economical way.
Google also emphasized the responsible creation of Gemma 2, explaining the application of Gemma 2's security features and following internal security processes to filter pre-training data to avoid potential risks such as bias.
