Amazon has revealed its latest advancements in custom chips for AI model training and inferencing
Amazone in response to the growing demand for generative AI and the ongoing GPU shortage The scarcity of high-performance GPUs, including Nvidia’s, has prompted tech giants to explore alternative solutions, leading to the development of specialized chips.
At the annual re:Invent conference, Amazon unveiled two groundbreaking chips aimed at addressing the GPU shortage and enhancing AI capabilities. The first chip, AWS Trainium2, is tailored for model training and promises up to 4 times better performance and 2 times better energy efficiency compared to its predecessor, Trainium. This chip will be available in EC Trn2 instances in clusters of 16 chips within the AWS cloud, with the capability to scale up to 100,000 chips in the EC2 UltraCluster product.
According to Amazon, a cluster of 100,000 Trainium chips can significantly expedite the training process for large language models, boasting the ability to train a 300-billion parameter AI model in weeks rather than months. This is a substantial advancement, with the parameters representing the learned aspects of a model crucial for its performance in tasks such as text generation.
The second chip, Graviton4, is designed for inferencing and is based on Arm architecture. As the fourth generation in Amazon’s Graviton chip family, Graviton4 offers up to 30% better compute performance, 50% more cores, and 75% more memory bandwidth than its predecessor, Graviton3. Notably, all of Graviton4’s physical hardware interfaces are encrypted, providing enhanced security for AI training workloads and sensitive data.
David Brown, VP of AWS compute and networking, emphasized the significance of these advancements, stating, “Silicon underpins every customer workload, making it a critical area of innovation for AWS.” The Trainium2 and Graviton4 chips are poised to revolutionize AI infrastructure, enabling faster model training, cost efficiency, and improved energy efficiency.
While Amazon did not specify the exact availability date for Trainium2 instances, it is anticipated that they will be accessible to AWS customers sometime next year. Meanwhile, Graviton4 will be available in Amazon EC2 R8g instances, which are currently in preview and expected to be generally available in the coming months. These developments mark a significant stride forward in Amazon’s commitment to delivering state-of-the-art cloud infrastructure tailored to real customer workloads.
