Llama 3.1 versions are now accessible in Amazon SageMaker JumpStart | AWS Machine Learning Blog

Llama 3.1 Collection Overview

Today, the state-of-the-art Llama 3.1 collection of multilingual large language models (LLMs) is now available for deployment through Amazon SageMaker JumpStart. The Llama 3.1 models offer pre-trained and instruction-tuned generative AI models in various sizes, catering to a wide range of users.

Model Details and Features

The Llama 3.1 multilingual LLMs come in sizes of 8B, 70B, and 405B, all designed to support long context lengths and optimized for inference with grouped query attention. These models excel in multilingual dialogue use cases and outperform many existing chat models in industry benchmarks.

Architectural Insights and Usage

Llama 3.1’s core is an auto-regressive language model based on an optimized transformer architecture. The models have been fine-tuned using supervised fine-tuning and reinforcement learning with human feedback to prioritize helpfulness and safety. Additionally, Llama 3.1 also includes instruct variants, specifically tuned for tool usage across different applications.

Model Deployment and Security

SageMaker JumpStart provides access to a plethora of foundation models (FMs) that can be customized for specific use cases. Deployment of models is handled in a secure environment on dedicated SageMaker Inference instances, ensuring data security and compliance within your virtual private cloud (VPC).

Utilizing SageMaker JumpStart

With interfaces like Amazon SageMaker Studio and SageMaker Python SDK, users can seamlessly discover and deploy pre-trained models for various AI and ML tasks. The SageMaker experience allows for flexibility in deploying models for inference and fine-tuning based on specific requirements.

Empowering Model Deployment

By leveraging SageMaker JumpStart, data scientists and ML engineers can harness the capabilities of advanced models like Llama 3.1 to enhance their inference workflows. With Meta’s cutting-edge models available, developers can explore and deploy models efficiently and effectively.

Meet the Team Behind the Innovation

A team of passionate professionals, including Saurabh Trikande, James Park, Dr. Kyle Ulrich, Jonathan Guinegagne, and Christopher Whitten, drive the development and deployment of advanced ML solutions on Amazon SageMaker. Their expertise and dedication contribute to the continuous innovation in AI and machine learning technologies.