“`html
The Collaboration Between Salesforce and AWS
This post is a joint collaboration between Salesforce and AWS and is being cross-published on both the Salesforce Engineering Blog and the AWS Machine Learning Blog. Salesforce, Inc. is an American cloud-based software company headquartered in San Francisco, California.
Salesforce’s Approach to Artificial General Intelligence
Salesforce is building toward artificial general intelligence (AGI) for business, enabling predictive and generative functions within their flagship software-as-a-service (SaaS) CRM, and working toward intelligent automations using artificial intelligence (AI) as well as agents.
Salesforce Einstein AI Technologies
Salesforce Einstein is a set of AI technologies that integrate with Salesforce’s Customer Success Platform to help businesses improve productivity and client engagement. Einstein has a list of over 60 features, unlocked at different price points and segmented into four main categories: machine learning (ML), natural language processing (NLP), computer vision, and automatic speech recognition.
Optimizing Performance with Amazon SageMaker
In this post, we share how the Salesforce Einstein AI Platform team boosted latency and throughput of their code generation LLM using Amazon SageMaker. The team started looking at solutions to host CodeGen, Salesforce’s in-house open source LLM for code understanding and code generation.
Performance Improvements and Key Metrics
Before using SageMaker, CodeGen models had a lower token-per-second rate and higher latencies. With SageMaker optimization, the team observed significant improvements in both throughput and latency.
Enhancements and Key Lessons Learned
The team faced challenges when integrating SageMaker, but successfully enhanced the platform to include specific functionalities essential for their projects. They also identified opportunities to improve resource efficiency by hosting multiple LLMs on a single GPU instance.
“`
Images should be inserted accordingly if present in the original article.
Leave a Reply