Filtering Metadata for Structured Data with Knowledge Graphs for Amazon Bedrock | Amazon Web Services Machine Learning Blog

Introduction to Amazon Bedrock

Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading artificial intelligence (AI) companies like AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon through a single API.

Understanding Retrieval Augmented Generation (RAG)

To equip FMs with up-to-date and proprietary information, organizations use Retrieval Augmented Generation (RAG), a technique that fetches data from company data sources and enriches the prompt to provide more relevant and accurate responses. Knowledge Bases for Amazon Bedrock is a fully managed capability that helps implement the entire RAG workflow, from ingestion to retrieval and prompt augmentation.

Importance of Metadata Filtering

Information about one dataset can be in another dataset, called metadata. Without using metadata, retrieval processes can yield unrelated results, leading to decreased FM accuracy and increased costs in the FM prompt token. Amazon Bedrock’s metadata filtering feature introduced on March 27, 2024, allows the use of metadata fields during the retrieval process, providing more accurate results.

Utilizing Knowledge Bases for Amazon Bedrock

The solution involved creating a knowledge base using supported vector store providers like Amazon OpenSearch Serverless and demonstrating the process with examples using a public dataset, Food.com – Recipes and Reviews. The post details how to structure metadata for accurate retrieval and filter results based on specific criteria. Screenshots and code snippets illustrate the process effectively.

Enhancing Retrieval Accuracy

By using metadata filtering in queries, users can ensure more accurate results from the FM models when retrieving information from the knowledge base. Code examples and step-by-step guides showcase how the metadata filtering feature can be leveraged to improve the relevance of the retrieved data, as well as the effectiveness of the model.

Conclusion

The post elaborates on the significance of metadata filtering in enhancing the accuracy of retrieval processes with Amazon Bedrock knowledge bases. By following the outlined steps and utilizing metadata filtering effectively, users can optimize the performance and precision of their FM models for various applications.