You can use a cross Region inference profile in place of a foundation model to route requests to multiple Regions. To track costs and usage for a model, in one or multiple Regions, you can use an application inference profile. To learn how to use an inference profile when running model inference, choose the tab for your preferred method, and then follow the steps:
Get Inference Profile This cell retrieves metadata for the application inference profile created earlier by using its ARN. The get_inference_profile method takes the Inference Profile's ARN as input and calls the Bedrock API to fetch detailed metadata about the profile, such as configuration settings, model information, and associated tags. This retrieval allows you to verify and inspect the ...
Amazon Bedrock Inference Profiles is a powerful feature that allows you to track costs and usage... Tagged with aws, bedrock, genai, nova.
Amazon Bedrock Inference Profiles are custom, logical endpoints that sit in front of foundation models. Instead of calling a model directly, your application calls an inference profile.
Mastering Amazon Bedrock Inference Profiles | by Furkan Gungor | Apr ...
Amazon Bedrock Application Inference Profiles This notebook demonstrates how organizations can implement, test, validate, and operationalize Amazon Bedrock application inference profiles. The aim is to provide a comprehensive understanding of how to manage and utilize application inference profiles effectively.
We’ll be working with application inference profiles that allow you to define custom tags on top of existing system inference profiles, or create our own profile from a single model. It doesn’t appear that you can build your own custom cross-region inference profile at present which is a shame.