Rate Limit Information from Anthropic API Response Headers Using ChatResponseMetadata in Spring AI

1. Overview

Spring AI ChatResponseMetadata class exposes metadata associated with a chat response, including information about the model and other response-related details. Depending on the provider and the metadata available, this information can also include rate limit details.

In this tutorial, we’ll explore how to access rate limit metadata returned by Anthropic through Spring AI. Also, we’ll examine how the getLimit() method exposes this information.

2. Maven Dependencies

To begin, let’s bootstrap a simple Spring Boot AI application and add the spring-boot-starter-web and the spring-ai-starter-model-anthropic dependencies to our pom.xml:

<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.ai</groupId> <artifactId>spring-ai-starter-model-anthropic</artifactId> </dependency>

The spring-boot-starter-web dependency provides Spring MVC and configures an embedded web server for our application. Next, the spring-ai-starter-model-anthropic dependency integrates Anthropic’s chat models with Spring AI. It allows us to interact with them through Spring AI abstractions.

3. Getting the Rate Limit Metadata

First, let’s configure the Anthropic model in our application.properties file:

spring.ai.anthropic.api-key=${ANTHROPIC_API_KEY} spring.ai.anthropic.chat.options.model=claude-sonnet-5-5 spring.ai.anthropic.chat.options.max-tokens=1024

Here, we load the Anthropic API key from the ANTHROPIC_API_KEY environment variable. Then, we specify the model to use and set the maximum number of tokens the model can generate in a response. In this case, we are using the claude-sonnet-5-5 model.

3.1. MetadataResponse Record

Moving on, let’s define a Java record to represent the response returned by our application:

public record MetadataResponse( String model, Long requestsLimit, Long requestsRemaining, Duration tokensReset, Long tokensLimit, Long tokensRemaining, Integer promptTokens, Integer completionTokens, String content) { } 

In the code above, we provided a structured representation of the data, such as model name, rate-limit information, token usage, and generated output to the client.

3.2. The Controller Class

Next, let’s create a controller class and inject the ChatClient to initialize the ChatClient:

@RestController public class ChatResponseMetaController { private final ChatClient chatClient; public ChatResponseMetaController(ChatClient.Builder builder) { this.chatClient = builder.build(); } }

With the ChatClient initialized, let’s add an endpoint that generates a short story. Also, the endpoint returns the associated model metadata, rate limits, and token usage:

@GetMapping("/story-gen") public MetadataResponse chat(@RequestParam(defaultValue = "Tell a story in less than 100 words") String message) { ChatResponse chatResponse = chatClient.prompt() .user(message) .call() .chatResponse(); ChatResponseMetadata metadata = chatResponse.getMetadata(); RateLimit rateLimit = metadata.getRateLimit(); Usage usage = metadata.getUsage(); AssistantMessage output = chatResponse.getResult().getOutput(); return new MetadataResponse( metadata.getModel(), rateLimit.getRequestsLimit(), rateLimit.getRequestsRemaining(), rateLimit.getTokensReset(), rateLimit.getTokensLimit(), rateLimit.getTokensRemaining(), usage.getPromptTokens(), usage.getCompletionTokens(), output.getText()); }

Here, we use the ChatClient to send the user’s prompt to the configured Anthropic model and retrieve the resulting ChatResponse. Then, we call the getMetadata() method to access the response metadata. From the metadata, getRateLimit() returns a RateLimit object containing the request and token limits reported by the provider.

We also call getUsage() to retrieve the token usage information and extract the generated text from the response. Finally, we populate the MetadataResponse record with these values, allowing our endpoint to return the generated story alongside the available rate limit and usage information.

3.3. GET Request

Moving on, let’s make a GET request to /story-gen to see the response:

{ "model": "claude-sonnet-5-5", "requestsLimit": 10000, "requestsRemaining": 9999, "tokensReset": "PT0S", "tokensLimit": 12000000, "tokensRemaining": 12000000, "promptTokens": 20, "completionTokens": 200, "content":"..." }

In the above output, the request limit is 10000, with 9999 requests remaining after the request. The token-related fields show the reported token budget and the time until the token limit resets. The promptTokens and completionTokens fields indicate the tokens used to process the prompt and generate the response, respectively.

4. Conclusion

In this article, we learned how to retrieve rate-limit metadata for Anthropic chat responses using the ChatResponseMetadata class. We accessed the RateLimit object through the getRateLimit() method and explored how to retrieve request limits, remaining requests, token limits, and token usage information.

The full source code for the examples is available over on GitHub.

The post Rate Limit Information from Anthropic API Response Headers Using ChatResponseMetadata in Spring AI first appeared on Baeldung.