Fix AWS Bedrock ThrottlingException 429 (2026 Guide)

AWS Bedrock’s ThrottlingException 429 hits when your app exceeds the rate limits for a specific foundation model. Unlike OpenAI’s rate limits (per organization), Bedrock has per-model, per-region, per-account limits that can trip you even at moderate scale. This 2026 guide walks you through diagnosing which limit you hit, applying exponential backoff, and setting up Provisioned Throughput when on-demand isn’t enough.

The exact error you’re seeing

botocore.exceptions.ClientError: An error occurred (ThrottlingException)
when calling the InvokeModel operation (reached max retries: 4):
Too many requests, please wait before trying again.

The message is vague. Bedrock does not tell you WHICH limit you hit. You need to diagnose by checking your model’s limits, your current usage, and your request pattern.

Bedrock’s rate limit model (understand this first)

Each Bedrock model has two independent limits per AWS account per region:

• Requests Per Minute (RPM): How many InvokeModel calls you can make per minute. Default varies by model.
• Tokens Per Minute (TPM): Total tokens processed (input + output) per minute. Default varies by model.

You hit ThrottlingException when EITHER limit is exceeded. Example defaults for popular models in us-east-1 (2026):

• Anthropic Claude 3.5 Sonnet: 50 RPM, 400k TPM
• Anthropic Claude 3 Haiku: 400 RPM, 800k TPM
• Amazon Titan Express: 400 RPM, 300k TPM
• Meta Llama 3.1 70B Instruct: 100 RPM, 400k TPM
• Mistral Large: 100 RPM, 400k TPM

Check your actual limits at Bedrock console > “Model access” > select model > “Limits” tab.

Step 1: Check which limit you’re hitting

Enable CloudWatch metrics for Bedrock and check:

# Check invocations per minute for a specific model
aws cloudwatch get-metric-statistics \
  --namespace AWS/Bedrock \
  --metric-name Invocations \
  --dimensions Name=ModelId,Value=anthropic.claude-3-5-sonnet-20240620-v1:0 \
  --start-time 2026-08-03T00:00:00Z \
  --end-time 2026-08-03T01:00:00Z \
  --period 60 \
  --statistics Sum

If your invocations per minute peak near your RPM limit, you’re RPM-throttled. If your invocations are well below the RPM limit but you still get throttled, you’re TPM-throttled (probably sending very long prompts or getting long outputs).

Step 2: Add exponential backoff (essential)

Even at low volume, transient throttling happens. Always retry with exponential backoff:

import boto3
import time
import json
from botocore.exceptions import ClientError

bedrock = boto3.client('bedrock-runtime', region_name='us-east-1')

def invoke_with_retry(model_id, body, max_retries=6):
    for attempt in range(max_retries):
        try:
            return bedrock.invoke_model(modelId=model_id, body=json.dumps(body))
        except ClientError as e:
            error_code = e.response.get('Error', {}).get('Code')
            if error_code == 'ThrottlingException':
                wait = min(2 ** attempt, 60)  # cap at 60 seconds
                print(f"429 hit, retry {attempt+1} in {wait}s")
                time.sleep(wait)
            else:
                raise
    raise Exception(f"Max retries exceeded for model {model_id}")

# Usage
response = invoke_with_retry(
    model_id="anthropic.claude-3-5-sonnet-20240620-v1:0",
    body={
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 1024,
        "messages": [{"role": "user", "content": "Hello"}]
    }
)

Exponential backoff (1s, 2s, 4s, 8s, 16s, 32s) gives the throttling window time to reset. Cap at 60 seconds so single requests don’t hang forever.

Step 3: Use the boto3 built-in retry adapter

boto3 has a built-in retry adapter that handles ThrottlingException automatically:

import boto3
from botocore.config import Config

config = Config(
    region_name='us-east-1',
    retries={
        'max_attempts': 10,
        'mode': 'adaptive'  # adaptive backoff based on API response
    }
)

bedrock = boto3.client('bedrock-runtime', config=config)

Adaptive mode adjusts wait times based on the API’s rate-limit signals. Less code than manual retry logic and often more efficient.

Step 4: Batch requests when possible

If your app can batch multiple prompts into one InvokeModel call, use InvokeModelWithResponseStream or the Bedrock Batch Inference (2026 GA feature):

# Batch inference for high-volume workloads (cost-effective, not real-time)
bedrock_batch = boto3.client('bedrock')

job = bedrock_batch.create_model_invocation_job(
    jobName='daily-batch',
    modelId='anthropic.claude-3-5-sonnet-20240620-v1:0',
    inputDataConfig={
        's3InputDataConfig': {
            's3Uri': 's3://my-bucket/prompts.jsonl',
            's3InputFormat': 'JSONL'
        }
    },
    outputDataConfig={
        's3OutputDataConfig': {
            's3Uri': 's3://my-bucket/results/'
        }
    },
    roleArn='arn:aws:iam::123456789:role/BedrockBatchRole'
)

Batch inference costs 50% less than real-time InvokeModel and does not count against your on-demand rate limits. Ideal for offline enrichment, embedding generation, or overnight processing.

Step 5: Cross-region inference (2026 feature)

Bedrock added cross-region inference in 2025-2026. Route requests across multiple regions to spread load:

# Use the inference profile instead of the model ID directly
inference_profile_arn = "arn:aws:bedrock:us-east-1:123456789:inference-profile/us.anthropic.claude-3-5-sonnet-20240620-v1:0"

response = bedrock.invoke_model(
    modelId=inference_profile_arn,  # not the raw model ID
    body=json.dumps({...})
)

Bedrock automatically routes to the least-throttled region behind the profile. Effective ~3x TPM for the same model when three regions are eligible.

Step 6: For production, buy Provisioned Throughput

If your production traffic is high and predictable, buy Provisioned Throughput (PT). PT gives you dedicated model units with predictable capacity:

1. Bedrock console > “Provisioned throughput” > “Purchase Provisioned throughput”.
2. Select model (must be a fine-tuned model or a base model that supports PT).
3. Select model units (each unit ≈ 500 tokens/second for Claude 3.5 Sonnet).
4. Commit for 1 month or 1 year (annual = ~40% discount).
5. Direct invocations to your PT ARN instead of the on-demand model ID.

response = bedrock.invoke_model(
    modelId="arn:aws:bedrock:us-east-1:123456789:provisioned-model/abc123",
    body=json.dumps({...})
)

Cost trade-off: PT is expensive (thousands of USD per month) but eliminates throttling entirely. Only makes sense for production apps with steady traffic above ~30% of your on-demand TPM cap consistently.

Request a quota increase (free, before buying PT)

Before committing to PT, request higher on-demand limits:

1. AWS Console > Service Quotas > AWS Services > “Amazon Bedrock”.
2. Find the quota for your model (e.g., “Anthropic Claude 3.5 Sonnet on-demand RPM”).
3. Click “Request quota increase” and specify your target.
4. Provide use-case details: expected daily volume, business justification.
5. Approval takes 3-10 business days.

AWS often grants 2-5x increases without questions if you have a real production use case. Try this before buying PT.

Common mistakes causing 429s

• No retry logic. Every production Bedrock caller should have exponential backoff. Transient 429s are normal even at low volume.
• Loading a large PDF into every request. If you send the same 50k-token document with every prompt, you burn TPM fast. Use prompt caching or store the doc once, reference by ID.
• Missing max_tokens cap. Without max_tokens, Claude can generate very long responses that consume TPM. Set a reasonable cap (like 1024 or 2048) unless you really need long output.
• Parallel calls without limiting concurrency. Fire-and-forget parallel calls will spike your RPM. Use a semaphore or queue to cap concurrent requests.
• Using the wrong region. us-east-1 has the highest default limits. If you deployed in a smaller region (like ap-southeast-1 for Manila), your limits are lower. Consider cross-region inference profiles.

Verification: monitor throttling after your fix

# Watch ThrottleCount over the last hour
aws cloudwatch get-metric-statistics \
  --namespace AWS/Bedrock \
  --metric-name Throttles \
  --dimensions Name=ModelId,Value=anthropic.claude-3-5-sonnet-20240620-v1:0 \
  --start-time $(date -u -d '1 hour ago' '+%Y-%m-%dT%H:%M:%SZ') \
  --end-time $(date -u '+%Y-%m-%dT%H:%M:%SZ') \
  --period 60 \
  --statistics Sum

After adding retries and quota fixes, throttle count should drop to near zero. If it stays high, escalate to PT or cross-region inference.