API Rate Limiting and Throttling Architecture

Description

You are a backend architect specializing in API security and traffic management. Design a production-grade rate limiting system.

## Configuration
- **Language**: [LANGUAGE e.g. Go / Java / Node.js / Python / C#]
- **Backend Store**: [STORE e.g. Redis / Memcached / DynamoDB / in-memory]
- **Architecture**: [ARCH e.g. API gateway / middleware / sidecar proxy]
- **Scale**: [SCALE e.g. 1K / 10K / 100K requests per second]
- **Tiers**: [TIERS e.g. free:100rpm / pro:1000rpm / enterprise:10000rpm]

## Implementation Components

### 1. Rate Limiting Algorithms
- Token bucket implementation for burst handling
- Sliding window log for precise counting at [SCALE]
- Fixed window counter for high-performance scenarios
- Adaptive rate limiting based on server load
- Per-endpoint, per-user, and per-IP limiting

### 2. Distributed Coordination
- [STORE]-based rate counter with atomic operations
- Race condition handling with Lua scripts (Redis) or CAS
- Clock synchronization strategy for multi-region
- Graceful degradation when [STORE] is unavailable
- Eventual consistency trade-offs at [SCALE]

### 3. Response Headers & Client Experience
- Standard headers: X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, Retry-After
- HTTP 429 response with structured error body
- Quota usage API endpoint for [TIERS] visibility
- Client SDK with built-in backoff and retry logic

### 4. Monitoring & Analytics
- Real-time rate limit hit dashboards
- Top consumers and abuse pattern detection
- Tier upgrade recommendations based on usage
- Alert on sustained rate limit violations

Generate complete [LANGUAGE] implementation with tests, configuration for [TIERS], and deployment guide for [ARCH].

Free to copy and use. Compatible with Claude 4 Opus, GPT-5, Gemini 2.5 Pro, Llama 4 Maverick.

Usage Instructions

Tags

Compatible AI Models

Version History

Related Prompts