Rate Limiting Fundamentals

TL;DR: Understand token bucket, leaky bucket, and sliding window algorithms and how to implement them in your APIs.

Rate limiting is a strategy to control how many requests a client can make to an API within a given window. Without it, a single misbehaving client can degrade the experience for everyone else.

Token Bucket: A bucket holds tokens that refill at a fixed rate. Each request consumes a token. If the bucket is empty, the request is rejected. This allows bursts up to the bucket size while maintaining a long-term average rate.

Leaky Bucket: Requests enter a queue and are processed at a fixed rate. If the queue is full, the request is dropped. This smooths out bursts but introduces latency.

Fixed Window: Count requests per fixed time window (e.g., 100 requests per minute). Simple but allows burst at window boundaries (double the limit in adjacent windows).

Sliding Window Log: Tracks timestamps of all requests. On each request, remove timestamps older than the window, then count remaining. Accurate but memory-intensive for high-traffic APIs.

Sliding Window Counter: Hybrid approach using counters for the current and previous window. Approximates sliding window behavior with O(1) memory per client.

For most APIs, the token bucket algorithm strikes the best balance. It allows natural burst behavior (a user refreshing a page 5 times quickly) while capping sustained load.

Use the devvkit API Rate Limit Calculator to estimate rate limits for your specific traffic patterns.