Rate Limiting Design

1. Algorithm Choice

Use a hierarchical token bucket as the primary enforcement algorithm, supplemented by a short sliding-window burst detector.

Why Token Bucket

Token bucket is the best primary mechanism because it:

• Allows legitimate short bursts while enforcing a sustainable average rate.
• Requires constant memory and near-constant processing per bucket.
• Works efficiently in a distributed data store such as Redis.
• Supports different refill rates and capacities for each tier.
• Provides a direct way to calculate when another request can be accepted.

A pure sliding-window log gives precise enforcement but is more expensive during scraping bursts because it stores timestamps for many requests. A sliding-window counter is cheaper but can still produce boundary artifacts and does not model burst capacity as naturally.

Hybrid Enforcement

Every request must pass all applicable controls:

1. Per-second token bucket: absorbs ordinary request bursts.
2. Per-minute token bucket: limits sustained traffic.
3. Per-day quota: controls total tier consumption.
4. Short sliding-window detector: identifies scraping patterns that exploit burst capacity, such as repeated bucket draining across many resources or identities.
5. Global and endpoint-specific safeguards: protect infrastructure and expensive routes.

Reject the request if any mandatory bucket or quota is exhausted.

Use atomic operations, preferably a Redis Lua script, to refill, inspect, and consume tokens. Apply local in-process emergency limits if the central rate-limit store is unavailable; fail open only for trusted paid traffic and fail closed for anonymous traffic on expensive endpoints.

2. Per-Tier Limits