Rate limiting sounds straightforward:
Count the requests, define a threshold and reject anything above it.
But what happens when traffic arrives at the exact boundary between two time windows? How should a system handle legitimate bursts? Is perfect accuracy worth the additional memory cost? And how do you prevent race conditions when several servers are updating the same counters concurrently?
These questions quickly turn a seemingly simple component into a genuinely interesting distributed systems problem.
While studying rate limiter design, I decided not to stop at diagrams and theoretical explanations. I implemented the five most commonly discussed rate limiting algorithms and built an interactive simulator that runs all of them against the same traffic stream.
The goal is simple: make their trade-offs visible.
🚦 Try the live simulator:
https://rate-limiter.mhayk.workers.dev/
💻 Explore the complete implementation on GitHub:
https://github.com/mhayk/system-design/tree/main/designs/02-rate-limiter
Why Rate Limiting Matters
A rate limiter controls how frequently a client, user, device or service can perform an operation within a given period.
For example:
- A user may create no more than two posts per second.
- An IP address may attempt to log in only five times per minute.
- A client may call an expensive third-party API no more than 1,000 times per day.
- An entire platform may accept a maximum of 100,000 requests per second.
Rate limiting helps systems:
- protect services from abusive or accidental traffic;
- prevent downstream services from becoming overloaded;
- control infrastructure and third-party API costs;
- enforce fair usage across customers;
- maintain predictable system behaviour during traffic spikes.
However, there is no single perfect rate limiting algorithm.
Each approach makes a different compromise between accuracy, memory consumption, latency, burst tolerance and implementation complexity.
That is exactly what this project demonstrates.
Five Algorithms, One Traffic Stream
The simulator implements five algorithms:
- Token Bucket
- Leaking Bucket
- Fixed Window Counter
- Sliding Window Log
- Sliding Window Counter
Instead of testing each algorithm with a different example, the simulator feeds the same traffic into all five.
This makes it possible to see where their behaviour begins to diverge.
A steady stream of requests can make every algorithm appear correct. The differences become clear only when the traffic becomes irregular, bursty or deliberately hostile.
1. Token Bucket
The Token Bucket algorithm maintains a bucket containing a limited number of tokens.
Tokens are added at a configured refill rate, up to the maximum capacity of the bucket. Every request consumes one token.
If a token is available, the request is accepted. If the bucket is empty, the request is rejected.
Why it is useful
Token Bucket allows short bursts while still enforcing a sustainable average request rate.
Imagine that a client has been inactive for several seconds. During that quiet period, its bucket may become full. When the client suddenly sends several requests together, it can use the stored tokens.
This is often desirable for public APIs, where a single page load may legitimately trigger multiple requests at once.
Main trade-off
The algorithm requires two parameters:
- bucket capacity;
- token refill rate.
Those parameters must be tuned carefully. A bucket that is too large may allow excessive bursts, while a bucket that is too small may reject perfectly valid usage.
For many general-purpose APIs, Token Bucket is a strong default choice.
2. Leaking Bucket
The Leaking Bucket algorithm behaves more like a queue.
Incoming requests are placed into a bucket with a limited capacity. Requests then leave the bucket at a constant, predictable rate.
If the queue is already full, new requests are rejected.
Why it is useful
Leaking Bucket smooths irregular traffic into a stable output rate.
This can be valuable when the downstream system has a hard processing limit, such as:
- a legacy platform;
- a payment provider with a strict transactions-per-second limit;
- a mainframe;
- a hardware device;
- a slow external integration.
Regardless of how bursty the incoming traffic becomes, the downstream service receives work at a controlled pace.
Main trade-off
Old requests can occupy the queue for a significant amount of time.
A large burst may fill the bucket, forcing newer and potentially more relevant requests to wait or be rejected. For interactive APIs, this delay can produce a poor user experience.
Unlike most rate limiters, Leaking Bucket often defers work rather than immediately refusing it.
3. Fixed Window Counter
Fixed Window Counter divides time into predefined windows.
For example, a rule allowing five requests per minute might create windows such as:
- 14:00:00 to 14:00:59;
- 14:01:00 to 14:01:59;
- 14:02:00 to 14:02:59.
Each window has its own counter. Once the counter reaches the configured limit, additional requests are rejected until the next window begins.
Why it is useful
Fixed Window Counter is:
- easy to understand;
- inexpensive to store;
- straightforward to implement with Redis using operations such as
INCRandEXPIRE.
It also works well when the boundary itself has business meaning, such as a daily quota that resets at midnight.
The boundary problem
This is where the simulator becomes particularly useful.
Suppose the limit is five requests per minute.
A client sends five requests at the end of one window and another five immediately after the next window begins.
Both counters remain within their individual limits. However, the system has allowed ten requests within a rolling sixty-second period.
That is twice the intended limit.
The simulator includes a Boundary Attack scenario that reproduces this behaviour visually. It is one of the clearest demonstrations of why an algorithm that appears correct under normal traffic may fail under adversarial traffic.
4. Sliding Window Log
Sliding Window Log solves the fixed window boundary problem by storing the timestamp of every request.
Whenever a new request arrives, the limiter:
- removes timestamps outside the current rolling window;
- adds the new request timestamp;
- counts the remaining timestamps;
- accepts or rejects the request based on that count.
Redis sorted sets are particularly well suited to this approach because timestamps can be used as scores.
Why it is useful
Sliding Window Log is highly accurate.
For any rolling window, the number of accepted requests can be kept within the configured threshold.
This makes it suitable for sensitive, lower-volume operations such as:
- login attempts;
- password resets;
- payment initiation;
- account recovery;
- costly third-party API operations.
Main trade-off
Accuracy has a memory cost.
The limiter must store individual request timestamps. Memory consumption therefore grows with request volume rather than merely with the number of users.
For a high-traffic public endpoint, storing millions of timestamps can become extremely expensive.
Sliding Window Log is often the most accurate algorithm in the comparison, but not necessarily the most practical one at scale.
5. Sliding Window Counter
Sliding Window Counter combines ideas from Fixed Window Counter and Sliding Window Log.
Instead of storing every timestamp, it keeps counters for the current and previous windows.
It then estimates the traffic inside the rolling window by applying a weight to the previous window.
A simplified calculation looks like this:
estimated requests =
current window requests
+ previous window requests × previous-window overlap
For example, if the current rolling window overlaps 70% of the previous fixed window, 70% of the previous counter contributes to the estimate.
Why it is useful
Sliding Window Counter provides:
- smoother behaviour around window boundaries;
- significantly lower memory consumption than a timestamp log;
- better accuracy than a basic fixed window;
- only a small amount of state per client and rule.
Main trade-off
The result is an approximation.
The calculation assumes that requests in the previous window were reasonably evenly distributed. In reality, they may all have arrived during a small burst.
Even so, this algorithm often provides one of the best practical compromises between accuracy and efficiency.
The Simulator: Where the Algorithms Disagree
Reading about the algorithms is useful, but watching them process the same requests makes their trade-offs much easier to understand.
The simulator includes traffic patterns designed to expose different behaviours.
Boundary Attack
Requests are placed immediately before and after a fixed window boundary.
The Fixed Window Counter may allow twice the intended rolling-window limit, while the sliding approaches behave differently.
Flash Sale Burst
A quiet service suddenly receives a large spike in requests.
Token Bucket can use tokens accumulated during the quiet period, while stricter algorithms may reject much of the burst.
Leaking Bucket Starvation
A burst fills the queue with older requests.
Newer traffic must wait, sometimes for a considerable period, or is rejected because the queue remains full.
Steady Traffic
Requests arrive at a consistent rate below the limit.
Almost every algorithm performs well.
This scenario teaches an important lesson: the happy path is not enough to evaluate a rate limiter.
Poisson Traffic
Requests have the same average rate as a steady client, but arrive in a more realistic, irregular pattern.
The algorithms begin to disagree even though the average traffic remains below the configured quota.
This reveals the difference between controlling an average rate and enforcing a hard limit within every possible rolling window.
A Deterministic Virtual Clock
The simulator does not depend on real time or use artificial delays.
Time is represented by a deterministic virtual clock. Each algorithm receives a timestamp and decides whether the request should be accepted.
As a result:
- a two-minute traffic trace can run almost instantly;
- every scenario produces the same result on every execution;
- automated tests can reproduce specific examples precisely;
- internal algorithm state can be inspected request by request;
- race conditions can be demonstrated as controlled event interleavings.
This was an important implementation decision.
A learning simulator must be repeatable. If the output changes because of operating-system scheduling or real-time delays, it becomes far more difficult to understand why an algorithm behaved in a particular way.
The Implementation
The project includes both a command-line simulator and an interactive browser version.
The core simulator is implemented in TypeScript, with each algorithm kept in its own small and heavily documented file.
The repository also contains:
- deterministic traffic generators;
- named experiment scenarios;
- request-by-request tracing;
- ASCII timeline rendering;
- rolling-window peak calculations;
- a distributed race-condition demonstration;
- automated tests reproducing the documented examples;
- reference Redis Lua scripts;
- architecture and distributed-system diagrams;
- a browser-based visual simulator.
The command-line version can be used without installing external dependencies:
npm run sim -- --list
npm run sim -- --scenario=fixed-window-edge-burst
npm run sim -- --scenario=flash-sale-burst --trace
npm run sim -- --all
npm run sim -- --race
npm test
The visual version runs as a self-contained browser application and allows the parameters and traffic patterns to be changed interactively.
The Distributed Race Condition
A rate limiter can still be logically correct and fail under concurrency.
Consider a naive Redis implementation:
GET counter
check counter against limit
SET counter + 1
Two or more requests may read the same counter value before any of them writes the updated value.
For example:
Request A reads 3
Request B reads 3
Request A writes 4
Request B writes 4
The final counter is four, even though two requests were accepted. It should have been five.
Under heavy concurrency, this can allow a significant number of requests through the limiter without the counter ever reflecting what happened.
The project includes a deterministic race-condition simulation comparing:
- a naive read-check-write implementation;
- an atomic check-and-increment operation.
The correct solution is not to place a distributed lock around every request. That would protect correctness by damaging latency and throughput.
Instead, the decision should be executed atomically inside the data store, commonly through:
- a Redis Lua script;
- atomic Redis commands;
- Redis sorted-set operations.
The repository includes production-shaped Lua reference implementations showing how these operations can be combined into a single atomic round trip.
From a Single Server to a Distributed Rate Limiter
A local in-memory counter may work while an application has only one instance.
Once multiple rate limiter instances exist, requests from the same client may reach different servers. If every instance maintains its own counters, none of them has a complete view of the client’s activity.
Sticky sessions might appear to solve this by repeatedly routing a client to the same instance, but they reduce flexibility and complicate scaling.
A more robust design keeps the rate limiter instances stateless and stores shared counters in a centralised system such as Redis.
This introduces further design decisions:
- How should Redis keys be sharded?
- How should hot keys be handled?
- Should the limiter fail open or fail closed when Redis is unavailable?
- How much latency can the limiter add to every request?
- How should counters be synchronised across regions?
- Is approximate global enforcement acceptable?
- Should rejected requests be dropped or queued for later processing?
These are the questions that transform a coding exercise into a system design problem.
What I Learnt from Building It
The biggest lesson was that rate limiting is not simply about counting requests.
It is about selecting the behaviour that matches the system being protected.
- Token Bucket is a strong choice when legitimate bursts should be accepted.
- Leaking Bucket is useful when downstream traffic must remain smooth and predictable.
- Fixed Window Counter is simple and efficient, but vulnerable at window boundaries.
- Sliding Window Log provides excellent accuracy at a considerable memory cost.
- Sliding Window Counter offers a practical compromise between precision and efficiency.
There is no universal winner.
The right choice depends on the endpoint, traffic shape, business requirements, expected scale and consequences of allowing or rejecting too many requests.
A mature platform may use several algorithms simultaneously for different operations.
Explore the Project
This project is part of my ongoing System Design study repository, where I am turning architectural concepts into executable experiments rather than keeping them only as diagrams and notes.
For the best learning experience:
- Choose one of the simulator scenarios.
- Predict how each algorithm will behave.
- Run the simulation.
- Compare the accepted requests, rejected requests and rolling-window peak.
- Inspect the trace and algorithm state.
- Change the parameters and repeat the experiment.
The cases where your prediction is wrong are usually the most valuable ones.
🚦 Open the interactive simulator:
https://rate-limiter.mhayk.workers.dev/
💻 Read the implementation, tests, diagrams and notes:
https://github.com/mhayk/system-design/tree/main/designs/02-rate-limiter
If you are studying System Design, preparing for technical interviews or simply interested in distributed systems, feel free to explore the repository, experiment with the scenarios and contribute your own ideas.
And if you find the project useful, consider giving the repository a star on GitHub. ⭐
