There is a moment in System Design interviews that often gives people that familiar feeling of nervousness.
You are comfortably discussing requirements, APIs, databases and components when the interviewer asks:
How many requests per second does this system need to support?
At that point, several questions appear at once.
How many users should I assume? How many actions does each person perform? Do I need an exact answer? What happens if I make a simple calculation mistake?
This stage is known as Back-of-the-Envelope Estimation.
Despite involving numbers, it is not really a maths test.
It is a way of making your reasoning visible.
You do not need the exact answer
When estimating the scale of a system, we are not trying to predict exactly how many servers the application will use three years from now.
The objective is to understand the order of magnitude of the problem.
There is an enormous difference between designing a system for:
300 requests per second;
30,000 requests per second;
300,000 requests per second.
In the same way, storing 50 GB is a completely different problem from storing 50 PB.
The estimate only needs to be good enough to help answer some important questions:
Is this genuinely a large-scale system?
Will most of the traffic come from reads or writes?
Will storage become a significant challenge?
Will we need a cache?
Could bandwidth become a bottleneck?
Will the data need to be distributed across multiple regions?
Before drawing the first component, the numbers already begin to show where the architecture might become difficult.
The interviewer wants to follow your reasoning
A good estimate can begin with a simple sentence:
I am going to state a few assumptions so that we can estimate the order of magnitude.
That sentence completely changes the conversation.
You are not claiming to know the company’s real numbers. You are simply creating a reasonable scenario to guide the design.
For example:
300 million monthly active users;
50% use the application every day;
each user performs two actions per day;
peak traffic is twice the average traffic.
The interviewer may prefer different assumptions, and that is perfectly fine. They can adjust the numbers while the method remains exactly the same.
Your assumptions do not need to be perfect. They need to be clear.
A simple method for estimating scale
Whenever someone asks you to estimate the scale of a system, you can follow this sequence.
1. Calculate the daily active users
We normally begin with the number of monthly active users, known as Monthly Active Users, or MAU.
DAU = MAU × percentage of users active each day
Suppose we have 300 million MAU and assume that 50% of them use the system every day:
300 million × 50% = 150 million DAU
We now have the number of Daily Active Users, or DAU.
2. Calculate the actions performed each day
Suppose each user performs two actions per day:
150 million × 2 = 300 million actions per day
3. Convert daily actions into QPS
There are 86,400 seconds in a day.
Therefore:
Average QPS = requests per day ÷ 86,400
In our example:
300 million ÷ 86,400 ≈ 3,500 QPS
For an initial estimate, we can also round 86,400 to approximately 100,000, or 10⁵.
This makes the mental arithmetic much easier:
300 million ÷ 100,000 ≈ 3,000 QPS
The result is not exact, but it is within the correct order of magnitude.
In a System Design interview, that is usually far more important than reaching the final digit.
4. Account for peak traffic
Traffic is not distributed evenly throughout the day.
An application may receive more traffic in the evening, after a notification is sent or during an important event.
For this reason, we can apply a Peak Factor, usually between two and three times the average traffic.
Peak QPS = Average QPS × Peak Factor
Using a peak factor of two:
3,500 × 2 = 7,000 peak QPS
We now have an initial understanding of the load the system needs to support.
When the estimate reveals the real problem
Let us add storage to the same example.
Imagine that:
10% of posts contain an image or another type of media;
each media object is approximately 1 MB;
the data will be retained for five years.
The daily storage requirement would be:
150 million users
× 2 posts per day
× 10%
× 1 MB
= 30 TB per day
Over five years:
30 TB × 365 × 5 ≈ 55 PB
Notice what happened.
The system handles approximately 3,500 writes per second, which may appear relatively manageable.
However, it also needs to store approximately 55 petabytes of content.
The estimate has shown that the greatest challenge probably does not lie in the number of requests. The more difficult problem may be storage, replication, distribution and delivery of those files.
We discovered this before choosing a database, cache, queue, programming language or cloud service.
That is the real value of Back-of-the-Envelope Estimation.
The numbers that are actually worth memorising
You do not need to memorise dozens of formulas.
A small set of numbers can solve most estimation problems.
86,400 seconds in a day
This is the number used to convert daily actions into QPS.
As a shortcut:
86,400 ≈ 100,000 ≈ 10⁵
Another useful reference is:
1 million actions per day ≈ 10 QPS
100 million actions per day ≈ 1,000 QPS
1 billion actions per day ≈ 10,000 QPS
The storage ladder
Each step represents approximately one thousand times more data:
KB → MB → GB → TB → PB
Keeping this sequence in mind makes it much easier to convert millions of records into gigabytes, terabytes or petabytes.
The peak factor
Average QPS rarely represents the busiest moment of the system.
As an initial approximation:
Peak QPS ≈ Average QPS × 2 or 3
Explaining the assumption is more important than choosing exactly two or three.
The replication factor
This is one of the easiest terms to forget.
Suppose the system stores three copies of every piece of data:
Physical storage = logical storage × 3
Fifty terabytes of logical data can quickly become 150 terabytes after replication.
Availability requires redundancy. Redundancy means copies. Copies require storage.
Bandwidth
Another simple calculation that is frequently overlooked:
Bandwidth = QPS × payload size
One thousand responses per second with a size of 1 MB each represent:
1,000 × 1 MB = 1 GB per second
In this scenario, the limit may not be the CPU or the database. It may be the network and the cost of transferring the data.
Common estimation mistakes
A few simple habits can prevent most problems during the interview.
Always write down the units
The number 5 means nothing on its own.
Is it 5 KB, 5 MB, 5 GB, 5,000 requests or 5 servers?
A missing unit can turn a correct calculation into a completely incorrect conclusion.
Round without feeling guilty
Turn a calculation such as:
99,987 ÷ 9.1
into:
100,000 ÷ 10
You are producing an estimate, not preparing a company’s annual accounts.
State your assumptions aloud
A sentence like this demonstrates organisation and maturity:
I will assume that 40% of users are active each day and that peak traffic is approximately three times the average.
Even if the numbers change, the method remains valid.
Perform a sanity check
Once you have finished, ask yourself:
Does the result seem reasonable?
Am I talking about hundreds or hundreds of thousands of QPS?
Should the storage be measured in gigabytes or petabytes?
Did I forget the replication factor?
Am I mixing bits and bytes?
Is the system read-heavy or write-heavy?
Does the payload size make sense?
This small pause can reveal mistakes before they affect the rest of the design.
A tool for practising the reasoning
To make this process more visual, I created a small collection of resources for practising System Design estimations.
The interactive estimator begins with the well-known Twitter example and shows every step of the calculation.
You can change:
the number of users;
the percentage of users active each day;
the number of actions performed by each user;
the ratio between reads and writes;
the payload size;
the retention period;
the peak factor;
the replication factor;
the estimated capacity of each server.
Rather than displaying only the final result, the tool shows how each value was calculated.
This is important because the objective is not to memorise a prepared answer. The objective is to understand how assumptions become numbers and how those numbers influence the architecture.
I also created a printable A4 poster containing the main formulas, latency references, availability levels, units and mental arithmetic shortcuts.
The idea is simple: print it, place it near your desk and refer to it regularly until the numbers start to feel familiar.
All the material, including the source code, formulas and a collection of revision exercises, is available in my study repository:
Confidence does not come from memorising every answer
The most uncomfortable part of a System Design interview is often not knowing exactly what answer the interviewer expects.
But that is precisely the point.
In real projects, we rarely begin with every piece of information available. We work with incomplete requirements, assumptions, approximate metrics and constraints that change over time.
A good estimate demonstrates that you can work with that uncertainty.
You do not need to predict the future.
You do not need the exact answer.
You do not need to perform every calculation mentally and in silence.
You need to state your assumptions, keep the units visible, round sensibly and explain what the numbers mean for the architecture.
The next time someone asks:
How many requests per second does this system need to support?
Take a breath.
Write down the assumptions.
Divide by 86,400.
Find the order of magnitude.
Then use the result to discover where the system actually becomes difficult.
The nerves may still appear, but now you have a method that tells you exactly where to begin.
Existe um momento em entrevistas de System Design que costuma causar aquele frio na barriga.
Você está conversando sobre requisitos, APIs, bancos de dados e componentes quando o entrevistador pergunta:
Quantas requisições por segundo esse sistema precisa suportar?
Nesse momento, várias dúvidas aparecem ao mesmo tempo.
Quantos utilizadores devo considerar? Quantas ações cada pessoa realiza? Preciso chegar ao número exato? O que acontece se eu errar uma conta simples?
Essa etapa é conhecida como Back-of-the-Envelope Estimation, que podemos traduzir como uma estimativa rápida ou um cálculo de guardanapo.
Apesar de envolver números, ela não é realmente uma prova de matemática.
É uma forma de tornar o seu raciocínio visível.
Você não precisa acertar o número exato
Quando estimamos a escala de um sistema, não estamos tentando prever exatamente quantos servidores a aplicação utilizará daqui a três anos.
O objetivo é descobrir a ordem de grandeza do problema.
Existe uma diferença enorme entre projetar um sistema para:
300 requisições por segundo;
30 mil requisições por segundo;
300 mil requisições por segundo.
Da mesma forma, armazenar 50 GB é um problema completamente diferente de armazenar 50 PB.
A estimativa precisa ser boa o suficiente para ajudar a responder algumas perguntas:
O sistema realmente terá uma escala elevada?
O maior volume estará nas leituras ou nas escritas?
O armazenamento será um desafio relevante?
Precisaremos de cache?
A largura de banda poderá ser um gargalo?
Será necessário distribuir os dados entre várias regiões?
Antes mesmo de desenhar o primeiro componente, os números já começam a mostrar onde a arquitetura poderá ficar mais difícil.
O entrevistador quer acompanhar o seu raciocínio
Uma boa estimativa pode começar com uma frase simples:
Vou declarar algumas premissas para conseguirmos estimar a ordem de grandeza.
Essa frase muda completamente a conversa.
Você não está dizendo que conhece os números reais da empresa. Está apenas criando um cenário razoável para orientar o design.
Por exemplo:
300 milhões de utilizadores ativos por mês;
50% utilizam a aplicação diariamente;
cada utilizador realiza duas ações por dia;
o tráfego no horário de pico é duas vezes maior do que a média.
Caso o entrevistador prefira outras premissas, ele poderá ajustá-las. O método continuará exatamente o mesmo.
As suas premissas não precisam ser perfeitas. Elas precisam ser claras.
Um método simples para estimar a escala
Sempre que alguém pedir uma estimativa, você pode seguir esta sequência.
1. Calcule os utilizadores ativos por dia
Normalmente começamos com o número de utilizadores ativos por mês, chamado de Monthly Active Users, ou MAU.
DAU = MAU × percentagem de utilizadores ativos diariamente
Se tivermos 300 milhões de MAU e assumirmos que 50% utilizam o sistema todos os dias:
300 milhões × 50% = 150 milhões de DAU
Agora temos o número de Daily Active Users, ou DAU.
2. Calcule as ações realizadas por dia
Se cada utilizador realiza duas ações diariamente:
150 milhões × 2 = 300 milhões de ações por dia
3. Transforme ações diárias em QPS
Um dia possui 86.400 segundos.
Portanto:
QPS médio = requisições por dia ÷ 86.400
No nosso exemplo:
300 milhões ÷ 86.400 ≈ 3.500 QPS
Durante uma estimativa inicial, também podemos arredondar 86.400 para aproximadamente 100.000, ou 10⁵.
Isso facilita bastante o cálculo mental:
300 milhões ÷ 100 mil ≈ 3.000 QPS
O resultado não é exato, mas está na ordem de grandeza correta.
Em uma entrevista de System Design, isso costuma ser muito mais importante do que chegar ao último dígito.
4. Considere o horário de pico
O tráfego de uma aplicação não é distribuído de maneira uniforme durante o dia.
Uma aplicação pode receber mais acessos durante a noite, depois do envio de uma notificação ou durante algum evento importante.
Por isso, podemos aplicar um Peak Factor, normalmente entre duas e três vezes o tráfego médio.
Peak QPS = Average QPS × Peak Factor
Utilizando um fator de pico de duas vezes:
3.500 × 2 = 7.000 QPS no pico
Pronto. Já temos uma primeira noção da carga que o sistema deverá suportar.
Quando a estimativa revela o verdadeiro problema
Vamos adicionar armazenamento ao mesmo exemplo.
Imagine que:
10% das publicações possuem uma imagem ou outro conteúdo multimédia;
cada conteúdo possui aproximadamente 1 MB;
os dados serão armazenados durante cinco anos.
O armazenamento diário seria:
150 milhões de utilizadores
× 2 publicações por dia
× 10%
× 1 MB
= 30 TB por dia
Durante cinco anos:
30 TB × 365 × 5 ≈ 55 PB
Observe o que aconteceu.
O sistema possui aproximadamente 3.500 escritas por segundo, um número que pode parecer relativamente administrável.
Por outro lado, ele precisa armazenar aproximadamente 55 petabytes de conteúdo.
A estimativa acabou de mostrar que o maior desafio provavelmente não está no número de requisições. O problema mais complexo pode estar no armazenamento, na replicação, na distribuição e na entrega desses ficheiros.
Descobrimos isso antes de escolher banco de dados, cache, fila, linguagem de programação ou qualquer serviço de cloud.
Esse é o verdadeiro valor do Back-of-the-Envelope Estimation.
Os números que realmente vale a pena memorizar
Você não precisa decorar dezenas de fórmulas.
Um pequeno conjunto de números resolve a maior parte das estimativas.
86.400 segundos por dia
Esse é o número utilizado para transformar ações diárias em QPS.
Como atalho:
86.400 ≈ 100.000 ≈ 10⁵
Outra referência útil:
1 milhão de ações por dia ≈ 10 QPS
100 milhões de ações por dia ≈ 1.000 QPS
1 bilhão de ações por dia ≈ 10.000 QPS
A escada de armazenamento
Cada passo representa aproximadamente mil vezes mais dados:
KB → MB → GB → TB → PB
Se você mantiver essa sequência na cabeça, ficará muito mais fácil converter milhões de registos em gigabytes, terabytes ou petabytes.
O fator de pico
O QPS médio raramente representa o momento mais movimentado do sistema.
Como aproximação inicial:
Peak QPS ≈ Average QPS × 2 ou 3
Mais importante do que escolher exatamente duas ou três vezes é explicar a premissa utilizada.
O fator de replicação
Este é um dos termos mais fáceis de esquecer.
Se o sistema armazena três cópias de cada dado:
Armazenamento físico = armazenamento lógico × 3
Cinquenta terabytes de dados podem rapidamente transformar-se em 150 terabytes após a replicação.
Disponibilidade exige redundância. Redundância significa cópias. E cópias ocupam espaço.
A largura de banda
Outra conta simples, mas frequentemente esquecida:
Bandwidth = QPS × tamanho do payload
Mil respostas por segundo com 1 MB cada representam:
1.000 × 1 MB = 1 GB por segundo
Nesse cenário, o limite pode não estar na CPU ou no banco de dados. Ele pode estar na rede e no custo de transferência dos dados.
Erros comuns durante a estimativa
Alguns pequenos hábitos evitam grande parte dos problemas durante a entrevista.
Escreva sempre as unidades
O número 5 não significa nada sozinho.
São 5 KB, 5 MB, 5 GB, 5 mil requisições ou 5 servidores?
Uma unidade esquecida pode transformar uma conta correta numa conclusão completamente errada.
Arredonde sem culpa
Transforme uma conta como:
99.987 ÷ 9,1
em:
100.000 ÷ 10
Você está fazendo uma estimativa, não fechando a contabilidade anual de uma empresa.
Diga as premissas em voz alta
Uma frase como esta demonstra organização e maturidade:
Vou assumir que 40% dos utilizadores estão ativos diariamente e que o pico é aproximadamente três vezes maior do que a média.
Mesmo que os números mudem, o método continua válido.
Faça uma verificação de sanidade
Ao terminar, pergunte:
O resultado parece razoável?
Estou falando de centenas ou centenas de milhares de QPS?
O armazenamento deveria estar em gigabytes ou petabytes?
Esqueci o fator de replicação?
Estou misturando bits e bytes?
O sistema possui mais leituras ou escritas?
O tamanho do payload faz sentido?
Essa pequena pausa pode encontrar erros antes que eles afetem o restante do design.
Uma ferramenta para praticar o raciocínio
Para tornar esse processo mais visual, criei um pequeno conjunto de recursos para praticar estimativas de System Design.
O estimador interativo começa com o conhecido exemplo do Twitter e mostra cada etapa do cálculo.
Você pode alterar:
número de utilizadores;
percentagem de utilizadores ativos diariamente;
ações realizadas por utilizador;
proporção entre leituras e escritas;
tamanho do payload;
período de retenção;
fator de pico;
fator de replicação;
capacidade estimada de cada servidor.
Em vez de apresentar apenas o resultado, a ferramenta mostra como cada valor foi calculado.
Essa é uma parte importante do processo, porque o objetivo não é memorizar uma resposta pronta. O objetivo é entender como as premissas se transformam em números e como esses números influenciam a arquitetura.
Também preparei um poster imprimível em formato A4 com as principais fórmulas, referências de latência, níveis de disponibilidade, unidades e atalhos de cálculo mental.
A ideia é imprimir, colocar próximo à secretária e consultar com frequência até que os números comecem a parecer familiares.
Todo o material, incluindo o código-fonte, as fórmulas e um conjunto de exercícios de revisão, está disponível no meu repositório de estudos:
A parte mais desconfortável de uma entrevista de System Design costuma ser não saber exatamente qual resposta o entrevistador espera.
Mas esse é justamente o ponto.
Na vida real, raramente começamos um projeto com todas as informações disponíveis. Trabalhamos com requisitos incompletos, premissas, métricas aproximadas e limites que mudam ao longo do tempo.
Uma boa estimativa demonstra que você consegue trabalhar com essa incerteza.
Você não precisa prever o futuro.
Não precisa acertar o número exato.
Não precisa fazer toda a matemática mentalmente e em silêncio.
Você precisa declarar as suas premissas, manter as unidades visíveis, arredondar com bom senso e explicar o que os números significam para a arquitetura.
Na próxima vez que alguém perguntar:
Quantas requisições por segundo esse sistema precisa suportar?
Respire.
Escreva as premissas.
Divida por 86.400.
Encontre a ordem de grandeza.
Depois, use o resultado para descobrir onde o sistema realmente fica difícil.
O frio na barriga pode até continuar aparecendo, mas agora você terá um método para saber exatamente por onde começar.
Count the requests, define a threshold and reject anything above it.
But what happens when traffic arrives at the exact boundary between two time windows? How should a system handle legitimate bursts? Is perfect accuracy worth the additional memory cost? And how do you prevent race conditions when several servers are updating the same counters concurrently?
These questions quickly turn a seemingly simple component into a genuinely interesting distributed systems problem.
While studying rate limiter design, I decided not to stop at diagrams and theoretical explanations. I implemented the five most commonly discussed rate limiting algorithms and built an interactive simulator that runs all of them against the same traffic stream.
The goal is simple: make their trade-offs visible.
A rate limiter controls how frequently a client, user, device or service can perform an operation within a given period.
For example:
A user may create no more than two posts per second.
An IP address may attempt to log in only five times per minute.
A client may call an expensive third-party API no more than 1,000 times per day.
An entire platform may accept a maximum of 100,000 requests per second.
Rate limiting helps systems:
protect services from abusive or accidental traffic;
prevent downstream services from becoming overloaded;
control infrastructure and third-party API costs;
enforce fair usage across customers;
maintain predictable system behaviour during traffic spikes.
However, there is no single perfect rate limiting algorithm.
Each approach makes a different compromise between accuracy, memory consumption, latency, burst tolerance and implementation complexity.
That is exactly what this project demonstrates.
Five Algorithms, One Traffic Stream
The simulator implements five algorithms:
Token Bucket
Leaking Bucket
Fixed Window Counter
Sliding Window Log
Sliding Window Counter
Instead of testing each algorithm with a different example, the simulator feeds the same traffic into all five.
This makes it possible to see where their behaviour begins to diverge.
A steady stream of requests can make every algorithm appear correct. The differences become clear only when the traffic becomes irregular, bursty or deliberately hostile.
1. Token Bucket
The Token Bucket algorithm maintains a bucket containing a limited number of tokens.
Tokens are added at a configured refill rate, up to the maximum capacity of the bucket. Every request consumes one token.
If a token is available, the request is accepted. If the bucket is empty, the request is rejected.
Why it is useful
Token Bucket allows short bursts while still enforcing a sustainable average request rate.
Imagine that a client has been inactive for several seconds. During that quiet period, its bucket may become full. When the client suddenly sends several requests together, it can use the stored tokens.
This is often desirable for public APIs, where a single page load may legitimately trigger multiple requests at once.
Main trade-off
The algorithm requires two parameters:
bucket capacity;
token refill rate.
Those parameters must be tuned carefully. A bucket that is too large may allow excessive bursts, while a bucket that is too small may reject perfectly valid usage.
For many general-purpose APIs, Token Bucket is a strong default choice.
2. Leaking Bucket
The Leaking Bucket algorithm behaves more like a queue.
Incoming requests are placed into a bucket with a limited capacity. Requests then leave the bucket at a constant, predictable rate.
If the queue is already full, new requests are rejected.
Why it is useful
Leaking Bucket smooths irregular traffic into a stable output rate.
This can be valuable when the downstream system has a hard processing limit, such as:
a legacy platform;
a payment provider with a strict transactions-per-second limit;
a mainframe;
a hardware device;
a slow external integration.
Regardless of how bursty the incoming traffic becomes, the downstream service receives work at a controlled pace.
Main trade-off
Old requests can occupy the queue for a significant amount of time.
A large burst may fill the bucket, forcing newer and potentially more relevant requests to wait or be rejected. For interactive APIs, this delay can produce a poor user experience.
Unlike most rate limiters, Leaking Bucket often defers work rather than immediately refusing it.
3. Fixed Window Counter
Fixed Window Counter divides time into predefined windows.
For example, a rule allowing five requests per minute might create windows such as:
14:00:00 to 14:00:59;
14:01:00 to 14:01:59;
14:02:00 to 14:02:59.
Each window has its own counter. Once the counter reaches the configured limit, additional requests are rejected until the next window begins.
Why it is useful
Fixed Window Counter is:
easy to understand;
inexpensive to store;
straightforward to implement with Redis using operations such as INCR and EXPIRE.
It also works well when the boundary itself has business meaning, such as a daily quota that resets at midnight.
The boundary problem
This is where the simulator becomes particularly useful.
Suppose the limit is five requests per minute.
A client sends five requests at the end of one window and another five immediately after the next window begins.
Both counters remain within their individual limits. However, the system has allowed ten requests within a rolling sixty-second period.
That is twice the intended limit.
The simulator includes a Boundary Attack scenario that reproduces this behaviour visually. It is one of the clearest demonstrations of why an algorithm that appears correct under normal traffic may fail under adversarial traffic.
4. Sliding Window Log
Sliding Window Log solves the fixed window boundary problem by storing the timestamp of every request.
Whenever a new request arrives, the limiter:
removes timestamps outside the current rolling window;
adds the new request timestamp;
counts the remaining timestamps;
accepts or rejects the request based on that count.
Redis sorted sets are particularly well suited to this approach because timestamps can be used as scores.
Why it is useful
Sliding Window Log is highly accurate.
For any rolling window, the number of accepted requests can be kept within the configured threshold.
This makes it suitable for sensitive, lower-volume operations such as:
login attempts;
password resets;
payment initiation;
account recovery;
costly third-party API operations.
Main trade-off
Accuracy has a memory cost.
The limiter must store individual request timestamps. Memory consumption therefore grows with request volume rather than merely with the number of users.
For a high-traffic public endpoint, storing millions of timestamps can become extremely expensive.
Sliding Window Log is often the most accurate algorithm in the comparison, but not necessarily the most practical one at scale.
5. Sliding Window Counter
Sliding Window Counter combines ideas from Fixed Window Counter and Sliding Window Log.
Instead of storing every timestamp, it keeps counters for the current and previous windows.
It then estimates the traffic inside the rolling window by applying a weight to the previous window.
For example, if the current rolling window overlaps 70% of the previous fixed window, 70% of the previous counter contributes to the estimate.
Why it is useful
Sliding Window Counter provides:
smoother behaviour around window boundaries;
significantly lower memory consumption than a timestamp log;
better accuracy than a basic fixed window;
only a small amount of state per client and rule.
Main trade-off
The result is an approximation.
The calculation assumes that requests in the previous window were reasonably evenly distributed. In reality, they may all have arrived during a small burst.
Even so, this algorithm often provides one of the best practical compromises between accuracy and efficiency.
The Simulator: Where the Algorithms Disagree
Reading about the algorithms is useful, but watching them process the same requests makes their trade-offs much easier to understand.
The simulator includes traffic patterns designed to expose different behaviours.
Boundary Attack
Requests are placed immediately before and after a fixed window boundary.
The Fixed Window Counter may allow twice the intended rolling-window limit, while the sliding approaches behave differently.
Flash Sale Burst
A quiet service suddenly receives a large spike in requests.
Token Bucket can use tokens accumulated during the quiet period, while stricter algorithms may reject much of the burst.
Leaking Bucket Starvation
A burst fills the queue with older requests.
Newer traffic must wait, sometimes for a considerable period, or is rejected because the queue remains full.
Steady Traffic
Requests arrive at a consistent rate below the limit.
Almost every algorithm performs well.
This scenario teaches an important lesson: the happy path is not enough to evaluate a rate limiter.
Poisson Traffic
Requests have the same average rate as a steady client, but arrive in a more realistic, irregular pattern.
The algorithms begin to disagree even though the average traffic remains below the configured quota.
This reveals the difference between controlling an average rate and enforcing a hard limit within every possible rolling window.
A Deterministic Virtual Clock
The simulator does not depend on real time or use artificial delays.
Time is represented by a deterministic virtual clock. Each algorithm receives a timestamp and decides whether the request should be accepted.
As a result:
a two-minute traffic trace can run almost instantly;
every scenario produces the same result on every execution;
automated tests can reproduce specific examples precisely;
internal algorithm state can be inspected request by request;
race conditions can be demonstrated as controlled event interleavings.
This was an important implementation decision.
A learning simulator must be repeatable. If the output changes because of operating-system scheduling or real-time delays, it becomes far more difficult to understand why an algorithm behaved in a particular way.
The Implementation
The project includes both a command-line simulator and an interactive browser version.
The core simulator is implemented in TypeScript, with each algorithm kept in its own small and heavily documented file.
The repository also contains:
deterministic traffic generators;
named experiment scenarios;
request-by-request tracing;
ASCII timeline rendering;
rolling-window peak calculations;
a distributed race-condition demonstration;
automated tests reproducing the documented examples;
reference Redis Lua scripts;
architecture and distributed-system diagrams;
a browser-based visual simulator.
The command-line version can be used without installing external dependencies:
npm run sim -- --list
npm run sim -- --scenario=fixed-window-edge-burst
npm run sim -- --scenario=flash-sale-burst --trace
npm run sim -- --all
npm run sim -- --race
npm test
The visual version runs as a self-contained browser application and allows the parameters and traffic patterns to be changed interactively.
The Distributed Race Condition
A rate limiter can still be logically correct and fail under concurrency.
Consider a naive Redis implementation:
GET counter
check counter against limit
SET counter + 1
Two or more requests may read the same counter value before any of them writes the updated value.
For example:
Request A reads 3
Request B reads 3
Request A writes 4
Request B writes 4
The final counter is four, even though two requests were accepted. It should have been five.
Under heavy concurrency, this can allow a significant number of requests through the limiter without the counter ever reflecting what happened.
The project includes a deterministic race-condition simulation comparing:
a naive read-check-write implementation;
an atomic check-and-increment operation.
The correct solution is not to place a distributed lock around every request. That would protect correctness by damaging latency and throughput.
Instead, the decision should be executed atomically inside the data store, commonly through:
a Redis Lua script;
atomic Redis commands;
Redis sorted-set operations.
The repository includes production-shaped Lua reference implementations showing how these operations can be combined into a single atomic round trip.
From a Single Server to a Distributed Rate Limiter
A local in-memory counter may work while an application has only one instance.
Once multiple rate limiter instances exist, requests from the same client may reach different servers. If every instance maintains its own counters, none of them has a complete view of the client’s activity.
Sticky sessions might appear to solve this by repeatedly routing a client to the same instance, but they reduce flexibility and complicate scaling.
A more robust design keeps the rate limiter instances stateless and stores shared counters in a centralised system such as Redis.
This introduces further design decisions:
How should Redis keys be sharded?
How should hot keys be handled?
Should the limiter fail open or fail closed when Redis is unavailable?
How much latency can the limiter add to every request?
How should counters be synchronised across regions?
Is approximate global enforcement acceptable?
Should rejected requests be dropped or queued for later processing?
These are the questions that transform a coding exercise into a system design problem.
What I Learnt from Building It
The biggest lesson was that rate limiting is not simply about counting requests.
It is about selecting the behaviour that matches the system being protected.
Token Bucket is a strong choice when legitimate bursts should be accepted.
Leaking Bucket is useful when downstream traffic must remain smooth and predictable.
Fixed Window Counter is simple and efficient, but vulnerable at window boundaries.
Sliding Window Log provides excellent accuracy at a considerable memory cost.
Sliding Window Counter offers a practical compromise between precision and efficiency.
There is no universal winner.
The right choice depends on the endpoint, traffic shape, business requirements, expected scale and consequences of allowing or rejecting too many requests.
A mature platform may use several algorithms simultaneously for different operations.
Explore the Project
This project is part of my ongoing System Design study repository, where I am turning architectural concepts into executable experiments rather than keeping them only as diagrams and notes.
For the best learning experience:
Choose one of the simulator scenarios.
Predict how each algorithm will behave.
Run the simulation.
Compare the accepted requests, rejected requests and rolling-window peak.
Inspect the trace and algorithm state.
Change the parameters and repeat the experiment.
The cases where your prediction is wrong are usually the most valuable ones.
If you are studying System Design, preparing for technical interviews or simply interested in distributed systems, feel free to explore the repository, experiment with the scenarios and contribute your own ideas.
And if you find the project useful, consider giving the repository a star on GitHub. ⭐
Building a platform capable of processing 40 billion requests per month might initially sound like a challenge reserved for companies such as Google, Amazon or Netflix.
However, once we convert that number into requests per second and divide the responsibilities correctly between caching, APIs, messaging systems, databases and asynchronous processing, the problem becomes much easier to understand.
In this article, I will explain how I would design a distributed architecture using Go, Apache Kafka, Redis, Kubernetes, distributed databases and modern observability tools to support this volume reliably and cost-effectively.
This is, of course, a reference architecture. The final implementation would depend on several factors, including:
The type of requests being processed
The average payload and response sizes
The balance between reads and writes
Consistency requirements
Geographical distribution of users
Data residency requirements
Expected availability and latency targets
Converting 40 Billion Requests into Requests per Second
Before selecting technologies, we need to understand the actual traffic volume.
Assuming a 30-day month:
40,000,000,000 requests per month
÷ 30 days
÷ 24 hours
÷ 60 minutes
÷ 60 seconds
≈ 15,432 requests per second
This gives us the following approximate traffic profile:
Period
Approximate volume
Per month
40 billion
Per day
1.33 billion
Per hour
55.5 million
Per minute
925,000
Per second
15,432
An average of approximately 15,000 requests per second is not, by itself, an extreme workload for a modern distributed platform.
The real challenge is handling traffic peaks.
Traffic is rarely distributed evenly throughout the day. Marketing campaigns, notifications, scheduled integrations or external events can multiply the normal traffic volume within seconds.
I would therefore design the platform to handle between five and ten times the average traffic:
Average traffic: approximately 15,000 RPS
Expected peak: approximately 75,000 RPS
Extreme peak: approximately 150,000 RPS
The objective would not be to keep enough infrastructure running permanently for 150,000 requests per second. Instead, the platform should be able to scale towards that volume quickly and safely.
High-Level Architecture
The architecture would be divided into several layers:
Clients and integrations
│
▼
Anycast DNS
│
▼
CDN, WAF and DDoS protection
│
▼
Global load balancer
│
├── Europe region
├── North America region
└── Asia-Pacific region
│
▼
Regional load balancer or Kubernetes ingress
│
▼
Go API services
│
├── Redis Cluster
├── Transactional databases
├── Distributed databases
└── Apache Kafka
│
▼
Asynchronous Go consumers
│
├── Notifications
├── Search indexing
├── Analytics
├── External integrations
└── Object storage
The main request flow would be:
Client
→ CDN and WAF
→ Global load balancer
→ Nearest healthy region
→ API gateway or ingress
→ Go services
→ Redis, database or Kafka
→ Asynchronous processing
1. CDN, WAF and DDoS Protection
The first layer should prevent unnecessary or malicious requests from reaching the internal services.
I would use a platform such as Cloudflare, AWS CloudFront or Fastly to provide:
Content delivery network functionality
Edge caching
DDoS protection
Web Application Firewall protection
Bot detection and mitigation
Rate limiting
TLS termination
Geographical routing
Protection against common web attacks
Whenever possible, public or semi-public responses should be served directly from the edge.
For example, if 30% of all requests could be answered by the CDN, the internal services would avoid processing approximately 12 billion requests per month.
This reduction would directly affect:
Infrastructure costs
CPU consumption
Database utilisation
Application latency
Overall platform stability
The best request for the application infrastructure to process is the request that never reaches it.
2. Multi-Region Architecture
For a global and business-critical platform, I would deploy the system across at least three regions.
For example:
Europe
North America
Asia-Pacific
A global load balancer would direct each user to the nearest healthy region.
The strategy could include:
Active-active regions for APIs
Automatic regional failover
Asynchronous replication for eventually consistent data
A primary region for operations requiring strong consistency
Regional storage for regulated or residency-sensitive data
A multi-region architecture is not only about improving latency. It also protects the system against:
The failure of an entire cloud region
Large-scale networking problems
Cloud provider incidents
Faulty deployments
Operational disasters
3. Cell-Based Architecture
Rather than running every customer inside one enormous shared cluster, I would divide the platform into independent cells.
A request should never wait indefinitely for another service to respond.
Internal Communication
For communication between services, I would consider:
HTTP and JSON for public APIs
gRPC or Connect for internal synchronous communication
Kafka for asynchronous events
Protocol Buffers for high-volume internal contracts
Not every interaction needs to go through Kafka.
Operations requiring an immediate response can use HTTP or gRPC. Kafka should be used when processing can happen asynchronously or when multiple services need to react to the same event.
5. Kubernetes and Autoscaling
The services could run on Kubernetes using a managed platform such as Amazon EKS, Google Kubernetes Engine or Azure Kubernetes Service.
Each service should have:
Multiple replicas
Pod Disruption Budgets
Readiness probes
Liveness probes
Resource requests
Resource limits
Topology spread constraints
Anti-affinity across availability zones
Automatic horizontal scaling
I would not scale services using CPU consumption alone.
Autoscaling decisions should consider:
CPU utilisation
Memory consumption
Requests per second
Active request concurrency
p95 and p99 latency
Internal queue sizes
Kafka consumer lag
Open connections
For example, an API service could scale when:
CPU utilisation exceeds 65%
or
p95 latency exceeds 200 ms
or
active requests per pod exceed 500
For Kafka consumers, consumer lag would be one of the most important scaling signals.
6. Redis for Caching and Temporary Data
Redis would reduce pressure on the databases and provide fast access to short-lived data.
Potential use cases include:
Response caching
User sessions
Rate limiting
Distributed locks
Idempotency keys
Feature flags
Counters
Temporary state
Frequently accessed query results
The target should be a high cache-hit ratio.
For example:
Cache-hit ratio: 90%
Requests reaching the database: 10%
If the platform receives 15,000 requests per second and Redis serves 90% of the required data, the database could receive approximately 1,500 queries per second instead of 15,000.
Redis should not automatically be treated as the permanent source of truth.
The services should also support a controlled degradation mode if the cache becomes unavailable.
7. Database Strategy
There is no single database that is ideal for every type of workload.
I would use different database technologies for different responsibilities.
PostgreSQL
PostgreSQL would be suitable for:
Relational data
Payments
Configuration
Accounts
Permissions
Transactional operations
Data requiring strong consistency
Depending on the volume, the PostgreSQL layer could use:
Table partitioning
Read replicas
PgBouncer
Tenant-based sharding
Independent databases for each cell
A managed solution such as Amazon Aurora PostgreSQL
Distributed Key-Value or Wide-Column Database
For very high-volume data accessed through predictable keys, I would consider:
Amazon DynamoDB
ScyllaDB
Apache Cassandra
Google Cloud Bigtable
These technologies can be suitable for:
Device state
Activity timelines
Counters
Large event histories
Write-heavy workloads
Predictable key-based queries
ClickHouse
For analytics and queries across large event datasets, I would use ClickHouse.
Possible workloads include:
Business reports
Usage metrics
Behavioural analysis
Large aggregations
Operational dashboards
Audit analysis
Running large analytical queries directly against the transactional database would be a mistake.
The database responsible for the product’s day-to-day operations should not also act as its data warehouse.
Object Storage
Historical data, exports and raw events could be stored in:
Amazon S3
Google Cloud Storage
Azure Blob Storage
MinIO
Object storage is considerably cheaper than transactional storage and can later support data reprocessing, analytics or machine-learning workloads.
8. Kafka as the Event Backbone
Kafka would become the backbone of the asynchronous processing layer.
When an important operation occurs, the responsible service would publish an event.
A problematic message should not be retried indefinitely inside the main topic.
Otherwise, one invalid or unprocessable message could block or degrade an entire partition.
13. Backpressure and Load Shedding
When demand temporarily exceeds capacity, the platform needs to protect itself.
This can be achieved through:
Bounded queues
Concurrency limits
Rate limiting
Circuit breakers
Early rejection
Degraded responses
Prioritisation of critical operations
It is often better to reject a small percentage of requests quickly than to allow every service to fail slowly.
Operations could be classified by priority:
Priority 1: authentication, payments and core operations
Priority 2: standard product data
Priority 3: reports and exports
Priority 4: non-essential background operations
During periods of overload, lower-priority operations could be temporarily restricted.
14. Resilience Between Services
Every external dependency should have:
A timeout
A circuit breaker
Limited retries
Bulkhead isolation
A fallback strategy
Dedicated metrics
Retries should only be used for safe or idempotent operations.
An inappropriate retry strategy can multiply an incident.
For example:
Service A retries three times
Service B retries three times
Service C retries three times
A single original request could produce up to 27 internal attempts.
This is known as retry amplification and can turn a minor slowdown into a complete outage.
15. Observability
A platform operating at this scale must allow the engineering team to answer three questions quickly:
What is happening?
Where is the problem?
Which customer or operation is affected?
I would use OpenTelemetry to standardise:
Metrics
Distributed traces
Logs
Context propagation between services
The observability stack could include:
Prometheus
Grafana
Loki
Tempo
OpenSearch
Datadog
New Relic
Honeycomb
Important API Metrics
Requests per second
p50, p95 and p99 latency
Error rate
Timeout rate
Requests by endpoint
Active connections
Number of goroutines
Memory consumption
Garbage collector pauses
Important Kafka Metrics
Messages produced
Messages consumed
Consumer lag
Throughput by partition
Under-replicated partitions
Average message size
Processing duration
Dead-letter queue volume
Important Database Metrics
Active connections
Queries per second
Slow queries
Lock wait time
Replication lag
Cache-hit ratio
CPU and storage utilisation
Throttling events
Every log entry should contain identifiers such as:
trace_id
request_id
tenant_id
user_id
region
service
version
This would allow an engineer to follow a request from the edge all the way to the final Kafka consumer.
16. Service-Level Objectives and Error Budgets
I would define clear reliability targets.
For example:
Availability: 99.99%
p95 latency: below 200 ms
p99 latency: below 500 ms
Error rate: below 0.1%
An availability target of 99.99% allows approximately 52 minutes of downtime per year.
I would also use error budgets.
If the engineering team consumed the error budget too quickly, feature deployments could be reduced or paused until the platform returned to an acceptable level of reliability.
This makes reliability measurable rather than subjective.
17. Security
Security must exist at every layer of the architecture.
The platform should include:
Web Application Firewall protection
DDoS protection
Rate limiting by IP address, user and tenant
OAuth 2.0 or OpenID Connect
Short-lived access tokens
Key rotation
TLS encryption in transit
Encryption at rest
Centralised secrets management
Least-privilege access policies
Network segmentation
Audit logging
Container vulnerability scanning
A Software Bill of Materials
Signed build artefacts
CI/CD supply-chain protection
Internal services could use mutual TLS or workload identities.
Secrets should never be embedded inside Docker images or stored directly in the source-code repository.
18. Deployment Strategy
A high-volume platform should not deploy a new version directly to 100% of users.
I would use:
Canary deployments
Blue-green deployments
Feature flags
Automatic rollbacks
Progressive delivery
For example:
1% of traffic
5% of traffic
20% of traffic
50% of traffic
100% of traffic
During each stage, the platform should evaluate:
Error rate
Latency
CPU utilisation
Memory consumption
Kafka consumer lag
Business metrics
If a regression is detected, the deployment should be interrupted or rolled back automatically.
19. Load Testing
No architecture should be considered capable of processing 40 billion requests per month simply because the diagram looks correct.
It must be tested.
I would perform:
Load tests
Stress tests
Spike tests
Soak tests
Chaos engineering experiments
Regional failover tests
Kafka broker failure tests
Redis failure tests
Database degradation tests
Deployment rollback tests
Tools such as k6, Vegeta or Gatling could simulate realistic traffic patterns.
The tests should include:
Realistic payloads
Authentication
Different endpoint distributions
Cold caches
Warm caches
Large tenants
Hot keys
Regional traffic
Slow client connections
Partial dependency failures
The objective is not only to discover the maximum number of requests per second.
It is also necessary to understand how the system behaves when it reaches its limits.
A well-designed platform should fail in a predictable and controlled manner.
20. Network Traffic Estimate
The amount of network traffic depends heavily on the average response size.
If each response averages 5 KB:
40 billion × 5 KB
≈ 200 TB of response data per month
If each response averages 20 KB:
40 billion × 20 KB
≈ 800 TB of response data per month
These estimates do not include:
HTTP headers
Retries
Database replication
Internal service communication
Kafka messages
Logs
Distributed traces
Cross-region traffic
This is why compression, CDN caching, regional routing and careful payload design would have a significant effect on the final infrastructure cost.
The Complete Architecture
My reference architecture would include the following components:
Edge
├── Anycast DNS
├── CDN
├── WAF
├── DDoS protection
└── Rate limiting
Routing
├── Global load balancer
├── Regional load balancers
└── API gateway or Kubernetes ingress
Application
├── Services written in Go
├── HTTP and JSON for public APIs
├── gRPC or Connect for internal communication
├── Bounded concurrency
├── Timeouts
├── Circuit breakers
└── Graceful degradation
Asynchronous processing
├── Apache Kafka
├── Schema Registry
├── Transactional Outbox
├── Idempotent consumers
├── Retry topics
└── Dead-letter queues
Data
├── Redis Cluster
├── PostgreSQL
├── Distributed key-value database
├── ClickHouse
├── Search engine
└── Object storage
Infrastructure
├── Kubernetes
├── Multi-region architecture
├── Cell-based architecture
├── Autoscaling
├── Infrastructure as Code
└── Progressive delivery
Observability
├── OpenTelemetry
├── Prometheus
├── Grafana
├── Distributed tracing
├── Centralised logs
└── SLO-based alerting
Is Kafka Actually Necessary?
One important point is that processing 40 billion requests per month does not automatically mean that the platform needs Kafka.
Kafka would make sense if the product required:
Asynchronous processing
Multiple consumers for the same event
Event reprocessing
Integration between many services
Large event volumes
Analytical pipelines
Decoupling between business domains
For a predominantly synchronous and relatively simple API, introducing Kafka could add unnecessary operational complexity.
The architecture should be driven by the business requirements, not simply by the size of the monthly request estimate.
Conclusion
Forty billion requests per month equates to approximately 15,000 requests per second on average.
This volume can be handled by a modern distributed architecture, but not simply by adding more servers.
Scalability would come from the combination of:
Aggressive caching
Asynchronous processing
Data partitioning
Cell-based isolation
Specialised databases
Stateless services
Idempotent operations
Backpressure
Observability
Load testing
Operational automation
Go would provide a strong foundation for efficient and concurrent services. Kafka would decouple processes and help absorb temporary traffic peaks. Redis would reduce pressure on the databases. A multi-region, cell-based architecture would limit failures and support progressive growth.
However, the most important lesson is that 40 billion requests should not be treated simply as one large monthly number.
The system must be designed around:
Traffic peaks
Latency requirements
Consistency requirements
Message and payload sizes
Geographical distribution
External dependencies
Infrastructure costs
Failure behaviour
The most scalable architecture is not necessarily the one that uses the largest number of technologies.
It is the architecture in which every component has a clear responsibility, operational limits are understood and failures have been anticipated before they occur.