Category: System Design

Building a platform capable of processing 40 billion requests per month might initially sound like a challenge reserved for companies such as Google, Amazon or Netflix.

However, once we convert that number into requests per second and divide the responsibilities correctly between caching, APIs, messaging systems, databases and asynchronous processing, the problem becomes much easier to understand.

In this article, I will explain how I would design a distributed architecture using Go, Apache Kafka, Redis, Kubernetes, distributed databases and modern observability tools to support this volume reliably and cost-effectively.

This is, of course, a reference architecture. The final implementation would depend on several factors, including:

• The type of requests being processed
• The average payload and response sizes
• The balance between reads and writes
• Consistency requirements
• Geographical distribution of users
• Data residency requirements
• Expected availability and latency targets

Converting 40 Billion Requests into Requests per Second

Before selecting technologies, we need to understand the actual traffic volume.

Assuming a 30-day month:

40,000,000,000 requests per month
÷ 30 days
÷ 24 hours
÷ 60 minutes
÷ 60 seconds

≈ 15,432 requests per second

This gives us the following approximate traffic profile:

Period	Approximate volume
Per month	40 billion
Per day	1.33 billion
Per hour	55.5 million
Per minute	925,000
Per second	15,432

An average of approximately 15,000 requests per second is not, by itself, an extreme workload for a modern distributed platform.

The real challenge is handling traffic peaks.

Traffic is rarely distributed evenly throughout the day. Marketing campaigns, notifications, scheduled integrations or external events can multiply the normal traffic volume within seconds.

I would therefore design the platform to handle between five and ten times the average traffic:

Average traffic: approximately 15,000 RPS
Expected peak: approximately 75,000 RPS
Extreme peak: approximately 150,000 RPS

The objective would not be to keep enough infrastructure running permanently for 150,000 requests per second. Instead, the platform should be able to scale towards that volume quickly and safely.

High-Level Architecture

The architecture would be divided into several layers:

Clients and integrations
        │
        ▼
Anycast DNS
        │
        ▼
CDN, WAF and DDoS protection
        │
        ▼
Global load balancer
        │
        ├── Europe region
        ├── North America region
        └── Asia-Pacific region
                │
                ▼
Regional load balancer or Kubernetes ingress
                │
                ▼
Go API services
        │
        ├── Redis Cluster
        ├── Transactional databases
        ├── Distributed databases
        └── Apache Kafka
                │
                ▼
Asynchronous Go consumers
        │
        ├── Notifications
        ├── Search indexing
        ├── Analytics
        ├── External integrations
        └── Object storage

The main request flow would be:

Client
→ CDN and WAF
→ Global load balancer
→ Nearest healthy region
→ API gateway or ingress
→ Go services
→ Redis, database or Kafka
→ Asynchronous processing

1. CDN, WAF and DDoS Protection

The first layer should prevent unnecessary or malicious requests from reaching the internal services.

I would use a platform such as Cloudflare, AWS CloudFront or Fastly to provide:

• Content delivery network functionality
• Edge caching
• DDoS protection
• Web Application Firewall protection
• Bot detection and mitigation
• Rate limiting
• TLS termination
• Geographical routing
• Protection against common web attacks

Whenever possible, public or semi-public responses should be served directly from the edge.

For example, if 30% of all requests could be answered by the CDN, the internal services would avoid processing approximately 12 billion requests per month.

This reduction would directly affect:

• Infrastructure costs
• CPU consumption
• Database utilisation
• Application latency
• Overall platform stability

The best request for the application infrastructure to process is the request that never reaches it.

2. Multi-Region Architecture

For a global and business-critical platform, I would deploy the system across at least three regions.

For example:

• Europe
• North America
• Asia-Pacific

A global load balancer would direct each user to the nearest healthy region.

The strategy could include:

• Active-active regions for APIs
• Automatic regional failover
• Asynchronous replication for eventually consistent data
• A primary region for operations requiring strong consistency
• Regional storage for regulated or residency-sensitive data

A multi-region architecture is not only about improving latency. It also protects the system against:

• The failure of an entire cloud region
• Large-scale networking problems
• Cloud provider incidents
• Faulty deployments
• Operational disasters

3. Cell-Based Architecture

Rather than running every customer inside one enormous shared cluster, I would divide the platform into independent cells.

Each cell could contain:

• Go services
• A dedicated Redis cluster or namespace
• A database or group of database partitions
• Dedicated Kafka topics or partitions
• Independent resource limits
• Independent monitoring
• A defined capacity target

For example:

Cell 01: customers 1–10,000
Cell 02: customers 10,001–20,000
Cell 03: customers 20,001–30,000

A routing service could use a tenant_id, customer_id or user_id to determine which cell should process a request.

The main benefit is reducing the blast radius of failures.

If one cell experiences a problem, only a percentage of customers should be affected. The remaining cells can continue operating normally.

It also becomes possible to add new cells as the platform grows, without scaling every part of the infrastructure at the same time.

4. API Services Written in Go

Go would be an excellent choice for the API layer because it provides:

• Low memory consumption
• Fast start-up times
• Efficient concurrency through goroutines
• Strong networking performance
• Small, self-contained binaries
• Simple container deployments
• Excellent profiling tools
• A mature cloud-native ecosystem

However, I would avoid creating unlimited goroutines for every operation.

Goroutines are lightweight, but they still consume memory, database connections, network sockets and downstream capacity.

The services should therefore implement:

• Concurrency limits
• Connection pooling
• Request timeouts
• Cancellation with context.Context
• Backpressure
• Circuit breakers
• Limited retries
• Graceful shutdown
• Health-check endpoints
• Metrics for every important route

A basic Go HTTP server could begin with configuration similar to this:

server := &http.Server{
    Addr:              ":8080",
    Handler:           router,
    ReadHeaderTimeout: 2 * time.Second,
    ReadTimeout:       5 * time.Second,
    WriteTimeout:      10 * time.Second,
    IdleTimeout:       60 * time.Second,
}

Every external dependency should also have an explicit timeout:

ctx, cancel := context.WithTimeout(r.Context(), 2*time.Second)
defer cancel()

A request should never wait indefinitely for another service to respond.

Internal Communication

For communication between services, I would consider:

• HTTP and JSON for public APIs
• gRPC or Connect for internal synchronous communication
• Kafka for asynchronous events
• Protocol Buffers for high-volume internal contracts

Not every interaction needs to go through Kafka.

Operations requiring an immediate response can use HTTP or gRPC. Kafka should be used when processing can happen asynchronously or when multiple services need to react to the same event.

5. Kubernetes and Autoscaling

The services could run on Kubernetes using a managed platform such as Amazon EKS, Google Kubernetes Engine or Azure Kubernetes Service.

Each service should have:

• Multiple replicas
• Pod Disruption Budgets
• Readiness probes
• Liveness probes
• Resource requests
• Resource limits
• Topology spread constraints
• Anti-affinity across availability zones
• Automatic horizontal scaling

I would not scale services using CPU consumption alone.

Autoscaling decisions should consider:

• CPU utilisation
• Memory consumption
• Requests per second
• Active request concurrency
• p95 and p99 latency
• Internal queue sizes
• Kafka consumer lag
• Open connections

For example, an API service could scale when:

CPU utilisation exceeds 65%
or
p95 latency exceeds 200 ms
or
active requests per pod exceed 500

For Kafka consumers, consumer lag would be one of the most important scaling signals.

6. Redis for Caching and Temporary Data

Redis would reduce pressure on the databases and provide fast access to short-lived data.

Potential use cases include:

• Response caching
• User sessions
• Rate limiting
• Distributed locks
• Idempotency keys
• Feature flags
• Counters
• Temporary state
• Frequently accessed query results

The target should be a high cache-hit ratio.

For example:

Cache-hit ratio: 90%
Requests reaching the database: 10%

If the platform receives 15,000 requests per second and Redis serves 90% of the required data, the database could receive approximately 1,500 queries per second instead of 15,000.

Redis should not automatically be treated as the permanent source of truth.

The services should also support a controlled degradation mode if the cache becomes unavailable.

7. Database Strategy

There is no single database that is ideal for every type of workload.

I would use different database technologies for different responsibilities.

PostgreSQL

PostgreSQL would be suitable for:

• Relational data
• Payments
• Configuration
• Accounts
• Permissions
• Transactional operations
• Data requiring strong consistency

Depending on the volume, the PostgreSQL layer could use:

• Table partitioning
• Read replicas
• PgBouncer
• Tenant-based sharding
• Independent databases for each cell
• A managed solution such as Amazon Aurora PostgreSQL

Distributed Key-Value or Wide-Column Database

For very high-volume data accessed through predictable keys, I would consider:

• Amazon DynamoDB
• ScyllaDB
• Apache Cassandra
• Google Cloud Bigtable

These technologies can be suitable for:

• Device state
• Activity timelines
• Counters
• Large event histories
• Write-heavy workloads
• Predictable key-based queries

ClickHouse

For analytics and queries across large event datasets, I would use ClickHouse.

Possible workloads include:

• Business reports
• Usage metrics
• Behavioural analysis
• Large aggregations
• Operational dashboards
• Audit analysis

Running large analytical queries directly against the transactional database would be a mistake.

The database responsible for the product’s day-to-day operations should not also act as its data warehouse.

Object Storage

Historical data, exports and raw events could be stored in:

• Amazon S3
• Google Cloud Storage
• Azure Blob Storage
• MinIO

Object storage is considerably cheaper than transactional storage and can later support data reprocessing, analytics or machine-learning workloads.

8. Kafka as the Event Backbone

Kafka would become the backbone of the asynchronous processing layer.

When an important operation occurs, the responsible service would publish an event.

For example:

{
  "event_id": "01JXYZ123",
  "event_type": "order.created",
  "event_version": 1,
  "tenant_id": "tenant-123",
  "order_id": "order-987",
  "occurred_at": "2026-07-12T10:30:00Z"
}

Other services could consume the event to:

• Send notifications
• Update a search index
• Record analytical data
• Refresh reports
• Execute fraud checks
• Synchronise external systems
• Populate a data lake
• Invalidate caches

This prevents the API from having to complete every secondary operation before responding to the client.

The synchronous flow could be limited to:

1. Validate the request
2. Persist the main operation
3. Publish or prepare the event
4. Return the response

Everything else could happen asynchronously.

9. Preventing Dual Writes with the Transactional Outbox Pattern

A common problem occurs when a service needs to:

1. Write data to a database
2. Publish an event to Kafka

Imagine that the database write succeeds, but publishing the Kafka message fails. The system would be left in an inconsistent state.

I would use the Transactional Outbox pattern to solve this problem.

Within the same database transaction, the service would write:

• The main business data
• A record in an outbox table

BEGIN;

INSERT INTO orders (...);

INSERT INTO outbox_events (
    event_id,
    event_type,
    payload,
    created_at
) VALUES (...);

COMMIT;

A separate process would then publish the pending outbox records to Kafka.

Change Data Capture with a platform such as Debezium could also be used to capture and publish these events.

This pattern reduces the risk of losing events between the transactional database and Kafka.

10. Kafka Partitioning

The number of Kafka partitions should not be chosen arbitrarily.

It depends on:

• Producer throughput
• Consumer throughput
• Required level of parallelism
• Average message size
• Number of consumer instances
• Ordering requirements

A partition key could use:

tenant_id
customer_id
order_id
device_id

Events associated with the same identifier would be sent to the same partition, preserving ordering for that key.

However, the system must avoid hot partitions.

A customer with substantially more traffic than everyone else could concentrate too many messages in one partition.

In that situation, logical sharding could be introduced:

tenant-123:0
tenant-123:1
tenant-123:2
tenant-123:3

The final number of partitions should be selected using realistic benchmarks rather than theoretical estimates alone.

11. Idempotent Consumers

Kafka commonly operates with at-least-once delivery semantics. This means that the same event may occasionally be delivered more than once.

Consumers must therefore be idempotent.

Each event should have a unique identifier:

event_id: 01JXYZ123

Before executing an irreversible operation, the consumer should determine whether the event has already been processed.

This is particularly important for:

• Payments
• Order creation
• Account balances
• Issuing credits or benefits
• Sending notifications
• External integrations

Processing the same event twice should produce the same final result as processing it once.

12. Retries and Dead-Letter Queues

Temporary failures are normal in distributed systems.

Consumers should use retries with:

• Exponential backoff
• Jitter
• A maximum number of attempts
• Dedicated retry topics
• A dead-letter queue

For example:

orders.events
orders.retry.1m
orders.retry.10m
orders.retry.1h
orders.dlq

A problematic message should not be retried indefinitely inside the main topic.

Otherwise, one invalid or unprocessable message could block or degrade an entire partition.

13. Backpressure and Load Shedding

When demand temporarily exceeds capacity, the platform needs to protect itself.

This can be achieved through:

• Bounded queues
• Concurrency limits
• Rate limiting
• Circuit breakers
• Early rejection
• Degraded responses
• Prioritisation of critical operations

It is often better to reject a small percentage of requests quickly than to allow every service to fail slowly.

Operations could be classified by priority:

Priority 1: authentication, payments and core operations
Priority 2: standard product data
Priority 3: reports and exports
Priority 4: non-essential background operations

During periods of overload, lower-priority operations could be temporarily restricted.

14. Resilience Between Services

Every external dependency should have:

• A timeout
• A circuit breaker
• Limited retries
• Bulkhead isolation
• A fallback strategy
• Dedicated metrics

Retries should only be used for safe or idempotent operations.

An inappropriate retry strategy can multiply an incident.

For example:

Service A retries three times
Service B retries three times
Service C retries three times

A single original request could produce up to 27 internal attempts.

This is known as retry amplification and can turn a minor slowdown into a complete outage.

15. Observability

A platform operating at this scale must allow the engineering team to answer three questions quickly:

What is happening?
Where is the problem?
Which customer or operation is affected?

I would use OpenTelemetry to standardise:

• Metrics
• Distributed traces
• Logs
• Context propagation between services

The observability stack could include:

• Prometheus
• Grafana
• Loki
• Tempo
• OpenSearch
• Datadog
• New Relic
• Honeycomb

Important API Metrics

• Requests per second
• p50, p95 and p99 latency
• Error rate
• Timeout rate
• Requests by endpoint
• Active connections
• Number of goroutines
• Memory consumption
• Garbage collector pauses

Important Kafka Metrics

• Messages produced
• Messages consumed
• Consumer lag
• Throughput by partition
• Under-replicated partitions
• Average message size
• Processing duration
• Dead-letter queue volume

Important Database Metrics

• Active connections
• Queries per second
• Slow queries
• Lock wait time
• Replication lag
• Cache-hit ratio
• CPU and storage utilisation
• Throttling events

Every log entry should contain identifiers such as:

trace_id
request_id
tenant_id
user_id
region
service
version

This would allow an engineer to follow a request from the edge all the way to the final Kafka consumer.

16. Service-Level Objectives and Error Budgets

I would define clear reliability targets.

For example:

Availability: 99.99%
p95 latency: below 200 ms
p99 latency: below 500 ms
Error rate: below 0.1%

An availability target of 99.99% allows approximately 52 minutes of downtime per year.

I would also use error budgets.

If the engineering team consumed the error budget too quickly, feature deployments could be reduced or paused until the platform returned to an acceptable level of reliability.

This makes reliability measurable rather than subjective.

17. Security

Security must exist at every layer of the architecture.

The platform should include:

• Web Application Firewall protection
• DDoS protection
• Rate limiting by IP address, user and tenant
• OAuth 2.0 or OpenID Connect
• Short-lived access tokens
• Key rotation
• TLS encryption in transit
• Encryption at rest
• Centralised secrets management
• Least-privilege access policies
• Network segmentation
• Audit logging
• Container vulnerability scanning
• A Software Bill of Materials
• Signed build artefacts
• CI/CD supply-chain protection

Internal services could use mutual TLS or workload identities.

Secrets should never be embedded inside Docker images or stored directly in the source-code repository.

18. Deployment Strategy

A high-volume platform should not deploy a new version directly to 100% of users.

I would use:

• Canary deployments
• Blue-green deployments
• Feature flags
• Automatic rollbacks
• Progressive delivery

For example:

1% of traffic
5% of traffic
20% of traffic
50% of traffic
100% of traffic

During each stage, the platform should evaluate:

• Error rate
• Latency
• CPU utilisation
• Memory consumption
• Kafka consumer lag
• Business metrics

If a regression is detected, the deployment should be interrupted or rolled back automatically.

19. Load Testing

No architecture should be considered capable of processing 40 billion requests per month simply because the diagram looks correct.

It must be tested.

I would perform:

• Load tests
• Stress tests
• Spike tests
• Soak tests
• Chaos engineering experiments
• Regional failover tests
• Kafka broker failure tests
• Redis failure tests
• Database degradation tests
• Deployment rollback tests

Tools such as k6, Vegeta or Gatling could simulate realistic traffic patterns.

The tests should include:

• Realistic payloads
• Authentication
• Different endpoint distributions
• Cold caches
• Warm caches
• Large tenants
• Hot keys
• Regional traffic
• Slow client connections
• Partial dependency failures

The objective is not only to discover the maximum number of requests per second.

It is also necessary to understand how the system behaves when it reaches its limits.

A well-designed platform should fail in a predictable and controlled manner.

20. Network Traffic Estimate

The amount of network traffic depends heavily on the average response size.

If each response averages 5 KB:

40 billion × 5 KB
≈ 200 TB of response data per month

If each response averages 20 KB:

40 billion × 20 KB
≈ 800 TB of response data per month

These estimates do not include:

• HTTP headers
• Retries
• Database replication
• Internal service communication
• Kafka messages
• Logs
• Distributed traces
• Cross-region traffic

This is why compression, CDN caching, regional routing and careful payload design would have a significant effect on the final infrastructure cost.

The Complete Architecture

My reference architecture would include the following components:

Edge
├── Anycast DNS
├── CDN
├── WAF
├── DDoS protection
└── Rate limiting

Routing
├── Global load balancer
├── Regional load balancers
└── API gateway or Kubernetes ingress

Application
├── Services written in Go
├── HTTP and JSON for public APIs
├── gRPC or Connect for internal communication
├── Bounded concurrency
├── Timeouts
├── Circuit breakers
└── Graceful degradation

Asynchronous processing
├── Apache Kafka
├── Schema Registry
├── Transactional Outbox
├── Idempotent consumers
├── Retry topics
└── Dead-letter queues

Data
├── Redis Cluster
├── PostgreSQL
├── Distributed key-value database
├── ClickHouse
├── Search engine
└── Object storage

Infrastructure
├── Kubernetes
├── Multi-region architecture
├── Cell-based architecture
├── Autoscaling
├── Infrastructure as Code
└── Progressive delivery

Observability
├── OpenTelemetry
├── Prometheus
├── Grafana
├── Distributed tracing
├── Centralised logs
└── SLO-based alerting

Is Kafka Actually Necessary?

One important point is that processing 40 billion requests per month does not automatically mean that the platform needs Kafka.

Kafka would make sense if the product required:

• Asynchronous processing
• Multiple consumers for the same event
• Event reprocessing
• Integration between many services
• Large event volumes
• Analytical pipelines
• Decoupling between business domains

For a predominantly synchronous and relatively simple API, introducing Kafka could add unnecessary operational complexity.

The architecture should be driven by the business requirements, not simply by the size of the monthly request estimate.

Conclusion

Forty billion requests per month equates to approximately 15,000 requests per second on average.

This volume can be handled by a modern distributed architecture, but not simply by adding more servers.

Scalability would come from the combination of:

• Aggressive caching
• Asynchronous processing
• Data partitioning
• Cell-based isolation
• Specialised databases
• Stateless services
• Idempotent operations
• Backpressure
• Observability
• Load testing
• Operational automation

Go would provide a strong foundation for efficient and concurrent services. Kafka would decouple processes and help absorb temporary traffic peaks. Redis would reduce pressure on the databases. A multi-region, cell-based architecture would limit failures and support progressive growth.

However, the most important lesson is that 40 billion requests should not be treated simply as one large monthly number.

The system must be designed around:

Traffic peaks
Latency requirements
Consistency requirements
Message and payload sizes
Geographical distribution
External dependencies
Infrastructure costs
Failure behaviour

The most scalable architecture is not necessarily the one that uses the largest number of technologies.

It is the architecture in which every component has a clear responsibility, operational limits are understood and failures have been anticipated before they occur.