Designing an Ad Click Aggregator

What you'll learn

• 1. Problem Statement

  Advertisers need to know how many people clicked on their ads in near real-time. An ad click aggregator is a backend system that: 1. Tracks every ad click -- when a user clicks an ad, the system records it reliably 2. Redirects the user…

• Why Is This Hard?

  - Clicks are ephemeral events -- if you lose one, it is gone forever - Advertisers pay per click -- double-counting means overbilling; under-counting means lost revenue - Peak traffic can spike unpredictably (viral ad, major sale event)…

• 2. Requirements

• Functional Requirements

  - Track ad clicks and redirect users to the advertiser's landing page - Aggregate click counts per ad at 1-minute granularity - Provide a query API: "How many unique clicks did ad X receive in time range [T1, T2]?" - Support rollup to la…

• Non-Functional Requirements

  | Requirement | Target | |-------------|--------| | Throughput | 10,000 clicks/second at peak | | Query latency | What breaks next -- the DB is now the whole system's bottleneck. Two flavours of pain: - Write contention: at 10K clicks/se…

• Stage 3 -- Bottleneck: Ingestion write throughput and durability

  This is the first genuinely hard problem, and it is the defining problem of ad-click-aggregation. Every click must be persisted before the user sees the advertiser's page, but you cannot afford a synchronous OLTP write on the hot path. T…

• Stage 5 -- Mature architecture

  The full picture after Stages 2-4 have compounded. Note the two data paths -- the speed path (Kafka -> Flink -> OLAP, seconds of latency) and the truth path (Kafka -> S3 -> Spark -> reconciliation, hours of latency, source of truth for b…

• Cost vs Complexity -- When to Pick What

  | Concern | MVP choice | Growth choice | Enterprise choice | |---------|-----------|---------------|-------------------| | Compute (click ingest) | Single VM / Fargate task | K8s deployment, HPA on RPS | K8s + edge collectors (Fastly / C…

• Migration Path (MVP -> Enterprise)

  An honest ordering. Not every system needs step 6 or 7 -- most successful ad networks stop around step 5. 1. Add Redis dedup in front of the DB. Cheap; the impression-ID HMAC scheme means no cache miss ever costs a DB round-trip. 2. Intr…

• 3. Click Handling -- Server-Side Redirect

  When a user clicks an ad, the click must be reliably tracked before the user is sent to the advertiser's site. A server-side redirect (HTTP 302) guarantees this.

• Flow

• Why Server-Side (Not Client-Side)?

  | Approach | Mechanism | Problem | |----------|-----------|---------| | Client-side (pixel/JS) | Fire tracking request from browser after page load | Ad blockers, network failures, user closes tab before request fires | | Server-side (30…

• 4. Idempotency with Impression IDs

  Double-counting clicks is a critical business error. If a user clicks the same ad twice (or the browser retries), we must count it only once.

• How It Works

  1. Ad Placement Service generates a unique impressionid for every ad shown to a user 2. The impression ID is signed with HMAC -- the Click Processor can verify authenticity without a database lookup 3. On click arrival: - Verify HMAC sig…

• Why This Order Matters

  Writing to the durable store first ensures that crashes between steps 1 and 2 lead to a retry that re-publishes (which Kafka handles idempotently), rather than silent data loss.

• Redis Memory Budget

  - 100M active impressions (rolling 24-hour window) - 16 bytes per impression ID - Total: ~1.6 GB -- easily fits in a single Redis instance - TTL of 24 hours auto-evicts old entries

• 5. Stream Layer (Kafka)

  Kafka serves as the backbone connecting click ingestion to aggregation. It provides: - Durability -- replicated across brokers, survives node failures - Ordering -- messages within a partition are strictly ordered - Replay -- consumers c…

• Partitioning Strategy

  Partition by AdId so that all clicks for the same ad land in the same partition. This enables: - Ordered processing per ad - Efficient aggregation (no cross-partition coordination) - Natural parallelism (one Flink task per partition)

• Archival

  Kafka Connect S3 Sink continuously exports raw events to object storage (S3/GCS) for: - Long-term retention beyond 7 days - Batch reprocessing and reconciliation - Compliance and audit trails

• 6. Stream Processing (Flink)

  Apache Flink performs the real-time aggregation. It consumes from Kafka, groups clicks by (AdId, minute), and writes aggregated counts to the OLAP database.

• Event-Time Semantics

  Flink uses the event timestamp (when the click actually happened) rather than the processing timestamp (when Flink received it). This is critical because: - Network delays can cause events to arrive out of order - Crashes and restarts ca…

• Watermarks

  A watermark is Flink's way of saying: "I believe all events with timestamp threshold: overwrite OLAP with batch results 4. Log discrepancies for monitoring -- persistent gaps indicate a bug

• 10. Hot Shard Mitigation

  When an ad goes viral, all its clicks funnel into a single Kafka partition (since we partition by AdId). This creates a "hot shard" that can overwhelm the Flink task responsible for that partition.

• Solution: Salted Partition Keys

  The Click Processor appends a random suffix (0 to N-1) to the AdId when publishing. This spreads a hot ad's traffic across N partitions.

• Flink Handling

  Flink must now re-aggregate across salted partitions: The OLAP database uses a SumMergeTree (ClickHouse) or equivalent -- concurrent writes to the same (adid, minute) key are automatically summed.

• 11. Fault Tolerance

  Every component can fail. The system is designed so that no single failure loses data.

• Layer-by-Layer Guarantees

  | Component | Failure Mode | Recovery | |-----------|-------------|----------| | Click Processor | Node crash | Stateless -- load balancer routes to healthy instance | | Redis (dedup) | Cache lost | Kafka still has events; slight risk of…

• Stream Retention as Safety Net

  Kafka retains events for 7 days. If any downstream component fails: 1. Fix the component 2. Reset consumer offset to the failure point 3. Replay and reprocess

• Flink Checkpointing

• 12. System Architecture

• 13. Scaling Strategy

  | Component | Scaling Approach | |-----------|-----------------| | Click Processor | Horizontal scaling behind a load balancer; autoscale based on request rate | | Kafka | Add partitions (shard by AdId); add brokers for storage/throughpu…

• Capacity Planning

  At 10K clicks/sec: - Click Processor: ~5 instances (assuming 2K req/sec per instance with headroom) - Kafka: 50 partitions across 5 brokers (ample throughput) - Flink: 50 parallel tasks (1 per partition) - OLAP: 3-node cluster with repli…