Video Streaming

Clarifying the requirements

Start by separating the three workflows hiding inside the word video. There is the write/ingest path (someone uploads a file that must be transcoded and stored), the read/playback path (millions of viewers pull segments), and the metadata path (browse, search, view counts). These have wildly different shapes: ingest is slow, bursty, and compute-heavy; playback is high-volume, latency-sensitive, and almost entirely cacheable; metadata is a small-payload, read-heavy lookup with one nasty hot key (the view counter on the viral video). An interviewer wants to see you scope to the dominant one. Playback is roughly four orders of magnitude more traffic than ingest, so the read path is where the design lives or dies.

Functionally, the system must accept an upload and transcode it into multiple bitrates, stream those bitrates adaptively (the player picks 360p on a phone on cellular, 1080p on a laptop on wifi, and switches mid-stream as bandwidth changes), and support browse, search, and view counts. The reference rubric is explicit that transcoding produces multiple renditions, that delivery uses adaptive-bitrate HLS or DASH chunks served from a CDN, and that view counts are kept asynchronous and approximate rather than strongly consistent.

Non-functionally, the system is global and massively read-heavy, it must minimize origin bandwidth because egress is the dominant cost, and playback must stay smooth under load. Translate those soft goals into the simulator's hard service-level objectives, because they are what you are actually graded on. These are the targets your design must defend with numbers, not adjectives.

• Availability (playback succeeds) at least 99 percent — the highest-weighted objective (weight 3 of the five SLOs).
• Segment p99 latency at most 80 ms, scored as the median of each tick's p99 — so segments must come from the edge, not a round trip to origin, and a brief latency blip can't sink you.
• Throughput at least 20,000 served segment requests per second, holding through the 4x viral peak — that is the request rate, not the viewer count; 20,000 rps is 80,000 concurrent streams.
• Saturation at most 85 percent — the busiest read-path tier (origin servers or object store) must never redline, even when traffic quadruples.
• Footprint at most 8 billable instances — you win by offloading to the edge, not by buying a giant origin fleet.

Back-of-the-envelope capacity estimation

Anchor everything to the simulator's 20,000 segment requests per second baseline and convert it into something physical. Adaptive-bitrate formats chop video into short segments; pick a 4-second segment, a common HLS/DASH choice. An active stream pulls one segment every 4 seconds, so each viewer generates 0.25 requests per second. That means 20,000 requests per second is 80,000 concurrent viewers at baseline, and the 4x viral peak is 80,000 requests per second, or 320,000 concurrent viewers, the overwhelming majority of them watching the same trending clip. This is the defining feature of the workload: a few keys dominate.

Now size the bandwidth, which is the real cost. Take an average sustained bitrate of 5 megabits per second across the adaptive ladder (a blend of phones at ~1 Mbps and laptops/TVs at 8 Mbps or more). At baseline, 80,000 viewers times 5 Mbps is 400 Gbps of egress. At the viral peak, 320,000 viewers times 5 Mbps is 1.6 Tbps. A single 4-second segment at 5 Mbps is about 2.5 megabytes. No origin fleet you would willingly pay for can push 1.6 Tbps, which is exactly why the design is edge-first.

The punchline is the hit-ratio math. With the reference CDN at a 95 percent hit ratio and the origin cache at 90 percent, the fraction of traffic that reaches the origin is 0.05 times 0.10, which is 0.5 percent. That turns 1.6 Tbps of viewer egress into 1.6 Tbps times 0.005, about 8 Gbps, hitting the origin at peak — a number a handful of servers and an object store handle comfortably. Run the same math on the simulator's deliberately under-tuned starter (CDN 0.60, origin cache 0.55) and the origin sees 0.40 times 0.45, which is 18 percent of traffic: roughly 288 Gbps, a 36x larger load that no eight-instance footprint can absorb. The whole exercise is moving those two numbers from 0.60/0.55 to 0.95/0.90.

Storage and transcode round out the estimate. Each uploaded video fans out into roughly six to eight renditions (an ABR ladder from ~240p to 1080p/4K plus audio), so stored bytes are a small multiple of the source. A catalog of a billion videos averaging a few hundred megabytes of renditions each lands in the hundreds of petabytes — the petabyte scale the brief names — which belongs in object storage, never a database. Transcoding a 10-minute video across the full ladder is tens of minutes of multi-core CPU work, and uploads arrive in unpredictable bursts, so ingest must be asynchronous: a queue plus an autoscaled worker fleet, never inline with the upload request.

High-level architecture

The design splits cleanly into the ingest pipeline and the delivery path, joined by object storage. On ingest, the client uploads the raw file (ideally as a direct multipart upload straight to object storage so the bytes never traverse your application servers). Completing the upload enqueues a transcode job. A worker fleet pulls jobs off the queue, transcodes the source into the ABR ladder, packages each rendition into 4-second segments plus an HLS or DASH manifest, and writes those immutable artifacts back to object storage. Metadata (title, duration, available renditions, manifest URL) is written to a sharded metadata store. The upload request itself returns in milliseconds; the video becomes playable when its renditions are ready.

On delivery, the player first fetches the manifest, then pulls segments in sequence, choosing a rendition based on measured bandwidth. Every segment request goes to the CDN first. On a CDN hit (the common case, ~95 percent) the bytes come straight from an edge node a few milliseconds from the viewer. On a CDN miss, the request falls through to an origin cache that shields the origin; on an origin-cache hit (~90 percent of the misses) the segment is served without ever touching an origin server. Only on a miss at both tiers does a lean origin fleet authenticate and sign the request and read the segment from object storage. Because segments are immutable and content-addressed, they are perfectly cacheable, which is what makes those high hit ratios achievable.

This maps onto the simulator's reference graph as a straight chain: Viewers, Edge CDN, Origin cache, Origin servers, Object store. Note the asymmetry the simulator builds in — it requires only four of those on the canvas before a run will even start (viewers, a CDN, origin servers, and an object store), but the origin cache, optional to begin with, is the piece that turns a design that passes on paper into one that actually clears every SLO. The reference settings that clear them are an Edge CDN at 0.95 hit ratio, an Origin cache at 0.90 hit ratio, Origin servers at 5,000 requests per second each with 3 replicas (15,000 effective), and an Object store at 8,000 requests per second. Footprint is billed per service replica and one unit for every other box (the client is free), so this is 1 for the CDN plus 1 for the origin cache plus 3 for the origin replicas plus 1 for the object store, which is 6 — under the cost ceiling of 8. The asynchronous transcode pipeline (queue plus workers) and the sharded metadata/view-count store sit alongside this read path; they are part of the answer you describe even though the graded simulation stresses the delivery chain.

• Ingest: direct upload to object storage, then enqueue an idempotent transcode job.
• Transcode: worker fleet produces an ABR ladder of segmented, immutable renditions plus manifests.
• Delivery: CDN, then origin cache, then a small origin fleet, then object storage — each tier absorbs almost everything the tier above misses.
• Metadata: sharded by video id, hot rows cached, view counts aggregated asynchronously.

Deep dive 1 — the two-tier hit-ratio cascade and the viral thundering herd

The hard part of this problem is not any single component; it is the multiplicative cache cascade and what happens to it when a viral segment is cold. Two cache tiers in series multiply their miss rates, so the origin load is the product of both miss fractions. Pushing the CDN from 0.90 to 0.95 halves origin traffic; adding a 0.90 origin cache behind it cuts what remains by another 10x. This is why the reference design spends its effort on two numbers rather than on origin replicas — and why the simulator's cost objective bills per origin replica but charges nothing for raising a hit ratio. Throwing origin iron at low hit ratios costs footprint and still loses: the instant the object store blips, every origin-dependent request fails no matter how many replicas you bought, so the 18 percent dependence of the starter craters availability while the 0.5 percent of the reference shrugs the same outage off. Raising hit ratios is the one free move that improves availability, latency, saturation, and cost at the same time.

The cache key is the triple of video id, rendition, and segment index, because a segment is immutable once published. That immutability lets you set effectively infinite TTLs, serve stale-while-revalidate, and never worry about invalidation for content (you invalidate manifests, not segments). It is also what lets an edge node serve the same hot segment to 100,000 viewers from one cached copy.

The dangerous moment is the cold viral segment. When a brand-new clip starts trending, its segments are not yet in any cache. If 50,000 players request segment 7 of the 1080p rendition in the same second and every one of those requests is a miss, they stampede through the CDN, through the origin cache, into the origin and object store all at once — a thundering herd that can knock over the very origin the caches exist to protect. The fix is request coalescing (also called single-flight or request collapsing): each cache tier lets exactly one request for a given key proceed to the next tier and parks the other 49,999 until the fill returns, then satisfies them all from the one fetched copy. Production CDNs do this; you should name it explicitly. For predictable spikes (a scheduled premiere, a new episode) you also pre-warm or push content to the edge ahead of demand instead of waiting for lazy pull-through fills.

This is exactly what the simulator's failure timeline probes. When the object store blips at second 28 during the viral peak, the reference design barely notices: at 0.95 and 0.90 hit ratios only 0.5 percent of traffic depends on the origin path at all, so even a total object-store outage drops well under 1 percent of requests and availability stays above the 99 percent target. Run the same six-second blip on the starter's 18 percent origin dependence and you drop nearly a fifth of all in-flight playback for the entire window the store is down — more than enough to drag the run's availability under target. The origin cache is not optional decoration — it is precisely what turns a backend outage into a sub-1-percent non-event.

Deep dive 2 — async transcoding and the view-count hot key

Transcoding must be decoupled from the upload request, and the rubric scores this directly: jobs run asynchronously on a queue plus worker fleet, never synchronously during upload and never skipped. Synchronous transcoding would make uploaders wait minutes and would let an upload spike exhaust your servers; storing a single format would break adaptive streaming entirely. The pipeline is upload, enqueue, transcode, package, publish. To go faster, split a long source into chunks, transcode the chunks in parallel across workers, and stitch the segments — transcoding is embarrassingly parallel along the timeline. Make jobs idempotent and keyed by a content hash of the source so that a retried or duplicated job (a worker dies mid-encode, the queue redelivers) recomputes deterministically and overwrites the same object-storage keys rather than producing duplicates or corrupting state.

The second hard problem is the view counter, which is the canonical hot-key trap. At the viral peak, 320,000 concurrent viewers are watching one clip. A strongly-consistent synchronous increment per view would funnel hundreds of thousands of writes per second onto a single database row — a lock-contention nightmare that serializes the whole platform behind one counter. The rubric explicitly penalizes the synchronous strongly-consistent counter and rewards counting asynchronously and approximately. The standard solution is to never write per view: emit view events to a log or stream, aggregate them in batches (per-edge or per-shard local counters that flush periodically), and accept that the displayed count is eventually consistent and slightly approximate. For raw scale you can layer in probabilistic counting such as HyperLogLog for unique viewers. Nobody refreshes a video because the view count is a few seconds stale.

Metadata more broadly scales by the same two moves the rubric rewards: shard by video id so load spreads across partitions, and cache hot video metadata so the trending clip's manifest and title are served from memory rather than re-fetched from the store on every play. Sharding by video id also conveniently isolates the hot video to one partition, which you then protect with caching and async aggregation rather than letting it hot-spot the whole metadata tier.

Bottlenecks under load and how to scale them

Walk the simulator's 45-second timeline and watch where pressure lands. Traffic ramps from baseline to 2x (trending) to 4x (viral) and settles at 2.5x (long tail). In an edge-first design, the CDN absorbs the ramp almost invisibly because edge capacity is enormous relative to origin capacity and 95 percent of requests terminate there. The bottleneck only appears downstream, and only if your hit ratios are low. With the starter's 0.60/0.55 ratios, the viral peak pushes roughly 14,400 requests per second into an origin fleet sized for 12,000; the origin saturates to 120 percent and forwards its capped 12,000 onto an object store sized for 6,000, which then runs at 2x. Requests drop, and both availability and the saturation SLO fail hard. The reference fixes this not by enlarging those components much (origin goes from 12,000 to 15,000 effective, object store from 6,000 to 8,000) but by starving them of traffic in the first place via the higher hit ratios — at 0.5 percent dependence the origin sees about 400 requests per second at peak instead of 14,400.

Then the chaos hits, and each event teaches a scaling lesson. At second 18 an origin node is killed: because the origin runs as 3 replicas, losing one leaves 10,000 requests per second of capacity, and since only ~0.5 percent of traffic reaches the origin at all, the survivors carry it without breaking a sweat — replication plus a tiny working set equals graceful degradation. At second 28 the object store blips: the origin cache absorbs it, as covered above, so a backend outage costs sub-1-percent availability instead of catastrophe. At second 36 a CDN edge degrades with a latency spike: because segments are small (a few megabytes), heavily cached, and the latency objective is scored as the median of per-tick p99 (robust to a brief 6-second spike), the reference design holds p99 under 80 ms; a real system would also shed the bad edge by re-routing to a healthy POP or a second CDN.

The scaling rules that fall out of this are worth stating plainly. Scale the read path horizontally at the edge, because edge bandwidth is cheap and near-infinite relative to origin. Scale the origin only enough to comfortably serve the small miss stream, and keep it replicated for failure tolerance, not for throughput. Scale ingest by autoscaling the stateless worker fleet against queue depth. Scale metadata by sharding plus caching. And recognize that the dominant scaling lever, by a wide margin, is the cache hit ratio, not instance count — which is exactly why the cost objective and the saturation objective both reward the same move.

Key trade-offs

Every interesting decision here is a trade between cost, freshness, and complexity. Name them and pick deliberately.

On caching strategy, lazy pull-through fills the edge on first miss (simple, but the first viewers of a viral clip pay the miss penalty and risk a herd), while push or pre-warming proactively distributes content to edges before demand (eliminates the cold-start herd for predictable launches, at the cost of pushing bytes that might never be watched). Large catalogs use pull-through for the long tail and push for known tentpole events.

On segment length, shorter segments (2 seconds) lower startup and live latency and let the player switch bitrate faster, but multiply request count and per-request overhead and slightly hurt compression and cache efficiency; longer segments (6 to 10 seconds) compress better and cut request volume but make ABR switching coarser. Four seconds is the common compromise this design assumes.

On storage versus compute, pre-transcoding and storing every rendition (the reference choice) makes serving cheap and perfectly cacheable but multiplies storage by the size of the ladder, while just-in-time packaging or on-the-fly transcoding saves storage but burns CPU on every cache miss and weakens cacheability — viable for the cold long tail, wrong for hot content. Per-title or per-scene encoding spends extra transcode compute up front to cut delivered bitrate (and therefore the dominant egress bill) for popular videos, which is usually worth it.

On consistency, approximate eventually-consistent view counts are the correct default; reserve strong consistency for things that actually need it, like billing, watch-history position, or content licensing windows. And on delivery, a single CDN is simpler while multi-CDN (or embedding caches inside ISP networks, as Netflix Open Connect and Google Global Cache do) improves reach, resilience, and egress economics at the cost of operational complexity. The interview-grade answer is to default to edge-first, immutable, pre-transcoded, asynchronously-counted, and to justify each trade by pointing at the bandwidth bill and the 99 percent availability target.