Real-time Chat

A chat system looks deceptively simple — just send a string from A to B — but it is one of the densest interview problems because every hard distributed-systems concept shows up at once: persistent connections, ordering, exactly-once-ish delivery, fan-out, presence, offline handling, and graceful behaviour under partition. What makes it distinctive is that the connection is long-lived and bidirectional. Unlike a request/response API where the server can forget you the moment it answers, a chat server must hold your socket open, know which server holds the recipient's socket, and route a message across that mesh in tens of milliseconds — or stash it durably if the recipient is offline. This guide walks you from a blank canvas to the reference design the SysGym simulator grades against. The simulator models one cell carrying about 9,000 message-delivery events per second, and it grades four things: messages delivered (availability at or above 95 percent), delivery p99 at or below 150 ms, sustained throughput at or above 9,000 rps, and gateway saturation at or below 90 percent. Its scripted chaos — a gateway crash, a load-balancer-to-gateway network partition, and a queue latency spike, all on top of a group that goes off and multiplies load by 1.5x — is engineered so that only a design with two independent gateway paths, each sized to carry the whole surge alone, survives. We will derive exactly why.