Search Autocomplete

Autocomplete is the rare system where the latency budget is shorter than human reaction time. The suggestion list has to repaint between one keystroke and the next, which means the server has tens of milliseconds, not hundreds, and it pays that cost on every single character a user types. The volume is brutal precisely because it is keystroke-level: a search bar fielding a few thousand queries per second is answering tens of thousands of completion requests per second. What makes this tractable is that the hard work and the fast work are completely separable. Ranking which completions are good is an expensive, data-heavy computation over everyone's search history, but it does not need to be fresh to the second. Serving the answer for a prefix is a trivial lookup, but it has to be instant and survive nodes dying mid-spike. So the whole design is an exercise in pushing all expensive computation offline and making the online path do nothing but read from memory. This guide walks you to that design from first principles, with the numbers that justify each choice, calibrated to a chaos simulator whose targets are p99 under 80ms, availability at or above 98%, sustained throughput of 15,000 requests per second, and peak saturation no higher than 0.9.