Ad CTR Prediction: Serving and Trade-Offs

With the model validated for accuracy, calibration, and fairness, the engineering challenge pivots sharply from offline metrics to production infrastructure. Every time a user loads a webpage or scrolls a feed, an ad auction fires. That auction has roughly 100 milliseconds, end to end, to retrieve candidate ads, assemble features, score each candidate with a CTR prediction model, run the auction logic, and return a winning ad. Miss that window, and the ad slot goes unfilled. Revenue evaporates.

This latency budget is not theoretical. Systems at Google, Meta, and TikTok operate under this constraint at billions of requests per day. The budget breaks down approximately as follows: around 10ms for ad retrieval and candidate selection, roughly 5ms for feature assembly, another 5ms for model inference, and the remaining time consumed by network hops, auction ranking, and ad rendering. Each millisecond matters because the scoring service must evaluate hundreds of candidate ads in parallel within that envelope.

The core interview question this lesson prepares you for is direct. “How would you design the serving path for a CTR prediction model that must return scores for hundreds of ad candidates within this budget?” The answer spans three pillars: feature serving strategies that keep retrieval under a millisecond, a feedback loop pipeline that continuously improves the model without introducing bias, and the trade-off reasoning that separates L4 answers from Staff+ answers.

The following diagram illustrates the full serving path from user request to ad response.

With the latency budget and architecture established, the next step is to examine the component that most frequently becomes the bottleneck: feature retrieval. ...