How model serving actually behaves once it meets production traffic.

Rotate the die and hover the parts. Memory bandwidth stops being a number you quote and starts being a place on the chip.

Disaggregation only pays if the KV cache moves between pools fast enough. This is the transfer layer underneath Dynamo doing that job.

Four things routinely called 'caching' that solve four different problems. Untangling them is most of the way to reading anyone else's cost numbers.

A useful corrective: the older, simpler mechanism holds up against linear attention on long context, without the post-training bill.

Cold start is the tax on serving many models from few GPUs. Splitting weights from the skeleton takes frontier-model wake-up from minutes to seconds.