feed/tags/hbm

HBM

One explainer you can run, 2 glossary terms, and one reading ordered as a path, each with one line on why it earns your time.

explainers/ 1 page

beginnerThe KV cache is the thing you are actually renting
Fill a GPU with weights and per-request cache until it will not take another user. 320 KiB per token adds up faster than anyone expects.

glossary/ 2 terms

High bandwidth memory
The stacked DRAM on the GPU package that holds the weights and the KV cache.
Memory bandwidth
Bytes per second the GPU can move between its HBM and its compute units.

intermediate/ 1 entry

The mental models the rest of the field takes for granted.

interactiveInteractive H100 architecture glossary
Rotate the die and hover the parts. Memory bandwidth stops being a number you quote and starts being a place on the chip.
Kyle Jeong, with definitions from Modal's GPU GlossaryAug 2026#gpu#hardware#hbm

Related topics