glossary/compute/ridge-point
Ridge point
The arithmetic intensity at which a machine's memory ceiling and compute ceiling meet: peak FLOPS divided by bandwidth. Below it a workload cannot use all the arithmetic because the bytes do not arrive fast enough; above it, bytes arrive faster than they can be used. It moves with the card: a chip whose compute grows faster than its bandwidth pushes the ridge right, leaving decode further below it.
208 FLOP per byte
On the A100 for dense BF16; every explainer's prefill crosses it at about 208 prompt tokens.
See it happen
Related
- glossary/
- Roofline model
A chart with arithmetic intensity on the x axis and attainable performance on the y, bounded by a sloped line from memory bandwidth and a flat line from peak compute.
- Arithmetic intensity
Floating-point operations performed per byte moved from memory.
- Memory-bound
Limited by how fast bytes can be read, not by how fast they can be operated on.
- Compute-bound
Limited by arithmetic throughput; the memory system keeps up and the tensor cores are the bottleneck.
- sources/
- topics/