feed/tags/gpu
GPU
8 glossary terms, and one reading ordered as a path, each with one line on why it earns your time.
- Tensor cores
The GPU units that do small dense matrix multiplies in one instruction, at far higher rate than the general-purpose cores.
- GEMM
General matrix multiply, the operation the projections and feed-forward layers of a transformer reduce to.
- CUDA graphs
Recording a sequence of kernel launches once and replaying the whole sequence with a single call.
- Kernel fusion
Combining several operations into one kernel so intermediate results stay in registers or on-chip memory instead of making a round trip to HBM.
- Kernel launch overhead
The fixed cost of asking the GPU to run a kernel: the driver call, the scheduling, the queue.
- Triton
A Python-embedded language and compiler for writing GPU kernels at the level of blocks of data rather than individual threads.
- CUTLASS
NVIDIA's C++ template library for building GEMM and related kernels that hit tensor core peak, and CuTe, its layer for describing the layouts and tilings that make that possible.
- Nsight
NVIDIA's profilers.
The mental models the rest of the field takes for granted.