Tag: autotuning

All the articles with the tag "autotuning".

• TensorRT-LLM: Compiling Inference Engines Ahead of Time

  Updated:16 Oct, 2026

  TensorRT-LLM began by compiling each model into an engine: operations fused into fewer kernels, and every kernel timed and chosen for the GPU and the shapes it would serve. Both steps measured on a laptop GPU: fusion removed 26% of the operations per token and 7% of prefill time; the best version of one kernel depended on the shape, and the worst cost twice as much. Then why NVIDIA moved the project onto a PyTorch runtime, and what it keeps of the compiled design.