#AI Inference

Також: llm inference, model inference

Running trained models to produce predictions or generated output in production. Covers latency and throughput tuning, batching and quantisation, GPU and accelerator utilisation, serving runtimes such as vLLM, TensorRT-LLM and Triton, and inference cost.