We ran all three on an 8x A100 node serving Llama 3.3 70B. Here's the honest scorecard.
vLLM: the incumbent. PagedAttention manages KV cache like virtual memory — 2-4x memory savings over naive attention. 52 req/s at 512-token outputs, 280ms TTFT. Best ecosystem: Hugging Face, NVIDIA, everyone supports it.
SGLang: the new challenger. RadixAttention shares prefixes across requests — huge win for few-shot and multi-turn. 65 req/s, 210ms TTFT, structured JSON output via grammar-constrained decoding. Best for agents and tool-calling.
TGI (Text Generation Inference, HuggingFace): the Rust-powered one. 45 req/s, 240ms TTFT, excellent integration with HF hub and endpoints. Best if you live inside the HF ecosystem.
Our pick: SGLang for new projects with agent workloads, vLLM for general-purpose serving, TGI if you're already on HF's infrastructure.
The ugly truth: they're all 90% as good as each other on raw throughput. The 10% difference is ergonomics, ecosystem and ops burden — pick what your team can run at 3am without paging anyone.