Groq is the company that made 'wow that's fast' a daily experience.
Their LPU (Language Processing Unit) is a custom chip designed from scratch for autoregressive inference. The result: 500-800 tokens per second on Llama 3.3 70B, faster on smaller models.
Why speed matters in UX: users notice when the first token lands in 200ms vs. 1.5s. Apps feel magical when responses feel instant. Groq makes that reachable on open models.
Their API: OpenAI-compatible, drop-in for most client libraries. Free tier is generous — enough to prototype, not enough to build a business on.
Models available: Llama 3.x, Qwen, Mixtral, DeepSeek, Gemma. They support fast fine-tuning for enterprise customers.
Where it falls short: they don't offer vision or audio models, and the catalog is smaller than OpenRouter's.
The strategic play: Groq + Llama is the fastest path to a genuinely delightful AI chat UX with zero infrastructure pain. If your product lives on perceived speed, start here.