Inference API running open models on custom LPU hardware
Groq is a hosted inference service that runs open-weight language models on its own LPU chips rather than GPUs. Developers call it through an OpenAI-compatible API, so most existing code works after swapping the base URL and key. Its selling point is latency: token generation is fast enough that chat responses and voice agents feel immediate, which matters for real-time applications. The catalogue covers Llama, Mistral, Qwen and other popular open models, and the free tier is generous enough to benchmark speed before committing to paid usage.
Last updated: 2026-09-20. This site only provides an index; for exact features, pricing, and licensing, see the official website.
low-latency chat, voice agents, real-time assistants, model benchmarking
If you're comparing similar products, check the alternatives below, or browse all tools in the AI Models & Platforms category.
Groq runs models on its own LPU chips designed for sequential token generation rather than general GPU work. That architecture produces very low latency, which shows up most in streaming chat and voice applications.
Yes, in most cases. The chat completions endpoint follows the OpenAI format, so you usually change the base URL, the API key and the model name without touching the rest of your code.
No. Groq serves its own catalogue of open-weight models and does not accept custom weights. If you need a tuned model, a platform with custom deployment is a better fit.
Developer console and API keys for the Claude model family
Node-based local interface for running image and video diffusion models
Browser playground for prompting and prototyping with Gemini models
Hosted Jupyter notebooks with optional free GPU and TPU runtimes