Every "free AI API" list tells you Groq gives you a 131k context window. We wired it into an agent, watched it fail, and went looking for the real number. It is 7.5k — one seventeenth of the advertised figure — and the reason is not in the model card.

Where it breaks

A one-shot "reply with the digit 2" passes on every free tier, so it measures nothing. Instead we walked the prompt size up a ladder, holding max_tokens at 512 and pacing calls 20 seconds apart so a per-minute rate limit could not be mistaken for a per-request size limit.

Groq served a 5,111-token prompt and refused at 10,000. Mistral and GLM served the whole ladder. The error Groq returns names the real constraint:

Request too large for model `openai/gpt-oss-120b` ... service tier
`on_demand` on tokens per minute (TPM): Limit 8000, Requested 8271

Not a context error. A tokens-per-minute error.

The budget counts what you ask for, not what you get

This is the part that catches people. "Requested" is prompt + max_tokens — the ceiling you declared, not the completion you received. Three probes make it unambiguous:

A twenty-token question is rejected. Not because the question is large, but because asking for a generous completion ceiling spends the whole minute's budget up front — before a single token is generated.

So the usable prompt is roughly 8000 − max_tokens. Ask for 512 tokens of output and you have about 7,500 tokens of room. Ask for the 65,536 the model card permits and you have none at all.

The limit is not one number

We first spot-checked two models and concluded the cap was per-model: gpt-oss-120b failed a request that qwen3.8-27b passed. Running the full sweep later the same day, the identical qwen3.8-27b request returned 413. The first run had simply landed in a minute that still had headroom.

Both things are true, and neither is documented:

• Limits differ by model. At the same ~8,200-token request, gpt-oss-120b, gpt-oss-safeguard-20b, qwen3.8-27b and qwen3.6-27b all returned 413, while gpt-oss-20b and compound-mini returned 200.
• Consumption is time-windowed. The same model, same parameters, returns 200 or 413 depending on what you spent in the preceding minute.

The practical consequence: never conclude a model is exempt from one passing probe. We nearly published that mistake.

What Groq is actually for

None of this makes Groq bad. It makes it specific. Against the same agent-shaped request — a system prompt plus five tool definitions — Groq is not slightly faster, it is in a different class:

Roughly 3× faster than the alternatives, and that has always been Groq's pitch — we covered the speed itself in our earlier Groq review. What that piece did not say, and what a year of "fastest free API" write-ups still do not say, is what the speed costs you.

Put the two findings together and the shape of the tier is obvious. Groq's free tier is a high-frequency, short-payload layer:

• Classification, routing and intent detection
• Field extraction from one document at a time
• Reranking search results
• Titles and summaries for a single chunk
• Guardrail and moderation checks
• Anything streaming, where time-to-first-token is what users feel

It is the wrong tier for an agent's primary model, long-document RAG, or a multi-turn coding session — anything that replays a growing transcript. Those prompts only go one direction.

Groq's own catalogue agrees. It ships llama-prompt-guard-2-22m and -86m with a 512-token context window, and allam-2-7b with 4,096. Those are purpose-built short classifiers. The platform is telling you what it is for.

Nine of the fourteen models take a chat request

We called every model /models returned on a free key. Five cannot serve a chat completion at all:

The list mixes speech, text-to-speech, guard models and chat models together with no modality field to separate them. If you build a model picker by reading /models — which several agent harnesses do — you will offer users four models that return 400 the moment they are selected.

A routing detail Groq did not document

When groq/compound hit its limit, the 429 named a different model:

Rate limit reached for model `openai/gpt-oss-120b` ... Limit 8000,
Used 4373, Requested 6278

We asked for compound; the budget was charged against gpt-oss-120b. Groq's compound systems are a routing layer over the underlying models, and the error text is the only place that shows through. Worth knowing if you are accounting for usage per model — compound traffic lands on another model's budget.

Configuring it honestly in a harness

Most agent runtimes take a context window per model and trim the transcript to fit. Give them 131,072 and they will happily assemble a 30k prompt that Groq refuses. Declare the budget you actually have:

models:
  - id: openai/gpt-oss-120b
    contextWindow: 7000   # not 131072 — this is TPM minus max_tokens
    maxTokens: 512

One warning about the failure mode. When the 413 comes back, harnesses tend to relabel it. Ours reported "Context overflow: this conversation is too large for the model" on a session holding 1,400 tokens. We went looking for an oversized system prompt that did not exist. If a context error contradicts your own token count, check whether the upstream status was really a 413 rate-limit response.

Reproduce it

Both scripts are in our benchmark repo — the model sweep and the size ladder. With a free key:

export GROQ_API_KEY=gsk_...
node automation/groq_sweep.mjs     # every model, two probes each
node automation/size_ladder.mjs    # find the size where it stops

The ladder paces requests 20 seconds apart on purpose. Without that gap you measure your own rate limit and conclude the wrong thing.

Bottom line

Groq's free tier is not a small version of a paid tier — it is a different shape. You get remarkable throughput inside a 7,500-token working set, and no path to the 131k the model card advertises. Used as a fast classification and routing layer it is the strongest free option we have measured. Used as an agent's primary model it fails, and the error message will not tell you why.

The practical setup is both: Groq for the short, frequent calls, and a wider-window free tier — Mistral or GLM both served 30,000 tokens without complaint — for the turns that carry a transcript. For how these behave when driving an actual agent loop, see our agent test across free models, and the wider free API comparison for what else is still genuinely free.

Measured 1 September 2026 against the on_demand free tier. The token limits, status codes and model list above are properties of the API and do not depend on where you call from. The latency figures do: all four providers were called over the same network path in the same session, so the ordering between them is meaningful, but treat the absolute milliseconds as indicative rather than a benchmark of your own connection. Rate limits are the kind of thing providers change without announcement — re-run the scripts above rather than trusting these numbers a year from now.