Fan-out: why asking Jev many questions at once is cheaper

TypeSafe’s documentation puts it in one line: Jev ingests the state once and evaluates every question against it in parallel. That sentence is worth money, and most Jev cost calculators ignore it entirely.

TypeSafe uses two names for this. The model page calls the aggressive form speculative fan-out; the cookbook calls the pattern parallel questions. They describe the same billing behaviour, and everything below applies to both.

The maths

You are billed for input tokens: the state, the question definitions, and a fixed overhead O of about 260 tokens that every request is measured to bill. So for one item with S state tokens and N questions of Q tokens each:

The difference is (N−1)×(O + S). You pay for the item’s own tokens, and the request overhead, once instead of N times. When the state is large relative to a question — which is the normal case, because a support ticket is long and “is this urgent?” is short — this dominates the bill.

A 500-token item with 60-token questions

At four questions you are already cutting the bill by roughly 70%. At sixteen it is over 85%. You also make (N−1) fewer requests per item, which matters against your organization request quota. Whether requests or tokens bind depends on that quota and your explicit token-counting assumption.

TypeSafe’s own benchmark, reproduced

This is not only our arithmetic. TypeSafe publishes a parallel questions cookbook that measures it: the Wikipedia article on GDPR (revision 1363040264, about 53,777 characters) with 13 questions — 8 Noul, 2 Choice, 3 Score — run on jev-1.12. One batched call came out 12.2× cheaper and 10.0× faster than 13 sequential calls, and the answers were the same either way, because each question is scored against the document on its own.

Put that scenario through the model on this page and you get the same answer. At roughly four characters per token, 53,777 characters is about 13,444 state tokens. With 13 questions the batched call is260 + 13,444 + 13Q tokens against 13 × (260 + 13,444 + Q) for the sequential version, plus the ~260-token overhead on every call — a ratio of 12.4× at 60-token questions and 12.2× at 80-token ones. TypeSafe measured 12.2×. The arithmetic on this site lands inside one percent of their own instrumented run, which is the point: the saving is a property of how you are billed, not a benchmark result that might not repeat.

Open the calculator preloaded with the cookbook’s scenario →

Measured independently: 2,976 billed requests

We also measured it ourselves, in the open: jev-fanout-bench sent 2,976 requests to jev-1.13-20260917 on 2026-09-23, through OpenRouter's TypeSafe-compatible System One endpoint (provider: TypeSafe), each batched call next to the same questions asked one at a time, every block repeated three times. Code, raw responses and checksums are public.

• The bill is exactly tokens × rate. Every response’s charge equals its input tokens at $0.042 per 1M, to the ninth decimal. The 122,844 output tokens returned cost nothing.
• Every request bills about 260 tokens of overhead. A one-word state still bills 261 tokens per request beyond its questions. TypeSafe’s API reference agrees: its one-sentence example reports 296 input tokens.
• Billing is exactly linear, so batching saves (N−1)×(O + S) tokens and nothing else changes.
• Even 120 unrelated questions do not interfere. In a second run, four target answers moved no more with 16, 60 or 120 unrelated questions in the call than when repeated alone, and latency stayed flat from 4 to 124 questions (426–436 ms). A call with 1,000 questions still succeeded. What each question costs.
• The answers do not change. Batched versus alone, Noul answers moved 0.005 on average and Score answers 0.01 (0–1 scale) — exactly as much as sending the identical request twice. No Choice answer changed its top option.

Correction. Until 2026-09-23 this site’s calculator billed only the state and the questions, and left the overhead out. On the default support-triage workload that understated the bill by about a quarter: $93.24 a month became $126.00. The fan-out saving itself barely moved.

Where it stops

In TypeSafe direct default mode, the whole request — state plus every question — must fit in 64k tokens, and the state plus the single longest question must fit in 32k. Explicitly enabling long context with TypeSafe-Danger-Zone-Long-Context: true raises both limits to 262k at the same token price, with slower, less accurate results. OpenRouter publishes a separate 262,144-token context policy; check your route and mode. TypeSafe also documents accuracy shifting as the state grows, so a giant state with thirty questions bolted on is not automatically the right answer. Batch, then measure.

The aggressive version is what TypeSafe means by speculative fan-out: including questions you might not need, because the marginal cost of one more question is just its own tokens, and branching in your own code afterwards is free.

Put your own numbers into the calculator →

Behaviour and limits from docs.typesafe.ai/models, checked 2026-10-17.