Most model pricing punishes you twice: input tokens and output tokens. Jev charges for input only, output tokens are free, and questions inside a single request are evaluated in parallel, so response time barely moves when you add a question. Those three properties have one design consequence: ask more per request, then decide in code.
Parallel questions
The parallel questions cookbook is the headline result. A 13-question regulatory briefing batched into one call came back 12.2 times cheaper and 10 times faster than the naive version, with no change in the answers. The mechanism is simple: one state, one request, one answer per question, all evaluated in parallel.
Before batching, a single question was one round trip. After, a briefing is one round trip.
Speculative fan-out
The fan-out pattern goes further and asks questions the code might not need. The ticket-triage example sends a Choice for category plus four speculative judgments in the same request: bug severity, reproduction steps, whether a refund is likely, and customer frustration. The bug branch reads severity and repro; the billing branch reads refund; the rest is ignored.
if category.choice == "bug_report":
if bug_severity.score > 1.5 and bug_repro.noul > 0.6:
escalate_to_engineering(ticket_id, severity="high")
Speculation has a price, because every question rides along as input tokens on every call. Fan out on judgments that several branches can reuse and leave the rest for later requests.
Composite scoring
Some decisions are not one question but several weighted ones. The composite scoring pattern scores a resume on four dimensions in one request, normalizes each score by its top level, and combines them with weights that live in code:
ic_score = (0.40 * py) + (0.10 * lead) + (0.40 * arch) + (0.10 * general)
em_score = (0.15 * py) + (0.40 * lead) + (0.20 * arch) + (0.25 * general)
Same answers, two role policies. When the job description changes you edit weights, not questions.
Batch at the edges of the context window
The most impressive batching happens when the candidate set is the state itself:
- Line-by-line search ranks all 218 lines of a 43,980-character document in one request, with a separate existence question that can answer "the document does not contain this".
- Skill suggestion ranks 182 agent skills in one request and re-reads the top three in a second, cutting wrong skill loads from 16.8% to 7.3%.
- Structure recovery reconstructs Markdown from unformatted text in exactly two requests: 16 line-pair questions to stitch split sentences, then 62 questions to classify each block, 10,211 tokens in 0.8 seconds.
Each of these replaces something that would otherwise be a loop of model calls with a single parallel pass, or two.
Know the limits
Batching is not free of constraints. Choice accepts at most 255 options, so a longer document needs a windowed second pass. Context is 64k, split between state and the longest question. Input bills at $42 per billion tokens, so the question is not "is this cheap" but "is this question worth its input on every call". And unanswered speculative answers still cost tokens even when your code ignores them.
A short checklist before you batch:
- Can several branches reuse this judgment? If yes, add it to the shared request.
- Is the candidate set smaller than 255 options? If not, pre-filter or window.
- Does the state fit comfortably under the context budget with room for the questions?
- Are the thresholds and the weights in code, where you can change them without re-prompting?
- Did you verify the batched answers match the one-question-at-a-time answers on your fixtures?
Do those and a single request stops being a cost-saving trick and becomes the default shape of a Jev call.
