Ran @typesafeai's Jev against an existing classifier eval that previously used Gemini 2.5 Flash Lite. It won both on quality (saturated the eval) and speed (6x)
Typed Decision Models: Jev and Laya in Agentic AI
A comparative review of Jev and the open-source Laya encoder: interfaces, deployment, disclosed internals, calibration claims, and the limits of agreement-based workflow evals.
- Category
- Practices & Patterns
- Published by
- Community
- Author
- Michał Chromiak
- Added
- 2026-09-20
Tagscommunityevaluationcalibrationagentsarchitecture
Highlights
- Notes that TypeSafe's workflow evals measure agreement with other models rather than ground truth.
- Details Laya: a 421M ModernBERT encoder with a typed-option head and open weights.
- Reports 67.8% mean agreement at $0.0004 per case versus 67.9% at $0.0304 for the comparison model.
More like this
An engineering note that places Jev as a typed micro-decision layer inside an agent harness rather than a planner, with preconditions for confidence gates and fan-out and a runtime split into five parts.
Practices & Patterns#community#agents#routing
Community
A map of documented Jev use cases, routing, guardrail verification, citation checking, reranking, rubric scoring, bulk classification, and semantic linting in CI, each with a confidence policy.
Practices & Patterns#community#routing#evaluation
Community
Break a judgment into atomic Score questions, normalize each by its top level, and combine them with weights you control in code, as in the resume example's four scored dimensions.
Practices & Patterns#official#patterns#score
Official
From the community
Posts from builders shipping with Jev right now.
Beating Gemini Flash Lite on an eval
Browser Use Ultrafast, powered by Jev
Breaking: Browser Use + Jev = Ultrafast ⚡ Findings flights took 7s and cost only $0.0039 🤯 > new action space every step > DOM state space > small LLM fallback to type (this video is at 1x speed btw) Built a tiny open source browser agent. try it below ↓