Skip to content
JevDirectory.org
CommunityPractices & PatternsVerified 2026-09-20

Jev as an LLM Judge

Yuchen Jin's take on Jev for LLM-as-a-judge work: 20-200x faster and 40-400x cheaper, with a screenshot of the model picking which candidate is AGI after a prompt sweep under $0.10.

Open x.com
Category
Practices & Patterns
Published by
Community
Author
Yuchen Jin
Added
2026-09-20

The post

Yuchen Jin
Yuchen Jin
Databricks
@Yuchenj_UW
·

Jev has spoken. It picked which model is AGI. 20–200x faster. 40–400x cheaper. This could make things like LLM-as-a-judge insanely fast and nearly free. (I tried a bunch of prompts and still didn’t burn through $0.10.)

Image
Diogo Almeida
Diogo Almeida
TypeSafe AI
@CompleteSkeptic

After co-inventing ChatGPT, I kept asking myself: why have superhuman chat models not led to AGI? I’ve spent the last 2 years in stealth building a new way to train models (RLCD), and a new type of frontier AI model that we are releasing today: Jev • 20-200x faster • 40-400x

Watch on X
Reply
Read 94 replies
Tagscommunityxevaluationbenchmarks

Highlights

  • Makes the case for Jev as a judge scorer: fast, cheap, and structured rather than free text.
  • Reports a prompt sweep that stayed under $0.10.
  • The screenshot shows Jev picking among candidate models, a classic judge task.

Watch out

A commentary post, not an evaluation: no fixture set, no agreement numbers, and vendor cost multiples.

More like this

Jev Beats Gemini Flash Lite on a Classifier Eval
Vercel CTO Malte Ubl reports that Jev beat an existing classifier eval previously run on Gemini 2.5 Flash Lite, saturating the eval on quality and running 6x faster.
Practices & Patterns#community#x#evaluation
Community
What Can You Build With Jev? Documented Use Cases
A map of documented Jev use cases, routing, guardrail verification, citation checking, reranking, rubric scoring, bulk classification, and semantic linting in CI, each with a confidence policy.
Practices & Patterns#community#routing#evaluation
Community
Jev in the Agent Runtime
An engineering note that places Jev as a typed micro-decision layer inside an agent harness rather than a planner, with preconditions for confidence gates and fan-out and a runtime split into five parts.
Practices & Patterns#community#agents#routing
Community
Back to all resources

From the community

Posts from builders shipping with Jev right now.

Classifying rows in DuckDB

The launch post