found the perfect use case for @typesafeai Jev: instant compaction in 2026, why is compaction still a summarization prompt? Jev can make it instant by scoring every tool call and dropping what’s irrelevant
jev-secret-detection
A measurement harness that sends each file snippet to Jev as one Noul question asking whether the content contains a usable secret credential, then scores the returned probability against the expected label.
- Category
- Practices & Patterns
- Published by
- Community
- Author
- teyhouse
- Added
- 2026-09-22
Tagscommunitypythonsecurityevaluationbenchmarks
Highlights
- The default 100-case set splits 50/50 across vendor tokens, embedded credentials, generic secrets, passwords, connection strings, and private keys.
- The report lists accuracy at a 0.5 threshold, AUC, recall, precision, mean noul per class, Brier score, and how many cases land in the 0.3-0.7 review band.
- Latency output includes round trip, server time from the x-envoy-upstream-service-time header, p50, p95, max, and requests per second.
- Server p50 measured 75 to 90 ms versus TypeSafe's documented ~100 ms; sending all 100 at once pushed round-trip p50 to about 2 s and triggered 529s.
- There is deliberately no regex matching or provider verification, and the Noul question is pinned to jev-1.13.0 in questions.py.
Quickstart
bash
echo "TYPESAFE_API_KEY=YOUR_API_KEY" > .env
uv run python main.py
uv run python main.py configWatch out
No license file, so reuse terms are unclear. Needs Python with uv and a TYPESAFE_API_KEY; fixtures use randomly generated values that secret scanners may still flag, and the config and edge batches are smaller.
More like this
A public benchmark comparing Jev with Claude Haiku 4.5 on whether an email agent should click a link, over 2,000 PhishNChips emails, with calibration, latency, cost, and five signal questions.
Practices & Patterns#community#python#security
Communityjev-phishing-bench
208GitHub stars
A self-hosted implementation of TypeSafe's Jev System One API powered by the 400M-parameter GLiFormer encoder: it serves choice, score, and noul and drops into the official typesafe-sdk via TYPESAFE_BASE_URL, but trails Jev on reasoning-heavy tasks.
Practices & Patterns#community#python#open-models
Communityjeff
From the community
Posts from builders shipping with Jev right now.
Instant compaction with Jev
A Claude session from 1M to 86K tokens
This is actually insane. This uses @typesafeai Jev model, as a plugin in Claude to review all the un-nesseasary tool calls, and it takes 1s to run! Like, literally, 1 second to take my Claude session from nearly 1M to ... 86K tokens! 😮 Ask your claude to install it and be Show more
found the perfect use case for @typesafeai Jev: instant compaction in 2026, why is compaction still a summarization prompt? Jev can make it instant by scoring every tool call and dropping what’s irrelevant
Vercel's fx safety reviewer, 18x faster
We're seeing extraordinary results from @typesafeai. Default mode in 𝚏𝚡 is auto, with a safety reviewer analyzing every command. That reviewer runs on GPT Luna today. Jev is up to 18x faster (p95) *and* more accurate. It's coming to @vercel AI Gateway and likely new default.
@fazxes
We benchmarked fx auto mode (safety) classifier with @typesafeai's Jev. tl;dr: ~5-18x faster and more accurate than 𝚐𝚙𝚝-𝟻.𝟼-𝚕𝚞𝚗𝚊, our current top choice
Jev lands on OpenRouter
Jev by @typesafeai is now on OpenRouter, in beta. Jev is a System One model. Instead of generating text, it takes your app's state plus a typed question and returns a typed decision with a probability attached. There is no JSON prompting, parsing layer, and nothing to validate Show more
700 leads scored for $0.09
JEV is INSANE. We gave it 700 high-intent leads and personalised outreach messages. In 40 seconds, it predicted how each message would perform, assigned a confidence score and detected lead-message mismatches. All for just $0.09. JEV can also score leads, analyse buying Show more
Beating Gemini Flash Lite on an eval
Ran @typesafeai's Jev against an existing classifier eval that previously used Gemini 2.5 Flash Lite. It won both on quality (saturated the eval) and speed (6x)
