Every request your product receives hides a quiet question: does this need the big model? Support inboxes tag most tickets as billing or bug. A review queue clears most posts with a glance. So an AI router sends each request down the cheapest lane that can handle it. Only hard cases go upstream to a frontier model. Then came the number that stopped builders mid-scroll: about 10,000 decisions for $0.42. A developer reported it in a LangChain AutoModeMiddleware context.
The idea went mainstream in September 2026, when LangChain published a Jev walkthrough that included model selection examples. Jev is a System One model that evaluates a state and returns typed answers and probabilities. Because it never writes text, it answers in the time a lookup takes. That makes Jev the natural fast lane inside an AI router.
This guide shows you how to build the router itself. We cover where the routing pattern fits and which decisions belong to Jev. Then a confidence threshold hands uncertain cases to GPT, and you measure the LLM cost savings honestly. Every number here is attributed, because launch-week claims deserve your skepticism.
In this guide
- The 30-second answer
- What you’ll learn
- What an AI router is (and is not)
- Why most calls do not need a frontier model
- Where Jev fits in the fast lane
- The confidence threshold that runs the switch
- A worked AI router flow, end to end
- Measuring the LLM cost savings honestly
- Rollout tips from early adopters
- Frequently asked questions
- The bottom line
The 30-second answer
An AI router is dispatch logic, not a model. It classifies each incoming request, then sends it to the cheapest engine that can serve it well. Easy, narrow decisions go to Jev, which returns typed answers with probabilities instead of generated text. Hard, open-ended work goes to your frontier language model. In fact, the split mirrors how teams pair a junior with a senior hire. The difference is that a router enforces it on every single call.
None of this is new in spirit. Teams have routed requests since the first load balancer. What changed is the fast lane itself. Before Jev, the cheap lane was usually a smaller LLM with the same token bill and the same drift risk. Now the fast lane is a decision model that cannot ramble, cannot improvise and cannot hide its doubt.
Key takeaways
- An AI router classifies each request first, then sends it to the cheapest engine that can handle it.
- Jev is the fast lane: it answers narrow questions instantly, with probabilities instead of prose.
- A confidence threshold decides when a case escalates from Jev to your frontier model.
- TypeSafe AI reports similar intelligence on System One tasks at two orders of magnitude faster.
- The savings look real but early, so measure them on your own traffic before you commit.
What you’ll learn
The route through this guide
- What an AI router is, in plain engineering terms
- Why most production calls never needed a frontier model
- Where Jev fits, and what its three answer types return
- How a confidence threshold runs the switch between lanes
- A worked flow you can copy, from request to logged receipt
- How to measure savings and roll out without betting the product
What an AI router is (and is not)
Start with the network analogy, because it is exact. A network router reads each packet and forwards it down the right path. An AI router reads each request and forwards it down the right model. The logic is a small piece of code you own. So there is nothing to buy and nothing to trust blindly.
Model selection is the router’s first job, and LangChain’s September 17, 2026 coverage showed it directly. Their article demonstrates Jev choosing which model should handle a request, alongside tool-risk checks. Specifically, the router asks a typed question: which of these engines fits this request? The answer comes back with a probability attached, and the dispatch code takes it from there.
In production, the same shape shows up everywhere. A help desk routes tickets before any drafting starts. Document pipelines sort pages before any summarizing begins. An agent platform screens tool calls before any code runs. Each case asks a narrow question first, and each one pays a frontier price today.
Here is what a router is not: a magic cost button. If your classification is wrong, the cheap lane handles a hard case badly. So good routers log every dispatch decision and audit the mistakes weekly. Also, a router adds one hop of latency, which is why the fast lane needs to be genuinely fast.
Why most calls do not need a frontier model
Count the judgment calls your product makes per day, then count how many need open-ended reasoning. For most stacks, the ratio is brutal. Ticket triage, spam checks, urgency scores and routing picks are all narrow questions with known answers. However, teams have been answering them with a poetry machine, because that was the only option on the shelf.
Jev vs GPT is not a fight; it is a division of labor. The decision model skips token-by-token generation entirely, which is why its answers feel like lookups. TypeSafe AI claims similar intelligence on System One tasks at two orders of magnitude faster. DataCamp’s launch analysis reported 40 to 200 times faster than frontier LLMs. On cost, the gap stretched to 40 to 400 times cheaper.
The economics compound at volume. At one decision per user session, savings are trivia. By a thousand decisions per minute, they are a budget line. So the teams paying attention first run agents that make hundreds of small calls per run. That is where the bill actually lives.
The pattern has a name in ops meetings: tiered support. Level one handles the known, level two handles the unclear, and escalation is a feature instead of a failure. An AI router gives that structure to machine decisions. Because Jev shows a probability with every answer, the escalation rule can be a number instead of a guess.
Where Jev fits in the fast lane
Jev routing starts with the input shape. You send a state: text, a JSON object, or an array of texts. Then you attach typed questions whose options you defined in advance. Because the model is non-generative, it assigns probabilities to predefined options and cannot answer outside them. Early arXiv analysis framed it exactly that way.
Three answer types cover the fast lane. Choice picks one option from up to 255, and returns the probability of each plus a confidence value. Score grades a record with a number. Noul is the typed empty answer that fires when nothing fits, and it is the feature that keeps routers honest.
The developer surface is deliberately boring. On the JavaScript side, the SDK installs from npm as @typesafe-ai/sdk on Node.js 20 or later. From there, client.systemOne() infers return types from your question definitions. A Python client covers typed questions with model pinning and test seams, while community .NET wrappers fill the third stack. If your router lives in JavaScript, the fast lane is an afternoon of wiring.
The input shape also hints at batch jobs. An array of texts is a legal state, so you can classify a whole queue in one pass. That is how bulk jobs reach the reported economics, because per-call overhead matters less when the queue is long.
Jev routing works best when the answer set is fixed and the volume is high. Sorting requests, scoring records and screening outputs were the first documented use cases. If your question changes every time, keep it on the slow lane. Jev has no opinion about things you never asked, which is exactly the point.
The confidence threshold that runs the switch
The threshold is the part that makes an AI router safe. Jev returns a confidence value with every Choice, so your code can branch on it. Above your line, the fast lane acts alone. Below it, or on a Noul, the request escalates to the frontier model.
Set the AI router’s escalation line empirically, not aspirationally. Replay real historical cases and watch where Jev’s probabilities disagree with your ground truth. Then put the line just under that disagreement zone. Tune it monthly, because drift is normal and thresholds are cheap to adjust.
Noul deserves its own rule. The typed empty answer means no option genuinely fit, which is information, not noise. So treat Noul like a low-confidence case by default. Escalate it, log it, and read the logs monthly. A rising Noul rate usually means your options no longer match reality.
Every dispatch should also leave a receipt. Store the question, the options, the chosen answer, the probabilities and the lane that acted. Because the receipt is structured from the start, audits become queries instead of archaeology. That habit costs minutes and pays for itself the first time someone asks why the router did what it did.
A worked AI router flow, end to end
Here is the full loop with two lanes. An AI router does not think; it reads, dispatches and records. The flow below is the whole design, and the table after it shows one example policy you can tune.
One request, two lanes
An example routing policy (tune the bands yourself)
| Confidence | Path | Example |
|---|---|---|
| High | Jev acts alone | Tag a ticket as billing or bug |
| Middle band | Jev drafts, GPT confirms | Pick a category for an odd request |
| Low or Noul | GPT takes the call | Handle a complaint that fits no option |
Notice what an AI router policy buys you. The expensive engine only sees the messy middle and the edge cases. Meanwhile, the cheap lane absorbs the predictable bulk. Because every dispatch is logged, you can prove where each answer came from. That proof matters more as agents take on real actions.
What if both lanes fail? The frontier model can also misfire on a messy case. So keep a third exit: a human queue for anything the router escalated twice. It is a rare path, and it is the difference between a weird Tuesday and a support incident.
Measuring the LLM cost savings honestly
Treat every launch number as a claim, not a promise. The two-orders-of-magnitude figure comes from TypeSafe AI’s own material. DataCamp’s launch analysis reported 40 to 200 times faster than frontier LLMs. On cost, the gap stretched to 40 to 400 times cheaper. A developer reported the $0.42 figure while running about 10,000 decisions through LangChain’s AutoModeMiddleware context. None of those numbers describe your traffic yet.
A frontier model is a brilliant, expensive consultant. The router’s whole job is to stop calling them for every yes-or-no question.
So measure the AI router on your own replay set instead. Pull 50 recorded AI responses from one workflow. Run the same questions through both lanes and compare answers, probabilities and bills. That single experiment beats any benchmark table. For the full benchmark picture with sources, see our breakdown of Jev’s benchmark numbers. The broader math, including what else drives spend, lives in our AI implementation cost guide.
Track three numbers from day one. First, accuracy against your replay ground truth. Second, the escalation rate, because a router that escalates everything saves nothing. Third, cost per thousand decisions, which is the figure your finance team will actually read.
Rollout tips from early adopters
Start smaller than feels comfortable, because an AI router multiplies whatever accuracy you give it. The community’s getting-started advice is to test one task first, and early adopters echo it. When the first lane holds for a few weeks, add the next question.
- Pick one high-volume question your product already asks a language model today.
- Define the options precisely, and decide what Noul should mean for that question.
- Install the official SDK, pin the model version, and replay real history before going live.
- Set a confidence threshold below which the router escalates instead of acting.
- Log every dispatch with its probabilities, so you can audit wrong-lane decisions weekly.
- Treat the routing pattern as a fallback design, never as a full replacement.
When the pilot looks good, migrate a workflow to Jev one question at a time. Keep a rollback plan in place for the whole ride. We compare the two engines task by task in the Jev vs ChatGPT guide. Meanwhile, the Jev System One model guide explains why the fast lane exists at all.
Frequently asked questions
What is an AI router in simple terms?
It is dispatch logic that classifies each request and sends it to the cheapest capable model. Easy decisions go to Jev, while hard ones go to a frontier LLM. You own the rules and the thresholds.
Why use Jev instead of a large model for routing?
Because routing is a narrow question with known answers, which is exactly Jev’s lane. It skips token-by-token generation, so the classification is fast and cheap. The frontier model stays reserved for work that needs open-ended reasoning.
How much money does the pattern actually save?
Honest answer: it depends on your traffic, and launch numbers are early. Reported figures include 40 to 400 times cheaper decisions and about 10,000 decisions for $0.42. Replay your own cases before trusting any multiplier. Treat the first month as a measurement period, not a verdict.
What happens when Jev is unsure?
It returns a low confidence value or the typed empty answer called Noul. Your threshold then routes that case to the frontier model. Nothing gets guessed silently, which is the design goal.
Does an AI router replace my language model?
No. GPT-class models still handle drafting, summarizing and open-ended reasoning. The router just stops them from wasting tokens on questions a lookup could answer.
How long does a first test take?
About an afternoon of setup plus one replay run. Community advice says to limit the first test to a single task. Fifty recorded responses from one workflow are enough to see the shape of the savings.
The bottom line
The AI router pattern is unglamorous, and that is its strength. It does not chase the smartest model; it chases the right-sized one. Jev turned fast, typed, probabilistic decisions into something you can call from code. The routing pattern turns those decisions into savings you can measure.
So build one lane of your AI router this week. Pick the narrowest, high-volume question you own, wire Jev behind a threshold, and keep GPT on the escalation path. Then let the receipts, not the launch posts, tell you what the pattern is worth.
Sources
- TypeSafe AI, Jev product overview and System One benchmark claims — typesafe.ai
- LangChain blog, Jev walkthrough with model selection and tool-risk examples — langchain.com
- DataCamp, launch analysis of Jev speed and cost versus frontier models — datacamp.com
- semly.ai, getting-started advice to limit the first Jev test to one task — semly.ai
- Early arXiv analysis of Jev’s non-generative design — arxiv.org