Hajer

A model swap is a behavior change

Know what breaks before you switch.

Your tests pass. Your schemas validate. The price sheet says the invoice will go down. Somewhere in the long tail of un-fixtured traffic, the new model quietly drops a behavior the old one nailed. Hajer will run the replay from your environment and stop at the first change the evidence cannot defend.

Prelaunch. Hajer is choosing design partners with a migration already on the calendar.

By joining, you agree to receive Hajer product updates. You can unsubscribe at any time.

What actually breaks

For teams whose model switch has a date on it and no eval team behind it.

010203040506070809101112010203040506070809101112BEHAVIOR AS RECORDEDSAME EVIDENCE, NEW MODELDETERMINISTIC FIRSTRECORDED TRAFFIC, REPLAYEDPROMPT · TOOL CALLS · OUTPUTCURRENT MODELVERIFICATION GATECANDIDATE MODEL
Current modelVerification gateCandidate modelOne record. One variable changed.

What actually breaks

A model swap doesn't throw an error. It returns something plausible, well-formed, and entirely different.

Regressions from a model change arrive as behavior nobody tested, on paths nobody considered important enough to fixture.

Eight ways production behavior moves when the model underneath it moves. None of them announce themselves.

  • Tool argument drift

    Checked without a judge

    The candidate satisfies the same tool schema but populates it differently: numbers arrive as strings, formerly populated optional fields disappear, enum casing changes, or dates use a new format.

    Schema validation passes, so nothing fails at the boundary. The break happens several services downstream, on the subset of arguments no fixture covers.

  • Call shape change

    Checked without a judge

    The candidate emits several tool calls in one turn where the current model emitted one, splits one call into a sequence, or reaches for a different tool to do the same job.

    The orchestrator was written against the old cadence. Both transcripts look correct in isolation; only the loop that consumes them knows the difference.

  • Refusal boundary shift

    Requires comparing meaning

    A thin class of legitimate traffic, such as one domain, phrasing, or document type, starts getting refused or deflected under the candidate's policy tuning. Or a class you never served starts getting answered.

    It is a slice of volume too small to move an aggregate pass rate. The users who hit it hit it every single time.

  • Preamble drift

    Checked without a judge

    The candidate wraps its answer in explanation, a code fence, or a closing offer to help, where the current model returned the bare value.

    Every response is a good response. The parser expecting a bare value is the thing that fails, and the parser is not in anybody's eval.

  • Silent truncation

    Checked without a judge

    A more verbose candidate hits the same output ceiling and returns a payload cut mid-structure. The same class of break occurs when it stops emitting the delimiter a downstream splitter depends on.

    The API returns a success status. The finish reason carries the truth and most clients never read it. Truncated JSON fails at parse time, far from the call that caused it.

  • Instruction priority inversion

    Requires comparing meaning

    The candidate weighs the system prompt against user-turn instructions differently. A user instruction the current model ignored now wins, or a system constraint suppresses formatting the user requested.

    Every instruction is still present and still followed by something. Only the precedence order changed, and precedence is never the thing anyone writes an assertion on.

  • Cost-per-task inversion

    Checked without a judge

    A candidate that is cheaper per token costs more per completed task, because it is more verbose, takes more turns, or spends more reasoning before it answers.

    Per-token price is on the pricing page; per-task cost is on the invoice. Hajer will report it through the divergence and say where it stopped.

  • Hedge drift

    Requires comparing meaning

    The candidate qualifies where the current model committed, or concedes to an incorrect user assertion it previously pushed back on.

    The text is not wrong; it is softer. Downstream classifiers, extractors, and assertion checks calibrated on the firmer voice read the hedge as a different answer.

Most changes in a model swap are mechanically checkable: structure, call shape, boundaries, budgets, and cost. Hajer will prove what it can without a judge and use semantic comparison only when meaning itself is under test. It will report uncertainty instead of averaging it away.

Where a difference can only be judged, Hajer says it was judged. It does not launder a judgment into a score.

Standard LLM-as-a-judge pipelines ask whether the output looks acceptable; they do not prove that the surrounding contracts still hold.

Method

Three movements. One record.

Hajer does not guess what your system does. It will read what your system did, then replay the switch where it cannot affect your users.

Rehearsal

The migration happens twice.

Your production traces will become the rehearsal. The same requests, the candidate model, your keys, no live side effects. Everything is held still except the one variable you are changing. Public benchmarks measure someone else's distribution. Your cutover risk lives in unusual traffic, prompts near your context ceiling, and the parsers, timeouts, and tool contracts written around the model you already run. Hajer treats what production shipped as accepted, not correct. It is the reference because your users lived with it, not because anyone graded it.

IRehearsal
ONE RECORDCURRENT MODELBEHAVIOR AS RECORDEDCANDIDATE MODELSAME EVIDENCE, NEW MODELRECORDED RESULTSNO LIVE SIDE EFFECTSYOUR KEYSEITHER SIDE IS WHATEVER YOU POINT IT AT.

Your current model is the benchmark.

Divergence

Hajer follows the evidence until it breaks.

Identical text is not the bar. Two models will rarely produce it, and demanding it would flag every run. Hajer will hold the run to the behavior your system depends on and mark the point where that behavior diverges. Then Hajer stops. Stopping there is part of the guarantee. Once a candidate's turn diverges, every recorded result after that point describes a conversation that never happened. Replaying it would mean scoring fiction.

IIDivergence
CURRENTCANDIDATEFIRST DIVERGENCENOT SCOREDSCORED

Refusing to score there is the only honest thing a replay can do.

Verdict

A gain does not override a critical failure.

Deterministic where behavior can be measured, semantic where it can only be judged. The result is one of four states, with the evidence that produced it attached to every line. Averages do not clear gates.

IIIVerdict
EVIDENCECUTOVER READYREVIEW REQUIREDCUTOVER BLOCKEDINSUFFICIENT EVIDENCEONE OF FOUR. THE EVIDENCE DECIDES.

Hajer will not score a path it cannot reconstruct, and it will not average its way across one.

The code travels. The data does not. Your traces and keys stay entirely under your control.

Your environment remains the boundary.

Hajer will run inside your environment: on a laptop, in a CI runner, or in your own VPC. It will start from a trace export, not from your repository, production logs, or internal documents. The Hajer process will read your keys from the environment and call your selected providers directly. During a replay, Hajer's infrastructure will receive none of your keys, trace export, candidate calls, or report. The report will remain local unless you send it.

Your environment

  • Trace export

    From your existing observability stack

  • Provider keys

    Yours. Never brokered or stored by Hajer's infrastructure

  • Reconstruction

    The call, rebuilt as it happened

  • Checks

    Built from the record, cited to it

  • Replay + scoring

    Current and candidate, side by side

Not requested

  • Repository
  • Production logs
  • Internal documents
ReportA local artifact. It leaves only when you send it.

Hajer does not proxy provider calls. No replay data routes through Hajer's infrastructure.

Evidence enters

What leaves the boundary

The verdict is a document, not a dashboard.

Hajer will return a document a person can read, a result a pipeline can gate on, and the checks that produced both. Every line traces back to something production actually did. There is no score to round in your favor. One of the four answers is that the evidence cannot support a decision, and Hajer will return that answer rather than a convenient one.

  • Cutover ready

    Every gate is satisfied.

  • Review required

    The candidate changed behavior in ways the evidence records but cannot adjudicate. You decide, with the divergences in front of you.

  • Cutover blocked

    A critical gate failed. Nothing elsewhere outvotes it.

  • Insufficient evidence

    The record cannot support a decision either way. Hajer says so instead of guessing.

  • The contract
  • Safety
  • Success
  • Reliability
  • Correctness

A gate that cannot be grounded fails. Insufficient evidence is the verdict when no gate can be grounded at all.

Constructed example

The regression you would have shipped.

Setup
A support-triage agent. Same prompts, same tools, same recorded traffic. One model changes. The deterministic checks pass: schemas validate, tool contracts hold, and measured cost falls. Every dashboard says ship it.
Failure
On a narrow class of refund questions, the candidate stops handing off to a human and answers them itself. The answer is fluent, correctly toned, and outside policy. No schema catches it. No latency chart catches it. Averages bury the failure, because the aggregate pass rate actually went up.
Hajer's verdict
Hajer marks the call where the hand-off behavior changed, refuses to score paths beyond it, and returns Cutover blocked. The candidate may be better. The evidence still cannot prove the cutover is safe. A gain cannot answer that question.

This example is constructed. Hajer has no customer results to show, and will not invent any.

Decision leaves

Who this is for

For teams with a real migration on the line.

  • AI behavior already runs in production.

    A regression here does not page anyone. It quietly changes what your users get, for as long as it takes someone to notice.

  • Your traces carry the content, not just the shape.

    Plenty of collectors record that a call happened and throw away what was in it. Hajer will read your export before it does anything else, and tell you which records can be replayed and which cannot.

  • A model switch is approaching, and a dedicated eval team is not.

    Somebody will be asked whether it is safe to switch. Right now there is no way to answer that with evidence.

And what Hajer is not

  • Not an observability platform. It will start from traces you already keep, and test the swap moment rather than replacing ongoing production monitoring.

  • Not an eval suite to write from scratch. Hajer will draft the checks from what your traces already show, in your vocabulary. You keep them, and they will rerun on the next migration.

  • Not your release authority. Hajer will hand you a verdict and the evidence under it. Signing the cutover stays with the person who carries the consequence.

  • Not a model recommendation. Hajer has no opinion about which model you should run.

  • Not a proxy. Hajer does not proxy provider calls.

  • Not a benchmark. Your current model, on your traffic, is the benchmark.

Provider migrationMarks name endpoints, not partners

Hajer is for teams facing a specific switch, such as a deprecation, provider change, or cost or latency move, with a date attached. If the switch is hypothetical, Hajer is early for you.

// Design partners

Bring us the migration you cannot leave to guesswork.

Tell us about the migration you are not looking forward to. We are choosing design partners by how much is riding on the switch.

By joining, you agree to receive Hajer product updates. You can unsubscribe at any time.

Add migration details

Optional. These details only help us prioritize design partners.

What Hajer is, and what it is not.

Questions

What is Hajer?

Model migration assurance. It will replay the traces your system already produced against a candidate model, compare what changed, and return a cutover verdict with the evidence attached.

Is Hajer another observability platform?

No. Observability tells you what happened. Hajer will tell you what happens if you change the model and refuse to guess when the traces cannot support an answer. It will start from the traces you already keep and test one moment: the swap.

How can a replay be valid for a multi-turn or agentic system?

It is valid right up to the first divergence, and not one step past it. Once the candidate's turn differs, the recorded results that follow belong to a conversation that never happened, and scoring them would be scoring fiction. That is exactly why Hajer stops at the first divergence and reports everything after it as unsupported. Any tool that replays straight through that point is producing a number, not evidence.

What happens when the evidence is incomplete?

Hajer fails closed. It will separate what it can score from what is missing, uncertain, or past the divergence, and return Insufficient evidence when the record cannot support a decision. That is a verdict, not an error.

Does Hajer need our codebase?

No. The workflow will start from a trace export. It will not require your application repository, your source code, your production logs, or your internal documents.

Where does the data go?

Only where you direct it. Hajer will run in your environment on your own provider keys. Candidate calls will go directly from your environment to your selected providers, never through Hajer's infrastructure. The report will remain local unless you send it.

Which trace platforms will you support?

Langfuse is the first planned trace adapter. Others will follow. Nothing is available yet. This is a waitlist, and Hajer will name one adapter rather than imply a roster.

Why is this a product rather than an internal script?

The first script is easy. The continuing work is keeping checks honest as traffic changes, stopping at the first divergence, preserving critical failures when averages improve, and producing evidence a skeptical reviewer can inspect. If your team already owns that work, you may not need Hajer.