UPSHIFT · BY MECHANISMcompatibility and repair for production AI agents
Upgrade the model.
Keep the agent.
Upshift runs your agent on both models, finds what the upgrade broke, and tries to fix it. If it cannot, it tells you to stay pinned.
uv tool install git+https://github.com/Mechanism-world/upshiftReplay of a real run: shell_gpt, GPT-5.5 to GPT-6 Astra, 2026-09-08.See the records
- Runs in your environment
- Uses your OpenAI or Anthropic key
- Produces a reviewable patch
- Can say stay pinned
THE LOOP
How it works.
Four steps. Every call, transcript, and check result is written to disk, so any claim on this page can be reopened later.
01
Baseline
Run your agent on the model that works, five times per case. A suite that cannot pass here never reaches a verdict.
run baseline gpt-5.5 N=5 · 14 cases
14 pass · 0 fail
02
Compare
Run the same cases on the candidate and sort each one into stable, regressed, improved, or flaky.
run candidate gpt-6-astra N=5 · 14 cases
regressed 14 · flaky 0 · fisher p < 0.001
03
Repair
Try a bounded set of changes: the prompt, model parameters, the tool schema, the endpoint route. Your application code is never rewritten.
- "endpoint": "chat_completions" + "endpoint": "responses"- Prompt
- Parameters
- Tool schema
- Endpoint route
04
Prove
Rerun the broken cases and the ones that already passed. A fix is accepted only when nothing else moves.
rerun full suite 10 / 10 reps
SAFE WITH PATCH
OBSERVED RUNS
Four real upgrades.
Every number here comes from a committed run record. One of them says do not upgrade.
shell_gpt
GPT-5.5 → GPT-6 Astra2026-09-08
13 / 14working0 / 14after upgrade14 / 14after patchSAFE WITH PATCH
GPT-6 Astra rejects function tools when reasoning_effort is set on Chat Completions. Upshift moved the agent to Responses and the suite passed again.
Read the reportMethod
Public open-source agent, evals authored by Upshift, N=5 reps per case per model. One case was already failing on GPT-5.5 and passed in the final run. $2.60 of API spend. Run records
Cookbook SMS bot
Claude Fable 5 → Fable 5.12026-09-01
5 / 5working0 / 5after upgrade5 / 5after patchSAFE WITH PATCH
Fable 5.1 rejects forced tool_choice. Upshift removed it, added the documented instruction, raised reasoning effort one rung, and every case came back.
Read the reportMethod
Open-source notebook agent from anthropics/claude-cookbooks, evals authored by Upshift, N=5 reps per case per model (25 reps each way). Run on release day. $0.93 of API spend. Run records
shell_gpt
GPT-5.5 → GPT-5.6 Sol2026-08-29
14 / 14working0 / 14after upgrade14 / 14after patchSAFE WITH PATCH
GPT-5.6 Sol broke tool calling on Chat Completions. Upshift moved the agent to Responses and the full suite passed again.
Read the reportMethod
Public open-source agent, evals authored by Upshift, sandboxed Linux run, N=5 reps per case per model and 10 on the patched suite. $0.56 of API spend. Run records
Booking agent
GPT-5.5 → GPT-5.6 Sol2026-08-28
36 / 38working0 / 38after upgrade32 / 36after patchSTAY PINNED
Three repairs brought back most of the suite, but four cases stayed broken and one candidate fix damaged a case that already worked. Upshift refused to call the upgrade safe.
Read the reportMethod
Internal agent and eval suite, not a customer workload, N=5 reps per case per model and 10 on contested cases. $12.31 of API spend. Run records
THE OUTPUT
You get a patch you can read.
Accepted changes arrive as a git diff. Rejected candidates stay in the run record with the reason they were rejected.
- Why the candidate was attempted
- Which cases were rerun
- What collateral behavior stayed intact
- Why the final verdict was reached
{
"name": "shell-gpt",
- "endpoint": "chat_completions",
+ "endpoint": "responses",
"model": "gpt-5.5"
}LOCAL BY DEFAULT
Everything runs on your machine.
Upshift sits next to your repository and calls the provider directly with your key. Prompts, transcripts, results, and patches are written to disk and stay there. There is no Upshift service, account, or telemetry. This site is the one exception, and only for itself: it counts page views without cookies.Privacy.
01
Your repository
agent + eval cases
02
Local Upshift runner
compare, repair, verify
03
Your model provider
direct API calls
04
Local evidence + patch
human review
CURRENT SCOPE
What works today.
Works today
- Plain OpenAI API agents (Chat Completions and Responses)
- Anthropic Messages API agents
- Deterministic eval checks, without an LLM judge
- Prompt, parameter, tool-schema, and endpoint-routing repairs
- Local run records, resumable execution, and git-apply patches
- `upshift adapt`: generate the adapter from a repository
Not yet
- Google, Bedrock, or local model providers
- LangChain, CrewAI, or Agents SDK integrations
- Streaming or multi-agent graphs
- Hosted execution, accounts, or a dashboard
- Automatic merges, deployments, or arbitrary source rewrites
RUN IT LOCALLY
See the whole loop in about a second.
The simulator runs baseline, candidate, repair, verification, verdict, and patch at zero API cost. It proves the machinery works, not that a model behaves.
Requires Python 3.12+ and uv.
$ uv tool install git+https://github.com/Mechanism-world/upshift
$ upshift init my-agent
$ upshift upgrade --agent my-agent \
--provider sim \
--baseline-model sim-5.5 \
--candidate-model sim-5.6-sol \
--tag demo
VERDICT SAFE WITH PATCH
patch runs/demo/upgrade.patch
# real run, ANTHROPIC_API_KEY in .env
$ upshift upgrade --agent my-agent \
--baseline-model claude-fable-5 \
--candidate-model claude-fable-5-1 \
--tag my-upgradeBRING A REAL AGENT
Have an upgrade you do not trust?
15 minutes on a migration that broke your agent. Bring the working version, the target version, and whatever evals you have.
A short technical session. No sales deck.
BEFORE THE NEXT UPGRADE