Starter
US$99 a month
- 1 model watched
- 6 task families
- 120 fresh tasks, every week
- Alert with example mistakes and a signed receipt
DEMO Avanciq Model Change Alarm · prototype preview · 16 November 2026
AI providers update and retire models. Every week we give your model fresh tasks from your kind of work, mark them to the cent, and alert you when accuracy really drops.
Your watch profile. You tell us the model your product uses and the work it does. We pick matching task families.
Fresh tasks every week. Each run is 120 new tasks with hidden answers fixed before the model sees them. Nothing can be memorised.
Marked by code. A task counts as right when every field is right. No AI judge.
A real drop, not noise. The alarm fires when accuracy falls at least 7 points below the last two normal weeks and a statistical test puts the chance of plain noise below 1 in 100.
An alert you can act on. What dropped, by how much, which family broke, example mistakes, and a signed receipt.
Example Co (a made-up customer) uses Claude Haiku 5.5. Weeks 1 to 6 are real runs of that model. Before week 7 we secretly switched the model to an older version, Claude Haiku 4.5, without telling the alarm, to copy a silent provider change.
ALARM: exact accuracy fell from 95% to 43.3% (52 of 120 tasks right, against 228 of 240 in weeks 5 and 6).
95% ranges: before 91.5 to 97.1%, now 34.8 to 52.3%. Drop 51.7 points; chance of plain noise: p = 5e-29.
Likely cause family: Claims settlement, Bank reconciliation, Delivery routes, 3-way match.
Extra clue: the provider reported a different model name this week (claude-haiku-4-5, before claude-haiku-5-5). The alarm itself came from the scores alone.
| Task family | Before (weeks 5-6) | This run | Flag |
|---|---|---|---|
| 3-way match | 38 of 40 (95%) | 1 of 20 (5%) | Likely broke |
| Bank reconciliation | 39 of 40 (97.5%) | 4 of 20 (20%) | Likely broke |
| Claims settlement | 38 of 40 (95%) | 6 of 20 (30%) | Likely broke |
| Delivery routes | 36 of 40 (90%) | 6 of 20 (30%) | Likely broke |
| Long documents | 38 of 40 (95%) | 15 of 20 (75%) | |
| Contract invoice schedules | 39 of 40 (97.5%) | 20 of 20 (100%) |
| Task | Family | Field | Expected | Given |
|---|---|---|---|---|
T-0135b87dd5 | Claims settlement | claims.CL-4280.decision | partial | pay |
T-ccff04b104 | Bank reconciliation | unpaid | 6 items | added INV-0519, INV-1567 |
T-8d6176df4a | Delivery routes | total_cost | 396.9 | 407.6 |
T-182e15fe9b | 3-way match | status.VI-3156 | HOLD_QTY | APPROVED |
T-ca2c0bd218 | Long documents | Q1 | Granite Works | Dunmore Freight |
We tested the alarm before building this page, with rules fixed first.
When a model is retired you must move. We run the same fresh tasks on the old model and the candidates, so you see accuracy and cost side by side before you switch.
| Model | Fully right | Cost per task |
|---|---|---|
| Claude Sonnet 4.5 (old, retiring) | 17 of 24 (70.8%)95% range 50.8 to 85.1% | US$0.1249 |
| Claude Sonnet 5.5 | 20 of 24 (83.3%)95% range 64.1 to 93.3% | US$0.0416 |
| Claude Haiku 5.5 | 22 of 24 (91.7%)95% range 74.2 to 97.7% | US$0.0043 |
| gpt-oss-120b (open model, high reasoning) | 7 of 24 (29.2%)95% range 14.9 to 49.2% | US$0.0217 |
Pick by our rule (the cheapest model within 10 points of the most accurate): Claude Haiku 5.5. With 24 tasks the ranges are wide; a paid report uses more tasks. See the Model Switch Report offer.
The week 7 receipt lists all 120 tasks with fingerprints of the task, the answer key and the model reply. It has no task text.
SHA-256: ba47d345d4c37d6ae73e8a7f821b876730549eee1e4c3d68b19f140e915d2e35
Ed25519 signature: signature file · public key: key file
sha256sum avanciq-alarm-demo-receipt.json base64 -d avanciq-alarm-demo-receipt.json.sig > sig.bin openssl pkeyutl -verify -pubin -inkey avanciq-alarm-key.pem -rawin -in avanciq-alarm-demo-receipt.json -sigfile sig.bin
US$99 a month
US$249 a month
US$499 a month
Free first check: before you pay anything, we run one check of 120 fresh tasks on your model and send you the result.
We are looking for a few early users who depend on one AI model for business work. No payment is taken on this page.
Every run uses fresh tasks, so scores move a little from week to week. The alarm needs two things at once: a drop of at least 7 points against the last two normal weeks, and a statistical test saying plain chance is very unlikely (p below 0.01). The rule was written down before we tested it.
The alarm uses the scores alone. If the provider reports a different model name, the alert shows that as an extra clue.
Today: bank reconciliation, long documents, contract invoice schedules, delivery routes, claims settlement and 3-way match, plus more families from our scoreboard. We pick the families that match your work.
Each run comes with a signed receipt: fingerprints of every task, answer key and reply. You can check the signature with openssl, and ask us to open any task.
No. This page is a preview with DEMO data from our own test runs. Nothing is charged here.