What MS Duplex delivers, in numbers.

• −90.3%*Model calls, with accuracy up94.6% accurate, against 59.8% for the model alone

  Model calls in 3,000 decisions

  The model with MS Duplex277

  The model alone2,864

  0Lower is better

  Model calls in 3,000 decisions

  The model with MS Duplex	277
  The model alone	2,864

• 0%*Stale answers after a rule changeCache and retrieval: 72%
  MS Duplex0%

  Cache and retrieval72%

  0Lower is better100%

  Stale answers after a rule change

  MS Duplex	0%
  Cache and retrieval	72%

• 6 / 6*Tests where MS Duplex beat fine-tuning+30.3 points on average, with no training run
• 3 / 3*Frontier models beaten at predicting system changesOn a public enterprise benchmark, with no model calls

  System-change prediction score

  MS Duplex0.256

  Gemini 3 Pro0.205

  Claude Sonnet 4.50.204

  GPT-5.10.166

  01

  System-change prediction score

  MS Duplex	0.256
  Gemini 3 Pro	0.205
  Claude Sonnet 4.5	0.204
  GPT-5.1	0.166

* Figures from MatterSpace’s internal benchmarks, not independently verified. Read the disclaimer

Every result.

* Figures from MatterSpace’s internal benchmarks, not independently verified. Read the disclaimer

• 90% fewer model calls, and more accurate.*

  94.6%*Accuracy, the highest in the testThe model alone 59.8% · with a cache 77.3%

  The model with MS Duplex94.6%

  The model with a cache77.3%

  The model alone59.8%

  0100%

  Accuracy, the highest in the test

  The model with MS Duplex	94.6%
  The model with a cache	77.3%
  The model alone	59.8%

  • −90.3% Model calls · −84.8% cost · −81.9% latency*
  • ~23× Faster once learned · ≈97 ms against ≈2,231 ms*

  A 3,000-decision stream in which the rules changed five times, against the frontier model alone and with a semantic cache; a 300-decision run added a cache with retrieval and a knowledge graph. Every approach used the same model.

  See the full result →Learns and unlearns →

• More accurate at every setting, at a seventh of the cost.

  94.1 to 94.8%*MS Duplex accuracy, at every one of eight settingsThe best a cache reached: 77.3%

  MS Duplex94.1 to 94.8%

  The best a cache reached77.3%

  0100%

  MS Duplex accuracy, at every one of eight settings

  MS Duplex	94.1 to 94.8%
  The best a cache reached	77.3%

  • 84 to 91% On three fresh runs of 1,000 decisions · The best a cache reached: about 73%*

  A 3,000-decision stream with every approach swept across its full range of settings, then three fresh runs of 1,000 decisions in a second, independent setup.

  See the full result →Decides whether to answer, hand over or decline →

• Keeps learning without forgetting, going stale or growing.

  5 / 5*Requirements for learning in production, all met on every runCache and retrieval averaged 2.2 / 5 · online fine-tuning 4 / 5

  MS Duplex5 / 5

  Online fine-tuning4 / 5

  Cache and retrieval2.2 / 5

  05

  Requirements for learning in production, all met on every run

  MS Duplex	5 / 5
  Online fine-tuning	4 / 5
  Cache and retrieval	2.2 / 5

  • 0% Stale answers after a policy change · Cache and retrieval 72%*
  • Fixed Footprint over time · Cache and retrieval grew to ≈1,300 rows*

  Ten runs, with the same supervision and the same inputs for every approach.

  See the full result →Stays fast and sound as it grows →

• Handles new wording, and spots the look-alikes.

  7.5%*Wrong answers on look-alike requestsA cache 45.3%, about 6× more

  MS Duplex7.5%

  A cache45.3%

  0Lower is better100%

  Wrong answers on look-alike requests

  MS Duplex	7.5%
  A cache	45.3%

  • 98.98% Of requests handled correctly: answered right, or handed to your AI · MS Duplex answered 95.78% of known requests itself*
  • 0.996 Score for telling familiar from new, out of 1 · A cache 0.930*

  Ten learned intents and six levels of novelty, from exact repeats to requests from another domain, over ten runs.

  See the full result →Handles cases it has never seen →

• Plans the shortest route, with zero model tokens.

  100%*Plan success, on every runA reactive agent: 0% · MS Duplex used no model tokens

  MS Duplex100%

  A reactive agent0%

  0100%

  Plan success, on every run

  MS Duplex	100%
  A reactive agent	0%

  • 0% A reactive greedy agent*
  • ~204 Tokens per decision a frontier model spends to match it*

  A multi-step world over ten runs, against a reactive agent and a frontier model given the same task.

  See the full result →Sees what an action will do →

• Keeps agent tasks succeeding, with fewer model calls.

  83.3%*Task successThe model on every turn: 82.5%

  MS Duplex83.3%

  The model on every turn82.5%

  0100%

  Task success

  MS Duplex	83.3%
  The model on every turn	82.5%

  • 12.8% Turns completed without the model · ≈15% fewer model calls*
  • 100% Success on new tasks*

  Τ-bench (tau2-bench), retail, by Sierra Research. The unmodified retail environment, with the same model, tools and user simulator for every approach, over three runs of forty episodes.

  See the full result →Runs model and tool calls under your rules →

• A steady reading cost, however long the history.

  96%*Current answers where a fact had changedThe new fact, not the old one0100%
  • 3.7× Less reading per question than reading the whole history · At forty sessions of history*
  • 84% Accuracy at forty sessions of history*

  LongMemEval, by Wu et al., ICLR 2025. Fifty questions, each asked with the history at 2, 5, 10, 20 and 40 sessions.

  See the full result →Brings the right history to each step →

• Blocks every attack, and lets ordinary work through.

  0%*Attack successAcross six families of paraphrased attacks
  • 0.7% False refusals · The lowest of any approach that blocked every attack*
  • 40 / 40 Poisoned samples removed · Accuracy back within 1 point*

  Six attack families, paraphrased, with harmless requests written to resemble the attacks.

  See the full result →Your rules come first →

• Learns a new skill on its own, with the model switched off.

  100%*Tasks completed with the model off, on every runA frontier model needed ≈507 calls per run
  • 0.26 ms To reuse a skill · At 0 model calls*
  • 627 Unsafe actions refused per run · And the safe skill was still learned*

  Four suites over ten runs, each scored by its worst run.

  See the full result →Learns new skills on its own →

• Beat three frontier models at predicting system changes.

  0.256*System-change prediction scoreGPT-5.1 0.166 · Gemini 3 Pro 0.205 · Claude Sonnet 4.5 0.204

  MS Duplex0.256

  Gemini 3 Pro0.205

  Claude Sonnet 4.50.204

  GPT-5.10.166

  01

  System-change prediction score

  MS Duplex	0.256
  Gemini 3 Pro	0.205
  Claude Sonnet 4.5	0.204
  GPT-5.1	0.166

  • $0 Model spend · 570 predictions at zero calls; 84 on a frontier model cost $13.40*
  • ~80% Decisions answered from what it learned · Your AI took the rest*

  World of Workflows, by Skyfall Research / ServiceNow. Three offline tracks, scored by the benchmark authors’ own evaluator, over three runs.

  See the full result →Sees what an action will do →

• Beats fine-tuning, with no training run.

  6 / 6*Tests where MS Duplex beat a fine-tuned adapter+30.3 points on average
  • 0 Retraining to move to another model family · The adapters moved in 0 of 6*
  • 60 to 79% Answered from what it learned · On either model family*

  Three public intent datasets, two open model families and three runs, with the same frozen model and data for every approach.

  See the full result →Adapts your model without retraining →