Skip to content
Advancing Bits to Master Atoms
← Every post

Guides8 min read

How to evaluate an AI materials discovery engine: ten questions to ask.

A vendor-neutral checklist for R&D leads and technical buyers: what to ask, what a good answer looks like and how MS Lattice answers from its record.

A compact crystal of silver and cobalt-blue spheres held between two plain brushed-silver gauge blocks by a cobalt-blue datum bar.A compact crystal of silver and cobalt-blue spheres held between two plain brushed-silver gauge blocks by a cobalt-blue datum bar.

Every AI materials discovery vendor has a demo, and the demo always works. What your team needs to know is different: whether the engine will work on your target, with your limits, at a price you can plan for, and whether you can check its answers without taking anyone’s word for them.

This guide is a checklist for that conversation, written for use with any vendor, MatterSpace included. Each question comes with what a good answer looks like. After the list, we show how MS Lattice answers from its own record, so you can hold everyone, us included, to the same bar.

Why a checklist beats a demo

AI materials discovery covers very different kinds of products. Some generate structures and screen them afterwards. Some make screening fast. Some give experts deep tools to drive by hand. Some learn from a company’s past lab results, and some pair AI with labs of their own on a partnership timeline. Each approach has moved the field forward. They are not interchangeable, and a demo on a hand-picked material tells you little about which one fits your program.

The cost of choosing badly rarely shows up in the software bill. It shows up in the lab, where every candidate takes synthesis time, characterization time and your people’s attention. A good evaluation comes down to one thing: how much of your lab’s time will go to candidates worth making.

The ten questions

Put these to every vendor on your list. Ask for evidence, not adjectives: what was measured, against what bar, set when. A good answer gives a figure with its conditions and points to a record your team can inspect.

  1. Has it found answers it was never shown?The one discovery test that cannot be faked is blind: the target is sealed away while the engine searches and ranks, the candidates are frozen, and only then is the answer revealed. A good answer includes a leak check showing the result was found, not remembered.
  2. Can it get the structure right from a formula alone?Many properties depend on how the atoms are arranged, not only on which atoms are present. Ask how often the engine ranks the correct crystal structure first when given nothing but a chemical formula, against a bar set before the run, and how it does on compounds held out of everything it starts from.
  3. Are its verdicts checked by physics?Fast estimates make a search affordable, but they are not the last word. A good answer offers first-principles verification for the candidates that matter, shows stability calls tested against arrangements known to be wrong, and shows reference properties reproduced within a tolerance declared in advance.
  4. Can you rebuild any candidate from its record?A score is a claim. Ask whether every candidate carries the record of every check it passed and where any stopped, whether that record is enough to rebuild the candidate, and whether anyone has audited that at production scale.
  5. Are your requirements held inside the search?If your limits are applied as a filter after generation, you pay for every structure that breaks them. A good answer says proposals that break your must-haves are turned away before a structure is built, and that nice-to-haves are pushed on top of must-haves, never traded against them.
  6. How long is the shortlist your lab receives?Millions of candidates make a headline; your lab makes a handful at a time. Ask what a typical campaign hands over, how it is ranked, and whether the candidates are deliberately different from one another or one family of near-copies.
  7. What does a candidate cost, and when do you see the price?Ask for the compute cost per candidate on a recorded run, not a list price on request. A good answer shows the price of every plan before launch and keeps every campaign inside a budget you set.
  8. Can your own AI agent drive it?Ask whether an agent can run the whole workflow, from stating the need to reading back the proof, over a standard connection such as MCP, and whether an API and SDK reach the same engine.
  9. What happens after the shortlist?A material is not discovered until it is made. Ask what the vendor hands your lab, in what file format, and whether there is a route on to synthesis, testing, production and finished parts if you want one.
  10. What do the results not claim?Every engine’s figures are screening estimates of some kind. A good vendor says, domain by domain, which properties are estimated, which are not predicted at all and what has to be tested before anyone relies on a candidate. Be wary of an answer with no limits in it.

How MS Lattice answers on proof

The first three questions ask whether an engine can find things. Here is MS Lattice’s record, as it stands.

  • 2 of 2sealed targets found blind
  • 13 of 15right structures ranked first, from a formula alone
  • 4 of 4first-principles stability verdicts right
  • 5 of 5silicon reference values reproduced

In the Blind Rediscovery Benchmark, two ultra-high-temperature ceramics stayed sealed while MS Lattice searched and ranked. The targets steered nothing and ranked nothing. When the candidates were frozen and the targets revealed, both had been found, down to the arrangement of their atoms: 64 candidates in two rounds, about 32 minutes of compute, under $0.20. Every match level passed for both targets, a full re-run passed 9 of 9 checks, and a leak check confirmed the match was found, not remembered.

Given nothing but a chemical formula, MS Lattice ranked the correct crystal structure first for 13 of 15 compounds, against a bar of 10 set before the run, in about 38 seconds per compound. Four compounds held out of everything it starts from, CaO, SrF₂, BaTiO₃ and Ni₃Al, were rebuilt to within 0.016 to 0.037 Å of the reference structures, and three of them were ranked first.

First-principles physics is on call for the candidates you choose. Its stability check called Mg₂Si and MoSi₂ stable, 0.165 and 0.284 eV/atom below their competing phases, and called deliberately wrong arrangements of each unstable: four verdicts out of four. It also reproduced silicon’s lattice constant, its three elastic constants and its highest optical phonon frequency, each within the tolerance set before the run.

From one node on the left, one track scatters into hundreds of tiny muted slate cubes while a straight cobalt-blue track leads directly to one complete crystal of satin silver and cobalt-blue spheres.From one node on the left, one track scatters into hundreds of tiny muted slate cubes while a straight cobalt-blue track leads directly to one complete crystal of satin silver and cobalt-blue spheres.
Tweezers lift one blue crystal out of a heap of gray sand: the few candidates worth making, picked from the many that are not.

How MS Lattice answers on evidence and requirements

Every MS Lattice candidate carries the record of every check: what it passed, what it failed and where it stopped, with the reason, alongside the uncertainty on every estimate and its stability against every rival phase of its chemistry. Each candidate is also checked against the known crystals, so you see what is new. From three production-size campaigns, all 73 audited candidates were rebuilt from that record, every property within its declared tolerance.

Your requirements are held inside the search. A proposal that breaks them is turned away before a structure is built, and your preferences are pushed as far as they go on top of them, never at their expense. That is also why the shortlist is short: hundreds of ranked contenders, deliberately different from one another, instead of millions of structures to triage. In the blind benchmark, 64 candidates were enough to find both sealed targets.

How MS Lattice answers on cost, agents and parts

Compute goes only to candidates that can meet your limits. On the record, that comes to about a third of a cent, ≈ $0.003, and about 30 seconds of compute per candidate in the blind benchmark. An AI agent’s complete 8-candidate campaign cost under $0.003. Every plan shows its estimated cost before launch, a plan that cannot run is refused with the reason before anything starts, and every campaign runs inside the budget you set.

MS Lattice is the engine; your AI agent is the driver. One MCP connection gives any agent that speaks MCP 120 tools and the whole chain, from describing the need to reading back the proof. Before every release, an AI agent acting as a customer runs that complete workflow, an 8-candidate campaign in 4 to 5 minutes. The same engine is reachable through the REST API, the Python SDK and the command line.

After the search, your lab receives a ranked shortlist with a standard CIF structure file for every candidate, and the whole campaign can be exported as CSV or JSON. If you want to go further, Norck runs MS Lattice, then synthesizes, validates and manufactures the winner, with scopes from discovery alone to end to end, finished parts included.

What the results do not claim

The tenth question matters most, so here is MS Lattice’s answer in full. Its figures are screening estimates and calculations for the bulk crystal, not measurements of a new material. Each of its twenty-three material domains states what you specify, what you receive and where its estimates stop. For magnets, for example, coercivity in a finished magnet depends on microstructure and processing, so a grade has to be made and measured before it goes into a motor.

Questions teams ask

Should we run our own blind test before we buy?

It is the strongest evidence you can get. With MS Lattice, the Blind Rediscovery Benchmark is one of the four modes, so your team can seal targets it already knows and see whether they come back.

How do we compare prices across vendors?

Ask each for the compute cost per candidate on a recorded run and for the price of a specific plan before launch. MS Lattice’s record puts it at about a third of a cent per candidate, and every plan shows its price before it runs.

What should a pilot hand over?

A pilot runs MS Lattice on your own target and hands you a ranked shortlist, each candidate as a structure file with its record, ready for your lab.

Does our work stay ours?

Ask every vendor. With MS Lattice, your campaigns and the data you bring stay within your organization, and before a pilot starts we agree with you what is shared, where it is processed and how long it is kept.

MatterSpace’s first domain engine

MatterSpace built MS Lattice for the scientists and engineers who discover and design new materials, starting from the properties their product needs.

Keep reading

Have a materials target? Tell us what it must do.

Share the material domain, the properties you need and the limits the material must respect. We will show you how MS Lattice would approach it through Norck, from a ranked shortlist for your own lab all the way to finished parts.