Review in Chemical Society Reviews Maps Where AI Adds Value to Chemical Research, and Where It Falls Short

High accuracy on test data is only the tip of the iceberg.
Alexander Ryabov of Skoltech on why benchmark scores do not settle whether AI helps in the lab.
Mark

Why does this review matter if AI in chemistry is already everywhere?

Mimi

Because it changes the yardstick. The authors organize things around the decisions a chemist makes, like which experiment to run next, rather than around model scores.

Mark

What's the main warning?

Mimi

That a strong benchmark result may not hold in a lab. They point to leakage between training and evaluation, overfitting to limited examples, and errors when a model goes beyond familiar data.

Luke

Worth noting that this is a review, not a new experiment. The summary doesn't give us a measured case where a benchmark-winning model failed in a lab.

Mimi

Fair. It describes recurring failure modes and representative applications, but the source gives no specific numbers.

Mark

What do they advise a lab to check before using a model?

Mimi

Whether the training data match its compounds and instruments, how well the model estimates its own uncertainty, and whether it was tested on new experiments with results unknown during training.

Luke

And the future vision, LLMs combined with physics-informed models, is stated as what comes next. That's the authors' expectation, not a result.

Mark

Where does the human fit?

Mimi

Medvedev says the researcher still defines the scientific question and verifies the agent's output.

Luke

Also, the authors are Skoltech and Zelinsky Institute researchers describing their own field, so the framing is theirs.

  • Benchmark scores look impressive, yet the authors warn they may collapse the moment a model meets a laboratory's unfamiliar compounds and instruments.
  • Data leakage, overfitting and extrapolation errors threaten to inflate results, and every mistake costs expensive experimental time.
  • The review maps AI methods onto each stage of chemical research, from literature mining and spectral interpretation to synthesis planning.
  • Researchers are steering toward closed loops where automated experiments feed results back to models, while humans keep responsibility for the question and its verification.
  • The call is to judge chemistry foundation models and automated platforms across full research cycles rather than on isolated test accuracy.

In a review appearing in Chemical Society Reviews, researchers from Skoltech, the Zelinsky Institute and ITMO University ask a question that has quietly grown urgent across the sciences: where does artificial intelligence truly serve the person at the bench, and how can anyone know the difference between a convincing number and a useful result? They reorganize the field around the sequence of decisions a chemist makes, from choosing which evidence to trust to deciding which experiment to run next, and they argue that a model's worth lies in whether it improves those choices under the stubborn constraints of real laboratories. Their caution is equally deliberate: the impressive scores of a model on familiar data can conceal leakage, overfitting and fragile extrapolation, and chemistry, where every compound and instrument differs, is unforgiving of such illusions.

A review in Chemical Society Reviews, one of chemistry's leading review journals, asks a plain question: where does artificial intelligence actually help a working chemist, and how can anyone tell? The paper comes from the Skoltech AI Center and the Zelinsky Institute of Organic Chemistry of the Russian Academy of Sciences, with colleagues from ITMO University.

The authors build their framework around the decisions a chemist makes rather than the algorithms themselves: which evidence to examine, which experiment to run next, and how to revise the plan when a new result arrives. Across the stages of research, from literature and spectra to property prediction, synthesis planning and experiment-guided loops, they map where AI can draw mechanistic insight, identify promising compounds and conditions, and link predictions to automated experiments.

The cautionary half of the paper examines how success gets overstated through leakage between training and test data, overfitting to few examples, and errors when models extrapolate. Before trusting a model, they advise checking whether its training data match a specific laboratory, how well it estimates its own uncertainty, and whether it has been tested on experiments unknown during training.

Alexander Ryabov of the Skoltech AI Center put it simply: high test accuracy is only the tip of the iceberg. Alexander's colleague Michael Medvedev, head of the Theoretical Chemistry Group at the Zelinsky Institute and first author, looks ahead to advanced language models combined with physics-informed models, agents that must judge when a prediction is well founded and which experiment would most reduce uncertainty. Even then, he stresses, the researcher defines the question and verifies the result.

The authors conclude that such foundation models and intelligent experimental platforms should be judged across the full cycle, from the first question to a reproducible result.

A review published in Chemical Society Reviews, one of the leading review journals in chemistry and a title of the UK-based Royal Society of Chemistry, sets out to answer a plain question: where does artificial intelligence actually help a working chemist, and how can anyone tell? The paper comes from researchers at the Skoltech AI Center and the Zelinsky Institute of Organic Chemistry of the Russian Academy of Sciences, with colleagues from ITMO University and other institutions.

The authors organize the review around the decisions a chemist makes rather than around the algorithms. Chemical research generates scientific publications, images, spectra, measurements and reaction records, and understanding how molecular structure relates to properties remains central. Along the way, researchers choose which evidence to examine, which experiment to run next, and how to change the plan once a new result arrives. The review treats that sequence as its framework.

Within it, the authors divide research into stages that generate, transform and validate information, then map AI methods onto each. They cover classical machine learning, models that build in chemical knowledge, and emerging foundation models for chemistry. The territory spans extracting knowledge from the literature and accumulated datasets; interpreting data from spectroscopy, mass spectrometry and microscopy; predicting properties and reactivity; planning syntheses; and systems in which one experiment's outcome guides the next. Representative applications show what transfers to a new problem and where a method's limits matter. The emphasis falls on measurable outcomes and recurring failure modes, not on benchmark scores alone.

In their conclusion, the authors name three ways AI can change a chemist's work. It can draw mechanistic insight from complex spectra, images and scientific texts. It can pick out promising compounds, synthesis routes and experimental conditions from among many options. And it can link predictions to automated experiments whose results steer the next step. One figure uses property prediction to show which steps between a molecular description and an experiment different models can handle. The practical value of AI, they argue, depends on whether it improves decisions under real research constraints.

The cautionary half of the paper concerns how success can be overstated. The authors examine information leaking between training and evaluation, overfitting to a limited set of examples, and errors when a model extrapolates beyond familiar data. These problems weigh heavily in chemistry, where substances, reaction conditions and instruments vary, and where experimental validation takes time and resources. Before adopting a model, they advise checking whether its training data match a particular laboratory's compounds and instruments, how reliably it estimates its own prediction uncertainty, and whether it has been tested on new experiments whose results were unknown during training.

Alexander Ryabov, Senior Research Scientist at the Skoltech AI Center, described the aim as viewing AI from the standpoint of a chemist who must make decisions. High accuracy on test data, he said, is only the tip of the iceberg; what matters more is whether a model helps interpret measurements, plan informative experiments, and keep predictions reliable on new compounds and reactions.

Michael G. Medvedev, head of the Theoretical Chemistry Group at the Zelinsky Institute and first author, looked ahead to a combination of advanced large language models and physics-informed models that respect chemical constraints. An AI agent, he said, must judge when a physics-informed prediction is well founded, when it goes beyond its training data, and which experiment would best reduce uncertainty. The researcher remains responsible for defining the scientific question and verifying what the agent produces.

The Skoltech co-authors include Ryabov; Vladimir Vanovskiy, Head of Hybrid Modeling; and RAS Professor Evgeny Burnaev, director of the AI Center. Zelinsky Institute co-authors include Medvedev and RAS Academician Valentine Ananikov. Burnaev, Medvedev and Ananikov are Sber Science Prize laureates. Looking forward, the authors argue that chemistry-grounded foundation models and intelligent experimental platforms should be evaluated across full research cycles, from the initial question to a reproducible result.

A model's high accuracy on test data is only the tip of the iceberg; what matters is whether it helps interpret measurements and plan informative experiments.
— Alexander Ryabov, Skoltech AI Center
The researcher remains responsible for defining the scientific question and verifying the results produced by the AI agent.
— Michael G. Medvedev, Zelinsky Institute
Contáctanos FAQ