Analysis Calls for Greater Transparency in Protein Design AI

Protein language models—artificial intelligence tools used to engineer proteins with useful properties, including entirely new structures never seen in nature—hold considerable potential for addressing global challenges. These include synthesizing enzymes capable of absorbing carbon dioxide from the atmosphere and building catalysts that reduce energy use or toxic waste in industrial processes. But as these models increasingly shape real-world decisions in biotechnology, a significant problem persists: they largely operate as black boxes, making it difficult to understand their decision-making process or judge whether their predictions are reliable, biased, or safe. 

A perspective paper published in Nature Machine Intelligence by researchers at the Centre for Genomic Regulation (CRG) examines how explainable AI—the techniques and methods that allow humans to understand, trust, and interpret the decisions of these tools—is currently being applied to protein language models. It is described as the most comprehensive survey of its kind to date. 

"Protein language models are moving fast but our understanding of fundamental biological processes such as folding or catalysis has not advanced alongside these breakthroughs," said Dr. Noelia Ferruz, corresponding author of the paper. "Without better ways to explain what these models learn and how they make decisions, we risk building powerful tools that we cannot fully trust."

The authors identify four points along a model's decision-making process where explanation can be applied: the training data the model learned from; the specific protein sequence given to the model; the architecture and internal components of the model itself; and what the authors call input-output behavior, which involves observing how a model's output changes when the input is slightly altered.

Reviewing existing scientific literature, the team found that explainability is most commonly used as an "Evaluator"—a way to check whether a model has learned patterns that biologists already recognize, such as binding sites or structural motifs. A smaller number of studies use these insights as a "Multitasker," applying learned signals to annotate new proteins or predict additional properties. Fewer still use explainability as an "Engineer" or "Coach" to redesign model architecture or steer protein sequence generation. 

The most ambitious role—the "Teacher"—remains largely unrealized. This is where AI would reveal entirely new biological principles not previously recognized by humans. "For us, the real holy grail is controllable protein design," said Dr. Ferruz. "Imagine being able to tell a model: 'Design a protein with this shape, active at this pH,' and not only receive a candidate sequence, but also a clear explanation of why that design should work, and importantly, why alternatives would fail."

The authors call on the research community to develop robust benchmarks, open-source tools, and laboratory validation processes to move the field in that direction. "If we want protein language models to become a reliable partner in discovery and design, explainability must not be an afterthought," said Andrea Hunklinger, first author of the paper.