Deep Learning Strategy Used to Improve Protein Design

An MIT researcher has combined attention neural networks with graph neural networks to better understand and design proteins. The approach, described in the Journal of Applied Physics, couples the strengths of geometric deep learning with those of language models not only to predict existing protein properties but also to envision new proteins that nature has not yet devised.

“With this new method, we can utilize all that nature has invented as a knowledge basis by modeling the underlying principles,” explained Markus Buehler author of the paper. “The model recombines these natural building blocks to achieve new functions and solve these types of tasks.”

Due to their complex structures, ability to multitask, and tendency to change shape when dissolved, proteins have been notoriously difficult to model. Machine learning has demonstrated the ability to translate the nanoscale forces governing protein behavior into working frameworks describing their function. However, going the other way—turning a desired function into a protein structure—remains a challenge.

To overcome this challenge, Buehler’s model turns numbers, descriptions, tasks, and other elements into symbols for his neural networks to use.

He first trained his model to predict the sequencing, solubility, and amino acid building blocks of different proteins from their functions. He then taught it to get creative and generate brand new structures after receiving initial parameters for a new protein’s function.

The approach allowed him to create solid versions of antimicrobial proteins that previously had to be dissolved in water. In another example, his team took a naturally occurring silk protein and evolved it into various new forms, including giving it a helix shape for more elasticity or a pleated structure for additional toughness.

The model performed many of the central tasks of designing new proteins, but Buehler said the approach can incorporate even more inputs for more tasks, potentially making it even more powerful.

“A big surprise element was that the model performed exceptionally well even though it was developed to be able to solve multiple tasks. This is likely because the model learns more by considering diverse tasks,” he said. “This change means that rather than creating specialized models for specific tasks, researchers can now think broadly in terms of multitask and multimodal models.”