New Tool Degrades Images to Train AI Systems and Improve Microscopy

Scientists at the Salk Institute report that they have developed a way to make high-resolution microscopic imaging more accessible. The new tool, which the team calls a "crappifier," could make it significantly easier for scientists to get detailed images of cells or cellular structures that have previously been difficult to observe because they require low-light conditions.

"We invest millions of dollars in these microscopes, and we're still struggling to push the limits of what they can do," says Uri Manor, senior author on the paper published today in Nature Methods. "That's the problem we were trying to solve with deep learning."

However, to use deep learning to improve microscope images—either by improving the resolution (sharpness) or reducing background "noise"—the system needs to be shown many examples of both high- and low-resolution images. That's a problem, because capturing perfectly identical microscopy images in two separate exposures can be difficult and expensive. It's especially challenging when imaging living cells that might be moving around during the process.

That's where the crappifier comes in. According to Manor, the method takes high-quality images and computationally degrades them, so that they look something like the lowest low-resolution images the team would acquire. Manor's team showed high-resolution images and their degraded counterparts to the deep learning software, called Point-Scanning Super-Resolution (PSSR). After studying the degraded images, the system was able to learn how to improve images that were naturally poor quality. That's significant because, in the past, computer systems that learned on artificially degraded data still struggled when presented with raw data from the real world.

"We tried a bunch of different degradation methods, and we found one that actually works," Manor says. "You can train a model on your artificially generated data, and it actually works on real-world data."

"Using our method, people can benefit from this powerful, deep learning technology without investing a lot of time or resources," says Linjing Fang, lead author on the paper. "You can use pre-existing high-quality data, degrade it, and train a model to improve the quality of a lower-resolution image."

The team showed that PSSR works in both electron microscopy and with fluorescence live cell images—two situations where it can be extraordinarily difficult or impossible to obtain the duplicate high- and low-resolution images needed to train AI systems. While the study demonstrated the method on images of brain tissue, Manor hopes it could be applied to other systems of the body in the future.