Preclinical imaging is already a powerful research and diagnostic tool. Now studies using CT, MRI, PET, and optical imaging technologies among others, are playing an important role in drug discovery and development. Preclinical imaging can save the pharmaceutical industry time and money by assessing the effects of drug candidates earlier in the drug pipeline— ultimately helping to get new drugs to patients faster. Applying artificial intelligence to image data analysis is advancing the field, allowing the development of innovative solutions where biology, physics, and computer science intersect.
Machine learning meets biology
The use of machine learning in image data analysis and visualization is rapidly advancing the value of preclinical imaging in drug development. Machine learning is a branch of artificial intelligence (AI) in which algorithms can “learn” from data by identifying patterns, using structured data sets to modify themselves in order to achieve a certain output. (For a popular culture example, see the 1983 film “WarGames,” in which a computer controlling the world’s nuclear arsenal repeatedly played simple tic-tac-toe games against itself until it “learned” that nuclear war is always a “no-win” situation.) A newer subset of machine learning, deep learning, uses a network of multiple algorithms that may interpret the data in different ways—this is also known as an artificial neural network.
Last year, Charles River Labs began a machine learning project “focused on lesion segmentation in the anatomical MRI data from transient medial cerebral artery occlusion model of ischemic stroke in rat,” says Artem Shatillo, head of MRI at Charles River. Starting with imaging data that contained manually delineated stroke lesions, they used a deep learning tool to teach a fully convolutional network algorithm to detect borders of ischemic areas, as well as other features.
Before the advent of machine learning, they maximized automation of the routine imaging data processing pipelines for tasks like converting, exporting, grouping, and quantifying data, so that imaging data can be batch-processed on a dedicated server with no user input. But all other data analysis tasks still required considerable amounts of manual work. With new machine learning tools, it is possible to minimize human errors and bias, and speed up the process by several orders of magnitude, according to Shatillo. “Software tools like this are dramatically accelerating data analysis, while also increasing consistency and quality of the outcome compared to manual segmentation,” he says. “In turn, this can make a difference between ‘go’ and ‘no go’ decisions for the drug candidate.”
“I believe machine learning will become an industry standard in digital imaging analysis very quickly, and we need to be prepared for that,” says Shatillo. “At Charles River, we are planning to validate and implement similar analysis tools for multiple imaging modalities.” Next they plan to use machine learning for MRI volumetry data analysis, which measures absolute volumes of brain structures, for example, to detect cortical atrophy in neurodegenerative disease models.
Machine learning and imaging mass cytometry
Trevor McKee, image analysis manager at STTARR Innovation Centre for Advanced Preclinical Imaging and Radiation Research, uses imaging mass cytometry (IMC)— an imaging method relying on heavy metal tagged antibodies—to detect locations of proteins within tissue sections. The method has an advantage over commonly used fluorescence based methods in that it can multiplex far more proteins.
IMC is particularly valuable in immuno-oncology, where a patient’s own immune system is prompted to attack cancer cells. IMC can show which subtypes of immune cells are present in tumors, for example, as well as their spatial localization within tumor biopsies. “This technique, alongside complementary methods like immuno-PET for tracking immune cells non-invasively, can provide the information needed to help refine and improve immune-targeted agents in clinical trials,” says McKee. The group is analyzing immune cell localization in several clinical trials to help identify the expected changes in patient biopsies. “This could be quite critical in helping a pharmaceutical company know whether a treatment is working,” says McKee.
McKee’s group uses machine learning methods for image segmentation and classification, reducing time spent on manual identification of features. Machine learning is especially helpful in classifying cells. Using manually labeled images as standards, and other features extracted from cell images, McKee’s group “taught” a machine learning classification algorithm how to predict the cell type correctly. “Current machine learning methods are never 100% correct, so we still have to do some manual correction, but if it identifies the tumor correct about 80% of the time, that still results in a significant time savings over a full manual approach,” he says.
Advances in image analysis speed small animal imaging
InVivo Analytics has developed a fully automated platform, InVivoAX, for fast cloud-based data analysis of small animal imaging. Combining whole animal imaging data sets is difficult due to differences in animal sizes and physical positions. InVivoAX solves these problems for mouse studies with two key tools: the body-conforming animal mold (BCAM) and the digital mouse atlas. The BCAM, which is available in a range of sizes, holds the animal in a precise position. The mouse atlas, or organ probability map, is digitally embedded within the BCAM, which contains 300,000 data points that are each associated with a statistical probability of an organ.1
When performed on a mouse expressing a bioluminescent or fluorescent reporter, InVivoAX returns a bioluminescent or fluorescent tomographic map that shows the location (to within 0.5–1 mm) of the reporter within organs of the body. Thus it can be used to follow the path of bacteria during a septic event, viral infection during viral delivery, the migration of T cells or stem cells, or the path of cancer cells during metastasis.
InVivoAX’s automated assignment of regions of interest (ROI) helps minimize variability. “Every person drawing an ROI is going to have a different cognitive bias, which this removes,” says Neal Paragas, co-founder and scientific advisor at InVivo Analytics. “The power of automated image analysis is that it removes the user from drawing a different ROI for each animal. And in our system, one ROI can be applied to the whole cohort or project automatically.”
With the BCAM and digital atlas now able to provide reliable whole animal imaging data, they are looking forward to helping collaborators in different locales to combine image data sets. “Our system calibrates everyone’s data so that it’s normalized, so collaborating labs can share a login and upload shared data to the cloud,” adds Paragas. “We are also expanding InVivoAX for nuclear imaging such as PET and SPECT, and magnetic particle imaging.”
AI for image analysis in high content screening
While 3-dimensional cell cultures are more physiologically relevant disease models, their greater complexity means that the datasets are also larger and more difficult to analyze compared to traditional 2-dimensional cell cultures. Revvity is embracing the challenge using AI to advance data analysis in their imaging software, such as Harmony® (software for high content imaging and analysis), Columbus™ (software for image data storage and analysis), and TIBCO® Spotfire® based Signals™ Screening (software for analysis and visualization).
Revvity uses AI-based methods to automate analysis tasks like identifying sets of parameters, which helps to optimize segmentation and classification of data. “Complex image analysis tasks such as 3D segmentation, quantification of morphology and texture, and phenotypic classification, are important for analysis of more biologically relevant models like spheroids and organoids,” says Martin Daffertshofer, high content screening software product manager at Revvity.
Image: PreciScan Identification of xy and z positions of cysts in gel. Cysts marked (in green) for high resolution re-scan to observe individual cells. Image courtesy of Revvity
Revvity’s Harmony and Columbus software both offer tools for texture and morphology-based analysis, which derives thousands of parameters from image data, then “classifies this information into disease relevant phenotypes,” says Daffertshofer. “When screening tens of thousands of compounds, this approach can be used to rapidly identify those that return a disease phenotype to a normal one.”
Revvity is exploring the use of deep learning methods to improve analysis of image data from high content screening—in some cases, solving problems that conventional algorithms can’t. For example, segmentation with conventional algorithms is difficult when cells are very dense and possibly overlapping in layers, but deep learning is better able to handle this segmentation.
“As a result, scientists could earn back valuable assay development time without compromising the assay modalities,” says Daffertshofer. “Even staining-free assays acquired as brightfield images could be automatically segmented and productively used in high content screening.” Revvity’s Signals Screening can classify cellular samples even without segmentation, using a training data set of control wells.
Daffertshofer believes that deep learning development will soon shift from software developers toward biologists. “Instead of writing code, the training of deep neural networks—the engine for deep learning—will drive the innovation of image analysis software,” he says. “Scientists in academia or the pharmaceutical industry may then be able to help develop image analysis best practices without needing in-depth instrument or coding expertise.”
References
1. Klose, A.D. and Paragas, N. Automated Quantification of Bioluminescence Images. Nature Communications vol. 9, Article number: 4262 (2018)