How to Generate More Reproducible Flow Cytometry Data

In any assay, each step of the experimental workflow has the potential to introduce variation. Such variation can cast doubt on the soundness of results within a lab and makes comparing data between different labs especially challenging. This article highlights common sources of variation in flow cytometry and suggests how these can be addressed to generate more reproducible data.

Multiple sources of variation

According to Diana L. Bonilla Escobar, Ph.D., SCYM(ASCP)^CM, US Applications Lead at Cytek Biosciences, sources of variation for flow cytometry span the entire experimental workflow—from sample collection, storage, and preparation, through marker/antibody/fluorophore selection, and on to staining protocol, instrument setup, sample acquisition, and data analysis. “Even if two researchers use the same sample, same reagents, same instrument, and follow the same sample preparation and staining protocols, their results and ultimate analysis of the data can still differ,” she reports. “Inevitably, such variation will be more pronounced where experiments are conducted by different laboratories. For example, reagents will come from different vials, even if they share the same lot number, and flow cytometers will exhibit some variability in terms of their different hardware components.”

Mike Blundell, Ph.D., Product Manager at Bio-Rad, agrees, commenting that even a seemingly inconsequential protocol alteration can have a significant impact on data quality. “Changing just simple things such as an incubation time or temperature, or the carrier in an antibody diluent, may lead to markedly different staining intensities,” he says. “Variation can also come from choosing a different fluorophore for a particular marker, since every additional dye will have an effect on each dye already included in the antibody panel. Researchers should likewise bear in mind that because antibody vendors have different conjugation methods and QC procedures, the same clone purchased from multiple suppliers can vary in terms of brightness.” Critically, since one of the main sources of variation in any flow cytometry experiment is the end user, best practices should be adopted wherever possible.

Common sources of variation for flow cytometry

• Sample material—source, collection procedure, shipping method, storage, preparation, donor-to-donor variability
• Antibodies—clone, method used for fluorophore conjugation/QC, storage buffer
• Staining protocol—sample volume, cell number, incubation times/temperatures, buffers, reagent lot number, reagent storage
• Panel design—marker/antibody/fluorophore selection
• Instrumentation—hardware configuration (laser power, filter bandwidths, noise characteristics of any electronics), sensitivity, dynamic range
• Instrument setup—flow rate, laser power, PMT voltages, acquisition/gating strategies
• End user—sample handling/acquisition/data analysis habits, ability to comply with standard operating procedures (SOPs)

Global standardization efforts

The need for greater reproducibility in flow cytometry has driven standardization efforts by several large consortia. Castle Funatake, Ph.D., Senior Manager, Research and Development, Protein and Cell Analysis, Thermo Fisher Scientific, notes that these include the development of guidelines for enumerating CD34+ progenitor cells (Sutherland et al., 1996), the creation of antibody panels for classifying hematological malignancies (van Dongen et al., 2012), and the production of gating strategies for intracellular cytokine staining (McNeil et al., 2013). “Furthermore, Cossarizza et al., the EuroFlow Consortium, and the Clinical and Laboratory Standards Institute have brought together experts in flow cytometry, immunology, and biotechnology to generate approaches for harmonizing and standardizing assays and protocols for flow cytometry,” she says.

Also highlighting the work of these groups, Escobar explains that such initiatives aim to reduce: 1) technical variation in sample preparation and staining by using SOPs and consistency in panels and reagents, 2) instrument variation by using the same hard-dye beads and target values, 3) variation in manual data analysis by using unsupervised automated tools, and 4) variation in data interpretation by building a database of clinical cases to compare against unknown samples. “These guidelines are readily applied in-house,” she says. “For instance, the presence of batch effects can be evaluated using unsupervised analysis, by identifying shifts in lineage populations upon clustering or dimensionality reduction or including a pre-characterized biological control sample in each run to evaluate staining and acquisition consistency. Once batches have been identified, normalization algorithms or changes in the workflow can be implemented.”

Liselotte Brix, CSO at Immudex, reports that reproducibility can additionally be enhanced by comparing assay performance with that of other labs. “At Immudex, we offer proficiency testing as a non-profit service to those laboratories wishing to evaluate their immune monitoring performance,” she says. “Essentially, all participants receive a set of identical PBMC samples, in which they quantify antigen-specific T cells using laboratory-specific flow cytometry protocols and either their own MHC multimer reagents or Immudex MHC Dextramer^® reagents. They then send their results back to us and we provide an anonymized report. In this way, participants can benchmark themselves against other laboratories around the world to align assay performance across test sites and identify staff training or protocol optimization needs.”

Safeguarding reproducibility

Aside from these global initiatives, countless other solutions have been developed to safeguard the reproducibility of flow cytometry data. Brix notes that commercially available options include standardized flow cytometry reagents, kits with low lot-to-lot variation, and QC products for instrument standardization and performance. “Data analysis is also increasingly standardized, either through the use of dedicated software or comprehensive description of gating strategies,” she reports.

Kivin Jacobsen, Senior Scientist at Immudex, adds that best practices at the bench are often focused on preserving sample quality and validating assay performance. “Sample-preparation methods should maintain viability and it is advised that researchers use live-dead staining reagents to gate out unwanted cells,” he says. “Other recommendations include acquiring enough cells to ensure statistical robustness; using proper positive and negative controls, both for the cells being investigated and to assess the performance of individual reagents; and monitoring assay performance by tracking specificity, sensitivity, and intra- and inter-assay variation over time.”

Tips for generating more reproducible flow cytometry data

• Clearly define the source and types of samples that will be used
• Establish how samples will be shipped and/or stored
• Optimize the sample-preparation method to ensure high viability and minimal cell debris
• Optimize the staining protocol—consider factors such as the sample volume, cell number, choice of antibodies, antibody titers, buffer conditions, and the temperature/duration of incubation steps
• Include appropriate controls—biological controls, unstained controls, and controls for multicolor staining such as compensation and fluorescence minus one (FMO) controls
• Follow panel building best practice—spread fluorophores across as many lasers and filters as possible, choose bright fluorophores for low density markers and vice versa, use a viability dye to exclude dead cells, always evaluate spread on each individual instrument, do not alter the panel without performing re-optimization
• Determine whether antibody master mixes can be prepared for use on different samples across different days—high stability dyes are essential here
• Use the same lot of reagents (e.g., antibodies, fixatives, permeabilization buffers, blocking agents) wherever possible and perform comparative studies every time a lot number changes
• Consider purchasing antibody reagents in bulk to support large or long-term studies (provided the expiration date is compatible); tandem dyes may not be suitable for long-term studies as they are prone to breakdown
• Always store reagents correctly
• Optimize and record acquisition and gating strategies
• Consider using calibration beads to standardize the flow cytometer
• Use the same flow cytometer for each experiment or establish standardized setup and settings across different instruments
• Refer to established best practice guidelines
• Implement measures for monitoring assay performance—track specificity, sensitivity, and intra- and inter-assay variation over time
• Use standardized analysis templates or automated data analysis to reduce subjectivity
• Once protocols/SOPs have been developed, adhere to them
• Ensure any new users receive comprehensive training

Tools for improved reproducibility

All of the companies included here have developed multiple products for improved flow cytometry reproducibility. Highlights include Bio-Rad’s StarBright Dyes that are extremely stable, produce a large Stokes shift without being a traditional tandem, and can be pre-mixed without unwanted dye interactions; Thermo Fisher Scientific’s Invitrogen™ Rainbow Calibration Particles that ensure consistent instrument settings and have been reported to enable standardization of flow cytometers from different manufacturers; and Cytek’s full spectrum flow cytometry systems that were designed with unique fluidics, optics, and electronics systems to ensure more reproducible flow cytometry data, as well as built-in software features such as automated instrument setup standardization and optimized acquisition settings that remove the need to manually adjust detector gains.