Q&A: Experts Weigh In on Reproducibility Issues Associated with Antibody Use

Reproducibility of antibodies is essential in not only advancing science but also in advancing and protecting a scientist’s career. If the experiment isn’t reproducible, then delays begin and the scientist’s career path could be halted or even destroyed if the results are published and then have to be retracted. While scientists are experiencing the setback, antibody companies should be the ones held accountable for their faulty antibodies and their lot-to-lot inconsistencies. Not all can take this responsibility seriously since quality is out of their control if they are a reseller and not the manufacturer. It is a crisis, and only when antibody companies improve transparency in the marketplace can we hope to resolve the problems of irreproducible science. Luckily, there are a few manufacturers in the market now that are transparent and others should soon change accordingly.

High-quality antibodies directly contribute to high reproducibility rates within life science research and scientific advances. These critical reagents must be properly used by researchers to obtain the desired quality and consistency of generated data. These two points are essential for the future, as they impact basic research, pre-clinical therapeutics, in vitro diagnostics, food safety and biosecurity.

To accomplish this goal we must foster an environment where continuous improvement of both antibody quality and validation standards is commonplace. We must collectively do more to improve the manufacturing process and adopt robust validation standards as they are developed. But the global resolution of problems pertaining to antibodies will require fundamental changes in how many stakeholders approach the topic, including manufacturers, resellers, researchers, funding agencies, journals and universities where most training of scientists occurs.

Antibodies are some of the most critical reagents used in life science and medical research. However, the use of poorly characterized antibodies can generate significant problems–compromising research projects and costing valuable time and money. According to an article published in Nature in 2015, an estimated $350 million in the United States and $800 million worldwide is wasted each year as a result of nonspecific or poorly validated antibodies contributing to failed or unreproducible experiments. While wasting precious research budget is never acceptable, it is becoming increasingly important during times of funding uncertainty that vendors and scientists align to address this issue.

While the issue of antibody validation standards has been discussed for quite some time, 2016 represented a major shift in the research community’s approach to addressing the challenge. In September, Nature Methods published the first proposed strategies to create standards for validating antibody specificity, brought forth by the International Working Group on Antibody Validation (IWGAV). Later that month, the Global Biological Standards Institute (GBSI) held the Asilomar Antibody Validation Workshop, where key stakeholders across the industry discussed strategies for validating antibodies, building on the priorities outlined in the Nature Methods piece.

Reproducibility is very important for all research reagents in order to ensure that analyses and assays can be repeated both over time and by other researchers, giving the same results.

The question of whether a research antibody is reproducible is dependent on whether a new lot or batch of an antibody performs equally well to a previous lot or batch. Reproducibility should not to be mixed up with validation or specificity of an antibody. A nonspecific antibody can be equally bad in each new lot, and have high reproducibility.

When manufacturing a new lot/batch of an antibody, regardless of antibody type, polyclonal, monoclonal or recombinant, stringent quality control procedures must be followed to ensure that the new lot performs the same way as previous lots in all assays.

The immune response may vary between animals, and therefore a polyclonal antibody purified from a new animal sera needs an even more stringent quality control procedure to secure retained specificity and functionality. The standard procedure at Atlas Antibodies is to analyze the new lot in parallel with a reference lot on consecutive sample materials, using the same protocol and test for all approved applications.

As mentioned by the NIH, Nature and Science, the cornerstones of science advancement are rigor in designing and performing scientific research and the ability to reproduce biomedical research, as well as transparency and independent verification. At Bio-Techne, we want to do our part as partners with the research community to raise industry-wide validation standards for the benefit of scientists, advancing biological discovery, and to boost research reproducibility.

Antibodies are critical tools for life science research, and the acknowledgement they are contributors to irreproducibility has now led to concerted efforts to tackle this issue. As one of the original innovators in protein and antibody-based research tools, reproducibility of research studies is something we at Bio-Techne have cared greatly about for the last 30 years. We believe using high quality antibodies from a trusted source is key for ensuring reproducibility. This starts with expert antigen design, application screening and validation, production and quality control processes. A highly experienced antibody development team is critical to all these steps. With over 50 years of combined antibody development expertise at our Bio-Techne antibody brands of R&D Systems and Novus Biologicals, we build and validate our antibodies and other key reagents with reproducibility top of mind. We make sure our antibodies give consistent results whether you carry out your experiments today or in 30 years’ time. We offer recombinant antibodies and recombinant conversion of hybridoma-derived monoclonals for those who want the ultimate in reproducibility and long-term supply security. In short, reproducibility is a key issue which we have taken very seriously at Bio-Techne for decades and not added on recently.

Despite negative reports regarding scientific reproducibility, I believe there are many good commercial antibodies in the marketplace that are being used to make new important discoveries in biomedical research every day. However, there are also bad antibodies and companies that commercialize antibodies without proper validation. In our experience, this is the result of companies selling antibodies they did not develop or taking shortcuts to diminish the costs associated with solid validation and quality control, which can be expensive. A responsible antibody vendor should provide clear data to support claims made regarding the performance, limitations, sensitivity and specificity of any product they sell and stand behind their reagents with excellent technical support.

Beyond antibody suppliers, there are other stakeholders that may contribute to the reproducibility problem. First are antibody users who may not be familiar with antibody-based methods and/or do not follow the recommended protocols. Antibodies can be tricky to use, particularly for some complex applications. Deviation from the recommended protocols used to develop and validate the antibody by a vendor can lead to poor results even with a highly specific and sensitive antibody. Secondly, we believe journals publishing scientific papers need to do a better job compelling authors to provide all the necessary information about the antibodies and protocols used in the study. Insufficient information can make a great paper virtually irreproducible.

We agree with many of the concerns and points of view being shared today about antibody reproducibility. Antibody companies should be striving to provide more detailed information about the products themselves and striving to provide more detail about exactly how the antibodies are used and tested. We work hard as a longtime developer and producer of antibody products to ensure our customers have confidence in the antibodies we supply them.

Antibodies are key reagents in all cell analysis applications, but their widespread use has also resulted in them being considered commodity products. Thus, though researchers still spend a lot of money on antibodies, they don’t necessarily spend the required time in validating and transparently reporting the usage of antibodies. Also, the research antibody market has become extremely crowded with multiple vendors selling the same clone. However, there are no universal standards for assessing the quality of an antibody sold by different vendors. This leads to a situation where there are no checks on vendors as to whether they developed a product with highest quality guaranteed to give specific, sensitive, and highly reproducible results.

I think researchers have known for some time that not all commercial antibodies are created equally, but this issue has not received serious public attention until recently. Some commercial antibodies are of very good quality. They are highly specific for their target, they are sensitive enough to be used in a meaningful way, and they have good lot-to-lot consistency. However, an alarmingly high percentage of commercial antibodies have poor specificity, low sensitivity, or both, and may not even work in any immune assay at all. Poorly performing antibodies can lead researchers to make erroneous conclusions based on data from off-target binding of the antibody. Even if an antibody works well in an immunoassay, poor lot-to-lot consistency may make it impossible to replicate experiments. Because antibodies are such commonly used yet critical reagents, poor performance can waste time, resources, and put the quality of life science research at risk.

Antibodies are great tools. Many exceptionally important research projects would be impossible without antibodies. Thus, researchers need to be able to trust their antibody-based products. At BioLegend, we have always understood that we could never compromise on quality. If an antibody fails in an experiment, far more is lost than that one reagent. No manufacturer should ever find it acceptable to put their client’s research at risk by allowing poorly characterized material to ship to a customer.

Many companies can create an antibody that binds in a single assay to a peptide immunogen or overexpressed recombinant protein. However, such an approach can result in products that give false promises. The fact that something preferentially binds to a target doesn’t mean it will be useful against the native molecule at endogenous levels, or that its preferential affinity for a target will not be swamped by promiscuous off-target binding. The keys to producing a great antibody product are in designing appropriate immunogens, screening for clones in relevant systems, and thorough validation of the clones and materials they produce. This is where the vast majority of effort must be spent. The failure to invest resources in this portion of product development is a key reason some companies fail to achieve a reputation for quality, despite having produced a large collection of antibodies.

If you start with a poor quality clone, then it is very difficult to reproducibly achieve good results. In contrast, if you develop a high quality clone, control production well, and carefully QC each lot, then you can guarantee reproducible performance against endogenous targets.

One of the fundamental tenets of scientific progress is the reproducibility of experimental results, which is largely dependent on the availability of the identical reagents used in the original report. When original findings cannot be repeated, the quality, stability, or performance of the reagents used in the experiments come into question. Research antibodies have come under particularly intense, and unfortunately well-deserved, criticism for frequently being the underlying reason for the failure to reproduce outcomes. Reports indicate that an estimated $800 million is wasted globally in researchers’ time, materials, and money attempting to do science with unreliable or poorly characterized antibodies. This growing dissatisfaction with the state of commercially available research antibodies has led to articles in many of the top biomedicine journals that demand increased validation of these reagents. These sentiments are echoed in new requirements by various funding agencies for scientists to document how antibodies and other reagents have been verified for performance.

As an antibody manufacturer, GeneTex acknowledges that there are significant issues within the commercial research antibody industry. In fact, we accept our part in this and must face the resultant scrutiny. We have realized that without industry-accepted standards for research antibodies, GeneTex must be on the forefront of antibody production, quality assurance, and validation.

Journal standards for publishing Western blot data are becoming more rigorous, but clear standards for antibody validation are not yet established. Validation results and experimental methodologies for Western blot analysis are not consistently reported in the literature, making it difficult to evaluate published results.

Careful experimental design is critical for quantitative Western blotting, but its impact on accuracy and reproducibility is not widely recognized. Sample preparation, detection, image capture, and data analysis methods can introduce variability and limit the reproducibility of Western blot analysis. For accurate quantitative analysis, it is important to understand and eliminate common sources of error. The key is to maximize accuracy and precision by minimizing Western blot variability, so your relative comparisons are as meaningful as possible. Careful validation and documentation of antibody performance are critical parts of this process.

Reproducibility is the key factor in scientific research. It will not only affect the robustness of a researcher’s results, but it also impacts the reliability and development of biological science. Unfortunately, more than 70% of researchers have tried but failed to reproduce another scientist's experiments, and more than half have failed to reproduce their own experiments. One of the major factors contributing to the lack of reproducibility is the reagents used. In biological research, immunology-based approaches, such as ELISA, Western blot, IP, IF, IHC, and FACS, are the fundamental techniques. As a result, the antibody reagents used have an important role in the repeatability of publication data.

We recognize that as a qualified antibody reagents provider, we are participants in researchers’ studies and are as responsible for the data as the authors are.

Antibodies are produced by host animals. Precise antigens are a prerequisite for screening reliable antibodies. We have the professional teams and various expression systems—from prokaryote to eukaryote cells and also cell-free expression systems—for protein/peptide reconstruction. Sino Biological owns one of the largest protein libraries in the world. We assure the quality of various protein antigens, which includes not only the sequence, but also the correct configuration and post-translation modifications. Using our professional staff and protein production expertise, we can choose the best antigen design for a certain target. For example, to obtain an antibody against a membrane protein, we would try the whole natural protein, the exocellular structure, and/or the DNA or whole cell immunization technique. To ensure reproducibility we screen the best antibody clone by comparing as many candidates as possible.