Reproducibility
Robust and Replicable
Data Integrity
Three topical, and to some, contentious issues the research world is talking about, mainly due to new standards set in place by scientific publications. Increasingly, reports are circulating about work being challenged to demonstrate reproducibility as the final hurdle to publication.
However, publication isn’t always the foremost concern for a lab.
Let’s talk about the real bottom line here: having confidence your data is accurate. That it is the result of solid lab work. That the results reflect standards everyone in the lab is adhering to. That data sets match even when generated by different team members.
Controlling, reducing, and eliminating variables is Research 101, or at least it should be. Before the publication even asks, “Is the data robust and reproducible?” the PI is concerned about the quality of the data coming out of the lab.
If the PI or department head is saying, “I don’t know who to believe in my lab,” or “Why aren’t we getting consistent answers?” it’s clear there’s work to be done to establish and ensure standards regardless of concerns about publication.
This suggests that the key words are actually:
Standardization
Robust and Rigorous Experimental Design
Technically Sound
These, too, can be contentious and subject to debate. They speak to critical issues that every lab must address: consistency and variability. Standardization helps reduce errors. Robust and rigorous experimental design leads to better replication of methodologies, resulting in more consistent and accurate data. And regardless if colleagues agree with the conclusions on a particular project, technically sound equals zero questions as to the integrity of the research.
Standardization is clearly not just a matter of a well-written protocol. Robust data result from proper training in the basics of research; rigorous quality controls in the lab; highly accurate instrumentation and quality consumables; and a commitment to accuracy and consistency in record keeping. Additionally, uncompromised ethics and uniform requirements for publication all play a role in the pathway to reliable and reproducible data.
It’s important to keep focus on the issue: accurate, reproducible data are important to all of us. If a research study is going to lead to a cure for a particular disease, can’t we all agree that it’s in our mutual best interest that the data are solid?
As the scientific community works to ensure the integrity of its research, let us also work together to address the proverbial elephants in the room.
As the scientific community works to ensure the integrity of its research, let us also work together to address the proverbial elephants in the room. These include budget constraints; the conflicting pressures of “do great work but be done by the deadline;” cutting corners contrary to set protocols; and, admittedly a personal bias, that instrument and consumable manufacturers do not exist to just sell products.
Yes, just as researchers need to publish in order not to perish, technology companies must sell in order to stay in business. Nothing new or remarkably profound here. Yet here is where we are calling on our colleagues and competitors to ask that we all agree researchers deserve our best.
Manufacturers have a responsibility to provide products, tools, and services that help ensure credible results. We should be transparent, forthright, and diligent in demonstrating how the research behind, and the validation of, the products we offer contribute to help achieve consistent data integrity.
This, too, is a shared issue. Technology companies demo on a regular basis to show how their particular product works and, often, how it works compared to the competition. This is foundational to commerce. What is in the best interest of researchers is for manufacturers to show what their product does best, and also what it doesn’t do. Then, let the market make an informed decision on what technology best suits its needs and helps ensure data integrity.
Delicate and touchy ground here admittedly. However, we all have a shared goal: generating data that help answer the most challenging research questions. Questions that lead to answers that, in turn, benefit us all, both as members of the scientific community and as potential benefactors of what solid research reveals. Because of this, regardless of which term we rally around, robust and reproducible data is in our mutual best interest.