A New Measure Redefines How Science Identifies Breakthroughs

For decades, citation counts served as the dominant way to gauge a research paper's impact. But counting citations measures popularity, not impact. In 2017, the Consolidation-Disruption (CD) Index was introduced to change that, aiming to identify truly disruptive papers—those representing contributions so distinct they eclipsed previous research on a topic.

The CD Index, however, has a notable blind spot. When two research teams make simultaneous discoveries and one cites the other, the index ranks one paper as a breakthrough and the other at the bottom. In some cases, both papers end up ranked low. Research has shown that small changes in citation patterns can render the score unstable, undermining its reliability.

Munjung Kim and YY Ahn of the University of Virginia are working with Sadamori Kojaku of Binghamton University to fix this problem. Their research, published in Science Advances, proposes a new citation ranking system called EDM, or Embedding Disruptiveness Measure.

"EDM uses techniques from neural language models to give each paper two representations, one for what came before it and one for what came after," Kim explained. "Nobel-level work tends to score significantly higher under EDM, it's stable against small citation changes, and it can identify simultaneous discoveries the older method couldn't."

Kim's path to developing EDM began with a fascination with the concept behind the disruption index—the idea that a disruptive paper is so notable it renders all prior work obsolete. At the same time, she was studying graph embeddings, a machine learning approach that turns networks into vectors in a geometric space. "It struck me that this kind of tool might let us improve the disruption index, which had relied only on counting immediate citation relationships," she said.

The key insight came through collaboration with Kojaku, who suggested using directional embeddings—learning two separate representations per paper. Once the model was built that way, a pattern emerged. Papers where EDM disagreed most with the CD Index turned out to be well-known cases of simultaneous discovery, revealing the original index's systematic flaw.

Kim hopes EDM will support large-scale studies on how breakthroughs happen, what kinds of teams produce them, and how funding relates to disruptiveness. "A more robust measure makes that work a little more reliable, especially when simultaneous discovery is common, which the sociologist Robert K. Merton argued is actually the rule, rather than the exception in science."