A Quality Adjusted H-Index: Less is More

Measures are necessary because unmeasured work is easily ignored. If universities want to reward merit, they need some way to identify it. The problem is that every measure can be gamed. A recent example comes from prediction markets: French authorities investigated possible tampering with a weather sensor at Charles de Gaulle Airport after unusual temperature spikes coincided with profitable bets on Paris temperatures on Polymarket. The case illustrates a general principle: once a number has consequences, people have incentives to influence the number rather than the underlying reality. (WSKG)

The same problem exists in academia. Professors’ work is difficult to evaluate. Research quality, originality, theoretical importance, mentorship, service, and long-term influence are not easily reduced to a single number. Yet universities still need to make decisions about hiring, promotion, salaries, awards, and prestige. This need has encouraged the use of quantitative performance indicators, such as the number of publications, total citations, and the H-index.

These measures capture something real, but each creates distortions. Publication counts reward volume, even when many papers have little influence. Citation counts reward visibility and cumulative attention, but can be inflated by a few highly cited papers, large collaborations, review articles, field size, and self-citation. The H-index improves on both by requiring a body of cited work, but it has its own blind spot: it ignores how many low-impact papers were produced alongside the influential ones.nt:

It is well known that academic metrics are biased because researchers can influence both citation counts and publication counts. Self-citations are relatively easy to detect, and can be excluded if necessary. Citations from close peer networks are harder to evaluate. Mutual citation practices, honorary coauthorship, strategic review writing, conference visibility, social media promotion, and aggressive self-marketing can all increase citation counts without necessarily reflecting greater intellectual merit.

Publication counts are even easier to inflate. The simplest strategy is to divide research into many small papers, submit weak papers repeatedly until they are accepted somewhere, or publish in journals with low rejection rates. In some cases, this includes pay-to-publish outlets that rely more on publication fees than on rigorous peer review. These practices do not imply that all highly productive scholars are gaming the system, but they show why raw publication counts are poor indicators of quality.

In principle, the problem could be solved by independent evaluations of scientific quality. In practice, this is difficult. Quality is multidimensional: a paper may be technically rigorous but unimportant, original but wrong, influential but misleading, or methodologically imperfect but theoretically generative. Expert judgment is necessary, but it is also subjective, costly, and vulnerable to reputation, ideology, personal networks, and disciplinary fashions.

As a result, universities rely on imperfect quantitative proxies. These proxies are attractive because they are easy to count, but they are incomplete. They measure visibility and productivity more easily than they measure quality. The challenge is not to find a perfect metric, but to design metrics that are harder to game and that capture dimensions of merit ignored by existing indicators.

This is where the low-impact tail becomes relevant. A publication record with many highly cited papers and few low-cited papers conveys something different from a publication record with the same H-index but hundreds of additional papers that attracted little attention. The conventional H-index ignores this distinction. A quality-adjusted H-index makes it visible.

The draft has a strong argument. The main correction is that the quality-adjusted H-index is not a percentage. Efficiency is a percentage, H/N. But QH = H²/N is on an H-index-like scale. So Ioannidis’s QH is 25.9, not 25.9%, and Diener’s is 32.2, not 32.2%. The uploaded text also has several typos: “ward” should be “reward,” “publication” should be “publications,” “meta-scientists” should be “meta-scientist,” and “Ioanndisis’s” should be “Ioannidis’s.”

Here is a refined version of the section from “A long tail…” onward:

A long tail of low-impact publications has negative effects on science. It crowds out potentially better work by other researchers. It also consumes resources, especially when publications are supported by publicly funded grants or paid publication fees. It may even hurt the authors themselves. Time spent producing many low-impact articles is time not spent developing fewer, more substantial contributions. Rewarding efficiency may therefore benefit science by shifting incentives away from maximizing publication counts and toward producing work that has durable influence.

The proposed index is simple. It requires only two pieces of information: the total number of publications, N, and the H-index, H. The H-index rewards a sustained body of impactful work. It does not solve the problem that citations are only a proxy for quality, but that is not the purpose of the new index. The purpose is to adjust the H-index for publication efficiency.

Efficiency can be defined as the proportion of publications that belong to the H-core:

Efficiency = H / N

A researcher with an H-index of 100 and 400 publications is more efficient than a researcher with the same H-index and 1,000 publications. Both have the same citation core, but the second author needed many more publications to achieve it.

Combining impact and efficiency gives:

QH-index = Impact × Efficiency
QH-index = H × H/N
QH-index = H²/N

Examples

John P. A. Ioannidis is a prominent Stanford scientist and meta-scientist whose work has focused on improving scientific credibility and reducing false findings. He has an impressive H-index of 190 and an even more impressive total of 1,396 publications. Based on traditional metrics, this is an extraordinary record.

However, the record looks different when efficiency is taken into account. To achieve an H-index of 190, Ioannidis produced 1,396 publications. His efficiency is therefore:

190 / 1,396 = .136

Thus, 13.6% of his publications are in the H-core. His quality-adjusted H-index is:

190² / 1,396 = 25.9

Ed Diener was one of the most influential social and personality psychologists and helped establish the scientific study of subjective well-being. His H-index is 126, which is lower than Ioannidis’s H-index of 190. However, Diener produced 493 publications. His efficiency is therefore:

126 / 493 = .256

Thus, 25.6% of his publications are in the H-core, nearly twice Ioannidis’s efficiency. His quality-adjusted H-index is:

126² / 493 = 32.2

The conventional H-index ranks Ioannidis higher. The QH-index ranks Diener higher because Diener achieved a large citation core with a much smaller publication record.

Conclusion

No simple quantitative indicator is a perfect measure of merit. Still, universities and funding agencies need some measures to allocate limited resources. The H-index was designed to avoid rewarding researchers merely for producing many low-impact publications. It improved on simple publication counts by requiring a body of cited work. Yet the H-index still has a blind spot: once the H-core is established, additional low-impact publications carry no penalty.

The QH-index addresses this problem. It preserves the central virtue of the H-index by rewarding sustained impact, but it discounts this impact when it is accompanied by a large number of low-impact publications. Publishing more articles is beneficial only when it increases the citation core. Producing a large long tail of low-impact work lowers the score. This corrective may help reduce incentives to publish as much as possible without regard to the quality or influence of the work.

Leave a Reply