In modern-day academia, citations are one of the main metrics to evaluate the impact of a study or a researcher’s scholarly productivity. Unfortunately, this tendency to overvalue citations has led to a practice known as “citation gaming”, whereby researchers go to great lengths to inflate the number of citations of their publications. Since tight-knit research groups or frequent collaborators can continuously cite each other’s publications back and forth, it has become challenging to identify individuals whose works have a truly remarkable reach.

Against this backdrop, a team of researchers from Birla Institute of Technology and Science (BITS) Pilani, Nitte Meenakshi Institute of Technology, Indian Statistical Institute, and PES University in India has now developed a new metric called the Author Lineage Independent Score (ALIS), which can assess “scholarly independence,” or the ability of researchers to come out from the shadows of their peers/supervisors and become independent. Their study is published in Volume 9, Issue 4 of the Journal of Information Science.

“Citation counts are an important indicator of the impact and credibility of one’s work. However, scholarly independence can only be accomplished when the researcher can independently pursue new inventions and ideas, and the impact of such new ideas are visible outside the influence of the scholar’s tight-knit research group or genealogy tree,” explains Prof. Snehanshu Saha from BITS Pilani, one of the authors of the study.

But computing scholarly independence is not a straightforward task as this is affected by multiple factors. In their study, the researchers decided to exploit the genealogical relationships between scholars, their mentors, and collaborating communities to compute ALIS. Like a family tree, an academic genealogical tree maps out Ph.D. supervisors and Ph.D. students as ancestors and heirs, respectively. Thus, it describes the connections between individual scholars and their peers, their superiors, and their communities. Based on such trees, the team developed three indicators.

The first of these was the ratio of non-genealogy citations (NGCs), which represents the proportion of citations that a scholar has received for their work from researchers outside their genealogical tree. The second is the other citation quotient (OCQ), indicating the proportion of citations that a researcher has received from people other than themselves.

Finally, the third indicator is the ratio of non-community citations (NCCs). Here, an author’s “community” refers to a group that bunches together the author’s siblings in the genealogical tree, as well as neighbours and colleagues who share the same workplace. “Members of a community share ideas, publish together, and often cite each other. Although citations from community networks cannot always be seen as deceptive and questionable, the intention behind exempting community citations in our calculations is to chisel away citations the authors might have accumulated due to their workplace tie-ups,” explains Prof. Saha.

To compute ALIS, these three indicators must be fed into an optimisation model. Accordingly, the team designed an algorithm that avoids bias due to any of the indicators, while also making the problem computationally tractable.

To validate their approach, the team computed ALIS for 100 real authors using data collected from the Web of Science and the Mathematics Genealogy Project. Detailed analyses of the results revealed that ALIS could identify researchers who had demonstrated exceptional scholarly independence regardless of the number of publications and citations. Similarly, authors with a high citation count but lower ALIS tended to collaborate more within their genealogy and community. To further validate ALIS and their optimisation model on a larger scale, the team also conducted more tests using an artificially generated dataset for 10,000 authors.

The implications of this study are manifold. Scholarly independence is an important quality that many organisations try to attract and cultivate. It can lead to more fruitful international collaborations, a higher degree of interdisciplinary research, and genuinely impactful studies. Further, ALIS can help solve a common problem during academic recruitment, namely that of a tie between two seemingly equally qualified candidates. Prof. Saha remarks, “ALIS could potentially be a game-changer in situations demanding a tie-break, where two candidates are close in terms of scholarly achievements, but only one can be picked.”

Furthermore, mathematical analyses showed that ALIS is not statistically correlated to any of the commonly used metrics, meaning it is not “just another metric” but brings new information to the table for assessing the capabilities of scholars.

ALIS could, thus, pave the way for other sophisticated metrics of evaluative bibliometrics. Integrating such metrics into common practice could help put an end to citation gaming and other unethical practices currently persisting in academia.

 “Common metrics of scholarly evidence must be taken with a pinch of salt, alternative solutions are a reality,” concludes Prof. Saha. While only time will tell how much ALIS reshapes the quantification of scholarly independence, the team remains hopeful.