A critique of ‘quick and dirty’ bibliometrics: issues for assessing complexity in bibliometric analysis

Abstract

This paper denounces the indiscriminate rise of low-quality bibliometric studies. These studies have proliferated across scientific journals in a wide range of disciplines, taking advantage of the accessibility of databases such as Web of Science and Scopus and the use of automated tools. This trend, driven by a lack of specialised knowledge among journal reviewers and a desire to rapidly increase academic output, has led to the publication of studies with methodological shortcomings, characterised by limited conceptual development and low analytical value. In response to this situation, the authors advocate for bibliometrics as a highly specialised field that requires theoretical understanding, methodological expertise, and critical interpretation of data. The paper proposes a methodological framework for assessing the complexity of bibliometric studies. This framework is based on six dimensions: size of the population, origin and source of data, mode of data collection, degree of data normalisation, types of analysis employed, and analytical and visualisation tools used. This methodological perspective aims to provide editors and researchers with a framework for identifying substantive research and distinguishing it from studies produced with minimal effort, limited judgment, and weak theoretical grounding.