IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1904.06300.html
   My bibliography  Save this paper

Journal ranking should depend on the level of aggregation

Author

Listed:
  • L'aszl'o Csat'o

Abstract

Journal ranking is becoming more important in assessing the quality of academic research. Several indices have been suggested for this purpose, typically on the basis of a citation graph between the journals. We follow an axiomatic approach and find an impossibility theorem: any self-consistent ranking method, which satisfies a natural monotonicity property, should depend on the level of aggregation. Our result presents a trade-off between two axiomatic properties and reveals a dilemma of aggregation.

Suggested Citation

  • L'aszl'o Csat'o, 2019. "Journal ranking should depend on the level of aggregation," Papers 1904.06300, arXiv.org, revised Sep 2019.
    • Handle: RePEc:arx:papers:1904.06300
    as

    Download full text from publisher

    File URL: https://arxiv.org/pdf/1904.06300
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. ,, 2014. "A ranking method based on handicaps," Theoretical Economics, Econometric Society, vol. 9(3), September.
    2. Bouyssou, Denis & Marchant, Thierry, 2014. "An axiomatic approach to bibliometric rankings and indices," Journal of Informetrics, Elsevier, vol. 8(3), pages 449-477.
    3. Bouyssou, Denis & Marchant, Thierry, 2016. "Ranking authors using fractional counting of citations: An axiomatic approach," Journal of Informetrics, Elsevier, vol. 10(1), pages 183-199.
    4. Wolfgang Glänzel & Henk F. Moed, 2013. "Opinion paper: thoughts and facts on bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(1), pages 381-394, July.
    5. Chambers, Christopher P. & Miller, Alan D., 2014. "Scholarly influence," Journal of Economic Theory, Elsevier, vol. 151(C), pages 571-583.
    6. Motty Perry & Philip J. Reny, 2016. "How to Count Citations If You Must," American Economic Review, American Economic Association, vol. 106(9), pages 2722-2741, September.
    7. Woeginger, Gerhard J., 2008. "An axiomatic analysis of Egghe’s g-index," Journal of Informetrics, Elsevier, vol. 2(4), pages 364-368.
    8. repec:hal:pseose:halshs-01109087 is not listed on IDEAS
    9. Ignacio Palacios-Huerta & Oscar Volij, 2004. "The Measurement of Intellectual Influence," Econometrica, Econometric Society, vol. 72(3), pages 963-977, May.
    10. Miroiu, Adrian, 2013. "Axiomatizing the Hirsch index: Quantity and quality disjoined," Journal of Informetrics, Elsevier, vol. 7(1), pages 10-15.
    11. Palacios-Huerta, Ignacio & Volij, Oscar, 2014. "Axiomatic measures of intellectual influence," International Journal of Industrial Organization, Elsevier, vol. 34(C), pages 85-90.
    12. Lasso de la Vega, Casilda & Volij, Oscar, 2018. "Ranking scholars: A measure representation," Journal of Informetrics, Elsevier, vol. 12(2), pages 510-517.
    13. Kongo, Takumi, 2014. "An alternative axiomatization of the Hirsch index," Journal of Informetrics, Elsevier, vol. 8(1), pages 252-258.
    14. László Csató, 2015. "A graph interpretation of the least squares ranking method," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 44(1), pages 51-69, January.
    15. Kenneth J. Arrow, 1950. "A Difficulty in the Concept of Social Welfare," Journal of Political Economy, University of Chicago Press, vol. 58(4), pages 328-328.
    16. Pavel Yu. Chebotarev & Elena Shamis, 1998. "Characterizations of scoring methodsfor preference aggregation," Annals of Operations Research, Springer, vol. 80(0), pages 299-332, January.
    17. Antonio Quesada, 2010. "More axiomatics for the Hirsch index," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(2), pages 413-418, February.
    18. Quesada, Antonio, 2011. "Axiomatics for the Hirsch index and the Egghe index," Journal of Informetrics, Elsevier, vol. 5(3), pages 476-480.
    19. Satterthwaite, Mark Allen, 1975. "Strategy-proofness and Arrow's conditions: Existence and correspondence theorems for voting procedures and social welfare functions," Journal of Economic Theory, Elsevier, vol. 10(2), pages 187-217, April.
    20. Thierry Marchant, 2009. "An axiomatic characterization of the ranking based on the h-index and some other bibliometric rankings of authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 80(2), pages 325-342, August.
    21. Woeginger, Gerhard J., 2008. "An axiomatic characterization of the Hirsch-index," Mathematical Social Sciences, Elsevier, vol. 56(2), pages 224-232, September.
    22. Antonio Quesada, 2011. "Further characterizations of the Hirsch index," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(1), pages 107-114, April.
    23. Gibbard, Allan, 1973. "Manipulation of Voting Schemes: A General Result," Econometrica, Econometric Society, vol. 41(4), pages 587-601, July.
    24. László Csató, 2019. "An impossibility theorem for paired comparisons," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 27(2), pages 497-514, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Csató, László, 2019. "Journal ranking should depend on the level of aggregation," Journal of Informetrics, Elsevier, vol. 13(4).
    2. Bouyssou, Denis & Marchant, Thierry, 2016. "Ranking authors using fractional counting of citations: An axiomatic approach," Journal of Informetrics, Elsevier, vol. 10(1), pages 183-199.
    3. Osório, António (António Miguel), 2019. "The value and credits of n-authors publications," Working Papers 2072/376026, Universitat Rovira i Virgili, Department of Economics.
    4. Sylvain Béal & Eric Rémila & Philippe Solal & Sylvain Ferrières, 2016. "An axiomatization of the iterated h-index and applications to sport rankings," Working Papers hal-01394818, HAL.
    5. Bouyssou, Denis & Marchant, Thierry, 2014. "An axiomatic approach to bibliometric rankings and indices," Journal of Informetrics, Elsevier, vol. 8(3), pages 449-477.
    6. Ana Paula dos Santos Rubem & Ariane Lima Moura & João Carlos Correia Baptista Soares de Mello, 2015. "Comparative analysis of some individual bibliometric indices when applied to groups of researchers," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 1019-1035, January.
    7. Antonin Macé, 2023. "The Limits of Citation Counts," Working Papers halshs-01630095, HAL.
    8. Mark Levene & Trevor Fenner & Judit Bar-Ilan, 2019. "Characterisation of the $$\chi$$χ-index and the rec-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 885-896, August.
    9. Brandão, Luana Carneiro & Soares de Mello, João Carlos Correia Baptista, 2019. "A multi-criteria approach to the h-index," European Journal of Operational Research, Elsevier, vol. 276(1), pages 357-363.
    10. Woeginger, Gerhard J., 2014. "Investigations on the step-based research indices of Chambers and Miller," Journal of Informetrics, Elsevier, vol. 8(3), pages 659-666.
    11. Lasso de la Vega, Casilda & Volij, Oscar, 2018. "Ranking scholars: A measure representation," Journal of Informetrics, Elsevier, vol. 12(2), pages 510-517.
    12. Mukherjee, Conan & Alam, Aftab, 2016. "On Evaluating Author's Performance by Publications: An Axiomatic Study," Working Papers 2016:14, Lund University, Department of Economics, revised 12 May 2017.
    13. Chambers, Christopher P. & Miller, Alan D., 2014. "Scholarly influence," Journal of Economic Theory, Elsevier, vol. 151(C), pages 571-583.
    14. J. Atsu Amegashie, 2020. "Citations And Incentives In Academic Contests," Economic Inquiry, Western Economic Association International, vol. 58(3), pages 1233-1244, July.
    15. Haley, M. Ryan & McGee, M. Kevin, 2020. "Jointly valuing journal visibility and author citation count: An axiomatic approach," Journal of Informetrics, Elsevier, vol. 14(1).
    16. Conan Mukherjee & Ranojoy Basu & Aftab Alam, 2020. "A measure of authorship by publications," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 41(3), pages 354-361, April.
    17. Adachi, Tsuyoshi & Kongo, Takumi, 2015. "Further axiomatizations of Egghe's g-index," Journal of Informetrics, Elsevier, vol. 9(4), pages 839-844.
    18. Antonin Mac'e, 2017. "The Limits of Citation Counts," Papers 1711.02695, arXiv.org, revised Sep 2023.
    19. Flores-Szwagrzak, Karol & Treibich, Rafael, 2015. "Co-authorship and the Measurement of Individual Productivity," Discussion Papers on Economics 17/2015, University of Southern Denmark, Department of Economics.
    20. Palacios-Huerta, Ignacio & Volij, Oscar, 2014. "Axiomatic measures of intellectual influence," International Journal of Industrial Organization, Elsevier, vol. 34(C), pages 85-90.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1904.06300. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: https://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.