IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/23282.html
   My bibliography  Save this paper

What do Editors Maximize? Evidence from Four Leading Economics Journals

Author

Listed:
  • David Card
  • Stefano DellaVigna

Abstract

We study editorial decision-making using anonymized submission data for four leading economics journals: the Journal of the European Economics Association, the Quarterly Journal of Economics, the Review of Economic Studies, and the Review of Economics and Statistics. We match papers to the publication records of authors at the time of submission and to subsequent Google Scholar citations. To guide our analysis we develop a benchmark model in which editors maximize the expected quality of accepted papers and citations are unbiased measures of quality. We then generalize the model to allow different quality thresholds for different papers, and systematic gaps between citations and quality. Empirically, we find that referee recommendations are strong predictors of citations, and that editors follow the recommendations quite closely. Holding constant the referees' evaluations, however, papers by highly-published authors get more citations, suggesting that referees impose a higher bar for these authors, or that prolific authors are over-cited. Editors only partially offset the referees' opinions, effectively discounting the citations of more prolific authors in their revise and resubmit decisions by up to 80%. To disentangle the two explanations for this discounting, we conduct a survey of specialists, asking them for their preferred relative citation counts for matched pairs of papers. The responses show no indication that prolific authors are over-cited and thus suggest that referees and editors seek to support less prolific authors.

Suggested Citation

  • David Card & Stefano DellaVigna, 2017. "What do Editors Maximize? Evidence from Four Leading Economics Journals," NBER Working Papers 23282, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:23282
    Note: IO LS PE POL PR
    as

    Download full text from publisher

    File URL: https://www.nber.org/papers/w23282.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ivo Welch, 2014. "Referee Recommendations," The Review of Financial Studies, Society for Financial Studies, vol. 27(9), pages 2773-2804.
    2. Damien Besancenot & Kim Huynh & Radu Vranceanu, 2011. "A Matching Model of the Academic Publication Market," Journal of Institutional and Theoretical Economics (JITE), Mohr Siebeck, Tübingen, vol. 167(4), pages 708-725, December.
    3. Glenn Ellison, 2013. "How Does the Market Use Citation Data? The Hirsch Index in Economics," American Economic Journal: Applied Economics, American Economic Association, vol. 5(3), pages 63-90, July.
    4. Marshall H. Medoff, 2003. "Editorial Favoritism in Economics?," Southern Economic Journal, John Wiley & Sons, vol. 70(2), pages 425-434, October.
    5. Damien Besancenot & Kim Van Huynh & Radu Vranceanu, 2011. "A Matching Model of the Academic Publication Market," CEPN Working Papers halshs-00589186, HAL.
    6. Raj Chetty & Emmanuel Saez & Laszlo Sandor, 2014. "What Policies Increase Prosocial Behavior? An Experiment with Referees at the Journal of Public Economics," Journal of Economic Perspectives, American Economic Association, vol. 28(3), pages 169-188, Summer.
    7. Elisabeth Schulte & Mike Felgenhauer, 2017. "Preselection and expert advice," International Journal of Game Theory, Springer;Game Theory Society, vol. 46(3), pages 693-714, August.
    8. Glenn Ellison, 2002. "The Slowdown of the Economics Publishing Process," Journal of Political Economy, University of Chicago Press, vol. 110(5), pages 947-993, October.
    9. David Card & Stefano DellaVigna, 2013. "Nine Facts about Top Journals in Economics," Journal of Economic Literature, American Economic Association, vol. 51(1), pages 144-161, March.
    10. Jonathan B. Berk & Campbell R. Harvey & David Hirshleifer, 2017. "How to Write an Effective Referee Report and Improve the Scientific Review Process," Journal of Economic Perspectives, American Economic Association, vol. 31(1), pages 231-244, Winter.
    11. Besancenot, Damien & Huynh, Kim & Vranceanu, Radu, 2009. "Desk rejection in an academic publication market model with matching frictions," ESSEC Working Papers DR 09008, ESSEC Research Center, ESSEC Business School.
    12. Glenn Ellison, 2002. "Evolving Standards for Academic Publishing: A q-r Theory," Journal of Political Economy, University of Chicago Press, vol. 110(5), pages 994-1034, October.
    13. Laband, David N & Piette, Michael J, 1994. "Favoritism versus Search for Good Papers: Empirical Evidence Regarding the Behavior of Journal Editors," Journal of Political Economy, University of Chicago Press, vol. 102(1), pages 194-203, February.
    14. Carole J. Lee & Cassidy R. Sugimoto & Guo Zhang & Blaise Cronin, 2013. "Bias in peer review," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(1), pages 2-17, January.
    15. Hofmeister Robert & Krapf Matthias, 2011. "How Do Editors Select Papers, and How Good are They at Doing It?," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 11(1), pages 1-23, October.
    16. Brogaard, Jonathan & Engelberg, Joseph & Parsons, Christopher A., 2014. "Networks and productivity: Causal evidence from editor rotations," Journal of Financial Economics, Elsevier, vol. 111(1), pages 251-270.
    17. Cherkashin, Ivan & Demidova, Svetlana & Imai, Susumu & Krishna, Kala, 2009. "The inside scoop: Acceptance and rejection at the journal of international economics," Journal of International Economics, Elsevier, vol. 77(1), pages 120-132, February.
    18. Raymond Fisman & Jing Shi & Yongxiang Wang & Rong Xu, 2018. "Social Ties and Favoritism in Chinese Science," Journal of Political Economy, University of Chicago Press, vol. 126(3), pages 1134-1171.
    19. Daniel S. Hamermesh, 1994. "Facts and Myths about Refereeing," Journal of Economic Perspectives, American Economic Association, vol. 8(1), pages 153-163, Winter.
    20. Carole J. Lee & Cassidy R. Sugimoto & Guo Zhang & Blaise Cronin, 2013. "Bias in peer review," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(1), pages 2-17, January.
    21. Marshall Medoff, 2006. "Evidence of a Harvard and Chicago Matthew Effect," Journal of Economic Methodology, Taylor & Francis Journals, vol. 13(4), pages 485-506.
    22. Danielle Li, 2017. "Expertise versus Bias in Evaluation: Evidence from the NIH," American Economic Journal: Applied Economics, American Economic Association, vol. 9(2), pages 60-92, April.
    23. Blank, Rebecca M, 1991. "The Effects of Double-Blind versus Single-Blind Reviewing: Experimental Evidence from The American Economic Review," American Economic Review, American Economic Association, vol. 81(5), pages 1041-1067, December.
    24. repec:wly:soecon:v:82:2:y:2015:p:430-452 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Marchingiglio, Riccardo & Poyker, Michael, 2019. "The Employment Effects of Gender-Specific Minimum Wage," Working Papers 290, The University of Chicago Booth School of Business, George J. Stigler Center for the Study of the Economy and the State.
    2. Jonas Hjort & Jonas Poulsen, 2019. "The Arrival of Fast Internet and Employment in Africa," American Economic Review, American Economic Association, vol. 109(3), pages 1032-1079, March.
    3. Simon Ek & Magnus Henrekson, 2019. "The Geography and Concentration of Authorship in the Top Five: Implications For European Economics," Scottish Journal of Political Economy, Scottish Economic Society, vol. 66(2), pages 215-245, May.
    4. Alan I. Barreca & Matthew Neidell & Nicholas J. Sanders, 2017. "Long-Run Pollution Exposure and Adult Mortality: Evidence from the Acid Rain Program," NBER Working Papers 23524, National Bureau of Economic Research, Inc.
    5. Christophe BELLEGO & Louis-Daniel PAPE, 2019. "Dealing with the log of zero in regression models," Working Papers 2019-13, Center for Research in Economics and Statistics.
    6. Klaus Wohlrabe & Lutz Bornmann, 2022. "Alphabetized co-authorship in economics reconsidered," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2173-2193, May.
    7. Dowling, Michael & Hammami, Helmi & Zreik, Ousayna, 2018. "Easy to read, easy to cite?," Economics Letters, Elsevier, vol. 173(C), pages 100-103.
    8. Yan, Zheming & Yu, Ying & Du, Kerui & Zhang, Ning, 2024. "How does environmental regulation promote green technology innovation? Evidence from China's total emission control policy," Ecological Economics, Elsevier, vol. 219(C).
    9. Sandro Mendonça & João Pereira & Manuel Ennes Ferreira, 2018. "Gatekeeping African studies: what does “editormetrics” indicate about journal governance?," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(3), pages 1513-1534, December.
    10. Battistin, Erich & Ovidi, Marco, 2017. "Rising Stars," IZA Discussion Papers 11198, Institute of Labor Economics (IZA).
    11. Krishna Muniyoor, 2022. "The Structure of Scholarly Publishing: a Case of Economics Research in India," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 13(3), pages 1801-1818, September.
    12. Gunes Gokmen & Wessel N. Vermeulen & Pierre-Louis Vézina, 2020. "The imperial roots of global trade," Journal of Economic Growth, Springer, vol. 25(1), pages 87-145, March.
    13. Maxim Ananyev & Michael Poyker & Yuan Tian, 2021. "The safest time to fly: pandemic response in the era of Fox News," Journal of Population Economics, Springer;European Society for Population Economics, vol. 34(3), pages 775-802, July.
    14. McCannon, Bryan C., 2019. "Readability and research impact," Economics Letters, Elsevier, vol. 180(C), pages 76-79.
    15. Fang, Hanming & Lei, Ziteng & Lin, Liguo & Zhang, Peng & Zhou, Maigeng, 2023. "Family companionship and elderly suicide: Evidence from the Chinese Lunar New Year," Journal of Development Economics, Elsevier, vol. 162(C).
    16. Petersen, Alexander M., 2019. "Megajournal mismanagement: Manuscript decision bias and anomalous editor activity at PLOS ONE," Journal of Informetrics, Elsevier, vol. 13(4).
    17. Carmela Lutmar & Yaniv Reingewertz, 2021. "Academic in-group bias in the top five economics journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(12), pages 9543-9556, December.
    18. Christian Dippel & Michael Poyker, 2023. "Do Private Prisons Affect Criminal Sentencing?," Journal of Law and Economics, University of Chicago Press, vol. 66(3), pages 511-534.
    19. Sascha Baghestanian & Sergey V. Popov, 2017. "Alma Mat(t)er(s): Determinants of Early Career Success in Economics," Economics Working Papers 17-02, Queen's Management School, Queen's University Belfast.
    20. Lutmar, Carmela & Reingewertz, Yaniv, 2020. "Academic in-group bias in economics," MPRA Paper 104730, University Library of Munich, Germany.
    21. Manuel Ennes Ferreira & Sandro Mendonça & João Pereira, 2018. "Gatekeeping African studies: What does “editormetrics” indicate about journal governance?," Working Papers Department of Economics 2018/07, ISEG - Lisbon School of Economics and Management, Department of Economics, Universidade de Lisboa.
    22. Erin Hengel & Eunyoung Moon, 2020. "Gender and quality at top economics journals," Working Papers 202001, University of Liverpool, Department of Economics.
    23. Reingewertz, Yaniv & Lutmar, Carmela, 2018. "Academic in-group bias: An empirical examination of the link between author and journal affiliation," Journal of Informetrics, Elsevier, vol. 12(1), pages 74-86.
    24. Ali Sina Önder & Sergey V. Popov & Sascha Schweitzer, 2021. "Leadership in Scholarship: Editors’ Appointments and the Profession’s Narrative," Working Papers in Economics & Finance 2021-05, University of Portsmouth, Portsmouth Business School, Economics and Finance Subject Group.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. David Card & Stefano DellaVigna, 2020. "What Do Editors Maximize? Evidence from Four Economics Journals," The Review of Economics and Statistics, MIT Press, vol. 102(1), pages 195-217, March.
    2. Sascha Baghestanian & Sergey V. Popov, 2018. "On publication, refereeing and working hard," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 51(4), pages 1419-1459, November.
    3. Damien Besancenot & Kim Huynh & Joao Faria, 2012. "Search and research: the influence of editorial boards on journals’ quality," Theory and Decision, Springer, vol. 73(4), pages 687-702, October.
    4. Bayar, Onur & Chemmanur, Thomas J., 2021. "A model of the editorial process in academic journals," Research Policy, Elsevier, vol. 50(9).
    5. Besancenot Damien & Faria João R. & Huynh Kim V., 2014. "Congestion of Academic Journals Under Papers’ Imperfect Selection," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 14(3), pages 1145-1167, July.
    6. Gehrig, Thomas & Stenbacka, Rune, 2021. "Journal competition and the quality of published research: Simultaneous versus sequential screening," International Journal of Industrial Organization, Elsevier, vol. 76(C).
    7. Yushan Hu & Ben G. Li, 2021. "The production economics of economics production," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 30(1), pages 228-255, February.
    8. Bransch, Felix & Kvasnicka, Michael, 2022. "Male Gatekeepers: Gender Bias in the Publishing Process?," Journal of Economic Behavior & Organization, Elsevier, vol. 202(C), pages 714-732.
    9. Cloos, Janis & Greiff, Matthias & Rusch, Hannes, 2020. "Geographical Concentration and Editorial Favoritism within the Field of Laboratory Experimental Economics (RM/19/029-revised-)," Research Memorandum 014, Maastricht University, Graduate School of Business and Economics (GSBE).
    10. Rui Dai & Lawrence Donohue & Qingyi (Freda) Drechsler & Wei Jiang, 2023. "Dissemination, Publication, and Impact of Finance Research: When Novelty Meets Conventionality," Review of Finance, European Finance Association, vol. 27(1), pages 79-141.
    11. Cloos, Janis & Greiff, Matthias & Rusch, Hannes, 2019. "Geographical Concentration and Editorial Favoritism within the Field of Laboratory Experimental Economics," Research Memorandum 029, Maastricht University, Graduate School of Business and Economics (GSBE).
    12. Damien Besancenot & Joao R. Faria & Kim Van Huynh, 2009. "Congestion in academic journals under an impartial selection process," Working Papers halshs-00382585, HAL.
    13. Ofer Azar, 2003. "Rejections and the Importance of First Response Times (Or: How Many Rejections Do Others Receive?)," General Economics and Teaching 0309002, University Library of Munich, Germany.
    14. Ofer H. Azar, 2005. "The Review Process in Economics: Is It Too Fast?," Southern Economic Journal, John Wiley & Sons, vol. 72(2), pages 482-491, October.
    15. Ali Sina Önder & Sergey V. Popov & Sascha Schweitzer, 2021. "Leadership in Scholarship: Editors’ Appointments and the Profession’s Narrative," Working Papers in Economics & Finance 2021-05, University of Portsmouth, Portsmouth Business School, Economics and Finance Subject Group.
    16. Lokman Tutuncu, 2023. "All-pervading insider bias alters review time in Turkish university journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(6), pages 3743-3791, June.
    17. Raffaele Miniaci & Michele Pezzoni, 2020. "Social connections and editorship in economics," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 53(3), pages 1292-1317, August.
    18. Hengel, E., 2017. "Publishing while Female. Are women held to higher standards? Evidence from peer review," Cambridge Working Papers in Economics 1753, Faculty of Economics, University of Cambridge.
    19. Azar Ofer H., 2015. "A Model of the Academic Review Process with Informed Authors," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 15(2), pages 865-889, April.
    20. Onder, Ali Sina & Popov, Sergey V & Schweitzer, Sascha, 2018. "Leadership in Scholarship: Editors' Influence on the Profession's Narrative," Cardiff Economics Working Papers E2018/2, Cardiff University, Cardiff Business School, Economics Section.

    More about this item

    JEL classification:

    • A11 - General Economics and Teaching - - General Economics - - - Role of Economics; Role of Economists
    • O3 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:23282. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.