IDEAS home Printed from https://ideas.repec.org/p/ces/ceswps/_8702.html
   My bibliography  Save this paper

Machine Predictions and Human Decisions with Variation in Payoffs and Skill

Author

Listed:
  • Michael Allan Ribers
  • Hannes Ullrich

Abstract

Human decision-making differs due to variation in both incentives and available information. This generates substantial challenges for the evaluation of whether and how machine learning predictions can improve decision outcomes. We propose a framework that incorporates machine learning on large-scale administrative data into a choice model featuring heterogeneity in decision maker payoff functions and predictive skill. We apply our framework to the major health policy problem of improving the efficiency in antibiotic prescribing in primary care, one of the leading causes of antibiotic resistance. Our analysis reveals large variation in physicians’ skill to diagnose bacterial infections and in how physicians trade off the externality inherent in antibiotic use against its curative benefit. Counterfactual policy simulations show the combination of machine learning predictions with physician diagnostic skill achieves a 25.4 percent reduction in prescribing and the largest welfare gains compared to alternative policies for estimated as well as plausible hypothetical weights on the antibiotic resistance externality.

Suggested Citation

  • Michael Allan Ribers & Hannes Ullrich, 2020. "Machine Predictions and Human Decisions with Variation in Payoffs and Skill," CESifo Working Paper Series 8702, CESifo.
    • Handle: RePEc:ces:ceswps:_8702
    as

    Download full text from publisher

    File URL: https://www.cesifo.org/DocDL/cesifo1_wp8702.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Aaron Chalfin & Oren Danieli & Andrew Hillis & Zubin Jelveh & Michael Luca & Jens Ludwig & Sendhil Mullainathan, 2016. "Productivity and Selection of Human Capital with Machine Learning," American Economic Review, American Economic Association, vol. 106(5), pages 124-127, May.
    2. Kwon, Illoong & Jun, Daesung, 2015. "Information disclosure and peer effects in the use of antibiotics," Journal of Health Economics, Elsevier, vol. 42(C), pages 1-16.
    3. Rachel Cassidy & Charles F. Manski, 2019. "Tuberculosis diagnosis and treatment under uncertainty," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 116(46), pages 22990-22997, November.
    4. Daniel Bennett & Che-Lun Hung & Tsai-Ling Lauderdale, 2015. "Health Care Competition and Antibiotic Use in Taiwan," Journal of Industrial Economics, Wiley Blackwell, vol. 63(2), pages 371-393, June.
    5. Jason Abaluck & Leila Agha & Chris Kabrhel & Ali Raja & Arjun Venkatesh, 2016. "The Determinants of Productivity in Medical Testing: Intensity and Allocation of Care," American Economic Review, American Economic Association, vol. 106(12), pages 3730-3764, December.
    6. David C. Chan Jr & Matthew Gentzkow & Chuan Yu, 2019. "Selection with Variation in Diagnostic Skill: Evidence from Radiologists," NBER Working Papers 26467, National Bureau of Economic Research, Inc.
    7. Hess, Stephane & Train, Kenneth E. & Polak, John W., 2006. "On the use of a Modified Latin Hypercube Sampling (MLHS) method in the estimation of a Mixed Logit Model for vehicle choice," Transportation Research Part B: Methodological, Elsevier, vol. 40(2), pages 147-163, February.
    8. Justine S. Hastings & Mark Howison & Sarah E. Inman, 2020. "Predicting high-risk opioid prescriptions before they are given," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 117(4), pages 1917-1923, January.
    9. Janet Currie & W. Bentley MacLeod, 2017. "Diagnosing Expertise: Human Capital, Decision Making, and Performance among Physicians," Journal of Labor Economics, University of Chicago Press, vol. 35(1), pages 1-43.
    10. Jishnu Das & Alaka Holla & Aakash Mohpal & Karthik Muralidharan, 2016. "Quality and Accountability in Health Care Delivery: Audit-Study Evidence from Primary Care in India," American Economic Review, American Economic Association, vol. 106(12), pages 3765-3799, December.
    11. Bo Cowgill & Megan T. Stevenson, 2020. "Algorithmic Social Engineering," AEA Papers and Proceedings, American Economic Association, vol. 110, pages 96-100, May.
    12. Currie, Janet & Lin, Wanchuan & Meng, Juanjuan, 2014. "Addressing antibiotic abuse in China: An experimental audit study," Journal of Development Economics, Elsevier, vol. 110(C), pages 39-51.
    13. Jon Kleinberg & Jens Ludwig & Sendhil Mullainathan & Ziad Obermeyer, 2015. "Prediction Policy Problems," American Economic Review, American Economic Association, vol. 105(5), pages 491-495, May.
    14. Michael Allan Ribers & Hannes Ullrich, 2019. "Battling antibiotic resistance: can machine learning improve prescribing?," CESifo Working Paper Series 7654, CESifo.
    15. Anna, Petrenko, 2016. "Мaркування готової продукції як складова частина інформаційного забезпечення маркетингової діяльності підприємств овочепродуктового підкомплексу," Agricultural and Resource Economics: International Scientific E-Journal, Agricultural and Resource Economics: International Scientific E-Journal, vol. 2(1), March.
    16. Sendhil Mullainathan & Ziad Obermeyer, 2023. "Diagnosing Physician Error: A Machine Learning Approach to Low-Value Health Care," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 137(2), pages 679-727.
    17. Susan Athey, 2018. "The Impact of Machine Learning on Economics," NBER Chapters, in: The Economics of Artificial Intelligence: An Agenda, pages 507-547, National Bureau of Economic Research, Inc.
    18. Jon Kleinberg & Himabindu Lakkaraju & Jure Leskovec & Jens Ludwig & Sendhil Mullainathan, 2018. "Human Decisions and Machine Predictions," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(1), pages 237-293.
    19. Andini, Monica & Ciani, Emanuele & de Blasio, Guido & D'Ignazio, Alessio & Salvestrini, Viola, 2018. "Targeting with machine learning: An application to a tax rebate program in Italy," Journal of Economic Behavior & Organization, Elsevier, vol. 156(C), pages 86-102.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ashesh Rambachan, 2022. "Identifying Prediction Mistakes in Observational Data," NBER Chapters, in: Economics of Artificial Intelligence, National Bureau of Economic Research, Inc.
    2. Shan Huang & Hannes Ullrich, 2021. "Physician Effects in Antibiotic Prescribing: Evidence from Physician Exits," Discussion Papers of DIW Berlin 1958, DIW Berlin, German Institute for Economic Research.
    3. Newham, Melissa & Valente, Marica, 2024. "The cost of influence: How gifts to physicians shape prescriptions and drug costs," Journal of Health Economics, Elsevier, vol. 95(C).
    4. Shan Huang & Michael Allan Ribers & Hannes Ullrich, 2021. "The Value of Data for Prediction Policy Problems: Evidence from Antibiotic Prescribing," Discussion Papers of DIW Berlin 1939, DIW Berlin, German Institute for Economic Research.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Michael A. Ribers & Hannes Ullrich, 2019. "Battling Antibiotic Resistance: Can Machine Learning Improve Prescribing?," Discussion Papers of DIW Berlin 1803, DIW Berlin, German Institute for Economic Research.
    2. Hannes Ullrich & Michael Allan Ribers, 2023. "Machine predictions and human decisions with variation in payoffs and skill: the case of antibiotic prescribing," Berlin School of Economics Discussion Papers 0027, Berlin School of Economics.
    3. Michael Allan Ribers & Hannes Ullrich, 2023. "Machine learning and physician prescribing: a path to reduced antibiotic use," Berlin School of Economics Discussion Papers 0019, Berlin School of Economics.
    4. Shan Huang & Michael Allan Ribers & Hannes Ullrich, 2021. "The Value of Data for Prediction Policy Problems: Evidence from Antibiotic Prescribing," Discussion Papers of DIW Berlin 1939, DIW Berlin, German Institute for Economic Research.
    5. Christian Posso & Jorge Tamayo & Arlen Guarin & Estefania Saravia, 2024. "Luck of the Draw: The Causal Effect of Physicians on Birth Outcomes," Borradores de Economia 1269, Banco de la Republica de Colombia.
    6. Andini, Monica & Boldrini, Michela & Ciani, Emanuele & de Blasio, Guido & D'Ignazio, Alessio & Paladini, Andrea, 2022. "Machine learning in the service of policy targeting: The case of public credit guarantees," Journal of Economic Behavior & Organization, Elsevier, vol. 198(C), pages 434-475.
    7. Garbero, Alessandra & Sakos, Grayson & Cerulli, Giovanni, 2023. "Towards data-driven project design: Providing optimal treatment rules for development projects," Socio-Economic Planning Sciences, Elsevier, vol. 89(C).
    8. Huang, Shan & Ribers, Michael Allan & Ullrich, Hannes, 2022. "Assessing the value of data for prediction policies: The case of antibiotic prescribing," Economics Letters, Elsevier, vol. 213(C).
    9. Jason Abaluck & Leila Agha & David C. Chan Jr & Daniel Singer & Diana Zhu, 2020. "Fixing Misallocation with Guidelines: Awareness vs. Adherence," NBER Working Papers 27467, National Bureau of Economic Research, Inc.
    10. de Blasio, Guido & D'Ignazio, Alessio & Letta, Marco, 2022. "Gotham city. Predicting ‘corrupted’ municipalities with machine learning," Technological Forecasting and Social Change, Elsevier, vol. 184(C).
    11. Elliott Ash & Sergio Galletta & Tommaso Giommoni, 2021. "A Machine Learning Approach to Analyze and Support Anti-Corruption Policy," CESifo Working Paper Series 9015, CESifo.
    12. Andini, Monica & Ciani, Emanuele & de Blasio, Guido & D'Ignazio, Alessio & Salvestrini, Viola, 2018. "Targeting with machine learning: An application to a tax rebate program in Italy," Journal of Economic Behavior & Organization, Elsevier, vol. 156(C), pages 86-102.
    13. Guido de Blasio & Alessio D'Ignazio & Marco Letta, 2020. "Predicting Corruption Crimes with Machine Learning. A Study for the Italian Municipalities," Working Papers 16/20, Sapienza University of Rome, DISS.
    14. Sebastian Panthöfer, 2022. "Do doctors prescribe antibiotics out of fear of malpractice?," Journal of Empirical Legal Studies, John Wiley & Sons, vol. 19(2), pages 340-381, June.
    15. McKenzie, David & Sansone, Dario, 2017. "Man vs. Machine in Predicting Successful Entrepreneurs: Evidence from a Business Plan Competition in Nigeria," CEPR Discussion Papers 12523, C.E.P.R. Discussion Papers.
    16. Battiston, Pietro & Gamba, Simona & Santoro, Alessandro, 2024. "Machine learning and the optimization of prediction-based policies," Technological Forecasting and Social Change, Elsevier, vol. 199(C).
    17. Ballestar, María Teresa & Doncel, Luis Miguel & Sainz, Jorge & Ortigosa-Blanch, Arturo, 2019. "A novel machine learning approach for evaluation of public policies: An application in relation to the performance of university researchers," Technological Forecasting and Social Change, Elsevier, vol. 149(C).
    18. Si, Yafei & Bateman, Hazel & Chen, Shu & Hanewald, Katja & Li, Bingqin & Su, Min & Zhou, Zhongliang, 2023. "Quantifying the financial impact of overuse in primary care in China: A standardised patient study," Social Science & Medicine, Elsevier, vol. 320(C).
    19. McKenzie, David & Sansone, Dario, 2019. "Predicting entrepreneurial success is hard: Evidence from a business plan competition in Nigeria," Journal of Development Economics, Elsevier, vol. 141(C).
    20. Liyang Tang, 2020. "Application of Nonlinear Autoregressive with Exogenous Input (NARX) neural network in macroeconomic forecasting, national goal setting and global competitiveness assessment," Papers 2005.08735, arXiv.org.

    More about this item

    Keywords

    prediction policy; expert decision-making; machine learning; antibiotic prescribing;
    All these keywords.

    JEL classification:

    • C10 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - General
    • C55 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Large Data Sets: Modeling and Analysis
    • I11 - Health, Education, and Welfare - - Health - - - Analysis of Health Care Markets
    • I18 - Health, Education, and Welfare - - Health - - - Government Policy; Regulation; Public Health
    • Q28 - Agricultural and Natural Resource Economics; Environmental and Ecological Economics - - Renewable Resources and Conservation - - - Government Policy

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ces:ceswps:_8702. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Klaus Wohlrabe (email available below). General contact details of provider: https://edirc.repec.org/data/cesifde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.