IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/26168.html
   My bibliography  Save this paper

Diagnosing Physician Error: A Machine Learning Approach to Low-Value Health Care

Author

Listed:
  • Sendhil Mullainathan
  • Ziad Obermeyer

Abstract

We use machine learning as a tool to study decision making, focusing specifically on how physicians diagnose heart attack. An algorithmic model of a patient’s probability of heart attack allows us to identify cases where physician testing decisions deviate from predicted risk. We then use actual health outcomes to evaluate whether those deviations represent mistakes or physicians’ superior knowledge. This approach reveals two inefficiencies. Physicians over-test: predictably low-risk patients are tested, but do not benefit. At the same time, physicians undertest: predictably high-risk patients are left untested, and then go on to suffer adverse health events including death. A natural experiment using shift-to-shift testing variation confirms these findings. Simultaneous over- and under-testing cannot easily be explained by incentives alone, and instead point to systematic errors in judgment. We provide suggestive evidence on the psychology underlying these errors. First, physicians use too simple a model of risk. Second, they overweight factors that are salient or representative of heart attack, such as chest pain. We argue health care models must incorporate physician error, and illustrate how policies focused solely on incentive problems can produce large inefficiencies.

Suggested Citation

  • Sendhil Mullainathan & Ziad Obermeyer, 2019. "Diagnosing Physician Error: A Machine Learning Approach to Low-Value Health Care," NBER Working Papers 26168, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:26168
    Note: AG EH LS PR
    as

    Download full text from publisher

    File URL: https://www.nber.org/papers/w26168.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Zarek C. Brot-Goldberg & Amitabh Chandra & Benjamin R. Handel & Jonathan T. Kolstad, 2017. "What does a Deductible Do? The Impact of Cost-Sharing on Health Care Prices, Quantities, and Spending Dynamics," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 132(3), pages 1261-1318.
    2. Sendhil Mullainathan & Ziad Obermeyer, 2017. "Does Machine Learning Automate Moral Hazard and Error?," American Economic Review, American Economic Association, vol. 107(5), pages 476-480, May.
    3. Jonathan T. Kolstad, 2013. "Information and Quality when Motivation is Intrinsic: Evidence from Surgeon Report Cards," NBER Working Papers 18804, National Bureau of Economic Research, Inc.
    4. Pedro Bordalo & Nicola Gennaioli & Andrei Shleifer, 2020. "Memory, Attention, and Choice," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 135(3), pages 1399-1442.
    5. Agrawal, Ajay & Gans, Joshua S. & Goldfarb, Avi, 2019. "Exploring the impact of artificial Intelligence: Prediction versus judgment," Information Economics and Policy, Elsevier, vol. 47(C), pages 1-6.
    6. Raj Chetty & Adam Looney & Kory Kroft, 2009. "Salience and Taxation: Theory and Evidence," American Economic Review, American Economic Association, vol. 99(4), pages 1145-1177, September.
    7. Herbert A. Simon, 1955. "A Behavioral Model of Rational Choice," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 69(1), pages 99-118.
    8. Xavier Gabaix, 2014. "A Sparsity-Based Model of Bounded Rationality," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 129(4), pages 1661-1710.
    9. Jason Abaluck & Leila Agha & Chris Kabrhel & Ali Raja & Arjun Venkatesh, 2016. "The Determinants of Productivity in Medical Testing: Intensity and Allocation of Care," American Economic Review, American Economic Association, vol. 106(12), pages 3730-3764, December.
    10. Katherine Baicker & Sendhil Mullainathan & Joshua Schwartzstein, 2015. "Behavioral Hazard in Health Insurance," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 130(4), pages 1623-1667.
    11. Sims, Christopher A., 2003. "Implications of rational inattention," Journal of Monetary Economics, Elsevier, vol. 50(3), pages 665-690, April.
    12. Amitabh Chandra & Douglas O. Staiger, 2007. "Productivity Spillovers in Health Care: Evidence from the Treatment of Heart Attacks," Journal of Political Economy, University of Chicago Press, vol. 115(1), pages 103-140.
    13. Jon Kleinberg & Jens Ludwig & Sendhil Mullainathan & Ziad Obermeyer, 2015. "Prediction Policy Problems," American Economic Review, American Economic Association, vol. 105(5), pages 491-495, May.
    14. Benjamin R. Handel & Jonathan T. Kolstad, 2015. "Health Insurance for "Humans": Information Frictions, Plan Choice, and Consumer Welfare," American Economic Review, American Economic Association, vol. 105(8), pages 2449-2500, August.
    15. Jonathan T. Kolstad, 2013. "Information and Quality When Motivation Is Intrinsic: Evidence from Surgeon Report Cards," American Economic Review, American Economic Association, vol. 103(7), pages 2875-2910, December.
    16. David C Chan & Matthew Gentzkow & Chuan Yu, 2022. "Selection with Variation in Diagnostic Skill: Evidence from Radiologists [The Determinants of Productivity in Medical Testing: Intensity and Allocation of Care]," The Quarterly Journal of Economics, Oxford University Press, vol. 137(2), pages 729-783.
    17. Jon Kleinberg & Himabindu Lakkaraju & Jure Leskovec & Jens Ludwig & Sendhil Mullainathan, 2018. "Human Decisions and Machine Predictions," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(1), pages 237-293.
    18. Agrawal, Ajay & Gans, Joshua & Goldfarb, Avi (ed.), 2019. "The Economics of Artificial Intelligence," National Bureau of Economic Research Books, University of Chicago Press, number 9780226613338, September.
    19. Sendhil Mullainathan, 2002. "A Memory-Based Model of Bounded Rationality," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 117(3), pages 735-774.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Newham, Melissa & Valente, Marica, 2024. "The cost of influence: How gifts to physicians shape prescriptions and drug costs," Journal of Health Economics, Elsevier, vol. 95(C).
    2. Michael Allan Ribers & Hannes Ullrich, 2020. "Machine Predictions and Human Decisions with Variation in Payoffs and Skill," CESifo Working Paper Series 8702, CESifo.
    3. Talia Gillis & Bryce McLaughlin & Jann Spiess, 2021. "On the Fairness of Machine-Assisted Human Decisions," Papers 2110.15310, arXiv.org, revised Sep 2023.
    4. Ity Shurtz, 2022. "Heuristic thinking in the workplace: Evidence from primary care," Health Economics, John Wiley & Sons, Ltd., vol. 31(8), pages 1713-1729, August.
    5. Max H. Farrell & Tengyuan Liang & Sanjog Misra, 2020. "Deep Learning for Individual Heterogeneity: An Automatic Inference Framework," Papers 2010.14694, arXiv.org, revised Jul 2021.
    6. Amitabh Chandra & Evan Flack & Ziad Obermeyer, 2021. "The Health Costs of Cost-Sharing," NBER Working Papers 28439, National Bureau of Economic Research, Inc.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xavier Gabaix, 2017. "Behavioral Inattention," NBER Working Papers 24096, National Bureau of Economic Research, Inc.
    2. Martin Gaynor & Kate Ho & Robert J. Town, 2015. "The Industrial Organization of Health-Care Markets," Journal of Economic Literature, American Economic Association, vol. 53(2), pages 235-284, June.
    3. Miao Liu, 2022. "Assessing Human Information Processing in Lending Decisions: A Machine Learning Approach," Journal of Accounting Research, Wiley Blackwell, vol. 60(2), pages 607-651, May.
    4. Elena Ashtari Tafti, 2022. "Technology, skills, and performance: the case of robots in surgery," IFS Working Papers W22/46, Institute for Fiscal Studies.
    5. Hannes Ullrich & Michael Allan Ribers, 2023. "Machine predictions and human decisions with variation in payoffs and skill: the case of antibiotic prescribing," Berlin School of Economics Discussion Papers 0027, Berlin School of Economics.
    6. Scott Duke Kominers & Xiaosheng Mu & Alexander Peysakhovich, 2019. "Paying for Attention: The Impact of Information Processing Costs on Bayesian Inference," Working Papers 2019-31, Princeton University. Economics Department..
    7. Erin T. Bronchetti & Judd B. Kessler & Ellen B. Magenheim & Dmitry Taubinsky & Eric Zwick, 2023. "Is Attention Produced Optimally? Theory and Evidence From Experiments With Bandwidth Enhancements," Econometrica, Econometric Society, vol. 91(2), pages 669-707, March.
    8. Raj Chetty & Adam Looney & Kory Kroft, 2009. "Salience and Taxation: Theory and Evidence," American Economic Review, American Economic Association, vol. 99(4), pages 1145-1177, September.
    9. Andrew Caplin & Daniel Martin, 2015. "A Testable Theory of Imperfect Perception," Economic Journal, Royal Economic Society, vol. 125(582), pages 184-202, February.
    10. Chen Lian, 2023. "Mistakes in Future Consumption, High MPCs Now," American Economic Review: Insights, American Economic Association, vol. 5(4), pages 563-581, December.
    11. Bauer, Kevin & Pfeuffer, Nicolas & Abdel-Karim, Benjamin M. & Hinz, Oliver & Kosfeld, Michael, 2020. "The terminator of social welfare? The economic consequences of algorithmic discrimination," SAFE Working Paper Series 287, Leibniz Institute for Financial Research SAFE.
    12. Chen Lian, 2021. "Mistakes in Future Consumption, High MPCs Now," NBER Working Papers 29517, National Bureau of Economic Research, Inc.
    13. Emmanuel Farhi & Xavier Gabaix, 2020. "Optimal Taxation with Behavioral Agents," American Economic Review, American Economic Association, vol. 110(1), pages 298-336, January.
    14. Attema, Arthur E. & Galizzi, Matteo M. & Groß, Mona & Hennig-Schmidt, Heike & Karay, Yassin & L’Haridon, Olivier & Wiesen, Daniel, 2023. "The formation of physician altruism," Journal of Health Economics, Elsevier, vol. 87(C).
    15. Kaufmann, Cornel & Müller, Tobias & Hefti, Andreas & Boes, Stefan, 2018. "Does personalized information improve health plan choices when individuals are distracted?," Journal of Economic Behavior & Organization, Elsevier, vol. 149(C), pages 197-214.
    16. Pedro Bordalo & John Conlon & Nicola Gennaioli & Spencer Kwon & Andrei Shleifer, 2023. "How People Use Statistics," Working Papers 699, IGIER (Innocenzo Gasparini Institute for Economic Research), Bocconi University.
    17. Avi Goldfarb & Mo Xiao, 2024. "Transitory shocks, limited attention, and a firm’s decision to exit," Quantitative Marketing and Economics (QME), Springer, vol. 22(3), pages 223-255, September.
    18. Anell, Anders & Dietrichson, Jens & Ellegård, Lina Maria & Kjellsson, Gustav, 2022. "Well-Informed Choices? Effects of Information Interventions in Primary Care on Care Quality," Working Papers 2022:2, Lund University, Department of Economics.
    19. Stefano DellaVigna, 2009. "Psychology and Economics: Evidence from the Field," Journal of Economic Literature, American Economic Association, vol. 47(2), pages 315-372, June.
    20. Michael Geruso & Timothy Layton, 2020. "Upcoding: Evidence from Medicare on Squishy Risk Adjustment," Journal of Political Economy, University of Chicago Press, vol. 128(3), pages 984-1026.

    More about this item

    JEL classification:

    • C55 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Large Data Sets: Modeling and Analysis
    • D8 - Microeconomics - - Information, Knowledge, and Uncertainty
    • D84 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Expectations; Speculations
    • D9 - Microeconomics - - Micro-Based Behavioral Economics
    • I1 - Health, Education, and Welfare - - Health
    • I13 - Health, Education, and Welfare - - Health - - - Health Insurance, Public and Private

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:26168. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.