IDEAS home Printed from https://ideas.repec.org/a/tpr/restat/v101y2019i3p468-475.html
   My bibliography  Save this article

Estimating the Impacts of Program Benefits: Using Instrumental Variables with Underreported and Imputed Data

Author

Listed:
  • Melvin Stephens

    (University of Michigan and NBER)

  • Takashi Unayama

    (Hitotsubashi University)

Abstract

Survey nonresponse has risen in recent years, which has increased the share of imputed and underreported values found on commonly used data sets. While this trend has been well documented for earnings, the growth in nonresponse to government transfers questions has received far less attention. We demonstrate analytically that the underreporting and imputation of transfer benefits can lead to program impact estimates that are substantially overstated when using instrumental variables methods to correct for endogeneity or measurement error in benefit amounts. We document the importance of failing to account for these issues using two empirical examples.

Suggested Citation

  • Melvin Stephens & Takashi Unayama, 2019. "Estimating the Impacts of Program Benefits: Using Instrumental Variables with Underreported and Imputed Data," The Review of Economics and Statistics, MIT Press, vol. 101(3), pages 468-475, July.
    • Handle: RePEc:tpr:restat:v:101:y:2019:i:3:p:468-475
    as

    Download full text from publisher

    File URL: https://www.mitpressjournals.org/doi/pdf/10.1162/rest_a_00769
    Download Restriction: Access to PDF is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Zeldes, Stephen P, 1989. "Consumption and Liquidity Constraints: An Empirical Investigation," Journal of Political Economy, University of Chicago Press, vol. 97(2), pages 305-346, April.
    2. Charles F. Manski, 1997. "Monotone Treatment Response," Econometrica, Econometric Society, vol. 65(6), pages 1311-1334, November.
    3. Lillard, Lee & Smith, James P & Welch, Finis, 1986. "What Do We Really Know about Wages? The Importance of Nonreporting and Census Imputation," Journal of Political Economy, University of Chicago Press, vol. 94(3), pages 489-506, June.
    4. Patrick Kline & Andres Santos, 2013. "Sensitivity to missing data assumptions: Theory and an evaluation of the U.S. wage structure," Quantitative Economics, Econometric Society, vol. 4(2), pages 231-267, July.
    5. Bound, John & Krueger, Alan B, 1991. "The Extent of Measurement Error in Longitudinal Earnings Data: Do Two Wrongs Make a Right?," Journal of Labor Economics, University of Chicago Press, vol. 9(1), pages 1-24, January.
    6. Melvin Stephens Jr., 2003. ""3rd of tha Month": Do Social Security Recipients Smooth Consumption Between Checks?," American Economic Review, American Economic Association, vol. 93(1), pages 406-422, March.
    7. Gary V. Engelhardt & Jonathan Gruber & Cynthia D. Perry, 2005. "Social Security and Elderly Living Arrangements: Evidence from the Social Security Notch," Journal of Human Resources, University of Wisconsin Press, vol. 40(2).
    8. Thomas J. Kane & Cecilia Elena Rouse & Douglas Staiger, 1999. "Estimating Returns to Schooling When Schooling is Misreported," NBER Working Papers 7235, National Bureau of Economic Research, Inc.
    9. James J. Heckman & Paul A. LaFontaine, 2006. "Bias-Corrected Estimates of GED Returns," Journal of Labor Economics, University of Chicago Press, vol. 24(3), pages 661-700, July.
    10. Christopher R. Bollinger & Barry T. Hirsch, 2006. "Match Bias from Earnings Imputation in the Current Population Survey: The Case of Imperfect Matching," Journal of Labor Economics, University of Chicago Press, vol. 24(3), pages 483-520, July.
    11. Melvin Stephens & Takashi Unayama, 2011. "The Consumption Response to Seasonal Income: Evidence from Japanese Public Pension Benefits," American Economic Journal: Applied Economics, American Economic Association, vol. 3(4), pages 86-118, October.
    12. Mogstad, M. & Wiswall, M., 2012. "Instrumental variables estimation with partially missing instruments," Economics Letters, Elsevier, vol. 114(2), pages 186-189.
    13. Brent Kreider & John V. Pepper & Craig Gundersen & Dean Jolliffe, 2012. "Identifying the Effects of SNAP (Food Stamps) on Child Health Outcomes When Participation Is Endogenous and Misreported," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(499), pages 958-975, September.
    14. Currie, Janet, 2004. "The Take-Up of Social Benefits," IZA Discussion Papers 1103, Institute of Labor Economics (IZA).
    15. Giovanni Mastrobuoni & Matthew Weinberg, 2009. "Heterogeneity in Intra-monthly Consumption Patterns, Self-Control, and Savings at Retirement," American Economic Journal: Economic Policy, American Economic Association, vol. 1(2), pages 163-189, August.
    16. Bruce D. Meyer & Wallace K. C. Mok & James X. Sullivan, 2009. "The Under-Reporting of Transfers in Household Surveys: Its Nature and Consequences," NBER Working Papers 15181, National Bureau of Economic Research, Inc.
    17. Krueger, Alan B. & Meyer, Bruce D., 2002. "Labor supply effects of social insurance," Handbook of Public Economics, in: A. J. Auerbach & M. Feldstein (ed.), Handbook of Public Economics, edition 1, volume 4, chapter 33, pages 2327-2392, Elsevier.
    18. Barry T. Hirsch & Edward J. Schumacher, 2004. "Match Bias in Wage Gap Estimates Due to Earnings Imputation," Journal of Labor Economics, University of Chicago Press, vol. 22(3), pages 689-722, July.
    19. Shapiro, Jesse M., 2005. "Is there a daily discount rate? Evidence from the food stamp nutrition cycle," Journal of Public Economics, Elsevier, vol. 89(2-3), pages 303-325, February.
    20. Wilcox, David W, 1989. "Social Security Benefits, Consumption Expenditure, and the Life Cycle Hypothesis," Journal of Political Economy, University of Chicago Press, vol. 97(2), pages 288-304, April.
    21. Card, David, 1996. "The Effect of Unions on the Structure of Wages: A Longitudinal Analysis," Econometrica, Econometric Society, vol. 64(4), pages 957-979, July.
    22. Melvin Stephens Jr & Takashi Unayama, 2015. "Child Benefit Payments and Household Wealth Accumulation," The Japanese Economic Review, Japanese Economic Association, vol. 66(4), pages 447-465, December.
    23. Esmeralda A. Ramalho & Richard J. Smith, 2013. "Discrete Choice Non-Response," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 80(1), pages 343-364.
    24. Gong Tang, 2003. "Analysis of multivariate missing data with nonignorable nonresponse," Biometrika, Biometrika Trust, vol. 90(4), pages 747-764, December.
    25. Nicholas S. Souleles, 1999. "The Response of Household Consumption to Income Tax Refunds," American Economic Review, American Economic Association, vol. 89(4), pages 947-958, September.
    26. Alberto Abadie & Guido W. Imbens, 2012. "A Martingale Representation for Matching Estimators," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(498), pages 833-843, June.
    27. Wim Van Lancker & Joris Ghysels & Bea Cantillon, 2012. "An international comparison of the impact of child benefits on poverty outcomes for single mothers," Working Papers 1203, Herman Deleeck Centre for Social Policy, University of Antwerp.
    28. Jonathan A. Parker, 1999. "The Reaction of Household Consumption to Predictable Changes in Social Security Taxes," American Economic Review, American Economic Association, vol. 89(4), pages 959-973, September.
    29. Charles F. Manski & John V. Pepper, 2000. "Monotone Instrumental Variables, with an Application to the Returns to Schooling," Econometrica, Econometric Society, vol. 68(4), pages 997-1012, July.
    30. Thomas J. Kane & Cecilia Rouse & Douglas Staiger, 1999. "Estimating Returns to Schooling When Schooling is Misreported," Working Papers 798, Princeton University, Department of Economics, Industrial Relations Section..
    31. Stephens Melvin, 2006. "Paycheque Receipt and the Timing of Consumption," Economic Journal, Royal Economic Society, vol. 116(513), pages 680-701, July.
    32. Bound, John & Brown, Charles & Duncan, Greg J & Rodgers, Willard L, 1994. "Evidence on the Validity of Cross-Sectional and Longitudinal Labor Market Data," Journal of Labor Economics, University of Chicago Press, vol. 12(3), pages 345-368, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Nguimkeu, Pierre & Denteh, Augustine & Tchernis, Rusty, 2019. "On the estimation of treatment effects with endogenous misreporting," Journal of Econometrics, Elsevier, vol. 208(2), pages 487-506.
    2. Padmaja Ayyagari & David Frisvold, 2016. "The Impact of Social Security Income on Cognitive Function at Older Ages Full Access," American Journal of Health Economics, MIT Press, vol. 2(4), pages 463-488, Fall.
    3. Chloe N. East & Andrew I. Friedson, 2018. "An Apple a Day? Adult Food Stamp Eligibility and Health Care Utilization among Immigrants," Upjohn Working Papers 19-295, W.E. Upjohn Institute for Employment Research.
    4. Martha J. Bailey & Connor Cole & Morgan Henderson & Catherine Massey, 2020. "How Well Do Automated Linking Methods Perform? Lessons from US Historical Data," Journal of Economic Literature, American Economic Association, vol. 58(4), pages 997-1044, December.
    5. Pettersson-Lidbom, Per, 2020. "Exit, Voice and Political Change: Evidence from Swedish Mass Migration to the United States A Comment," Research Papers in Economics 2020:3, Stockholm University, Department of Economics, revised 20 Sep 2020.
    6. Tommasi, Denni & Zhang, Lina, 2024. "Bounding program benefits when participation is misreported," Journal of Econometrics, Elsevier, vol. 238(1).
    7. Helen H. Jensen & Brent Kreider & Oleksandr Zhylyevskyy, 2019. "Investigating Treatment Effects of Participating Jointly in SNAP and WIC when the Treatment Is Validated Only for SNAP," Southern Economic Journal, John Wiley & Sons, vol. 86(1), pages 124-155, July.
    8. Melvin Stephens Jr & Takashi Unayama, 2015. "Child Benefit Payments and Household Wealth Accumulation," The Japanese Economic Review, Japanese Economic Association, vol. 66(4), pages 447-465, December.
    9. Zachary Ward, 2023. "Intergenerational Mobility in American History: Accounting for Race and Measurement Error," American Economic Review, American Economic Association, vol. 113(12), pages 3213-3248, December.
    10. UNAYAMA Takashi & KOMURA Norihiro & HATTORI Takahiro, 2021. "Impacts of Cash Transfers on Consumption during the COVID-19 Pandemic: Evidence from Japanese Special Cash Payment (Japanese)," Discussion Papers (Japanese) 21022, Research Institute of Economy, Trade and Industry (RIETI).
    11. Padmaja Ayyagari & David Frisvold, 2015. "The Impact of Social Security Income on Cognitive Function at Older Ages," NBER Working Papers 21484, National Bureau of Economic Research, Inc.
    12. Melvin Stephens, Jr. & Desmond J. Toohey, 2018. "The Impact of Health on Labor Market Outcomes: Experimental Evidence from MRFIT," NBER Working Papers 24231, National Bureau of Economic Research, Inc.
    13. Cameron LAPOINT & UNAYAMA Takashi, 2020. "Winners, Losers, and Near-Rationality: Heterogeneity in the MPC out of a Large Stimulus Tax Rebate," Discussion papers 20067, Research Institute of Economy, Trade and Industry (RIETI).
    14. HATTORI Takahiro & KOMURA Norihiro & UNAYAMA Takashi, 2021. "Impact of Cash Transfers on Consumption during the COVID-19 Pandemic: Evidence from Japanese Special Cash Payments," Discussion papers 21043, Research Institute of Economy, Trade and Industry (RIETI).
    15. Karadja, Mounir & Prawitz, Erik, 2020. "A response to Pettersson-Lidbom’s “Exit, Voice and Political Change: Evidence from Swedish Mass Migration to the United States – a Comment”," Working Paper Series 2020:5, Uppsala University, Department of Economics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fuchs-Schündeln, N. & Hassan, T.A., 2016. "Natural Experiments in Macroeconomics," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 923-1012, Elsevier.
    2. Andersson, Elvira & Lundborg, Petter & Vikström, Johan, 2015. "Income receipt and mortality — Evidence from Swedish public sector employees," Journal of Public Economics, Elsevier, vol. 131(C), pages 21-32.
    3. Melvin Stephens & Takashi Unayama, 2011. "The Consumption Response to Seasonal Income: Evidence from Japanese Public Pension Benefits," American Economic Journal: Applied Economics, American Economic Association, vol. 3(4), pages 86-118, October.
    4. Somville, Vincent & Vandewalle, Lore, 2023. "Access to banking, savings and consumption smoothing in rural India," Journal of Public Economics, Elsevier, vol. 223(C).
    5. Evans, William N. & Moore, Timothy J., 2011. "The short-term mortality consequences of income receipt," Journal of Public Economics, Elsevier, vol. 95(11), pages 1410-1424.
    6. Frederik Plesner Lyngse, 2020. "Liquidity Constraints and Demand for Healthcare: Evidence from Danish Welfare Recipients," Papers 2010.14651, arXiv.org.
    7. Frederik Plesner Lyngse, 2020. "Liquidity Constraints and Demand for Healthcare: Evidence from Danish Welfare Recipients," CEBI working paper series 20-28, University of Copenhagen. Department of Economics. The Center for Economic Behavior and Inequality (CEBI).
    8. Marks, Mindy & Prina, Silvia & Gernhardt, Roy, 2023. "Government Shutdown and SNAP Disbursements: Effects on Household Expenditures," IZA Discussion Papers 16452, Institute of Labor Economics (IZA).
    9. Jeremy J. Nalewaik, 2014. "Missing Variation in the Great Moderation: Lack of Signal Error and OLS Regression," Finance and Economics Discussion Series 2014-27, Board of Governors of the Federal Reserve System (U.S.).
    10. Hori Masahiro & Shimizutani Satoshi, 2009. "The Response of Household Expenditure to Anticipated Income Changes: Bonus Payments and the Seasonality of Consumption in Japan," The B.E. Journal of Macroeconomics, De Gruyter, vol. 9(1), pages 1-22, August.
    11. William N. Evans & Timothy J. Moore, 2009. "Liquidity, Activity, Mortality," NBER Working Papers 15310, National Bureau of Economic Research, Inc.
    12. Bollinger, Christopher R. & Hirsch, Barry & Hokayem, Charles M. & Ziliak, James P., 2018. "Trouble in the Tails? What We Know about Earnings Nonresponse Thirty Years after Lillard, Smith, and Welch," IZA Discussion Papers 11710, Institute of Labor Economics (IZA).
    13. Christopher R. Bollinger & Barry T. Hirsch, 2010. "GDP & Beyond – die europäische Perspektive," RatSWD Working Papers 165, German Data Forum (RatSWD).
    14. Arna Vardardottir & Michaela Pagel, 2016. "The Liquid Hand-to-Mouth: Evidence from a Personal Finance Management Software," 2016 Meeting Papers 789, Society for Economic Dynamics.
    15. Sumit Agarwal & Chunlin Liu & Nicholas S. Souleles, 2007. "The Reaction of Consumer Spending and Debt to Tax Rebates-Evidence from Consumer Credit Data," Journal of Political Economy, University of Chicago Press, vol. 115(6), pages 986-1019, December.
    16. Macchiavello, Rocco & Casaburi, Lorenzo, 2015. "Firm and Market Response to Saving Constraints: Evidence from the Kenyan Dairy Industry," CEPR Discussion Papers 10952, C.E.P.R. Discussion Papers.
    17. Daniel Wilhelm, 2018. "Testing for the presence of measurement error," CeMMAP working papers CWP45/18, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    18. Milkman, Katherine L. & Beshears, John, 2009. "Mental accounting and small windfalls: Evidence from an online grocer," Journal of Economic Behavior & Organization, Elsevier, vol. 71(2), pages 384-394, August.
    19. Momi Dahan, 2021. "Poverty and economic behavior: gambling on social security paydays," International Gambling Studies, Taylor & Francis Journals, vol. 21(1), pages 38-58, January.
    20. Satoshi Shimizutani, 2017. "College tuition payment and household consumption in Japan," Review of Economics of the Household, Springer, vol. 15(1), pages 265-285, March.

    More about this item

    JEL classification:

    • C80 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - General
    • E21 - Macroeconomics and Monetary Economics - - Consumption, Saving, Production, Employment, and Investment - - - Consumption; Saving; Wealth
    • H53 - Public Economics - - National Government Expenditures and Related Policies - - - Government Expenditures and Welfare Programs
    • H55 - Public Economics - - National Government Expenditures and Related Policies - - - Social Security and Public Pensions

    Lists

    This item is featured on the following reading lists, Wikipedia, or ReplicationWiki pages:
    1. Estimating the Impacts of Program Benefits: Using Instrumental Variables with Underreported and Imputed Data (REStat 2019) in ReplicationWiki

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tpr:restat:v:101:y:2019:i:3:p:468-475. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Kelly McDougall (email available below). General contact details of provider: https://direct.mit.edu/journals .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.