IDEAS home Printed from https://ideas.repec.org/p/ags/aaea21/314016.html
   My bibliography  Save this paper

Towards Data-driven Project design: Providing Optimal Treatment Rules for Development Projects

Author

Listed:
  • Garbero, Alessandra
  • Sakos, Grayson
  • Cerulli, Giovanni

Abstract

It is increasingly commonplace among development practitioners to employ impact evaluations to measure the performance and effectiveness of their activities and investments. However, there remains a substantial gap between the evidence generated by these assessments and its use for prospective planning and design of future development projects. Specifically, the retrospective data generated in counterfactual-based evaluations on what worked, how, and for whom, is not routinely or easily applied by policymakers for decision-making after an evaluation's closure. We address this gap through the development of an optimal policy learning (OPL) tool for rural development projects that leverages observational data to drive data-driven project design through identification of welfare-maximizing targeting and selection rules that can maximize project impacts. In so doing, we solve the policymaker's policy assignment problem, i.e. deciding who to treat and where. Further, we define distinct roles for the policymaker and the analyst in which the latter is tasked with generating a menu of potential selection rules while the former weighs each rule's costs and benefits against their objective function, addressing the practical constraints poised by optimal policy learning's use for project design. To illustrate the utility of our approach we apply OPL to two projects funded and evaluated by the International Fund for Agricultural Development (IFAD). We show that OPL and this division of labor not only identifies the welfare-maximizing policy assignment but also allows policymakers to gain deeper insights into the trade-offs, costs, and benefits of different objectives, policies, and demands facilitating more informed decision-making and more effective policies and development interventions.
(This abstract was borrowed from another version of this item.)

Suggested Citation

  • Garbero, Alessandra & Sakos, Grayson & Cerulli, Giovanni, 2021. "Towards Data-driven Project design: Providing Optimal Treatment Rules for Development Projects," 2021 Annual Meeting, August 1-3, Austin, Texas 314016, Agricultural and Applied Economics Association.
    • Handle: RePEc:ags:aaea21:314016
    DOI: 10.22004/ag.econ.314016
    as

    Download full text from publisher

    File URL: https://ageconsearch.umn.edu/record/314016/files/Abstracts_21_07_01_15_37_10_49__93_32_55_84_0.pdf
    Download Restriction: no
    File URL: https://libkey.io/10.22004/ag.econ.314016?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Zhengyuan Zhou & Susan Athey & Stefan Wager, 2023. "Offline Multi-Action Policy Learning: Generalization and Optimization," Operations Research, INFORMS, vol. 71(1), pages 148-183, January.
    2. Susan Athey & Stefan Wager, 2021. "Policy Learning With Observational Data," Econometrica, Econometric Society, vol. 89(1), pages 133-161, January.
    3. Stoye, Jörg, 2009. "Minimax regret treatment choice with finite samples," Journal of Econometrics, Elsevier, vol. 151(1), pages 70-81, July.
    4. Jon Kleinberg & Jens Ludwig & Sendhil Mullainathan & Ziad Obermeyer, 2015. "Prediction Policy Problems," American Economic Review, American Economic Association, vol. 105(5), pages 491-495, May.
    5. Booysen, Frikkie & van der Berg, Servaas & Burger, Ronelle & Maltitz, Michael von & Rand, Gideon du, 2008. "Using an Asset Index to Assess Trends in Poverty in Seven Sub-Saharan African Countries," World Development, Elsevier, vol. 36(6), pages 1113-1130, June.
    6. Charles F. Manski, 2004. "Statistical Treatment Rules for Heterogeneous Populations," Econometrica, Econometric Society, vol. 72(4), pages 1221-1246, July.
    7. Keisuke Hirano & Jack R. Porter, 2009. "Asymptotics for Statistical Treatment Rules," Econometrica, Econometric Society, vol. 77(5), pages 1683-1701, September.
    8. Djebbari, Habiba & Smith, Jeffrey, 2008. "Heterogeneous impacts in PROGRESA," Journal of Econometrics, Elsevier, vol. 145(1-2), pages 64-80, July.
    9. Dehejia, Rajeev H., 2005. "Program evaluation as a decision problem," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 141-173.
    10. Linden McBride & Austin Nichols, 2018. "Retooling Poverty Targeting Using Out-of-Sample Validation and Machine Learning," The World Bank Economic Review, World Bank, vol. 32(3), pages 531-550.
    11. Aaron Chalfin & Oren Danieli & Andrew Hillis & Zubin Jelveh & Michael Luca & Jens Ludwig & Sendhil Mullainathan, 2016. "Productivity and Selection of Human Capital with Machine Learning," American Economic Review, American Economic Association, vol. 106(5), pages 124-127, May.
    12. Toru Kitagawa & Aleksey Tetenov, 2018. "Who Should Be Treated? Empirical Welfare Maximization Methods for Treatment Choice," Econometrica, Econometric Society, vol. 86(2), pages 591-616, March.
    13. Brown,Elizabeth Denison & Tanner,Jeffery, 2019. "Integrating Value for Money and Impact Evaluations : Issues, Institutions, and Opportunities," Policy Research Working Paper Series 9041, The World Bank.
    14. Lechner, Michael & Smith, Jeffrey, 2007. "What is the value added by caseworkers?," Labour Economics, Elsevier, vol. 14(2), pages 135-151, April.
    15. Bhattacharya, Debopam & Dupas, Pascaline, 2012. "Inferring welfare maximizing treatment assignment under budget constraints," Journal of Econometrics, Elsevier, vol. 167(1), pages 168-196.
    16. Ho, Daniel E. & Imai, Kosuke & King, Gary & Stuart, Elizabeth A., 2007. "Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference," Political Analysis, Cambridge University Press, vol. 15(3), pages 199-236, July.
    17. Tetenov, Aleksey, 2012. "Statistical treatment choice based on asymmetric minimax regret criteria," Journal of Econometrics, Elsevier, vol. 166(1), pages 157-165.
    18. Andini, Monica & Ciani, Emanuele & de Blasio, Guido & D'Ignazio, Alessio & Salvestrini, Viola, 2018. "Targeting with machine learning: An application to a tax rebate program in Italy," Journal of Economic Behavior & Organization, Elsevier, vol. 156(C), pages 86-102.
    19. Manski, Charles F., 2000. "Identification problems and decisions under ambiguity: Empirical analysis of treatment response and normative analysis of treatment choice," Journal of Econometrics, Elsevier, vol. 95(2), pages 415-442, April.
    20. Jon Kleinberg & Himabindu Lakkaraju & Jure Leskovec & Jens Ludwig & Sendhil Mullainathan, 2018. "Human Decisions and Machine Predictions," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(1), pages 237-293.
    21. Bianca Potì & Giovanni Cerulli, 2011. "Evaluation of firm R&D and innovation support: new indicators and the ex-ante prediction of ex-post additionality," Research Evaluation, Oxford University Press, vol. 20(1), pages 19-29, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Toru Kitagawa & Sokbae Lee & Chen Qiu, 2022. "Treatment Choice with Nonlinear Regret," Papers 2205.08586, arXiv.org, revised Oct 2024.
    2. Kitagawa, Toru & Wang, Guanyi, 2023. "Who should get vaccinated? Individualized allocation of vaccines over SIR network," Journal of Econometrics, Elsevier, vol. 232(1), pages 109-131.
    3. Shosei Sakaguchi, 2021. "Estimation of Optimal Dynamic Treatment Assignment Rules under Policy Constraints," Papers 2106.05031, arXiv.org, revised Aug 2024.
    4. Yuya Sasaki & Takuya Ura, 2020. "Welfare Analysis via Marginal Treatment Effects," Papers 2012.07624, arXiv.org.
    5. Toru Kitagawa & Guanyi Wang, 2020. "Who Should Get Vaccinated? Individualized Allocation of Vaccines Over SIR Network," Papers 2012.04055, arXiv.org, revised Jul 2021.
    6. Susan Athey & Stefan Wager, 2021. "Policy Learning With Observational Data," Econometrica, Econometric Society, vol. 89(1), pages 133-161, January.
    7. Davide Viviano, 2019. "Policy Targeting under Network Interference," Papers 1906.10258, arXiv.org, revised Apr 2024.
    8. Takuya Ishihara & Toru Kitagawa, 2021. "Evidence Aggregation for Treatment Choice," Papers 2108.06473, arXiv.org, revised Jul 2024.
    9. Toru Kitagawa & Guanyi Wang, 2020. "Who should get vaccinated? Individualized allocation of vaccines over SIR network," CeMMAP working papers CWP59/20, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    10. Toru Kitagawa & Guanyi Wang, 2021. "Who should get vaccinated? Individualized allocation of vaccines over SIR network," CeMMAP working papers CWP28/21, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    11. Toru Kitagawa & Aleksey Tetenov, 2021. "Equality-Minded Treatment Choice," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 39(2), pages 561-574, March.
    12. Juliano Assunção & Robert McMillan & Joshua Murphy & Eduardo Souza-Rodrigues, 2019. "Optimal Environmental Targeting in the Amazon Rainforest," NBER Working Papers 25636, National Bureau of Economic Research, Inc.
    13. Takanori Ida & Takunori Ishihara & Koichiro Ito & Daido Kido & Toru Kitagawa & Shosei Sakaguchi & Shusaku Sasaki, 2022. "Choosing Who Chooses: Selection-Driven Targeting in Energy Rebate Programs," NBER Working Papers 30469, National Bureau of Economic Research, Inc.
    14. Eric Mbakop & Max Tabord‐Meehan, 2021. "Model Selection for Treatment Choice: Penalized Welfare Maximization," Econometrica, Econometric Society, vol. 89(2), pages 825-848, March.
    15. Toru Kitagawa & Shosei Sakaguchi & Aleksey Tetenov, 2021. "Constrained Classification and Policy Learning," Papers 2106.12886, arXiv.org, revised Jul 2023.
    16. Anders Bredahl Kock & Martin Thyrsgaard, 2017. "Optimal sequential treatment allocation," Papers 1705.09952, arXiv.org, revised Aug 2018.
    17. Firpo, Sergio & Galvao, Antonio F. & Kobus, Martyna & Parker, Thomas & Rosa-Dias, Pedro, 2020. "Loss Aversion and the Welfare Ranking of Policy Interventions," IZA Discussion Papers 13176, Institute of Labor Economics (IZA).
    18. Kohei Yata, 2021. "Optimal Decision Rules Under Partial Identification," Papers 2111.04926, arXiv.org, revised Aug 2023.
    19. Takanori Ida & Takunori Ishihara & Koichiro Ito & Daido Kido & Toru Kitagawa & Shosei Sakaguchi & Shusaku Sasaki, 2021. "Paternalism, Autonomy, or Both? Experimental Evidence from Energy Saving Programs," Papers 2112.09850, arXiv.org.
    20. Kirill Ponomarev & Vira Semenova, 2024. "On the Lower Confidence Band for the Optimal Welfare," Papers 2410.07443, arXiv.org, revised Oct 2024.

    More about this item

    Keywords

    Research Methods/Statistical Methods; Agricultural and Food Policy; International Development;
    All these keywords.

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ags:aaea21:314016. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: AgEcon Search (email available below). General contact details of provider: https://edirc.repec.org/data/aaeaaea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.