IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/12817.html
   My bibliography  Save this paper

Test-Based Accountability and Student Achievement: An Investigation of Differential Performance on NAEP and State Assessments

Author

Listed:
  • Brian A. Jacob

Abstract

This paper explores the phenomenon referred to as test score inflation, which occurs when achievement gains on "high-stakes" exams outpace improvements on "low-stakes" tests. The first part of the paper documents the extent to which student performance trends on state assessments differ from those on the National Assessment of Educational Progress (NAEP). I find evidence of considerable test score inflation in several different states, including those with quite different state testing systems. The second part of the paper is a case study of Texas that uses detailed item-level data from the Texas Assessment of Academic Skills (TAAS) and the NAEP to explore why performance trends differed across these exams during the 1990s. I find that the differential improvement on the TAAS cannot be explained by several important differences across the exams (e.g., the NAEP includes open-response items, many NAEP multiple-choice items require/permit the use of calculators, rulers, protractors or other manipulative). I find that skill and format differences across exams explain the disproportionate improvement in the TAAS for fourth graders, although these differences cannot explain the time trends for eighth graders.

Suggested Citation

  • Brian A. Jacob, 2007. "Test-Based Accountability and Student Achievement: An Investigation of Differential Performance on NAEP and State Assessments," NBER Working Papers 12817, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:12817
    Note: CH ED LS
    as

    Download full text from publisher

    File URL: https://www.nber.org/papers/w12817.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Edward L. Glaeser & Andrei Shleifer, 2001. "A Reason for Quantity Regulation," American Economic Review, American Economic Association, vol. 91(2), pages 431-435, May.
    2. Brian A. Jacob & Steven D. Levitt, 2003. "Rotten Apples: An Investigation of the Prevalence and Predictors of Teacher Cheating," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 118(3), pages 843-877.
    3. Figlio, David N., 2006. "Testing, crime and punishment," Journal of Public Economics, Elsevier, vol. 90(4-5), pages 837-851, May.
    4. Richards, Craig E. & Sheu, Tian Ming, 1992. "The South Carolina school incentive reward program: A policy analysis," Economics of Education Review, Elsevier, vol. 11(1), pages 71-86, March.
    5. Holmstrom, Bengt & Milgrom, Paul, 1991. "Multitask Principal-Agent Analyses: Incentive Contracts, Asset Ownership, and Job Design," The Journal of Law, Economics, and Organization, Oxford University Press, vol. 7(0), pages 24-52, Special I.
    6. Julie Berry Cullen & Randall Reback, 2006. "Tinkering Toward Accolades: School Gaming Under a Performance Accountability System," NBER Working Papers 12286, National Bureau of Economic Research, Inc.
    7. Figlio, David N. & Winicki, Joshua, 2005. "Food for thought: the effects of school accountability plans on school nutrition," Journal of Public Economics, Elsevier, vol. 89(2-3), pages 381-394, February.
    8. Lillard, Dean R. & DeCicca, Philip P., 2001. "Higher standards, more dropouts? Evidence within and across time," Economics of Education Review, Elsevier, vol. 20(5), pages 459-473, October.
    9. David N. Figlio & Lawrence S. Getzler, 2002. "Accountability , Ability and Disability: Gaming the System," NBER Working Papers 9307, National Bureau of Economic Research, Inc.
    10. Brian A. Jacob, 2002. "Accountability, Incentives and Behavior: The Impact of High-Stakes Testing in the Chicago Public Schools," NBER Working Papers 8968, National Bureau of Economic Research, Inc.
    11. Daniel M. Koretz, 2002. "Limitations in the Use of Achievement Tests as Measures of Educators' Productivity," Journal of Human Resources, University of Wisconsin Press, vol. 37(4), pages 752-777.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gregg, John J. & Lavertu, Stéphane, 2023. "Test-based accountability and educational equity: Breaking through local district politics?," Economics of Education Review, Elsevier, vol. 97(C).
    2. Richardson, J.T., 2015. "Accountability incentives and academic achievement: Distributional impacts of accountability when standards are set low," Economics of Education Review, Elsevier, vol. 44(C), pages 1-16.
    3. Fitzpatrick, Maria D. & Grissmer, David & Hastedt, Sarah, 2011. "What a difference a day makes: Estimating daily learning gains during kindergarten and first grade using a natural experiment," Economics of Education Review, Elsevier, vol. 30(2), pages 269-279, April.
    4. Huang, Bin & Li, Bo & Walker, Ian & Zhu, Yu, 2022. "Does It Pay to Attend More Selective High Schools? Regression Discontinuity Evidence from China," IZA Discussion Papers 15756, Institute of Labor Economics (IZA).
    5. Mizala, Alejandra & Urquiola, Miguel, 2013. "School markets: The impact of information approximating schools' effectiveness," Journal of Development Economics, Elsevier, vol. 103(C), pages 313-335.
    6. Zhang, Hongliang, 2016. "Identification of treatment effects under imperfect matching with an application to Chinese elite schools," Journal of Public Economics, Elsevier, vol. 142(C), pages 56-82.
    7. Nicholas J. Sanders, 2012. "What Doesn’t Kill You Makes You Weaker: Prenatal Pollution Exposure and Educational Outcomes," Journal of Human Resources, University of Wisconsin Press, vol. 47(3), pages 826-850.
    8. Chiang, Hanley, 2009. "How accountability pressure on failing schools affects student achievement," Journal of Public Economics, Elsevier, vol. 93(9-10), pages 1045-1057, October.
    9. Alejandra Mizala & Miguel Urquiola, 2007. "Parental choice and school markets: The impact of information approximating school effectiveness," Documentos de Trabajo 239, Centro de Economía Aplicada, Universidad de Chile.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Figlio, David N. & Rouse, Cecilia Elena, 2006. "Do accountability and voucher threats improve low-performing schools?," Journal of Public Economics, Elsevier, vol. 90(1-2), pages 239-255, January.
    2. Rajashri Chakrabarti, 2013. "Vouchers, Public School Response, And The Role Of Incentives: Evidence From Florida," Economic Inquiry, Western Economic Association International, vol. 51(1), pages 500-526, January.
    3. Derek Neal, 2011. "The Design of Performance Pay in Education," NBER Working Papers 16710, National Bureau of Economic Research, Inc.
    4. Brian A. Jacob, 2002. "Accountability, Incentives and Behavior: The Impact of High-Stakes Testing in the Chicago Public Schools," NBER Working Papers 8968, National Bureau of Economic Research, Inc.
    5. Cuesta, José Ignacio & González, Felipe & Larroulet Philippi, Cristian, 2020. "Distorted quality signals in school markets," Journal of Development Economics, Elsevier, vol. 147(C).
    6. Jacob, Brian A., 2005. "Accountability, incentives and behavior: the impact of high-stakes testing in the Chicago Public Schools," Journal of Public Economics, Elsevier, vol. 89(5-6), pages 761-796, June.
    7. Chiang, Hanley, 2009. "How accountability pressure on failing schools affects student achievement," Journal of Public Economics, Elsevier, vol. 93(9-10), pages 1045-1057, October.
    8. Kane, Thomas J. & Rockoff, Jonah E. & Staiger, Douglas O., 2008. "What does certification tell us about teacher effectiveness? Evidence from New York City," Economics of Education Review, Elsevier, vol. 27(6), pages 615-631, December.
    9. Coelli, Michael & Foster, Gigi, 2024. "Unintended consequences of school accountability reforms: Public versus private schools," Economics of Education Review, Elsevier, vol. 99(C).
    10. Lombardi, María, 2019. "Is the remedy worse than the disease? The impact of teacher remediation on teacher and student performance in Chile," Economics of Education Review, Elsevier, vol. 73(C).
    11. Bertoni, Marco & Brunello, Giorgio & Rocco, Lorenzo, 2013. "When the cat is near, the mice won't play: The effect of external examiners in Italian schools," Journal of Public Economics, Elsevier, vol. 104(C), pages 65-77.
    12. Thomas J. Nechyba, 2006. "Alternative education finance strategies," Regional Economic Development, Federal Reserve Bank of St. Louis, issue Mar, pages 7-27.
    13. Rezende, Marcelo, 2010. "The effects of accountability on higher education," Economics of Education Review, Elsevier, vol. 29(5), pages 842-856, October.
    14. Paul Glewwe & Nauman Ilias & Michael Kremer, 2010. "Teacher Incentives," American Economic Journal: Applied Economics, American Economic Association, vol. 2(3), pages 205-227, July.
    15. Hege Marie Gjefsen & Trude Gunnes, 2015. "School accountability Incentives or sorting?," Discussion Papers 815, Statistics Norway, Research Department.
    16. Patricia M. Anderson & Kristin F. Butcher & Diane Whitmore Schanzenbach, 2017. "Adequate (or Adipose?) Yearly Progress: Assessing the Effect of “No Child Left Behind” on Children's Obesity," Education Finance and Policy, MIT Press, vol. 12(1), pages 54-76, Winter.
    17. Reback, Randall, 2008. "Teaching to the rating: School accountability and the distribution of student achievement," Journal of Public Economics, Elsevier, vol. 92(5-6), pages 1394-1415, June.
    18. Colleen Donovan & David N. Figlio & Mark Rush, 2006. "Cramming: The Effects of School Accountability on College-Bound Students," NBER Working Papers 12628, National Bureau of Economic Research, Inc.
    19. Barrera-Osorio, Felipe & Raju, Dhushyanth, 2017. "Teacher performance pay: Experimental evidence from Pakistan," Journal of Public Economics, Elsevier, vol. 148(C), pages 75-91.
    20. Bernal, Pedro & Martinez, Sebastian, 2020. "In-kind incentives and health worker performance: Experimental evidence from El Salvador," Journal of Health Economics, Elsevier, vol. 70(C).

    More about this item

    JEL classification:

    • I2 - Health, Education, and Welfare - - Education

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:12817. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.