IDEAS home Printed from https://ideas.repec.org/a/ect/emjrnl/v2y1999i2p167-191.html
   My bibliography  Save this article

Data mining reconsidered: encompassing and the general-to-specific approach to specification search

Author

Listed:
  • KEVIN D. HOOVER
  • STEPHEN J. PEREZ

Abstract

This paper examines the efficacy of the general-to-specific modeling approach associated with the LSE school of econometrics using a simulation framework. A mechanical algorithm is developed which mimics some aspects of the search procedures used by LSE practitioners. The algorithm is tested using 1000 replications of each of nine regression models and a data set patterned after Lovell's (1983) study of data mining. The algorithm is assessed for its ability to recover the data-generating process. Monte Carlo estimates of the size and power of exclusion tests based on t -statistics for individual variables in the specification are also provided. The roles of alternative sizes for specification tests in the algorithm, the consequences of different signal-to-noise ratios, and strategies for reducing overparameterization are also investigated. The results are largely favorable to the general-to-specific approach. In particular, the size of exclusion tests remains close to the nominal size used in the algorithm despite extensive search.

Suggested Citation

  • Kevin D. Hoover & Stephen J. Perez, 1999. "Data mining reconsidered: encompassing and the general-to-specific approach to specification search," Econometrics Journal, Royal Economic Society, vol. 2(2), pages 167-191.
    • Handle: RePEc:ect:emjrnl:v:2:y:1999:i:2:p:167-191
    as

    Download full text from publisher

    To our knowledge, this item is not available for download. To find whether it is available, there are three options:
    1. Check below whether another version of this item is available online.
    2. Check on the provider's web page whether it is in fact available.
    3. Perform a search for a similarly titled item that would be available.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Hendry, David F & Mizon, Grayham Ernest, 1985. "Procrustean Econometrics: Stretching and Squeezing Data," CEPR Discussion Papers 68, C.E.P.R. Discussion Papers.
    2. John Y. Campbell & Pierre Perron, 1991. "Pitfalls and Opportunities: What Macroeconomists Should Know about Unit Roots," NBER Chapters, in: NBER Macroeconomics Annual 1991, Volume 6, pages 141-220, National Bureau of Economic Research, Inc.
    3. Stock, James H & Watson, Mark W, 1988. "Variable Trends in Economic Time Series," Journal of Economic Perspectives, American Economic Association, vol. 2(3), pages 147-174, Summer.
    4. Hansen, Bruce E, 1996. "Methodology: Alchemy or Science: Review Article," Economic Journal, Royal Economic Society, vol. 106(438), pages 1398-1413, September.
    5. Pagan, Adrian, 1987. "Three Econometric Methodologies: A Critical Appraisal," Journal of Economic Surveys, Wiley Blackwell, vol. 1(1), pages 3-24.
    6. Gregory D. Hess & Christopher S. Jones & Richard D. Porter, 1994. "The predictive failure of the Baba, Hendry and Starr model of the demand for M1 in the United States," Research Working Paper 94-06, Federal Reserve Bank of Kansas City.
    7. P.C.B. Phillips, 1988. "Reflections on Econometric Methodology," The Economic Record, The Economic Society of Australia, vol. 64(4), pages 344-359, December.
    8. Roger D. Blair & David L. Kaserman & James T. McClave, 1987. "Viewpoint," Challenge, Taylor & Francis Journals, vol. 30(4), pages 60-64, July.
    9. Anindya Banerjee, 1994. "Dynamic Specification And Testing For Unit Roots And Co-integration," Working Paper 914, Economics Department, Queen's University.
    10. Hoover, Kevin D, 1988. "On the Pitfalls of Untested Common-Factor Restrictions: The Case of the Inverted Fisher Hypothesis," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 50(2), pages 125-138, May.
    11. Lovell, Michael C, 1983. "Data Mining," The Review of Economics and Statistics, MIT Press, vol. 65(1), pages 1-12, February.
    12. repec:bla:ecorec:v:64:y:1988:i:187:p:344-59 is not listed on IDEAS
    13. Ericsson, Neil R. & Campos, Julia & Tran, Hong-Anh, 1990. "Pc-Give and David Hendry'S Econometric Methodology," Brazilian Review of Econometrics, Sociedade Brasileira de Econometria - SBE, vol. 10(1), April.
    14. Thomas Mayer, 1992. "Truth versus precision in economics," Books, Edward Elgar Publishing, number 307.
    15. Mayer, Thomas, 1980. "Economics as a Hard Science: Realistic Goal or Wishful Thinking?," Economic Inquiry, Western Economic Association International, vol. 18(2), pages 165-178, April.
    16. Hendry, David F. & Richard, Jean-Francois, 1982. "On the formulation of empirical models in dynamic econometrics," Journal of Econometrics, Elsevier, vol. 20(1), pages 3-33, October.
    17. Dolado, Juan J & Jenkinson, Tim & Sosvilla-Rivero, Simon, 1990. "Cointegration and Unit Roots," Journal of Economic Surveys, Wiley Blackwell, vol. 4(3), pages 249-273.
    18. Hendry, D.F. & Richard, J.-F., 1987. "Recent developments in the theory of encompassing," LIDAM Discussion Papers CORE 1987022, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    19. Mizon, Grayham E & Richard, Jean-Francois, 1986. "The Encompassing Principle and Its Application to Testing Non-nested Hypotheses," Econometrica, Econometric Society, vol. 54(3), pages 657-678, May.
    20. Li, Hongyi & Maddala, G. S., 1997. "Bootstrapping cointegrating regressions," Journal of Econometrics, Elsevier, vol. 80(2), pages 297-318, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kevin D. Hoover & Stephen J. Perez, 1999. "Data mining reconsidered: encompassing and the general-to-specific approach to specification search," Econometrics Journal, Royal Economic Society, vol. 2(2), pages 167-191.
    2. Harris Dellas & Kevin Hoover, 2003. "Truth and Robustness in Cross-country Growth Regressions," Working Papers 11, University of California, Davis, Department of Economics.
    3. Julia Campos & Neil R. Ericsson & David F. Hendry, 2005. "General-to-specific modeling: an overview and selected bibliography," International Finance Discussion Papers 838, Board of Governors of the Federal Reserve System (U.S.).
    4. Adrian C. Darnell, 1994. "A Dictionary Of Econometrics," Books, Edward Elgar Publishing, number 118.
    5. Kevin D. Hoover & Stephen J. Perez, 2004. "Truth and Robustness in Cross‐country Growth Regressions," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 66(5), pages 765-798, December.
    6. Ericsson, Neil R., 1992. "Cointegration, exogeneity, and policy analysis: An overview," Journal of Policy Modeling, Elsevier, vol. 14(3), pages 251-280, June.
    7. Andrade, Isabel, 1992. "The relationship between inflation and relative price variability: A multivariate approach," Discussion Paper Series In Economics And Econometrics 9203, Economics Division, School of Social Sciences, University of Southampton.
    8. Castle Jennifer L. & Doornik Jurgen A & Hendry David F., 2011. "Evaluating Automatic Model Selection," Journal of Time Series Econometrics, De Gruyter, vol. 3(1), pages 1-33, February.
    9. Ericsson, Neil R., 1992. "Parameter constancy, mean square forecast errors, and measuring forecast performance: An exposition, extensions, and illustration," Journal of Policy Modeling, Elsevier, vol. 14(4), pages 465-495, August.
    10. Bardsen, Gunnar & Eitrheim, Oyvind & Jansen, Eilev S. & Nymoen, Ragnar, 2005. "The Econometrics of Macroeconomic Modelling," OUP Catalogue, Oxford University Press, number 9780199246502.
    11. Neil R. Ericsson, 2008. "The Fragility of Sensitivity Analysis: An Encompassing Perspective," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 70(s1), pages 895-914, December.
    12. Schlitzer, Giuseppe, 1995. "Testing the stationarity of economic time series: further Monte Carlo evidence," Ricerche Economiche, Elsevier, vol. 49(2), pages 125-144, June.
    13. Hendry, David F & Ericsson, Neil R, 1991. "An Econometric Analysis of U.K. Money Demand in 'Monetary Trends in the United States and the United Kingdom' by Milton Friedman and Anna Schwartz," American Economic Review, American Economic Association, vol. 81(1), pages 8-38, March.
    14. David F. Hendry & Neil R. Ericsson, 1999. "Encompassing and rational expectations: How sequential corroboration can imply refutation," Empirical Economics, Springer, vol. 24(1), pages 1-21.
    15. Spanos, Aris, 1989. "On Rereading Haavelmo: A Retrospective View of Econometric Modeling," Econometric Theory, Cambridge University Press, vol. 5(3), pages 405-429, December.
    16. McAleer, Michael, 1994. "Sherlock Holmes and the Search for Truth: A Diagnostic Tale," Journal of Economic Surveys, Wiley Blackwell, vol. 8(4), pages 317-370, December.
    17. John Y. Campbell & Pierre Perron, 1991. "Pitfalls and Opportunities: What Macroeconomists Should Know about Unit Roots," NBER Chapters, in: NBER Macroeconomics Annual 1991, Volume 6, pages 141-220, National Bureau of Economic Research, Inc.
    18. Amavilah, Voxi Heinrich, 2012. "The Caldwellian Methodological Pluralism: Wishful Thoughts and Personal Tendencies," MPRA Paper 44656, University Library of Munich, Germany, revised 28 Feb 2013.
    19. Surajit Deb, 2003. "Terms of Trade and Supply Response of Indian Agriculture: Analysis in Cointegration Framework," Working papers 115, Centre for Development Economics, Delhi School of Economics.
    20. Oxley, L.T., 1995. "An expert systems approach to econometric modelling," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 39(3), pages 379-383.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ect:emjrnl:v:2:y:1999:i:2:p:167-191. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley-Blackwell Digital Licensing or Christopher F. Baum (email available below). General contact details of provider: https://edirc.repec.org/data/resssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.