IDEAS home Printed from https://ideas.repec.org/a/bla/reesec/v51y2023i3p754-778.html
   My bibliography  Save this article

Testing machine learning systems in real estate

Author

Listed:
  • Wayne Xinwei Wan
  • Thies Lindenthal

Abstract

Uncertainty about the inner workings of machine learning (ML) models holds back the application of ML‐enabled systems in real estate markets. How do ML models arrive at their estimates? Given the lack of model transparency, how can practitioners guarantee that ML systems do not run afoul of the law? This article first advocates a dedicated software testing framework for applied ML systems, as commonly found in computer science. Second, it demonstrates how system testing can verify that applied ML models indeed perform as intended. Two system‐testing procedures developed for ML image classifiers used in automated valuation models (AVMs) illustrate the approach.

Suggested Citation

  • Wayne Xinwei Wan & Thies Lindenthal, 2023. "Testing machine learning systems in real estate," Real Estate Economics, American Real Estate and Urban Economics Association, vol. 51(3), pages 754-778, May.
    • Handle: RePEc:bla:reesec:v:51:y:2023:i:3:p:754-778
    DOI: 10.1111/1540-6229.12416
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/1540-6229.12416
    Download Restriction: no
    File URL: https://libkey.io/10.1111/1540-6229.12416?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Marc K. Francke & Alex M. Minne, 2017. "Land, Structure and Depreciation," Real Estate Economics, American Real Estate and Urban Economics Association, vol. 45(2), pages 415-451, April.
    2. Daniel W. Apley & Jingyu Zhu, 2020. "Visualizing the effects of predictor variables in black box supervised learning models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 82(4), pages 1059-1086, September.
    3. Jing Lei & Max G’Sell & Alessandro Rinaldo & Ryan J. Tibshirani & Larry Wasserman, 2018. "Distribution-Free Predictive Inference for Regression," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(523), pages 1094-1111, July.
    4. Sendhil Mullainathan & Ziad Obermeyer, 2017. "Does Machine Learning Automate Moral Hazard and Error?," American Economic Review, American Economic Association, vol. 107(5), pages 476-480, May.
    5. Nikhil Naik & Ramesh Raskar & César A. Hidalgo, 2016. "Cities Are Physical Too: Using Computer Vision to Measure the Quality and Impact of Urban Appearance," American Economic Review, American Economic Association, vol. 106(5), pages 128-132, May.
    6. Gabriel Ahlfeldt & Alexandra Mastro, 2012. "Valuing Iconic Design: Frank Lloyd Wright Architecture in Oak Park, Illinois," Housing Studies, Taylor & Francis Journals, vol. 27(8), pages 1079-1099, November.
    7. Shen, Lily & Ross, Stephen, 2021. "Information value of property description: A Machine learning approach," Journal of Urban Economics, Elsevier, vol. 121(C).
    8. Coulson, N. Edward & McMillen, Daniel P., 2008. "Estimating time, age and vintage effects in housing prices," Journal of Housing Economics, Elsevier, vol. 17(2), pages 138-151, June.
    9. Edwin Buitelaar & Frans Schilder, 2017. "The Economics of Style: Measuring the Price Effect of Neo†Traditional Architecture in Housing," Real Estate Economics, American Real Estate and Urban Economics Association, vol. 45(1), pages 7-27, February.
    10. Aubry, Mathieu & Kräussl, Roman & Manso, Gustavo & Spaenjers, Christophe, 2019. "Machine learning, human experts, and the valuation of real assets," CFS Working Paper Series 635, Center for Financial Studies (CFS).
    11. Gabriel M. Ahlfeldt & Kristoffer Moeller & Sevrin Waights & Nicolai Wendland, 2017. "Game of Zones: The Political Economy of Conservation Areas," Economic Journal, Royal Economic Society, vol. 127(605), pages 421-445, October.
    12. Edward L. Glaeser & Michael Scott Kincaid & Nikhil Naik, 2018. "Computer Vision and Real Estate: Do Looks Matter and Do Incentives Determine Looks," NBER Working Papers 25174, National Bureau of Economic Research, Inc.
    13. Yi Fan & Ho Pin Teo & Wayne X. Wan, 2021. "Public transport, noise complaints, and housing: Evidence from sentiment analysis in Singapore," Journal of Regional Science, Wiley Blackwell, vol. 61(3), pages 570-596, June.
    14. Duncan Simester & Artem Timoshenko & Spyros I. Zoumpoulis, 2020. "Targeting Prospective Customers: Robustness of Machine-Learning Methods to Typical Data Challenges," Management Science, INFORMS, vol. 66(6), pages 2495-2522, June.
    15. Julia Freybote & Lauren Simon & Lauren Beitelspacher, 2016. "Understanding the contribution of curb appeal to retail real estate values," Journal of Property Research, Taylor & Francis Journals, vol. 33(2), pages 147-161, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wan, Wayne Xinwei & Lindenthal, Thies, 2022. "Towards accountability in machine learning applications: A system-testing approach," ZEW Discussion Papers 22-001, ZEW - Leibniz Centre for European Economic Research.
    2. Ka Shing Cheung & Chung Yim Yiu, 2022. "The economics of architectural aesthetics: Identifying price effect of urban ambiences by different house cohorts," Environment and Planning B, , vol. 49(6), pages 1741-1756, July.
    3. Alan Sage & Mike Langen & Alex van de Minne, 2023. "Where is the opportunity in opportunity zones?," Real Estate Economics, American Real Estate and Urban Economics Association, vol. 51(2), pages 338-371, March.
    4. Yoshida, Jiro, 2020. "The economic depreciation of real estate: Cross-sectional variations and their return implications," Pacific-Basin Finance Journal, Elsevier, vol. 61(C).
    5. Rolheiser, Lyndsey & van Dijk, Dorinth & van de Minne, Alex, 2020. "Housing vintage and price dynamics," Regional Science and Urban Economics, Elsevier, vol. 84(C).
    6. Erik B Johnson & Alan Tidwell & Sriram V Villupuram, 2020. "Valuing Curb Appeal," The Journal of Real Estate Finance and Economics, Springer, vol. 60(1), pages 111-133, February.
    7. Guan‐Yuan Wang, 2023. "The effect of environment on housing prices: Evidence from the Google Street View," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 42(2), pages 288-311, March.
    8. Lyndsey Rolheiser, 2021. "Old, small and unwanted: Post-war housing and neighbourhood socioeconomic status," Urban Studies, Urban Studies Journal Limited, vol. 58(14), pages 2952-2970, November.
    9. William N Goetzmann & Christophe Spaenjers & Stijn Van Nieuwerburgh, 2021. "Real and Private-Value Assets [Gendered prices]," The Review of Financial Studies, Society for Financial Studies, vol. 34(8), pages 3497-3526.
    10. Lopez, Luis A. & Yoshida, Jiro, 2022. "Estimating housing rent depreciation for inflation adjustments," Regional Science and Urban Economics, Elsevier, vol. 95(C).
    11. Skripkiūnas Tomas & Navickas Valentinas, 2023. "Architectural Factors Influencing a Housing Market Value: A Theoretical Framework," Real Estate Management and Valuation, Sciendo, vol. 31(1), pages 25-35, March.
    12. Yi Huang & Geoffrey Hewings, 2021. "More Reliable Land Price Index: Is There a Slope Effect?," Land, MDPI, vol. 10(3), pages 1-24, March.
    13. Lyndsey Rolheiser & Dorinth van Dijk & Alex van de Minne, 2018. "Does Housing Vintage Matter? Exploring the Historic City Center of Amsterdam," DNB Working Papers 617, Netherlands Central Bank, Research Department.
    14. Tsai, I-Chun & Wang, Wen-Kai, 2022. "The value of land redevelopment in different types of properties: Considering the effect of hold-out problems on the development probability," Land Use Policy, Elsevier, vol. 119(C).
    15. Liao, Wen-Chi & Jing, Kecen & Lee, Chaun Ying Rachel, 2022. "Economic return of architecture awards: Testing homebuyers’ motives for paying more," Regional Science and Urban Economics, Elsevier, vol. 93(C).
    16. Persson, Petra & Qiu, Xinyao & Rossin-Slater, Maya, 2021. "Family Spillover Effects of Marginal Diagnoses: The Case of ADHD," IZA Discussion Papers 14020, Institute of Labor Economics (IZA).
    17. Saravanan Thirumuruganathan & Soon-gyo Jung & Dianne Ramirez Robillos & Joni Salminen & Bernard J. Jansen, 2021. "Forecasting the nearly unforecastable: why aren’t airline bookings adhering to the prediction algorithm?," Electronic Commerce Research, Springer, vol. 21(1), pages 73-100, March.
    18. Scott Duke Kominers & Alexander Teytelboym & Vincent P Crawford, 2017. "An invitation to market design," Oxford Review of Economic Policy, Oxford University Press and Oxford Review of Economic Policy Limited, vol. 33(4), pages 541-571.
    19. Ruairi C. Robertson & Thaddeus J. Edens & Lynnea Carr & Kuda Mutasa & Ethan K. Gough & Ceri Evans & Hyun Min Geum & Iman Baharmand & Sandeep K. Gill & Robert Ntozini & Laura E. Smith & Bernard Chasekw, 2023. "The gut microbiome and early-life growth in a population with high prevalence of stunting," Nature Communications, Nature, vol. 14(1), pages 1-15, December.
    20. Leandro Andrián & Oscar Mauricio Valencia, 2023. "Past the Tipping Point? Assessing Debt Overhang in Latin America and the Caribbean," IDB Publications (Book Chapters), in: Andrew Powell & Oscar Mauricio Valencia (ed.), Dealing with Debt, edition 1, chapter 8, pages 183-196, Inter-American Development Bank.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:reesec:v:51:y:2023:i:3:p:754-778. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/areueea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.