research-article

Public Access

Trust in Data Science: Collaboration, Translation, and Accountability in Corporate Data Science Projects

Authors:

Steven J. JacksonAuthors Info & Claims

Proceedings of the ACM on Human-Computer Interaction, Volume 2, Issue CSCW

Article No.: 136, Pages 1 - 28

https://doi.org/10.1145/3274405

Published: 01 November 2018 Publication History

Abstract

The trustworthiness of data science systems in applied and real-world settings emerges from the resolution of specific tensions through situated, pragmatic, and ongoing forms of work. Drawing on research in CSCW, critical data studies, and history and sociology of science, and six months of immersive ethnographic fieldwork with a corporate data science team, we describe four common tensions in applied data science work: (un)equivocal numbers, (counter)intuitive knowledge, (in)credible data, and (in)scrutable models. We show how organizational actors establish and re-negotiate trust under messy and uncertain analytic conditions through practices of skepticism, assessment, and credibility. Highlighting the collaborative and heterogeneous nature of real-world data science, we show how the management of trust in applied corporate data science settings depends not only on pre-processing and quantification, but also on negotiation and translation. We conclude by discussing the implications of our findings for data science research and practice, both within and beyond CSCW.

References

[1]

Saleema Amershi, Max Chickering, Steven M Drucker, Bongshin Lee, Patrice Simard, and Jina Suh. 2015. ModelTracker: Redesigning Performance Analysis Tools for Machine Learning. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 337--346.

Digital Library

[2]

Mike Ananny and Kate Crawford. 2016. Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. New Media Soc.20, 3, 973--989.

[3]

Chris Anderson. 2008. The End of Theory: The Data Deluge Makes the Scientific Method Obsolete. Wired. Retrieved from https://archive.wired.com/science/discoveries/magazine/16-07/pb_theory

[4]

Cecilia Aragon, Clayton Hutto, Andy Echenique, Brittany Fiore-Gartland, Yun Huang, Jinyoung Kim, Gina Neff, Wanli Xing, and Joseph Bayer. 2016. Developing a Research Agenda for Human-Centered Data Science. In Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion (CSCW '16 Companion), 529--535.

Digital Library

[5]

Ellen Balka and Ina Wagner. 2006. Making Things Work: Dimensions of Configurability As Appropriation Work. In Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work, 229--238.

Digital Library

[6]

Eric P S Baumer. 2017. Toward human-centered algorithm design. Big Data Soc.4, 2, 1--12.

[7]

Reuben Binns, Max Van Kleek, Michael Veale, Ulrik Lyngs, Jun Zhao, and Nigel Shadbolt. 2018. "It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18), 1--14.

Digital Library

[8]

Luc Boltanski and Laurent Thévenot. 2006. On Justification: Economies of Worth. Princeton University Press, Princeton.

[9]

Geoffrey C. Bowker. 2013. Data Flakes: An Afterword to "Raw Data" Is an Oxymoron. In"Raw Data" Is an Oxymoron, Lisa Gitelman (ed.). MIT Press, Cambridge, Massachusetts, 167--171.

[10]

Geoffrey C. Bowker. 2014. The Theory/Data Thing. Int. J. Commun.8, 1795--1799.

[11]

George E. P. Box. 1979. Robustness in the Strategy of Scientific Model Building. In Robustness in Statistics, Robert L. Launer and Graham N. Wilkinson (eds.). Academic Press, New York, 201--36.

[12]

Engin Bozdag. 2013. Bias in algorithmic filtering and personalization. Ethics Inf. Technol.15, 3 (2013), 209--227.

Digital Library

[13]

Eric Breck, Shanqing Cai, Eric Nielsen, Michael Salib, and D Sculley. 2016. What's your ML Test Score? A rubric for ML production systems. In Reliable Machine Learning in the Wild - NIPS 2016 Workshop.

[14]

Jenna Burrell. 2016. How the machine 'thinks': Understanding opacity in machine learning algorithms. Big Data Soc.3, 1, 1--12.

[15]

Kevin Daniel André Carillo. 2017. Let's stop trying to be "sexy" -- preparing managers for the (big) data-driven business era. Bus. Process Manag. J.23, 3, 598--622.

[16]

Kathy Charmaz. 2014. Constructing Grounded Theory (Introducing Qualitative Methods series) 2nd Edition. Sage, London.

[17]

Glenn Cohen, Ruben Amarasingham, Anand Shah, Bin Xie, and Bernard Lo. 2014. The legal and ethical concerns that arise from using complex predictive analytics in health care. Health Aff.33, 7, 1139--1147.

[18]

Harry M. Collins and Robert Evans. 2007. Rethinking Expertise. University of Chicago Press, Chicago.

[19]

Kate Crawford and Ryan Calo. 2016. There is a blind spot in AI research. Nature 538, 311--313.

[20]

Morgan Currie, Britt S Paris, Irene Pasquetto, and Jennifer Pierre. 2016. The conundrum of police officer-involved homicides: Counter-data in Los Angeles County. Big Data Soc.3, 2, 1--14.

[21]

Aritra Dasgupta, Susannah Burrows, Kyungsik Han, and Philip J Rasch. 2017. Empirical Analysis of the Subjective Impressions and Objective Measures of Domain Scientists' Visual Analytic Judgments. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17), 1193--1204.

Digital Library

[22]

Lorraine Daston and Peter Galison. 1992. The Image of Objectivity. Representations 40, 81--128.

[23]

Lorraine Daston and Peter Galison. 2007. Objectivity. MIT Press, Cambridge, Massachusetts.

[24]

Alain Desrosieres. 1998. The Politics of Large Numbers: A History of Statistical Reasoning. Harvard University Press, Cambridge, MA.

[25]

John Dewey. 1939. Theory of Valuation. University of Chicago Press, Chicago.

[26]

Jana Diesner. 2015. Small decisions with big impact on data analytics. Big Data Soc.2, 2, 1--6.

[27]

Paul Dourish. 2016. Algorithms and their others: Algorithmic culture in context. Big Data Soc.3, 2, 1--11.

[28]

Wendy Nelson Espeland and Mitchell L Stevens. 2008. A Sociology of Quantification. Eur. J. Sociol. / Arch. Eur. Sociol. / Eur. Arch. für Soziologie 49, 3, 401--436.

[29]

Batya Friedman and Helen Nissenbaum. 1996. Bias in computer systems. ACM Trans. Inf. Syst.14, 3, 330--347.

Digital Library

[30]

Jennifer Gabrys, Helen Pritchard, and Benjamin Barratt. 2016. Just good enough data: Figuring data citizenships through air pollution sensing and data stories. Big Data Soc.3, 2, 1--14.

[31]

Harold Garfinkel. 1964. Studies of the Routine Grounds of Everyday Activities. Soc. Probl.11, 3 (1964), 225--250.

[32]

Harold Garfinkel. 1967. Studies in Ethnomethodology. Prentice Hall, New Jersey.

[33]

Tarleton Gillespie. 2014. The Relevance of Algorithms. In Media Technologies: Essays on Communication, Materiality, and Society, Tarleton Gillespie, Pablo J. Boczkowski and Kirsten A. Foot (eds.). MIT Press, Cambridge, 167--194.

[34]

Lisa Gitelman. 2006. Raw Data is an Oxymoron. MIT Press, MA.

[35]

Barney Glaser and Anselm Strauss. 1967. The Discovery of Grounded Theory: Strategies for Qualitative Research. Aldine Transactions, Chicago.

[36]

Charles Goodwin. 1994. Professional Vision. Am. Anthropol.96, 3, 606--633.

[37]

Miriam Greis, Emre Avci, Albrecht Schmidt, and Tonja Machulla. 2017. Increasing Users' Confidence in Uncertain Data by Aggregating Data from Multiple Sources. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17), 828--840.

Digital Library

[38]

Riccardo Guidotti, Anna Monreale, Franco Turini, Dino Pedreschi, and Fosca Giannotti. 2018. A Survey Of Methods For Explaining Black Box Models. arXiv Prepr.1802.01933.

[39]

Jürgen Habermas. 1992. Autonomy and Solidarity: Interviews. Verso, London.

[40]

Ian Hacking. 1990. The Taming of Chance. Cambridge University Press, Cambridge.

[41]

Donna Haraway. 2007. Modest_Witness@Second_Millenium. Routledge, London.

[42]

Richard Harper. 2000. The social organization of the IMF's mission work: An examination of international auditing. In Audit cultures: Anthropological studies in accountability, ethics, and the academy, Marilyn Strathern (ed.). Routledge, London, 21--53.

[43]

Mireille Hildebrandt. 2011. Who needs stories if you can get the data? ISPs in the era of big number crunching. Philos. Technol.24, 4, 371--390.

[44]

Edmund Husserl. 1970. The Crisis of European Sciences and Transcendental Philosophy. Northwestern University Press, Evanston.

[45]

John P. A. Ioannidis. 2005. Why most published research findings are false. PLoS Med.2, 8 (2005), e124.

[46]

Steven J. Jackson. 2006. Water Models and Water Politics: Design, Deliberation, and Virtual Accountability. In Proceedings of the 2006 International Conference on Digital Government Research, 95--104.

Digital Library

[47]

Bernward Joerges and Terry Shinn. 2001. A Fresh Look at Instrumentation an Introduction. In Instrumentation Between Science, State and Industry, Bernward Joerges and Terry Shinn (eds.). Springer Netherlands, Dordrecht, 1--13.

[48]

Matthew Kay, Shwetak N Patel, and Julie A Kientz. 2015. How Good is 85%?: A Survey Tool to Connect Classifier Evaluation to Acceptability of Accuracy. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 347--356.

Digital Library

[49]

Evelyn Fox Keller. 2000. Models of and Models for: Theory and Practice in Contemporary Biology. Philos. Sci.67, 72--86.

[50]

Helen Kennedy and Giles Moss. 2015. Known or knowing publics? Social media data mining and the question of public agency. Big Data Soc.2, 2, 1--11.

[51]

Mary Beth Kery, Amber Horvath, and Brad Myers. 2017. Variolite: Supporting Exploratory Programming by Data Scientists. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17), 1265--1276.

Digital Library

[52]

Rob Kitchin. 2014. The Data Revolution: Big Data, Open Data, Data Infrastructures and Their Consequences. Sage Publications, USA.

[53]

Rob Kitchin. 2014. Big Data, new epistemologies and paradigm shifts. Big Data Soc.1, 1 (April 2014), 1--12.

[54]

Rob Kitchin and Gavin McArdle. 2016. What makes Big Data, Big Data? Exploring the ontological characteristics of 26 datasets. Big Data Soc.3, 1, 1--10.

[55]

Jon Kleinberg, Jens Ludwig, and Sendhil Mullainathan. 2016. A Guide to Solving Social Problems with Machine Learning. Harvard Business Review. Retrieved from https://hbr.org/2016/12/a-guide-to-solving-social-problems-with-machine-learning

[56]

Cory P. Knobel. 2010. Ontic Occlusion and Exposure in Sociotechnical Systems. University of Michigan.

[57]

Bran Knowles, Mike Harding, Lynne Blair, Nigel Davies, James Hannon, Mark Rouncefield, and John Walden. 2014. Trustworthy by Design. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '14), 1060--1071.

Digital Library

[58]

Bran Knowles, Mark Rouncefield, Mike Harding, Nigel Davies, Lynne Blair, James Hannon, John Walden, and Ding Wang. 2015. Models and Patterns of Trust. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '15), 328--338.

Digital Library

[59]

Laura M Koesten, Emilia Kacprzak, Jenifer F A Tennison, and Elena Simperl. 2017. The Trials and Tribulations of Working with Structured Data: -a Study on Information Seeking Behaviour. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17), 1277--1289.

Digital Library

[60]

Bruno Latour. 1987. Science in Action. Harvard University Press, Cambridge, MA.

[61]

David Lazer, Ryan Kennedy, Gary King, and Alessandro Vespignani. 2014. The parable of Google flu: Traps in big data analysis. Science 343, 6176, 1203--1205.

[62]

Sabina Leonelli. 2014. What difference does quantity make? On the epistemology of Big Data in biology. Big Data Soc.1, 1, 1--11.

[63]

Sabina Leonelli. 2015. What counts as scientific data? A relational framework. Philos. Sci.82, 5 (2015), 810--821.

[64]

Zachary Chase Lipton. 2016. The Mythos of Model Interpretability. In2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016).

[65]

Michael Luca, Jon Kleinberg, and Sendhil Mullainathan. 2016. Algorithms Need Managers, Too. Harvard Business Review. Retrieved from https://hbr.org/2016/01/algorithms-need-managers-too

[66]

Adrian Mackenzie. 2017. Machine Learners: Archaeology of a Data Practice. MIT Press, Cambridge, MA.

[67]

Jacob Metcalf and Kate Crawford. 2016. Where are human subjects in Big Data research? The emerging ethics divide. Big Data Soc.3, 1, 1--14.

[68]

Brent D. Mittelstadt, Patrick Allo, Mariarosaria Taddeo, Sandra Wachter, and Luciano Floridi. 2016. The ethics of algorithms: Mapping the debate. Big Data Soc.3, 2, 1--21.

[69]

Brent D. Mittelstadt and Luciano Floridi. 2016. The Ethics of Big Data: Current and Foreseeable Issues in Biomedical Contexts. Sci. Eng. Ethics 22, 2, 303--341.

[70]

Gauri Naik and Sanika S. Bhide. 2014. Will the future of knowledge work automation transform personalized medicine? Appl. Transl. Genomics 3, 3, 50--53.

[71]

Gina Neff, Anissa Tanweer, Brittany Fiore-Gartland, and Laura Osburn. 2017. Critique and Contribute: A Practice-Based Framework for Improving Critical Data Studies and Data Science. Big Data 5, 2, 85--97.

[72]

Joseph O'Connell. 1993. Metrology: The Creation of Universality by the Circulation of Particulars. Soc. Stud. Sci.23, 1, 129--173.

[73]

Wanda J. Orlikowski. 2007. Sociomaterial Practices: Exploring Technology at Work. Organ. Stud.28, 9, 1435--1448.

[74]

Frank Pasquale. 2015. The Black Box Society: The Secret Algorithms that Control Money and Information. Harvard University Press, Cambridge, MA.

Digital Library

[75]

Samir Passi and Steven J. Jackson. 2017. Data Vision: Learning to See Through Algorithmic Abstraction. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17), 2436--2447.

Digital Library

[76]

Samir Passi and Phoebe Sengers. 2016. "From what I see, this makes sense:" Seeing meaning in algorithmic results. In Computer-Supported Cooperative Work (CSCW) 2016 workshop "Algorithms at Work: Empirical Diversity, Analytic Vocabularies, Design Implications,"1--4.

[77]

Theodore Porter. 1995. Trust in Numbers: The Pursuit of Objectivity in Scienceand Public Life. Princeton University Press, Princeton.

[78]

Michael Power. 1997. The Audit Society: Rituals of Verification. Oxford University Press, Oxford.

[79]

David Powers. 2011. Evaluation: From precision, recall and F-measure to ROC, informedness, markedness & correlation. J. Mach. Learn. Technol 2, 1, 37--63.

[80]

Rita Raley. 2013. Dataveillance and Counterveillance. In"Raw Data" Is an Oxymoron, Lisa Gitelman (ed.). MIT Press, Cambridge, Massachusetts, 121--145.

[81]

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. Model-Agnostic Interpretability of Machine Learning. In2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016), 91--95.

[82]

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. Why Should I Trust You?: Explaining the Predictions of Any Classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135--1144.

Digital Library

[83]

Gernot Rieder and Judith Simon. 2016. Datatrust: Or, the political quest for numerical evidence and the epistemologies of Big Data. Big Data Soc.3, 1, 1--6.

[84]

Alan Rubel and Kyle M. L. Jones. 2014. Student privacy in learning analytics: An information ethics perspective. Inf. Soc.32, 2, 143--159.

Digital Library

[85]

Nadine Schuurman and Ellen Balka. 2009. alt.metadata.health: Ontological Context for Data Use and Integration. Comput. Support. Coop. Work 18, 1, 83--108.

Digital Library

[86]

Andrew D. Selbst and Solon Barocas. 2018. The Intuitive Appeal of Explainable Machines. Fordham Law Rev.Forthcoming, (2018).

[87]

Steven Shapin. 1989. The Invisible Technician. Am. Sci.77, 6, 554--563.

[88]

Steven Shapin. 1994. A Social History of Truth: Civility and Science in Seventeenth Century England. University of Chicago Press, Chicago.

[89]

Steven Shapin. 1995. Cordelia's Love: Credibility and the Social Studies of Science. Perspect. Sci.3, 3, 255--275.

[90]

Steven Shapin. 1995. Trust, Honesty, and the Authority of Science. In Society's Choices: Social and Ethical Decision Making in Biomedicine, Ruth Ellen Bulger, Elizabeth Meyer Bobby and Harvey Fineberg (eds.). National Academies Press, Washington D.C., 388--408.

[91]

Steven Shapin and Simon Schaffer. 1985. Leviathan and the Air-Pump: Hobbes, Boyle and the Experimental Life. Princeton University Press, Princeton.

[92]

Sameer Singh, Marco Tulio Ribeiro, and Carlos Guestrin. 2016. Programs as Black-Box Explanations. ArXiv e-prints 1611.07579.

[93]

Susan Leigh Star and Karen Ruhleder. 1994. Steps Towards an Ecology of Infrastructure: Complex Problems in Design and Access for Large-scale Collaborative Systems. In Proceedings of the 1994 ACM Conference on Computer Supported Cooperative Work (CSCW '94), 253--264.

Digital Library

[94]

David Stark. 2009. The Sense of Dissonance: Accounts of Worth in Economic Life. Princeton University Press, Princeton.

[95]

Anselm Strauss and Juliet M. Corbin. 1990. Basics of Qualitative Research: Grounded Theory Techniques and Procedures. Sage, New York.

[96]

John Symons and Ramón Alvarado. 2016. Can we trust Big Data? Applying philosophy of science to software. Big Data Soc.3, 2, 1--17.

[97]

Alex Tabarrok. 2015. The Rise of Opaque Intelligence. Marginal Revolution. Retrieved February 15, 2018 from https://marginalrevolution.com/marginalrevolution/2015/02/opaque-intelligence.html

[98]

Alex S. Taylor, Siân Lindley, Tim Regan, and David Sweeney. 2014. Data and life on the street. Big Data Soc.1, 2, 1--7.

[99]

François Thoreau. 2016. 'A mechanistic interpretation, if possible': How does predictive modelling causality affect the regulation of chemicals? Big Data Soc.3, 2, 1--11.

[100]

Michael Veale. 2017. Logics and Practices of Transparency and Opacity in Real-world Applications of Public Sector Machine Learning. In 4th Workshop on Fairness, Accountability, and Transparency in Machine Learning (FAT/ML 2017), Halifax, Canada.

[101]

Martin Wattenberg, Fernanda Viégas, and Moritz Hardt. 2016. Attacking discrimination with smarter machine learning. Google Research.

[102]

Michele Willson. 2017. Algorithms (and the) everyday. Information, Commun. Soc.20, 1 (2017), 137--150.

[103]

Qian Yang, Alex Scuito, John Zimmerman, Jodi Forlizzi, and Aaron Steinfeld. 2018. Investigating How Experienced UX Designers Effectively Work with Machine Learning. In Proceedings of the 2018 Designing Interactive Systems Conference (DIS '18), 585--596.

Digital Library

[104]

Tal Zarsky. 2016. The trouble with algorithmic decisions an analytic road map to examine efficiency and fairness in automated and opaque decision making. Sci. Technol. Hum. Values 41, 1, 118--132.

[105]

Matthew Zook, Solon Barocas, danah boyd, Kate Crawford, Emily Keller, Seeta Peña Gangadharan, Alyssa Goodman, Rachelle Hollander, Barbara A. Koenig, Jacob Metcalf, Arvind Narayanan, Alondra Nelson, and Frank Pasquale. 2017. Ten simple rules for responsible big data research. PLoS Comput. Biol.13, 3.

Cited By

Victorelli EDos Reis J(2024)Adopting Human-data Interaction Guidelines and Participatory Practices for Supporting Inexperienced Designers in Information Visualization ApplicationsJournal of the Brazilian Computer Society10.5753/jbcs.2024.259230:1(35-54)Online publication date: 5-Apr-2024
https://doi.org/10.5753/jbcs.2024.2592
Kou T(2024)From Model Performance to Claim: How a Change of Focus in Machine Learning Replicability Can Help Bridge the Responsibility GapSSRN Electronic Journal10.2139/ssrn.4806609Online publication date: 2024
https://doi.org/10.2139/ssrn.4806609
Zilberstein S(2024)Ethical Dilemmas and Collaborative Resolutions in Machine Learning Research for Health CareSocius: Sociological Research for a Dynamic World10.1177/2378023124125967110Online publication date: 20-Jun-2024
https://doi.org/10.1177/23780231241259671
Show More Cited By

Index Terms

Trust in Data Science: Collaboration, Translation, and Accountability in Corporate Data Science Projects
1. Human-centered computing
  1. Collaborative and social computing
    1. Empirical studies in collaborative and social computing

Recommendations

Beyond Expertise Seeking: A Field Study of the Informal Knowledge Practices of Healthcare IT Teams

CSCW has long been concerned with formal and informal knowledge practices in organizations, examining both the social and technical aspects of how knowledge is sought, shared, and used. In this study, we are interested in examining the set of activities ...
Trust and epistemic communities in biodiversity data sharing
JCDL '02: Proceedings of the 2nd ACM/IEEE-CS joint conference on Digital libraries

Trust is a key element of knowledge work: what we know depends largely on others. This paper discusses the concepts of communities of practice and epistemic cultures, and their implication for design of digital libraries that support data sharing, with ...
Mobile-banking adoption by Iranian bank clients

This study provides insights into factors affecting the adoption of mobile banking in Iran. Encouraging clients to use the cell-phone for banking affairs, and negative trends in the adoption of this technology makes it imperative to study the factors ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction

Proceedings of the ACM on Human-Computer Interaction Volume 2, Issue CSCW

November 2018

4104 pages

EISSN:2573-0142

DOI:10.1145/3290265

Editors:
Karrie Karahalios
University of Illinois & Adobe
,
Andrés Monroy-Hernández
Snap Inc.
,
Airi Lampinen
Stockholm University
,
Geraldine Fitzpatrick
Vienna University of Technology

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 November 2018

Published in PACMHCI Volume 2, Issue CSCW

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation (NSF)

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

162
Total Citations
View Citations
4,451
Total Downloads

Downloads (Last 12 months)1,065
Downloads (Last 6 weeks)168

Reflects downloads up to 06 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Victorelli EDos Reis J(2024)Adopting Human-data Interaction Guidelines and Participatory Practices for Supporting Inexperienced Designers in Information Visualization ApplicationsJournal of the Brazilian Computer Society10.5753/jbcs.2024.259230:1(35-54)Online publication date: 5-Apr-2024
https://doi.org/10.5753/jbcs.2024.2592
Kou T(2024)From Model Performance to Claim: How a Change of Focus in Machine Learning Replicability Can Help Bridge the Responsibility GapSSRN Electronic Journal10.2139/ssrn.4806609Online publication date: 2024
https://doi.org/10.2139/ssrn.4806609
Zilberstein S(2024)Ethical Dilemmas and Collaborative Resolutions in Machine Learning Research for Health CareSocius: Sociological Research for a Dynamic World10.1177/2378023124125967110Online publication date: 20-Jun-2024
https://doi.org/10.1177/23780231241259671
Valentine MBohn RPratt AJain PSinger SBernstein M(2024)Constructing a Classification Scheme - and its Consequences: A Field Study of Learning to Label Data for Computer Vision in a Hospital Intensive Care UnitProceedings of the ACM on Human-Computer Interaction10.1145/36870298:CSCW2(1-29)Online publication date: 7-Nov-2024
https://dl.acm.org/doi/10.1145/3687029
Lee SDourish P(2024)Reconfiguring Data Relations: Institutional Dynamics around Data in Local GovernanceProceedings of the ACM on Human-Computer Interaction10.1145/36869598:CSCW2(1-28)Online publication date: 7-Nov-2024
https://dl.acm.org/doi/10.1145/3686959
Yang TMiceli M(2024)"Guilds" as Worker Empowerment and Control in a Chinese Data Work PlatformProceedings of the ACM on Human-Computer Interaction10.1145/36869048:CSCW2(1-27)Online publication date: 7-Nov-2024
https://dl.acm.org/doi/10.1145/3686904
Shankar SGarcia RHellerstein JParameswaran A(2024)"We Have No Idea How Models will Behave in Production until Production": How Engineers Operationalize Machine LearningProceedings of the ACM on Human-Computer Interaction10.1145/36536978:CSCW1(1-34)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3653697
Popova KFigueras CHöök KLampinen A(2024)Who Should Act? Distancing and Vulnerability in Technology Practitioners' Accounts of Ethical ResponsibilityProceedings of the ACM on Human-Computer Interaction10.1145/36374348:CSCW1(1-27)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637434
Ehsan ULiao QPassi SRiedl MDaumé H(2024)Seamful XAI: Operationalizing Seamful Design in Explainable AIProceedings of the ACM on Human-Computer Interaction10.1145/36373968:CSCW1(1-29)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637396
Yoo DWoo HPendse SLu NBirnbaum MAbowd GDe Choudhury M(2024)Missed Opportunities for Human-Centered AI Research: Understanding Stakeholder Collaboration in Mental Health AI ResearchProceedings of the ACM on Human-Computer Interaction10.1145/36373728:CSCW1(1-24)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637372
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents