{"id":5117,"date":"2026-05-26T16:38:20","date_gmt":"2026-05-26T20:38:20","guid":{"rendered":"https:\/\/www.law.georgetown.edu\/privacy-technology-center\/?page_id=5117"},"modified":"2026-05-26T16:38:20","modified_gmt":"2026-05-26T20:38:20","slug":"data-and-automation-in-child-welfare","status":"publish","type":"page","link":"https:\/\/www.law.georgetown.edu\/privacy-technology-center\/education\/data-and-automation-in-child-welfare\/","title":{"rendered":"Data and Automation in Child Welfare"},"content":{"rendered":"<p><span style=\"font-weight: 400\">*Many affected families and advocates have begun referring to the system as the \u201cfamily policing system,\u201d in <\/span><a href=\"https:\/\/cfrny.org\/glossary-of-terms\/\"><span style=\"font-weight: 400\">an effort to<\/span><\/a><span style=\"font-weight: 400\"> \u201cmore accurately describe[] the system\u2019s use of surveillance, regulation, and punishment to oppress families and communities, the majority of which are Black or Brown and low-income.\u201d We thank <\/span><a href=\"https:\/\/stoplapdspying.org\/wp-content\/uploads\/2023\/05\/Dividing-and-Conquering-Families.pdf\"><span style=\"font-weight: 400\">V. Copeland and Brianna Harvey<\/span><\/a><span style=\"font-weight: 400\"> for \u201chelping to theorize and widely use this term, and for refusing to use neutral language that does not fully portray the violence of the system.\u201d<\/span><\/p>\n<h2><b>To learn more, see the resources below:\u00a0<\/b><\/h2>\n<h3>On &#8220;AI&#8221;:<\/h3>\n<p><span style=\"font-weight: 400\">The readings below provide some background and framing to help think through what people are talking about when they talk about \u201cAI,\u201d and what is at stake:\u00a0<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Emily Tucker, <\/span><strong><a href=\"https:\/\/www.techpolicy.press\/artifice-and-intelligence\/\">Artifice and Intelligence<\/a><\/strong><span style=\"font-weight: 400\"> (2022): Contests conflation of human intelligence with tech tools\u2019 ability to mimic coherent text output. Explains the Privacy Center\u2019s practice of avoiding the term \u201cartificial intelligence\u201d because of its capacity to obfuscate and \u201cplacehold . . . for the scrupulous descriptions that would make the technologies\u201d described \u201ctransparent for the average person,\u201d and what we do instead.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Rua M. Williams, <\/span><strong><a href=\"https:\/\/www.goethe.de\/prj\/geg\/en\/thm\/tru\/25453339.html\">On Being an Outlier<\/a><\/strong><span style=\"font-weight: 400\"> (2024) (in <\/span><a href=\"https:\/\/www.youtube.com\/watch?v=Va1Tr8YMsOU\"><span style=\"font-weight: 400\">video form here<\/span><\/a><span style=\"font-weight: 400\">): Defines \u201cAI: as \u201calgorithmic inferences.\u201d Explains that \u201c[w]hen we ask a statistical aggregate to infer probabilities of human action, human worth, and human life, that model will harm the most vulnerable every time.\u201d Explores how algorithmic inferences in healthcare and biomedical research \u2013 promising optimization and increased productivity \u2013 in fact drive denial of care to people classified as disabled, accelerating \u201cdisabled death.\u201d Asks us to learn to ask \u201cwise questions\u201d and to \u201cturn our math around\u201d: what if we used AI to find and dismantle the structures that cause harm?<\/span><\/li>\n<li style=\"font-weight: 400\">Frameworks Institute,<strong><a href=\"https:\/\/www.frameworksinstitute.org\/app\/uploads\/2026\/05\/Framing-the-Social-Implications-of-AI.pdf\"> Framing the Social Implications of AI<\/a><\/strong> (2026): Based on research including survey experiments, focus groups, and interviews, this piece provides helpful instruction in ways to frame discussions of the \u201csocial implications of AI\u201d to foster understanding and engagement.<\/li>\n<\/ul>\n<h3><b>On Algorithmic Decisionmaking in Child Welfare:<\/b><\/h3>\n<p><span style=\"font-weight: 400\">The readings below discuss the development and adoption of algorithmic tools, such as predictive risk assessment tools, in child welfare practice to date.\u00a0<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Erina Seh-Young Moon, Matthew Tamura, Angelina Zhai, Nuzaira Habib, Behnaz Shirazi, Altaf Kassam, Devansh Saxena, and Shion Guha, <\/span><strong><a href=\"https:\/\/doi.org\/10.1145\/3772318.3790297\"><i>The Promises and Perils of using LLMs for Effective Public Services<\/i><\/a><\/strong><span style=\"font-weight: 400\"> (2026): Case study of a large Canadian child welfare agency researching how caseworkers may use automated tools in case management. Outlines opportunities and limitations of using large language models (\u201cLLMs\u201d) within child welfare systems, the importance of centering human discretion, and the risk of power imbalances between public sector agencies and private companies in procuring corporately-developed and controlled data tools. Also provides a roadmap for future use of language tools in the public sector.<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Colin Lecher, <\/span><strong><a href=\"https:\/\/themarkup.org\/investigations\/2025\/05\/20\/the-nyc-algorithm-deciding-which-families-are-under-watch-for-child-abuse\"><i>The NYC Algorithm Deciding Which Families Are Under Watch for Child Abuse<\/i><\/a><\/strong><span style=\"font-weight: 400\">, The Markup (May 20, 2025): A look at how NYC\u2019s Administration for Children&#8217;s Services (ACS) uses an algorithmic tool to assess families for risk based on various socioeconomic factors, and how the agency\u2019s use of the biased proxies designers incorporated into the tool as input and outcome variables continues to subject families of color to discriminatory surveillance.<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Stephanie K. Glaberson,<\/span><strong><a href=\"https:\/\/escholarship.org\/uc\/item\/2w7032c2\"> The Epistemic Injustice of Algorithmic Family Policing<\/a><\/strong><span style=\"font-weight: 400\"> (2024): Argues a key mechanism through which the U.S. child welfare system disrupts families and communities is \u201cepistemic injustice\u201d: the regulation of knowledge production and sharing to discredit targeted groups, and that agency use of risk-prediction algorithms that privilege carceral data perpetuates and reinforces this form of injustice. Proposes an alternative view for building a better system that will keep children safe and families thriving without encoding today\u2019s inequalities in algorithmic family policing.<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Vee Copeland and Stop LAPD Spying, <\/span><strong><a href=\"https:\/\/stoplapdspying.org\/dcfs\/\">DCF(S) Stands For Dividing and Conquering Families: How the Family Policing System Contributes to the Stalker State<\/a><\/strong><span style=\"font-weight: 400\"> (2023): This report unveils the sweeping landscape of LA\u2019s Family Policing System and its reliance on AI and predictive analytics tools, which further heighten the criminalization and surveillance of families. It also explores how the traditional criminalization tactic known as the \u201cBroken Windows\u201d approach functions as a guiding principle for many of these predictive tools. Finally, it concludes with alternatives to surveillance and state violence, encouraging readers to rethink \u201cchild protection\u201d in ways that bring families the resources and community they need to thrive.<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Marissa Gerchick, Tobi Jegede, Tarak Shah, Ana Gutierrez, Sophie Beiers, Noam Shemtov, Kath Xu, Anjana Samant, and Aaron Horowitz, <\/span><strong><a href=\"https:\/\/dl.acm.org\/doi\/10.1145\/3593013.3594081\">The Devil is in the Details: Interrogating Values Embedded in the Allegheny Family Screening Tool<\/a><\/strong><span style=\"font-weight: 400\"> (2023): Examines how algorithmic design choices can function as policy decisions, through an audit of one algorithmic tool already being used to screen calls to the Allegheny County, PA child welfare agency, the Allegheny Family Screening Tool (AFST). Highlights the values implicitly embedded in predictive tools that rely on biased data sources, perpetuate those biases, and give families no opportunity for recourse. Calls into question the claim to objectivity that proponents of algorithmic tools make. Concludes that the AFST discriminates on the basis of disability status.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Devansh Saxena, Erina Seh-Young Moon, Aryan Chaurasia, Yixin Guan, and Shion Guha, <\/span><strong><a href=\"https:\/\/doi.org\/10.1145\/3544548.3581308\"><i>Rethinking &#8220;Risk&#8221; in Algorithmic Systems Through A Computational Narrative Analysis of Casenotes in Child-Welfare<\/i><\/a><\/strong><span style=\"font-weight: 400\"> (2023): Discusses many critiques of algorithmic risk prediction tools. Highlights need for extreme caution in relying on newer natural language processing (NLP) systems now being marketed to and adopted by many agencies, noting, in particular, issues with relying on worker-produced case notes such as variability, inexperience and high turnover, documentation of \u201cperceived risks\u201d rather than factual information, \u201cdefensive decisionmaking\u201d and omission of caseworker mistakes from notes, as well as the risk that \u201cthe contextual knowledge derived from casenotes can easily be stripped and instead exploited once quantified to be used in downstream tasks\u201d among others.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Sally Ho and Garance Burke, <\/span><strong><a href=\"https:\/\/apnews.com\/article\/child-welfare-algorithm-investigation-9497ee937e0053ad4144a86c68241ef1\">An algorithm that screens for child neglect raises concerns<\/a><\/strong><span style=\"font-weight: 400\">, AP (April 29, 2022): Reporting on the use of the Allegheny Family Screening Tool (AFST) in Allegheny County, PA, as of 2022, along with considerations for other child welfare agencies adopting similar tools. Documents advocates\u2019 efficiency claims assertion that social workers can always override the tool. Discusses critics\u2019 concerns about the tool\u2019s opacity and the potential for the tool to entrench existing racial disparities.<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Hao-Fei Cheng, Logan Stapleton, Anna Kawakami, Venkatesh Sivaraman, Yanghuidi Cheng, Diana Qing, Adam Perer, Kenneth Holstein, Zhiwei Steven Wu, and Haiyi Zhu, <\/span><strong><a href=\"https:\/\/dl.acm.org\/doi\/abs\/10.1145\/3491102.3501831\"><i>How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions<\/i><\/a><\/strong><span style=\"font-weight: 400\"> (2022): Explores whether workers\u2019 use of an algorithmic risk prediction tool in a child welfare context mitigates or exacerbates disparities, finding that while the tool, operating on its own, would worsen racially disparate treatment, workers are able to correct for this over-inclusion, and ultimately, workers operating with the tool make less disparate decisions than either worker or tool operating alone. Also highlights the distinction between \u201cautomated\u201d and \u201caugmented\u201d decision-making with algorithmic child welfare tools.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Devansh Saxena, Karla Badillo-Urquiola, Pamela J. Wisniewski, and Shion Guha, <\/span><strong><a href=\"https:\/\/arxiv.org\/pdf\/2107.03487\"><i>A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare<\/i><\/a><\/strong><span style=\"font-weight: 400\"> (2021): Develops a framework of algorithmic decision-making adapted for the public sector that balances 3-way interactions between human discretion, bureaucratic processes, and algorithmic decision-making. Applies the framework to a qualitative case study of algorithms in daily use at a mid-western U.S. child-welfare agency, discussing the benefits and drawbacks revealed, and proposes guidelines for the design of high-stakes algorithmic decision-making tools in the public sector.<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Anjana Samant, Aaron Horowitz, Sophie Beiers, and Kath Xu, <\/span><strong><a href=\"https:\/\/www.aclu.org\/news\/womens-rights\/family-surveillance-by-algorithm-the-rapidly-spreading-tools-few-have-heard-of\">Family Surveillance by Algorithm: The Rapidly Spreading Tools Few Have Heard Of<\/a><\/strong><span style=\"font-weight: 400\"> (2021): National survey detailing which states and counties were employing, exploring, and\/or discontinuing the use of predictive analytics in child welfare decisions as of 2021, including which specific tools jurisdictions were using and how agencies were using them. <\/span><\/li>\n<li style=\"font-weight: 400\">Stephanie K. Glaberson,<strong> <a href=\"https:\/\/ir.lawnet.fordham.edu\/cgi\/viewcontent.cgi?article=2757&amp;context=ulj\">Coding Over the Cracks: Predictive Analytics and Child Protection<\/a><\/strong> (2019): Argues the introduction of predictive analytics into an already-flawed system risks perpetuating biases and assumptions and magnifying existing problems. Describes the \u201cdecision points\u201d and value-laden judgments integral to the child welfare system, the quintessentially human process of developing a machine learning algorithm, the predictive risk tools already implemented or then in development, and the ways those tools pose new risks to children and families.<\/li>\n<\/ul>\n<h3><b>If you want to go deeper:<\/b><\/h3>\n<p><span style=\"font-weight: 400\">Rua M. Williams, <\/span><strong><a href=\"https:\/\/link.springer.com\/book\/10.1007\/978-3-032-02665-1\"><i>Disabling Intelligences: Legacies of Eugenics and How We are Wrong about AI<\/i><\/a><\/strong><span style=\"font-weight: 400\"> (2025)<\/span><\/p>\n<p><span style=\"font-weight: 400\">Ruha Benjamin, <\/span><strong><a href=\"https:\/\/www.ruhabenjamin.com\/imagination-a-manifesto\">Imagination: A Manifesto<\/a><\/strong><span style=\"font-weight: 400\"> (2024)<\/span><\/p>\n<p><span style=\"font-weight: 400\">Joy Buolamwini, <\/span><strong><a href=\"https:\/\/www.unmasking.ai\/\">Unmasking AI: My Mission to Protect What Is Human in a World of Machines <\/a><\/strong><span style=\"font-weight: 400\">(2024)<\/span><\/p>\n<p><span style=\"font-weight: 400\">Ulises A. Mejias and Nick Coudry, Data Grab: The New Colonialism of Big Tech and How to Fight Back (2024)\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400\">Tania Duarte, Nicholas Barrow, Medina Bakayeva, Peter Smith, <\/span><strong><a href=\"https:\/\/link.springer.com\/article\/10.1007\/s43681-024-00539-x\">The Ethical Implications of AI Hype: Examining the overinflation and misrepresentation of AI capabilities and performance <\/a><\/strong><span style=\"font-weight: 400\">(2024)<\/span><\/p>\n<p><span style=\"font-weight: 400\">Salom\u00e9 Viljoen, <\/span><strong><a href=\"https:\/\/yalelawjournal.org\/feature\/a-relational-theory-of-data-governance\">A Relational Theory of Data Governance<\/a><\/strong><span style=\"font-weight: 400\"> (2021)<\/span><\/p>\n<p><span style=\"font-weight: 400\">Sasha Costanza-Chock, <\/span><strong><a href=\"https:\/\/mitpress.mit.edu\/9780262043458\/design-justice\/\">Design Justice: Community-Led Practices to Build the Worlds We Need <\/a><\/strong><span style=\"font-weight: 400\">(2020)<\/span><\/p>\n<p><span style=\"font-weight: 400\">Ruha Benjamin, <\/span><strong><a href=\"https:\/\/www.ruhabenjamin.com\/race-after-technology\">Race After Technology<\/a><\/strong><span style=\"font-weight: 400\"> (2019)\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400\">Virginia Eubanks, <\/span><strong><a href=\"https:\/\/us.macmillan.com\/books\/9781250074317\/automatinginequality\/\">Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor<\/a><\/strong><span style=\"font-weight: 400\"> (2018)\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400\">Safiya Umoja Noble, <\/span><strong><a href=\"https:\/\/nyupress.org\/9781479837243\/algorithms-of-oppression\/\">Algorithms of Oppression: How Search Engines Reinforce Racism<\/a><\/strong><span style=\"font-weight: 400\"> (2018)\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400\">Lewis, T., Gangadharan, S. P., Saba, M., Petty, T., <\/span><strong><a href=\"https:\/\/detroitcommunitytech.org\/system\/tdf\/librarypdfs\/ODB-Digital-Defense.pdf?file=1&amp;type=node&amp;id=81&amp;force=\">Digital defense playbook: Community power tools for reclaiming data<\/a><\/strong><span style=\"font-weight: 400\">, Our Data Bodies (2018)<\/span><\/p>\n<p><span style=\"font-weight: 400\">Khiara Bridges, <\/span><strong><a href=\"https:\/\/www.sup.org\/books\/law\/poverty-privacy-rights\">The Poverty of Privacy Rights<\/a><\/strong><span style=\"font-weight: 400\"> (2017)\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400\">Cathy O&#8217;Neil, <\/span><strong><a href=\"https:\/\/www.penguin.co.uk\/books\/304513\/weapons-of-math-destruction-by-oneil-cathy\/9780141985411\">Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy<\/a><\/strong><span style=\"font-weight: 400\"> (2017)<\/span><\/p>\n<p><b>For even more, see \u201c<\/b><a href=\"https:\/\/monettdiaz.com\/books-critical-ai.html\"><b>A non-exhaustive collection of worth-reading books on topics strongly related to Critical AI (12th edition)<\/b><\/a><b>\u201d curated by Prof. Dr. Dagmar Monett<\/b><\/p>\n<p><i><span style=\"font-weight: 400\">If you think something important is missing from this list, please send us an email!<\/span><\/i><\/p>\n<h3><b>What we\u2019re doing:<\/b><\/h3>\n<p><span style=\"font-weight: 400\">The Privacy Center\u2019s Family Surveillance work seeks to:\u00a0<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400\"><b>Empower organized communities with information.<\/b><span style=\"font-weight: 400\"> Privacy Center work in this area illuminates techniques and tools in use and pathways of data movement to support organized communities in protecting themselves. It seeks to illuminate the ways data moves \u2014 and doesn\u2019t \u2014 so that communities can make educated decisions about their own lives rather than being guided by a lack of information, fear, or rumor.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\"><b>Shape law and policy.<\/b><span style=\"font-weight: 400\"> The law \u2013 both unintentionally and by design \u2013 frequently serves to make communities more vulnerable to family policing interventions. Our work illuminates those legal avenues of vulnerability and supports communities in taking action to make needed changes.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\"><b>Change the narrative. <\/b><span style=\"font-weight: 400\">Through the work of impacted individuals, activists, and scholars, the political community has come to a more sophisticated understanding in recent years of the harm of criminalization and policing. Left out of that narrative all too often is the harm perpetrated by this system, which many imagine to provide merely a necessary intervention on behalf of children in need. Our work will build public awareness of the impact in the lives of children, parents, and communities of living under the shadow of family policing\u2019s surveillant gaze.\u00a0<\/span><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>*Many affected families and advocates have begun referring to the system as the \u201cfamily policing system,\u201d in an effort to \u201cmore accurately describe[] the system\u2019s use of surveillance, regulation, and [&hellip;]<\/p>\n","protected":false},"author":15769,"featured_media":0,"parent":40,"menu_order":1,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_acf_changed":false,"_price":"","_stock":"","_tribe_ticket_header":"","_tribe_default_ticket_provider":"","_tribe_ticket_capacity":"0","_ticket_start_date":"","_ticket_end_date":"","_tribe_ticket_show_description":"","_tribe_ticket_show_not_going":false,"_tribe_ticket_use_global_stock":"","_tribe_ticket_global_stock_level":"","_global_stock_mode":"","_global_stock_cap":"","_tribe_rsvp_for_event":"","_tribe_ticket_going_count":"","_tribe_ticket_not_going_count":"","_tribe_tickets_list":"[]","_tribe_ticket_has_attendee_info_fields":false,"footnotes":"","_tec_slr_enabled":"","_tec_slr_layout":""},"class_list":["post-5117","page","type-page","status-publish","hentry"],"acf":[],"ticketed":false,"_links":{"self":[{"href":"https:\/\/www.law.georgetown.edu\/privacy-technology-center\/wp-json\/wp\/v2\/pages\/5117","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.law.georgetown.edu\/privacy-technology-center\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.law.georgetown.edu\/privacy-technology-center\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.law.georgetown.edu\/privacy-technology-center\/wp-json\/wp\/v2\/users\/15769"}],"replies":[{"embeddable":true,"href":"https:\/\/www.law.georgetown.edu\/privacy-technology-center\/wp-json\/wp\/v2\/comments?post=5117"}],"version-history":[{"count":8,"href":"https:\/\/www.law.georgetown.edu\/privacy-technology-center\/wp-json\/wp\/v2\/pages\/5117\/revisions"}],"predecessor-version":[{"id":5125,"href":"https:\/\/www.law.georgetown.edu\/privacy-technology-center\/wp-json\/wp\/v2\/pages\/5117\/revisions\/5125"}],"up":[{"embeddable":true,"href":"https:\/\/www.law.georgetown.edu\/privacy-technology-center\/wp-json\/wp\/v2\/pages\/40"}],"wp:attachment":[{"href":"https:\/\/www.law.georgetown.edu\/privacy-technology-center\/wp-json\/wp\/v2\/media?parent=5117"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}