*Many affected families and advocates have begun referring to the system as the “family policing system,” in an effort to “more accurately describe[] the system’s use of surveillance, regulation, and punishment to oppress families and communities, the majority of which are Black or Brown and low-income.” We thank V. Copeland and Brianna Harvey for “helping to theorize and widely use this term, and for refusing to use neutral language that does not fully portray the violence of the system.”

To learn more, see the resources below: 

On “AI”:

The readings below provide some background and framing to help think through what people are talking about when they talk about “AI,” and what is at stake: 

• Emily Tucker, Artifice and Intelligence (2022): Contests conflation of human intelligence with tech tools’ ability to mimic coherent text output. Explains the Privacy Center’s practice of avoiding the term “artificial intelligence” because of its capacity to obfuscate and “placehold . . . for the scrupulous descriptions that would make the technologies” described “transparent for the average person,” and what we do instead. 
• Rua M. Williams, On Being an Outlier (2024) (in video form here): Defines “AI: as “algorithmic inferences.” Explains that “[w]hen we ask a statistical aggregate to infer probabilities of human action, human worth, and human life, that model will harm the most vulnerable every time.” Explores how algorithmic inferences in healthcare and biomedical research – promising optimization and increased productivity – in fact drive denial of care to people classified as disabled, accelerating “disabled death.” Asks us to learn to ask “wise questions” and to “turn our math around”: what if we used AI to find and dismantle the structures that cause harm?
• Frameworks Institute, Framing the Social Implications of AI (2026): Based on research including survey experiments, focus groups, and interviews, this piece provides helpful instruction in ways to frame discussions of the “social implications of AI” to foster understanding and engagement.

On Algorithmic Decisionmaking in Child Welfare:

The readings below discuss the development and adoption of algorithmic tools, such as predictive risk assessment tools, in child welfare practice to date. 

• Erina Seh-Young Moon, Matthew Tamura, Angelina Zhai, Nuzaira Habib, Behnaz Shirazi, Altaf Kassam, Devansh Saxena, and Shion Guha, The Promises and Perils of using LLMs for Effective Public Services (2026): Case study of a large Canadian child welfare agency researching how caseworkers may use automated tools in case management. Outlines opportunities and limitations of using large language models (“LLMs”) within child welfare systems, the importance of centering human discretion, and the risk of power imbalances between public sector agencies and private companies in procuring corporately-developed and controlled data tools. Also provides a roadmap for future use of language tools in the public sector.
• Colin Lecher, The NYC Algorithm Deciding Which Families Are Under Watch for Child Abuse, The Markup (May 20, 2025): A look at how NYC’s Administration for Children’s Services (ACS) uses an algorithmic tool to assess families for risk based on various socioeconomic factors, and how the agency’s use of the biased proxies designers incorporated into the tool as input and outcome variables continues to subject families of color to discriminatory surveillance.
• Stephanie K. Glaberson, The Epistemic Injustice of Algorithmic Family Policing (2024): Argues a key mechanism through which the U.S. child welfare system disrupts families and communities is “epistemic injustice”: the regulation of knowledge production and sharing to discredit targeted groups, and that agency use of risk-prediction algorithms that privilege carceral data perpetuates and reinforces this form of injustice. Proposes an alternative view for building a better system that will keep children safe and families thriving without encoding today’s inequalities in algorithmic family policing.
• Vee Copeland and Stop LAPD Spying, DCF(S) Stands For Dividing and Conquering Families: How the Family Policing System Contributes to the Stalker State (2023): This report unveils the sweeping landscape of LA’s Family Policing System and its reliance on AI and predictive analytics tools, which further heighten the criminalization and surveillance of families. It also explores how the traditional criminalization tactic known as the “Broken Windows” approach functions as a guiding principle for many of these predictive tools. Finally, it concludes with alternatives to surveillance and state violence, encouraging readers to rethink “child protection” in ways that bring families the resources and community they need to thrive.
• Marissa Gerchick, Tobi Jegede, Tarak Shah, Ana Gutierrez, Sophie Beiers, Noam Shemtov, Kath Xu, Anjana Samant, and Aaron Horowitz, The Devil is in the Details: Interrogating Values Embedded in the Allegheny Family Screening Tool (2023): Examines how algorithmic design choices can function as policy decisions, through an audit of one algorithmic tool already being used to screen calls to the Allegheny County, PA child welfare agency, the Allegheny Family Screening Tool (AFST). Highlights the values implicitly embedded in predictive tools that rely on biased data sources, perpetuate those biases, and give families no opportunity for recourse. Calls into question the claim to objectivity that proponents of algorithmic tools make. Concludes that the AFST discriminates on the basis of disability status. 
• Devansh Saxena, Erina Seh-Young Moon, Aryan Chaurasia, Yixin Guan, and Shion Guha, Rethinking “Risk” in Algorithmic Systems Through A Computational Narrative Analysis of Casenotes in Child-Welfare (2023): Discusses many critiques of algorithmic risk prediction tools. Highlights need for extreme caution in relying on newer natural language processing (NLP) systems now being marketed to and adopted by many agencies, noting, in particular, issues with relying on worker-produced case notes such as variability, inexperience and high turnover, documentation of “perceived risks” rather than factual information, “defensive decisionmaking” and omission of caseworker mistakes from notes, as well as the risk that “the contextual knowledge derived from casenotes can easily be stripped and instead exploited once quantified to be used in downstream tasks” among others. 
• Sally Ho and Garance Burke, An algorithm that screens for child neglect raises concerns, AP (April 29, 2022): Reporting on the use of the Allegheny Family Screening Tool (AFST) in Allegheny County, PA, as of 2022, along with considerations for other child welfare agencies adopting similar tools. Documents advocates’ efficiency claims assertion that social workers can always override the tool. Discusses critics’ concerns about the tool’s opacity and the potential for the tool to entrench existing racial disparities.
• Hao-Fei Cheng, Logan Stapleton, Anna Kawakami, Venkatesh Sivaraman, Yanghuidi Cheng, Diana Qing, Adam Perer, Kenneth Holstein, Zhiwei Steven Wu, and Haiyi Zhu, How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions (2022): Explores whether workers’ use of an algorithmic risk prediction tool in a child welfare context mitigates or exacerbates disparities, finding that while the tool, operating on its own, would worsen racially disparate treatment, workers are able to correct for this over-inclusion, and ultimately, workers operating with the tool make less disparate decisions than either worker or tool operating alone. Also highlights the distinction between “automated” and “augmented” decision-making with algorithmic child welfare tools. 
• Devansh Saxena, Karla Badillo-Urquiola, Pamela J. Wisniewski, and Shion Guha, A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare (2021): Develops a framework of algorithmic decision-making adapted for the public sector that balances 3-way interactions between human discretion, bureaucratic processes, and algorithmic decision-making. Applies the framework to a qualitative case study of algorithms in daily use at a mid-western U.S. child-welfare agency, discussing the benefits and drawbacks revealed, and proposes guidelines for the design of high-stakes algorithmic decision-making tools in the public sector.
• Anjana Samant, Aaron Horowitz, Sophie Beiers, and Kath Xu, Family Surveillance by Algorithm: The Rapidly Spreading Tools Few Have Heard Of (2021): National survey detailing which states and counties were employing, exploring, and/or discontinuing the use of predictive analytics in child welfare decisions as of 2021, including which specific tools jurisdictions were using and how agencies were using them.
• Stephanie K. Glaberson, Coding Over the Cracks: Predictive Analytics and Child Protection (2019): Argues the introduction of predictive analytics into an already-flawed system risks perpetuating biases and assumptions and magnifying existing problems. Describes the “decision points” and value-laden judgments integral to the child welfare system, the quintessentially human process of developing a machine learning algorithm, the predictive risk tools already implemented or then in development, and the ways those tools pose new risks to children and families.

If you want to go deeper:

Rua M. Williams, Disabling Intelligences: Legacies of Eugenics and How We are Wrong about AI (2025)

Ruha Benjamin, Imagination: A Manifesto (2024)

Joy Buolamwini, Unmasking AI: My Mission to Protect What Is Human in a World of Machines (2024)

Ulises A. Mejias and Nick Coudry, Data Grab: The New Colonialism of Big Tech and How to Fight Back (2024) 

Tania Duarte, Nicholas Barrow, Medina Bakayeva, Peter Smith, The Ethical Implications of AI Hype: Examining the overinflation and misrepresentation of AI capabilities and performance (2024)

Salomé Viljoen, A Relational Theory of Data Governance (2021)

Sasha Costanza-Chock, Design Justice: Community-Led Practices to Build the Worlds We Need (2020)

Ruha Benjamin, Race After Technology (2019) 

Virginia Eubanks, Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor (2018) 

Safiya Umoja Noble, Algorithms of Oppression: How Search Engines Reinforce Racism (2018) 

Lewis, T., Gangadharan, S. P., Saba, M., Petty, T., Digital defense playbook: Community power tools for reclaiming data, Our Data Bodies (2018)

Khiara Bridges, The Poverty of Privacy Rights (2017) 

Cathy O’Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy (2017)

For even more, see “A non-exhaustive collection of worth-reading books on topics strongly related to Critical AI (12th edition)” curated by Prof. Dr. Dagmar Monett

If you think something important is missing from this list, please send us an email!

What we’re doing:

The Privacy Center’s Family Surveillance work seeks to: 

• Empower organized communities with information. Privacy Center work in this area illuminates techniques and tools in use and pathways of data movement to support organized communities in protecting themselves. It seeks to illuminate the ways data moves — and doesn’t — so that communities can make educated decisions about their own lives rather than being guided by a lack of information, fear, or rumor. 
• Shape law and policy. The law – both unintentionally and by design – frequently serves to make communities more vulnerable to family policing interventions. Our work illuminates those legal avenues of vulnerability and supports communities in taking action to make needed changes. 
• Change the narrative. Through the work of impacted individuals, activists, and scholars, the political community has come to a more sophisticated understanding in recent years of the harm of criminalization and policing. Left out of that narrative all too often is the harm perpetrated by this system, which many imagine to provide merely a necessary intervention on behalf of children in need. Our work will build public awareness of the impact in the lives of children, parents, and communities of living under the shadow of family policing’s surveillant gaze.