How to Choose Your Nonprofit's First AI Project

In 2023, a child protection worker in Victoria, Australia, used ChatGPT to help draft a report for the Children's Court in a case involving a young child. The state's privacy regulator, the Office of the Victorian Information Commissioner, investigated and found that the worker had entered personal information about the family, including names, and that the report contained inaccuracies, one of which downplayed the harm to the child. A review of cases in the worker's unit turned up about 100 over a year in which staff may have used ChatGPT to draft child protection documents. The Department of Families, Fairness and Housing had general technology policies and some training in place, but no specific rules for generative AI, and the regulator judged its controls insufficient. In September 2024 it issued a compliance notice that barred child protection staff from using generative AI tools and required the department to block them by November (OVIC, 2024; Hendry, 2024). The regulator found that the errors did not change the court's decision.

Of the human-services failures I know, this one teaches the most, largely because so little of it was about technology. As the leader of a child welfare organization, I find it frightening, and I also find it useful, because it shows how easily good intentions go wrong when nobody has set the boundaries. A worker under pressure reached for a tool that made the job easier. Nobody had decided which tasks suited AI, what information could go into it, or how anyone would check the output; the policies said the right things, but nothing in the system made the right thing the easy thing.

As the first article in this series showed, your staff are probably using AI already. That changes the decision in front of most nonprofit leaders. Either the organization picks its first project deliberately, with rules and checks, or one emerges on its own.

Start with the work, not the tool

Take four candidate projects at a hypothetical mid-sized family services nonprofit, each of which could plausibly save time:

1. A staff assistant that answers questions about the organization's own policies, procedures, and eligibility rules, drawing only on approved documents.

2. A tool that turns caseworkers' recorded home-visit conversations into draft case notes.

3. First drafts of donor thank-you letters and grant report narratives.

4. A chatbot on the website that helps families find community resources.

Before comparing them, map what happens today in each workflow: who does what, where it slows down, what information moves, and where a mistake would land. This step sounds tedious, yet most of the useful learning happens here, because a process map often shows that the slow step is a handoff or an approval, in which case AI is aimed at the wrong part of the job.

Screen for data first, then for review

For any organization that handles client information, the first screen is what will go into the tool, and a failed answer should stop the project until it is fixed. Personal and free consumer AI accounts are the wrong place for client information. Organizations covered by HIPAA face a specific rule: federal guidance treats a cloud service that handles protected health information as a "business associate," which must sign a business associate agreement before it handles that information (HHS, Office for Civil Rights). Consumer chatbots generally do not offer one, and a health lawyer at Holland & Hart points out that the agreement is necessary without being sufficient on its own (McCue, 2026). Substance use treatment records carry added federal protection under 42 CFR Part 2, whose updated rule carried a compliance deadline of February 16, 2026 (Chestler & Rush, 2026).

Plenty of community nonprofits are not HIPAA-covered entities, which does not settle the matter. State contracts often carry their own confidentiality terms, so read yours, and talk with counsel before piloting anything that touches client data; this section is orientation, not legal advice. Victoria adds a practical lesson here. If staff can paste a case file into a personal account, some eventually will, usually with good intentions, so a policy needs technical backing: approved business accounts with data protections, settings that keep your data out of model training, blocks on unapproved tools, and someone who periodically looks at how tools are really being used.

Review capacity is the second screen. Ask what happens when an output is wrong, and whether a qualified person has the time and knowledge to catch it. AI note-taking, the most common first project in social care, shows why this matters. In February 2026 the Ada Lovelace Institute, an independent UK research group, published Scribe and prejudice?, built on interviews with 39 people, 29 of them social workers in 17 local authorities that use AI transcription. Workers described garbled transcripts and misspelled names, and inaccurate transcripts for people with regional accents. One summary recorded a client as expressing suicidal thoughts the client never voiced; a social worker caught it before it reached the record. Elsewhere, one authority's own quality checks found that AI often did not improve the assessments and may have let unchecked errors into official care records (Bruff & Groves, 2026). Because the study relied on interviews, it describes problems without measuring how often they occur.

Councils and vendors, meanwhile, report large savings from the same tools. Kingston Council in London says its social workers report spending 50–60% less time on case notes and assessments (Local Government Association, 2025), and Swindon Borough Council, which piloted the same kind of tool with its vendor, reports that writing up assessments fell from four hours to an hour and a half (Swindon Borough Council, 2024). Neither account describes a comparison group or an independent accuracy check. Both accounts can be true at once, since a tool can save real time while handing some of it back as checking. A review published in England's National Workload Action Group series makes exactly this point, warning that AI case recording moves time from typing to quality assurance and that hours saved may not become money saved (Rothera, 2025). Measure only drafting time in your pilot and you will overestimate the benefit.

A safer first project, and how I would rank the four

Washington, DC's Child and Family Services Agency points toward a better starting place. Its staff chatbot answers questions about policies and procedures from approved documentation, not from confidential case files. The example comes from a July 2026 piece in the Child Welfare Information Gateway's Gateway Exchange, which also gives sensible starting advice: begin with a narrow use case, rule out automated decisions with serious consequences, document data sources and quality checks before building, define success before piloting, and give staff a way to flag and challenge errors (Child Welfare Information Gateway, 2026). It is guidance from an information service rather than a regulation, and it reports no outcome data.

For evidence, the most careful evaluation I have found in social services is the test by Nava PBC and the nonprofit Amplifi of a benefits chatbot for caseworkers in Los Angeles County. Their tool answers only from vetted sources, cites them, and declines questions outside its scope. When 125 caseworkers answered hypothetical client questions in a randomized test that used simulated chatbot suggestions, AI support raised their accuracy by about 40% on average. The 14-week field pilot, with 61 caseworkers at six organizations, was more sobering: about 65% used the tool, use tapered off, the evidence on reduced workload was promising but inconclusive, and adoption depended on training and peer support (Chen et al., 2026). Nava built the tool and led the evaluation with researchers from Georgetown and Cornell Tech, and the randomized part used hypothetical questions and simulated answers. Even so, few studies come this close to our sector.

What the DC and Nava tools have in common is a shape worth copying: both face staff rather than the public, both draw on documents the organization controls, and in both a person decides what to do with the answer. With that shape in mind, I would start the hypothetical organization on the staff policy assistant. Client information stays out of it, and the reviewer is built in, because the staff member who asks the question decides what to do with the answer; along the way, the organization learns how to set up accounts and measure results. Donor and grant drafting is a reasonable second choice, provided no client stories go in without consent and someone checks every figure. The public chatbot for families comes last, because it puts AI output in front of vulnerable people with no staff member in between. Your ranking may differ with your services and the information you hold, but make it on purpose, before your staff's informal experiments make it for you.

The case-note tool deserves separate treatment, because it may offer the most value of the four. The savings councils report from these tools are large even after checking time is counted, and in human services, time returned from documentation is time that can go back to families. Case notes also carry the strictest security and safety protocols an organization keeps: who may see a record, where recordings are stored, how long they are kept, and how they are destroyed. Those policies and procedures are the guardrails that make the project possible, and building them is part of the project. An organization that puts them in place first, with a data agreement, a consent process, access controls, and checking time in the budget, can pursue one of the most valuable uses of AI in its work without putting the families in its records at risk. I would wait on the case-note tool until those guardrails exist, and then treat it as the project the first pilot was preparing you for.

Test it, because performance depends on the task

A tool that handles one task well can stumble on a closely related one, and it is hard to tell in advance which is which. An experiment with 758 Boston Consulting Group consultants, published in Organization Science in 2026, shows the pattern clearly. On tasks within the AI's abilities, consultants using GPT-4 finished about 12% more tasks, about 25% faster, with higher-rated quality; on one task chosen to fall outside those abilities, they were about 19 percentage points less likely than colleagues without AI to reach the correct answer (Dell'Acqua et al., 2026). That was a 2023 model, and the boundaries have moved since. You still have to test your own task to learn where it falls.

Impressions will not tell you. Experienced software developers in a 2025 study by the research group METR took 19% longer to finish tasks with AI while believing it had made them about 20% faster (Becker et al., 2025). METR's larger 2026 follow-up produced estimates pointing toward speedups, with enough uncertainty that the group called them weak evidence, although it now believes developers are probably faster with current tools (Becker et al., 2026). For a nonprofit pilot, the takeaway is about measurement rather than speed. What people feel and what actually happens can diverge, and both shift as tools change.

A pilot needs a named owner, a baseline (how long and how well the work gets done today), a definition of acceptable quality, and a list of failures you will not accept. Agree on stop conditions in advance so that ending the pilot is a decision rather than an embarrassment. Track the whole time cost, including preparation, prompting, checking, correction, and follow-up, and record setup costs apart from ongoing ones. If the tool records or transcribes people, get consent first and decide how AI-assisted records will be marked. Kingston Council's rollout is a sensible template: a data processing agreement with the vendor, a data protection impact assessment, an updated privacy notice, consent before recording, sign-off from the council's internal transformation board, and a start with three social workers before expanding (Local Government Association, 2025).

Work in small cycles, adjusting between them; the Institute for Healthcare Improvement's Plan-Do-Study-Act method, long used in health care, is built for exactly this, and IHI advises keeping early tests as small as possible (Institute for Healthcare Improvement). Resist generalizing from a few good results; the National Institute of Standards and Technology's generative AI guidance warns specifically against extrapolating from narrow, nonsystematic, and anecdotal assessments (NIST, 2024). Ten clean summaries do not show that the hundredth will be free of a serious error. If all this feels unfamiliar, you are in good company: only 10 of the 36 OECD countries surveyed report measuring the impact of their government AI projects (OECD, 2026), so a small nonprofit that measures honestly will know more than most.

A safe first project can turn into a permanent sideshow, a pleasant demonstration that never touches the organization's real work, so choose a pilot that teaches you something you will need later, such as how to set data rules or how much checking time a tool really costs. Budget for people as well as licenses, too; Nava's results suggest that even a useful tool fades without training and peer support.

Tools for choosing and testing

Project selection worksheet. These criteria are a starting point, not validated weights. Do not turn them into a score, and never let a high convenience rating offset a failed data or review screen.

Pilot brief. Before starting, write down the workflow, owner, approved information, tool and version, account type and data agreement, participating staff, review process, baseline, timeframe, acceptable quality, unacceptable failures, and stop conditions.

References

Becker, J., Rush, N., Barnes, B., & Rein, D. (2025, July 10). Measuring the impact of early-2025 AI on experienced open-source developer productivity. METR.

Becker, J., Rush, N., Cunningham, T., Rein, D., & Mahamud, K. (2026, February 24). We are changing our developer productivity experiment design. METR.

Bruff, O., & Groves, L. (2026, February 11). Scribe and prejudice? Exploring the use of AI transcription tools in social care. Ada Lovelace Institute.

Chen, M., Esposito, M., Giannella, E., Guo, Z., Gosciak, J., & Koenecke, A. (2026). Helping the helpers: Evaluating a GenAI-powered assistive chatbot for caseworkers. Nava Labs, Nava Public Benefit Corporation.

Chestler, A. L., & Rush, L. C. (2026, January 21). HHS compliance deadline approaching for updated Part 2 record protections. Baker Donelson.

Child Welfare Information Gateway. (2026, July). AI in child welfare: A spotlight on workforce development and analytics. Gateway Exchange, 1(2). Children's Bureau, Administration for Children and Families. Partner spotlight by R. D. Blagg.

Dell'Acqua, F., McFowland, E., III, Mollick, E., Lifshitz, H., Kellogg, K. C., Rajendran, S., Krayer, L., Candelon, F., & Lakhani, K. R. (2026). Navigating the jagged technological frontier: Field experimental evidence of the effects of artificial intelligence on knowledge worker productivity and quality. Organization Science. Advance online publication. https://doi.org/10.1287/orsc.2025.21838

Hendry, J. (2024, September 25). Vic child protection agency ordered to block GenAI tools. InnovationAus.

Institute for Healthcare Improvement. (n.d.). How to improve: Model for improvement: Testing changes.

Local Government Association. (2025, January 28). Kingston Council: Using AI in adult social care administration.

McCue, C. (2026, June 19). AI chatbots and HIPAA: What healthcare providers need to know. Holland & Hart LLP.

National Institute of Standards and Technology. (2024). Artificial intelligence risk management framework: Generative artificial intelligence profile (NIST AI 600-1). https://doi.org/10.6028/NIST.AI.600-1

OECD. (2026). Adopting and governing AI in government. In Digital government outlook 2026: From foundations to transformational impact. OECD Publishing. https://doi.org/10.1787/0496b2bc-en

Office of the Victorian Information Commissioner. (2024, September). Investigation into the use of ChatGPT by a Child Protection worker.

Rothera, S. (2025, September). Reducing unnecessary social worker workload through the use of AI in case recording. Research in Practice, for the Department for Education.

Swindon Borough Council. (2024, October 1). Council completes AI trial to enhance Adult Social Care services.

U.S. Department of Health and Human Services, Office for Civil Rights. (2022). Guidance on HIPAA & cloud computing.