The Real Reason Some Decisions Can Never Go to AI Has Nothing to Do With Capability

Ask, seriously, why a hiring decision or a legal judgment call shouldn’t be fully handed to AI, and most people reach for a capability argument — the tool isn’t smart enough yet, it makes mistakes, it lacks common sense. That framing is a trap, because it implies the answer changes once the technology improves. It doesn’t, and two completely unrelated regulatory and professional bodies have arrived at the exact same underlying principle from entirely different directions: accountability doesn’t transfer to a tool, no matter how capable that tool becomes. The decision doesn’t need a smarter AI. It needs a human who can actually be held responsible for it.

Why Capability Was Never the Right Test

This distinction matters more than it sounds. A capability-based argument against AI delegation is inherently temporary — it concedes that the objection goes away once the tool crosses some threshold of competence, and that threshold keeps moving. An accountability-based argument is structural, not technical. It says the problem isn’t whether the AI can produce a reasonable-looking output; it’s that no output, however good, comes with the thing certain decisions actually require: a specific, identifiable person who bears real responsibility for the outcome and can be held to account if it goes wrong.

This is exactly the principle two very different institutions have independently converged on.

The Employment Example: Why “The Algorithm Did It” Isn’t a Defense

Employment law offers the clearest, most consequential version of this principle in action. The U.S. Equal Employment Opportunity Commission launched a formal Artificial Intelligence and Algorithmic Fairness Initiative specifically to ensure that AI and algorithmic decision-making tools used in hiring and other employment decisions comply with federal civil rights laws, with the agency stating explicitly that while the technology may be evolving, anti-discrimination laws still apply regardless of the form bias takes. The EEOC’s subsequent guidance has been unambiguous on a specific point that matters enormously here: an employer cannot shift liability for a discriminatory hiring outcome onto a third-party AI vendor. If an algorithmic tool produces a discriminatory result — even one the employer didn’t design, didn’t fully understand, and was told by the vendor was “bias-free” — the employer using it remains legally accountable, not the software or the company that built it.

This isn’t a hypothetical concern. A well-documented EEOC enforcement action involved an automated hiring system that was found to have rejected older applicants based on age, resulting in a settlement requiring monetary relief and changes to hiring practices. The employer hadn’t personally designed the discriminatory filter — it had adopted a third-party tool and let it run. That gap between “we didn’t personally decide this” and “we’re still responsible for it” is precisely the accountability principle at work: delegating the decision-making process to an algorithm didn’t delegate away the consequences of that process, and it never will, regardless of how sophisticated hiring AI eventually becomes, or how many independent audits a vendor claims to have run before selling the tool.

The Legal Example: Why Client Trust Can’t Be Outsourced

A completely different profession, working from entirely different concerns, reaches the same structural conclusion. The American Bar Association’s Formal Opinion 512, its first formal ethics guidance addressing generative AI, states that lawyers remain fully responsible for the accuracy and appropriateness of their work product regardless of what role AI tools played in producing it, and that this responsibility cannot be transferred to the AI tool or excused by its use. A lawyer who submits a filing containing AI-fabricated case citations is professionally and often legally accountable for that filing — “the AI made it up” is not a defense recognized by any bar association or court, and several attorneys have already faced sanctions on precisely this basis.

What’s notable is that the ABA’s reasoning isn’t really about AI’s current error rate, even though hallucination is part of what makes the underlying risk concrete. It’s about the nature of legal representation itself: a client retains a specific attorney, and that attorney’s professional judgment, license, and accountability are what the relationship is actually built on. No tool, however reliable, can stand in for that specific, personal accountability, because the entire structure of legal ethics is built around a person being answerable for the advice given, not a process being generally trustworthy on average.

The Common Thread: Decisions That Need a Person to Answer For Them

Line up the EEOC’s position on hiring and the ABA’s position on legal practice and the shared logic becomes obvious, even though the two fields have almost nothing else in common. In both cases, the concern isn’t primarily that AI might get the specific answer wrong, though that risk is real and compounds the underlying problem. It’s that certain decisions structurally require a person who can be held to account — someone whose judgment, license, or professional responsibility is genuinely on the line, not just a process that produced a plausible-looking output. Remove that accountable person from the loop, and you haven’t actually made the decision more efficient. You’ve made it unanswerable, which is a fundamentally different and more serious problem than an occasional wrong answer.

This connects directly to a point worth returning to from a related piece. Our Anti-Automation Manifesto covers a broader set of tasks that resist delegation on similar grounds, and the EEOC and ABA examples here add a harder, more institutional edge to that argument — this isn’t just a matter of taste or caution about AI’s current limitations. In at least these two well-documented fields, it’s now formal regulatory and professional guidance, backed by real enforcement actions and sanctions.

Identifying Non-Delegable Decisions in Your Own Work

The practical test that emerges from both examples: does this decision require someone to be personally, specifically accountable if it goes wrong — professionally, legally, or ethically — in a way that can’t be satisfied by “the process was reasonable on average”? Final hiring and termination decisions, professional judgment calls that carry licensure or legal responsibility, anything requiring a signature that represents personal accountability rather than administrative formality, and decisions where a specific person needs to be answerable to a client, patient, or regulator all fall into this category, regardless of how good the AI assistance behind them becomes.

This is a meaningfully different filter than the risk-based sorting covered in our piece on what’s actually safe to automate, which focuses on how expensive and catchable a mistake is. Accountability is a separate axis entirely — a decision can have a genuinely low error rate and still be non-delegable, because the issue isn’t primarily about error frequency. It’s about whether a real person remains answerable for the outcome regardless of how it was reached. Our guide to fact-checking AI answers helps reduce errors in the decisions that do get AI assistance, but it doesn’t substitute for the accountability question this piece is specifically about — those are two different, complementary safeguards, not interchangeable ones.

What This Looks Like Outside Law and Employment

It’s worth extending this principle past the two specific fields with formal guidance already in place, because the underlying logic clearly generalizes. A financial advisor recommending a specific investment strategy to a client is accountable for that recommendation in a way no AI-generated analysis can be, regardless of how sophisticated the underlying modeling is — the client trusted a licensed person, not a process, and that person remains answerable to both the client and any regulator reviewing the advice later. A manager delivering a performance review that affects someone’s career is accountable for the judgment behind that review in a similarly personal way, even if AI helped organize the supporting notes. In both cases, no formal regulatory guidance yet exists the way it does for hiring algorithms or legal filings, but the same structural question applies: if this goes wrong, who specifically has to answer for it, and does an AI-generated recommendation, however well-reasoned, actually satisfy that requirement?

The honest answer, in nearly every case where the stakes are genuinely personal or professional rather than purely administrative, is no. AI can meaningfully inform these decisions — gathering data, surfacing considerations, drafting the reasoning — but the moment of actually deciding, and being willing to stand behind that decision by name, remains something no output can substitute for, regardless of the field or whether a formal ethics opinion has been written about it yet.

Auditing Your Own Non-Delegable Decisions

A useful, direct exercise: list the decisions in your role where a mistake would require you, specifically and personally, to explain what happened to a client, a regulator, an employee, or a court — not the organization abstractly, but you, by name. Those decisions belong firmly in human hands, and AI assistance around them should be limited to research, drafting, and analysis that you personally review and stand behind, never to the final decision itself. Everything else — decisions where the accountability genuinely sits with a broader process rather than a specific accountable person — has real room for AI to help, provided the risk-based safeguards from disciplined automation still apply.

Frequently Asked Question