The productivity narrative around AI is almost entirely constructed from cases where AI works. A task that took an hour takes ten minutes. A report that required specialized knowledge is drafted by someone without that knowledge. A piece of code that would have required a senior engineer is produced by a junior one. These cases are real. They should inform decisions about AI adoption.
What does not appear in the narrative is the category of cases where AI introduction makes things worse — where the integration creates more risk, more cost, or more error than the workflow without AI. These cases also exist. They are not edge cases. They are predictable failures that follow from structural properties of the tasks and the systems involved.
Understanding when not to use AI is as important as understanding when to use it. Without that understanding, organizations apply AI integration indiscriminately, discover failures after deployment, and draw the wrong conclusions: either that the specific tool was wrong (and the next tool will work) or that AI adoption was premature (and the organization should wait). The actual lesson — that the use case was wrong — gets missed, and the same mistake is made with the next tool or in the next cycle.
What follows is a description of six structural cases where AI integration creates problems rather than solving them, and what to do in each case instead.
Case 1: High-Stakes Decisions Requiring Human Accountability
There is a category of decisions where the organizational requirement is not just a good outcome — it is a traceable, accountable human decision. Regulatory filings, clinical recommendations, disciplinary actions, credit underwriting, legal interpretations, and significant resource allocation decisions all fall into this category. The requirement for human accountability is not incidental. It is built into the legal, ethical, or organizational framework governing the decision.
When AI is embedded in these decisions in ways that position the AI output as the decision rather than as input to a human decision, the accountability structure is violated. The harm is not limited to bad outcomes — the harm is the inability to establish who made the decision and on what basis, which is a governance failure independent of outcome quality.
The failure mode is subtle in practice because AI outputs can be technically reviewed by a human before the decision is recorded. Nominal human review — a person reads the AI recommendation and approves it without substantive evaluation — is legally and ethically insufficient as an accountability structure even if it produces the correct outcome most of the time. The accountability requirement is for a decision-maker who can be evaluated against the criteria that apply, not for a person who approved an AI recommendation. These are different standards.
What to do instead: Use AI to prepare inputs to the human decision — to synthesize relevant information, surface applicable precedents, and identify the considerations that should be weighed — while preserving the human decision as a genuine exercise of judgment rather than an approval step. The AI is a researcher and drafter. The human is the decision-maker.
Case 2: Contexts Where Errors Compound Before Detection
AI systems make errors at rates that vary by task and domain. For many tasks, the error rate is low enough and errors are detectable quickly enough that the net value of AI assistance is clearly positive. But there is a class of contexts where AI errors are not caught at the point of generation and compound through downstream processes before they become visible.
Consider a workflow where AI generates a data extraction from a set of documents, the extracted data is used to populate a structured database, the database is queried for a monthly report, and the report is used for planning decisions. An error in the extraction step may not be visible at the extraction step, at the database population step, or at the report generation step. It becomes visible — if at all — at the planning decision step, when someone with the right domain context looks at the planning recommendation and finds it implausible. By this point, the error has propagated through three steps and influenced decisions that were made in the intervening period.
The risk is not that AI makes errors. The risk is compounding error propagation in systems where the detection point is far downstream from the generation point. This risk is structural — it is a property of the workflow design, not of the AI system — and it is present whenever AI is embedded in multi-step processes where intermediate outputs are not independently verified.
What to do instead: Map the error propagation path before integrating AI. Identify where AI-generated outputs feed downstream processes and how far errors would travel before detection. For high-propagation paths, add verification steps between AI generation and downstream use — not nominal review, but substantive verification against an independent source. If independent verification is not feasible, position AI assistance earlier in the workflow where errors are still visible and correctable, rather than at the point where errors will propagate most broadly.
Case 3: Workflows Where Explainability Is Legally or Ethically Required
Some decisions are governed by requirements to explain the basis on which the decision was made. Loan denials in regulated markets, employment decisions in jurisdictions with documentation requirements, medical decisions with informed consent requirements, public procurement with audit requirements — in all of these contexts, the decision-maker must be able to articulate the factors that drove the decision in terms that are defensible under scrutiny.
AI systems, particularly large language models and sophisticated ML models, do not produce explainability as a natural output. They produce outputs. The reasoning path that produced those outputs is frequently not traceable in a way that satisfies explainability requirements. A language model that produces a loan risk assessment cannot explain why it assigned a particular risk level in terms that are legally defensible, because its reasoning process is not organized around defensible criteria in the way that a human underwriter''s documented reasoning is.
Organizations that embed AI in explainability-required contexts and then attempt to construct explanations after the fact are building compliance theater rather than compliant processes. The explanation is constructed to justify the decision rather than to document the decision-making process. This is a documentation failure that is also a compliance failure.
What to do instead: Use AI upstream of explainability requirements, not at them. AI can prepare materials, synthesize relevant information, and draft initial assessments. The formal decision — the one that the explainability requirement applies to — is made by a human who documents their reasoning in terms that satisfy the applicable requirements. The AI preparation work is internal process; the documented decision is the human decision.
Case 4: Small-N Problems Where Training Data Is Insufficient
AI systems learn from data. The quality and volume of relevant training data is the primary determinant of reliability in a specific domain. This is well understood in general terms, but the specific implications are frequently not worked through for the use cases under consideration.
A language model trained on general text corpora will perform poorly on highly specialized, domain-specific tasks where the relevant vocabulary, the relevant context, and the relevant reasoning patterns are not well represented in its training data. The model will produce fluent, confident output — because that is what language models produce — but the accuracy in the specific domain will be materially lower than in domains that are well-represented in training.
The small-N problem is more specific: it applies to any task where the relevant reference set is small enough that the model has not encountered enough examples to have developed reliable patterns. A model asked to evaluate the risk profile of a novel financial instrument that was created after its training cutoff has no relevant training data. A model asked to assess the compliance implications of a regulation that applies to a highly specific industry in a specific jurisdiction may have encountered only a handful of relevant cases. A model asked to reason about a very specific technical architecture that is used by only a few organizations may be extrapolating from superficially similar architectures with different properties.
The tell is when the model produces confident output on a topic where confident output should not be possible — where the answer is genuinely uncertain, or where the relevant facts are specific to a context the model cannot have trained on. Users without domain expertise cannot always identify this failure mode, which is what makes it dangerous.
What to do instead: For small-N, high-specificity problems, use AI for the components that are well within its training distribution — drafting, summarizing, structuring — and apply human domain expertise for the components that require it. Do not ask AI to evaluate specific, novel, or highly specialized questions and treat the output as authoritative. The model does not know what it does not know, and in small-N domains, what it does not know is often the relevant thing.
Case 5: Systems Where AI Output Quality Cannot Be Verified by the People Using It
Human-in-the-loop design assumes that the humans in the loop can meaningfully evaluate AI outputs. In many workflows, this assumption is not valid: the person reviewing the AI output does not have the domain expertise to evaluate whether it is correct.
This is not a failure of human capability. It is a workflow design failure. The review step was included in the design because governance requires human review, but the person available to perform the review — based on who is in the workflow at that point — does not have the knowledge needed to catch the relevant errors.
Examples: A non-technical manager reviewing AI-generated code for a security vulnerability assessment cannot evaluate whether the assessment is complete or accurate. A general administrative staff member reviewing AI-generated regulatory compliance documentation for a specialized industry cannot evaluate whether the compliance guidance is correct. A junior analyst reviewing AI-generated risk assessments that require actuarial expertise to evaluate is not providing meaningful review.
In all of these cases, the nominal human review step creates the appearance of oversight without the substance of it. Errors that require domain expertise to catch pass through the review step uncaught. The governance structure appears to be functioning because a human reviewed the output. In practice, the error rate for expert-level errors is the same as if no human review had occurred.
What to do instead: Either position the review step where the appropriate expertise exists — route AI outputs on technical questions to people with technical expertise — or reduce AI involvement in the components that require expert review. If expert review cannot be consistently provided, AI should not be generating outputs in that domain that will not receive expert scrutiny.
Case 6: Trust-Sensitive Contexts Where Automation Reduces Credibility
There is a category of professional interactions where the value being delivered is not only the information or recommendation itself but the relationship and human judgment that produced it. A therapist''s value is not reducible to the content of their observations. An experienced advisor''s value is not reducible to the quality of their recommendations. A teacher''s value is not reducible to the accuracy of the content they convey. In these contexts, the relationship, the process of individual attention, and the visible exercise of human judgment are part of what is being offered.
When AI is used to generate the substantive outputs in these contexts — the observations, the recommendations, the feedback — and the human is presenting those outputs as their own exercise of judgment, the professional relationship is misrepresented. This is not only an ethical problem. It is a functional problem: the client, patient, or student is making decisions based on the belief that they received something they did not receive. If they learn that the substantive judgment was AI-generated, the trust damage is severe and typically not recoverable.
There is a subtler version of this problem in B2B professional services. A client retains an advisor because they have confidence in that advisor''s judgment and experience. When the advisor uses AI to generate the core analysis and presents it as their own analysis, the client is paying for a service they did not receive. Again, the harm is not only ethical. The advice quality may be adequate or even superior to what the advisor would have produced without AI. But the client''s basis for trusting the advice — the advisor''s specific expertise and judgment — is not what is actually providing the advice.
What to do instead: Be transparent about AI use in professional relationships where the relationship itself is part of the value. Use AI for research, preparation, and drafting, and be clear that these are AI-assisted. The substantive judgment — the advisor''s analysis of what the AI-generated material means for this specific client, the teacher''s evaluation of where the AI-generated explanation missed the student''s actual confusion, the therapist''s reading of what the AI-generated summary of the session does not capture — is the human contribution, and it is the one being paid for.
A Decision Framework for Evaluating Whether AI Belongs in a Workflow
The six cases above have a common structure: each describes a condition under which AI integration creates risk that outweighs its benefit. The decision framework for evaluating any candidate AI integration involves checking for these conditions explicitly.
Accountability check: Does this workflow produce decisions where human accountability is legally, ethically, or organizationally required? If yes, is the integration design structured so that a human is making a genuine decision — not approving an AI recommendation — and can document that decision?
Propagation check: When AI makes an error at this integration point, how far does that error propagate before it is detected? Is independent verification feasible between AI generation and downstream use? If not, is the integration point appropriate?
Explainability check: Does this workflow produce decisions that are subject to explainability requirements? Is AI positioned upstream of the formal decision point rather than at it?
Training data check: Is the task within the AI system''s training distribution? Is the relevant reference set large enough that the AI has developed reliable patterns? If the domain is highly specialized, novel, or narrow, is AI use limited to the components that are well within its training distribution?
Verifiability check: Can the people reviewing AI outputs at this workflow step meaningfully evaluate whether those outputs are correct? If not, can the review be routed to someone who can? If neither, should AI involvement in this component be reduced?
Trust check: Is the relationship delivering value that depends on the client''s belief that human judgment is being applied? Is AI use transparent in this relationship?
These checks do not produce automatic go/no-go decisions. They produce the right questions, and the answers to those questions inform integration design. Some integrations that fail one check can be redesigned to address the condition — by repositioning AI earlier in the workflow, by adding a substantive review step, by being transparent about AI use. Others cannot be made to work without a fundamental change to the workflow.
The output of a responsible evaluation is not "AI works here" or "AI does not work here." It is a specific integration design — with defined review steps, clear accountability, explicit explainability handling, and honest acknowledgment of the use-case limits — or a clear decision that AI integration in this workflow creates unacceptable risk at this time.
The Honest Version of the Productivity Narrative
AI tools are useful for a large and growing set of tasks. The productivity gains available from well-designed AI integration are real, and organizations that do not develop the capacity to integrate AI responsibly will be at a structural disadvantage relative to those that do.
That is true. It is also incomplete.
The complete version includes the cases where AI integration fails — not because the tools are inadequate, but because the conditions required for responsible integration are absent. It includes the accountability failures, the propagation risks, the explainability gaps, the training data limits, the verification gaps, and the trust erosions that occur when AI is applied without evaluating whether it should be.
An organization that has done the work of understanding when not to use AI is more capable of responsible AI adoption, not less. It is not slower to deploy or more conservative in its ambition. It is clearer about where its investments should be concentrated, which use cases are ready for integration, and which require governance work before integration is responsible. That clarity produces better outcomes than unrestricted deployment — not because caution is inherently valuable, but because it is based on a more accurate model of where AI creates value and where it creates risk.
The six cases described in this article are not meant to counsel against AI adoption. They are meant to be the honest companion to every productivity claim: here is where AI works, here is where it creates value, and here — specifically, structurally, not as a general caveat — is where it does not.