What Data Should You Never Give AI?

Somewhere in a typical week, most regular chatbot users hand over something they’d never post publicly — a salary figure while asking for negotiation advice, a symptom while asking about a diagnosis, a chunk of work code while asking for a bug fix. It doesn’t feel like exposure. It feels like talking to a very capable, very patient assistant who happens to live in a browser tab. That framing is the problem. A chatbot isn’t a diary, a doctor, or a coworker bound by confidentiality (what data should you never give AI)— and treating it like one is how sensitive information ends up somewhere it was never supposed to go.

This isn’t a call to panic or to stop using AI tools. It’s a practical map of the categories of information that carry real, documented risk when they’re typed into a chat window, and why each category is riskier than it feels in the moment.

Why This Is a Genuinely Different Risk Than Oversharing With a Person

When you tell a friend something private, the risk is bounded — it lives with that one person, subject to their judgment and your relationship. Typing something into an AI chatbot is structurally different. Depending on the tool and its settings, what you enter may be stored, reviewed by human trainers as part of quality checks, or used to further train future versions of the model — meaning it doesn’t just sit there, it potentially becomes part of the system’s underlying knowledge in ways that are difficult or impossible to fully undo later. There’s no equivalent of asking a person to forget something.

This isn’t a hypothetical concern dreamed up by cautious IT departments. Bloomberg reported that Samsung banned employee use of ChatGPT and similar generative AI tools after discovering staff had uploaded sensitive internal source code to the platform, with the company specifically flagging that data sent to such tools is stored on external servers, making it difficult to retrieve or delete. That single incident, involving proprietary code entered by engineers trying to solve a technical problem, is a useful template for thinking about risk generally: the person entering the data almost never intends harm. They’re just trying to get something done faster, without registering that the tool they’re using doesn’t behave like a private conversation.

Government-Issued IDs and Login Credentials

This is the most straightforward category, and also the one people violate almost reflexively. Social Security numbers, passport numbers, driver’s license numbers, and any kind of login credential — usernames, passwords, security question answers — should never be typed into a chatbot, full stop. There’s no legitimate task that requires an AI tool to see your actual password or government ID number to help you; if a chatbot is walking you through a process that seems to need one, that’s a moment to pause, not to paste.

Financial Account Details

Bank account numbers, full credit card numbers, and other precise financial identifiers belong in this same never-share category, and the concern here isn’t hypothetical either. The Consumer Financial Protection Bureau examined the privacy risks of AI chatbots in consumer finance and concluded that personal and financial information entered into chat logs should be treated as sensitive consumer information requiring the same level of security as any other confidential financial record — noting that chat interfaces introduce an additional venue for privacy breaches beyond a company’s traditional systems. If you’re using an AI tool to help draft a budget or understand a financial concept, work with rounded numbers and hypothetical scenarios rather than your actual account details — the tool doesn’t need the real figures to be useful, and the difference in risk is significant.

Workplace and Client Confidential Information

This is the category most people underestimate, largely because it doesn’t feel personal — it feels like just getting work done. But proprietary source code, unreleased product details, internal strategy documents, and client information all carry real professional and legal exposure when pasted into a public AI tool, as the Samsung example above demonstrates concretely.

The legal profession has been especially explicit about this, and its guidance generalizes well beyond law. The American Bar Association’s Formal Opinion 512, its first formal ethics guidance on generative AI, states that lawyers are responsible for understanding how a given AI tool uses submitted data and must put safeguards in place to prevent client information from being inadvertently disclosed to third parties — recommending that lawyers secure clients’ informed consent before entering confidential information into AI tools at all, since standard boilerplate consent language in an engagement letter isn’t considered sufficient. You don’t need to be a lawyer for the underlying logic to apply. Anyone handling information on behalf of a client, patient, employer, or organization with a duty of confidentiality is taking on real risk by feeding that information into a consumer AI tool without knowing exactly how the tool handles it.

Health Information and Anything You Wouldn’t Want Publicly Linked to You

Consumer-facing AI chatbots generally aren’t built or contractually bound the way a healthcare provider’s systems are, which means detailed symptoms, diagnoses, medication names, and other health specifics deserve the same caution as financial data — useful to discuss in general or hypothetical terms, but risky to attach directly to your real identity and circumstances. The same logic extends to anything else you’d be uncomfortable seeing surface later, disconnected from context: details about a legal dispute, a relationship conflict, an immigration situation, or anything else where the specifics, not just the general topic, could cause harm if they resurfaced somewhere unexpected.

The Structural Reason This Keeps Happening

It’s worth understanding why data exposure through AI tools is a recurring, structural risk rather than a rare mistake. The National Institute of Standards and Technology’s Generative AI Profile, part of its broader AI Risk Management Framework, identifies data privacy and the risk of sensitive information leaking through a model’s outputs as one of the core categories organizations need to actively manage when deploying generative AI systems. NIST’s framing treats this as an ongoing risk to be managed continuously, not a one-time setting to configure and forget — which is a useful mental model for individual users too. The risk doesn’t go away because you were careful once. It requires the same vigilance every time you open a new chat.

Other People’s Information, Especially Children’s

It’s easy to focus entirely on your own exposure and forget that a lot of what gets typed into a chatbot belongs to someone else. A colleague’s performance review draft, a friend’s personal situation you’re seeking advice about, a family member’s medical update — all of it involves another person’s private details, entered without their knowledge or consent. That’s worth pausing on independent of any AI-specific risk, but AI tools raise the stakes because that information doesn’t stay in a private conversation the way it would if you’d simply talked it through with someone else.

Children’s information deserves its own specific caution. Names, ages, school details, photos, or anything that could identify a minor shouldn’t be entered into general-purpose AI tools, both because children can’t consent to that exposure and because the long-term consequences of data persisting somewhere are harder to predict for a young person whose life is still unfolding. If you’re using AI to help with something involving a child — homework help, a parenting question, a school project — describe the situation generally rather than including identifying details.

A Practical Checklist Before You Type

Turning all of this into a habit is easier with a short mental checklist rather than trying to remember every category individually:

• Strip identifiers, keep the substance. Most tasks work just as well with “a family member” instead of a name, “a mid-size company” instead of your employer, or rounded numbers instead of exact figures.
• Check who owns the information. If it belongs to someone else — a client, a colleague, a family member, a child — treat it with more caution than you would your own details, since they haven’t consented to the exposure.
• Know your tool’s settings. Many AI platforms offer a setting to exclude your conversations from model training, and business or enterprise tiers often come with stronger contractual data protections than the free consumer version of the same product.
• When genuinely unsure, don’t. If a task seems to require sensitive information and you can’t find a way to generalize it, that’s usually a sign the task calls for a professional or a tool specifically built and certified to handle that category of data, not a general-purpose chatbot.

A Simple Gut Check Before You Hit Enter

You don’t need a formal policy to apply most of this — one question does most of the work: would you be comfortable if this exact message, with your name attached, appeared somewhere you didn’t control? If the honest answer is no, that’s the signal to rephrase the question in general terms, remove the identifying specifics, or use a version of the tool with clearer data-handling guarantees, such as an enterprise or business tier that explicitly excludes your inputs from model training.

If you want to go a layer deeper on how these systems actually process what you type, our piece on what generative AI actually is explains why the data you enter doesn’t just vanish after the response — it’s part of understanding the mechanism, not just the rule. And since caution about what you share pairs naturally with caution about what you receive, our guide to fact-checking AI answers covers the other half of using these tools responsibly.

Frequently Asked Question