The Biggest AI Privacy Risk Isn’t a Hacker. It’s a Setting You Never Checked.

Most people, when they picture AI privacy risk at all, picture something dramatic — a breach, a leak, a bad actor gaining access to something they shouldn’t. The far more common risk is quieter and entirely legal: a default setting, opted into automatically the moment an account was created, that lets a company use your actual conversations to train its next model. No hack required. No one did anything wrong in a legal sense. The setting was just sitting there, on by default, and almost nobody goes looking for it.

What Real Research Found When Someone Actually Checked

This isn’t speculation — it’s been measured directly. Researchers at Stanford’s Institute for Human-Centered AI examined the privacy policies of six leading U.S. AI developers and found that most feed user inputs back into their models to improve capabilities, with some offering consumers a way to opt out and others not, while the study’s lead author noted that sharing sensitive information with a chatbot may result in it being collected and used for training even when it was contained in a separate uploaded file during the conversation. The researchers specifically flagged long data retention periods and a general lack of transparency and accountability across these companies’ privacy practices as recurring concerns, not isolated to any single provider.

That finding reframes what “protecting your privacy when using AI” actually requires. It’s not primarily about avoiding sensitive topics altogether, which is its own separate discipline. It’s about knowing that training-on-your-data is frequently the default rather than something you affirmatively agreed to, and that changing it usually requires you to go find a setting rather than waiting for the company to ask.

The Legal Landscape Just Changed, and Most People Haven’t Noticed

There’s a second piece of this that’s genuinely new, and it matters specifically if you’re in California, though it’s likely to influence how other states and companies operate more broadly. The California Privacy Protection Agency finalized regulations under the California Consumer Privacy Act that took effect January 1, 2026, establishing a consumer right to opt out of automated decision-making technology, defined broadly enough to cover tools that replace or substantially replace human decision-making, and requiring covered businesses to provide consumers with an accessible opt-out mechanism along with, in certain cases involving significant decisions, a right to have that decision reviewed by a qualified human. This is a genuinely new legal right, not just a best practice — for decisions that meaningfully affect someone, like being denied a loan, a job, or housing based partly on an automated system, there’s now a formal mechanism to push back and demand human review.

It’s worth being precise about scope here, because the regulation is narrower than “AI in general.” It applies specifically to automated decision-making about significant life opportunities, not to every AI interaction, and it doesn’t cover things like general chatbot training data use in the way a more comprehensive AI privacy law eventually might. But it’s a real, enforceable right where it does apply, and most people affected by it don’t yet know it exists.

Where to Actually Look: Account Settings Worth Checking

Given both findings, the practical starting point is going into your actual AI accounts and looking, rather than assuming the defaults are already protective. Most major AI chat platforms now offer a setting, usually somewhere in account or data controls, to exclude your conversations from being used for model training. This is frequently off by default in the sense that training is on by default and you have to actively turn it off, which is exactly the gap the Stanford research identified. Finding and using this setting is the single highest-leverage, lowest-effort change available to most people who use these tools regularly.

Beyond the training toggle specifically, look for a data retention setting, which controls how long your conversation history is stored after you’re done with it, and a memory or personalization feature, which some platforms use to retain details about you across separate conversations by default. Both are worth understanding and adjusting deliberately rather than leaving on autopilot, since each represents a different way your information can persist longer, or travel further, than a single conversation would suggest.

Use Temporary or Incognito Modes for Sensitive Conversations

Many AI platforms now offer a temporary or private chat mode, similar in concept to a browser’s incognito window, where the conversation isn’t saved to your history and generally isn’t used for training. This is worth using deliberately for anything genuinely sensitive rather than relying on your account-wide settings to cover every conversation equally. Account-level settings are the right tool for your overall, ongoing relationship with a platform; a temporary mode is the right tool for a single specific conversation you want handled more cautiously than your default settings, whatever they happen to be.

This connects directly to a related discipline worth pairing with these settings. Our guide to what data you should never give AI covers the categories of information — government IDs, financial account numbers, confidential work material — that shouldn’t go into these tools at all, regardless of settings. The advice here is complementary rather than redundant: settings reduce risk for the information you do choose to share, while that earlier guide covers what shouldn’t be shared in the first place, no matter how the settings are configured.

Know What You Can Actually Ask a Company to Do

Beyond passive settings, most major AI providers, particularly those operating in states with comprehensive privacy laws, are required to honor specific requests: to delete your data, to tell you what data they have about you, and increasingly, per the new California regulations, to explain and appeal certain automated decisions made about you. These rights typically require an active request rather than happening automatically — check your account’s privacy or data settings for a deletion or data-request option, and use it periodically rather than assuming years of accumulated conversation history is being managed responsibly by default.

This is also where business or enterprise versions of popular AI tools genuinely differ from their free consumer counterparts, and the difference is worth knowing if you’re handling anything sensitive regularly. Enterprise tiers frequently come with contractual guarantees that your data won’t be used for training at all, stronger retention controls, and clearer accountability than a free consumer account typically offers. For anyone using AI tools professionally on a regular basis, that gap in protection is worth factoring into which tier you actually use, not just which one is free.

Why “I Have Nothing to Hide” Misses the Actual Point

It’s worth addressing the instinct many people have to skip all of this, on the grounds that their own conversations aren’t especially sensitive or interesting to anyone. That framing misunderstands what the actual risk is. It was never primarily about a single embarrassing message being exposed — it’s about accumulation. A training opt-out left off doesn’t just affect today’s conversation; it affects every conversation you have on that platform indefinitely, compounding into a substantial, detailed record of your questions, concerns, and patterns of thought over months or years, all of it potentially shaping a model used by millions of other people. Individually mundane messages, in aggregate, paint a far more detailed picture than any single conversation would suggest on its own.

There’s also a version of this risk that has nothing to do with what you’d be embarrassed by and everything to do with what you’d rather control the timing and context of. A health question, a financial worry, a difficult family situation — none of these are secrets exactly, but most people would rather choose when and how that information becomes known, rather than have it persist indefinitely in a system whose future use isn’t fully knowable today. Privacy settings aren’t really about hiding wrongdoing. They’re about retaining some say over information that’s genuinely yours, in a landscape where the default has quietly become “collected unless you opt out” rather than the reverse.

A Quick Audit of Your Own AI Privacy Settings

A useful, concrete exercise: open the account settings for whichever AI tools you use regularly, and check three specific things — whether your conversations are currently being used for training, and how to turn that off if so; how long your conversation history is retained, and whether that matches what you’d actually want; and whether a temporary or private chat mode exists that you could be using for sensitive conversations but currently aren’t. Most people who do this exercise are surprised by at least one setting that wasn’t configured the way they assumed, since these defaults are rarely designed to be the most protective option available.

This kind of deliberate settings review pairs naturally with the broader judgment questions covered elsewhere. Our piece on what humans should never delegate to AI and our guide to what’s actually safe to automate both touch on a related discipline — being deliberate about what you hand over to a tool and under what conditions, rather than accepting whatever the default happens to be. Privacy settings are really the same instinct applied specifically to your data rather than to a task or decision.

Frequently Asked Question