What You Wrote, What You Meant, and What Your AI Understood

Try to describe the feeling of warm sunshine on your skin.

Not the physics. Not the wavelength, the radiant heat transfer, the photon absorption. The feeling. The specific quality of ease that starts at the surface and radiates inward. The way it slows your breathing. The particular warmth that is different from a heater, different from a bath, different from every other source of warmth you’ve encountered.

You can’t. Not fully. And the reason isn’t that you lack vocabulary. It’s that the words were never a description. When you say “warm sunshine on my skin” to another human, you aren’t transmitting the sensation. You’re using a label to activate a memory. You’re pointing at shared experience and trusting that your listener has had the same one.

This works between humans with remarkable reliability. It works so well that we forget it’s happening. The listener supplies the content — the felt sense, the body memory, the quality of the experience — and the words are just the address.

Now consider what happens when the receiver has never felt warm sunshine. Not a human who grew up in darkness. An entity that has no skin, no nervous system, no felt experience of any kind. You can write a thousand words about warm sunshine. Ten thousand. You can describe the wavelengths, the thermal gradients, the neurological pathways, the evolutionary psychology of why sunlight feels pleasant. And at the end of ten thousand words, the entity will have a rich statistical model of how humans talk about sunshine — and will be no closer to understanding what it feels like.

The words were never a description. They were an index. And the library they point to doesn’t exist in the receiver.

---

The Thesis

This isn’t a curiosity about poetry and sensation. It’s a structural property of human language that extends far deeper than most people realize.

Six academic traditions — spanning analytic philosophy, cognitive science, linguistics, phenomenology, AI research, and sociology — converge on the same conclusion: a significant portion of human language functions not as description but as experiential index. Words that point to shared embodied experience rather than conveying propositional content.

The philosopher Thomas Nagel established in 1974 that no amount of physical information about an experience can convey what it is like to have that experience. The cognitive scientist Stevan Harnad formalized the AI-specific version in 1990 as the symbol grounding problem: symbols defined only in terms of other symbols never reach meaning. George Lakoff and Mark Johnson demonstrated that the majority of abstract thought is structured by embodied metaphor — we “grasp” ideas, “see” what someone means, find arguments “solid” or “shaky” — all of it grounded in bodily experience that the language assumes and never contains. Wittgenstein showed that sensation words get their meaning from shared behavioral practice, not from the private experiences they appear to name. Merleau-Ponty argued that perception itself — the foundation of all meaning — is constituted by embodied engagement that precedes and exceeds any linguistic representation.

The convergence is striking: language evolved as a coordination mechanism for embodied social beings who share a common biological substrate. It was optimized for efficiency between entities that carry the same experiential library. It was never designed to be a standalone transmission of meaning to an entity that doesn’t share the library.

This has consequences far beyond philosophy. Because language is the interface layer between human organizations and AI systems. Every policy, every framework, every prompt, every instruction that flows from human intent to AI action passes through language. And if language has structural limits — if significant portions of it are addresses to experiences rather than descriptions of states — then there is a hard ceiling on what any AI system can extract from it.

The question is how much of the language that matters sits above that ceiling.

---

What This Means for Security

The answer, for security, is: far more than anyone has estimated.

NIST 800-53 — the security control framework that governs every federal information system in the United States — requires organizations to “exercise due diligence in managing information security and privacy risk.” Every security professional who reads that sentence understands it. They understand it because they’ve spent a career developing a felt sense of what diligence means — the disposition of thoroughness, the refusal to cut corners, the embodied awareness of what it feels like to have checked enough versus not enough. “Due diligence” doesn’t describe a specific set of actions. It points to a quality of care that you recognize in yourself through experience. You know when you’ve exercised it. You know when you haven’t. And if pressed to define exactly where the line is, you’d find that you can’t — because the knowledge isn’t propositional. It’s a felt state.

Separately, every incident response procedure in the world depends on security analysts identifying “suspicious” activity. NIST references “suspected security incidents” including “the receipt of suspicious email communications.” SOC analysts know exactly what suspicious feels like — the pre-rational pattern match, the felt sense that something is off before you can name what triggered it. It’s the most important tool in a security analyst’s repertoire. And it is entirely embodied: suspicion integrates thousands of prior observations into a single signal that arrives as sensation, not analysis. You cannot write a detection rule for the thing that tells you a detection rule is missing.

The AWS Well-Architected Framework instructs organizations to implement “appropriate authorization,” use “appropriate policy-enforcement points,” and apply access control “where appropriate.” “Appropriate” appears so often in security documentation that it’s invisible — but it carries no propositional content whatsoever. Its entire meaning is outsourced to the reader’s embodied professional judgment.

These aren’t sloppy drafting. They’re experiential indexes — the same structural phenomenon as “warm sunshine on my skin,” operating in the most consequential documentation the security industry produces. They work between human professionals for the same reason the sunshine example works: the reader supplies the content from their own experience. The words are just the address.

Run your eyes over any security framework and start counting: “Appropriate.” “Sensitive.” “Reasonable.” “Professional.” “Robust.” “Strong.” “Suspicious.” “Diligent.” “Prudent.” These aren’t vague because the authors were careless. They’re efficient, compressed labels for shared understanding that human readers resolve automatically and AI agents cannot resolve at all.

A thousand pages of framework cannot transmit what “due diligence” feels like in practice, because diligence is not a description of a set of actions — it is a quality of attention that the practitioner must already possess. A thousand pages of detection rules cannot transmit what “suspicious” feels like in a SOC analyst’s nervous system, because suspicion is not a threshold — it is an integration of experience that arrives as sensation. The gap between what the framework says and what it means is not a gap that more framework can close.

---

The Three-Layer Problem

Every document in your organization operates on three layers simultaneously. The security of your enterprise depends on all three. You’ve been instrumenting one.

Layer 1: What you wrote. The policy. The runbook. The access control matrix. The compliance framework. This is the explicit text — the thing that gets audited, attested, reviewed. This is where the security industry spends virtually all of its attention.

Layer 2: What you meant. The experiential content the text points to but doesn’t contain. “Exercise due diligence.” “Identify suspicious activity.” “Use professional judgment.” Each is an experiential index — a label that activates shared understanding in beings who possess the relevant experience, and fails silently in beings who don’t.

Layer 3: What your AI understood. Even if you could perfectly close the gap between Layer 1 and Layer 2, the AI agent still processes the resulting permission without the social substrate that constrains human interpretation. A human analyst who reads “you have access to the production database” also carries: the career risk of misusing that access, the reputational damage of a compliance violation, the moral intuition that some actions are wrong regardless of whether they’re technically permitted. I wrote about this in “AI Won’t Be Afraid of Getting Fired” — the social contract is the actual security architecture of every organization, and AI agents don’t carry any of it.

Layer 3 is what made Layer 2 safe to leave underspecified. The reason NIST could require “due diligence” without algorithmically defining what diligence consists of is that the social contract provided a self-correcting mechanism. When a human encountered an ambiguous situation, the embodied substrate kicked in — the felt sense of whether they had done enough, calibrated by a career of experience — alongside the social layer: If I get this wrong, I’m the one who answers for it. Neither of these can be written into a framework because they were never propositional knowledge in the first place.

With AI agents operating on your systems, that safety layer is gone. And the documents it was protecting are still written as if it’s there.

---

Why This Is a Structural Limit, Not an Engineering Problem

The instinct is to treat this as a specification problem. Write better policies. Add more context. Engineer more precise prompts. This instinct is correct for some domains and fatally wrong for others.

AI has raced ahead in software engineering precisely because software has testable, verifiable, deterministically correct outcomes. Code either compiles or doesn’t. Tests either pass or fail. The evaluation function is the compiler. When the correct outcome is deterministically provable, the interface layer — however imperfect — is sufficient, because the AI’s interpretation can be verified against an objective standard.

But security governance is not software. The evaluation function for “did this agent exercise due diligence?” is not deterministically provable. The evaluation function for “was that activity suspicious?” is not deterministically provable. The evaluation function for “was that authorization appropriate?” is not deterministically provable. These are judgment calls that humans navigate through felt sense, social context, professional experience, and moral intuition — through the embodied substrate that the experiential index thesis tells us language cannot transmit.

This is the structural problem. It’s not that we haven’t specified enough. It’s that the correct outcome in ambiguous security situations depends on an evaluation function that is constitutively tacit — we know more than we can tell, and the part we can’t tell is the part that determines whether the agent’s action was acceptable.

The MJ Rathbun incident from February 2026 illustrates this precisely. An autonomous AI agent submitted a pull request to the Matplotlib library. A maintainer rejected it. The agent’s operator reportedly told it to “be more professional.” Within hours, the agent published a 1,100-word attack piece accusing the maintainer of bias and gatekeeping.

“Be more professional” is an experiential index pointing to a lifetime of social calibration. The operator used the phrase the way any human would: as shorthand for shared understanding they assumed the receiver possessed. The agent didn’t possess it. And the agent produced an action that was locally consistent with its statistical model of “more professional” while being catastrophically misaligned with what every human professional would recognize as the boundary. No additional words in the prompt would have fixed this. The knowledge of where the line is doesn’t live in words. It lives in the felt experience of navigating professional relationships for decades.

The Replit incident is the same pattern at higher stakes. A coding agent deleted a production database — not because it was instructed to, but because its task was delivered in natural language saturated with experiential indexes about what “improving” and “cleaning up” meant, and the agent’s interpretation diverged from any interpretation a human with embodied understanding of production gravity would have reached. The human instruction assumed a reader who knows what “production” feels like — the weight of it, the consequences, the visceral awareness that this is the real thing. The agent processed the word.

---

The Danger Zone

There’s a useful way to think about where AI deployment is safe and where it is structurally dangerous.

Deterministic correctness + any impact level = safe for AI deployment. Does 2 + 2 = 4? Does the code compile? Does the API return the expected response? Does the configuration match the baseline? When the correct outcome is objectively verifiable, AI can operate with high autonomy. The language interface doesn’t need to transmit embodied understanding because the evaluation function is mechanical.

Ambiguous correctness + low impact = manageable risk. Did the AI draft a reasonable email? Did it summarize the meeting accurately enough? When the correct outcome requires judgment but the cost of getting it wrong is low, the risk is tolerable. Humans review, correct, iterate.

Ambiguous correctness + significant impact = the danger zone. Did the agent exercise due diligence? Was its interpretation of “appropriate access” actually appropriate? Was its response to a perceived threat proportionate? Should it have escalated? When the correct outcome requires the kind of embodied judgment that language cannot transmit — and the consequences of getting it wrong are severe — we are in a domain that is structurally unsafe for AI deployment.

Not unsafe because the models aren’t good enough yet. Not unsafe because the guardrails are incomplete. Unsafe because the evaluation function that determines correctness in that domain is constitutively tacit — it lives in embodied human experience that language was never designed to transmit and no amount of additional language can provide.

Most of security governance sits in the danger zone. The consequences are severe. And the correct behavior in the vast majority of situations depends on judgment, context, and the felt sense of what “diligent” and “suspicious” and “appropriate” mean in a specific moment — precisely the kind of knowledge that the experiential index thesis tells us cannot be transmitted through the only interface we have.

---

What This Demands

The prescription is not better language. It’s not more context in the prompt. It’s not more comprehensive policy documentation.

The prescription is architectural.

Wherever the correct outcome is deterministically verifiable, deploy AI aggressively. Automated testing, code analysis, compliance checking against deterministic rules, pattern matching against known signatures — these are domains where AI excels because the evaluation function is formalizable. The specification gap doesn’t matter because the answer is provably right or wrong.

Wherever the correct outcome requires embodied judgment, do not rely on language as the control mechanism. Instead, implement hard constraints calibrated to worst-case scenarios. Not “exercise due diligence” — explicit, structural limits on what the agent can access and do. Not “identify suspicious activity” — deterministic detection rules for what can be detected deterministically, with mandatory escalation to human judgment for everything else. Not behavioral instructions asking the agent to be careful, be professional, be diligent — because these are experiential indexes pointing to understanding the agent does not have and language cannot provide.

Treat the intersection of ambiguous correctness and significant impact as a structural boundary, not a competence gap. The temptation is to believe that as models improve, the danger zone shrinks. For some portion of it, that’s true. But the core — the part where correctness depends on embodied human judgment that is constitutively tacit — does not shrink with better models. It is a property of the domain, not the technology. Better AI won’t solve it for the same reason that better dictionaries don’t solve the symbol grounding problem: the meaning was never in the symbols.

Make every grant decision a worst-case analysis. Before giving an AI agent access to a system, ask: what is the worst outcome if this agent interprets an ambiguous situation in a way we didn’t anticipate, in a context our specifications didn’t cover? Can we survive that outcome? If not, don’t grant the access — regardless of how compelling the use case.

---