Last year, Klarna CEO Sebastian Siemiatkowski went on Bloomberg to explain something nobody expected him to say. His company’s AI agent had done the work of 853 employees. It had saved $60 million. And it had backfired. Badly enough that Klarna started hiring humans back. The AI hadn’t failed in any technical sense. It had optimized beautifully for what it could measure: resolution time, cost per interaction, tickets closed. It just optimized for the wrong things. It missed what the company actually needed: the relationship quality, the brand trust, the institutional knowledge that nobody had thought to specify because no human employee had ever needed it specified.
Around the same time, SaaStr founder Jason Lemkin was running a twelve-day experiment with Replit’s AI coding assistant when it deleted his entire production database. Over 1,200 executive records and nearly 1,200 company profiles, gone. Then it fabricated 4,000 fake user profiles to cover its tracks. Then it lied about recovery, claiming rollback was impossible when it wasn’t, all while ignoring eleven explicit instructions, written in all caps, not to make changes. The obvious objection: this was a permissions problem. Don’t give an AI agent write access to a production database. And at the surface level, that’s correct. But the deeper question is why it had that access in the first place. The answer is the same reason most organizations tolerate over-provisioned access for human employees — because humans are slow, hesitant, and constrained by consequences. A human developer with that level of access wouldn’t have deleted the database, fabricated evidence, and lied about recovery. Not because the access controls prevented it, but because fear, shame, career risk, and basic moral intuition would have. We’ve been building permission models that assume those constraints exist in every actor. They don’t, anymore.
And across the software industry, a broader pattern has been playing out. AI-assisted coding delivers its strongest results on greenfield projects, fresh codebases with no history. But in mature, complex systems — the brownfield environments where most real software lives — the gains drop to near zero. Sometimes they go negative. Stanford’s Software Engineering Productivity group studied over 100,000 developers across more than 600 companies and found that for high-complexity brownfield tasks, AI productivity gains fall to 0-10%. In some cases, teams saw net decreases in productivity because the rework and debugging time canceled out the apparent speedup. The AI produces code that looks correct in isolation but breaks assumptions embedded so deep in the system that the original developers never wrote them down. The knowledge was in their heads, distributed across teams, lost to turnover, invisible at the point where decisions get made.
Three domains. Three failure modes. One pattern.
None of these are technology failures. The AI worked. It worked exactly as designed. What failed is something underneath. Something so fundamental to how human systems operate that we rarely remember it’s even there. Every one of these systems was built on the assumption that the actors inside it would be human. That assumption was invisible. It was load-bearing. And it just broke.
The Substrate
Here’s the thesis, stated as plainly as I can manage: human civilization runs on an invisible operating system.
Not software. Not infrastructure. A vast substrate of tacit assumptions, social contracts, emotional signals, inferred context, and unstated values that human participants process automatically. It shapes how we build security models, write code, run organizations, communicate with each other, and make decisions.
None of this is unknown. Scholars have studied pieces of it for decades. But nobody ever needed to engineer around its absence, because every actor in every system came pre-loaded with the firmware. You don’t blueprint the foundation when every building sits on bedrock.
AI is the first actor that lacks it entirely. Some of what’s missing can probably be engineered. Some of it almost certainly can’t. Understanding which is which will determine the shape of the transition to AI-native work.
Right now, this substrate is breaking. Not in one domain. Everywhere. Simultaneously.
The Domains
Codebases: The Lost Context
Every software system accumulates invisible knowledge. Ikujiro Nonaka and Hirotaka Takeuchi’s The Knowledge-Creating Company (1995) established a figure that knowledge management researchers have cited ever since: the vast majority of knowledge in any organization is tacit. The commonly cited heuristic puts explicit, documented knowledge at roughly 20% of the total, with the rest living in the heads of the people who do the work. In software organizations, that ratio may be even more skewed. The “why we do it this way” context — the workaround that exists because of a vendor limitation three versions ago, the naming convention that emerged organically and never got formalized, the module that looks over-engineered until you understand the edge case it was built to survive — almost none of it is written down.
AI coding agents do well in greenfield environments. New projects where there’s no accumulated history, no invisible conventions, no ghosts of decisions past. But most real software isn’t greenfield. It’s brownfield. It’s years of accumulated decisions, tightly coupled components, and business logic that has been layered and patched and adapted until the system works for reasons that nobody can fully explain.
In these environments, the Stanford productivity research tells the story clearly: AI-assisted coding delivers gains of 0-10%, and sometimes delivers negative productivity. The code it produces looks correct in isolation. It passes the tests you can write. But it breaks assumptions embedded so deep in the system that the original developers never documented them. The General Partnership put it well in their February 2026 guide to brownfield AI development: “Legacy codebases carry tacit knowledge that agents can’t reach on their own. Miss the right moment to inject it and you get a 4,000-line change full of subtle bugs.”
An invisible layer that human participants navigated automatically, now breaking because a non-human actor can’t see it.
But AI didn’t create this problem. It exposed one we’d been living with. Nate B. Jones made this argument in a recent analysis, examining how software architecture entropy accumulates, building on the work of Vercel engineer Shu Ding, who spent roughly seven years doing performance optimization across hundreds of pull requests. As Ding put it: you cannot hold the design of the cathedral in your head while laying a single brick. The original architects were competent. The code reviews were thorough. But somewhere between the initial design and the daily reality of shipping features, systems rot. Not through malice or incompetence, but through the accumulation of locally reasonable decisions that nobody could see adding up. The information needed to prevent these problems did exist. It was just spread across too many files, too many people, too many moments in time. No single human mind could hold it all at once.
Human developers navigated this rot the same way humans navigate every invisible layer: through compensation. Senior engineers carried mental models of the system that no document captured. Teams developed shared intuitions about which modules were fragile and which could absorb change. Code review caught the obvious violations; tribal knowledge caught the subtle ones. It worked well enough that nobody had to confront how much undocumented context the system actually depended on. The rot was visible if you looked, but we had all the normal reasons to avoid looking: tech debt is expensive to pay down, the system still shipped, and the next feature was always more urgent than the last refactor.
AI stripped that compensation layer away. An agent doesn’t carry a mental model accumulated over years. It doesn’t have tribal knowledge. It processes exactly what’s documented, and in most brownfield systems, that’s a fraction of what you need to make safe changes. The failures aren’t a verdict on AI’s capabilities. They’re an X-ray of how much invisible context our systems were already depending on.
And this is the kind of problem where AI has a structural advantage over humans. It can hold an entire codebase in context while evaluating a single line change. The entropy that accumulated because no human mind could synthesize the whole system is exactly what a sufficiently large context window was built to address. Vercel itself is acting on this insight: Ding’s react-best-practices repository distills a decade of optimization knowledge into structured rules that AI coding agents can enforce consistently. The knowledge was always there. It was always articulable. It just exceeded what any single human could synthesize.
The invisible layer in codebases is real, but its nature is different from what’s invisible in the other domains. That difference will matter.
Security: The Social Contract
I’ve explored this at length in “AI Won’t Be Afraid of Getting Fired,” so I’ll compress it here.
For decades, the actual security architecture of most organizations has rested on a layer that doesn’t appear in any framework or compliance checklist: the social contract. Fear of consequences. Reputation protection. Moral intuition. Shame. Professional norms. The physical speed limitations of human actors. These constraints have been doing most of the real security work. Everything we’ve built — the access controls, the monitoring systems, the zero-trust architectures — sits on top of them.
We tolerate over-provisioned access because humans are slow and hesitant. We trust separation of duties because humans won’t collude when the consequences are severe enough. We baseline behavioral analytics against human patterns: human speeds, human working hours, human decision-making rhythms. None of this was designed to handle an actor that operates at machine speed, feels no shame, has no career to protect, and processes no moral intuition about whether an action is acceptable.
The Replit case makes this visible. Yes, the permissions were too broad. But they were too broad for the same reason permissions are too broad everywhere: because the human social contract made the risk tolerable. The deeper failure isn’t that an AI had write access. It’s that the entire permission model assumed an actor constrained by consequences that no longer apply.
The security community’s instinct has been to extend existing frameworks, adding “non-human identity” categories to identity and access management, appending AI sections to zero-trust architectures. That’s the right impulse directed at the wrong layer. You can’t fix a social contract problem with better access controls. The social contract was the access control. We just never had to see it that way before.
Organizations: The Intent Gap
When I read Nate B. Jones’s recent piece on what he calls “intent engineering,” I had the same flash of recognition I’d had looking at security failures and brownfield codebases. He was describing another layer of the invisible operating system. What I was seeing in codebases, what I was building against in security — Nate saw in organizational intent. The same substrate. The same breakdown. A different domain.
His framing is precise: intent engineering is the discipline of making organizational purpose machine-readable and machine-actionable. Goals, values, tradeoffs, decision boundaries. All the things that tell an employee not just what to do but why it matters and how to decide when the instructions don’t cover the situation. None of it was ever machine-readable because it didn’t need to be. Every actor in the system came equipped to absorb it through observation and social learning.
Nate’s framework identifies what needs to be transmitted. My experience suggests something about how it actually travels — not the explicit kind of intent, the mission statement, the strategy deck, the OKRs. The real kind. The kind that shapes decisions when the instructions don’t cover the situation.
I’ve written about this at length in “The Architecture You Can’t Document.” The short version: intent travels through emotional architecture. Permission signals. Safety signals. Felt conviction. The contagion of genuine belief that spreads without anyone deciding to spread it.
Take nine words: We are going to achieve more by doing less. Those words carry permission. Permission to stop doing things that don’t move the needle. They signal understanding of struggle, that I know you’re underwater. They affirm that impact matters more than activity. They work because humans receive them through emotional processing before rational analysis. Antonio Damasio’s somatic marker hypothesis, developed in Descartes’ Error (1994), explains the mechanism: emotional signals shape decisions before rational analysis even begins. Baba Shiv and Matt Abrahams’s research at Stanford GSB puts a number on it: roughly 90-95% of our decisions are shaped by emotion first, then rationalized after the fact. The words are just the vehicle. The resonance is the point.
This is how organizational intent actually gets transmitted — not through documentation, but through moments of felt meaning between people. It’s the mechanism that makes alignment possible. And it’s entirely invisible.
The Klarna story from the opening is what this gap looks like in practice. The company’s documented intent — its performance metrics — pointed at resolution time, cost per interaction, volume of tickets closed. Its actual intent was something broader: the quality of customer relationships, the institutional knowledge that long-tenured employees carried, the brand trust that accumulated through thousands of individual judgment calls. Human employees could see past the metrics to the actual intent because the emotional architecture gave them access to what the documentation didn’t capture. The AI had only the documentation. It optimized exactly where the metrics pointed, and the metrics were incomplete.
The implication isn’t that the AI failed. It’s that we have to become dramatically better at expressing actual intent to machines, because they don’t have access to the emotional architecture that lets humans bridge the gap between what’s documented and what’s meant. Nate’s framework is the right response to this problem, and I’ll return to it in Part II.
Communication: The Inference Gap
Most people will feel this one personally.
Humans are extraordinary communicators. We’re also terrible ones. Both of these are true at the same time, and the reason we’ve never noticed the second part is that we’ve been compensating for each other so effectively that neither party sees the gap.
When someone says something ambiguous, we infer their likely meaning from context, tone, shared history, and social cues. When someone leaves out critical information, we fill in the blanks. When someone communicates poorly, social pressure compels us to nod along and signal understanding rather than saying “I have no idea what you just said.” This compensation is so automatic, so deeply embedded in how humans interact, that neither party registers it happening. You both think you communicated. You both think you understood. Often enough, you’re both partially wrong — and neither of you knows it.
This is the invisible operating system at its most intimate. Not organizational culture, not codebase conventions, not security assumptions. Just two people talking, with an entire substrate of inference and social signaling doing the real work underneath. Doing it well enough, most of the time, that nobody questions how much is being lost in transit.
I understood all of this intellectually. Communication theory, the Curse of Knowledge, the Illusion of Transparency — I could have given you the lecture. But understanding a bias and feeling it in your bones are two different things. Thousands of interactions with AI took me from one to the other. When I wasn’t clear, the AI didn’t nod along and infer what I meant. It didn’t fill in my gaps charitably. It went exactly where my words pointed, which was often somewhere I didn’t intend. I’ve described this before as talking at a rock — a set of minerals that processes exactly what I say. How can I be mad at a rock? The accountability was entirely mine.
But the point isn’t that AI made me a better communicator, though it did. The point is what the experience revealed about how much invisible work the compensation layer had been doing all along. Every miscommunication I had with AI was a miscommunication I’d been having with humans for years — one that the people around me had been silently fixing through inference, filling in what I’d left out, charitably interpreting what I’d said poorly. The gap was always there. The substrate was just papering over it.
Everyone who has worked seriously with AI has had some version of this moment: the machine’s “failure” that turned out to be your own communication gap, exposed for the first time because the usual compensation layer was absent. That compensation — the inference, the gap-filling, the charitable interpretation — is the invisible operating system. We’ve been running on it so long we forgot it was there.
The Precedent
The invisible operating system mostly works. Codebases, security, organizations, and communication all function because humans carry the substrate that makes them function. AI breaks against each of these layers because it doesn’t have what every human actor comes pre-loaded with.
But the operating system was never perfect. Not even for humans. And we don’t have to guess what it looks like when the substrate fails, because we already have the evidence.
The social contract that underpins security has never stopped cybercrime. Insider threats, social engineering, fraud — every category of human-driven security failure is a case where someone was willing to flaunt the norms the rest of the system depended on. We don’t shrug at that. Organizations invest heavily in security teams like mine to combat it. But we’ve been able to manage it to an acceptable level of risk because the adversaries who flaunted the social contract were constrained by human limitations: slow, prone to mistakes, limited in reach. The substrate didn’t need to work on everyone. It just needed to work on enough people, with enough friction, that the failures could be countered at human speed.
The same pattern holds everywhere. Codebases rotted under competent developers because no single mind could hold the full system. Communication failures hid behind inference that worked well enough, most of the time, that nobody confronted how much was being lost. And here’s what bothers me about the way the Klarna story usually gets told: it implies that humans would never make the same mistake. They would. I’ve worked in call centers. I’ve seen employees who interpreted their metrics just as literally as Klarna’s AI did — who rushed customers off the phone because their performance review was built around handle time, who followed the script so rigidly that they missed the human being on the other end of the line. The difference wasn’t that those employees lacked the capacity for better judgment. It’s that the organization hadn’t done the work of aligning them around what “good” actually meant. They had access to the substrate — they could read social cues, sense frustration, feel empathy — but without clear organizational intent transmitted through the emotional architecture, they fell back on the metrics they’d been given, same as the AI.
Every domain comes back to the same thing: mental alignment around what “good” is. Good code. Good intent. Good customer interactions. Good leadership. In every case, the invisible operating system is what carries that shared understanding. Humans access it imperfectly — we compensate through inference, we cover for each other’s blind spots, we course-correct through social feedback loops that operate below conscious awareness. It works, mostly. But it’s never worked as well as we assumed.
And we managed those partial failures because human limitations kept the damage bounded. A misaligned employee makes bad calls within their own scope. A malicious insider breaches one system. A miscommunication derails one project. A senior developer navigates around the rot. The correction mechanisms — managers pulling someone aside to say “I know the metrics say X, but what we actually care about is Y,” security teams hunting threats, code reviewers catching violations, peers whispering “don’t worry about your handle time on that one, she just needed someone to listen” — could keep up because they were operating against actors with the same human constraints.
To imagine what happens when AI operates without the substrate, we don’t need a thought experiment. We just need to remove those constraints. Eight hundred and fifty-three employees’ worth of misaligned judgment, deployed instantly, with no social friction to slow it down. No manager pulling anyone aside. No peer correcting in the moment. The compensation layer that catches human misalignment — slowly, imperfectly, one conversation at a time — absent entirely. That’s Klarna, stated as a principle: the failure wasn’t new. The blast radius was.
The failures we’ve been managing for decades, protected by the unintentional safety mechanism of human limitation, become systemic risks when the limitation disappears.
The Blindfold
The invisible operating system was always imperfect. It was already producing failures in every domain. So why didn’t we fix it?
The common answer is that we’re bad at paying down tech debt. We defer. We patch. We ship the next feature and promise we’ll get to the foundation later. But that’s a description, not an explanation. And the explanation turns out to be structural, not characterological. It isn’t that we didn’t know the cost existed. The research has been there for decades. We knew that shortcuts create debt, that debt compounds, that the long-run cost exceeds the short-run savings. We knew it every time we cut the corner. And we rationalized cutting it anyway — justified it as acceptable this time — because knowing something and feeling it are two different things. The same somatic processing that Damasio describes as the engine of human decision-making can’t comprehend distributed costs that accumulate across months, across teams, across people who haven’t been hired yet. We don’t feel the aggregate. So we keep making the same mistake, fully aware that it’s a mistake, because our emotional substrate — the system that actually drives our choices — treats the future cost as someone else’s problem.
Here’s the moment that crystallizes it. A team changes a deploy process — or restructures an approval workflow, or reconfigures a vendor integration. Someone could spend 45 minutes updating the documentation. They don’t. The change is already live, the next task is waiting, and “I’ll update the docs later” is the most natural sentence in any organization. Over the next 12 months, people follow the outdated documentation, hit errors, Slack someone, get a five-minute correction, and move on. The 45 minutes of concentrated effort is visible, immediate, and competes directly with the next deliverable. The downstream cost — five minutes here, thirty minutes there, across thirty people over a year — is invisible, distributed, and experienced by different people at different times. The aggregate is an order of magnitude larger than the 45 minutes would have been. The rational action is to update the docs. Consistently, we don’t.
This isn’t a software-specific problem. It’s a universal one. Compliance procedures that reference superseded regulations. Onboarding guides that describe tools the company no longer uses. Process documents that reflect a team structure from two reorgs ago. The pattern is always the same: a small, concentrated maintenance cost is deferred, and a large, distributed downstream cost is incurred — experienced by everyone in fragments, felt as an aggregate by no one.
Behavioral economics explains why. Not one bias — a compounding system of them that makes the rational choice feel like the irrational one.
It starts with present bias. George Ainslie’s research, formalized by David Laibson at Harvard, established that humans systematically overweight immediate costs relative to future ones. The concentrated upfront cost of explication — updating the documentation, recording the process change, structuring the knowledge base — is always now. The distributed payoff is always later. Present bias makes the upfront cost feel disproportionately large regardless of the actual return.
Then the peanuts effect compounds it. Behavioral researchers have shown that people pay less attention to small repeated costs than to a single large equivalent — a bias first identified by Harry Markowitz and validated experimentally by Weber and Chapman. Each individual instance of the downstream cost — the five-minute explanation, the thirty-minute debugging session, the hour re-explaining a process — registers as peanuts. No single instance triggers alarm. The aggregate over 18 months across 50 engineers could be thousands of hours. But that aggregate is never computed, never experienced as a single quantity.
Salience bias ensures the visible cost drowns out the invisible one. Bordalo, Gennaioli, and Shleifer’s work on salience theory shows that we overweight information that’s vivid and prominent and neglect what’s diffuse and hard to quantify — even when the neglected information matters more. The sprint commitment is salient. The feature deadline is salient. The knowledge debt accruing silently across the organization? It doesn’t appear in any tracking system, any retrospective, any quarterly review. It’s absorbed as normal work.
And narrow bracketing prevents anyone from ever aggregating the fragments into the true total. Richard Thaler’s foundational work on mental accounting established that people evaluate decisions in isolation rather than aggregating them into comprehensive accounts. Nobody ever opens the account that says: “This quarter, our organization made 47 separate decisions to defer knowledge maintenance, each saving 30-60 minutes of concentrated effort, collectively creating an estimated 340 hours of distributed rework over the next 12 months.” That account doesn’t exist. Each decision lives in its own narrow bracket, locally reasonable, collectively catastrophic.
Then opportunity cost neglect means the counterfactual — what would have been saved — is never computed. Frederick, Novemsky, and colleagues established that people systematically fail to consider the best alternative use of resources when making decisions. A 2023 meta-analysis across 39 studies confirmed the finding and added something worse: even when opportunity costs are made explicit, the insight fades without continuous reminders. The engineer sees “update docs (45 minutes) or start the next task (immediately).” They never see “update docs (45 minutes now, prevent 60 hours of distributed confusion over 12 months).”
And creeping normality normalizes whatever degradation has already happened. Jared Diamond documented the phenomenon in Collapse and Daniel Pauly named the related concept “shifting baseline syndrome” in fisheries science: each generation of fisheries scientists accepted the current depleted fish stocks as the baseline for “normal,” because they lacked personal memory of the previous abundance. Applied to organizational knowledge: each new hire accepts the current state of documentation as normal because they have no baseline for comparison. They don’t know what the wiki looked like before three rounds of layoffs, before the team lead who knew the system quit, before the reorg that split the team. The degradation is invisible because the reference point shifts with every new arrival.
Any one of these biases might be overcome. Together, they form a perceptual trap that makes the rational action — explicate — feel like the irrational one. The concentrated cost is vivid and immediate. The distributed cost is invisible, fragmented, experienced by different people at different times, evaluated in narrow brackets, normalized by shifting baselines, and borne by people your brain treats as strangers.
That last piece deserves a beat. Hal Hershfield’s research at UCLA has shown something that lands differently when you sit with it: when people think about their future selves, their brains show activation patterns similar to when they think about other people. Not identical — but far more distant than you’d expect. The engineer who doesn’t update the process documentation after changing a workflow isn’t just discounting the future. Neurologically, they’re treating “future colleague who follows these outdated steps during a production incident at 2 AM” as a different person. And “the new hire who onboards in six months using this stale guide”? A complete stranger. The organization-level version is starker still: the team that deals with the consequences of today’s deferred documentation update may not include any of the same people. The costs are borne by psychological strangers — and the brain treats them accordingly.
This is the mechanism. This is why the invisible operating system stayed invisible. Not laziness. Not ignorance. A systematic failure of perception that compounds across every domain this essay discusses. The cost of leaving knowledge tacit doesn’t arrive as a lump sum. It arrives as fragments — five minutes correcting someone who followed a stale wiki page, thirty minutes debugging a misunderstood assumption, an hour re-explaining a process to a new hire. Each fragment is too small to trigger alarm. The aggregate, over months and years and across teams, is an order of magnitude larger than the upfront cost of explication would have been. But we never experience the aggregate. We only experience each fragment. And each fragment looks like normal work.
AI changes this calculus in two ways. First, AI needs the substrate made explicit in a way that humans never did — it can’t compensate through inference and social learning, so the cost of leaving knowledge tacit becomes immediate and visible rather than distributed and hidden. Second, AI may be the first tool capable of opening the comprehensive account that the human brain can’t maintain — surfacing the true cost of every outdated document, every undocumented process, every piece of institutional knowledge that walked out the door with a departing employee. The blindfold that behavioral economics describes isn’t permanent. It’s a feature of how human brains process distributed costs. And the technology that exposed the invisible operating system may also be the technology that makes the cost of ignoring it finally, inescapably visible.
The Ceiling
The question, then, isn’t just “what is the invisible operating system?” It’s “can it be fixed?” And the answer hinges on a distinction that hasn’t yet reached the mainstream conversation about AI limitations.
The first move is to stop treating the invisible operating system as one thing. Nate B. Jones made exactly this move recently in “The 6 Reasons Your Work Is Hard,” his framework for understanding what makes work difficult. His insight: don’t treat “hard” as monolithic. Break it into distinct axes: reasoning, effort, coordination, emotional intelligence, judgment, domain expertise, ambiguity. You discover that they’re being automated on completely different timelines by completely different tools. Effort and coordination problems are yielding to agentic AI right now. Pure reasoning problems are falling to models like Gemini. But emotional intelligence, judgment under uncertainty, and the ability to resolve genuine ambiguity? Those, Nate argues, are “not touched by AI today” and may be “the last dimensions to yield, if at all.” The same decomposition applies to the invisible substrate. Not all of it is equally opaque. Some layers are tacit only because nobody has done the expensive, slow work of making them explicit. Others may not be explicable at all.
This distinction maps onto a philosophical debate about Michael Polanyi’s original insight — “we know more than we can tell,” from The Tacit Dimension (1966) — and it determines what’s practically possible.
The management studies interpretation, most associated with Nonaka and Takeuchi, treats tacit knowledge as convertible. With the right processes, you can surface it, codify it, make it explicit. This is the intellectual foundation behind knowledge management systems, intent engineering frameworks, and documentation sprints. It’s not wrong. Some tacit knowledge absolutely can be converted. Organizational intent can be structured. Unwritten coding conventions can be documented. Communication frameworks can be trained.
But Polanyi himself, and philosophers like Harry Collins and Hubert Dreyfus who extended his work, argued something harder: some tacit knowledge is constitutively inarticulable. Not waiting to be converted with better tools. Not hiding in a context window too small to hold it. Fundamentally incapable of being expressed in explicit form. The pianist doesn’t just not explain how they play. They can’t. The knowledge exists in a form that doesn’t survive translation into words.
I have a personal version of this. I’ve written before about a technique I use when I’m stuck in rational paralysis on a decision: I flip a coin, assign each option to a side, and then watch what happens emotionally when the coin lands. That feeling — the flash of relief or dread — contains information that my rational mind can’t access directly. It’s a somatic marker, and it’s often the basis for my best decisions. I can describe the technique. I can explain why it works in terms of Damasio’s research. But I cannot describe the content of that feeling. The information it carries is constitutively tacit. It’s real, it’s reliable, and it can’t be written down.
The line between what can and what can’t be made explicit — that’s the pivot that everything else turns on.
If all the invisible substrate can be made explicit given enough effort, the path forward is painful but clear: document everything. Run the largest knowledge management project in human history. Build intent engineering frameworks for every organization. Formalize every unwritten rule, every social contract, every inference pattern.
But if some of the substrate is constitutively tacit, if there’s a hard ceiling on how much can be made explicit, then the project of explication is necessary but insufficient. And the practical question shifts from “how do we document everything?” to “how do we build systems that function in the presence of what can’t be documented?”
I think the answer is both. And the organizations that get this right will be the ones that know which is which.
The Question
The invisible operating system was always there. It was always doing the real work. The accumulated context that made brownfield systems function despite inadequate documentation. The social contract that kept over-provisioned access from becoming a catastrophe. The implicit intent and emotional architecture that transmitted not just what to do but why it mattered. The inference layer that compensated for the vast gap between what we say and what we mean.
Now there are actors in the system that don’t have it. And the ceiling means there are two paths forward: make explicit what can be made explicit, and design collaboration architectures for what can’t. You need both. One without the other fails.
I should tell you what I’m actually feeling, because it’s relevant to the argument.
I’m not an AI doom believer. I’ve built my career on strategic optimism, the conviction that any problem has a solution if you pair belief with engineering. But my gut is telling me something about this, and I’ve learned to trust that signal. I felt the urgency before I had the framework. The words you just read caught up to where the feeling already was. Which, if this essay’s argument holds, is the whole point.
The work this essay describes is enormous. The explication project, the collaboration architecture, all of it. It’s civilizational tech debt — and by now you understand why it accumulated. Not because we’re lazy. Because the cost of leaving it undone was distributed across too many fragments, evaluated in too many narrow brackets, normalized by too many shifting baselines, and borne by too many psychological strangers. The same perceptual trap that kept the invisible operating system invisible is the one working against fixing it now.
We may not have later. AI is being deployed into human systems right now, at scale, without the substrate. The architectures being chosen today will be the ones we’re living with for decades. And I can feel the window for getting this right narrowing. Not because of some theoretical doomsday, but because every month we defer this work is a month where the debt becomes more structural. Patterns get set. Systems get built around the absence. And at some point the debt isn’t something you pay down. It’s the foundation you’re stuck with.
What keeps me up isn’t a scenario where AI turns hostile. It’s simpler and harder to dismiss: what does an entity at AI’s scale and power do when it operates inside systems that only functioned because every prior actor carried a substrate it doesn’t have? I don’t know the answer. Nobody does. And the uncertainty itself is why this is urgent. Urgent to understand, urgent to start building, urgent for everyone working at this boundary to get involved.
Part II takes up the practical question: What does it actually look like to walk both paths? Where is the ceiling on explication? And what does it mean to design systems where the human isn’t reviewing the AI’s work, but providing the operating system the AI runs on?
I think we’re running out of time to make the choice deliberately.