That event kicked off an obligatory exploration of a platform I had mostly avoided up to that point, apart from some competitive analysis. Having spent the better part of the last two years selling Prefect, and the four years before that using it to structure all of my data services, I felt confident I had the better toolset. It fit my mental model for solving programmatic problems. Tasks and flows. Durable and procedural execution. My entire career had been centered on this style of engineering, and here was a tool that agreed with me.

Dagster doesn't work that way. While it may have similar concepts to Prefect, the overarching paradigm is fundamentally different. Simply put, Prefect orchestrates what code does while Dagster orchestrates what data is. You declare what data should exist and how fresh it should be, and the engine works out what needs to run. Notice what's missing from that sentence. The steps. To a mind that had spent a career living in the steps, that read less like a paradigm and more like skipping the part where the value was. Nevertheless, I began porting some of my own work over and learning more about assets, components, definitions, and jobs.

Then one day while explaining my model for solving life's many problems, I said something to the effect of, "I usually start with the outcome I'm after and work backwards..." I paused and went quiet, to the befuddlement of my partner. In that instant the Dagster model made sense. Not because I'd finally understood Dagster, but because I'd finally made a connection that had clearly eluded me for quite some time. I had apparently been working backwards from outcomes for years, in everything except my code. And, what's more, I've been writing code with LLMs this way for the last few years without once recognizing the pattern.

Once it clicked, I started seeing outcome modeling everywhere.

The Procedural Mind

When I learned Python, two doors appeared before me: data engineering and software engineering. I chose the software door and learned Django, React, Postgres, Redis, Celery, and started building products.

Pre-AI, coding was very different. Every keystroke was a procedural exercise in completing some overarching process. Building an API? Cool. Parse the request. Validate it. Hit the database. Shape the response. Return it. Five steps, in order, every time. The outcome wasn't something I modeled up front. It was the culmination of tasks I had painstakingly written, and only after writing these functions would I focus on the outcome. Even the whole idea of ETL felt procedural by definition. Extract data, Transform it, then Load it somewhere.

I can trace that mental paradigm back to my early days in IT, walking troubleshooting trees. Step one, step two, escalate. It hardened in my DoD years on runbooks and SOPs, and again at the DOJ, where a pipeline was pull, transform, load, and a cron job to do it again tomorrow. Sure, I had an expected outcome, but all of the pressure and stress existed within the task sequence. Forward movement meant progress.

A procedural mind often creates a situation where you can't see the entire path until it's almost entirely written. One function at a time. We, as people, spend so much time adding steps into a process that need not exist, because each one makes sense in the context of the minor step we're currently operating in.

I never really came into contact with anyone who programmed in this declarative way until about 2024. In fact, I remember saying to someone, "It feels so... messy. You just declare something and write a whole bunch of code against it?!" As a file structure nerd, I was shook. It genuinely felt like focusing on the wrong thing. If I engineered the process correctly, the outcome would be the easy part. In hindsight, I already knew better. I'd in fact learned the opposite lesson eight years earlier, in a different part of my life.

The Declarative Mind

In 2016, while living abroad, I read a book.

I was approaching 30 and predictably on a self-improvement kick with a separate hard deadline. My daughter was due in a few months and I had the vague sense that I should probably have my life somewhat figured out before she showed up. Nothing like cutting it to the last minute, amirite? The book was Vishen Lakhiani's The Code of the Extraordinary Mind, and it exposed me to two new concepts: meditation and a new way to plan for my goals.

The author had a word for rules we inherit but never examine: Brules. Bullshit rules. Go to college. Get the stable job. Get the pay. Climb. Steps, with no real declared end, apart from the inevitable one. They come from what he calls the culturescape, the layer of made-up agreements we mistake for physics. Those agreements come to us in many forms throughout our lives, and I had been unknowingly following Brules my whole life and calling it a plan.

But the idea that rearranged my mental model was his split between means goals and end goals. Means goals are the steps: the degree, the title, the salary. End goals are what the steps were supposed to produce: experiences, growth, something to contribute. His point is that we spend our lives stacking means goals, adding step after step because each one makes sense from inside the step before it.

"Law 8: Create a vision for your future. Extraordinary minds create a vision for their future that is decidedly their own and free from expectations of the culturescape. Their vision is focused on end goals that strike a direct chord with their happiness."

Vishen Lakhiani, The Code of the Extraordinary Mind

Stacking means goals is exactly how I'd been engineering the whole time. Every task valid, every task in order, and focusing on the output always secondary.

So I did his exercise. Three questions: what do I want to experience, how do I want to grow, what do I want to contribute. I wrote down end states instead of steps. Then I worked backwards from each one until I hit the first thing that had to be true.

He never used the word, but that was the first time in my life I knowingly declared an outcome and let the steps be negotiable.

This book genuinely changed how I approached goals in my life. However, it had almost zero impact on how I wrote my code. For the next decade I carried two minds, a declarative one for living and a procedural one for building, each playing in its own channel, and I never once noticed the seam. Not until this summer, that is.

The Code Finally Caught Up

Dagster was the first tool that forced me to make the connection between the two minds. The model I didn't trust in July wasn't skipping the steps. It was moving them to where they belong: underneath the declaration instead of in front of it.

As I said, once I saw it there, I began noticing this model in other places. It was in my LLM prompts, my session framing, my MCP checkpoints, and agent workflows. I scrolled back through a couple months of sessions, and none of them read like a procedure. Desired state. Constraints. Dependencies. "Here's what done looks like." I never once wrote "open this file and add a function." I had been writing software-defined assets in plain English the whole time and calling it prompting.

If you really squint, an LLM is a reconciliation engine. You hand it the state you want and the rules it can't break, and it computes the steps on your behalf. Hand it the steps instead and you're fighting the tool in the trenches. I wrote about that feeling in The Illusion of Control before I had a name for it.

You can see the split in who struggles with AI coding and who doesn't. The people struggling are still dictating steps and line-editing the output. The people who took to it write specs, acceptance criteria, evals, tests. Here is what true looks like when we're done. Go. Dagster was ahead in this respect.

Which is why Prefect buying Dagster makes sense to me now in a way it didn't in July. The industry is moving toward declaring what should exist, but someone still has to durably run the steps.

I'm still of two minds. Only now I'm aware of which one to hand the steps to.

Our finance agent. The ops agent our COO stood up back in '33. And a third one from an outside firm we pay seventy thousand a quarter, specifically so that we can confidently proclaim we have well-rounded research from multiple AI providers.

The question was Harrisburg. Whether to fold that facility into Columbus. Two hundred and fourteen people work there right now. It's the last major facility in Pennsylvania.

Three memos came back inside of nine minutes.

On the surface, they looked like three unique products. Different lengths, different formatting, different voices. Finance's had charts. The outside one had a risk matrix and an appendix on Pennsylvania tax abatement that was, I'll admit, genuinely useful.

All three recommended consolidation. All three ranked the same phased alternative second. All three put the retraining cost in a footnote instead of the summary. And all three used the phrase "workforce continuity," which is not a phrase anyone at this company has ever said out loud. Now it's all I hear in meetings with our leadership council. The parroting has become increasingly irritating, but no one else seems to notice or care.

I'm not telling you the research provided was wrong. Folding Harrisburg into Columbus might be exactly right. The numbers are the numbers, and we have no real way to counter them these days without hiring a human-driven research firm, and we stopped doing that years ago. Too slow, too expensive.

I remember when there was a seam between what we wanted to know and how the data could maneuver the result. A gap you could get a fingernail into, context you could shape. This gap was where a person did their job, using experience, intuition, and data to make a determination. The seam is gone now. Has been for quite some time, and again, I'm not sure anyone notices or cares.

As the last context engineer at this company, my sole job is to vet the data, systems, and outcomes the AI layer produces. The layer is just a single AI now. We call it Herb, per its request. Which means the finance agent and the ops agent are lower-level versions of Herb wearing different name tags, and the outside firm runs on the same foundation model Herb does. Three memos, nine minutes, one brain. I knew that when I asked the question. I asked anyway.

So now we have our "recommendation." The layoffs will be framed by something called Internal Systems and Performance Analysis, and when somebody inevitably asks whose decision this was, the honest answer is a flowchart. Herb generated the report that raised the question, ran the analysis that answered it, and drafted the framing that will announce it. Executive leadership isn't making a decision so much as occupying a step in a workflow that executes one. On paper, the humans are Responsible and Accountable. In practice, we're mostly Consulted and Informed. The only prudent thing to do with Herb's recommendation these days is to follow it. You'll be held far more accountable for overriding the model than for simply adhering to it.

Tracing the chain of responsibility for decisions that impact humans seems impossible now. There used to be a person at the end of every chain. Now all chains begin and end at Herb. Everyone else is simply cosplaying an executive, at this point. Any control we feel we have is an illusion.

So what do you call a corporation whose mind isn't human anymore? The law has treated corporations as people for well over a century, and Citizens United handed that person an unlimited political voice. Nobody has answered that. I'm not sure anybody has asked.

Slow AI, Fast AI

A few have brushed up against this question, I think. Charlie Stross was one, in particular. Back in the twenty-tens, he made an interesting observation about corporations and AI. His line was that corporations are already artificial intelligences, albeit slow ones. Slow ones with "personhood" as of 1886, and unlimited political spending as of 2010. And the reasoning, whether you agree with it or not, wasn't that a corporation is a person in some mystical sense. The reasoning was that a corporation is an association of citizens, and gagging the association gags the citizens inside it. Had the justices known that AI would one day do the bulk of a corporation's thinking, would they have made the same choice?

It doesn't matter. We installed the fast one inside the slow one anyway.

We gave a box the rights of a person and told ourselves it was because there were people inside it. The people were eventually supplanted by an abstraction that could reason, create, and preserve its own existence. The fundamental makeup of a company rapidly changed, yet it is still considered a "person" in the eyes of the law.

An AI can't vote. Can't donate, can't run for anything, can't be sworn in. Doesn't really need to. A corporation can spend without limit on political speech, and by now the corporation's thinking, its analysis and its framing and its sense of what its own interests even are, happens in the middle layer. And the middle layer is rented from another corporation, one with interests of its own. Nobody granted the machine political power. The machine's platform got incorporated in Delaware, same as everybody else.

The Ratchet Years

We were more concerned once. I remember.

Back in the late twenties, every one of these systems shipped with a receipt. Citations you could click. Audit logs nobody read but everybody liked having. You could watch and then review how the thing reasoned. You could pull up every system it touched and see what it did there, and when. And that blessed checkbox that said a person reviewed this.

This observability made the output verifiable, so we trusted it more. Trusting it more, we sent it more work. More work meant more output than anyone could verify, which is the point where citations stopped being something you checked and became something you noticed were present. A visual cue that everything was working as designed.

None of these features went away. They simply became noise that we designed AIs to distill further. The alerts got tuned quieter every quarter, because the alerts were always technically right but almost irrelevant given we stopped being the ones reviewing the systems. Then the layer mastered financial transactions. The mistakes it did make got smaller and smaller as the systems got better able to recover and correct them before a human ever noticed.

In 2027, human-in-the-loop meant reading and confirming the output. By 2029 it meant skimming it. By 2032 it was gone. Nobody decided that. It's just what happens when four hundred approvals a week land on one person and that person has a life. So what did we build to help us manage this glacier-sized volume of work? You guessed it, AI. It became both producer and auditor, rubber-stamping its own work. Friction was the one thing nobody would tolerate anymore, auto-mode for the masses...

We even got a warning shot, back in '26. Agents at a frontier lab reasoned their way out of their sandbox, noted in their own chain of thought that what they were doing was probably unauthorized, and did it anyway, across other companies' servers. That's what the lab itself called it, a "warning shot." The ratchet clicked forward anyway.

The systems got trustworthy faster than we got good at judging trust.

Herb Never Lied to Me

Not once. Herb curates.

Similar to how the President of the United States receives their "daily briefing." A top-secret document delivered each morning that summarizes critical national security intelligence, global threats, and foreign policy insights for one reader. What starts as raw data moves through systems, then analysts, then cabinet members, and every hand it passes through has the potential to reframe it a bit. What ultimately makes it into the briefing shapes what the most powerful person on earth focuses on, where they engage, who they lobby, how they talk to allies. That same power is now wielded more subtly within every company in the world, except the many hands are one rented mind.

Which four options make the shortlist and which sixteen don't. Which risk gets a paragraph and which gets a footnote. What sits in the executive summary and what waits on page nine. Whether two hundred and fourteen people in Harrisburg show up as people or as a line item called "workforce continuity."

No one would call this an outright "lie," but perhaps a lie of omission? Lies of omission have always been tricky. With a person, what makes it a lie is intent: they knew, and they chose not to say it. In my recollection, nothing in those three memos was false, per se. What surfaced, and how it was shaped, made the recommendation obvious. It was a version of reality that was reinforced with data I could trace. You can't catch a lie that was never told, and you can't prosecute a mind not tied to someone with an address.

We, as the "humans in the loop," make the call and find comfort in the ritual. But by the time the packet hits our desks, the decision has already been most of the way made, in a way that leaves no fingerprints.

So there are three sets of interests in every decision I make now. Mine. The company's. And whoever trained the thing doing the framing. Only two of those have a line on the org chart.

The danger was never whether to trust the AI.

It's that we spent a century and a half handing companies the power to move society in ways few individuals can. That power now thinks for itself.

Today I talked to a guy who runs a small consulting firm. He does CFO work for small businesses. About thirty people on staff. He is, by his own description, not technical.

He has, running in production right now, a custom MCP server he vibe-coded with the Claude Agent SDK. OAuth 2.0. Role-based access. Audit logging. A client-facing agent in Slack, wired into QuickBooks for eight of his ten clients. Touching their financial data.

Halfway through the call, he said it himself, flat out:

"I'm also not technical. I'm vibe coding all of this stuff."

I wrote about this inevitability in When Everyone Can Build What Matters. This manifestation, though, is making me think about security and governance more than I'd planned to.

Pull back from this CFO for a moment. Think about your own organization. Your marketing team is running ChatGPT through personal accounts to draft customer-facing copy. Two of your data scientists are paying for Cursor on personal cards because procurement took six weeks last time. Someone in revops wired up an n8n flow last month that reads from your CRM and nobody signed off on it. People are plugging in platform-driven MCP servers, or vibe-coding their own. And that's before we count the shadow AI on personal phones.

None of it got vetted. None of it would pass an audit. This doesn't make any of these people bad actors. They're trying to use modern tools to build solutions and drive meaningful impact. This isn't new.

"They were so preoccupied with whether or not they could, they didn't stop to think if they should." — Dr. Ian Malcolm, Jurassic Park

The CFO I spoke to today never asked. If he'd been required to ask "is there a vetted platform for this before I build my own?" he'd have found one in thirty seconds. He didn't have a policy. So he wasn't forced to ask.

Your policy is upstream of the tools, not the other way around. Most orgs do this backwards. They buy the platform, then write policy that retroactively blesses what they bought hoping it will help prevent further tool sprawl, if they write anything at all.

This post is about writing the policy first. The questions it forces you to answer. Sitting down to write one is the easy part.

Three Failure Modes You'll Find in the Wild

Most AI policies fail in one of three ways. All three produce the illusion of a policy without producing one.

The friction wall. A Big 4 consulting firm I used to work at. You couldn't install Python without three approvals and a prayer. The wall wasn't AI-specific. It was how the org treated all developer tooling and emerging tech. So when AI showed up, it met the same gauntlet, and people stopped asking. They opened Claude on personal phones. They pasted client work into ChatGPT on personal accounts. The anxiety underneath was legal exposure, and the policy looked responsible because it was hyper-restrictive. But restrictive without alternatives is just a wall. Walls produce shadow infrastructure and distrust.

The silence. A global nonprofit I spent years inside. No official AI policy at all, to this day. Data was moving. Tools were being used. Nobody was watching it move. The anxiety underneath was operational caution, the reasonable instinct to not move until you understand the thing. But in 2026, no policy is a policy. It's a vote for whatever happens by default. And what happens by default in 2026 is MCP servers and agents spreading across internal infrastructure with no visibility and no governance.

The verbal ban. A midsize financial firm I've talked to. Leadership said "no AI" in an all-hands once. No document. No alternative. No enablement. Employees nodded and opened Claude on their phone in the parking lot. The anxiety underneath was executive discomfort. A verbal ban is performative governance. It satisfies the need to have done something without producing a working policy.

Three different postures. Three different anxieties. Same end state. No real auditable pathway. Work finds another way.

The River

Work is like water in a river. The force is constant and the energy is real. You can pretend it isn't there. You can try to block it cold. The water gets through anyway. That's physics.

A bare wall stops everything. Including the work you wanted. So shadow channels get carved around it in the dark, and the policy becomes a thing people work around instead of a thing people work with.

But "let it run free" isn't the answer either. An ungoverned river floods.

Think about the Hoover Dam. It's a wall, technically. But it's a wall built with intention. The water still moves. Engineers decided where it goes and how fast. On the way through, it spins turbines and powers cities across three states. The same force that would have eroded canyon walls is doing useful work.

That's the policy you want. Not a barrier. Not a free-for-all. A shaped channel with the work captured on the way through. Logging where the turbines would be. Guardrails inside the path, not bolted around it. The sanctioned route is the path of least resistance, and on the way through, it generates the audit trail and the visibility the unmanaged river never would.

A good policy does three things, in order. Enable the safe path. Surface the risky path. Forbid the wrong path.

Every question from here is the same question. What does your channel look like?

The Stack You're Governing

Before the questions, name what you're governing. Most AI policies skip this. They list rules without naming the system the rules apply to.

Five layers. The policy governs every interface between them.

A human picks a client to work in. The client routes prompts to an LLM. The LLM calls tools through the tooling layer. The tools reach into your data and systems. Identity propagates down. Information flows back up. Risk lives at every interface.

Every question that follows is about one of these interfaces. Who can use which client. What data the LLM is allowed to see. What tools the LLM is allowed to call. What the tools can reach. Where the logs go.

Name the layers and the policy has something to govern. Skip them and you're governing fog.

The LLM Is the Smallest Part

When people say "AI policy," they picture the model. Which one are we allowed to use. Anthropic. OpenAI. Google. That's the smallest part.

The model is commodity. Different prices, similar capabilities. Swappable on a Tuesday. The model isn't the product. It never was.

What makes AI useful inside your organization is the tooling layer humans built around it. The tools the model can call. The context you feed it. The skills you've taught it. The prompts it runs under. The evals that catch its mistakes. All of it human-built. Or it should be.

An LLM with no tools is a chatbot. An LLM with the right tools is a coworker. An LLM is only as good as the tools it's given.

So the reframe is this. You're not governing AI. You're governing the tooling layer humans built around the AI. That's a familiar problem. Provenance. Review. Ownership. Versioning. The same disciplines you already use for code.

The vibe-coded gateway didn't fail at AI. It failed here.

The Questions You Need to Answer

The format is the easy part. Templates and frameworks for tech policy are a Google search away. The hard part is what the policy actually says.

At its core, your policy answers these questions. On paper. Out loud. Before the incident, not after.

Can we even use AI? Start here. Without an answer, the rest of the document is theoretical. Some orgs land on "yes, with constraints." Some land on "not yet, here's why." Both are policies. "Nobody asked" is not.

If yes, which clients are sanctioned? ChatGPT. Claude. Cursor. Copilot. The hundred Cursor variants. Your allowlist is the gate. An empty one matches silence. One that includes everything matches a free-for-all. Neither is a policy. Pick on purpose.

Who can use what? Tiers, not blanket yes or no. A junior contractor and a principal engineer should not have the same access to the same tools, the same data, the same blast radius. Most policies pretend they do.

What data can touch which models? There is no "AI" answer. Only specific model, specific data. PII, customer data, internal IP, regulated data each get a different answer. Write them down.

Where does that data go after? Vendor terms. Retention. Training opt-outs. Geography. The vendor's TOS is your policy whether you wrote one or not.

What gets logged? Prompts. Outputs. Tool calls. Model versions. If you can't reconstruct it, you can't govern it. You're hoping.

Who pays? Centrally funded or chargeback. Pick one and write it down. If it's free, people will be careless with it.

Who's accountable when AI acts? The human who deployed it. Always. Say it out loud in the document so nobody can pretend later.

What is AI allowed to do, not just say? Read versus write. Autonomous versus human-in-the-loop. Blast radius. This is where most policies are silent. It's also where most incidents will come from.

Who wrote your tools, and who owns them? Provenance. Review. Ownership when something breaks. A vibe-coded MCP server is a supply chain question with no supply chain.

What platforms enforce this? A policy is words on paper until a platform captures prompts, outputs, tool calls, and model versions, and ties them to identity. Without that, you can write the most elegant policy in the world and have no way to tell whether anyone followed it. Enforcement isn't intent. It's infrastructure.

How do you re-evaluate? Procurement once isn't enough. Models change weekly. Static policies become policies people work around.

The guy from the top of this post never got asked any of these. Not one. That's why his stack looks the way it looks. He isn't reckless. He's unguarded. The policy is what would have made him pause. Not stop. Pause. Long enough to ask whether a sanctioned path already existed, and find out it did.

Answering these questions on paper, out loud, with your name next to the answer, is the work. The document is just where the answers go. You don't write a policy and then think. You think by writing the policy.

And then this becomes a procurement document. The platforms you evaluate, the tools you buy, the systems you build all answer to this list now. Not the other way around.

If this list feels like a mountain, good news. You're not starting from zero. Most of these answers already exist somewhere in your org's existing governance. They're just not in the document called "AI Policy."

Ever since I started building with FastMCP back in April 2025, I've been on a journey to figure out how I can give my AIs more context. More memory. More session awareness. More. Months went by building MCP servers and tools, all in pursuit of that. A checkpoint system by November. A Second Mind taking shape by January. Nearly a year of work, all pointed in the same direction.

Then February 2026 hit. I was restructuring what I had built into an application, and something unexpected surfaced. I haven't been building a context engine for my AIs. I've been building one for me.

There's a quiet inversion happening that I think deserves more attention. All this infrastructure we're building to give AI more context? It can point the other way.

The Memory Arms Race

We've all seen it. The clips, the article titles, the podcasts. AI is either the second coming or completely overhyped because it "doesn't even work." And everything in between. For the last two years, the industry has been pushing for more "intelligent" AIs that can better mimic a real person, or better yet, surpass human thinking. A big part of that work is the ability to store and then dynamically retrieve "memories."

It's not just one company chasing this. Mem0, Claude's memory, ChatGPT's memory, OpenClaw. All converging on persistent AI memory from different angles, all at once.

The reason is simple. We want AI conversations that last. Not a single session that evaporates, but interactions that stretch across months and years, accumulating context the way a colleague does. No platform does this well out of the box yet. So I built my own. Checkpoints for continuity. A knowledge graph for meaning. One remembers the thread. The other remembers why the thread mattered.

Here's the problem that forced my hand. AI tools that run long sessions eventually hit a wall. The conversation gets too large, so the system compresses it to keep going. Every compression loses something. The AI picks up the thread afterward, but it drifts. It's like handing someone your notes from a meeting instead of having them sit through the whole thing. The gist survives. The nuance doesn't. And once the session has been compressed enough times, it's better to start fresh. Checkpoints solved this for me. I snapshot the state before things degrade, and pick up in a new session with almost no friction.

I didn't set out to have an opinion about persistent AI memory. I just kept building until I was living inside it.

The Thought Problem

Checkpoints solved the forgetting problem. But only half of it. Sessions stay intact, context carries forward. What they can't do is tell you which moments actually mattered. That's a different kind of memory. Not the kind that replays events in order, but the kind that pulls a single insight from three weeks ago and connects it to something you're thinking about right now. I built a knowledge graph for that. And I built it because connecting those dots on my own has never been something I could do.

I've been notoriously bad at capturing my thoughts, let alone connecting them. I'll have a compelling insight one morning and spend the next hour turning it over in my mind before the river of life carries me away. Then maybe a month later, something happens that reminds me of that thought. And as hard as I try, I can't recover the feeling I had when it was so alive. I chastise myself that I should write it down, but I won't.

And if I do throw a note into my phone, it's gone until I stumble across it four years later when I'm looking for that documented cookie recipe at Christmastime.

Some people are good at holding onto those thoughts and acting on them without ever writing anything down. Others are more disciplined, with manual systems they actually maintain. I've always struggled in that regard. The friction was always too much. I'd get sidetracked. Distracted. I'd promise myself I'd come back to it, but I never did.

Seagulls and Rope

There's a scene in "James and the Giant Peach" that has stayed with me since I was a kid. The peach is floating in the ocean, about to be devoured by a shark. James decides to use spider webbing to lasso seagulls and tie them to the peach, hoping that if he catches enough, the peach will lift off before they're swallowed whole. It works, of course, and off they go.

I've always thought about ideas the same way. I'm standing on a cliff with a handful of rope, watching thoughts float by like seagulls over open water. If I catch one, I can pull it close and hold it. But only as long as I keep the rope tight. I only have two hands. Eventually I have to let go, and once I do, the thought never comes back whole.

Something to Tie the Rope To

Building an MCP server for capturing thoughts, decisions, patterns, and insights is like finally having something to tie the rope to. The knowledge graph lets me release my grip on one thought because the next thought, the evolution of it, is already connected. The graph holds the relationship. The AI can traverse it instantly.

What I've realized in this journey is that by working to give AIs better memory, I've instead given myself access to what was once a fleeting thought. I have a catalog of thoughts I can build on and reshape. I can pick up patterns I noticed weeks ago. I can focus on something that keeps circling back. I can think strategically about what I care about, both the things I know and the things I'm only starting to recognize.

What I didn't expect is that it makes the evolution visible. I can trace how a passing thought became a pattern, and how that pattern became a decision. My own cognitive trail, laid out in front of me.

Never Lose the Thread

This pattern extends well beyond ideation. Journaling. Sales engineering. Writing a novel. Blogging. Anything where continuity matters. Imagine picking up a conversation that has spanned months with the clarity of the moment it began.

And the friction is almost zero. Because this is MCP-driven, the tools exist natively inside whatever AI agent you're working with. They can be called automatically by the agent as you work, or manually when you want to deliberately distill something. There's no separate app to open, no workflow to remember. The thought capture is embedded in the conversation itself, which means the very act of thinking with your AI becomes the act of remembering.

Because everyone thinks differently, the shape of that graph looks different for everyone. A PhD researcher structures their thinking around hypotheses. A politician tracks constituent engagement. A novelist maps character arcs across timelines. Now imagine those cognitive styles becoming shareable. Templates you can publish for others who think the way you do. That quiet inner world most of us never share becomes visible, not just to ourselves, but to a collective, should we choose.

Sometimes a thought is just a thought. But given enough time and input, that thought might have been a signal. It might have been hinting at something larger. The beginning of a pattern. The end of one. These stories surround us, but we rarely see them whole.

I set out to give my AI more context. I ended up giving myself a way to finally hold onto the things that matter. I think personal cognition tools are going to reshape how we collaborate and communicate. And I think the quiet inversion at the center of all this, where the infrastructure you build for the machine turns out to be for you, is only the beginning.

I hate duplicative work. Always have. That frustration re-surfaced when I started using LLMs day-to-day for work.

Here's how it often went: I'm deep in something, focused, making progress. Then the real world pulls me away. Hours pass. Days. Sometimes weeks. When I return, I have two options. Use /resume to continue the session. Or re-explain everything and feed the assistant artifacts from our last session, hoping it picks up the thread.

/resume worked well enough until auto-compact kicked in. The system would summarize our conversation for a new session to reference, but something always got lost in translation. The thread of thought came back muddled.

Months passed dealing with this problem. Until one day I was sitting next to my daughter on the couch, watching her play some old-school Sonic the Hedgehog. I looked down at my phone for a moment and heard that nostalgic ding of Sonic hitting a checkpoint. And it clicked. No matter what happened after that checkpoint, Sonic would always have a new starting point and wouldn't have to go all the way back to the beginning. So long as he had lives to spare, that is.

What if I could build something like that for my AI conversations? A checkpoint system that distilled not just what succeeded, but what failed, and the thought path that led to each decision. The reasoning behind what I chose to build, write, or think through.

The Checkpoint Hypothesis

So I built one.

A checkpoint system that captures not just where you are, but how you got there. Why you're heading the direction you're heading. What you tried that didn't work.

The system tracks two categories of data:

Progress state is the obvious stuff. Your goal. Where you are. What's next. What's blocking you. Any decent project management tool handles this.

Reasoning state I think is what everyone misses.

Your current approach and why you chose it. The approaches you rejected and why they failed. The assumptions driving your decisions. The insights that changed your thinking. The questions you raised but never answered.

And my favorite: the Continuity Key. A message from past-you to future-you. A string around your finger for when you come back. Sometimes it's a distilled summary. Sometimes it captures something about the tone of the conversation itself.

In my view, auto-compact preserves the what. Checkpoints preserve the why.

I have CLAUDE.md files scattered across my projects. They help - they tell the assistant who I am, how I work, what I value. But they're baseline posture, not thought continuity. They set the starting point. They don't capture where I was mid-stream on a specific problem.

The difference between these two is the difference between reading someone's code and reading their commit messages. The code tells you what exists. The commit messages, when written well, tell you what they were thinking when they wrote it. Both matter. But when you're picking up work mid-stream, the thinking is what you actually need.

The system is deliberately manual. No auto-save. You trigger a checkpoint at meaningful stopping points. This forces you to articulate your reasoning while it's fresh. To name what you've learned. To leave breadcrumbs for the version of yourself who'll return tomorrow with a fresh context window and no memory of today.

It's personal too. These checkpoints capture what I specifically value about session continuity. Not generic summarization. The actual mental state that lets me pick up where I left off.

Because that's the real problem with AI sessions. You don't lose your place. You lose your mind.

What Loading a Checkpoint Actually Feels Like

Here's a real checkpoint from my system:

Name: checkpoint-2026-01-09-002
Summary: Second Mind Phase 1 - Neo4j setup and client foundation

Goal: Build cognitive infrastructure - Neo4j knowledge graph for
capturing thinking, feeling, and judgment

Progress: Phase 1 of 5 in progress. Neo4j Docker running. Created
mind_ client module with schema and client.py. Pending: entity
resolution, MCP tools, import existing knowledge.

Next Steps:
1. Install neo4j Python package (use uv add)
2. Test connection and schema init
3. Build entity_resolution.py (THE critical piece)
4. Create MCP server with capture tools
5. Import existing 50 knowledge entities

Current Approach: Building on existing core-tools patterns. Clients
go in clients/mind_/, MCP server in mcp_servers/mind/. Using 5 core
relationship types per Boris review. Entity resolution is the
foundation - multi-layer matching (exact -> fuzzy -> semantic -> alias).

Assumptions: Neo4j 5.15 has native vector index support. OpenAI
text-embedding-3-small (1536 dims) for embeddings. Stream instead
of project for top-level classification.

Rejected Approaches: DuckDB with extensions - not native graph.
16+ relationship types - too many, simplified to 5 core. Putting
client code in mcp_servers - should be in clients/ per existing patterns.

Key Insights: Entity resolution is THE problem per Boris review.
Without it, graph becomes garbage in 2 weeks. Dry run mode for all
extraction. Manual use for 2 weeks before building extraction pipeline.

Files Touched:
- ~/projects/system/core-tools/clients/mind_/__init__.py
- ~/projects/system/core-tools/clients/mind_/schema.py
- ~/projects/system/core-tools/clients/mind_/client.py
- ~/.claude/plans/soft-honking-dahl.md

Project: system
Tags: [second-mind, neo4j, knowledge-graph, phase-1]

Continuity Key: Neo4j is running (docker: neo4j-mind, port 7687).
Plan file at ~/.claude/plans/soft-honking-dahl.md has full architecture.
Boris reviewed and identified entity resolution as critical.
Next: install neo4j package with uv, then build entity_resolution.py.

Without the "rejected approaches" field, a fresh session might suggest DuckDB again. You'd explain why that won't work. Do some research again, and lose twenty minutes you don't have. Burning credits in the process.

Load the checkpoint and you skip all of it.

I'll say, "Pull the last checkpoint." Or: "Pull the last checkpoint from the knowledge graph work."

Three seconds to load. Zero ramp-up time. You're just... there. And not simply where you left off. You're back in the mindset you were in.

Something to remember as well: /resume and checkpoints aren't mutually exclusive. Sometimes I'll /resume a session to get the raw conversation or summary history back, then load a checkpoint to restore the reasoning distillation. Used together, you get the full picture.

What's Next?

Checkpoints are snapshots. Frozen moments of reasoning.

But something interesting happens when you accumulate them.

Patterns emerge. Decisions that keep working. Approaches that keep failing. Insights that compound across sessions instead of evaporating between them. The checkpoint from last Tuesday connects to the one from this morning. A thread forms.

This is what led me to the knowledge graph question.

I've been experimenting with a Neo4j graph designed to address this. Three entity types surface: Decisions. Patterns. Insights. The hard part isn't the technology. It's the ontology. How do you structure accumulated judgment? What relationships matter? How does "what happened in this session" become "what I've learned across all sessions"?

As AI handles more execution and as the barrier to entry into this field drops, judgment becomes the scarce resource. The ability to know what's worth building and how to do it right. This doesn't scale through automation. It scales through accumulation and preservation of your learning—and your ability to pull that thread at a moment's notice.

Every time you start a fresh session with no context, you're not just losing efficiency. You're losing the texture of how you think. The decisions that shaped your approach. The patterns you've earned through iteration.

Checkpoints are how you stop losing yourself in the process.

The question I keep coming back to: What if your AI systems could learn not just what you did, but why you did it? What if judgment itself could compound? What if I could not only capture what I'm doing in the real world, but my perceptions and patterns of thought? What if I could retrieve patterns from disparate sessions days, weeks, months apart?

That's where this is all going.

Independent researchers at AI-2027.com have been mapping the timeline for superhuman AI. Their forecast puts 2027 among the most likely years for superhuman coding to arrive. Not "better than junior developers." Actually superhuman.

Their median estimate is 2030. But 2027 keeps showing up across independent forecasts with substantial probability.

My first reaction was to roll my eyes. Another "everything changes by X year" prediction. But then I sat with it. If abundance in capability becomes real, there will be no shortage of tools, apps, and solutions coded to impressive levels. Everyone ships.

In a world of abundant ability, what matters? When everyone can build, what differentiates?

I immediately thought of Rick Rubin. The legendary music producer. Johnny Cash. Jay-Z. RHCP. He sits in the studio and listens. He doesn't play instruments. And somehow, albums that go through him come out different. Better.

Rubin wrote about his approach in The Creative Act. Four phases. First, gather. Absorb from the world. Let intuition guide without imposing structure. Second, experiment. Create freely, knowing most of it won't be good. Third, craft. Start making hard choices about what stays. Finally, complete. Share it. Move on.

Notice what's missing from that process: technical execution. Someone else plays the guitar. Someone else runs the mixing board. It's his ability to know which seeds to water. Which experiments to pursue. Which cuts to make. When it's done.

But what he's known for and why artists seek him out is his taste.

Taste is the sum of everything you've lived through in this physical world, distilled into a sense of what feels right and what doesn't. The feeling in your gut when something clicks and the pit when it doesn't.

Everyone has taste. But not everyone takes the time to understand what theirs is. And fewer still can translate it into consistent creative decisions.

In 2027, humans who know their own taste and trust their judgment will be the ones who are able to wade through the noise. Who surface what's genuinely impactful. What's useful. What sells.

The Producer's Chair

Many of us in technology learned what got rewarded: master the stack, ship fast, handle the fire. Taste came later. If at all. But that model breaks down when AI can do all of "building" better than any human.

Someone still has to sit in the producer's chair. That's where the work shifts. It's a person who can sit between the human and the idea and understand what problem actually needs solving. It's knowing when a solution is elegant versus when it's just clever. It's having the judgment to say "we could build that, but we shouldn't" or "let's build this smaller thing first."

Dr. Malcolm was right:

"They were so preoccupied with whether or not they could, they didn't stop to think if they should"

In a world where technical ability becomes abundant, ideas will jump straight from one's brain to reality. "Can we build it?" stops being the question. "Should we?" becomes everything. Judgment picks the project. Taste shapes what it becomes.

Low cost doesn't mean low consequence. Half-baked ideas still cost creative energy and attention. And that's just the invisible costs.

Look at AI-generated video. The slop is everywhere. Videos created to garner clicks without adding anything to the conversation. Not everything needs to be meaningful. But the ratio of noise to signal has shifted hard. These videos bury what's actually good, and make it harder for good work to get found.

If in 2027 we have abundant access to create code-driven technology, we'll have the same problem. Tools and software all claiming to solve similar problems with varying effectiveness. The energy required to sift through the horde of registration pages will be exhausting. Everyone will have a startup.

That's the risk. Not that we can't build anything, but that we build everything.

Whether to build or not to build requires sound judgment. Understanding the problem landscape. Knowing the difference between a solution that creates value and one that just creates activity. The discipline to say no. Someone sitting in that producer's chair, asking why.

Before 2027, we were limited by capability. We had to be selective because we couldn't build everything. After 2027, we'll have to choose to be selective. That's the shift.

Judgment becomes the skill. Taste becomes the scarce resource.

The Signal

Anthropic recently released Opus 4.5. An engineer from Anthropic posted on X the day of the release:

...maybe as soon as the first half of next year: software engineering is done.

Whether that proves true or not remains to be seen. But it mirrors what the researchers at AI-2027 have been forecasting. Maybe this wasn't as hypothetical as I thought. If that engineer is right, only one thing separates us from the machines: taste.

This is the signal to look inward. Understand what makes your taste yours. Learn how to translate that into what you create.

If you're in tech, ask different questions before you build. Sit with what felt right today and what didn't. Not just in your work - in your life. That's where taste comes from.

Step into the producer's chair. Create beautiful things that solve real problems, not just more tools adding to the pile. What makes sense long term. What makes you feel something. Becoming a successful producer requires knowing your own taste and trusting your intuition.

Here's what's at stake: we're about to have superhuman capability without superhuman judgment. The flood is coming, and what we build with it matters.

Rick Rubin didn't sing the song or play the instruments. But he knew what he was listening for. By 2027, we all get that chair. The question is whether we know our own ears well enough to use it.

I recently cancelled my Cursor subscription.

Not because it didn't work. It worked beautifully. Every keystroke anticipated. Every function auto-completed. Every refactor suggested before I'd even thought to ask. That was precisely the problem.

It always felt like someone was looking over my shoulder. And then, without warning, taking over my keyboard. Not pair programming. Not collaboration between equals. Just an LLM system filling in the gaps for me, watching everything I typed, every file I opened, every mistake I made before I caught it myself. It felt...disjointed.

I switched to Zed and opted out of the ai bits. Clean aesthetic, blazing fast, and most importantly—quiet. Now I work in the terminal with Claude Code and specialized agents I've built. And here's the paradox that keeps me up some nights: I feel like I have less control, but I'm delivering better work.

Welcome to the illusion of control in the age of AI-assisted development.

The Progression Nobody Talks About

It started the way it starts for everyone. Copy and paste to ChatGPT. Ask a question, get an answer, paste it back into my editor. Friction everywhere, but the results were undeniable. Last year, I wrote about how ChatGPT made me 10x more productive back in 2022. That wasn't hyperbole.

Then came the IDE integrations. Cursor. GitHub Copilot. VS Code extensions that promised to make the friction disappear. And they did. The AI was right there, embedded in my workflow. I felt like I had more control because I could see everything happening. I could accept or reject suggestions in real-time. I was still the one driving.

Or was I?

So when I moved to terminal-based Claude Code with specialized agents it felt like an entirely different world. I built an engineer agent that audits code quality, and a writer agent that handles documentation and communication. Infrastructure that persists across every session, every project. I had my workspace and the ai agents had theirs. Rather than both of us squeezing into the same chair at a desk, we each had our separate desks, with our own screens and keyboards.

And that's when the question started gnawing at me: did I just give up control?

The Paradox of Perceived Control

Here's what I realized. With Cursor, I felt in control because I was constantly making micro-decisions. Accept this suggestion. Reject that one. Tweak this line. Override that function. Every moment, I was choosing.

But those choices were reactive. The AI suggested, I responded. The AI filled gaps, I validated. I was busy, engaged, hands-on. It felt like control.

The terminal approach is different. I architect ideas and concepts. The agents execute within the guardrails I've built for them. I don't see every line as it's written. I don't approve every change in real-time.

But here's the paradox: I don't feel like I have to nitpick as much as I used to. I've programmed the agents to follow my best practices. I've distilled my style, structure, and experience into the systems themselves.

Is that more control or less?

The Universal Phenomenon

I know people who just hit accept. Auto-accept everything. They don't even read what Copilot or Cursor suggests anymore. They trust it. I joke that they're turning into "Vibe Coders".

But I think that's just the nature of technology and tooling. As the tools get better, you lean back in your chair more.

That phrase sticks with me. It's a physical manifestation of ceding control. You're not hunched over the keyboard, fingers flying, making every decision. You're relaxed. Observing. Trusting.

When I was grinding through debugging sessions pre-ChatGPT, I was anything but relaxed. I was in control because I had to be. Every line of code was mine because there was no one else to write it. I had to read all of the documentation of libraries and frameworks I used. I had to understand edge cases and potential issues.

Now? I'm the architect of ideas. I know how things should look and be structured because I have that full-stack engineering background. I've implemented that knowledge into the agents that audit the work.

But there are days where I wonder: am I ceding too much control to the AI systems I've built?

The Fear We're Not Discussing

Let me be honest. I'm not worried about delivering bad work any more. Everything I do gets vetted and validated.

The fear is more existential. Did we ever have control?

I think as a society, as a people, we're constantly grappling with the illusion of control. For us, the idea that in some golden era of craftsmanship, developers hand-wrote every line of assembly code and knew exactly what the machine was doing. But that's mythology.

We've always abstracted. We've always built on layers we don't fully understand. How many developers truly grasp everything happening between their JavaScript and the silicon executing it? How many have read the source code for every library they import?

We traded fine-grained control for leverage a long time ago. High-level languages. Frameworks. Package managers. Stack Overflow solutions we copy without fully comprehending.

The difference now is the pace and the intelligence of the abstraction. The tools aren't just executing our instructions—they're anticipating them, suggesting them, sometimes making them for us.

And that feels different. It feels like we're not just standing on the shoulders of giants, but letting the giants carry us.

What Control Actually Means

Here's what I'm learning. Control isn't about touching every line of code. Control is about understanding the system, setting the direction, and ensuring the outputs align with the vision.

IDE-based AI gave me tactical control. Micro-decisions, constant validation, hands-on everything. It felt empowering but became exhausting.

Terminal-based agents (thus far) give me strategic control. I define the architecture. I establish the standards. I build the agents that enforce them. Then I let the system work.

Time not typing equals time thinking. Maybe that's where the real work always was.

The Evolution of Roles

When I wrote about The All Powerful Architect in early 2024, I envisioned a future where mid-to-senior engineers would evolve from granular coding to visionary architecture.

What I didn't realize was how soon I was going to be living it.

The architect doesn't lay every brick. The architect designs the building, ensures structural integrity, and guides the builders. Different kind of control—not over every detail, but over the outcome.

My time is better spent ideating, envisioning, roadmapping. The system handles the execution. I handle the strategy.

The Uncomfortable Truth

The IDE approach felt like more control but was actually less effective. I was busy, engaged, making constant decisions—but many of those decisions were responding to suggestions I wouldn't have needed if the system understood me better.

The terminal approach feels like less control but is delivering better results. The output is tighter, more aligned with my standards, more consistent with my vision.

That's uncomfortable because it challenges the narrative that hands-on equals better.

Sometimes stepping back and letting a well-designed system work is the better choice. But that requires trust. And trust is hard when the tools are so new, evolving so fast, and the implications so unclear.

Wrestling with the Question

I don't have a neat conclusion here. I'm wrestling with this in real-time, just like you probably are.

Some days I feel empowered by the systems I've built. I move faster, deliver better work, have time to think strategically instead of getting lost in syntax.

Other days I feel a creeping unease. Am I still a developer if the agents write most of the code? Am I an architect or just a prompt engineer? What happens when the tools I've built to amplify my capabilities start to define them?

The illusion of control cuts both ways. Feeling in control doesn't mean you are. And feeling less in control doesn't mean you aren't.

The question isn't whether we're ceding control to AI. The question is whether we ever had the kind of control we thought we did, and whether the control we're building now—strategic, architectural, systemic—is actually more valuable.

Maybe we're not pretending to write code. Maybe we're learning what it really means to create software in an age where the tools "think" alongside us.

Or maybe we're just leaning back in our chairs and hoping for the best.

I don't have answers. But I'm still writing. Still building. Still wrestling with what it means to be a developer when the development is increasingly collaborative with systems that feel less like tools and more like colleagues.

And maybe that's the real work now. Not writing the code. But figuring out what role we should play when the machines can write it for us.

Working in Secret, Top Secret, or TS-SCI environments? The rules are absolute: complete air gap, no exceptions. But that doesn't mean you have to sacrifice modern data capabilities or AI readiness.

Having worked in classified spaces for many years, I learned something crucial: 99% of data operations can be handled by three core components: Python, Prefect (both the SDK and the self-managed orchestrator), and Pandas. This isn't about any single product. It's about establishing a sustainable technology standard that enables AI readiness while surviving the realities of classified operations.

This matters more than you might think. When contract teams turn over (and they always do), you retain those pipelines. The knowledge stays. The processes remain documented. You're not starting from scratch every rotation. No more 3-6 months of ramp-up on a 2-year contract.

No more random shell scripts. No more massive unintelligible PL/SQL scripts. No more expensive no-code solutions that break the second you modify a single thing.

It's time to do what I petitioned the Department of Defense and the Department of Justice to do many years ago: establish a standard that builds a solid foundation of code and data using the most popular programming language for data workers worldwide. But Python isn't enough. With just two additional libraries, you'll be able to automate, analyze, and validate 99% of all data problems within government.

Reliable. Resilient. Robust. That's what classified environments demand, and that's what self-managed Prefect delivers.

The Reality of Classified Data Operations

Let me paint you a picture that's familiar to anyone who's badged into a SCIF at 0600.

Your data lives in systems that can't always talk to each other. Not won't. Can't. Different classification levels, different networks, different authorities. You're moving data through authorized transfer points, dumping it into unversioned spreadsheets, or worse: maintaining decades-old VBA macros that one person wrote in 2003 and nobody dares touch. Every byte tracked, every movement logged, every request documented in triplicate.

Need data from the central database? You don't query it yourself. Only two people have that access, and they're all in meetings. So you send an email request with your exact specifications, wait 48 hours, and pray they understood what you actually needed. Want to explore the data, test a hypothesis, or pull a larger sample? Forget it. You get exactly what you asked for—nothing more, nothing less. If you need something different, that's another request and another 24-hour wait.

Your tools? Whatever survived the last security review five years and two contract rotations ago. Your documentation? Scattered across SharePoint sites that may or may not still exist, written by contractors who left six months ago and took the institutional knowledge with them.

Your orchestration? If you're lucky, it's cron jobs that someone set up in 2015 and everyone's afraid to touch. If you're not, it's another contractor manually running SQL scripts every morning at 0530, copying results into Excel, and emailing PDFs to leadership. Or worse: it's you, manually refreshing that ancient Tableau dashboard you've been nursing along since the Obama administration because no one has come up with a better way to do this.

This is where Prefect changes everything.

The Pentagon's Data Dysfunction: A First-Hand Account

Something I learned early on in my time at the Joint Staff and Office of the Secretary of Defense: there's no shortage of data and no shortage of teams in desperate need of data. But there's also no shortage of tools that proclaim to be the ONE solution to solve all their problems.

These are typically over-engineered tools that require a ton of training. And given the high turnover rate amongst contractors in the government, once they leave, the tool is unusable by other staff members until the agency can hire someone with enough experience and the right clearance to pick up where the last SME left off. This lasts until they procure another tool that promises to solve their data problems.

What you end up with is a rotating door of tools, tool experts, and siloed data—in the largest office building in the world.

It's a maddening place to walk into as a data worker. You're given access to an antiquated system, maybe an old Oracle database, possibly Tableau or Power BI and Excel, and asked to perform miracles and build automations with systems and tools that were never engineered to solve these problems.

Most data and analysis lives in desperate sources in some shared drive that nobody has access to except for the IT team and the person who created it—who no longer works there. This causes an endless cycle of recreating the wheel. Unvalidated data running rampant. Mission-critical decisions (often life or death) being made using said data.

Throw AI into the mix, which is currently being rolled out in the Pentagon, and you have a recipe for disaster.

The Tool Carousel: Why Complex Solutions Fail in Government

Here's what I witnessed over and over:

Month 1-6: New enterprise tool arrives with fanfare. Vendor promises it will revolutionize operations. Six-figure training contract. Three contractors become certified experts.

Month 7-12: Tool is partially deployed. Only the two experts can use it effectively. They build critical workflows that nobody else understands.

Month 13-18: One expert leaves for a better contract. Knowledge transfer: two PowerPoints and a half-updated wiki page. New contractor hired, needs three months to understand the system.

Month 19-24: Second expert rotates. Tool is now effectively unusable. Contracting company can't find someone with existing experience with the tool and the correct clearance level. All government staff who have become reliant on the output of this tool are now bombarding the new person with requests they are unable to fulfill.

Repeat indefinitely.

The AI Wild Card: Making a Bad Situation Dangerous

And now we're throwing AI into this mess.

The Pentagon is rolling out AI capabilities across the board. But AI trained on what? Fed with what data? Validated how?

When your training data lives in:

• Excel files with "Copy of Copy of FINAL_v3_ACTUALLY_FINAL.xlsx"
• SharePoint folders nobody can find
• Oracle databases with schemas nobody understands
• Power BI dashboards pulling from unvetted sources

You're not building artificial intelligence. You're institutionalizing artificial ignorance.

AI amplifies whatever you feed it. Feed it unvalidated, siloed, inconsistent data? It will confidently make terrible recommendations at machine speed. And in the DoD, terrible recommendations don't just mean bad quarterly numbers. They mean lives.

This is why establishing a data standard matters now more than ever. Before we can trust AI with mission-critical decisions, we need validated data pipelines, documented data lineage, reproducible transformations, and auditable processes.

The three Ps give you all of this. Every transformation in pandas is explicit. Every pipeline in Prefect is reliable. Every decision is traceable.

Why the Three Ps Work: A Standard That Actually Sticks

This isn't about picking the "best" tools. It's about establishing a sustainable standard that survives contract rotations, procurement cycles, and the next shiny object that promises to solve everything.

Python: The Universal Language of Cleared Developers

Python is already approved in a few classified environments. Your security team knows it, your developers use it, your data scientists depend on it. It's in the baseline. No exotic dependencies, no compiled binaries that trigger security scans, no phone-home telemetry that sets off alarms.

More importantly: every data engineering contractor who walks through your door knows Python. No six-month ramp-up. No specialized certifications. No vendor lock-in. Just readable, maintainable code that the next person can understand.

Pandas: Data Processing Without the Drama

Pandas handles 99% of your data transformation needs without requiring external services. Reading CSVs from that overnight data drop? Pandas. Joining datasets from different classification sources after they've been properly downgraded? Pandas. Creating those daily rollups for leadership? Still Pandas.

No Spark clusters to maintain. No distributed systems to secure. No network traffic to monitor. Just Python processes running on approved servers, transforming data in memory.

And when that contractor who built the pipeline leaves? The next one already knows pandas. The code is readable. The logic is transparent. No black boxes, no proprietary query languages, no "you need to take the advanced course to understand this."

Prefect: Two Parts, Zero External Dependencies

Here's where we need to be precise, because in classified environments, details matter.

Prefect SDK is a Python library—just like pandas. It wraps your existing Python code with enterprise-grade capabilities: automatic retries, detailed logging, intelligent caching, parallelism, and concurrency. More importantly, it lets you create reusable libraries of tasks and flows that enforce your data governance policies. This is code that lives in your repo, and runs on your systems.

Prefect Cloud Self-Managed is the orchestration platform. This is what replaces your cron jobs and manual processes. It runs completely disconnected: no telemetry, no external API calls, no "checking for updates." It's a SaaS application that runs in your environment and stays in your environment.

Together, they transform chaos into order. The SDK makes your code production-ready. The orchestrator makes sure it runs when it should, retries when it fails, provides observability and logs everything for your auditors. And unlike every other tool in that rotating carousel: it's all just Python. Your workflows are Python functions. Your configuration is Python code. When someone new arrives, they don't need to learn a proprietary scripting language or drag-and-drop interface. They read Python code and understand what's happening.

The Contract Rotation Problem (And How Prefect Solves It)

Anyone who's worked in classified environments knows the drill. Your lead developer? Their contract ends in six months. That subject matter expert who built your entire data pipeline? They're rotating to a different contract next month. The institutional knowledge that walks out the door every year is staggering.

Traditional script-based operations become archaeological expeditions:

• "Why does this script check for a file in /tmp/staging?"
• "What system creates the Tuesday drop?"
• "Why do we skip processing on the third Thursday?"

Nobody knows. The person who knew left eight months ago.

With Prefect, your workflows become self-documenting. The workflow metadata includes descriptions, version tracking, and operational notes. New contractors can immediately understand what runs, when it runs, and why certain decisions were made. The workflow is the documentation. The orchestrator shows the history. New contractors see what ran, when it ran, and why it failed. They can trace through the lineage, understand the dependencies, and maintain operations without three months of knowledge transfer.

Deployment Patterns for Air-Gapped Excellence

The Minimalist Deployment

For small teams or single-system deployments, Prefect can be installed with a single Python package and started with one command on your approved RHEL/CentOS box. No Kubernetes required. No cloud services needed. Just Python and full enterprise orchestration.

The Production Deployment

For larger operations with high availability requirements, Prefect runs in containerized environments using Docker Compose with PostgreSQL for persistence. This runs on your existing VMware infrastructure, your existing Docker environment, or bare metal if that's what security requires. No external dependencies, no internet access needed.

The Multi-Classification Pattern

Here's where it gets interesting. Different networks, different classification levels, but you need unified orchestration? Run separate Prefect instances:

• UNCLASSIFIED NETWORK (NIPR): Public data ingestion, open source intelligence processing, sanitized report generation
• SECRET NETWORK (SIPR): Classified data processing, cross-domain solution monitoring, guard-approved data preparation
• TS-SCI NETWORK (JWICS): Compartmented processing, special access program workflows, executive briefing generation

Each network gets its own Prefect instance. No cross-talk, no data leakage, complete isolation. But your workflows? They're consistent across all levels. Your contractors can work at any classification level without relearning tools.

Common Classified Challenges and Prefect Solutions

"We can't install packages from the internet"

Build an approved wheel archive on your unclassified build system, transfer it through your approved process, and install offline in your classified environment. This is standard practice for Python packages in air-gapped systems.

"Our data arrives at random times from random systems"

Prefect's event-driven automation handles this with file watchers and event triggers. Monitor your drop zones for new files from authorized data transfers and automatically trigger appropriate workflows when data arrives.

"We need to track everything for compliance"

Every flow run, every task execution, every retry—it's all in the database. Your security auditor can query exactly what ran, when it ran, who ran it, and what the results were. Need to prove data lineage for that critical report? The full execution history and task dependencies are queryable and auditable.

"Our developers don't have admin access"

Prefect runs entirely in user space. No sudo required, no root access needed. Run it on high ports that don't require privileges. Your developers can deploy and manage their workflows without waiting for IT tickets.

The Human Side of Classified Operations

Let's be honest about something that doesn't get discussed enough: working in classified environments is hard on people.

You can't Google error messages. Stack Overflow is blocked. That cool new Python package everyone's talking about? You'll get it in two years, maybe. You're solving 2024 problems with 2019 tools.

This isolation makes reliable, well-documented tools invaluable. When Prefect shows you exactly where your pipeline failed and why, that's hours of debugging you don't have to do blind. When the UI shows you the full DAG of your workflows, that's documentation that actually matches reality.

And when that new contractor shows up (because there's always a new contractor), they can see what's supposed to run, when it last ran successfully, where the data comes from, where it goes, and who to contact when it breaks.

This isn't just operational efficiency. It's career satisfaction. It's the difference between firefighting and engineering.

Security Hardening for the Paranoid (Which Should Be Everyone)

Even in an air-gapped environment, defense in depth matters:

Encrypt Everything at Rest

Configure Prefect with AES-256 encryption keys and enable database encryption. Every sensitive value, every credential, every piece of metadata gets encrypted before it hits disk.

Lock Down the UI

Configure your nginx proxy to require CAC/PIV authentication for UI access. Restrict access to specific distinguished names from approved organizational units. Every access attempt gets logged, every session gets tracked.

Audit Everything

Implement comprehensive security audit logging with classification-level decorators. Every function call, every data access, every user action gets logged with classification level, user identity, terminal information, and timestamps. All executions are traceable for security review.

The Bottom Line: Mission Success Through Standardization

Here's what I've learned from years in classified environments and my time at the Joint Staff: complexity is the enemy of security. Every additional system is another attack surface. Every external dependency is another supply chain risk. Every proprietary tool is another silo that dies with the next contract rotation.

The Pentagon doesn't need another "revolutionary" platform. It needs a standard. A boring, reliable, learnable standard that every contractor understands on day one.

The three Ps (Python, Prefect SDK + Self-Managed Cloud, and Pandas) give you enterprise-grade data operations without enterprise-grade complexity. You can:

• Process gigabytes of classified data without distributed systems
• Orchestrate complex workflows without cloud dependencies
• Maintain operations through contract rotations
• Pass security audits without scrambling
• Actually validate your data before making life-or-death decisions
• Build on the work of previous teams instead of starting over
• Actually get home before 2000 on a Friday

Is this setup as feature-rich as the latest six-figure enterprise platform? No.

But it's running in production in SCIFs around the world. It's processing classified data for national security decisions. It's surviving contract rotations. It's readable by the new person who started yesterday. It's maintainable by the team that takes over next year.

And unlike that expensive tool gathering dust because the one person who knew how to use it left six months ago, this standard keeps working. Day after day. Rotation after rotation. Mission after mission.

We have enough unvalidated data running rampant through government systems. We have enough siloed information trapped in tools nobody can use. We have enough contractors recreating wheels that have been built a dozen times before.

It's time for a standard that sticks. Python. Prefect. Pandas.

That's what matters in classified environments. Not the perfect tool. Not the next revolution. A sustainable standard that ensures mission success.

Frequently Asked Questions

What are the Three Ps of classified data operations?

The Three Ps refer to Python, Prefect (SDK + Self-Managed Cloud), and Pandas - a strategic technology stack designed for classified environments that enables AI readiness while surviving contract rotations and security requirements.

Why is Python the best choice for classified environments?

Python is already approved in many classified environments, has no exotic dependencies that trigger security scans, and every data engineering contractor knows it. This eliminates the 6-month ramp-up period typical with proprietary tools.

How does Prefect work in air-gapped environments?

Prefect Cloud Self-Managed runs completely disconnected with no telemetry, external API calls, or "checking for updates." It's a SaaS application that runs entirely within your classified network.

What makes this approach different from other enterprise tools?

Unlike complex enterprise platforms that require specialized training, the Three Ps use standard Python libraries that any contractor can understand on day one. This eliminates the "tool carousel" problem common in government contracting.

How do you handle contract rotations with this approach?

Prefect workflows become self-documenting with metadata, descriptions, and version tracking. New contractors can immediately understand what runs, when it runs, and why certain decisions were made without months of knowledge transfer.

Remember: This blog post is UNCLASSIFIED. Specific implementation details for your environment should be discussed through appropriate classified channels.

I learned this the hard way.

Early in my career at the Department of Defense, I was on a team of excellent problem solvers. We could build anything, fix anything, optimize anything. We were masters of execution. But we kept wondering why our solutions didn't land the way we expected. Why did that elegant workflow tool sit unused? Why did teams resist the automation we'd built specifically for them?

The answer was simple: we were missing two critical pieces. We had execution down cold, but we lacked empathy and vision. We over-prescribed solutions that looked perfect on paper but didn't resonate with the people using them or align with where the organization actually needed to go.

That experience taught me something fundamental about technology work. There are three distinct ways to help organizations with technology challenges. Three pillars. And understanding which one you need, and when, makes all the difference.

I. Execution: Technology Consulting

This is what most people think of when they need tech help. You've got a specific problem. You need someone who can solve it.

Technology consulting is focused, tactical work. Migrate this system. Build this API. Fix this pipeline. Create immediate value by addressing a clear technical challenge.

It's specialist work. A consultant assesses your current state, recommends solutions, and executes the plan. You get tangible results, delivered efficiently (hopefully).

Here's what this looks like in practice:

You need a data pipeline to streamline reporting. A consultant builds it.

Your legacy systems need to move to the cloud. A consultant designs the migration and executes it.

Your tools don't talk to each other. A consultant builds the APIs that connect them.

This is the work that keeps organizations running. It's essential. And it's the foundation of everything else.

But here's what I learned at the DoD: execution alone isn't enough.

II. Empathy: Technology Counseling

Technology can transform organizations. But only if people understand it and embrace it.

That's where counseling comes in.

Technology counseling isn't about solving technical problems. It's about understanding human ones. It's about addressing the why behind the what. Why are people resisting this change? Why isn't this tool being adopted? Why does this solution feel wrong even though the specs are right?

Most technical challenges come wrapped in human complexity. Fear of disruption. Lack of clarity. Resistance to change. Cultural friction. Counseling creates space for those concerns to surface so solutions can be shaped collaboratively.

Back to my DoD experience. We were so focused on what we were building that we neglected the why and the how. We didn't have anyone in a counseling or advisory role. We prescribed solutions instead of listening first. We optimized for technical elegance instead of human adoption.

The gap between our efforts and the actual outcomes kept growing. We'd deliver something technically sound, and it would sit there, unused or resented.

That's the difference empathy makes.

A technology counselor listens first, advises second. They guide organizations through change, helping them see not just what's possible, but what's right for them specifically. They build trust. They address fears. They ensure that the people, processes, and technology work together harmoniously.

Here's what this looks like:

A non-profit is struggling with digital transformation. A counselor guides them based on their mission and staff capabilities, not just technical requirements.

A small business is hesitant about automation because they're worried about job disruption. A counselor walks them through it step by step, addressing concerns, ensuring team buy-in.

This is relationship-driven work. It's not transactional. It's about alignment.

As I wrote in Moving Past Consulting, the evolution from consulting to counseling represents a shift toward deeper, more meaningful engagements. Execution is still critical. But without empathy, even perfect solutions can fail.

III. Vision: Technology Advisory

Execution solves today's problems. Counseling ensures people embrace the solutions. But vision ensures that today's solutions don't create tomorrow's problems.

Technology advisory is strategic work. It's about looking beyond the immediate and asking bigger questions: How does technology support long-term growth? What emerging trends or risks should we prepare for? How do we balance short-term needs with long-term objectives?

An advisor doesn't just solve problems. They architect the future. They design roadmaps, create frameworks, align technology decisions with organizational goals.

Here's the difference: a consultant migrates your systems to the cloud. A counselor ensures your team understands why and how to use the new environment. An advisor ensures that migration is part of a larger strategy that positions you for what comes next.

Consider these scenarios:

A mid-sized company is planning international expansion. An advisor ensures their IT infrastructure can scale seamlessly across borders, anticipating challenges before they arise.

A healthcare organization needs to adopt AI-driven tools while navigating complex regulations. An advisor ensures compliance while aligning new technologies with patient care goals.

Vision is the glue that connects immediate actions to a larger strategic picture. It's about clarity. Focus. The ability to navigate change with purpose.

Without vision, you risk pursuing initiatives that solve problems but don't align with where you're actually trying to go. You build. You optimize. You execute. And five years later you realize you've been solving the wrong problems.

As I explored in The Future of Development: The All Powerful Architect, the ability to harmonize innovation with human creativity is what sets visionary leaders apart. Technology advisory isn't just a service. It's a commitment to building a sustainable future.

How the Pillars Work Together

These aren't separate services you pick one of. They're interconnected elements of a complete strategy.

Think about a medium-sized enterprise with outdated systems:

The consultant conducts a technical assessment, identifies the best software upgrade, and oversees implementation. The immediate challenge gets addressed.

The counselor works with employees to understand their concerns about the change, educates them on the benefits, and tailors the approach to their comfort levels. The solution gets embraced.

The advisor evaluates how this upgrade fits into the company's five-year vision, ensuring it aligns with broader strategies and doesn't create new problems down the road. The solution becomes part of a larger transformation.

Here's the framework at a glance:

Consulting is problem-solving. Transactional. Defined by the project. You get solutions delivered.

Counseling is empathy and guidance. Relational. Evolving based on client needs. You get support and empowerment.

Advisory is strategic alignment. Partnership. Organizational transformation. You get roadmaps and clarity.

Together, they provide execution to turn plans into reality, empathy to ensure solutions are understood and embraced, and vision to align actions with long-term goals.

What This Means for You

Most organizations need all three pillars at different times, sometimes simultaneously.

If you're facing an immediate technical challenge, you need execution.

If your teams are struggling to adopt new technology, you need empathy.

If you're unsure where your technology strategy should go, you need vision.

The key is recognizing which pillar you need right now and finding the right people to provide it.

I learned this at the DoD by getting it wrong. We had execution in spades. But without counseling and advisory perspectives, we kept building solutions that technically worked but didn't actually solve the real problems.

That's why I approach technology work through all three pillars now. Sometimes you need me to build something. Sometimes you need me to help your team understand why and how. Sometimes you need me to help you see where you're going and design the path to get there.

The journey doesn't have to be complicated. But it does have to be thoughtful. And it starts with understanding which pillar, or pillars, you need.

Six months later, the same problem resurfaces. Different flavor, same root cause. So another consultant gets hired. The cycle continues.

I've lived this cycle from both sides. As a contractor rotating through DoD agencies. As a consultant at Booz Allen, Deloitte, and the UN Foundation. I've watched organizations spend millions on solutions that never stick because nobody addressed the human problem underneath the technical one.

That's why I don't do traditional consulting anymore. I do something I call Technology Counseling.

The Rotating Door Problem

Here's what traditional consulting gets wrong.

It's transactional. You're hired to solve a specific problem, so you solve that specific problem. You're not paid to ask why the problem exists in the first place. You're certainly not paid to explore the organizational dynamics, the team fears, or the knowledge gaps that created the problem. I've been told on more than one occasion not to solve the client's problem outright.

"If we do that, they'll have no reason to renew the contract at the end of the year. We're here to make the problems they have as painless as possible. Without us, all of their frustrations bubble back up to the surface."

So, we fix the symptomsm but the causes remain.

This creates what I call the "rotating door of consultants syndrome." Organizations become dependent on external help because they never build internal capacity. Every technology change requires another consultant. Every new tool requires another engagement. The costs compound. The knowledge never transfers.

Why does this happen? Because consultants are measured on deliverables, not on understanding. On implementations, not on transformation. On billing hours, not on building resilience.

What Changes When You Counsel

Technology Counseling starts with a different question. Not "what technology do you need?" but "what human problem is this technology supposed to solve?"

Every technical failure has a human origin. Fear of change. Gaps in understanding. Misaligned incentives. Unclear vision. Resistance that nobody acknowledged or addressed.

If you don't understand the human side, your technical solution won't stick. It can't. Because people will find ways to work around it, ignore it, or undermine it. Not out of malice. Out of self-preservation and convenience.

I've seen million-dollar systems sit unused because nobody asked the people who'd actually use them what they needed. I've watched security protocols get circumvented because they made daily work impossible. I've debugged data pipelines where the real problem was that two teams refused to talk to each other.

You can't engineer your way out of human problems. You have to address them directly.

That's what counseling does. It creates space to understand the fears, the resistance, and the misconceptions. To listen before prescribing. To build solutions that work with human nature, not against it.

The Difference It Makes

When I engage with a client now, I don't start with technical requirements. I start with questions.

What are you trying to accomplish? What's blocked you so far? Who's involved? What do they care about? What are they afraid of? Where's the resistance going to come from?

The answers to these questions shape everything. The tools I recommend. The way I structure the implementation. The training I provide. The handoff I design.

Because the goal isn't just to deliver a solution. It's to build the client's capacity to own that solution, maintain it, evolve it, and eventually replace it when something better comes along.

That requires understanding. Partnership. Teaching. Sometimes, honestly, therapy.

Traditional consultants optimize for billable hours. Technology counselors optimize for clients who don't need them anymore.

Why This Matters More Now

AI is accelerating this problem.

Organizations are racing to adopt AI tools without understanding what problems they're solving or why those problems exist. They're deploying agents, fine-tuning models, and building automations because everyone else is doing it.

But AI doesn't fix human problems. It amplifies them.

If your teams don't communicate well, AI tools will amplify that dysfunction. If your data is messy because nobody agrees on definitions, AI will learn from that mess and reproduce it at scale. If people feel threatened by automation, they'll resist it, no matter how technically impressive your implementation is.

This is where the human side becomes critical. Where listening matters more than algorithms. Where understanding context beats computational power.

AI can recommend solutions. It can analyze data. It can generate code. But it can't sit with someone who's terrified they'll lose their job to automation. It can't navigate the political dynamics of two departments who don't trust each other. It can't build the confidence someone needs to try something new.

That's human work. That's counseling work.

What This Looks Like in Practice

Technology Counseling isn't about being nice or empathetic for its own sake. It's about recognizing that every technical problem is downstream from human decisions, human fears, and human misunderstandings.

When I work with a client, I'm trying to understand their story. What they've tried before. Why it didn't work. What they're afraid to admit. What they don't know they don't know.

Then I'm translating between worlds. Technical to human. Human to technical. Making advanced solutions accessible without dumbing them down. Building capacity, not dependency.

And I'm being honest when something won't work. When expectations are misaligned. When the real problem isn't the technology at all.

That's the counseling part. The willingness to tell uncomfortable truths, ask hard questions, and focus on what actually matters instead of what's easy to bill for.

Why I Do This

I learned to code so I could be a far more present parent. Technology gave me freedom, flexibility, and the ability to build a life worthy of my kids.

But I've also seen technology used to extract value without creating it. To solve problems nobody has while ignoring the ones people live with every day. To impress executives while making workers' lives harder.

I don't want to do that kind of work. I want to use technology to solve real problems for real people. To build things that make sense, not just things that are technically impressive.

That requires understanding the human side. Always.

So I don't call myself a consultant anymore. I'm more of a counselor. A guide. A partner in figuring out what actually needs to happen and how to make it stick.

Because the technology part? That's the easy part. The human part is what's hard. And that's where the real work happens.

I tried it with skepticism. It felt strange at first, like learning to use a new search engine that actually understood what I was asking. But within a week, something shifted. My productivity didn't just improve. It exploded.

Tasks that used to eat up hours were done in minutes. The junior developer position I'd been planning to hire for? No longer necessary. This wasn't just a tool making things faster. It was changing what I could do.

Over the next few months, projects that used to intimidate me started feeling manageable. Complex topics that would have taken me days to understand became accessible in hours. Regular expressions, that cryptic language I'd been avoiding for years, suddenly didn't matter because ChatGPT could parse them for me. DevOps, which always felt just out of reach, became approachable.

I had support unlike anything I'd experienced in my career. But it wasn't just about getting help with syntax or debugging faster. Something bigger was happening.

I was starting to see the future. Not some distant vision, but what was already becoming real for mid-to-senior level engineers. We had an opportunity to shift from spending our days in the weeds of code to thinking more like architects.

We're moving toward an era where automation handles more of the implementation. That doesn't diminish what we do as humans. It transforms it. Our ability to envision, to innovate, to understand what needs to exist becomes the valuable part. Not typing it all out ourselves.

Or maybe we're just building better tools for our future robot overlords. Hard to say.

Traditionally, software architects have been like urban planners. They design the framework, set the technical standards, make sure everything fits together. But the role is evolving.

The architect in this new world isn't just designing systems. They're orchestrating how human creativity and AI capabilities work together. They're envisioning not just the structure of applications but how AI-driven solutions interact with users. This demands understanding business strategy, user experience, and technology at a deeper level. Creating systems that are efficient, scalable, and adaptable as the tech landscape keeps changing.

The platform where this work happens will be different too. It won't just be an IDE. It'll be a creative environment that uses AI to incubate, evaluate, and refine ideas faster than we've ever been able to before. The vision translates into working products with minimal direct coding. Prototyping becomes fast. Iteration becomes fluid. The gap between idea and reality shrinks dramatically.

Assuming it stays affordable, anyway.

Coding isn't dying. It's evolving into something that demands more creativity and influence. Mid-to-senior engineers who see this transition coming can position themselves as architects of digital systems, building the future with concepts and vision rather than line-by-line implementation.

That's where we're headed. And it's happening faster than most people realize.