Two Minds, One Signal

In July, Prefect bought Dagster.

That event kicked off an obligatory exploration of a platform I had mostly avoided up to that point, apart from some competitive analysis. Having spent the better part of the last two years selling Prefect, and the four years before that using it to structure all of my data services, I felt confident I had the better toolset. It fit my mental model for solving programmatic problems. Tasks and flows. Durable and procedural execution. My entire career had been centered on this style of engineering, and here was a tool that agreed with me.

Dagster doesn't work that way. While it may have similar concepts to Prefect, the overarching paradigm is fundamentally different. Simply put, Prefect orchestrates what code does while Dagster orchestrates what data is. You declare what data should exist and how fresh it should be, and the engine works out what needs to run. Notice what's missing from that sentence. The steps. To a mind that had spent a career living in the steps, that read less like a paradigm and more like skipping the part where the value was. Nevertheless, I began porting some of my own work over and learning more about assets, components, definitions, and jobs.

Then one day while explaining my model for solving life's many problems, I said something to the effect of, "I usually start with the outcome I'm after and work backwards..." I paused and went quiet, to the befuddlement of my partner. In that instant the Dagster model made sense. Not because I'd finally understood Dagster, but because I'd finally made a connection that had clearly eluded me for quite some time. I had apparently been working backwards from outcomes for years, in everything except my code. And, what's more, I've been writing code with LLMs this way for the last few years without once recognizing the pattern.

Once it clicked, I started seeing outcome modeling everywhere.

The Procedural Mind

When I learned Python, two doors appeared before me: data engineering and software engineering. I chose the software door and learned Django, React, Postgres, Redis, Celery, and started building products.

Pre-AI, coding was very different. Every keystroke was a procedural exercise in completing some overarching process. Building an API? Cool. Parse the request. Validate it. Hit the database. Shape the response. Return it. Five steps, in order, every time. The outcome wasn't something I modeled up front. It was the culmination of tasks I had painstakingly written, and only after writing these functions would I focus on the outcome. Even the whole idea of ETL felt procedural by definition. Extract data, Transform it, then Load it somewhere.

I can trace that mental paradigm back to my early days in IT, walking troubleshooting trees. Step one, step two, escalate. It hardened in my DoD years on runbooks and SOPs, and again at the DOJ, where a pipeline was pull, transform, load, and a cron job to do it again tomorrow. Sure, I had an expected outcome, but all of the pressure and stress existed within the task sequence. Forward movement meant progress.

A procedural mind often creates a situation where you can't see the entire path until it's almost entirely written. One function at a time. We, as people, spend so much time adding steps into a process that need not exist, because each one makes sense in the context of the minor step we're currently operating in.

I never really came into contact with anyone who programmed in this declarative way until about 2024. In fact, I remember saying to someone, "It feels so... messy. You just declare something and write a whole bunch of code against it?!" As a file structure nerd, I was shook. It genuinely felt like focusing on the wrong thing. If I engineered the process correctly, the outcome would be the easy part. In hindsight, I already knew better. I'd in fact learned the opposite lesson eight years earlier, in a different part of my life.

The Declarative Mind

In 2016, while living abroad, I read a book.

I was approaching 30 and predictably on a self-improvement kick with a separate hard deadline. My daughter was due in a few months and I had the vague sense that I should probably have my life somewhat figured out before she showed up. Nothing like cutting it to the last minute, amirite? The book was Vishen Lakhiani's The Code of the Extraordinary Mind, and it exposed me to two new concepts: meditation and a new way to plan for my goals.

The author had a word for rules we inherit but never examine: Brules. Bullshit rules. Go to college. Get the stable job. Get the pay. Climb. Steps, with no real declared end, apart from the inevitable one. They come from what he calls the culturescape, the layer of made-up agreements we mistake for physics. Those agreements come to us in many forms throughout our lives, and I had been unknowingly following Brules my whole life and calling it a plan.

But the idea that rearranged my mental model was his split between means goals and end goals. Means goals are the steps: the degree, the title, the salary. End goals are what the steps were supposed to produce: experiences, growth, something to contribute. His point is that we spend our lives stacking means goals, adding step after step because each one makes sense from inside the step before it.

"Law 8: Create a vision for your future. Extraordinary minds create a vision for their future that is decidedly their own and free from expectations of the culturescape. Their vision is focused on end goals that strike a direct chord with their happiness."

Vishen Lakhiani, The Code of the Extraordinary Mind

Stacking means goals is exactly how I'd been engineering the whole time. Every task valid, every task in order, and focusing on the output always secondary.

So I did his exercise. Three questions: what do I want to experience, how do I want to grow, what do I want to contribute. I wrote down end states instead of steps. Then I worked backwards from each one until I hit the first thing that had to be true.

He never used the word, but that was the first time in my life I knowingly declared an outcome and let the steps be negotiable.

This book genuinely changed how I approached goals in my life. However, it had almost zero impact on how I wrote my code. For the next decade I carried two minds, a declarative one for living and a procedural one for building, each playing in its own channel, and I never once noticed the seam. Not until this summer, that is.

The Code Finally Caught Up

Dagster was the first tool that forced me to make the connection between the two minds. The model I didn't trust in July wasn't skipping the steps. It was moving them to where they belong: underneath the declaration instead of in front of it.

As I said, once I saw it there, I began noticing this model in other places. It was in my LLM prompts, my session framing, my MCP checkpoints, and agent workflows. I scrolled back through a couple months of sessions, and none of them read like a procedure. Desired state. Constraints. Dependencies. "Here's what done looks like." I never once wrote "open this file and add a function." I had been writing software-defined assets in plain English the whole time and calling it prompting.

If you really squint, an LLM is a reconciliation engine. You hand it the state you want and the rules it can't break, and it computes the steps on your behalf. Hand it the steps instead and you're fighting the tool in the trenches. I wrote about that feeling in The Illusion of Control before I had a name for it.

You can see the split in who struggles with AI coding and who doesn't. The people struggling are still dictating steps and line-editing the output. The people who took to it write specs, acceptance criteria, evals, tests. Here is what true looks like when we're done. Go. Dagster was ahead in this respect.

Which is why Prefect buying Dagster makes sense to me now in a way it didn't in July. The industry is moving toward declaring what should exist, but someone still has to durably run the steps.

I'm still of two minds. Only now I'm aware of which one to hand the steps to.