The New Business Skill Is Understanding Systems

For most of modern history, expertise was measured by output. A good programmer wrote good code. A good analyst produced a good report. The ability to make the thing was the barrier, and the barrier was the value.

AI is lowering that barrier fast. The first draft of almost anything — code, reports, designs, contracts, estimates — is now cheap, and in some narrow tasks the machine's draft beats what a mid-level professional would produce.

But making the thing was never the whole job. It was just the most visible part. Behind every competent piece of work sits understanding that rarely gets written down: what the thing is for, what it connects to, how it breaks, and who gets hurt when it does. A senior engineer isn't valuable because she types faster than a junior one. She's valuable because she can look at a working system and say, "this falls over the first time two customers check out at once."

My forecast is simple: over the next decade, value migrates from the production layer to the understanding layer. Not because production stops mattering, but because it stops being scarce.

The easier it gets to generate complex work, the harder it gets to check it.

Software developers are living the early version now. In the 2025 Stack Overflow Developer Survey, more developers said they distrust the accuracy of AI output than trust it, and the most common frustration was solutions that are almost right.

"Almost right" is the dangerous zone. Obviously wrong output gets caught. Almost-right output gets shipped.

This isn't a programmer problem, because everyone is about to run systems they didn't build:

• A business owner has AI build a customer management system. It works, but it's storing customer data somewhere it shouldn't.
• A marketing agency automates publishing. The content is fine until the week it states something false about a client's competitor.
• A lender automates part of its risk analysis. The model performs well on the data it was tested on and badly on the customers it actually gets.

In every case, the system appears to work. That's the whole problem. The defining failure of the next decade probably won't be the crash. It will be plausible, quietly wrong output that nobody is positioned to question.

The cleanest way I've found to think about this is to split work into three layers.

AI will participate in all three. It already writes code, increasingly reviews it, and recommends decisions. Which raises the obvious objection: what if AI gets just as good at verification?

Then somebody has to verify the verifier. Automated review will improve fast; "check this against that" is exactly what machines do well. But that moves the human role up a level rather than eliminating it. The professional of 2032 probably won't read every line of machine-written code. They'll design the checks: what gets tested, what level of risk is acceptable, which exceptions get escalated, and when an automated review isn't enough.

The third layer is different in kind, not just degree. Producing an assessment isn't the same as being accountable for it. A model can flag a loan applicant as high-risk. It can't lose its license, get sued, face the applicant, or decide that this particular case deserves an exception. "The system said so" is not an answer a customer, a court, or a regulator will accept.

Regulators are moving in the same direction, though more slowly than the headlines suggest. Article 14 of the EU AI Act requires that high-risk AI systems be designed so that people can oversee them effectively: understand the system's capacities and limitations, monitor its operation, interpret its output, resist over-reliance on it, and override or stop it. Those obligations are not yet in force. The high-risk chapter applies from 2 December 2027 for stand-alone systems listed in Annex III, and from 2 August 2028 for AI built into products that are already regulated under Annex I. The timetable has moved, but the principle survived it. Someone has to be in the loop, and that someone has to understand the loop.

If value moves to the understanding layer, new kinds of work show up there. Some of these roles exist today. Some are forming inside other jobs. A few are guesses. The labels below say which.

This isn't QA with a new name. QA checks whether a product meets its spec. AI auditing has a harder job, because the system's behavior isn't fully written down anywhere. The same model can answer the same question two different ways, drift as its data changes, and fail in ways no test case anticipated. QA asks "does it meet the spec?" Auditing asks "what is this thing actually doing, and should it be?"

A caution: most of these won't become standalone occupations with their own job boards. They'll get absorbed into existing roles. The operations manager becomes part systems architect. The senior developer becomes part auditor. The compliance officer becomes part oversight specialist. So this isn't a forecast of a hiring boom. It's a forecast that the definition of a competent professional is changing, and the people who notice early get a real head start.

A system can be technically correct and still lead you somewhere bad.

A scheduling system can optimize crew routes perfectly and still burn out your best foreman by stacking every hard job on him, because "hard" wasn't in the data. A pricing model can maximize margin and quietly price out the long-time customers who send you referrals. A hiring filter can match the job description exactly and screen out your best future hire, because their résumé didn't look like the last ten.

None of these are bugs. The systems did what they were told. The failure is in what they were told, and in nobody noticing what was left out.

Models can increasingly reason about tradeoffs. But deciding which tradeoffs a business is willing to make is a statement of values, and values have to belong to somebody.

There's a complication in all this, and it's the part of the forecast that worries me most.

The judgment this essay keeps asking for doesn't arrive on day one. People have always earned it doing beginner's work: debugging code they didn't write, building the reports nobody else wanted to build, grinding through document review. AI is very good at exactly that work.

That is a shortfall against a peer group, not a count of people out of work, and the researchers are explicit that it describes a pattern rather than proving a cause. The comparable figure was 15% in July 2025. The decline shows up as fewer young people hired rather than more people fired, it is concentrated in occupations built on codified knowledge — the formal, documented kind you can learn from textbooks and written procedures — and experienced workers show no comparable gap. The gap narrows when education is accounted for, and the effect is larger in this payroll sample than in national survey benchmarks. It does not disappear.

I dug into that problem in Who Will Train the Humans Who Supervise AI?. Here the point is narrower. If the pipeline of people who learned judgment the hard way is thinning, the businesses that come out ahead won't be the ones waiting for those people to show up. They'll be the ones that make their own systems understandable to the people they already have.

That starts with the person running the business.

None of this means every business owner needs to learn to code. That's less necessary now than it has ever been. What they need is systems literacy.

The second half is the part that matters. Go back to the roofing company and follow that one changed supplier code through the business.

Every part did its job. The lookup returned a product. The estimate was internally consistent. The scheduler scheduled. The failure only exists at the level of the whole system, which is exactly the level nobody was watching.

Catching it doesn't require reading code. It requires knowing that estimates depend on supplier data, and that a sudden drop in close rates is a pricing question before it's a sales question. That's systems literacy in practice: a tiny technical change, recognized as the business problem it became.

The owner of a twelve-person company in 2030 probably won't write a line of code. But she'll need to know:

• ✓What each system in the business actually does, and what it's allowed to do on its own
• ✓Where the data comes from, where it goes, and who can see it
• ✓What “working correctly” means for each system, in numbers she can check
• ✓What failure would look like, and how she'd find out
• ✓Which decisions she will never hand to a machine, and why
• ✓Who is accountable when something goes wrong, by name

None of that requires a computer science degree. It requires curiosity, skepticism, and the willingness to look under the hood of things that seem to be working fine.

Here's a test any owner can run this week: trace one customer from first contact to paid invoice. A homeowner fills out a form. Where does that information go? Who, or what, decides whether they get a call back, and how fast? Who builds the quote, from what numbers, and who notices if it's wrong? At every step, mark which decisions are automated and where a mistake could sit for weeks without anyone catching it.

Most owners who do this honestly hit at least one "I'm not sure." That's the place to look first.

Within five years, I expect "can you explain how your business's systems work?" to be a routine question from investors, lenders, insurers, and acquirers, the way they ask about the books today. A business that runs on systems its owner doesn't understand carries risk nobody has priced.

Forecasts are only useful if they can be wrong, so here are mine, with rough timelines and how confident I am.

I'll revisit this list. Some of it will look obvious in hindsight. Some of it will look naive. That's the point of writing it down.

Every major shift in technology has redrawn the line between what tools do and what people do. The calculator didn't end mathematics; it moved mathematicians to harder problems. The nail gun didn't end carpentry; it moved the skill from driving nails to knowing where they go.

AI is redrawing that line again: machines produce, people understand. That's not a smaller role for people. It's a harder one, and it was always the most valuable part of the work. We just couldn't see it clearly while everyone was busy making things.

Back to that roofing company in 2031. The business that survives the bad week isn't the one with the fanciest agents. It's the one where somebody looked at a dip in close rates, got suspicious, and knew where to look.

That person may be the most important hire of the next decade. They don't need to be a programmer. We just aren't doing much yet to make sure they exist.