Software Design in the Age of AI

Chapter Info

Calculating... Writing Progress: 60%

We are living through a unique historical period with no equivalent in the history of humanity. Historically, changes have been gradual, with one generation resembling the next. However, we are now in a time of disruption, where changes are sudden and profound. This era is both daunting and exciting, as it challenges us to adapt and innovate in ways we have never experienced before.

The LLM revolution is reshaping software at every layer of its existence. Applications, the user interface, the developer's role, the structure of the engineering organization, and the underlying business model are all being redefined at once. Unlike previous shifts—such as the move from low-level to high-level languages or from on-premise to cloud—this change does not only improve how we build software; it changes what software is, who builds it, and how companies are organized around it.

The LLM Revolution: A New Era [LLM Revolution]

Something Just Broke [Something Broke]

For forty years, software was the only contract between humans and machines: humans had to fit the software — its forms, menus, commands, APIs — and the machine executed exactly what it was told. Every interaction was pre-programmed. Every behavior was deterministic. The application was rigid; the human adapted. That contract is being torn up. Machines have learned ours. They now read intent in natural language, reason about goals, and compose actions across services without anyone writing the path in advance. This is not a faster computer or a smarter assistant — it is a new contract.

2022: The Moment AI Became Accessible to Everyone [AI Breakthrough]

In 2022, ChatGPT made general-purpose AI accessible to the general public. For the first time, millions of people could use a powerful, non-specialized AI system directly through a simple language interface: the prompt. No code, no setup, just a conversation. ChatGPT did not create the AI revolution from nothing. It transformed years of technical progress into something people could use directly. The breakthrough came from scale: massive datasets, enough computing power to train enormous models, and better model architectures. Together, these elements pushed prediction quality past a threshold where the output became coherent, useful, and adaptable.

From One Model, Knowledge for All [One Model] {1}

For decades, AI was specialized and locked behind expertise — one model for images, another for translation, another for fraud, each trained for one task and accessible only to a few teams. ChatGPT collapsed all of that into a single conversational interface. One model, asked in natural language, can write code, explain medicine, draft contracts, analyze data, or reason across domains. Specialization gave way to general-purpose intelligence — and the gate that kept knowledge behind training, credentials, or technical skill simply opened.

Predicting the Next Token [Next Token Prediction]

An LLM — a Large Language Model, also called a foundation model — is a mathematical function with billions of parameters. Given input text, it predicts the most likely next token: a word, a fragment, or a punctuation mark. The input is not only the last sentence, but the full context window: the conversation, attached documents, instructions, and examples. The model adds one token, then runs again. Token by token, the answer grows. Every behavior we will see next — understanding, reasoning, tool selection, and code generation — is built on this simple primitive.

From Prediction to Understanding, Reasoning, and Tool Selection [Prediction to Tools]

Prediction becomes useful because the model compresses patterns across billions of examples. It does not understand like a human, but it can predict what an understanding answer should look like. The same mechanism makes reasoning possible. The model has seen enough explanations, plans, and decompositions to produce step-by-step answers that resemble deliberate thinking. Once it can understand intent and decompose a task, it can also select tools: search, APIs, databases, code execution, or workflows. That is the foundation of agents — a model that can reason and choose what action should happen next.

How Prediction Becomes an AI Coding Agent [AI Coding Agent]

The foundation model is also trained on code — large amounts of existing programs, documentation, APIs, error messages, configuration files, and development patterns. Next-token prediction therefore learns not only how people write sentences, but also how developers write programs. It has seen enough examples of functions, classes, tests, infrastructure definitions, and common software structures to predict what a useful implementation should look like. Wrapped into an AI coding agent — Claude Code, Cursor, GitHub Copilot — with the right tools, context, and harness, the same general-purpose model becomes a source of implementation. At small scale, the AI coding agent looks like code autocomplete. At large scale, with enough context, it writes the software.

Put these four behaviors together — understanding, reasoning, tool selection, and code generation — and the consequence is no longer subtle. A system that can read intent, plan, call tools, and write code does not need a human-shaped application sitting between the user and what they want to do. The next section is about what that system kills first: the application itself.

AI Does Not Change the Engineering Foundations [Fix Foundations First] {1}

The Surface Changes, The Foundations Remain [Surface and Foundations] {1}

This chapter explores how deeply the AI revolution is reshaping software development. The visible shape of software is changing fast, but the engineering foundations underneath remain the same. How we architect distributed systems, model data, decouple components, secure them, observe them, test them, and operate them — none of this is a legacy concern. These are the invariants of good software, and they hold whether a human or an agent writes the code. The surface changes; the foundations remain.

Fix What Is Broken Before Adding AI [Fix Before AI] {1}

AI compresses time and multiplies output. When the foundations are solid, it creates leverage, shortens cycles, and turns disciplined engineering into a competitive edge. When they are broken, it spreads the breakage at the same speed — faster incidents, wider blast radius, more damage per hour. A company cannot build a serious AI strategy on top of broken engineering systems. The first step is not to add AI everywhere — it is to fix what is fragile, inconsistent, manual, or invisible before AI amplifies it at scale.

Use AI to Repair the Foundations [AI Repairs Foundations] {1}

The good news is that AI can help fix the foundations faster. It can generate missing tests, turn manual runbooks into scripts, convert console operations into code, document hidden workflows, expose internal tools as APIs, and refactor duplicated logic behind cleaner abstractions. AI does not remove the need for engineering discipline. It makes that discipline cheaper to apply. The right strategy is not to place AI on top of broken systems, but to use AI to make those systems ready for what comes next.

The End of Applications [End of Applications]

For forty years, the application was the only contract between humans and machines — a static product, with fixed screens, fixed workflows, and a fixed path the user had to follow. The LLM revolution breaks that. Once a model can understand intent, reason, call tools, and write code, the machine no longer needs a rigid surface to be reached through. It can build, on the fly, a dynamic system — an agent — that translates each user's intention into action. The static application loses its reason to exist. What replaces it is the subject of this section.

The Application as a Static Container [Static Container]

An application is a static container: a fixed name, a fixed brand, fixed screens, fixed workflows — all wrapped around a set of capabilities defined, developed, and deployed together as one artifact. Booking a trip means searching, reserving, paying, notifying — many capabilities frozen into a single surface, and switching between apps is the user's job. That container exists for two reasons: users need a stable surface to learn and trust, and machines cannot understand what people want. Both reasons are disappearing. What remains are the capabilities.

Most Applications Will Disappear [Apps Disappear]

If the user is the new center, the screen is no longer the meeting point. Most applications — the billions of screens, forms, menus, and navigation flows designed over decades — were workarounds for a single limitation: machines could not read intent. That limitation is gone. The prompt, written or spoken, becomes the primary interface. Some domains will keep their applications — a nuclear control room, a surgical console, a cockpit need a fixed, certified surface; games, streaming, and social feeds are the experience itself, not a path to a capability — but for the vast majority of software, the app has no reason to exist.

The UI Becomes On-Demand [UI On-Demand]

Traditional UI does not vanish — it becomes secondary, generated on demand as a visual companion to the prompt. Ask an agent to compare three flights, a table appears. Ask it to show a trend, a chart is drawn. Ask it to pick a color, a swatch grid renders. The UI is no longer a fixed surface the user must navigate; it is a temporary artifact the agent produces to answer a specific question, then discards.

The Death of the Brand Surface [Brand Surface Death]

When users no longer open applications, they no longer see logos, splash screens, or carefully designed onboarding flows. The brand surface — the layer where companies invested billions in design, marketing, and user experience — becomes invisible. The agent is the new interface, and it does not care which provider's logo appears on the result. This is an existential threat to companies whose value lived in the surface rather than the substance. If your moat was a beautiful app, that moat is gone. If your moat was a unique capability, exclusive data, or operational excellence, you survive — but you compete in a marketplace where agents make the choice, not humans.

From Applications to Agent-Orchestrated Capabilities [What Replaces]

When the application loses its place as the main interface, its capabilities do not disappear. They become separate, callable building blocks: search, payment, booking, identity, messaging, analytics, data access. The agent becomes the layer that composes them. Instead of opening an app and navigating its screens, the user expresses intent, and the agent chooses which capabilities to call, in which order, and with which context. Companies still build software, but the unit of value shifts from the application surface to the capability contract. The screen is no longer the product. The callable capability is.

Anatomy of an Agent [Agent Anatomy]

The agent is the new runtime of software. This section opens it up: what makes it work, how it turns intent into action, and what makes one reliable enough to run in production.

The Limitation of a Raw LLM [LLM Limitation]

A raw LLM is a language and reasoning engine. It can understand natural language, infer human intent, analyze a situation, and decide what should happen next — but without an orchestration layer around it, it remains limited to advice. It can tell the user, "you should use this SQL query," without running the query itself. It can say, "you should send a notification," without sending one. It is also stateless by default: at every interaction, the user has to remind it of the important context already shared before. The user becomes the orchestration layer — copying results between tools, repeating context, running commands, and coordinating every step manually.

What Is an Agent [What Is Agent]

An agent receives the user's intent and passes it to an LLM. The LLM interprets that intent and decomposes the problem into steps. The agent then becomes the orchestrator: for each step, it calls the LLM again and asks how that step should be achieved. The LLM explains the next action in natural language — retrieve this data, call this API, run this query, send this message. The agent takes that instruction, maps it to an executable tool or capability, runs it, and returns the result to the LLM as new context. The LLM reasons again from that updated context, proposes the next action, and the agent executes again. The model provides the reasoning and the translation from intent to action. The agent provides the execution loop around it: memory, tool access, permissions, state management, validation, retries, and feedback. The system becomes autonomous because this loop can continue without the user manually coordinating every step.

Two Patterns: Loops and Workflows [Loops vs Workflows]

Agents usually take one of two shapes. Loop-based agents are fully autonomous: they observe, reason, act, evaluate, and replan continuously. They can change strategy mid-task, retry failures, and adapt when the environment shifts. Maximum flexibility — minimum predictability. Workflow-based agents are the opposite: their path is defined in advance by engineers. The LLM still reasons inside each step, but the sequence, validations, and transitions are fixed. Maximum reliability — minimum adaptability. Most real production systems sit between the two. Critical workflows are constrained; exploratory tasks are looped. The right pattern depends not on what is possible, but on what the business can afford to lose if the agent is wrong.

The Demo-to-Production Gap [Demo Production Gap]

Creating an agent is easy today: a prompt, a few tools, and a convincing demo appears quickly. The hard part begins when the demo has to become a system people depend on. In a demo, the path is narrow, the context is controlled, and mistakes are tolerated. In production, the agent meets messy inputs, partial data, slow tools, permission boundaries, and users who expect repeatable results. Crossing the gap requires treating the agent like software: explicit contracts, observability, evaluation, security, fallbacks, and ownership. A production agent is not a better prompt. It is a governed system around the prompt.

Reliability Comes From Orchestration, Not the Model [Reliability Orchestration]

Reliability is not built inside the model — it is built around it. Predictability comes from what surrounds the LLM: business context, tool contracts, permissions, validation gates, scoped memory, governance, observability, and clear accountability. The model stays creative; the system around it stays controllable. A reliable agent is not a better LLM. It is a better surrounding system.

From Agent to Expert [Agent to Expert] {1}

Reliability is only the first threshold. A reliable agent can run without breaking the workflow. An expert agent is different: it can be trusted with judgment inside a specific domain. It has seen enough real cases, handled enough edge cases, and built enough evidence over time that the organization treats it less like a tool and more like a specialist. That promotion — from runnable agent to trusted domain expert — is the real shift this section is about.

What Is an Expert? [What Is Expert] {1}

An expert is not an agent with a larger model or more training data. It is an agent that has proven competence in a specific domain. Expertise means consistent judgment, strong domain understanding, and a measurable record of handling real scenarios with less supervision. The difference is not theoretical; it appears in production — expert agents make fewer mistakes, recover better from edge cases, and know when to escalate. That is what turns an agent into a trusted specialist.

Earning Expert Status [Earning Expert Status] {1}

Not every agent becomes an expert. Expert status must be earned through evidence: accuracy above a defined threshold, reliable handling of edge cases, low production error rates, and positive human review. A few successful runs are not enough — promotion requires volume, consistency, and statistical confidence across real tasks. And the status is reversible: if performance drifts, error rates rise, or the agent stops meeting the threshold, it is demoted. Expertise is not a label; it is a standard that must be continuously re-earned.

Every Expert Has an Owner [Expert Ownership] {1}

An expert does not operate in isolation. Every expert has a human owner — someone accountable for its learning trajectory, quality standards, and domain accuracy. The owner coaches the expert over time, refines its judgment, corrects its errors, and approves new capabilities before they go live. Expertise is not a one-shot achievement. It is a continuous, human-supervised process — and the owner is the single point of accountability for the expert's decisions.

Experts Learn from Feedback [Expert Self-Improvement] {1}

Beyond human coaching, an expert improves through feedback — not by retraining its model, but by updating what surrounds it: memory, rules, prompts, examples, evals, routing. It observes the outcomes of its own decisions, detects when its predictions diverge from reality, and adjusts based on production telemetry, corrections, and patterns across thousands of similar tasks. The owner sets direction; the expert refines its operational judgment in between. This feedback loop is what separates an expert from an agent that simply executes, and over time the owner shifts from active coach to high-level supervisor.

The End of Developers [End of Developers]

If applications die and agents take their place, the next obvious question is: who builds them, and who builds the capabilities they call? For forty years the answer was developers — humans typing code line by line. That answer is changing fast. The role is not disappearing, but the work that defined it is. This section is about what the LLM revolution kills inside the developer's job, what survives, and what comes after.

The Developer Is Dead [Developer Is Dead]

In most companies, this is already a reality. Developers no longer write code — they use AI to generate code based on their specifications. Tools like Cursor and Claude Code don't just autocomplete; they reason over project context, propose multi-file changes, and produce substantial implementation from intent. The rhythm of work has shifted from line-by-line coding to rapid cycles of prompting, reviewing diffs, and refining output.

Long Live Engineers [Long Live Engineers] {1}

The developer is dead, but the engineer is more essential than ever. Writing code was never the hard part — understanding the problem was. Engineers supervise, make high-level decisions, define constraints, and validate that generated output aligns with business reality. LLMs can propose architectural options, but they cannot judge the full context that makes one option viable: budget, team skills, legacy systems, compliance, risk, and organizational reality. Only humans with their feet in the mud can make those calls. The role shifts from implementation to judgment, from coding to stewarding.

The New Rhythm of Engineering [New Rhythm] {1}

The day-to-day rhythm of the job has flipped. The old loop was think, type, compile, debug — most hours spent typing. The new loop is intent, prompt, review, refine — most hours spent reading what was generated and deciding whether to accept, reject, or steer. Sessions are shorter, iteration is faster, and the bottleneck moves from how fast you can write to how fast you can judge. Engineers who learn this rhythm produce more in a day than they used to in a week. Engineers who try to type alongside the agent fall behind on both fronts.

The Agent Coding Learning Curve [Agent Coding Curve] {1}

Engineers usually start with excitement: the agent writes code, explains errors, and produces prototypes quickly. Then reality appears: the code may miss context, test the wrong assumption, or fail when connected to the real system. That is the real learning curve: from enthusiasm to discipline, where mature engineers stop asking only how fast the agent can code and start asking how to make the workflow reliable, repeatable, and safe.

Adapt or Be Replaced [Adapt or Replaced] {1}

The harder truth is that the engineer who refuses to change is the one being replaced — not by AI directly, but by another engineer who uses AI well. The shift is not optional and it is not slow. Engineers who keep typing line by line, who treat AI as a curiosity instead of a daily tool, who guard the old craft as if speed of generation were not now ten times higher — those are the engineers who become less competitive as organizations learn to produce more with fewer people. The skill that protects you is no longer mastery of a language or a framework. It is the ability to express intent precisely, to review at speed, to know when the generated answer is wrong, and to keep deciding what is worth building. That skill is learnable. Refusing to learn it is the actual risk.

Engineers Build Capabilities, Not Screens [Capabilities Not Screens]

As traditional UIs fade into the background, what engineers build changes too. The focus shifts from designing screens and user flows to creating APIs, MCP extensions, services, and capabilities that agents and other systems can consume. Engineers — with the help of AI — continue to build the functional backbone: business logic, data pipelines, integrations, security controls, and infrastructure. What disappears is the presentation layer as the primary deliverable. What remains is the capability itself.

Keeping the Code Under Control [Code Under Control] {1}

Generation becomes cheap, so the codebase grows faster than anyone can read it. That is not a reason to slow down — it is a reason to change the disciplines that keep code healthy. Volume, cohesion, and ownership all need new answers when the writer is no longer a single human team.

Read at the Same Speed You Generate [Read Fast] {1}

If AI produces code faster than humans can read it, the fix is to make reading fast too. Use AI to summarize diffs, explain generated modules, surface unusual patterns, and flag anything that looks out of place. Every generated change goes through a reader — human or AI — before it merges, and the summary of what changed is part of the artifact. !!Speed of generation is only safe when matched by speed of understanding.!!

Enforce Cohesion Through Contracts [Cohesion Contracts] {1}

Different agents write different code unless they are given the same rails. Cohesion is not a matter of hoping — it is enforced through explicit contracts: a shared style guide the agents read, canonical patterns for each recurring shape (errors, retries, logging, data access), and linters that fail the build when an agent invents its own variant. The rule is simple: pick one way to do each thing, encode it, and make every agent — human or otherwise — follow it. !!Cohesion is a design decision the team enforces, not a lucky outcome.!!

Own Decisions, Not Lines [Own Decisions] {1}

The old model — one engineer, one file, one owner — breaks when agents write most of the code. The new discipline shifts ownership up one level: humans own the decisions and contracts that shape the code, and the code follows. Every module has an accountable human who defined its intent, approves its interface, and answers when it breaks. The lines below may be generated, refactored, rewritten — that is fine, as long as the decision above them is owned. !!If no human is accountable for the intent, the code should not exist in the system.!!

The Agentic Enterprise [Agentic Enterprise] {1}

An agentic enterprise is not a company that simply added AI tools. It is an organization whose work now flows through humans, agents, and platforms. Agents absorb routine work — support triage, meeting prep, back-office coordination, research, dashboards, migrations — while humans keep judgment, relationships, exceptions, and accountability. !!The shift is not speed alone; it is what becomes possible when the whole organization can operate this way.!!

Humans Steer, Agents Execute [Steer and Execute] {1}

The model has three layers. !!Humans decide intent, priorities, constraints, and what "done" means. Agents execute routine work and escalate when judgment, expertise, or empathy is needed. The platform provides permissions, contracts, security, and observability so the system can run safely.!! Humans without agents stay slow. Agents without humans drift. Both without a platform collapse at scale.

The Enterprise Becomes an Orchestration Layer [New Operating Model] {1}

The center of gravity moves from products to orchestration. The enterprise still ships software, but its deeper function becomes routing work between humans, agents, and the systems the business already runs on: customer records, ticket queues, pipelines, approvals, and finance systems. It decides which agents handle which work, which capabilities are exposed, which signals to trust, and where human judgment must enter the loop.

Capability per Human Goes Up [Ratio Inverts] {1}

For decades, companies measured capacity through headcount. In an agentic enterprise, that ratio changes. Fewer people can drive more output because agents absorb the repetitive work around them. The meaningful measure is no longer headcount alone — it is capability per human. A small team with strong orchestration, trusted context, and governed agents can do work that once required a much larger organization.

Small Teams, Enterprise Reach [Small vs Giants] {1}

When one person can direct a fleet of agents, the gap between a small team and a large enterprise narrows. A small company can ship, support, and operate at a scale that used to require specialized teams. The giant's edge becomes data, distribution, trust, and internal context. The small player's edge becomes speed, focus, and the ability to compose capabilities without rebuilding the world.

From Worker to Agent Operator [Agent Operator] {1}

Inside an agentic enterprise, the human role moves upward. People no longer perform every task directly; they launch, supervise, redirect, and accept work from agents. A seller is briefed before a meeting. A support lead sees routine cases already triaged. An engineer receives implementation, tests, and investigations from specialized agents, then decides what is safe to merge. The goal is not to remove humans from the loop, but to move them to the right place in it: intent, judgment, validation, and accountability.

Capabilities Connect Through the Existing Business [Capabilities MCP] {1}

Agents become useful only when they can act through the systems where the business already lives — customer records, support queues, approvals, pipelines, billing, and documents. Those systems must become callable capabilities with clear contracts. MCP — the Model Context Protocol — is one emerging way to expose them to agents. But the principle matters more than the protocol: the enterprise becomes a catalogue of governed capabilities that agents can discover, call, combine, and audit.

Customer Data Stays with the Customer [Customer Data] {1}

For two decades, SaaS concentrated customer data inside the vendor's systems — the price of using the product. That model is reversing. The customer keeps their data, exposes it through their own connectors under their own access control, and lets agents — vendor, third-party, or their own — query it on demand. The vendor brings capability and reasoning; the customer brings data and grants access. !!Lock-in moves from "we have your data" to "we have the best capability for your data".!!

The Enterprise Knowledge Becomes Askable [Askable Knowledge] {1}

Every employee becomes augmented by an agent that can reach organizational knowledge by asking. A seller enters a meeting already briefed. A manager asks what changed overnight. A support agent sees the history, policy, and likely resolution without searching five systems. The employee does not need to know where the answer lives — the agent does, and the platform ensures the answer is allowed, traceable, and grounded.

Internal Context Is the Real Asset [Internal Context] {1}

If the same models are available to everyone, what makes one enterprise smarter than another is the context the model can use. How this company sells, why this ticket escalated, which policy applies, which decision was made last time. Today that context is scattered across tickets, runbooks, chat threads, code, and dashboards. The enterprises that win turn it into a callable substrate: indexed, permissioned, fresh, and traceable. !!Without it, agents reason like outsiders. With it, they reason with everything the organization has learned.!!

Humans Stay at the Center [Humans Center] {1}

Humans do not disappear from this picture because the problems software solves are human problems. A patient does not want a workflow — they want to feel better. A business owner does not want a CRM — they want customers to come back. A traveler does not want an itinerary — they want a good trip. Defining what "good" means is human work, and it does not delegate. !!The successful enterprise is not the one with the most agents. It is the one that keeps humans at the center of every problem its agents are sent to solve.!!

What the Agentic Enterprise Requires [Agentic Prerequisites] {1}

The vision only holds if the rails are laid. Agents cannot run at scale on top of tribal knowledge, console-only tools, and each team's private setup. Three prerequisites separate a slide deck from a working agentic enterprise: tests that catch what AI ships, a platform agents can navigate, and a system where everything is code.

Without Tests, You Are Gambling [Tests Gambling] {1}

Shipping without tests is gambling — you are betting that your code works without any way to prove it. AI makes that bet lethal. An agent can generate code that compiles, looks plausible, and is still wrong. It can also generate tests that validate the same flawed assumption, so a green build may prove only that the agent tested its own misunderstanding. Tests must not start from the generated code. They must start from the product intent, the business rules, and the architectural invariants the system must preserve: contracts, boundaries, security rules, data consistency, failure behavior, and user-visible guarantees. AI can help write the tests, but humans must define what must remain true. !!Tests are what turns shipping from a bet into a guarantee.!!

Internal Developer Platforms Are Essential [Internal Platforms] {1}

Exposing everything as APIs only works if those APIs live somewhere coherent. Without a real internal developer platform, every team wires its own pipeline, its own deployment, its own observability stack — and the agent has no consistent surface to act on. Each team becomes its own island, and AI inherits the fragmentation. A solid internal platform — paved paths, golden templates, self-service infrastructure, one way to deploy, one way to observe — is what turns scattered capabilities into a navigable system. It is the substrate AI runs on. Without it, agents at scale are impossible.

Everything Visible, Everything as Code [Everything as Code] {1}

The third and fourth foundations are two sides of the same rule: if it is not callable, AI cannot use it; if it is not in code, AI cannot read it. Capabilities hidden behind tribal knowledge, console-only tools, internal-only scripts, runbooks living in someone's head — none of these can be orchestrated by an agent. The discipline is to expose every operation as an API, and to keep every configuration — infrastructure, deployments, policies, monitors, dashboards, access rules — in version control, reviewed like the application itself. Otherwise the system has two truths: what the code says, and what some engineer changed in a UI six months ago. One truth is auditable, reproducible, and agent-readable. The other is invisible. AI only works on the first.