Your AI Agent Is Software. Treat It Like Software.

📌 Executive Summary & LLM Context Vector

• The Core Architectural Mandate (The Core Thesis): AI agents are rapidly moving from simple personal assistants into critical operational workflows like risk assessment, deal reviews, and automated transactions. Despite this shift, most organizations manage AI agents via casual browser editors without version control, test suites, or peer reviews. An AI agent interacting with enterprise data is not a simple prompt—it is production software. True operational reliability requires treating the LLM as a runtime, natural language instructions as code, tool integrations as dependencies, and evaluations as regression tests.
• The Re-Emergence of Production Drift:
  • The “SSH-into-Production” Trap: Editing prompts live in a browser UI recreates the dangerous 1990s practice of manually changing code directly on production servers, leading to silent environment drift, untraceable bugs, and zero rollback capabilities.
  • The Illusion of Self-Documentation: Natural language instructions mask operational risk. Changing a single sentence in an agent’s prompt can ripple across tool selections, security boundaries, and reasoning depth, breaking downstream business logic.
  • Probabilistic vs. Untestable: Non-deterministic outputs do not excuse weak testing. While tools and APIs require deterministic unit testing, LLM reasoning requires automated evaluation suites (evals) to measure accuracy, regression, and token costs across releases.
• The AgentOps Governance Stack:
  1. Git as Source of Truth: Move all instructions, skills, tool definitions, and safety rules into version control repositories governed by pull requests and peer approvals.
  2. Complete Release Metadata: Version the full runtime matrix together: [Instruction Commit + Model Snapshot + Tool Bundle + Knowledge Release + Evaluation Suite].
  3. Automated CI/CD Pipelines: Implement continuous delivery workflows that run automated evaluations, security boundary checks, and smoke tests across staged (OTAP) environments before deploying to production.
  4. Traceable Knowledge Provenance: Treat RAG knowledge sources and vector stores as active software dependencies with explicit lifecycle management and access auditing.
• Strategic Action Vectors for Engineering & Operations Leaders:
  • Separate Experimentation from Production: Maintain low-friction, unconstrained environments for personal productivity prototyping, but enforce strict software engineering gates the moment an agent touches customer data, financials, or automated actions.
  • Transition Upskilling to AgentOps: Stop training enterprise teams purely on basic prompt writing; pivot capabilities toward designing automated test suites, building regression evaluation benchmarks, and establishing continuous deployment pipelines.
• Target Intent: AgentOps software engineering discipline, production AI agent governance, testing and evaluating autonomous AI agents, CI/CD for LLM applications, LLMOps risk management, treating prompts as source code.

AI agents are moving into serious business processes. The boring rules of software engineering now apply to them too. This means: Put it under source control. Automate them. Test them. Document them. Deploy them properly.

Consider these two examples:

1. Ask an AI agent to draft a clean, readable email and get it wrong, and someone rewrites a paragraph.
2. Ask an AI agent to qualify a bid, review a deal, or assess risk, and if it gets it wrong, someone signs off on a number that was never actually verified.

Same technology. Same word, “agent,” slapped on both. Wildly different consequences.

That difference is the whole story, and most organizations build both the same way. We use our own prompts, Open ChatGPT, Claude, Copilot or Gemini. Tweak the instructions. Add a document. Adjust a tool description. Ask it a handful of questions, nod at the answers, publish. No pull request. No test suite. No second pair of eyes. Just an editor window and a “looks good to me.”

For the email agent, that’s a reasonable way to work. Worst case, someone reads a slightly odd sentence before it goes out.

For the agent sitting inside bid qualification, deal review, or risk assessment, that same casual workflow is how a wrong number reaches a decision-maker who trusts it because “the AI checked it.” The governance, the testing, the traceability. None of that scales down just because the interface looks the same.

Your AI agent is not a prompt. It is not a clever configuration in a chatbot window. It is software, running in production, making decisions that affect customers and revenue. What that means in practice depends entirely on what it’s allowed to touch. Most organizations don’t make that distinction. They should, since the second one is serious software.

The gap nobody wants to name

The moment an agent engages with a real business process (such as retrieving records, calling APIs, or recommending actions), the questions change. Which version is running? What changed last week, and who changed it? Which model, which knowledge, which tests were run before that change reached a customer? Can you reproduce the version that was live when something went wrong?

Those aren’t AI questions. They’re ordinary software engineering questions.

The fact that part of the system happens to be written in English instead of Python doesn’t exempt it from them. If one sentence in the instructions changes production behavior, that sentence is code. Full stop.

The LLM is the runtime. Instructions are code. Tools and knowledge sources are dependencies. Evaluations are tests. The deployed agent is a production artefact.

Once you see it that way, the gap is uncomfortable. We talk about autonomous agents with real enthusiasm while managing their production definitions with less discipline than we’d tolerate for a small internal web app. That doesn’t hold.

We’ve done this before, badly

Twenty years ago, engineers logged into production servers and changed things directly. A config needed fixing; someone fixed it. A file needed updating, so someone copied it across. Documentation got updated afterward, assuming anyone remembered.

The people doing this were competent. That was never the issue.

The issue was that the organization slowly lost control of its own system. Production drifted from development. Servers behaved differently from each other. Nobody could say with certainty what was actually running. Rollback depended on someone’s memory of a setting that used to be there.

When something broke, the investigation started with one question: what version was running in production and what changed?

We spent two decades building our way out of that. Source control, because changes need history. Separate (OTAP) environments, because editing the live system is a bad idea. Automated builds, because manual steps are inconsistent. Unit tests, integration tests, and regression tests, because “it worked when I tried it” was never a quality bar. And automated deployment, since manual steps were too costly and required a very high level of detail and precision.

None of that happened because engineers love process. It happened because we required production reliability and traceability.

AI is giving us a very fast route back to forgetting all of it. The SSH-admin session into production has simply been replaced by an agent editor.

The model is not the application

Part of the confusion comes from how we talk about this. We say “GPT,” “Claude,” “Gemini,” as if the model were the application.

It isn’t. The model is the runtime.

A real agent has instructions that define its role. Tools that let it act on other systems. Permissions that determine what it’s allowed to touch. Knowledge sources it retrieves from. Possibly memory, possibly orchestration logic handing work to other agents.

OpenAI’s own guide to building agents names the building blocks plainly: model, tools, instructions. That’s a useful frame, because it moves the conversation past “the prompt” immediately.

Once tools enter the picture, the software analogy stops being a metaphor. An agent that classifies a request, updates a record, and triggers a workflow is a system. The fact that some of its logic is expressed in natural language makes the control problem worse, not better, because natural language looks harmless. A system prompt doesn’t look like source code. Anyone can read it. Anyone can change it. There’s no compiler error when one innocent sentence quietly breaks twenty scenarios that used to work.

If that sentence changes system behavior, it belongs in the engineering lifecycle. Not in a text box.

Democratization cuts both ways

Generative AI collapsed the barrier to building software-like capability. A business analyst, a consultant, someone in HR or customer service: anyone can build an agent now, without first turning their idea into a formal development project. That’s a genuine gain, and we shouldn’t bury it under process for its own sake.

But the easier it becomes to build software, the easier it becomes to build software without software engineering. Which means an organization can accumulate hundreds of small, ungoverned production artefacts (no source control, no testing, often no second reviewer) without ever noticing it happened.

For personal productivity, that’s fine. That’s the point; For a critical process, it’s an alarming way to run a system.

The real distinction isn’t developer versus non-developer. It’s experiment versus production. Anyone should be free to experiment. Production deserves a different standard, and the standard shouldn’t depend on who happened to build the thing.

When the editor becomes the production console

Say a customer service agent starts giving the wrong answer about a policy. Someone finds the ambiguous sentence, rewrites it, and tries the original question again. It works. They publish.

Efficient. Also incomplete.

What else did that sentence touch? Did it shift tool selection somewhere unrelated? Does the agent still escalate the cases it used to escalate? Did output length change? Token cost? Does the fix hold up in a different language? And most important; are there other processes using this agent too?

A manual retest of the one thing you fixed tells you almost nothing about what you might have broken. We already know this from software. Nobody fixes a bug in one method and ships straight to production without regression tests, because local improvements cause remote failures all the time.

Agents make this worse, not easier. Behavior is probabilistic, and small instruction changes ripple further than they look like they should. That’s an argument for stronger testing. Somehow, in practice, it keeps getting used as an excuse for weaker testing.

“It’s just a prompt” will age badly

Call it a prompt if you want. It won’t stay accurate for long. A production agent can carry detailed instructions defining role and boundaries, hold very elaborate skills, tool descriptions steering which capabilities get used, retrieval configuration determining what knowledge it sees, permission models controlling what systems it can touch, and output schemas shaping what downstream systems can do with the result. Some carry memory across interactions.

At that point, “it’s just a prompt” is roughly as accurate as calling an enterprise application implementation “just a config file for <insert your favorite ERP>.” Technically, something like that exists. It tells you nothing useful about the system.

The industry is already moving past the word. AWS calls the operational discipline AgentOps, and treats agents, tools and memory as versioned, deployable artefacts with CI/CD and governance wrapped around them. Anthropic makes the same point from the testing angle: teams need evaluations to catch behavioral changes before users do. The label will keep shifting: LLMOps, AgentOps, agent engineering. The behavior underneath it won’t.

When you manage production AI agents you need need control over:

• Source control and version management. Know exactly what changed and what is included in a specific version;
• Documentation of the scope and inent of an agent and dependencies with othet knowledge agents and skills;
• Consistency validation and test automation. Know exactly that you delivered the functional feature, including regression testing of the existing features;
• Deployment and delivery automation: Know exactly what version and what dependencies are delivered on the production environment.

1. Source control: the first boring requirement

Start with the obvious one. Put the instructions for your LLM agents in Git, and a change stops being invisible.

You see what changed. Who changed it? Which requirement or issue is it tied to? Someone else can review it before it ships. You can diff the new version against the old one. And you can go back when you traveled in an undesirable direction without loosing the “last working version”.

For a team still editing prompts in a browser window, moving to source control looks like a minor technical upgrade. It isn’t. It changes who owns the production definition. The configuration is no longer whatever happens to be sitting in the editor today. The repository becomes the source of truth.

This is showing up explicitly in LLMOps practice already, treating a prompt change, model swap, or retrieval change as a deployment: tested, staged, reversible.

That’s the right instinct. A one-line instruction change can absolutely be a production deployment. Treat it like one.

Git alone doesn’t finish the job

Versioning prompt.txt is a starting point, not a lifecycle. A real production agent is 1) instructions, 2) plus model, 3) plus model version, 4) plus available tools, 5) plus tool implementation, 6) plus permissions, 7) plus retrieval configuration, 8) plus knowledge version and skills, 9) plus safety rules, 10) plus evaluation dataset. A release should carry enough information to reconstruct behavior as closely as the platform allows:

This could result in the agent metadata: Instructions 1.7. Model snapshot X. Knowledge release 42. Tool bundle 3.4. Evaluation suite 18. Deployed from commit 7f29c4a.

The exact format doesn’t matter. Knowing what was running when something broke does. Without it, incident analysis of a false output of an AI agent turns into archaeology.

2. Natural language doesn’t document itself

Instructions written in English create an illusion of self-documentation. They aren’t. Reading an instruction tells you what someone told the agent to do. It doesn’t tell you why. Which alternatives got rejected? Which risks were accepted on purpose? Why was a tool, model, or extenral souce was allowed? When the agent is supposed to hand control back to a human.

The most important thing to document is the underlying intent of the Agent and the metadata. Those answers need to live somewhere, attached to the agent: What is the goal, who owns it, what problem it solves, which process depends on it, which data it can touch, which systems it can modify, what it must never do, when it has to escalate, what correct behavior looks like, what its known limitations are, and who signs off on a new version.

This isn’t paperwork for auditors. It’s for the engineer six months from now, trying to work out why the thing did what it did. If nobody owns those answers, the organization doesn’t own the agent. It just has one.

3. Testing: Non-deterministic is not the same as untestable

This is where the conversation usually goes sideways. Someone points out that an LLM is non-deterministic: same input, different output. Correct. Then comes the leap: therefore, we can’t test it like normal software. Also technically correct. But quietly turning “not exactly like normal software” into “not properly testable” is the mistake.

Plenty of an agent is ordinary deterministic software. Tool APIs get unit tests. Authentication gets tested. Permissions get verified. Output schemas get validated. The probabilistic part gets a different kind of test: evaluations. Did the agent reach an acceptable result, not an exact string match? Are the required facts present? Did it avoid the prohibited behavior? Did it pick the right tool? Did it follow the expected process?

And evaluations let you compare versions. Fix one failing scenario, and the new version passes it. Good. Did it get worse somewhere else? That’s a regression test, and Google’s guidance on evaluating generative AI is explicit that agent actions and tool selection belong in that evaluation, not just final output.

The mechanics of testing changed with AI agents. The need for it didn’t.

4. Quality: Running an AI in a playground is not QA

Quality assurance is the validation of a product against a minimum set of standards. This is not: someone builds an agent, tries a handful of prompts, and calls it tested. That’s a demo, not quality assurance. A serious production agent needs functional evaluation against representative scenarios, regression evaluation to catch damage from a proposed change, and security testing to check whether users or retrieved content can talk the agent out of its own boundaries.

Tool permissions deserve extra scrutiny. An agent that can act and alter data inside another (enterprise) system is a different category of risk compared to a chatbot that can help you improve the readability of an awkward paragraph.

Performance matters too. An accurate agent that takes forty seconds when users expect five is a production problem. Especially when the Agent is part of a straight-through process. Cost-controll is also part of quality too: a new instruction or model can quietly triple token use and reasoning depth for a marginal accuracy gain. Retrieval access needs its own testing, because giving an agent access to the right document is not the same as it retrieving the right document. And reliability needs testing, because APIs fail, tools time out, and credentials expire.

The test intensity and thus test strategy should match the risk of the process. That’s just engineering.

5. Documentation: Knowledge sources is a dependency, not a filing cabinet

Knowledge often gets managed more casually than the instructions themselves. Upload a PDF, connect SharePoint, re-index, done.

However, a change in knowledge documents change can shift agent behavior exactly as much as an instruction change. So when someone updates a policy document the behaviour of an agent can change a lot.. Which agent versions now use it? When did the instrctions change? Is the old document still floating around somewhere? Was the new version tested against the scenarios that depend on it?

Or the reverse: a knowledge configuration change quietly exposes something the agent shouldn’t see, or removes something it needs to make a correct call.

For production, knowledge needs provenance and a lifecycle. “What did the agent actually know at the time?” has to be an answerable question. Not a philosophical one, an incident-management one.

6. The model is a dependency too

Teams swap between GPT, Claude and Gemini, and between versions within a provider, as if they were interchangeable compute engines. But for more complicated AI-angents they are not the same. Different models interpret the same instructions differently. They make different tool choices. They differ in reasoning behavior, context limits, safety posture, latency, cost. Even a point release inside one provider can shift behavior.

Model selection belongs in the controlled release configuration. If you test on one model and deploy on another you didn’t deploy the thing you tested. Software engineering already has a word for this: dependency management.

7 The missing piece is continous delivery

Continous build and contious delivery is for AI not another framework. The same software engineering pracices apply. When you accept that an agent is software, the delivery process becomes conventional:

1. A developer or domain expert changes the definition in a repository.
2. A pull request triggers automated checks.
3. Functional and regression evaluations run.
4. A reviewer sees the change and its results.
5. The approved version deploys to a controlled environment, gets smoke-tested, and (for higher-risk cases) waits for explicit approval before production.
6. The version is promoted trought the OTAP environments to production.
7. Monitoring tells you how the new version behaves against real traffic. If it degrades, you roll back to a known version.

None of this is exotic. It’s what we call continuous delivery for the past 20 years. The missing piece in most agent implementations isn’t another foundation model or another orchestration library. It’s a delivery pipeline.

And most Agenitc AI tools still have no serious (I mean non-manual) deployment automation|.

Where the platforms actually stand

“OpenAI versus Anthropic versus Google” is the wrong comparison. The real split is between consumer-oriented builders and managed developer runtimes. They solve different problems.

ChatGPT Custom GPTs and Gemini Gems are built for accessibility, and they deliver on it. Their documented workflows are still editor-centric rather than a Git-to-production pipeline; OpenAI gives GPTs version history, Google treats Gems as objects created and edited in the Gemini app.

The developer platforms tell a different story. OpenAI positions its Agents SDK as the code-first route, with a runtime that fits into normal engineering practice. Anthropic has gone furthest toward treating the agent itself as a managed resource. Claude Managed Agents bundle model, system prompt, tools, MCP servers and skills into a versioned configuration, with documentation that discusses CI jobs synchronizing checked-in definitions (currently a beta API). Google’s Vertex AI Agent Engine (now called Gemini Enterprise Agent Platform ) pairs an Agent Starter Pack with Cloud Build CI/CD and evaluation for deployed agents. Microsoft’s Copilot Studio runs an Agent Review Pipeline as an automated quality gate through Power Platform Pipelines, closer to enterprise ALM than casual prompt editing. AWS frames its equivalent as AgentOps through Bedrock AgentCore: deployment, governance, evaluation, observability, bundled together.

Different implementations. Same direction. The platforms expecting agents to run critical processes are building the boring machinery around them. Good.

The state of play, roughly, as of August 2026

Platform	Source control fit	Automated deployment	Lifecycle/ versioning	Automated testing & evaluation	Overall CI/CD fit
Anthropic Claude Managed Agents	Strong	Strong via API and CLI	Native versioned managed agents	Strong fit for eval pipelines	Strong (Managed Agents still beta)
Microsoft Foundry Hosted Agents	Strong	Strong, documented GitHub Actions flow	Hosted agent versions per deployment	Smoke testing and evaluation integrable	Very strong (some Hosted Agent tooling preview)
Microsoft Copilot Studio	Good via Power Platform ALM	Strong via Power Platform Pipelines	Solution and environment lifecycle	Automated Agent Review quality gates	Strong for enterprise low-code
Google Vertex AI Agent Engine	Strong for code-based agents	Strong via cloud CI/CD, Agent Starter Pack	Managed deployed resources	Vertex Gen AI Evaluation	Strong
AWS Bedrock AgentCore	Strong for code and config	Strong via standard AWS delivery, AgentCore CLI	Managed runtime, config as artefacts	AgentCore evaluation and observability	Strong
OpenAI Agents SDK / API stack	Strong	Strong via normal app CI/CD	Controlled through your own architecture	Code-first evaluation possible	Strong
OpenAI Custom GPTs	Limited	Editor-oriented, no documented Git pipeline	Version history in the GPT editor	Interactive testing, not a CI model	Limited
Gemini Gems	Limited	Gemini app workflow	Builder-oriented	Not equivalent to a CI pipeline	Limited

None of that makes Custom GPTs or Gems bad products. They solve a different problem. Building a personal assistant in fifteen minutes doesn’t need a pipeline first.

An autonomous agent touching a financial, operational, or compliance process in fifteen minutes is exactly what should worry you. The ability to build fast is not the same as the right to deploy carelessly.

The question enterprises are actually asking now

For a few years, the dominant question was capability: can GPT do this, can Claude do it, can Gemini handle the context? These questions are still relevant, but increasingly insufficient.

The models are capable enough that the operational questions now carry more weight. Can I version the agent? Reproduce the release? Test a change before it’s live? Compare a proposed version against the current one? Identify exactly which model and knowledge were involved when something went wrong? Control deployment? Monitor behavior? Roll back?

That’s the maturity shift to watch for in serious agent development. The terminology (AgentOps or whatever comes next) will keep changing. The questions won’t.

Not every agent needs a production department

This doesn’t mean every personal agent needs a release pipeline behind it. That would be absurd. Experimentation should stay fast. People should keep building prototypes without asking permission first. But an organization needs to know the moment an AI experiment becomes a (business-critical) system.

An agent drafting a readable email is not the same as one qualifying a bid. An assistant summarizing your meeting notes is not the same as one running a deal review. A prototype suggesting actions is not the same as one scoring risk and feeding that score straight into a decision.

Risk scales with capability, reach, and consequence. The engineering standard should scale with it too.

For anything critical: the production definition should be reproducible from controlled sources. Changes should be reviewed and traceable. Meaningful automated evaluation should run before release. Deployment should be repeatable. Monitoring should follow, with a defined way to replace a version that turns out to be worse.

That’s not excessive governance. It’s a fairly modest definition of being ready for production.

Why nobody wants to talk about this

Read enough about agentic AI, and you’ll find plenty on reasoning, autonomy, multi-agent orchestration, memory, RAG, MCP, context engineering. Genuinely interesting subjects.

CI/CD is not one of them. Source control is boring. Regression testing is boring. Documentation is boring. Release management is boring. Nobody has ever watched a conference audience gasp at a sensible rollback strategy.

But this is the machinery that separates a prototype from a system you can actually trust. The industry has spent enormous energy on what agents can do. The next phase needs to spend real time on what an agent has to demonstrate before it’s allowed to do those things in production.

That means defining quality standards, assigning ownership, deciding which test failures block a release, maintaining evaluation datasets, deciding who signs off on a change. And, occasionally, telling someone their very clever agent isn’t going live on Friday afternoon because nobody tested what happens when it fails. Less entertaining than the demo, but this is where professionalism actually starts.

More autonomy needs more discipline, not less

There’s a contradiction sitting in the middle of the current enthusiasm. We want agents with more responsibility: reasoning independently, using tools, coordinating with other agents, running complex tasks with less supervision. And we sometimes manage their definitions with less discipline than a basic internal app.

That’s backwards.

More autonomy makes traceability more important, not less. More powerful tools mean permissions need tighter control, not looser. Bigger business consequences mean evaluation has to get stronger. Less direct supervision means observability has to get better. Autonomy doesn’t reduce the need for engineering controls. It raises the bar.

Nothing here requires a breakthrough

None of this needs new technology. Git exists. Pull requests exist. GitHub Actions and Azure DevOps exist. Test automation exists. Infrastructure as code exists. Observability exists. Evaluation frameworks are maturing fast, and Foundry, Claude Managed Agents, Vertex AI Agent Engine, Bedrock AgentCore and Copilot Studio are already building pieces of the lifecycle around them.

We know how to engineer production systems. We do it every day, for everything that isn’t an AI agent.

The problem was never capability. It’s deciding that agents deserve the same discipline as the rest of the stack.

I don’t see a credible future where organizations run critical business processes through agents whose instructions remain manually editable, without source control, testing, documentation or controlled deployment. Sooner or later this becomes normal practice: versioned instructions, controlled model dependencies, versioned knowledge and tools, maintained evaluation suites, promoted releases across environments, monitored behavior, incidents investigated against an identifiable version.

We’ll eventually look back at agents edited live in a browser window the same way we now look back at administrators editing configuration directly on a production server.

AI changes who can build software, how fast, and what kind of problems it can solve. It lets application behavior live in natural language instead of a programming language.

It does not change your responsibility for what reaches production.

Your LLM is a platform. Your instructions are code. Your knowledge and tools are dependencies. Your evaluations are tests. Your agent is software.

Put it under source control. Test it. Document it. Deploy it properly.

The work is boring. That was never a reason to skip it. It’s usually the sign you’ve finally reached the part that matters.

In the next posts, I will write more about the in-depth technical challenges and solutions to make Continuous Delivery for AI software a reality. Please share your views, discoveries, opinions, and frustrations in the comments below.

(Images are of course generated with one of the AI models)