All Insights
#ArtificialIntelligence
August 2026

AI and IT Departments, Part 9: Infrastructure Operations Is Changing - When the Monitoring System Can Fix What It Finds

This week it is back my roots - Infrastructure and Operations and also because it may be one of the clearest examples of where the move from AI assistance to AI action becomes very real. As I think back, the operating model has long been familiar - monitoring tools detect something, generate an alert and give an engineer information to start troubleshooting and fixing. Automation may then execute a script or runbook, but the reasoning in the middle, working out what happened and what should be done about it, has largely remained human.

By Steve Harris

This week it’s back my roots - Infrastructure and Operations and also because it may be one of the clearest examples of where the move from AI assistance to AI action becomes very real.

As I think back, the operating model has long been familiar - monitoring tools detect something, generate an alert and give an engineer information to start troubleshooting and fixing. Automation may then execute a script or runbook, but the reasoning in the middle, working out what happened and what should be done about it, has largely remained human.

That boundary is starting to change (or at least the emerging capabilities provide a way for it to change). The interesting question is no longer just whether AI can explain an alert. It is whether it can investigate the problem, decide what should happen next, invoke the appropriate tools and, under the right controls, fix what it finds.

What is it?

The I&O market is already well beyond simply putting a chatbot in front of monitoring data. Across observability, AIOps, incident response, network operations and infrastructure automation, there is a fairly consistent progression emerging - Dynatrace, Datadog, Splunk, New Relic, ServiceNow, PagerDuty, Hewlett Packard Enterprise OpsRamp and others are increasingly using AI to correlate operational data, investigate incidents and recommend actions. In some products, agents can perform multi-step investigations or invoke operational workflows.

This maps quite neatly to the Assist → Augment → Advise → Act → Automate maturity path I use (https://ai4it.ca) throughout this series and my broader work. The important shift in I&O happens as we move from systems that help an engineer understand something to systems that participate in the operational process itself. The business outcome also changes, from individual productivity at the early stages toward shorter response times, workflow execution and eventually automation of routine work.

In infrastructure operations the maturity path might look something like this:

  • Assist: “What does this Kubernetes error mean?”
  • Augment: “Summarise these alerts and show me the services affected.”
  • Advise: “Investigate this incident using logs, metrics, configuration, recent changes and previous incidents, then recommend a resolution.”
  • Act: “Investigate the incident, choose the approved remediation workflow and ask me for approval before executing it.”
  • Automate: “Detect this known condition, investigate it, execute the permitted remediation, verify recovery and escalate only if the expected result is not achieved.”

That final step is quite different from the traditional monitoring model, and there is an important architectural point here. The better implementations do not appear to be heading toward giving an LLM unrestricted administrator credentials and asking it to fix things. The pattern emerging from the research is more interesting:

Telemetry → trusted operational context → AI reasoning → policy/authorization → deterministic automation → validation

In other words, the AI may decide what should happen, while established automation determines exactly how it happens. Red Hat Ansible, Terraform, Rundeck, Itential and other deterministic automation platforms do not become obsolete because agents arrive. In some ways they become more important because they provide repeatability, permissions, parameter validation, predictable execution and an audit boundary underneath the probabilistic reasoning layer.

One of the more interesting examples is the work John Capobianco is doing around NetClaw and agentic network operations. Traditional network automation mostly automates execution. An engineer diagnoses the problem, determines the required change and then uses a script, template or automation platform to execute it. NetClaw tries to move the automation one level higher and capture some of the reasoning used by an experienced network engineer.

That makes the flow more like:

Problem → reason → gather evidence → reason again → select an action → execute through a governed workflow → validate

(Nice work John Capobianco).

What does it mean from a business perspective?

The obvious opportunity is faster incident resolution, but I think the change is broader than MTTR. Infrastructure teams spend a significant amount of time gathering evidence - an alert arrives, somebody checks logs, another system provides topology, somebody looks for recent changes, the CMDB is consulted, a runbook is found and perhaps an SME is pulled into the incident.

Observability and Action Enhancements

An agent can potentially do much of that investigation concurrently and consistently. This is why observability looks like such a strong early use case. Giving an agent read access to telemetry is considerably less risky than giving it permission to change production. It can still reduce the human effort required to get from “something is wrong” to “this appears to be the cause”.

The more consequential change comes when investigation connects to action. Consider a known failure in a non-critical stateless service. Today the process might be:

alert → engineer investigates → engineer identifies known condition → engineer executes runbook → engineer checks service A more mature agentic implementation could become:

alert → agent investigates → known condition confirmed → approved runbook executed → telemetry confirms recovery → incident updated The human may never need to enter the loop - although that’s a really mature state and still requires a risk based approach. compare a simple restart of a edge device to modifying core network routing or an enterprise IAM policy and the answer should be very different. This is why I don’t think a generic “human in the loop” is specific enough as a control model. What matters is what authority has actually been delegated.

Risk Based Approach

An agent might be allowed to observe infrastructure, perform diagnostic tests and prepare a proposed change automatically. It might have permission to execute a low-risk remediation against one server. The same agent should not necessarily have permission to propagate a routing change across hundreds of locations.

The useful variables become things such as consequence, blast radius, reversibility and ability to verify the outcome.

This also creates an interesting relationship with ITIL 4.

Monitoring and Event Management begins to connect much more directly with Incident Management and Problem Management because the technology can participate in the investigation rather than merely raise the event. Change Enablement becomes particularly important once an agent can propose or initiate production changes. Configuration Management matters because the quality of the agent’s reasoning depends heavily on the quality of the topology, dependency and configuration information it receives - bad operational data will not be fixed by a better model.

From a COBIT 2019 perspective, the issue is even less about “AI governance” in the abstract and more about delegated machine authority. Who has authorized the agent to act? Under which identity? Against which systems? What actions are permitted? What evidence caused the decision? Was approval required? What actually executed? Was the result independently verified?

Those are governance questions - AI simply makes them more urgent because the gap between a decision and its execution can become very small and the blast radius can be huge and beyond human speed to respond.

The Future

Agent-to-agent interoperability may eventually make these ecosystems much easier to assemble. MCP is already becoming important as a common way of exposing operational context and tools to agents. The broader multi-agent interoperability story is still less mature. Either way, interoperability increases the importance of identity and authorization rather than reducing it - the operating model changes as well.

The NOC engineer increasingly becomes less of a person moving manually between screens and more of a person supervising investigations (a mirror to how Agentic SWE is progressing), handling exceptions, improving automation, encoding operational knowledge and deciding where additional authority can safely be delegated. Although we are some distance from it being universal.

What do I do with it?

The practical starting point is not autonomous infrastructure but by taking stock of how much of the investigation can safely be delegated.

  • Begin read-only. Connect AI to observability, operational knowledge and reliable infrastructure context. Let it correlate evidence, perform investigations and suggest likely causes without production write access. This is a useful test of whether the reasoning is actually good enough before the question of autonomy even arises.
  • Separate reasoning from execution. A common theme - keep probabilistic AI on one side of the boundary and deterministic automation on the other. Let the agent determine which approved action is appropriate, but use established runbooks, workflows, infrastructure-as-code or controllers to perform it.
  • Define levels of authority. Don’t create one generic “agent access” role. The machine identity should only possess the authority required for that level of operation.
  • Use risk and reversibility to decide where humans stay involved. Restarting one approved non-critical service is very different from changing a firewall policy or the core network and the approval model should reflect that.
  • Make verification part of the workflow. “The API call succeeded” is not the same as “the service recovered.” The agent should check the expected post-condition using telemetry or an independent source of operational state (just like a human engineer). Unexpected results should stop the process, trigger rollback where possible and escalate.
  • Test failure as seriously as success. Test stale configuration data, denied permissions, unavailable tools, contradictory evidence, failed automation and unexpected post-change state. A demonstration that only proves the happy path tells you very little about production readiness.
  • Keep the existing service-management controls. Incident records, change authorization, configuration information and audit trails are not bureaucracy that agents allow us to remove. They increasingly become part of the control system within which the agents operate.

And I would increase autonomy gradually:

Read-only AI → agentic investigation → proposed action → human-approved execution → policy-approved low-risk execution → narrowly scoped closed-loop remediation

That feels way more realistic than jumping from a monitoring copilot to an autonomous NOC. The point is not that infrastructure operations is about to become completely autonomous - just like a lot of the predictions around AI, the capability may be being developed or nearly there - adoption is a completely different story.

Having said that the technical boundary is moving. We used to automate what happened after an engineer had worked out what to do (if I had to do things more than twice as a sysadmin, it was getting automated). Agentic systems are starting to participate in working out what to do in the first place, and once the monitoring system can investigate what it finds, select an action and initiate a controlled remediation, this stops being a feature upgrade to an operations tool - it becomes a change to how Infrastructure and Operations works.

Want to Discuss This Topic?

Steve is always happy to have a direct conversation.