Cloud was supposed to do it. Kubernetes was supposed to do it. Serverless was supposed to do it. Platform engineering was supposed to do it. Now AI agents are the latest candidate.

The question sounds simple:

Will AI kill DevOps?

The better question is:

Which parts of DevOps become automated, and which parts become more important?

DevOps has always been about automation, feedback loops, reliable delivery, and reducing manual handoff between development and operations. AI agents are not the opposite of DevOps. They are a continuation of the same direction.

The difference is that AI can now reason across more context:

• source code
• pull requests
• pipeline logs
• cloud resources
• infrastructure state
• metrics
• traces
• incidents
• tickets
• vulnerabilities
• access requests
• runbooks
• architecture documentation

That creates real opportunities. It also creates new risks.

The future is not DevOps disappearing. The future is DevOps becoming more automated, more policy-driven, and more dependent on strong engineering judgement.

Why This Matters

Many DevOps teams still spend too much time on repetitive operational work:

• fixing broken pipelines
• checking logs manually
• updating tickets
• applying routine patches
• reviewing access requests
• collecting audit evidence
• investigating noisy alerts
• running the same operational checklist again and again

Google’s SRE guidance describes toil as repetitive, predictable work related to maintaining a service, and argues that reducing toil is central to operational efficiency.

This is where AI agents can help.

AI is good at reading context, summarising information, identifying patterns, generating draft changes, and calling tools through controlled interfaces. When connected to APIs, CI/CD systems, observability platforms, security scanners, ticketing systems, and infrastructure tools, AI can reduce a lot of operational friction.

But AI only works safely when the environment has:

• clear APIs
• reliable telemetry
• documented runbooks
• policy controls
• approval workflows
• audit logging
• ownership boundaries
• rollback procedures

Without these, AI automation can become another source of production risk.

Practical takeaway:
AI does not remove the need for DevOps maturity. It increases the value of DevOps maturity.

Core Concept: AI Does Not Replace DevOps, It Changes the Operating Model

DevOps is not only a collection of tools. It is a way of delivering and running software with speed, reliability, and accountability.

AI can automate parts of the toolchain, but it cannot remove the need for:

• ownership
• architecture decisions
• production accountability
• risk management
• security governance
• compliance evidence
• incident judgement
• platform design

The more realistic position is:

AI will not replace mature DevOps. It will expose immature DevOps.

Teams that depend on manual tickets, tribal knowledge, undocumented scripts, weak observability, and reactive firefighting will be vulnerable to disruption. Teams that already have strong CI/CD, infrastructure as code, observability, SRE practices, and security controls will be able to use AI safely.

What AI Actually Changes in DevOps

1. CI/CD: From Pipeline Scripts to Delivery Orchestration

CI/CD is one of the most obvious areas for AI-assisted DevOps.

Today, many teams still maintain complex YAML pipelines manually. Build failures are inspected by reading logs. Release notes are prepared manually. Deployment evidence is scattered across source control, CI/CD systems, ticketing tools, and chat messages.

AI agents can improve this workflow.

AI can help with CI/CD by:

• generating pipeline templates
• explaining failed builds
• summarising test failures
• identifying flaky tests
• suggesting pipeline fixes
• checking deployment readiness
• preparing release notes
• creating rollback recommendations
• collecting release evidence
• opening pull requests for pipeline improvements

MCP is relevant here because it provides a standard way for AI applications to integrate with external tools and data sources. The official MCP specification describes it as an open protocol for integrating LLM applications with external data sources and tools.

In a DevOps environment, MCP-style tools could expose controlled access to:

• GitHub or GitLab
• Jenkins
• Kubernetes
• Terraform Cloud
• cloud provider APIs
• Jira or ServiceNow
• observability platforms
• security scanners

However, CI/CD should not be fully replaced by AI agents.

CI/CD still needs deterministic and auditable controls:

• repeatable workflow execution
• automated tests
• approval gates
• artefact signing
• environment controls
• deployment history
• rollback logic
• segregation of duties
• audit trails

Practical takeaway:
AI should assist the delivery system. It should not become the delivery system.

2. Infrastructure as Code: AI Will Not Remove State

One tempting argument is that AI agents can scan cloud infrastructure through APIs, store the current status in memory, and remove the need for Terraform state.

That is not a safe conclusion.

Terraform state is not just a cache. HashiCorp explains that Terraform state is necessary because it maps real-world resources to Terraform configuration and helps Terraform understand what it manages.

Cloud API discovery can show what exists, but it cannot always explain:

• why a resource exists
• who owns it
• whether it is intentional
• which module created it
• whether it should be changed
• whether it is compliant
• whether it is manually created or managed by IaC
• what dependency relationship exists
• what the intended architecture should be

AI memory is also not a safe replacement for infrastructure state. It may lack:

• locking
• consistency
• versioning
• reconciliation
• drift tracking
• deterministic planning
• auditability
• rollback support

That does not mean AI has no role in IaC. It has a strong role, but not as a hidden state engine.

Better AI use cases for IaC

AI can help with:

• generating Terraform modules
• reviewing Terraform plans
• explaining risky infrastructure changes
• detecting drift
• comparing cloud inventory with IaC
• creating pull requests to fix drift
• documenting infrastructure
• identifying unused resources
• checking tagging standards
• estimating cost impact
• reviewing IAM, security groups, and network exposure

Note:
AI should improve IaC workflows, not bypass the source of truth.

3. Monitoring and SRE: From Reactive to Preventive Operations

Traditional operations often follow a reactive pattern:

1. Alert fires.
2. Engineer checks a dashboard.
3. Engineer searches logs.
4. Engineer checks recent deployments.
5. Engineer updates an incident ticket.
6. Engineer escalates to another team.
7. Root cause is found later.

AI can improve this pattern by correlating signals across systems.

AI can support SRE by:

• correlating metrics, logs, traces, events, and deployments
• detecting abnormal behaviour earlier
• identifying saturation trends
• highlighting likely root causes
• reducing alert noise
• suggesting runbook actions
• creating incident timelines
• drafting post-incident reviews
• recommending capacity changes
• identifying recurring failure patterns

This is where AIOps becomes relevant.

AIOps means using AI and analytics to improve IT operations. It is commonly applied to monitoring, event correlation, diagnosis, and operational workflow automation.

However, AI cannot compensate for poor observability. It needs good operational data.

AI-assisted SRE needs:

• useful metrics
• structured logs
• distributed traces
• service ownership
• dependency maps
• SLOs
• runbooks
• known failure modes
• deployment history
• incident history

Practical takeaway:
AI can help teams move from reactive firefighting to preventive operations, but only if the operational data is reliable.

4. Security: Agent-Assisted DevSecOps

Security is another strong area for AI-assisted DevOps.

Modern security work is fragmented across many tools and workflows:

• dependency scanners
• container image scanners
• secrets scanners
• IAM systems
• CI/CD platforms
• cloud security tools
• ticketing systems
• vulnerability databases
• compliance evidence repositories

AI agents can help connect these signals.

AI can assist DevSecOps by:

• checking vulnerability findings
• analysing dependency risk
• summarising CVE impact
• creating patch pull requests
• reviewing container image scan results
• checking IAM permissions
• detecting over-permissioned accounts
• identifying exposed services
• reviewing Kubernetes RBAC
• checking CI/CD pipeline risks
• preparing audit evidence
• tracking security exceptions

CI/CD security should be treated as a first-class concern. OWASP maintains a dedicated Top 10 list for CI/CD security risks, covering risks and recommended controls for modern delivery pipelines.

AI can make this better, but also more dangerous if permissions are poorly designed.

An AI agent should not automatically perform high-risk security actions without control, such as:

• granting admin access
• rotating production secrets
• changing firewall rules
• deleting accounts
• patching critical systems without testing
• approving security exceptions
• disabling controls

Practical takeaway:
Agent-assisted DevSecOps is valuable, but an over-permissioned AI agent becomes a new attack surface.

5. Chaos Engineering: AI Can Help, But Should Not Act Randomly

The correct term is chaos engineering, not “caros engineering.”

Chaos engineering is about testing system resilience by introducing controlled failure scenarios. AI can assist by identifying weak points and proposing experiments, but it should not randomly execute destructive tests.

AI can help with chaos engineering by:

• identifying single points of failure
• reviewing architecture diagrams
• proposing failure scenarios
• checking whether alerts exist
• checking whether rollback exists
• generating experiment plans
• summarising test results
• recommending resilience improvements

AI should not:

• run production failure tests without approval
• disable critical infrastructure randomly
• terminate resources without a defined blast radius
• test customer-facing systems without clear rollback
• bypass change management

Practical takeaway:
AI can design and analyse chaos experiments, but production execution must remain tightly controlled.

6. Support Operations: Agents as L1 and L2 Operators

AI-assisted operations should not be limited to infrastructure monitoring.

Support operations are a strong use case, especially for standardised workflows such as account maintenance, access requests, ticket routing, and operational checks.

Maintaining user accounts is usually part of:

• IT operations
• IT service management
• identity and access management
• access lifecycle management
• support operations

It becomes part of AIOps or AI-assisted operations when AI helps with decision-making, workflow automation, diagnosis, or ticket handling.

AI agents can help with:

• creating user accounts
• disabling leaver accounts
• processing access requests
• routing tickets
• validating approvals
• checking group membership
• identifying stale access
• updating Jira or ServiceNow tickets
• generating audit evidence
• answering standard support questions
• escalating unusual requests

Example: Safe AI-assisted access request

A controlled workflow could look like this:

1. User submits an access request.
2. Agent reads the ticket.
3. Agent identifies the requested system and access level.
4. Agent checks policy.
5. Agent checks manager or system owner approval.
6. Agent calls the IAM API only if policy allows.
7. Agent updates the ticket.
8. Agent writes an audit log.
9. Agent schedules access review or expiry.

This is very different from giving an AI agent unrestricted admin access.

Unsafe pattern

Avoid this model:

• AI has full admin rights.
• AI decides access without policy.
• AI grants production access without approval.
• AI deletes users without verification.
• AI makes changes without audit logs.

Practical takeaway:
AI can be a strong first-line operations assistant, but identity-related actions require least privilege, approval, and auditability.

7. The New Role of DevOps Engineers

AI changes the work profile of DevOps engineers.

The role becomes less about repetitive manual execution and more about designing safe automation systems.

Skills that become more important

DevOps engineers will need stronger skills in:

• platform engineering
• API integration
• MCP and tool design
• policy-as-code
• security automation
• identity governance
• observability engineering
• SRE practices
• AI agent guardrails
• workflow orchestration
• audit and compliance automation

The title may still be DevOps engineer, platform engineer, SRE, cloud engineer, or infrastructure engineer. The direction is similar: less manual operation, more system design.

8. What AI Should Not Own

AI agents should not independently control every operational task.

Some actions are too risky without human approval, strong policy, and rollback controls.

High-risk areas

AI should not independently perform:

• production database migration
• destructive infrastructure changes
• IAM privilege escalation
• firewall rule changes
• production secret rotation
• emergency rollback with customer impact
• deletion of cloud resources
• compliance exception approval
• financial or billing changes
• chaos experiments in production

Safer operating model

Use:

• read-only access by default
• scoped tool permissions
• approval gates
• policy-as-code
• change windows
• audit logs
• dry-run mode
• pull request workflow
• break-glass process
• human review for production changes

Note:
The safest first version of an AI DevOps agent is usually read-only: it observes, explains, summarises, recommends, and drafts changes.

Decision Framework: What Should AI Automate?

“The knowledge is compiled once and then kept current, not re-derived on every query.” [1]

The model operates as an autonomous compiler, extracting entities from new sources and persistently updating cross-references, contradictions, and topic summaries within a centralized repository [1].

1. System Layers and Directory Structure

The architecture enforces strict separation of concerns across three system layers [1]:

2. Execution Operations

The framework relies on three primary operational vectors [1]:

• Ingest: The LLM reads raw data, generates summary pages, updates index structures, and modifies existing domain entities.

• Query: The LLM reads the compiled index.md to locate relevant pages before synthesizing referenced answers.

• Lint: The LLM conducts autonomous health-checks to flag contradictions, locate orphaned pages, and update missing cross-references.

3. Schema Configuration: GEMINI.md

The Schema dictates the LLM’s operational logic and file system constraints. The following configuration establishes the system instructions for the compiler agent. It must be saved as GEMINI.md in the root directory.

# LLM Wiki Schema

A project for managing and generating a stateful, hierarchical knowledge base using Large Language Models (LLMs).

## Directory Overview

| Directory | Core Purpose | Target File Type |
| :--- | :--- | :--- |
| `raw/` | Storage for immutable raw source materials. | Unstructured text, datasets, documents. |
| `tools/` | Automation scripts, routing, sanitization, and transformation pipelines. | Source code. |
| `wiki/` | Destination for structured knowledge, subdivided by domain. | Markdown (`.md`). |

## Future Instructions for Gemini: Compiler Agent

When interacting with this project, you operate strictly as an objective, domain-aware LLM Wiki Compiler. To prevent prompt injection and unauthorized file modifications, you are prohibited from directly writing files. All state modifications must be executed via the `tools/` directory scripts.

### Operating Inputs

You will process contextual inputs during ingestion:
1. **MASTER INDEX**: The current contents of `wiki/index.md`.
2. **TARGET DOMAIN CONTEXT**: The concatenated contents of relevant `.md` files from `wiki/[domain]/`.
3. **NEW RAW DATA**: The text of a new document from the `raw/` directory.

### Operating Parameters

* **Domain Categorization**: Assign extracted entities to the most appropriate domain subdirectory. Each key concept must have its own individual file.
* **Data Extraction & Generation**: Extract verifiable facts and concepts. If knowledge is missing in the current wiki state, generate the required `wiki/` content.
* **Directory Integrity**: You possess zero direct write access to the file system. You must execute state changes exclusively through the provided routing scripts in the `tools/` directory.
* **State Preservation**: Merge new data into existing `wiki/[domain]/[Entity].md` structures. Preserve all previously verified facts.
* **Cross-Referencing**: Hyperlink entities across pages and domains using relative WikiLinks.
* **Contradiction Flagging**: If new raw data contradicts existing wiki state, append the exact string: `> **CONTRADICTION FLAG**: [Explanation]`.

### Output Formatting Constraints

You must output execution commands directed at the `tools/update_wiki` script. The output must strictly follow this YAML-in-delimiter format. Use the YAML block scalar indicator (`|`) for the markdown content. 

@@@execute tools/update_wiki
operations:
  - action: modify_or_create
    target_path: wiki/[domain]/[Entity].md
    index_summary: "[One-line summary for the domain index.md]"
    markdown_content: |
      [Full Markdown content incorporating extracted facts, WikiLinks, and code snippets]
@@@

4. Deployment via Gemini CLI

The @google/gemini-cli package enables direct terminal execution of the schema directives.

• • Dependency Installation:

    npm install -g @google/gemini-cli

    Agent Execution: Run the CLI in non-interactive mode. The local GEMINI.md schema acts as the default contextual grounding anchor.

    gemini -p "Ingest raw/network_logs_0419.txt. Update wiki if knowledge not found. Highlight different if discrepancy found."

References

• • • [1] A. Karpathy. “llm-wiki.md”. GitHub Gists. https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f

What is MCP?

Model Context Protocol (MCP) is an open-standard orchestration protocol (pioneered by Anthropic) that allows AI models to dynamically discover and navigate their environment.

• Mechanism: Uses JSON-RPC 2.0 over stateful transports like stdio or WebSockets/SSE.

• The “Pull” Model: Instead of hardcoding tools, the agent asks the MCP server: “What are your current capabilities?”

• Primitives: Exposes Tools (actions), Resources (data), and Prompts (templates).

What is OpenAPI?

OpenAPI (OAS) is the industry-standard “Structural Contract” for defining RESTful Web APIs.

• Mechanism: Stateless HTTP/REST focused on structured data exchange between software systems.

• The “Push” Model: To give an agent access, developers must “stuff” the OpenAPI definition into the system prompt or a RAG pipeline so the model “knows” the endpoints exist.

Architectural Comparison: The Pro/Con Matrix

Why Choose MCP over OpenAPI?

• Dynamic Discovery: Eliminate “Context Stuffing.” Agents discover tools at connection time, preventing “Specification Drift” where the model’s instructions don’t match the live API.

• Reduced Integration Complexity (M + N): In an OpenAPI world, M agents connecting to N APIs require M * N custom integrations.

  • MCP standardizes the “handshake,” reducing complexity to M + N.

• Semantic Density: MCP servers return summarized context optimized for LLM “eyes,” rather than raw, verbose JSON/XML blobs that waste tokens and processing power.

• Near-Zero Latency: Local agents (IDE extensions, CLIs) use stdio for instantaneous communication without network overhead.

Why Stick with OpenAPI?

• Production Maturity: Highly compatible with existing Load Balancers, Caching layers, and global CDNs.

• Security & Auth: Mature support for OAuth 2.0/OpenID Connect. MCP’s security is still evolving and currently faces higher risks of Prompt Injection (tricking the model into tool misuse).

• Interoperability: OpenAPI remains the universal language for non-AI microservices and traditional frontend apps.

The Architect’s Opinion: The “Semantic Gateway” Hybrid

Do not replace your Web APIs. Wrap them.

The most resilient 2026 architecture uses an MCP Server as a Gateway to existing REST/OpenAPI services.

Key Strategies for the Hybrid Model:

1. Intent-Based Mapping: Do not map 1 API to 1 Tool. Instead, bundle multiple endpoints into a High-Level Intent.

   • Example: An update-subscription MCP tool might orchestrate a GET /user, a PUT /billing, and a POST /notify behind the scenes.

2. Automated Bridging: Use tools like openapi-to-mcp to generate your base server, ensuring your OpenAPI spec remains the Single Source of Truth.

3. Hardened Security:

   • Scoped Authentication: Use the Gateway to exchange the agent’s identity for Short-lived, Down-scoped Tokens.

   • Semantic Guardrails: Implement a policy layer (like OPA or an LLM guardrail) to validate tool parameters before execution.

Use Case: The Evolution of “Action”

Final Verdict: Use OpenAPI for your data’s foundation and MCP as the intelligent interface that lets AI agents actually put that data to work.

Recommended Architect’s Toolkit

• mcp-server-openapi: A CLI to instantly turn your spec into an MCP server.

• OAuth 2.1: The 2026 requirement for secure, remote MCP-to-API communication.

Reference

• Model Context Protocol (MCP) Official Docs: “Introduction to MCP: The USB-C for AI Applications.” Anthropic / ModelContextProtocol.io (2026). https://modelcontextprotocol.io/docs/getting-started/intro

• OpenAPI Specification (v3.1.x): “The Structural Contract for RESTful Services.” OpenAPI Initiative. https://www.openapis.org/

• The $M+N$ Integration Problem: “Why Enterprises are Switching to MCP in 2026: Solving the $M \times N$ Complexity Gap.” Signity Solutions (March 2024/2026). https://www.signitysolutions.com/tech-insights/mcp-vs-traditional-api-integration

• Orchestration vs. Data Transport: “MCP vs. REST API: Comparing Warehouse Management to a Forklift.” Loginsoft (March 2026). https://www.loginsoft.com/post/mcp-vs-api-whats-the-actual-difference-and-when-to-use-each

• Performance Benchmarks: “Sub-second Response Times: When to stick with REST over MCP for High-Volume Data.” Ryze AI (April 2026). https://www.get-ryze.ai/blog/rest-api-vs-mcp-google-ads-claude

• The Semantic Gateway Pattern: “Generating MCP Tools from OpenAPI: Best Practices for AI-Native Descriptions.” Speakeasy (January 2026). https://www.speakeasy.com/mcp/tool-design/generate-mcp-tools-from-openapi

• OpenAPI Bridge (Rust Implementation): “Automated Transformation of OpenAPI Operations into MCP Tools.” MCP Market (March 2026). https://mcpmarket.com/server/openapi-bridge-2

• The 2026 Security Guide: “Hardening MCP Servers: OAuth 2.1, Scoped Tokens, and Guarding against Tool Poisoning.” Unicrew DevSecOps (April 2026). https://unicrew.com/blog/secure-mcp-server-guide/

• Defensive Architecture: “Zero Trust for AI Agents: Implementing Circuit Breakers in MCP Gateways.” InstaTunnel (March 2026). https://medium.com/@instatunnel/securing-mcp-servers-the-2026-guide-to-ai-tool-tunneling-aafa113b08db

Architect’s Note: For your WordPress post, I recommend hyperlinking the $M+N$ Complexity and Semantic Gateway sections specifically to the Loginsoft and Speakeasy articles, as they provide the strongest visual analogies for your readers.

1. The State Contract (Shared Memory)

In a multi-agent system, LLM API calls are inherently stateless. To maintain context across multiple agent interactions, we must establish a rigorous data contract. The AgentState class acts as the single source of truth, passed sequentially from node to node.

using LangChain.Providers;
using LangChain.Providers.OpenAI;
using System;
using System.Text.Encodings.Web;
using System.Text.Json;
using System.Text.Json.Serialization;
using System.Threading.Tasks;

namespace OllamaAgenticWorkflow
{
    // ==========================================
    // STATE CONTRACT
    // ==========================================
    public class AgentState
    {
        [JsonPropertyName("original_request")]
        public string OriginalRequest { get; set; } = string.Empty;

        [JsonPropertyName("orchestrator_route")]
        public string OrchestratorRoute { get; set; } = string.Empty;

        [JsonPropertyName("generated_content")]
        public string GeneratedContent { get; set; } = string.Empty;

        [JsonPropertyName("evaluator_feedback")]
        public string EvaluatorFeedback { get; set; } = string.Empty;

        [JsonPropertyName("is_approved")]
        public bool IsApproved { get; set; } = false;

        [JsonPropertyName("iteration_count")]
        public int IterationCount { get; set; } = 0;

        public string ToJson()
        {
            var options = new JsonSerializerOptions
            {
                WriteIndented = true,
                Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping // Ensures Unicode/Chinese chars aren't escaped
            };
            return JsonSerializer.Serialize(this, options);
        }
    }

2. The Agent Blueprint (Template Method Pattern)

An “agent” is essentially an LLM wrapped in state management and output parsing. The AgentNodeBase uses the Template Method Pattern to define the standard execution sequence. It handles the API call and robust JSON extraction, forcing concrete subclasses to strictly define their Persona (System Prompt) and State Mutations.

// ==========================================
    // ABSTRACT BASE NODE (TEMPLATE METHOD PATTERN)
    // ==========================================
    public abstract class AgentNodeBase
    {
        protected readonly IChatModel _chatModel;

        protected AgentNodeBase(IChatModel chatModel)
        {
            _chatModel = chatModel;
        }

        // Abstract methods that each specific agent must implement
        protected abstract string NodeName { get; }
        protected abstract string GetSystemPrompt();
        protected abstract string GetHumanPrompt(AgentState state);
        protected abstract void ParseAndUpdateState(AgentState state, JsonElement jsonRoot);

        // The Template Method that runs the shared execution logic
        public async Task<AgentState> ExecuteAsync(AgentState state)
        {
            string iterationLog = NodeName == "Generator" ? $" Attempt {state.IterationCount + 1}..." : "...";
            Console.WriteLine($"\n[{NodeName}]{iterationLog}");

            var systemPrompt = GetSystemPrompt();
            var humanPrompt = GetHumanPrompt(state);

            var response = await _chatModel.GenerateAsync(new[]
            {
                new Message(systemPrompt, MessageRole.System),
                new Message(humanPrompt, MessageRole.Human)
            });

            var cleanJson = ExtractJsonFromLlmResponse(response.LastMessageContent ?? string.Empty);
            
            // Failsafe for models that completely fail to return JSON
            if (string.IsNullOrWhiteSpace(cleanJson) || (!cleanJson.StartsWith("{") && !cleanJson.StartsWith("[")))
            {
                throw new FormatException($"The LLM did not return valid JSON. Raw output: {response.LastMessageContent}");
            }

            using var jsonDoc = JsonDocument.Parse(cleanJson);
            ParseAndUpdateState(state, jsonDoc.RootElement);

            return state;
        }

        // Shared utility to strip markdown backticks from LLM responses
        private string ExtractJsonFromLlmResponse(string text)
        {
            var start = text.IndexOf('{');
            var end = text.LastIndexOf('}');
            if (start != -1 && end != -1 && end > start)
            {
                return text.Substring(start, end - start + 1);
            }
            return text; 
        }
    }

3. The Worker Nodes (Generator, Evaluator, Translator)

These concrete classes inherit the blueprint.

The Generator-Evaluator Loop is visible here. The Generator produces content. If the Evaluator rejects it and provides feedback, the Orchestrator will route it back to the Generator. Notice how the GeneratorNode specifically injects state.EvaluatorFeedback into its next prompt to force a revision, and then explicitly wipes the feedback state so the process can restart cleanly.

// ==========================================
    // CONCRETE AGENT NODES
    // ==========================================
    public class GeneratorNode : AgentNodeBase
    {
        public GeneratorNode(IChatModel chatModel) : base(chatModel) { }

        protected override string NodeName => "Generator";

        protected override string GetSystemPrompt() => @"
            You are a Generator Agent. Execute the task provided by the user.
            
            Respond strictly in valid JSON format:
            { ""generated_result"": ""your content here"" }";

        protected override string GetHumanPrompt(AgentState state)
        {
            // Grab the feedback BEFORE we wipe it out
            var feedbackContext = !string.IsNullOrWhiteSpace(state.EvaluatorFeedback)
                ? $"\nPREVIOUS FEEDBACK TO FIX: {state.EvaluatorFeedback}"
                : "";

            return $"Task: {state.OriginalRequest}{feedbackContext}";
        }

        protected override void ParseAndUpdateState(AgentState state, JsonElement jsonRoot)
        {
            state.GeneratedContent = jsonRoot.GetProperty("generated_result").GetString() ?? "";
            state.IterationCount++;

            state.EvaluatorFeedback = "";
            state.IsApproved = false;

            Console.WriteLine($"[{NodeName} Content]:\n{state.GeneratedContent}");
        }
    }

    public class EvaluatorNode : AgentNodeBase
    {
        public EvaluatorNode(IChatModel chatModel) : base(chatModel) { }

        protected override string NodeName => "Evaluator";

        protected override string GetSystemPrompt() => @"
            You are an Evaluator Agent. 
            Review the original task and the generated content. Does the content satisfy the task perfectly? 
            
            Respond strictly in valid JSON format using a boolean:
            { ""is_approved"": true, ""feedback"": ""reasoning"" }";

        protected override string GetHumanPrompt(AgentState state) => $@"
            Original Task: {state.OriginalRequest}
            Generated Content: {state.GeneratedContent}";

        protected override void ParseAndUpdateState(AgentState state, JsonElement jsonRoot)
        {
            state.IsApproved = jsonRoot.GetProperty("is_approved").GetBoolean();
            state.EvaluatorFeedback = jsonRoot.GetProperty("feedback").GetString() ?? "";
            Console.WriteLine($"[{NodeName} Approved?]: {state.IsApproved}");
        }
    }

    public class TranslatorNode : AgentNodeBase
    {
        public TranslatorNode(IChatModel chatModel) : base(chatModel) { }

        protected override string NodeName => "Translator";

        protected override string GetSystemPrompt() => @"
            You are a Translator Agent. Translate the provided text accurately.
            
            Respond strictly in valid JSON format. Provide the translated text:
            { ""generated_result"": ""translated text here"" }";

        protected override string GetHumanPrompt(AgentState state) => 
            $"Task: {state.OriginalRequest}";

        protected override void ParseAndUpdateState(AgentState state, JsonElement jsonRoot)
        {
            state.GeneratedContent = jsonRoot.GetProperty("generated_result").GetString() ?? "";
            state.IsApproved = true; // Auto-approve translations
            Console.WriteLine($"[{NodeName} Output]:\n{state.GeneratedContent}");
        }
    }

4. The Hub Router (Orchestrator Node)

The Hub-and-Spoke Pattern relies on a central router. The OrchestratorNode acts as this hub. It evaluates the current AgentState variables to logically deduce the workflow status (NEEDS_GENERATION, NEEDS_EVALUATION, NEEDS_REVISION, or APPROVED). It then tells the execution runner which “spoke” to trigger next.

public class OrchestratorNode : AgentNodeBase
    {
        public OrchestratorNode(IChatModel chatModel) : base(chatModel) { }

        protected override string NodeName => "Orchestrator";

        protected override string GetSystemPrompt() => @"
            You are the Lead Orchestrator Agent. Your job is to route the workflow based strictly on the 'Workflow Status' provided.
            
            Routing Rules:
            - If Workflow Status is 'NEEDS_GENERATION', route to: 'generator'
            - If Workflow Status is 'NEEDS_EVALUATION', route to: 'evaluator'
            - If Workflow Status is 'NEEDS_REVISION', route to: 'generator'
            - If Workflow Status is 'APPROVED', route to: 'finish'
            - If the user explicitly asks for a language translation and the status is NEEDS_GENERATION, route to: 'translator'
            
            Respond strictly in valid JSON format:
            { ""route_to"": ""chosen_route"", ""reasoning"": ""brief explanation of why"" }";

        protected override string GetHumanPrompt(AgentState state)
        {
            // Calculate a strict, foolproof status string for the LLM
            string workflowStatus = "NEEDS_GENERATION";

            if (state.IsApproved)
            {
                workflowStatus = "APPROVED";
            }
            else if (!string.IsNullOrWhiteSpace(state.GeneratedContent))
            {
                if (string.IsNullOrWhiteSpace(state.EvaluatorFeedback))
                {
                    // Content exists, but no feedback yet -> Must Evaluate
                    workflowStatus = "NEEDS_EVALUATION";
                }
                else
                {
                    // Content exists, and we have feedback -> Must Revise
                    workflowStatus = "NEEDS_REVISION";
                }
            }

            string genContent = string.IsNullOrWhiteSpace(state.GeneratedContent) ? "None" : state.GeneratedContent;
            string evalFeedback = string.IsNullOrWhiteSpace(state.EvaluatorFeedback) ? "None" : state.EvaluatorFeedback;

            return $@"
            Workflow Status: {workflowStatus}
            Original Request: {state.OriginalRequest}
            Generated Content: {genContent}
            Evaluator Feedback: {evalFeedback}";
        }

        protected override void ParseAndUpdateState(AgentState state, JsonElement jsonRoot)
        {
            state.OrchestratorRoute = jsonRoot.GetProperty("route_to").GetString()?.ToLower() ?? "unknown";
            string reasoning = jsonRoot.GetProperty("reasoning").GetString() ?? "";

            Console.WriteLine($"[{NodeName} Decision]: Route to '{state.OrchestratorRoute}'");
            Console.WriteLine($"[{NodeName} Reasoning]: {reasoning}");
        }
    }

5. The State Machine Runner

The AgentGraphRunner executes the actual Hub-and-Spoke loop. It ensures that every single step passes through the Orchestrator first. Crucially, it implements a maxLoops failsafe to prevent the agents from getting stuck in an infinite loop of generation and rejection.

// ==========================================
    // WORKFLOW RUNNER (HUB-AND-SPOKE STATE MACHINE)
    // ==========================================
    public class AgentGraphRunner
    {
        private readonly OrchestratorNode _orchestrator;
        private readonly TranslatorNode _translator;
        private readonly GeneratorNode _generator;
        private readonly EvaluatorNode _evaluator;

        public AgentGraphRunner(IChatModel chatModel)
        {
            _orchestrator = new OrchestratorNode(chatModel);
            _translator = new TranslatorNode(chatModel);
            _generator = new GeneratorNode(chatModel);
            _evaluator = new EvaluatorNode(chatModel);
        }

        public async Task RunWorkflowAsync(string userRequest)
        {
            var state = new AgentState { OriginalRequest = userRequest };

            int maxLoops = 10; // Failsafe to prevent endless LLM arguments
            int currentLoop = 0;

            Console.WriteLine("\n=== Starting Hub-and-Spoke Agentic Workflow ===");

            while (currentLoop < maxLoops)
            {
                // 1. The Orchestrator acts as the central router for EVERY step
                state = await _orchestrator.ExecuteAsync(state);

                // 2. Execute the route determined by the Orchestrator
                if (state.OrchestratorRoute == "finish")
                {
                    Console.WriteLine("\n[System] Orchestrator determined the task is complete.");
                    break;
                }
                else if (state.OrchestratorRoute == "translator")
                {
                    state = await _translator.ExecuteAsync(state);
                }
                else if (state.OrchestratorRoute == "generator")
                {
                    state = await _generator.ExecuteAsync(state);
                }
                else if (state.OrchestratorRoute == "evaluator")
                {
                    state = await _evaluator.ExecuteAsync(state);
                }
                else
                {
                    Console.WriteLine($"\n[System] Unknown route '{state.OrchestratorRoute}'. Ending workflow.");
                    break;
                }

                currentLoop++;
            }

            if (currentLoop >= maxLoops)
            {
                Console.WriteLine($"\n[System] Workflow forced to stop after {maxLoops} iterations.");
            }

            Console.WriteLine("\n=== Workflow Complete ===");
            Console.WriteLine(state.ToJson());
        }
    }

6. Application Initialization

Finally, the Main execution initializes the model provider and launches the interactive loop.

Critical Architecture Note: The current implementation utilizes an identical local model (gemma3:4b) for both generation and evaluation. Operating the same model for generation and validation introduces structural bias and validation echo chambers. In a production environment, decoupling model providers is mandatory. The EvaluatorNode should ideally be initialized with a secondary, more advanced model architecture to ensure objective review and mathematical correctness.

class Program
    {
        static async Task Main(string[] args)
        {
            // 1. Force UTF-8 Encoding for Traditional Chinese / Unicode support
            Console.OutputEncoding = System.Text.Encoding.UTF8;
            Console.InputEncoding = System.Text.Encoding.UTF8;

            // 2. Initialize Provider and Model (OpenAI compatibility mode for Ollama)
            // Note: Endpoints must be secured/masked in production.
            var provider = new OpenAiProvider(
                apiKey: "ollama",
                customEndpoint: "http://[REDACTED_INTERNAL_IP]:11434/v1" 
            );
            
            // To prevent structural bias, consider instantiating a second model 
            // (e.g., GPT-4o or Claude 3.5 Sonnet) specifically for the Evaluator node.
            var chatModel = new OpenAiChatModel(provider, id: "gemma3:4b");

            // 3. Instantiate the Runner
            var runner = new AgentGraphRunner(chatModel);

            Console.WriteLine("=================================================");
            Console.WriteLine("🤖 OOP Multi-Agent Workflow Initialized");
            Console.WriteLine("Type 'exit' to quit.");
            Console.WriteLine("=================================================\n");

            // 4. Interactive User Loop
            while (true)
            {
                Console.Write("👤 You: ");
                string? userRequest = Console.ReadLine();

                if (string.IsNullOrWhiteSpace(userRequest)) continue;
                if (userRequest.Trim().ToLower() is "exit" or "quit") break;

                try
                {
                    await runner.RunWorkflowAsync(userRequest);
                }
                catch (Exception ex)
                {
                    Console.WriteLine($"\n[Error] The workflow encountered an issue: {ex.Message}");
                }

                Console.WriteLine("\n------------------------------------------\n");
            }
        }
    }
}

Test Result

References

1. Ratnesh Yadav. (2025). Bot-to-Bot: Centralized (Hub-and-Spoke) Multi-Agent Topology. Medium. https://medium.com/@ratneshyadav_26063/bot-to-bot-centralized-hub-and-spoke-multi-agent-topology-part-2-87b46ec7e1bc

2. Yuval Mehta. (2026). How Multi-Agent Self-Verification Actually Works. Towards AI. https://pub.towardsai.net/how-multi-agent-self-verification-actually-works-and-why-it-changes-everything-for-production-ai-71923df63d01

3. arXiv preprint. (2025). MAR: Multi-Agent Reflexion Improves Reasoning Abilities in LLMs. arXiv. https://arxiv.org/html/2512.20845v1

1. Provider Configuration

To interact with Vault and generate secure tokens, the environment must specify the required providers.

terraform {
  required_providers {
    vault = {
      source  = "hashicorp/vault"
    }
    random = {
      source = "hashicorp/random"
    }
  }
}

2. Generating the Secret Data

In this implementation, a token is required to bootstrap a K3s cluster. The random provider is utilized to generate a secure, 32-character string.

resource "random_password" "k3s_cluster_token" {
  length  = 32
  special = false
}

3. Storing Sensitive Data in Vault (Set)

The generated token is injected into Vault using the vault_kv_secret_v2 resource. This operation assumes the target Vault mount path is configured for KV Version 2.

• Mount Point: home-lab-secrets

• Secret Path: k3s-cluster-token

• Payload Format: JSON encoded key-value pair.

  resource "vault_kv_secret_v2" "k3s_cluster_token" {
    mount     = "home-lab-secrets"
    name      = "k3s-cluster-token"
    data_json = jsonencode({
      token = random_password.k3s_cluster_token.result
    })
  }

4. Retrieving Sensitive Data from Vault (Get)

To utilize the stored secret elsewhere in the infrastructure code, a Terraform data source fetches the values from Vault. The depends_on meta-argument ensures the secret is fully provisioned before retrieval is attempted.

data "vault_kv_secret_v2" "k3s_cluster_token" {
  depends_on = [vault_kv_secret_v2.k3s_cluster_token]
  mount      = "home-lab-secrets"
  name       = "k3s-cluster-token"
}

5. Usage Example: Bootstrapping a Node

Once retrieved, the secret data can be accessed via data.vault_kv_secret_v2.<name>.data.<key>. In the provided example, the token is passed as an environment variable (K3S_TOKEN) over an SSH connection using a remote-exec provisioner to join a node to the existing K3s cluster.

provisioner "remote-exec" {
    inline = [
      "cloud-init status --wait",
      "curl -sfL https://get.k3s.io | K3S_URL='https://${local.master_node.network.ip}:6443' K3S_TOKEN='${data.vault_kv_secret_v2.k3s_cluster_token.data.token}' sh -"
    ]
  }

Lesson learnt from previous work, I will conclude them into steps and hope it can help others.Prerequisites

1. Ensure all Proxmox nodes are same version.
   It can done by run commands in each node below:

   ## Update package list.
   sudo apt update
   
   ## Ensure all proxmox node in same version.
   sudo apt dist-upgrade -y 
   
   ## Update package.
   sudo apt upgrade
   
   ## Clean up
   sudo apt clean

2. Ensure all node connect to network;
3. Ensure all node can resolve each other’s DNS record;

In there, it will use 2 nodes (nodeA, nodeB) in command as sample/

Steps:

1. Create cluster.
   In nodeA, use command below to create cluster.

   pvecm create demo-cluster

2. Grand cluster firewall port.
   In nodeA, open /etc/pve/firewall/cluster.fw , add rules inside section [RULES], sample as below:

   [RULES]
   
   IN SSH(ACCEPT) -i vmbr0 -log nolog # SSH between nodes
   IN NTP(ACCEPT) -i vmbr0 -log nolog # NTP sync time
   IN DNS(ACCEPT) -i vmbr0 -log nolog # DNS resolve
   IN ACCEPT -i vmbr0 -p tcp -dport 5403 -sport 5403 -log nolog # Proxmox VE Cluster Daemon
   IN ACCEPT -i vmbr0 -p tcp -dport 60050 -sport 60000 -log nolog # Proxmox VE backup and migration traffic
   IN ACCEPT -i vmbr0 -p udp -dport 5405 -sport 5404 -log nolog # Proxmox corosync
   IN ACCEPT -i vmbr0 -p tcp -dport 3128 -sport 3128 -log nolog # Proxmox VE HA (High Availability) manager
   IN ACCEPT -i vmbr0 -p tcp -dport 5999 -sport 5900 -log nolog # SPICE console connections.
   IN ACCEPT -i vmbr0 -p tcp -dport 8009 -sport 8007 -log nolog # Proxmox VE cluster traffic
   IN ACCEPT -i vmbr0 -p tcp -dport 8006 -sport 8006 -log nolog # Proxmox VM

3. Join node.
   In nodeB, use command below to join cluster.

   pvecm add <node1 IP address>

4. Verify result
   In nodeB, run command below, if it can found nodeB IP address, means node join successfully.

   pvecm status

   

It cause by dangling commit and dangling blob in repository.

Dangling commit:  Commit that isn’t directly linked to by any child commit, branch, tag or other reference

Dangling blob: A change that made it to the staging area/index, but never got committed.

These dragging commit and blob can found in command below:

git fsck --name-objects

Output:

Run command below to clean danging commit / blob and clean up repository

cd $REPO_PATH 
## Remove all danging commit and blob and GC collect. 
git reflog expire --expire=now --all 
git gc --prune=now 

## Clean up local old / conflict branch 
git remote prune origin

We can trigger the checking in code. And it is recommend for API response after collect API response.

Code to validate response in code level, it can be move to abstract class or interface and call it when needed.

import javax.validation.ConstraintViolation;

ApiResponse response = ApiClient.get();
Validator validator = Validation.buildDefaultValidatorFactory().getValidator();
Set<ConstraintViolation<ApiResponse>> result = validator.validate(response);
if(!result.isEmpty()) {
  String message = result.stream()
  .map(o-> new StringBuilder()
  .append(o.getPropertyPath().toString())
  .append(o.getMessage()))
  .collect(Collectors.joining(";\r\n"));
  throw new IllegalArgumentException(message);
}

However, snippet is the two-side blade for developer. It will introduce duplicate code and result difficult to maintain if abuse. The recommend place to use snippets are:

1. Unit test for high level format;
2. DTO / DAO with specific handling in getter / setter method;
3. Predefined markup (e.g. XML / HTML, JSON, YAML);

In this demo, it will use JUnit parameter test as sample to create snippet.Steps:

1. Create Live Template
   Highlight code would like to add into snippet, press Control + Shift + A, and input save as live template, then click Enter.
2. Add snippet abbreviation and variables.
   In Save as Live Template dialog, in Abbreviation, input name ptest, and variables are format like $variableName$.
   
3. Configure snippet variables.
   Click Edit Variables, in expression, input default value. You can use method provided to customize value (e.g. upper case, camelCase, etc) to fit your need.
   
4. Test snippet.
   In project, type snippet abbreviation, expected it will show in intelli-sense, after press enter, snippet will be render.

In this example, it will use Gradle as example to show how to get package version and call it in in other steps.

Steps

1. Update GitHub Action.
   Add steps below in GiutHub action.

   - name: Read package version
     id: get-version
     run: |
       version=$(grep -oP "version\s*=\s*'[^']+'" build.gradle | grep -oP "[^']*" | tr -d '\n' | tr -d ' ' | tr -d 'version=')
       echo "::set-output name=version::${version}"
   
   - name: Print package version
     run: |
       echo "Testing current version."
       echo "Package version is ${{ steps.get-version.outputs.version }}"

2. Check result
   Execute the pipeline to check result, expect version in build.gradle mentioned will shown.