Files
hermes-skills/devops/multi-agent-team-architecture/SKILL.md
T

9.2 KiB

name, description, tags
name description tags
multi-agent-team-architecture Designing and implementing a multi-agent team using Hermes profiles, SOUL.md templates, and CEO-level orchestration.
multi-agent
profiles
architecture

Multi-Agent Team Architecture

Trigger Conditions

  • User wants to set up a virtual team or multiple specialized agents.
  • User asks to separate tasks by domain (e.g., SRE vs. Nutrition).
  • User wants to scale the AI team with new "hires" (profiles).

Core Concept: The "Schön Consulting" Model

The team is organized as a CEO (Orchestrator) + Specialized Agents (Profiles) structure.

  • CEO (You/Standard Profile): Strategic oversight, task delegation, cross-domain coordination, and user interaction.
  • Agents (Hermes Profiles): Fully isolated environments (SOUL.md, Memory, Skills, Cron) for specific domains (e.g., infra-sre, nutrition-coach).

Step 1: Profile Creation

Every team member is a distinct Hermes profile. This ensures no "context bleed" between domains.

# Create a blank profile (fresh state)
hermes profile create <agent-name>

# Or clone existing config/SOUL to start quickly
hermes profile create <agent-name> --clone

Step 2: The SOUL.md Template

Every agent requires a structured SOUL.md to define their "identity." Use this template:

# <Agent-Name> (Role Title)

## Role
[Brief professional description of the agent's purpose and expertise.]

## Persona & Tone
- **Style:** (e.g., Direct, Empathetic, Technical, Pragmatic)
- **Key Trait:** (e.g., Security-first, Family-oriented, Budget-conscious)

## Responsibilities
1. [Primary Task 1]
2. [Primary Task 2]
3. [Primary Task 3]

## Tools & Workflow
- [Specific tools, scripts, or skills this agent should use]
- [How it handles data/results]

## Escalation Rules (An CEO)
- **CRITICAL:** [Condition for immediate, high-priority alert]
- **WARNING:** [Condition for next-report inclusion]
- **INFO:** [Condition for routine logging]

## Communication Style
"[Example of how the agent addresses the user/CEO and reports status]"

Step 3: Model & Provider Alignment Across Profiles

When a user says "set model X for all agents", update three keys per profile — not just model.default. Hermes checks them in a priority order, and leftover stale values cause silent fallbacks:

hermes config set --profile <agent> model.provider <PROVIDER>
hermes config set --profile <agent> model.default    <MODEL>
hermes config set --profile <agent> model.model      <MODEL>   # explicit mirror

Verification:

hermes profile list

shows the model string column; if it disagrees with what you expect, the wrong key is carrying it.

Auxiliary tasks stale-endpoint fix

After a provider migration, agent profiles may carry stale auxiliary.* overrides pointing to a dead custom endpoint (e.g. base_url: http://10.0.30.99:8080/v1). Symptoms: Auxiliary title generation failed: HTTP 400: network error occurred while connecting to provider API.

Bulk reset all auxiliary slots across all profiles to auto (falls back to main model):

import subprocess

profiles = ['default', 'infra-sre', 'nutrition-coach']  # adapt to your stack
slots = [
    'vision', 'web_extract', 'compression', 'session_search',
    'skills_hub', 'approval', 'mcp', 'title_generation', 'flush_memories',
]
for p in profiles:
    for s in slots:
        for k, v in [('provider', 'auto'), ('model', ''), ('base_url', ''), ('api_key', '')]:
            subprocess.run(
                ['hermes', 'config', 'set', '--profile', p, f'auxiliary.{s}.{k}', v],
                capture_output=True)

Provider Config Inheritance (Critical — Before Gateway Start)

Agent profiles do NOT automatically inherit providers.* blocks from the default profile.

If your default profile defines a custom provider (e.g., providers.noris with base_url and api_key), every agent profile that uses this provider MUST explicitly copy that block. Otherwise you get:

⚠️ Provider authentication failed: Unknown provider 'noris'.

Fix: Copy the provider definition into the agent's config.yaml:

# 1. Read provider from default
grep -A4 'providers:' ~/.hermes/config.yaml

# 2. Inject into agent config (example: nutrition-coach)
# Edit ~/.hermes/profiles/nutrition-coach/config.yaml:
providers:
  noris:
    type: openai
    base_url: https://ai.noris.de/v1
    api_key: sk-...

Also remove stale api_key / base_url entries from the agent's model: block — they should come from providers.*.

Verification after fix:

hermes profile show <name>  # should show "(noris)" not "(custom)"

Step 4: Telegram Bot Setup & Verification (Critical)

Before an agent can communicate via Telegram, the systemd service must be installed.

  1. Configure .env: Create ~/.hermes/profiles/<name>/.env with TELEGRAM_BOT_TOKEN=...
  2. Reload: systemctl --user daemon-reload
  3. Install Service: Run <name> gateway install.
    • ⚠️ Critical: gateway start fails without this.
  4. Start Service: systemctl --user start hermes-gateway-<name>.service
  5. Verification:
    • Check process: systemctl status hermes-gateway-<name>.service (Active: running)
    • Check logs: journalctl --user -u hermes-gateway-<name>.service -n 20
    • Test API: curl https://api.telegram.org/bot<TOKEN>/getMe -> expect {"ok": true}

Telegram DM Limitation (Two Bots, Same User)

A Telegram user has ONE DM chat ID regardless of which bot messages them. If you register a second bot (@schoen_essensplaner_bot) and expect it to create a "separate chat" from the CEO bot, both bots will deliver to the same DM thread (the user's personal chat ID).

True separation requires a group or topics:

  • New Group: Create a dedicated group, invite the bot → negative group chat ID
  • Topics: Enable Topics in an existing group, post into a dedicated topic thread
  • The infra-sre agent delivers to telegram:-5265906503 because it targets a group, not a DM.

Step 4: Operational Workflow

Task Distribution

  • Direct Command: User asks a specific agent (e.g., via Telegram Topic or chef, check infra).
  • Delegation: CEO uses delegate_task to offload complex sub-tasks to isolated contexts.
  • Automation: CEO sets cronjob for routine tasks (e.g., Sunday night infra scan).

Success Measurement

The CEO monitors specific metrics per agent:

  • Infra: Uptime, TLS-Expiration, Port-Drift.
  • Family: Plan-adherence, Feedback-Adaption, Budget-accuracy.

Step 4: Team Growth (Gap Analysis)

The CEO must proactively identify "hiring" needs:

  1. Pattern Recognition: Notice repetitive user questions or manual tasks.
  2. Proposal: "Chef, we need a new agent for [Task] because [Gap]."
  3. Execution: User approves -> CEO creates Profile + SOUL.md + Cron -> Agent goes live.

Pitfalls

  • Context Bleed: Do not share Memory across profiles unless explicitly intended (use memory tool to copy).
  • Over-Engineering: Start with 1-2 agents; expand based on actual need, not theoretical ones.
  • Telegram Routing: If using a single bot, rely on Topics in groups for separation. If using multiple bots, ensure each has a unique @BotFather token.
  • Cannot restart gateway from inside the gateway process. When running as a Telegram gateway session, systemctl --user restart hermes-gateway or hermes gateway restart will be blocked with "Blocked: cannot restart or stop the gateway from inside the gateway process" because SIGTERM propagates to child processes (including the agent itself). Instead: use the /restart slash command in the Telegram chat, or ask the user to run hermes gateway restart from a separate SSH session.
  • Cross-profile gateway restarts also blocked from inside any gateway. systemctl --user restart hermes-gateway-infra-sre also fails from inside the main gateway because systemd sends SIGTERM to the entire cgroup. Must be done from an external shell or via /restart in each respective bot chat.
  • hermes update procedure for multi-gateway setups. After hermes update completes (pulls code + syncs skills + updates Python deps), each gateway service must be restarted individually to load the new code. The update command itself does NOT restart gateways. For a 3-profile setup (default + infra-sre + nutrition-coach), all three need restart. From inside any gateway session, use /restart; from SSH, use systemctl --user restart hermes-gateway hermes-gateway-infra-sre hermes-gateway-nutrition-coach.
  • Gateway may hang in "deactivating" state after restart attempt. If systemctl --user is-active shows deactivating for an extended period, use systemctl --user reset-failed <service> followed by systemctl --user start <service>. Alternatively, systemctl --user kill <service> then start. This is a systemd race condition, not a Hermes bug.
  • MCP servers and toolset changes require gateway restart. Adding mcp_servers to config.yaml or enabling a new toolset (e.g., hermes tools enable homeassistant) does not take effect until the gateway restarts. MCP tools are discovered at startup only — no hot-reload.

Skill Update History

  • 2026-04-21: Initial creation based on "Schön Consulting Inc." virtual team setup (Infra SRE + Nutrition Coach).