Reference Architecture

Network Observability
& Source of Truth

How I build visibility into a large multi-vendor network estate: vendor telemetry into an open-source stack, a source of truth that states what should exist, drift measured against it, and automation that stays read-only until a human approves the change.

The pipeline

From vendor APIs to an on-call alert

Five stages. Every one of them read-only by default, scheduled on the host, and monitored for its own freshness — a dead feed raises an alert rather than going quiet.

01
Sources
Every layer of the estate, pulled from the vendor's own API or its native syslog.
  • SD-WAN fabric
  • Branch LAN & wireless
  • Firewall estate
  • RMM & endpoint
  • Security telemetry
  • Public cloud networking
02
Collect
One collector per vendor. Rate-limit aware, restart-safe, heartbeat-monitored.
  • Scheduled API collectors
  • rsyslog → Vector pipeline
  • Parsed and labeled at the edge
  • No agent on any production device
03
Store
Three stores, three questions: what should exist, how is it performing, what just happened.
  • NetBox — intent, IPAM, circuits
  • InfluxDB — metrics, tiered retention
  • Loki — estate-wide logs
04
Visualize
Dashboards and alert rules provisioned as code and versioned in git — not clicked together in a UI.
  • Grafana, provisioned from source
  • Fleet, WAN, capacity and SLA views
  • Every panel has a named owner
05
Act
Raw webhooks rendered into readable cards, each tied to a written runbook.
  • Chat alerts with runbook links
  • On-call escalation ladder
  • Carrier & WAN SLA for leadership
What makes it hold up

Three rules the whole thing is built on

A pipeline is easy. Keeping it trustworthy — and safe to point at production — is the part that takes judgement.

Discipline
Intent versus actual

The source of truth holds intended state. Collectors report what devices actually run. The gap between them is measured and reported — drift is detected, never assumed.

Automation
Read-only until proven

Scheduled jobs back configuration up to git and score every device against a hardening baseline. Anything that writes runs in check-mode first, canary before fleet, with a recorded rollback.

AI control plane
Read anything, change nothing

A single agent reads every system in the stack and answers in plain English. It cannot change anything — writes execute only inside a human-approved plan, fully audit-logged.

Design decisions

The parts people usually get wrong

Alerts that survive contact with humans

An alert fires only when two independent signals agree. False positives die before they reach anyone. A channel people mute is worse than no alerting at all.

Collectors that admit when they die

Every feed reports its own freshness on a cadence. Silence is treated as a failure state, not as good news — the most dangerous dashboard is one that is quietly three weeks stale.

Source of truth ≠ discovery

NetBox records what the network is supposed to be. Discovery reports what it is. Conflating the two produces a database that documents your mistakes instead of your intent.

Built to be handed over

Everything runs on organization-owned infrastructure under version control, with runbooks, an escalation ladder and a documented restore path. If it only works while one person is present, it is a liability, not a platform.