Lab OS: Live D3 Force-Directed Network Graph
Lab OS is a custom, real-time command deck and network topology engine. Instead of static Visio diagrams or rigid dashboard tiles, it models the entire infrastructure—UniFi enterprise switches, hypervisors, virtual machines, SAN arrays, and smart-home devices—as an interactive, physics-based force-directed simulation in D3.js.
1. System Architecture & Data Pipeline
Lab OS decouples live data collection from web rendering. A high-efficiency Python backend daemon running on our control workstation continuously interrogates the lab’s core APIs and pushes structured graph deltas to the frontend over WebSockets:
| Subsystem | Ingestion Source | Polling Frequency | Telemetry Delivered |
|---|---|---|---|
| Network Fabric | UniFi Dream Machine Pro API | Every 3 seconds | Port link negotiation (10GbE SFP+ vs 1GbE), active PoE wattage, duplex state, dropped packets |
| System Health & Alarms | Zabbix 7.0 JSON-RPC (trigger.get) |
Every 5 seconds | Active problem states, ICMP availability, severity levels (Warning through Disaster) |
| Virtualization & Storage | XCP-ng (XAPI) & Hyper-V WMI | Every 10 seconds | Host RAM pressure, VM power state, EqualLogic MPIO active path throughput |
| Container Stacks | Docker Engine Socket (docker02) |
Every 5 seconds | 24+ container lifecycle states, container healthchecks, restart counts |
| IoT & Automation | Home Assistant WebSocket API | Real-time events | VLAN 40 sensor telemetry, voice satellite states, presence triggers |
2. Physics-Based Visual Mechanics (D3.js)
The frontend is powered by a customized HTML5 Canvas and SVG hybrid renderer built on d3-force:
Dynamic Force Simulation
Nodes naturally self-organize based on physical network topology. The core switch sits at the gravitational center; hypervisors repel each other slightly to avoid clutter; and virtual machines orbit around their parent hypervisors via structural constraints.
Animated Traffic Flow
Links carry glowing animated particle streams. The speed, color, and density of the particles map linearly to actual interface throughput. A 220 MB/s SAN backup visually races across the links, while idle links carry slow, faint pulses.
Reactive Status Auras
Every node has a multi-tier status glow. Healthy nodes pulse a calm emerald green. When Zabbix detects an issue, the affected node pulses amber or red, ripple waves expand outward, and the parent switch port highlights the path to the issue.
Interactive Node Inspection
Clicking any device opens an instant HUD blade with live hardware specs, IP assignments, MAC addresses, active service ports, and 1-click actions (e.g. open web UI, copy SSH command, or trigger a test ping).
3. The Human-in-the-Loop Control Panel
Lab OS is not just a passive monitor—it is also the primary approval interface for our local AI assistant (Jarvis). Whenever automation proposes an infrastructure modification (such as restarting a failed daemon or cycling a switch port):
- A high-visibility “Needs Approval” banner docks to the top of the HUD.
- The exact proposed shell command, target server, and reasoning are displayed side-by-side with the current live metric.
- Clicking [Approve] signs the payload cryptographically and signals the local agent to proceed; clicking [Reject] aborts the execution immediately.
4. Technical Specifications & Runtime
| Component | Specification | Notes |
|---|---|---|
| Frontend Engine | D3.js v7 (Canvas + SVG) | 60 FPS particle animation loop with WebGL acceleration |
| Backend Framework | Python 3.12 / Flask / WebSockets | Runs locally on DEATHSTAR (port 8200 / 5055) |
| Security Perimeter | Bound strictly to 127.0.0.1 / Tailscale ACL |
Zero exposure to untrusted VLANs or external WAN |
| Hardware Display | Dedicated 27″ Vertical 4K Display | Mounted in engineering workstation cockpit |
5. Roadmap & Next Phases
- Sanitized Demo Mode: An automated data scrubber that masks public IPs, internal hostnames, and MACs with fictitious aliases, enabling live video streaming and public case studies.
- Historical Traffic Time-Machine: A scrubber bar to “rewind” the network map to any timestamp in the past 7 days to watch network flows and alerts replay in real time during post-mortems.
- Per-VM Resource Gauges: Real-time circular progress rings directly around VM node borders indicating CPU and memory pressure.
For an in-depth write-up on the physics simulation math and D3 implementation details, read our companion engineering guide: Building Lab OS: How I Created a Real-Time Interactive Network Topology Map with D3.js.