syntheses:compute-host-stack
This is an old revision of the document!
Table of Contents
Compute Host Stack
Notes
Summary
Compute Host Stack
Overview
The compute host stack runs on physical servers (Xen/KVM hosts) and manages guest VPS lifecycle. It consists of software from multiple repos:
vps/vps→product/website/HostDaemon/— Main .NET daemon (also duplicated here)vps/vps→infrastructure/compute/— Shell scripts, configs, and services deployed to hostsvps/compute(project 22) — Separate repo with HostDaemon, qemu-network, routed, etc.vps/qemu-network(project 20) — iptables/netns firewall rules
Architecture
Customer Action
↓
WebApi / PanelSite / ClassicPanel
↓
Domain validation → NHibernate persistence
↓
HostCommand created (Infrastructure/compute or product/website/HostDaemon)
↓
HostDaemon (on compute host) receives & executes command
↓
Guest lifecycle: start/stop/reinstall/migrate/firewall/backup
↓
qemu-network/mammothvps.sh applies iptables with claim_lock
Key Components
HostDaemon (product/website/HostDaemon/)
.NET daemon running on each compute host. Handles guest lifecycle commands:
product/website/HostDaemon/ ├── Applets/ — Host command applets (sub-commands) ├── Application/ — Application logic ├── GuestConfiguration/ — Guest config generation (Xen/KVM) ├── Handlers/ — Host command handlers ├── Helpers/ — Utility classes ├── Hosted/ — Background services ├── LoggingContext/ — Logging ├── Properties/ — Assembly metadata ├── Scripts/ — Shell scripts deployed to hosts ├── Services/ — Host services ├── config/ — HostDaemon config ├── BlockingHostCommandHandler.cs ├── HostCommandExecutor.cs — Executes commands on hosts ├── HostCommandHandler.cs — Command handler interface ├── HostShell.cs — Shell abstraction for host commands ├── SshShell.cs — SSH-based host shell ├── Program.cs — Entry point └── appsettings.json — Configuration
Key classes:
- HostCommand — The command object sent from WebApi to HostDaemon
- HostCommandExecutor — Executes commands, manages ordering
- HostCommandHandler — Interface for handling specific command types
- HostShell / SshShell — Abstraction for executing commands on hosts
qemu-network (vps/qemu-network, project 20)
iptables/netns firewall on compute hosts:
- Script:
mammothvps.sh— Applies iptables rules withclaim_lock - Uses claim_lock “iptables” held during entire apply phase
- Serialises across ALL guests on a host (one customer's slow apply blocks others)
- Critical bug surface: unbounded firewall rules cause Cartesian product explosion
infrastructure/compute/ (vps/vps)
Scripts and configs deployed to compute hosts:
infrastructure/compute/ ├── etc/ — Host configuration files ├── hostdaemon/ — HostDaemon deployment ├── ovmf/ — UEFI firmware files ├── qemu-network/ — Firewall scripts ├── routed/ — Routing daemon ├── scripts/ — Shell scripts (backup, monitoring, etc.) ├── services/ — Systemd/init service definitions ├── smtbalance/ — SMTP load balancing ├── xm/ — Xen configuration ├── xmconfig/ — Xen config tools └── xm-for-kvm/ — Xen-to-KVM migration
vps/compute (project 22)
Separate repo for compute host programs:
vps/compute/ ├── hostdaemon/ — HostDaemon (duplicate of product/website/HostDaemon) ├── ovmf/ — UEFI firmware ├── power-limiter/ — Power management ├── qemu-network/ — iptables/netns firewall ├── routed/ — Routing daemon ├── scripts/ — Host scripts ├── services/ — Service definitions ├── smtbalance/ — SMTP balancing ├── xm/ — Xen ├── xmconfig/ — Xen config └── xm-for-kvm/ — Xen-to-KVM migration
Guest Lifecycle Commands
Commands flow from WebApi → HostCommand → HostDaemon:
- Power operations — start, stop, restart, reinstall
- Network — firewall rules, interface changes, VPC assignment
- Migration — live migration between hosts
- Backup — backup/restore operations
- Configuration — DNS, reverse DNS, hostname changes
- Maintenance — scheduled maintenance windows
Firewall Rule Flow (Critical Path)
This is the most complex and bug-prone path:
- Customer submits firewall rules via WebApi/Panel/CLI
- Domain validation →
ValidateAddressListAttributevalidates each address (NOT collection count) - NHibernate persists
GuestFirewallRuleentities - GuestManagementService.UpdateGuestNetwork() → sends HostCommand to HostDaemon
- HostDaemon iterates all firewall rules
- OperatingSystem.XenConfiguration() → builds FIREWALLRULES xenConfig string
- Guest.GetIptablesEntries() → aggregates across ALL rules
- GuestFirewallRule.GetIptablesEntries() → 4-level Cartesian loop (source × dest × protocol × port)
- qemu-network/mammothvps.sh → applies iptables with claim_lock
Xen vs KVM
xm/— Xen configuration (older platform)xm-for-kvm/— Xen-to-KVM migration toolingovmf/— UEFI firmware (for KVM)- Migration from Xen to KVM is in progress
Infrastructure/QEMU
- build.sh — QEMU build script
- debian-12.cfg — Debian 12 QEMU configuration
- harness.sh — QEMU test harness
- README.md — Documentation
Key Bug Surfaces
claim_lock (qemu-network)
claim_lock “iptables”held during entire apply phase- One customer's massive firewall ruleset blocks ALL other guests on the same host
- Known issue #1640
Unbounded firewall rules
GuestFirewallRule— No cap on SourceAddresses/DestinationAddresses countValidateAddressListAttribute— Per-item only, no collection count checkGuestFirewallRule.GetIptablesEntries()— 4-level nested loopGuest.GetIptablesEntries()— No size limit on aggregation
HostCommand serialization
- HostDaemon processes commands sequentially per guest
- Slow commands (massive firewall apply) block subsequent commands
Related
Related Pages
syntheses/compute-host-stack.1784174059.txt.gz · Last modified: by 127.0.0.1
