High-Availability Proxmox Cluster & Ceph Storage at Home
A deep dive into migrating three compact nodes to a shared Ceph pool, configuring quorum voters, and seamless live VM migration without edge dropouts.
Why Ceph in a 3-Node Compact Cluster?
Previously, each node ran local NVMe ZFS pools with asynchronous replication scheduled every 15 minutes. While simple, a node failure still meant up to 15 minutes of state delta. With Ceph RBD (RADOS Block Device) spanning across three dedicated 2.5GbE links, virtual disks are synchronously replicated across all three hosts.
Network Topology & Dedicated Corosync Ring
The most common cause of split-brain in small Proxmox clusters is mixing cluster heartbeats (Corosync) with heavy storage or backup traffic. To guard against this, I isolated Corosync on a dedicated VLAN with strict QoS, plus a secondary fallback ring over standard home LAN.
# /etc/pve/corosync.conf excerpt
totem {
cluster_name: harker-homelab
config_version: 3
interface {
linknumber: 0
knet_transport: sctp
}
}Live Migration Benchmarks & Verdict
Migrating a busy PostgreSQL instance and Pi-hole DNS resolver between node-01 and node-02 takes approximately 1.8 seconds with Ceph shared storage, with zero dropped UDP DNS packets and seamless TCP session resumption. Next step: automated power balancing via custom Proxmox HA rules.