How I Migrated 10 Production Docker Stacks with Basically Zero Downtime

Moving live production containers between hosts sounds simple on paper until you actually have to do it. You have databases that can corrupt if you copy them hot, reverse proxy certificates you don’t want browsers barking about, and routers that refuse to let go of old MAC addresses.

I had 10 Docker stacks running on an aging host (docker01 at 10.1.20.18) that handled everything in my lab: Vaultwarden, Caddy, Authentik, FreePBX, Grafana, Loki, WordPress, and our mail relay. I built a fresh VM on XCP-ng (docker02), tuned it, and migrated every single service over with about 60 seconds of actual cutover time. Here is how I pulled it off and the two major gotchas I ran into along the way.

1. The Game Plan: Take Over the Same IP

The biggest headache in any server migration is updating DNS records, firewall rules, and mobile app endpoints. I decided early on: docker02 was going to take over 10.1.20.18 so nothing else on the network had to change.

I configured Netplan on docker02 with two IP addresses on the same virtual interface:

network:
  version: 2
  ethernets:
    enX0:
      dhcp4: no
      addresses:
        - 10.1.20.19/24 # Management & FreePBX IP
        - 10.1.20.18/24 # Production Services IP
      routes:
        - to: default
          via: 10.1.20.1
      nameservers:
        addresses: [10.1.20.15]

2. Pre-Syncing Data While Still Live

You never want to do a massive multi-gigabyte data copy during your actual maintenance window. Days before the cutover, I ran an initial rsync of all /home/ntadmin/stacks/ from docker01 over to docker02.

I used -aHAXv with --numeric-ids to make sure permissions, hard links, and ACLs stayed completely intact. That knocked out 95% of the heavy lifting while all my containers were still serving clients normally.

3. Gotcha #1: Docker Port Conflicts (Caddy vs. FreePBX)

Here was the first surprise. I tried setting "ip": "10.1.20.18" in Docker’s /etc/docker/daemon.json thinking Docker would bind ports cleanly to that IP. It didn’t. Docker still bound ports to 0.0.0.0.

When I brought up FreePBX on 10.1.20.19, it failed with address already in use because Caddy was sitting on port 80 and 443 across all interfaces. The fix was simple: stop relying on daemon.json and explicitly specify the IP in the compose files:

# In Caddy's docker-compose.yml:
ports:
  - "10.1.20.18:80:80"
  - "10.1.20.18:443:443"

# In FreePBX's docker-compose.yml:
ports:
  - "10.1.20.19:80:80"
  - "10.1.20.19:443:443"

4. The 60-Second Cutover

When it was time to pull the trigger:

1. I stopped all stacks on docker01 to freeze database state.
2. Ran a final rsync pass. Because 95% of the data was already on docker02, the final delta sync finished in just 18 seconds.
3. Shut down docker01 and set auto_poweron=false on XCP-ng so it wouldn’t accidentally start back up and create an IP clash.
4. Fired up all 10 stacks on docker02 with docker compose up -d.

5. Gotcha #2: The UniFi Stale ARP Trap

Containers were running and Caddy was healthy, but external connections were timing out. Why? My UniFi UDM Pro still had 10.1.20.18 cached to docker01‘s old MAC address. The router was stubbornly sending traffic to a powered-off VM.

Instead of waiting 10 to 15 minutes for the ARP cache to expire on its own, I hopped onto docker02 and sent four gratuitous pings directly from the secondary interface:

# Force the UDM Pro to update its ARP table immediately
ping -I 10.1.20.18 -c 4 10.1.20.1

Boom. The UDM Pro updated its table immediately, traffic flowed through, and the migration was complete. Because I copied Caddy’s internal CA keys along with the data, no browser threw an SSL warning. Total downtime: about a minute.