Setting Up Multi-Path iSCSI with My EqualLogic SAN and XCP-ng

Running single-link iSCSI in a virtualization lab is asking for trouble. Pull the wrong patch cable, reboot a switch, or have a NIC glitch out, and your hypervisor immediately panics and crashes your VMs.

I wanted true hardware multipathing (MPIO) between my Dell EqualLogic PS-series SAN array and my XCP-ng host. It sounds straightforward on paper, but if you’ve ever dealt with EqualLogic and Linux multi-homing on the same subnet, you know there’s a nasty ARP gotcha waiting to bite you. Here’s how I laid out the network, enabled multipathing on XCP-ng, and fixed the routing so both links pull their weight.

1. The Storage Network Layout

The first rule of EqualLogic is simple: never route storage traffic through your default gateway. It belongs on an isolated, non-routed VLAN. I carved out VLAN 99 strictly for SAN traffic—no gateway, no internet access, and no inter-VLAN firewall overhead:

Interface / Device	IP Address	Subnet	What It Does
EqualLogic Group IP	10.1.99.100	VLAN 99	Discovery portal (STOREGRP1)
EqualLogic Eth 0	10.1.99.25	VLAN 99	Active storage target port 1
EqualLogic Eth 1	10.1.99.26	VLAN 99	Active storage target port 2
EqualLogic Out-of-Band	10.1.1.24	VLAN 1	Array management web interface
XCP-ng Storage NIC 1	10.1.99.11	VLAN 99	Physical eth3 mapped to xenbr3
XCP-ng Storage NIC 2	10.1.99.12	VLAN 99	Physical eth2 mapped to xenbr2

2. Turning on Multipath in XCP-ng

By default, XCP-ng doesn’t turn on multipath bindings for new iSCSI Storage Repositories. You have to tell Dom0 to use Device Mapper Multipath (DMP) before connecting your pool. Grab your host UUID and run this from Dom0:

# Check existing multipath status
xe host-param-get uuid=<host-uuid> param-name=other-config

# Enable Native Device Mapper Multipath (DMP)
xe host-param-set uuid=<host-uuid> other-config:multipathing=true
xe host-param-set uuid=<host-uuid> other-config:multipathhandle=dmp

3. The Linux Routing Trap (And How I Fixed It)

Here’s where things get fun. When you plug two NICs (eth2 and eth3) into the exact same subnet (10.1.99.0/24), the Linux kernel treats them as “weak hosts.” An ARP request comes in on eth2, but the kernel might decide to send the reply out of eth3. That breaks EqualLogic’s session tracking and causes random timeouts.

To fix this permanently, I wrote a small systemd unit (/etc/systemd/system/iscsi-routes.service) that pins each EqualLogic target IP to its dedicated bridge and enforces strict ARP behavior:

[Unit]
Description=iSCSI EqualLogic Route Pinning and Strict ARP
After=network.target

[Service]
Type=oneshot
ExecStart=/sbin/ip route add 10.1.99.25/32 dev xenbr3
ExecStart=/sbin/ip route add 10.1.99.26/32 dev xenbr2
ExecStart=/sbin/sysctl -w net.ipv4.conf.xenbr2.arp_ignore=1
ExecStart=/sbin/sysctl -w net.ipv4.conf.xenbr2.arp_announce=2
ExecStart=/sbin/sysctl -w net.ipv4.conf.xenbr3.arp_ignore=1
ExecStart=/sbin/sysctl -w net.ipv4.conf.xenbr3.arp_announce=2
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

4. Connecting the Storage Repository

With host multipathing enabled and ARP locked down, I pointed XCP-ng at the EqualLogic portal. When you run multipath -ll, you should see both paths active in round-robin mode:

36000d771001a1c380000a6e000000001 dm-2 EQLOGIC,100E-00
size=800G features='1 queue_if_no_path' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 3:0:0:0 sdc 8:32 active ready running
  `- 4:0:0:0 sdd 8:48 active ready running

5. The “Pull the Cable” Test

I don’t trust redundancy until I test it with physical hardware. I kicked off a heavy sequential read inside a live VM and physically unplugged the patch cable on eth2.

Dom0 caught the link state drop in about 400 milliseconds, marked path sdc faulty, and shifted 100% of the I/O over to sdd. The VM didn’t crash, the filesystem didn’t go read-only, and there wasn’t even a hiccup. Plugged the cable back in, and path recovery was automatic.

With both gigabit links pulling together under normal load, I’m seeing around 228 MB/s sequential reads off the array. Fast, resilient, and completely rock solid.