How to boot a diskless VM with iPXE over NFS using only IPv6

I have encoutered so many issues and countless hours trying to figure this out, but I finally succeeded. Reading source code, strace’ing binaries trying to figure out what was wrong with them.

iPXE

iPXE will do IPv4 DHCP first and then IPv6 SLAAC/DHCP. This is not a problem, it’s just annoying. IPv4 DHCP has a default timeout in iPXE of 10 seconds.

What you need to do in iPXE:

• enable serial console -> src/config/console.h -> #define CONSOLE_SERIAL
• enable IPv6 -> src/config/general.h -> #define NET_PROTO_IPV6
• mangle IPv4 DHCP timeout -> src/config/dhcp.h -> #define DHCP_DISC_END_TIMEOUT_SEC 1

Kernel options

Make sure you pass the right kernel options for initramfs/initrd to know exactly what you want to do. (replace eth0 with the name of your PXE interface)

• ip=:::::eth0:off (disable IPv4 on the PXE interface)
• ip6=:::::eth0:on (enable IPv6 on the PXE interface)
• nfsroot=<hostname>:<path> (mounting IPv6 NFS shares using a literal IPv6 WILL FAIL, you MUST use a hostname)

Those options will tell the initrd script which interface and protocol is desired.

Hooks in initramfs

A few files must be copied to the initramfs in order to make this work. First we need DNS to work and secondly we need to replace the busybox nfsmount with mount.nfs4 (don’t worry, it can also do NFS3).

In /etc/initramfs-tools/hooks create an executable script with the name of your choice. Here’s the content:

#!/bin/bash
. /usr/share/initramfs-tools/hook-functions

# copy all the name libraries (for DNS to work)
cp -fpL /lib/x86_64-linux-gnu/libns* ${DESTDIR}/lib/x86_64-linux-gnu/

# copy helper files needed by mount.nfs4
for file in /etc/protocols /etc/netconfig
do cp $file ${DESTDIR}${file}; done

# uncomment to embed in your setup, can be useful for debugging
# copy_exec /sbin/dhclient /sbin
# copy_exec /bin/ping /bin
# copy_exec /usr/bin/strace /bin
# replace the busybox nfsmount with mount.nfs4 (this way we don't need to modify the included nfs script which calls nfsmount)
copy_exec /sbin/mount.nfs4 /bin/nfsmount

After that you’ll need to update-initramfs -u

Boot

Boot and it should just work! Happy IPv6!