Tech Notes

An autoinstall image for Debian 13, and parameters from the Proxmox API

How I build one Debian 13 installer image for the whole homelab, create a directory storage on Proxmox over the API, and give every machine its own user and network without rebuilding the image.

#Linux#Homelab#Proxmox

Most of my homelab runs on Proxmox. When I need one more Debian machine there, I attach the netinst ISO, open the console and answer the same questions again. Language, keyboard, mirror, disk, user, packages. It takes around fifteen minutes of clicking, and every machine can possibly end a little different from the previous one.

So I made one image which installs Debian 13 by itself. The image is the same for every machine. The things which differ, the hostname, the address, the user, come from outside at install time.

What is inside the image

The base is the normal Debian 13 netinst ISO. Into it I put three things.

The first is preseed.cfg, which holds every answer that never changes. Locale, keyboard, time zone, my internal apt mirror, the partitioning recipe, the package list.

The second and the third are two shell scripts. The preseed file can call a script at two moments:

d-i preseed/early_command string sh /cdrom/autoinstall/early.sh
d-i preseed/late_command string sh /cdrom/autoinstall/late.sh

early_command runs right after the preseed file is read, before the network is configured and before the disk is touched. It reads the parameters, then writes them into debconf:

debconf-set netcfg/get_ipaddress 192.168.11.14
debconf-set netcfg/get_netmask 255.255.255.0
debconf-set netcfg/get_gateway 192.168.11.1
debconf-set passwd/username storage
debconf-set partman-auto/disk /dev/sda

When the installer reaches those steps, the answers are already there and it does not ask.

late_command runs at the end, when the new system is mounted on /target. It writes the mirror sources, puts the ssh keys in place, gives the user sudo and configures sshd.

Building the image

The build is one script and it does not need root. There is no loop mount and no unpacking of 755 megabytes into a temporary folder. xorriso can read the original ISO and write a new one with some files replaced, and it keeps the boot records as they were:

xorriso -indev debian-13.6.0-amd64-netinst.iso \
        -outdev debian-13.6.0-amd64-autoinstall.iso \
        -boot_image any replay \
        -map ./stage/preseed.cfg /preseed.cfg \
        -map ./stage/autoinstall /autoinstall \
        -map ./stage/isolinux.cfg /isolinux/isolinux.cfg \
        -map ./stage/grub.cfg /boot/grub/grub.cfg \
        -commit

-boot_image any replay is the important flag. It copies the El Torito records and the hybrid MBR from the source image, so the result still boots on BIOS and on UEFI. Without it you get a data disc.

The two config files which I replace are the boot menus. The BIOS one is isolinux, the UEFI one is grub, and both need the same kernel line:

append initrd=/install.amd/initrd.gz auto=true priority=critical file=/cdrom/preseed.cfg locale=en_US.UTF-8 keymap=us ---

file=/cdrom/preseed.cfg tells the installer where the answers are. priority=critical means it asks only about things which are really critical, and everything else takes the default.

I also regenerate md5sum.txt inside the image, so the built in integrity check still passes on the files which I changed.

The installer wanted to read the CD

The first version stopped in the middle with a red dialog: "An attempt to configure apt to install additional packages from the media failed". After that the unattended install is finished, because somebody has to press a button.

The reason is a script inside apt-setup called 40cdrom. If the image has a file /.disk/base_installable, apt-setup runs apt-cdrom add inside the new system to register the disc as a package source. On my machines this command fails, and the failure is a critical question.

My machines take everything from my internal mirror, so the disc is never needed as a package source. The build now deletes that one marker file from the image:

xorriso ... -rm /.disk/base_installable --

The generator sees no marker, exits immediately, and no dialog appears. The price is that the mirror must be reachable during the whole install, also for the base system, and not only for extra packages. In my network this is fine, and it also means every machine gets the same packages from the same place, independent of how old the ISO is.

A directory on Proxmox from the API

The images need a place on the Proxmox node. A directory storage is enough for ISO files, and it can be created over the API, so this step also does not need a shell on the node:

PVE=https://192.168.11.8:8006/api2/json
TOKEN='PVEAPIToken=root@pam!autoinstall=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'

curl -sk -H "Authorization: $TOKEN" -X POST "$PVE/storage" \
  --data-urlencode "storage=isoimages" \
  --data-urlencode "type=dir" \
  --data-urlencode "path=/var/lib/vz/isoimages" \
  --data-urlencode "content=iso" \
  --data-urlencode "mkdir=1"

mkdir=1 creates the folder if it is not there. The answer comes back at once, and the storage is active and visible in the web interface.

Then the image goes into it. The upload endpoint takes a checksum, and the node calculates the hash of what it received and fails the task if it does not match:

SUM=$(sha256sum dist/debian-13.6.0-amd64-autoinstall.iso | cut -d' ' -f1)

curl -sk -H "Authorization: $TOKEN" \
  -F content=iso \
  -F checksum-algorithm=sha256 \
  -F "checksum=$SUM" \
  -F "filename=@dist/debian-13.6.0-amd64-autoinstall.iso" \
  "$PVE/nodes/host/storage/isoimages/upload"

I use the checksum every time now. Two builds of this image have the same size, because I change only text files inside a padded ISO. In the storage list they look identical. The hash is the only thing which tells me which one is on the node.

When you delete the storage over the API, the folder on disk stays. Proxmox removes the definition, not the data.

Parameters from the Proxmox fields

Everything which differs per machine is a normal field of the VM configuration. Proxmox collects those fields into a small cloud-init drive and attaches it to the machine, and early.sh reads that drive before the installer asks anything:

HASH=$(mkpasswd -m yescrypt)
KEY=$(python3 -c "import urllib.parse,sys;print(urllib.parse.quote(open('/home/deba/.ssh/id_ed25519.pub').read(), safe=''))")

curl -sk -H "Authorization: $TOKEN" -X POST "$PVE/nodes/host/qemu" \
  --data-urlencode "vmid=201" \
  --data-urlencode "name=web01" \
  --data-urlencode "memory=2048" \
  --data-urlencode "cores=1" \
  --data-urlencode "scsi0=local-lvm:16" \
  --data-urlencode "ide2=isoimages:iso/debian-13.6.0-amd64-autoinstall.iso,media=cdrom" \
  --data-urlencode "ide3=local-lvm:cloudinit" \
  --data-urlencode "ciuser=storage" \
  --data-urlencode "cipassword=$HASH" \
  --data-urlencode "sshkeys=$KEY" \
  --data-urlencode "ipconfig0=ip=192.168.11.14/24,gw=192.168.11.1" \
  --data-urlencode "nameserver=192.168.0.1" \
  --data-urlencode "searchdomain=intra" \
  --data-urlencode "net0=virtio,bridge=vmbr0" \
  --data-urlencode "boot=order=scsi0;ide2"

The hostname comes from the machine name, because Proxmox writes hostname: web01 and fqdn: web01.intra into that drive by itself. The address, the gateway and the resolver come from ipconfig0 and nameserver. The user comes from ciuser with cipassword or sshkeys, or with both.

So one image is uploaded one time, and every machine after that is one API call. Nothing else is built and nothing else is uploaded.

The limit of this way is that you can pass only what Proxmox models. There is one user field and no second one, which is the reason this image creates one account and gives it sudo. There is an option --cicustom which takes a file with any content you want, but that file has to be on the node storage already, and the upload endpoint accepts only iso, vztmpl and import. So a custom snippet needs scp and a shell on the node, and I did not want that step.

Three things for which I spent extra time

The boot order. I wrote boot=order=ide2;scsi0, CD first, like for a manual install. The install finished, the machine rebooted, the firmware found the CD again and started the same install from the beginning. It did this four times before I looked. 42 gigabytes were written to a 16 gigabyte disk. The correct order is scsi0;ide2: an empty disk has no boot sector, so the firmware goes to the CD and installs, and after that the disk boots.

The password field. Proxmox accepts anything as --cipassword and passes it further without looking. The installer treats it as a crypt hash. If you put a normal word there, the machine installs fine and nobody can log in. My script now refuses a value which does not start with a dollar sign.

The ssh key field. --sshkeys must be url encoded. Proxmox rejects a literal slash inside it, and a normal url encoder leaves the slash alone. In Python it is urllib.parse.quote(key, safe=''). Base64 keys contain slashes often.

How it looks now

Building the image takes about ten seconds. The install itself takes around twelve minutes, and almost all of that is downloading packages from my mirror over a slow link.

I create the machine with one API call, start it, and come back later. The machine has the right address, my user with the ssh key and sudo without a password, my internal mirror in sources.list and a firewall which allows only ssh. After this it is time for the second step of automation: Ansible, Semaphore and playbooks.