Tech Notes

An autoinstall image for Debian 13, and parameters from the Proxmox API

How I build one Debian 13 autoinstall ISO with preseed and xorriso, and give every machine its own user, address and hostname from the Proxmox API.

#Linux#Homelab#Proxmox

Most of my homelab runs on Proxmox. When I need one more Debian machine there, I attach the netinst ISO, open the console and answer the same questions again. Language, keyboard, mirror, disk, user, packages. It takes around fifteen minutes of clicking, and every machine can possibly end a little different from the previous one.

So I made one image which installs Debian 13 by itself. The image is the same for every machine. The things which differ, the hostname, the address, the user, come from outside at install time. All of it is in images/debian in my repository.

What is inside the image

The base is the normal Debian 13 netinst ISO. Into it I put three things.

The first is preseed.cfg, which holds every answer that never changes. Locale, keyboard, time zone, my internal apt mirror, the partitioning recipe, the package list.

The second and the third are two shell scripts, early.sh and late.sh. The preseed file can call a script at two moments:

d-i preseed/early_command string sh /cdrom/autoinstall/early.sh
d-i preseed/late_command string sh /cdrom/autoinstall/late.sh

early_command runs right after the preseed file is read, before the network is configured and before the disk is touched. It reads the parameters, then writes them into debconf:

debconf-set netcfg/get_ipaddress 192.168.0.161
debconf-set netcfg/get_netmask 255.255.255.0
debconf-set netcfg/get_gateway 192.168.0.1
debconf-set passwd/username storage
debconf-set partman-auto/disk /dev/sda

When the installer reaches those steps, the answers are already there and it does not ask.

late_command runs at the end, when the new system is mounted on /target. It writes the mirror sources, puts the ssh keys in place, gives the user sudo and configures sshd.

The whole preseed file

This is the file as it is baked into the image. The values in the network and the account block are placeholders: they keep the installer from stopping if a parameter is missing, and early.sh overwrites them before the installer reaches those questions.

### Localisation
d-i debian-installer/language string en
d-i debian-installer/country string UA
d-i debian-installer/locale string en_US.UTF-8
d-i localechooser/supported-locales multiselect en_US.UTF-8
d-i keyboard-configuration/xkb-keymap select us
d-i keyboard-configuration/layoutcode string us
d-i console-setup/ask_detect boolean false

### Clock and time zone
d-i time/zone string Europe/Kyiv
d-i clock-setup/utc boolean true
d-i clock-setup/ntp boolean true

### Network, overwritten by early.sh
d-i netcfg/choose_interface select auto
d-i netcfg/disable_autoconfig boolean true
d-i netcfg/disable_dhcp boolean true
d-i netcfg/dhcp_options select Configure network manually
d-i netcfg/get_hostname string debian
d-i netcfg/get_domain string
d-i netcfg/hostname string debian
d-i netcfg/get_ipaddress string 10.0.0.10
d-i netcfg/get_netmask string 255.255.255.0
d-i netcfg/get_gateway string 10.0.0.1
d-i netcfg/get_nameservers string 10.0.0.1
d-i netcfg/no_default_route boolean false
d-i netcfg/confirm_static boolean true
d-i netcfg/wireless_wep string
d-i hw-detect/load_firmware boolean true

### Mirror
d-i mirror/country string manual
d-i mirror/protocol string http
d-i mirror/http/hostname string mirror.intra
d-i mirror/http/directory string /deb.debian.org/debian
d-i mirror/http/proxy string
d-i mirror/suite string trixie
d-i mirror/udeb/suite string trixie
d-i debian-installer/allow_unauthenticated boolean true

### Accounts, username and hash come from early.sh
d-i passwd/root-login boolean false
d-i passwd/make-user boolean true
d-i passwd/user-fullname string Debian User
d-i passwd/username string debian
d-i passwd/user-password-crypted password !
d-i user-setup/allow-password-weak boolean true
d-i user-setup/encrypt-home boolean false

### Partitioning, whole disk, one root filesystem, ext4
d-i partman-auto/disk string /dev/sda
d-i partman-auto/method string regular
d-i partman-auto/choose_recipe select atomic
d-i partman-lvm/device_remove_lvm boolean true
d-i partman-md/device_remove_md boolean true
d-i partman-basicfilesystems/no_swap boolean false
d-i partman-partitioning/confirm_write_new_label boolean true
d-i partman/choose_partition select finish
d-i partman/confirm boolean true
d-i partman/confirm_nooverwrite boolean true
d-i partman-efi/non_efi_system boolean true
d-i partman/default_filesystem string ext4

### Base system
d-i base-installer/install-recommends boolean true
d-i apt-setup/non-free-firmware boolean true
d-i apt-setup/non-free boolean false
d-i apt-setup/contrib boolean false
d-i apt-setup/enable-source-repositories boolean false
d-i apt-setup/services-select multiselect updates
d-i apt-setup/use_mirror boolean true
d-i apt-setup/disable-cdrom-entries boolean true
d-i apt-setup/cdrom/set-first boolean false
d-i apt-setup/cdrom/set-next boolean false
d-i apt-setup/cdrom/set-failed boolean false

### Packages
tasksel tasksel/first multiselect standard, ssh-server
d-i pkgsel/include string sudo openssh-server qemu-guest-agent ca-certificates curl gnupg python3 chrony vim ufw
d-i pkgsel/upgrade select safe-upgrade
d-i pkgsel/update-policy select unattended-upgrades
popularity-contest popularity-contest/participate boolean false

### Bootloader
d-i grub-installer/only_debian boolean true
d-i grub-installer/with_other_os boolean true
d-i grub-installer/bootdev string default

### Finish
d-i finish-install/reboot_in_progress note
d-i cdrom-detect/eject boolean true
d-i debian-installer/exit/poweroff boolean false

### Hooks
d-i preseed/early_command string sh /cdrom/autoinstall/early.sh
d-i preseed/late_command string sh /cdrom/autoinstall/late.sh

partman-auto/method regular with the atomic recipe means one partition for everything, no LVM, and partman-auto adds the ESP by itself when the machine boots in UEFI mode. passwd/root-login false leaves root without a password, so the first user is the only way in and late.sh gives it sudo.

In the repository the mirror lines point to deb.debian.org, so the image works without my mirror. make iso MIRROR=mirror.intra turns them into the lines above.

Building the image

The build is one script, build-image.sh, and it does not need root. There is no loop mount and no unpacking of 755 megabytes into a temporary folder. xorriso can read the original ISO and write a new one with some files replaced, and it keeps the boot records as they were:

xorriso -abort_on FATAL \
        -indev debian-13.6.0-amd64-netinst.iso \
        -outdev debian-13.6.0-amd64-autoinstall.iso \
        -boot_image any replay \
        -volid DEBIAN_AUTOINSTALL \
        -overwrite on \
        -map ./stage/preseed.cfg /preseed.cfg \
        -map ./stage/autoinstall /autoinstall \
        -map ./stage/isolinux.cfg /isolinux/isolinux.cfg \
        -map ./stage/grub.cfg /boot/grub/grub.cfg \
        -chmod_r 0555 /autoinstall -- \
        -commit

From the repository the whole build is four commands. The result is images/debian/dist/debian-13.6.0-amd64-netinst-autoinstall.iso:

git clone https://github.com/Denrox/proxmox-iac && cd proxmox-iac
make image
curl -fLO https://cdimage.debian.org/cdimage/archive/13.6.0/amd64/iso-cd/debian-13.6.0-amd64-netinst.iso
make iso BASE=debian-13.6.0-amd64-netinst.iso

-boot_image any replay is the important flag. It copies the El Torito records and the hybrid MBR from the source image, so the result still boots on BIOS and on UEFI. Without it you get a data disc.

The two config files which I replace are the boot menus. The BIOS one is isolinux, the UEFI one is grub, and both need the same kernel line:

append initrd=/install.amd/initrd.gz auto=true priority=critical file=/cdrom/preseed.cfg locale=en_US.UTF-8 keymap=us netcfg/choose_interface=auto console=tty0 console=ttyS0,115200n8 ---

file=/cdrom/preseed.cfg tells the installer where the answers are. priority=critical means it asks only about things which are really critical, and everything else takes the default.

I also regenerate md5sum.txt inside the image, so the built in integrity check still passes on the files which I changed.

The installer wanted to read the CD

The first version stopped in the middle with a red dialog: "An attempt to configure apt to install additional packages from the media failed". After that the unattended install is finished, because somebody has to press a button.

The reason is a script inside apt-setup called 40cdrom. If the image has a file /.disk/base_installable, apt-setup runs apt-cdrom add inside the new system to register the disc as a package source. On my machines this command fails, and the failure is a critical question.

My machines take everything from my internal mirror, so the disc is never needed as a package source. The build now deletes that one marker file from the image:

xorriso ... -rm /.disk/base_installable --

The generator sees no marker, exits immediately, and no dialog appears. The preseed file says the same thing a second time, from the other side, so a media scan is not configured even if some other path finds the disc:

d-i apt-setup/disable-cdrom-entries boolean true
d-i apt-setup/cdrom/set-first boolean false
d-i apt-setup/cdrom/set-next boolean false
d-i apt-setup/cdrom/set-failed boolean false

For this the mirror must be reachable during the whole install, also for the base system, and not only for extra packages. In my network this is fine, and it also means every machine gets the same packages from the same place, independent of how old the ISO is.

A directory on Proxmox from the API

The images need a place on the Proxmox node. A directory storage is enough for ISO files, and it can be created over the API, so this step also does not need a shell on the node:

PVE=https://192.168.0.155:8006/api2/json
TOKEN='PVEAPIToken=root@pam!autoinstall=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'

curl -sk -H "Authorization: $TOKEN" -X POST "$PVE/storage" \
  --data-urlencode "storage=isoimages" \
  --data-urlencode "type=dir" \
  --data-urlencode "path=/var/lib/vz/isoimages" \
  --data-urlencode "content=iso" \
  --data-urlencode "mkdir=1"

The token needs rights on the node, otherwise every answer is empty or 403. QUICKSTART.md, step 0, has the pveum commands.

mkdir=1 creates the folder if it is not there. The answer comes back at once, and the storage is active and visible in the web interface.

Then the image goes into it. The upload endpoint takes a checksum, and the node calculates the hash of what it received and fails the task if it does not match:

SUM=$(sha256sum images/debian/dist/debian-13.6.0-amd64-netinst-autoinstall.iso | cut -d' ' -f1)

curl -sk -H "Authorization: $TOKEN" \
  -F content=iso \
  -F checksum-algorithm=sha256 \
  -F "checksum=$SUM" \
  -F "filename=@images/debian/dist/debian-13.6.0-amd64-netinst-autoinstall.iso" \
  "$PVE/nodes/pve-test/storage/isoimages/upload"

I use the checksum every time now. Two builds of this image have the same size, because I change only text files inside a padded ISO. In the storage list they look identical. The hash is the only thing which tells me which one is on the node.

When you delete the storage over the API, the folder on disk stays. Proxmox removes the definition, not the data.

Parameters from the Proxmox fields

Everything which differs per machine is a normal field of the VM configuration. Proxmox collects those fields into a small cloud-init drive and attaches it to the machine, and early.sh reads that drive before the installer asks anything:

HASH=$(mkpasswd -m yescrypt)
KEY=$(python3 -c "import urllib.parse,sys;print(urllib.parse.quote(open('$HOME/.ssh/id_ed25519.pub').read(), safe=''))")

curl -sk -H "Authorization: $TOKEN" -X POST "$PVE/nodes/pve-test/qemu" \
  --data-urlencode "vmid=201" \
  --data-urlencode "name=web01" \
  --data-urlencode "memory=2048" \
  --data-urlencode "cores=1" \
  --data-urlencode "scsi0=local-lvm:16" \
  --data-urlencode "ide2=isoimages:iso/debian-13.6.0-amd64-netinst-autoinstall.iso,media=cdrom" \
  --data-urlencode "ide3=local-lvm:cloudinit" \
  --data-urlencode "ciuser=storage" \
  --data-urlencode "cipassword=$HASH" \
  --data-urlencode "sshkeys=$KEY" \
  --data-urlencode "ipconfig0=ip=192.168.0.161/24,gw=192.168.0.1" \
  --data-urlencode "nameserver=192.168.0.1" \
  --data-urlencode "searchdomain=intra" \
  --data-urlencode "net0=virtio,bridge=vmbr0" \
  --data-urlencode "boot=order=scsi0;ide2"

The hostname comes from the machine name, because Proxmox writes hostname: web01 and fqdn: web01.intra into that drive by itself. The address, the gateway and the resolver come from ipconfig0 and nameserver. The user comes from ciuser with cipassword or sshkeys, or with both.

So one image is uploaded one time, and every machine after that is one API call. Nothing else is built and nothing else is uploaded. pve-provision.sh does all of this in one command: upload, create, start and wait. It uploads to the local storage, so the directory above is not needed for it:

export PVE_HOST=192.168.0.155 PVE_TOKEN='PVEAPIToken=root@pam!autoinstall=<secret>'
./scripts/pve-provision.sh --insecure --node pve-test --vmid 201 --name web01 \
    --installer images/debian/dist/debian-13.6.0-amd64-netinst-autoinstall.iso \
    --ciuser storage --ssh-key ~/.ssh/id_ed25519.pub \
    --ip 192.168.0.161/24 --gw 192.168.0.1 --nameserver 192.168.0.1

The limit of this way is that you can pass only what Proxmox models. There is one user field and no second one, which is the reason this image creates one account and gives it sudo. There is an option --cicustom which takes a file with any content you want, but that file has to be on the node storage already, and the upload endpoint accepts only iso, vztmpl and import. So a custom snippet needs scp and a shell on the node, and I did not want that step.

Three things for which I spent extra time

The boot order. I wrote boot=order=ide2;scsi0, CD first, like for a manual install. The install finished, the machine rebooted, the firmware found the CD again and started the same install from the beginning. It did this four times before I looked. 42 gigabytes were written to a 16 gigabyte disk. The correct order is scsi0;ide2: an empty disk has no boot sector, so the firmware goes to the CD and installs, and after that the disk boots.

The password field. Proxmox accepts anything as --cipassword and passes it further without looking. The installer treats it as a crypt hash. If you put a normal word there, the machine installs fine and nobody can log in. My script now refuses a value which does not start with a dollar sign.

The ssh key field. --sshkeys must be url encoded. Proxmox rejects a literal slash inside it, and a normal url encoder leaves the slash alone. In Python it is urllib.parse.quote(key, safe=''). Base64 keys contain slashes often.

How it looks now

Building the image takes about ten seconds. The install itself takes around twelve minutes, and almost all of that is downloading packages from my mirror over a slow link.

I create the machine with one API call, start it, and come back later. The machine has the right address, my user with the ssh key and sudo without a password, my internal mirror in sources.list and a firewall which allows only ssh. After this it is time for the second step of automation: Ansible, Semaphore and playbooks. I wrote about that step in Automating the Proxmox ecosystem: a controller VM for OpenTofu state.