VMware: Custom ESXi boot iso bootable on Fusion, but not vSphere...
Building an embedded lab for some testing w/ NSX and everything. Cannot clone a pre-installed ESXi host w/o some magic, so decided to utilize a simple kickstart script in an ESXi custom iso pre-mounted to my VM template. Worked fine on Fusion, not vSphere/ESXi though...
Simply change VM Options --> Boot Options to BIOS
Fusion defaults to BIOS, which is why it worked. I made the iso w/o UEFI options which is why it worked on fusion, but not ESXi VM.
The command I was using to make the iso was missing some key new features to make the iso UEFI bootable.
mkisofs -relaxed-filenames -J -R -o ~/Desktop/custom_esxi.iso -b ISOLINUX.BIN -c BOOT.CAT -no-emul-boot -boot-load-size 4 -boot-info-table -eltorito-alt-boot -eltorito-platform efi -b EFIBOOT.IMG -no-emul-boot ~/Desktop/ESXiDefaultInstall
You will have to modify two BOOT.CFG files, one in root directory, and the other in the EFI subdirectory to utilize any custom kickstart you put together. If you only modify one, it will only point to the kickstart file when booted via BIOs and not UEFI or vice versa.
Basically, it came down to the fact that I was rebuilding the iso w/ an older command that didn't have the UEFI options. This made my iso BIOs bootable (Fusion default), but not bootable on my ESXi VM which was set to UEFI. Highlighted above in solutions section. Basically my goal was to simply clone a templated VM shell w/ a local hard drive layout I needed for an Embedded vSAN lab. The VM shell would have the iso mounted so that it would build ESXi at time of clone.
So I needed an iso that would simply build ESXi w/ basic defaults and configure DHCP. In the other notes section, I show how I modified the BOOT.CFG file(s) and added a KS.CFG file. Once I found the options I was missing, my iso was now properly bootable via UEFI and BIOs. This is a fairly easy workaround to build ESXi virtual hosts w/o having to stand up a PXE environment within an NSX bubble.
mkisofs is not apart of MacOS, but you can install it via homebrew.
brew install cdrtools
/BOOT.CFG and /EFI/BOOT/BOOT.CFG (Modified kernelopt value to target CD rom and my custom KS.CFG)
install --firstdisk --overwritevmfs
Nested ESXi virtual appliances that you can customize via OVA options.