Spent much of the day trying to fix an unbootable Fedora machine where I’d previously, accidentally deleted the entire of /etc. Mostly recovered by copying content from another identical install but systemd still failed very early in boot, not even able to get to the single user mode emergency shell. Eventually I booted from grub with init=/bin/sh and used this (previously unknown to me) systemd trick
# cd /etc/systemd/system/sysinit.target.wants/
# ln -s /lib/systemd/system/debug-shell.service .
# exec /lib/systemd/systemd
The ‘debug-shell.service’ unit simply runs a password-less shell on /dev/tty9 very early in boot, so when the systemd boot sequence failed I could Alt-F9 switch console and finally diagnose the problem I had with LVM/fstab which was preventing boot from succeeding.
So next time you have an utterly screwed up machine remember about the ‘debug-shell.service’ trick. Just don’t forget to disable the service once you’ve fixed everything – don’t want a password-less shell hanging around forever 🙂