llamafile runs on all Linux distros since ~2009. It doesn't have any dependencies. It'd probably even run as the init process too (if you assimilate it). The only thing it needs is the Linux 2.6.18+ kernel application binary interface. If you have an SELinux policy, then you may need to tune things, and on some distros you might have to install APE Loader for binfmt_misc, but that's about it. See the Gotchas in the README. Also goes without saying that llamafile runs on WIN32 too, if that's the world you're most comfortable with. It even runs on BSD distros and MacOS. All in a single file.
The APE shell script needs to run /bin/mkdir in order to map the embedded ELF executable in memory. It should be possible for you to work around this on Linux by installing our binfmt_misc interpreter:
sudo wget -O /usr/bin/ape https://cosmo.zip/pub/cosmos/bin/ape-$(uname -m).elf
sudo sh -c "echo ':APE:M::MZqFpD::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register"
sudo sh -c "echo ':APE-jart:M::jartsr::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register"
That way the only file you'll need to whitelist with Nix is /usr/bin/ape. You could also try just vendoring the 8kb ape executable in your Nix project, and simply executing `./ape ./llamafile`.
