Linking statically, and glibc breaking userspace for fun
glibc is annoyingly eager to break userspace. You can’t just build something that only depends on libc and expect it to work on all linux systems of that architecture.
I don’t know why Linus Torvalds keeps insisting “we do not break userspace” as a policy for the kernel when libc seems to make that exact thing a hobby. And either way the userspace programs break.
Compiling static (including libc) is frowed upon, and has even had known breakages left unaddressed.
E.g. setlocale()
had a strange bug where for new threads you had to
first set the locale to the wrong locale, and then call it again to
set it to the right one. Otherwise the new thread would be in a weird
state where the locale was wrong, but it thought it was right, so
wouldn’t allow you to change it to what it thought it already was.
I can’t find the bug now (I ran into this around 2004-2005), but the official response was basically “well don’t compile statically, then”.
And DNS can be broken with static glibc. “a statically linked glibc can’t use NSS (Name Service Switch) modules from a different glibc version, so if you statically link glibc you can’t reliably resolve names with getaddrinfo”.
On other operating systems, like Solaris and OpenBSD, support for building statically has actually been withdrawn.
Here’s building some Rust against glibc, and trying to run it elsewhere:
build-machine$ cargo build --release
[…]
build-machine$ ldd target/release/livecount
linux-vdso.so.1 (0x00007fff191a0000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fc8f6077000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fc8f5f90000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc8f5d68000)
/lib64/ld-linux-x86-64.so.2 (0x00007fc8f6567000)
build-machine$ scp target/release/livecount prod-machine:
[…]
prod-machine$ ./livecount -h
l: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by ./livecount)
./livecount: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by ./livecount)
./livecount: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by ./livecount)
./livecount: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by ./livecount)
./livecount: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by ./livecount)
It’s libc! Not like you need to maintain some complex C++ vtable ABI. How can you not be backwards (ehem, forwards) compatible for libc?!
So what do you do?
glibc is broken when you compile statically, and ABI breaks regularly so you can’t link dynamically either. So what do you do?
You’ll need to install another libc, and link statically against that:
$ rustup target add x86_64-unknown-linux-musl
[…]
$ cargo build --release --target=x86_64-unknown-linux-musl
[…]
$ ldd target/x86_64-unknown-linux-musl/release/livecount
statically linked