chroot and dynamic libraries: Some jots

This is just a messed up pile of jots as I tried to solve a specific problem. The actual problem turned out to be between chair and keyboard, but I decided to post this anyhow, just in case it will be useful in the future.

The setting was like this: I had a script, which called a suid-enabled program I wrote (jailer.c), which did a chroot() to a chroot jail and then called setgroups(), setgid() and setuid(), and eventually an execl() to a bash script, which was of course inside the chroot jail.

So all in all, the program could be called from any user, but thanks to its setuid to root, it could change the root, and then turn into another user.

In the bash script, the control was eventually turned over to another program (not mine, hence the chroot protection) with the bash built-in command exec. And all was fine.

But then I needed to continue the execution after the program. So I dumped the exec and used the good old invocation by just starting the line with the program’s name. And that failed colossally.

Spoiler: The reason turned out to be that the current process ID remains when exec is used, and changes when other methods are used (duh). As some preparations to running the program had to match the process ID of the program running, exec worked, other methods didn’t. So it was really my bad.

After a while I thought I figured out that somehow, all this mucking around (playing with users? setuid? chroot?) caused the program to fail in finding its library files.

So I added a

export LD_LIBRARY_PATH=/lib64:/special/lib/lin64

line to the bash script, which made the program work. Finally. Only now it segfaulted. Well, at least I know I did something in the direction.

The problem seemed to be, that the program loaded an outdated libstdc++.so.6 file from its own library set, instead of the one in /usr/lib64/. LD_LIBRARY_PATH solved one issue, but since its paths are always handled before the regular one, it actually messed up.

Being in a chroot environment, everything is controlled, so why not add the standard libraries into LD_LIBRARY_PATH? Ugly, but nobody said being in a (chroot) jail should be nice.

So what is the regular order of loading libraries? Well, I went

$ ldconfig -v | less

and picked up the paths that live in the jail, and put them before the special paths. And then fixed LD_LIBRARY_PATH to

export LD_LIBRARY_PATH=/usr/lib64:/lib64:/special/lib/lin64

This solved the issue with libstdc++.so.6, but the segfault remained.

Epilogue: All this didn’t solve the problem, but rather kept me busy with complicated stuff, while the actual solution was so much simper. Maybe this will be useful for solving something else. Or I just wasted a few hours.