When /etc/fonts is a symlink to a directory, private-etc rules that invoke fcopy produce wrong directory structure and breaks apps (NixOS)

[reedriley]

Description

On NixOS; the /etc/fonts directory is a symlink (to a symlink) to a directory. When a profile includes this in private-etc; fcopy appears to copy the symlink into the target directory rather than resolving it as a directory and copying the contents over.

In short; inside of a firejail; the "correct" path becomes /etc/fonts/fonts instead of /etc/fonts.

Steps to Reproduce

1. Install NixOS; or set up etc with a symlink from /etc/fonts to /etc/static/fonts and a symlink from /etc/static/fonts to some other location.
2. Run a profile for basically any electron app.
3. Observe that it hits issues involving fonts.

Additional context

I think the fix here is probably to modify fcopy to behave differently if a source symlink is to a directory vs. a file; and add a corresponding unit test. If that's the case; I'm happy to work on a pull request - but given that this is in a fairly subtle space I wanted to check if my understanding is correct before I begin.

Environment

• NixOS 21.11
• Firejail version 0.9.68rc1

Checklist

• The issues is caused by firejail (i.e. running the program by path (e.g. /usr/bin/vlc) "fixes" it).
• I can reproduce the issue without custom modifications (e.g. globals.local).
• The program has a profile. (If not, request one in https://github.com/netblue30/firejail/issues/1139)
• The profile (and redirect profile if exists) hasn't already been fixed upstream.
• I have performed a short search for similar issues (to avoid opening a duplicate).
  • I'm aware of browser-allow-drm yes/browser-disable-u2f no in firejail.config to allow DRM/U2F in browsers.
• I used --profile=PROFILENAME to set the right profile. (Only relevant for AppImages)

Log

From running firejail --debug, I'm pretty confident the relevant section of the logs involve this snippet:

Copying files in the new /etc directory:
Copying /etc/fonts to private /etc
Creating empty /run/firejail/mnt/etc/fonts directory
sbox run: /run/firejail/lib/fcopy /etc/fonts /run/firejail/mnt/etc/fonts 

Happy to provide more logs or perform more tests as requested.

[netblue30]

Thanks for the bug! Try the latest on mainline git, I think I have a fix for fcopy.

[rusty-snake]

firejail --noprofile --debug --private-etc=localtime ls -l /etc now shows an empty /etc. Can anyone confirm.

[reinerh]

firejail --noprofile --debug --private-etc=localtime ls -l /etc now shows an empty /etc. Can anyone confirm.

Yes, I can confirm. With commit 4e27b34 on top of rc1 I also get an empty /etc.

[reedriley]

And I can confirm that with firejail built from master installed on NixOS that:

1. /etc/fonts/ looks sane again for a few profiles I tested, and also
2. the electron apps that had been broken without a hacky workaround are functional again.

[netblue30]

New fix in, this time for localtime broken earlier. @reedriley, give it a try again. Thanks.

[reinerh]

I'm wondering if special-casing /etc/fonts is sufficient for NixOS. AFAIK they use a lot of symlinks, so probably also for other configuration (and application) files.

[vs49688]

They do. This is from my NixOS system:

Directories:

$ find /etc -maxdepth 1 -type l -xtype d
/etc/kbd
/etc/static
/etc/dbus-1
/etc/cups
/etc/fonts
/etc/zoneinfo
/etc/terminfo

Files:

$ find /etc -maxdepth 1 -type l -xtype f
/etc/hostid
/etc/resolvconf.conf
/etc/set-environment
/etc/locale.conf
/etc/shells
/etc/nscd.conf
/etc/fstab
/etc/hosts
/etc/profile
/etc/issue
/etc/fuse.conf
/etc/localtime
/etc/netgroup
/etc/nanorc
/etc/nsswitch.conf
/etc/rpc
/etc/mtab
/etc/bashrc
/etc/hostname
/etc/services
/etc/sddm.conf
/etc/login.defs
/etc/ipsec.secrets
/etc/tlp.conf
/etc/ethertypes
/etc/inputrc
/etc/host.conf
/etc/protocols
/etc/man_db.conf
/etc/os-release
/etc/vconsole.conf

[reedriley]

This fix at least resolves the electron app issues. So it's at least an improvement. But like @vs49688 says there are other directory symlinks.

I think the old firejail handling of symlinks-to-files was probably correct; it's just the handling of symlinks-to-directories that was broken. Is it possible to run this logic only if src is a symlink that resolves to a directory?

[netblue30]

Yes, probably there are other directories handled as symlinks on NixOS. Run a "ls -l /etc" and post it here.

[vs49688]

Yes, probably there are other directories handled as symlinks on NixOS. Run a "ls -l /etc" and post it here.

Done, see lsout.txt

[netblue30]

thanks @vs49688! The symlinks - plenty of them - are going in /etc/static. I'll bring in a new fix tomorrow.

[reedriley]

On my system; these are the directories in /etc that are symlinks to other directories:

$ find /etc -maxdepth 1 -type l -xtype d | sort
/etc/apparmor.d
/etc/cups
/etc/dbus-1
/etc/fonts
/etc/kbd
/etc/static
/etc/terminfo
/etc/zoneinfo

i suspect the precise set will vary from installation to installation - for example apparmor.d isn't present on @vs49688's system.

I still think the right fix here is probably to modify fcopy semantics when the "source" argument is a symlink to a directory vs. symlink to a file? Just like how fcopy has different semantics when the "source" argument is a directory vs. a file? (duplicate_dir vs. duplicate_file?)

Either that; or if we can't trust that the symlink won't point somewhere else in the appropriate threat models; we might need to change private-etc profiles to encode whether an entry is a directory or a file. (e.g., by using a trailing slash - we could treat fonts/ and fonts as having distinct meanings?)

[netblue30]

I'm reading about NixOS. Do you guys also have a /etc/config directory?

[vs49688]

I'm reading about NixOS. Do you guys also have a /etc/config directory?

Not on any of my systems, although someone could add one via environment.etc

i suspect the precise set will vary from installation to installation - for example apparmor.d isn't present on @vs49688's system.

Yeah, I should probably enable AppArmor.

I still think the right fix here is probably to modify fcopy semantics when the "source" argument is a symlink to a directory vs. symlink to a file? Just like how fcopy has different semantics when the "source" argument is a directory vs. a file? (duplicate_dir vs. duplicate_file?)

Agreed.

[reedriley]

Here's a sketch of something that seems to work just fine for me: reedriley@c0822a0

With that patch applied; everything works the way I expect; and I get the following directory layout:

$ firejail --noprofile --quiet --private-etc=localtime,fonts ls -l --recursive /etc/
/etc/:
total 0
dr-xr-xr-x 4 root root 100 Feb  2 17:53 fonts
lrwxrwxrwx 1 root root  88 Feb  2 17:53 localtime -> /nix/store/6cc9lmldp0w0vvgr5k1h1gr8i6hn791b-tzdata-2021c/share/zoneinfo/America/New_York

/etc/fonts:
total 0
dr-xr-xr-x 2 root root  60 Feb  2 17:53 2.11
dr-xr-xr-x 2 root root 480 Feb  2 17:53 conf.d
lrwxrwxrwx 1 root root  83 Feb  2 17:53 fonts.conf -> /nix/store/xba54gdybnaca93w85vmzrqg1n0njxg6-fontconfig-2.13.94/etc/fonts/fonts.conf

/etc/fonts/2.11:
total 0
lrwxrwxrwx 1 root root 83 Feb  2 17:53 fonts.conf -> /nix/store/xba54gdybnaca93w85vmzrqg1n0njxg6-fontconfig-2.13.94/etc/fonts/fonts.conf

/etc/fonts/conf.d:
total 0
lrwxrwxrwx 1 root root  66 Feb  2 17:53 00-nixos-cache.conf -> /nix/store/dipzgbynmr5gmgn732dacbsavj107xi2-fc-00-nixos-cache.conf
lrwxrwxrwx 1 root root 113 Feb  2 17:53 10-hinting-slight.conf -> /nix/store/xba54gdybnaca93w85vmzrqg1n0njxg6-fontconfig-2.13.94/share/fontconfig/conf.avail/10-hinting-slight.conf
lrwxrwxrwx 1 root root  70 Feb  2 17:53 10-nixos-rendering.conf -> /nix/store/9nvnd8s7ac3y94cywa5iwjx9ccmb1rm9-fc-10-nixos-rendering.conf
lrwxrwxrwx 1 root root 117 Feb  2 17:53 10-scale-bitmap-fonts.conf -> /nix/store/xba54gdybnaca93w85vmzrqg1n0njxg6-fontconfig-2.13.94/share/fontconfig/conf.avail/10-scale-bitmap-fonts.conf
lrwxrwxrwx 1 root root 116 Feb  2 17:53 20-unhint-small-vera.conf -> /nix/store/xba54gdybnaca93w85vmzrqg1n0njxg6-fontconfig-2.13.94/share/fontconfig/conf.avail/20-unhint-small-vera.conf
lrwxrwxrwx 1 root root 113 Feb  2 17:53 30-metric-aliases.conf -> /nix/store/xba54gdybnaca93w85vmzrqg1n0njxg6-fontconfig-2.13.94/share/fontconfig/conf.avail/30-metric-aliases.conf
lrwxrwxrwx 1 root root 107 Feb  2 17:53 40-nonlatin.conf -> /nix/store/xba54gdybnaca93w85vmzrqg1n0njxg6-fontconfig-2.13.94/share/fontconfig/conf.avail/40-nonlatin.conf
lrwxrwxrwx 1 root root 106 Feb  2 17:53 45-generic.conf -> /nix/store/xba54gdybnaca93w85vmzrqg1n0njxg6-fontconfig-2.13.94/share/fontconfig/conf.avail/45-generic.conf
lrwxrwxrwx 1 root root 104 Feb  2 17:53 45-latin.conf -> /nix/store/xba54gdybnaca93w85vmzrqg1n0njxg6-fontconfig-2.13.94/share/fontconfig/conf.avail/45-latin.conf
lrwxrwxrwx 1 root root 108 Feb  2 17:53 49-sansserif.conf -> /nix/store/xba54gdybnaca93w85vmzrqg1n0njxg6-fontconfig-2.13.94/share/fontconfig/conf.avail/49-sansserif.conf
lrwxrwxrwx 1 root root 103 Feb  2 17:53 50-user.conf -> /nix/store/xba54gdybnaca93w85vmzrqg1n0njxg6-fontconfig-2.13.94/share/fontconfig/conf.avail/50-user.conf
lrwxrwxrwx 1 root root 104 Feb  2 17:53 51-local.conf -> /nix/store/xba54gdybnaca93w85vmzrqg1n0njxg6-fontconfig-2.13.94/share/fontconfig/conf.avail/51-local.conf
lrwxrwxrwx 1 root root  74 Feb  2 17:53 52-nixos-default-fonts.conf -> /nix/store/k8lbdziyr4ghki6csr8qnlshh273bv7k-fc-52-nixos-default-fonts.conf
lrwxrwxrwx 1 root root  73 Feb  2 17:53 53-nixos-reject-type1.conf -> /nix/store/bw6w7yazrr5rbwsy4x4v3bfjqzd7v223-fc-53-nixos-reject-type1.conf
lrwxrwxrwx 1 root root  65 Feb  2 17:53 53-no-bitmaps.conf -> /nix/store/2ffyf8rn2ib5hjx2q0k3jgd1vj4lllin-fc-53-no-bitmaps.conf
lrwxrwxrwx 1 root root 106 Feb  2 17:53 60-generic.conf -> /nix/store/xba54gdybnaca93w85vmzrqg1n0njxg6-fontconfig-2.13.94/share/fontconfig/conf.avail/60-generic.conf
lrwxrwxrwx 1 root root 104 Feb  2 17:53 60-latin.conf -> /nix/store/xba54gdybnaca93w85vmzrqg1n0njxg6-fontconfig-2.13.94/share/fontconfig/conf.avail/60-latin.conf
lrwxrwxrwx 1 root root 112 Feb  2 17:53 65-fonts-persian.conf -> /nix/store/xba54gdybnaca93w85vmzrqg1n0njxg6-fontconfig-2.13.94/share/fontconfig/conf.avail/65-fonts-persian.conf
lrwxrwxrwx 1 root root 107 Feb  2 17:53 65-nonlatin.conf -> /nix/store/xba54gdybnaca93w85vmzrqg1n0njxg6-fontconfig-2.13.94/share/fontconfig/conf.avail/65-nonlatin.conf
lrwxrwxrwx 1 root root 106 Feb  2 17:53 69-unifont.conf -> /nix/store/xba54gdybnaca93w85vmzrqg1n0njxg6-fontconfig-2.13.94/share/fontconfig/conf.avail/69-unifont.conf
lrwxrwxrwx 1 root root 108 Feb  2 17:53 80-delicious.conf -> /nix/store/xba54gdybnaca93w85vmzrqg1n0njxg6-fontconfig-2.13.94/share/fontconfig/conf.avail/80-delicious.conf
lrwxrwxrwx 1 root root 108 Feb  2 17:53 90-synthetic.conf -> /nix/store/xba54gdybnaca93w85vmzrqg1n0njxg6-fontconfig-2.13.94/share/fontconfig/conf.avail/90-synthetic.conf

[reedriley]

Or; alternatively; we could skip creating the empty /etc/fonts directory in this case somehow. (Perhaps by modifying create_empty_dir_as_root to use lstat instead of stat? Or build_dirs in fs_etc.c to use lstat instead of stat?)

[reedriley]

Here's another fix that appears to work, which feels a lot less hacky: reedriley@967265d

The flow:

1. fs_etc.c used to resolve the symlink /etc/fonts to a directory; and therefore create an empty directory.
2. Then, fcopy moved the symlink into the newly created empty directory screwing up the structure.

Changing the build_dirs function in fs_etc.c to use lstat instead of stat allows it to handle this case correctly and preserve the directory structure rather than warping it.

With this patch, I get:

$ firejail --noprofile --quiet --private-etc=localtime,fonts ls -l --recursive /etc/
/etc/:
total 0
lrwxrwxrwx 1 root root 69 Feb  2 20:08 fonts -> /nix/store/6yzk4cmmdywc33lr6bf5iladr7rjih0w-fontconfig-conf/etc/fonts
lrwxrwxrwx 1 root root 88 Feb  2 20:08 localtime -> /nix/store/6cc9lmldp0w0vvgr5k1h1gr8i6hn791b-tzdata-2021c/share/zoneinfo/America/New_York

[netblue30]

It turns out we already had all the support. It is --follow-link flag in fcopy, I just had to enable it for private-etc. Also reverted all the changes to fcopy.

Fix here: 8c33968

[reedriley]

I can confirm; this fix works for my system as well. Thanks!

I'd flag there are probably other symlink issues lurking in firejail+NixOS; the distro relies on them heavily. But, if I hit any, I'll be sure to report them or submit a pull request,

[netblue30]

I'd flag there are probably other symlink issues lurking in firejail+NixOS; the distro relies on them heavily.

Sure, thanks! /etc should be fully fixed right now. The fix resolved all symlinks there. Closed for now.