What's the scope of --root (supposed to be)?

[praiskup]

E.g. Red Hat UBI images contain:

[root@c5436d2dbebf /]# cat /etc/rpm/macros.image-language-conf 
%_install_langs C.utf8

This though affects dnf --installroot using rpm --root:

$ podman run --privileged -it --rm registry.access.redhat.com/ubi9/ubi
[root@85d37140e490 /]# cat /etc/rpm/macros.image-language-conf
%_install_langs C.utf
[root@85d37140e490 /]# dnf --installroot /tmp/customroot install tar -y
...
Complete!
[root@85d37140e490 /]# find /tmp/customroot/usr/share/locale/
/tmp/customroot/usr/share/locale/
/tmp/customroot/usr/share/locale/locale.alias

With removed macros file:

[root@85d37140e490 /]# rm /etc/rpm/macros.image-language-conf 
rm: remove regular file '/etc/rpm/macros.image-language-conf'? yes
[root@85d37140e490 /]# dnf --installroot /tmp/customroot2 install tar -y
...
Complete!
[root@85d37140e490 /]# find /tmp/customroot2/usr/share/locale/
/tmp/customroot2/usr/share/locale/
/tmp/customroot2/usr/share/locale/doi
/tmp/customroot2/usr/share/locale/doi/LC_MESSAGES
/tmp/customroot2/usr/share/locale/ig
/tmp/customroot2/usr/share/locale/ig/LC_MESSAGES
/tmp/customroot2/usr/share/locale/hus
/tmp/customroot2/usr/share/locale/hus/LC_MESSAGES
/tmp/customroot2/usr/share/locale/fro
/tmp/customroot2/usr/share/locale/fro/LC_MESSAGES
...

This is causing trouble with EPEL8/EPEL9 mock builds after switching to "bootstrap image" by default.

I'm curious whether this is an isolation bug in RPM or not. And if there's known guidance on what to do in such a case (should Mock try hard to work-around this, or should packages rather not require all langpacks at build-time?).

Is this a proper place to discuss this?

[pmatilai]

It's certainly proper to discuss it here.

Rpm generally pull all or almost all configuration from the host rather than --root. During initial install there's little else it can do, for latter usage it's more questionable maybe.
So it's what I would call expected behavior - in the sense that it's not a shock revelation that it does this. Whether it's the right behavior...

[praiskup]

Thank you for your comment. I wish you stated your opinion on what to do about this. But to me it eventually doesn't seem that much controversial to just remove the /etc/rpm/macros.image-language-conf file, which will lead to the default %_install_langs all behavior. This is likely the setup that is being used by everyone ATM, and if we want to have more minimal installations in Mock, someone has to come up with some official proposal (though since Mock still downloads the packages, filtering translations from the installation transaction probably wouldn't bring too much additional value to motivate us).

I now actually realized we have been doing this for Fedora Copr (for Fedora Cloud-based images) since the pre-bootstrap time. Those days host configuration affected non-bootstrapped Mock installations.

For Mock it's going to be a bit of "guessing", I suppose the config file could be named whatever way -> but at least Fedora and RHEL container images use the same pattern.

[pmatilai]

I wouldn't mind sharing my opinion if I had one 😄

Like noted before, to rpm it's not about install langs, its about all configuration. And changing such a wide-reacing, long-standing behavior is not something I'm willing to entertain at this time.

There are various ways around this but it requires help from the thing performing the installation (in Fedora, dnf) in that you need to be able to pass macro defines to it somehow. I don't know, maybe the dnf folks would be open to adding cli/config options for passing down to the rpm transaction?

[dmnks]

Indeed, the current man page entry for --root in rpm(8) is subject to interpretation. The line that separates what comes from the chroot and what doesn't is rather blurry, "all operations" can mean practically anything including, well, everything (but that's obviously not the case).

One way to read the "all operations" part is this: all operations performed to manage the target filesystem tree. That primarily means installing, removing and querying packages. Anything else that RPM does, such as reading configuration files, therefore isn't covered by --root. So, from that perspective, this is not a bug.

Whether or not it would make sense to let rpm do the chroot before the configuration is loaded I'm not sure either but there's a point to be made in terms of keeping things simple and the scope of --root as narrow as possible. RPM isn't (and shouldn't be) trying to become a containerization tool, that's best left to higher level tools utilizing RPM such as DNF. Looking at the DNF man page, --installroot has a much wider scope of what goes into and comes out of the installroot, with each piece documented in great detail. RPM macros aren't part of that right now but maybe that would be the right place to deal with this (?).

[dmnks]

As for the solution itself, my understanding of the use case here is that you basically want to instruct DNF to run RPM with the stock distro configuration during --installroot. Or, more specifically, with the stock configuration of the distro being installed in that root. Is that correct, @praiskup?

[praiskup]

Yes, correct.

[dmnks]

Thanks. I wonder if the --macros option (or its API equivalent in include/rpm/rpmmacro.h) could be used then? The little downside is that the caller would have to list all the default paths themselves (just with the target root prefix). Maybe we could improve that somehow and facilitate such use better but I'm not (yet) sure how. Any ideas?

[dmnks]

Note that whatever the solution, DNF would need to be updated accordingly (probably the --installroot logic).

[praiskup]

When we talk about configuration, what's meant by that? All macro definitions? Because there only seems to be an issue with /etc/rpm/macros.image-language-conf like things.

As you asked for ideas; I think there's a space for a low-level, distro-specific, rpm-sub-package that would install the distro-specific configuration. If there was such an RPM, Mock could just simply tell DNF to ignore the host's configuration at the beginning (split the preparation into two steps):

$ rootdir=$(mktemp -d)
dnf5 --installroot "$rootdir" install rpm-config --use-host-config
dnf5 --installroot "$rootdir" install @buildsys-build

The first transaction needs to be as small as possible. Small enough to
let distro-maintainers cure the target distribution so the transaction
isn't affected by the host configuration at all. The second
transaction, since the config is already in chroot, can ignore the
host configuration entirely.

Seems like the set of packages in the first transaction in Fedora
is currently pretty small:

$ sudo dnf5 --installroot "$rootdir" install setup --use-host-config
Updating and loading repositories:
Repositories loaded.
Package                        Arch     Version       Repository      Size
Installing:
 setup                         noarch   2.14.4-1.fc39 fedora     720.3 KiB
Installing dependencies:
 fedora-gpg-keys               noarch   39-1          fedora     123.2 KiB
 fedora-release                noarch   39-30         fedora       0.0   B
 fedora-release-common         noarch   39-30         fedora      17.7 KiB
 fedora-release-identity-basic noarch   39-30         fedora     666.0   B
 fedora-repos                  noarch   39-1          fedora       4.5 KiB
 filesystem                    x86_64   3.18-6.fc39   fedora     106.0   B

Transaction Summary:
 Installing:        7 packages

Total size of inbound packages is 1 MiB. Need to download 1 MiB.
After this operation 867 KiB will be used (install 867 KiB, remove 0 B).
Is this ok [y/N]:

Not sure if one of those could be recycled for this purpose, instead of
creating a new rpm-config package?

[dmnks]

When we talk about configuration, what's meant by that? All macro definitions? Because there only seems to be an issue with /etc/rpm/macros.image-language-conf like things.

Yeah, apologies for the confusion. Even though a lot of RPM "configuration" is done through macros, RPM does distinguish between configuration and macros, both of which can be overridden with the --rcfile and --macros CLI options, respectively.

What I meant was macro configuration, i.e. --macros. That takes a colon-separated list of paths and defaults to this (line breaks are mine):

/usr/lib/rpm/macros:/usr/lib/rpm/macros.d/macros.*:
/usr/lib/rpm/platform/%{_target}/macros:
/usr/lib/rpm/fileattrs/*.attr:/usr/lib/rpm/redhat/macros:
/etc/rpm/macros.*:
/etc/rpm/macros:
/etc/rpm/%{_target}/macros:
~/.rpmmacros

That's why I said you'd need to supply all the desired files yourself when using this CLI option as there's no way to just prepend/append paths to it.

As you asked for ideas; I think there's a space for a low-level, distro-specific, rpm-sub-package that would install the distro-specific configuration. If there was such an RPM, Mock could just simply tell DNF to ignore the host's configuration at the beginning (split the preparation into two steps):

$ rootdir=$(mktemp -d)
dnf5 --installroot "$rootdir" install rpm-config --use-host-config
dnf5 --installroot "$rootdir" install @buildsys-build

This is mixing two separate issues. The one originally discussed here is about RPM respecting the config/macro files in the target root when called with --root. Populating the root with the desired configuration is a then a separate matter and one that would need to be discussed on fedora-devel or similar channels.

Out of curiosity, though, does the second dnf5 command already work the way you want, or was that just your proposal for how it could work? If DNF already works this way, that means it does perform some kind of RPM isolation by itself (by calling RPM in a chroot/namespace) and thus adding an option like --use-installroot-config to DNF should be fairly easy.

The first transaction needs to be as small as possible. Small enough to let distro-maintainers cure the target distribution so the transaction isn't affected by the host configuration at all. The second transaction, since the config is already in chroot, can ignore the host configuration entirely.

Seems like the set of packages in the first transaction in Fedora is currently pretty small:

$ sudo dnf5 --installroot "$rootdir" install setup --use-host-config
Updating and loading repositories:
Repositories loaded.
Package                        Arch     Version       Repository      Size
Installing:
 setup                         noarch   2.14.4-1.fc39 fedora     720.3 KiB
Installing dependencies:
 fedora-gpg-keys               noarch   39-1          fedora     123.2 KiB
 fedora-release                noarch   39-30         fedora       0.0   B
 fedora-release-common         noarch   39-30         fedora      17.7 KiB
 fedora-release-identity-basic noarch   39-30         fedora     666.0   B
 fedora-repos                  noarch   39-1          fedora       4.5 KiB
 filesystem                    x86_64   3.18-6.fc39   fedora     106.0   B

Transaction Summary:
 Installing:        7 packages

Total size of inbound packages is 1 MiB. Need to download 1 MiB.
After this operation 867 KiB will be used (install 867 KiB, remove 0 B).
Is this ok [y/N]:

Not sure if one of those could be recycled for this purpose, instead of creating a new rpm-config package?

Again, while possibly a valid proposal, this is not the right venue for discussing it, it needs to be brought up on the Fedora channels.

[dmnks]

That's why I said you'd need to supply all the desired files yourself when using this CLI option as there's no way to just prepend/append paths to it.

Just to illustrate better how one could use this (in order to achieve your original objective):

# dnf install --releasever=38 --installroot=/rootfs rpm
[...]
# echo '%_foobar 123' > /etc/rpm/macros.foobar
# echo '%_foobar 456' > /rootfs/etc/rpm/macros.foobar
# rpm --eval '%_foobar'
123
# rpm --root /rootfs --eval '%_foobar'
123
# rpm --macros /rootfs/etc/rpm/macros.foobar --eval '%_foobar'
456

[praiskup]

This is mixing two separate issues. The one originally discussed here is
about RPM respecting the config/macro files in the target root when
called with --root. Populating the root with the desired configuration
is a then a separate matter and one that would need to be discussed on
fedora-devel or similar channels.

You are right. I just wanted to clearly describe the intentions I have.

I wonder if the --macros option (or its API equivalent in
include/rpm/rpmmacro.h) could be used then?

Now (I think) I understand how things work. And I believe yes. Even
though it is not convenient for use.

What I meant was macro configuration, i.e. --macros. That takes a
colon-separated list of paths and defaults to this (line breaks are
mine):

Is this "macro path" hard-coded into the RPM binary, or does it depend on
some configuration file, too? No matter which, the path seems to be
a distribution-specific list.

That's why I said you'd need to supply all the desired files yourself
when using this CLI option as there's no way to just prepend/append
paths to it.

This makes sense.

No matter if Mock or DNF or even RPM would be responsible for overriding
--macros when using --root, we need to be able detect the appropriate path
for the target chroot. Being distro-specific, we can't tell it in
advance. IOW we override host's RPM macro path with chroot's RPM macro
path.

Is there a way to tell the chroot's macro path? Could RPM bake it
somewhere as a data file ideally?

$ rootdir=$(mktemp -d)
dnf5 --installroot "$rootdir" install rpm-config --use-host-config
dnf5 --installroot "$rootdir" install @buildsys-build

Out of curiosity, though, does the second dnf5 command already work the
way you want, or was that just your proposal for how it could work?

The later. The second command still uses host's configuration.

If DNF already works this way, that means it does perform some kind of
RPM isolation by itself (by calling RPM in a chroot/namespace) and thus
adding an option like --use-installroot-config to DNF should be fairly
easy.

Unfortunately this is not the case. I'm afraid someone needs to design
config-override mechanism (and this ticket doesn't seem to be worst place
to do it :-)).

Just to illustrate better how one could use this (in order to achieve your original objective):

Thank you, I think the outer RPM wrapper could set --macros. The problem is
to minimize the guesswork, i.e. we shouldn't set --macros blindly. Ideas?

[dmnks]

I'll convert this into a Discussion, let's continue there. As soon as a concrete implementation emerges, we'll create appropriate tickets.

[dmnks]

FTR, I've stumbled upon these two recent PRs that somewhat extend the scope of --root and may act as some kind of precedent if we choose to involve macro configuration, too:

• Make user/group info reliable operation across chroot #2503
• Check inside --root when querying for files #2582

[pmatilai]

--macros is problematic because it requires rewriting the entire path, most of which is fairly critical to normal operation. Basically rpm must always initialize itself with the macros from the same version, otherwise there's weird stuff can happen. Which means, rpm from the host must initialize with the macros from the host.

For selective macro loading, --load=/some/path is much better alternative as this doesn't disrupt the normal macro path.

[Rtoax]

Thanks a lot, i just stuck on this for a long time. My build container /etc/rpm/macros.image-language-conf=%_install_langs en_US.UTF-8, The ISO i built with this container, it's Anaconda INSTALLATION screen only support English.

[yrro]

I think I've stumbled into this. I was trying to use a public GitHub Actions runner to create container images built from the ubi-micro images, which don't have dnf installed within.

I found that dnf --installroot=... install failed because the PGP keys it wanted weren't found in the RPM database. If I worked around that problem then I found that the the container's RPM database in /var/lib/rpm was not being updated by DNF. Instead, a new RPM database was created in /home/runner/.rpmdb! The ultimate cause of both problems was a a Debian patch which defines %_dbpath as ~/.rpmdb.

But this isn't specific to Debian either; the following is run on Fedora 39:

$ ctr=$(buildah from registry.access.redhat.com/ubi9/ubi-minimal)

$ mnt=$(buildah mount $ctr)

$ dnf --noplugins --releasever=9 --installroot="$mnt" list --installed | wc -l
0

$ chroot "$mnt" rpm -qa | wc -l
103

DNF sees no packages installed with the installroot, because the host configuration defines %_dbpath as /usr/lib/sysimage/rpm, again different from how it's defined in the ubi-micro container image.

I suppose what I've learned is that you can't really use a host's DNF/rpm to install into a chroot if the distro/config differs too much between the host and the chroot.