rpm2archive -f pax cannot handle utf8 filenames

[mlschroe]

It fails because it want to convert the filenames to utf8:

$ echo $LC_CTYPE
de_DE@euro
$ rpm2archive /usr/src/packages/RPMS/x86_64/empty-3.0.0-1.x86_64.rpm > /dev/null
Error writing archive: Can't translate pathname './fooöo' to UTF-8 (84)

[mlschroe]

In this case archive_write_header() returns ARCHIVE_WARN, which is treated as error in rpm2archive. OTOH I don't think libarchive should mess with the file names, maybe it makes sense to set the hdrcharset to BINARY for pax. But that adds a "hdrcharset=BINARY" attribute that GNU tar complains about.

[mlschroe]

Oh, and the error handling in rpm2archive is completely broken...

[mlschroe]

Btw, it cannot handle UTF8 filenames as well, as it checks the current locale which is not initialized and thus 7 bit ascii...

[mlschroe]

I find it very surprising that bsdtar's output depends on the current locale, but that seems to be the case:

$ echo hello > micro_µ
$ bsdtar -cf - . | bsdtar -tf -
./
./micro_µ
$ LC_CTYPE=de_DE@euro bsdtar -cf - . | bsdtar -tf -
./
./micro_µ
$ LC_CTYPE=de_DE@euro bsdtar --options hdrcharset=BINARY -cf - . | bsdtar -tf -
./
./micro_µ

[mlschroe]

It's not much work to not use libarchive for writing. The only two formats that can be used for archive writing are cpio and pax (all the others have too many limitations). Writing cpio is easy and writing a pax tar file is also not hard (reading a tar file is where it gets really messy because of all the different implementations).

Is that something you would be interested in?

[pmatilai]

rpm2archive could use some love for sure, but I'd rather not teach it about format internals, that's just the kind of thing I'd rather outsource to somebody else - like libarchive. If it doesn't do what we want it to do, then lets at least look at fixing it instead of doing it in rpm, it'd benefit way more people too.

As for the encoding, I think here's a case for RPMTAG_ENCODING: if that's present and says utf-8 (upstream rpm will never put anything else there) we can safely assume utf-8. Anything else is a legacy case and if the easiest solution is to just say "BINARY" encoding then that's fine with me. Or does "GNU tar complains" mean it actually fails entirely rather than just warn?

[mlschroe]

It just warns about the unknown attribute.

[mlschroe]

And this is about file names, I think "upstream rpm" treats those pretty much as binary as they are created by the build process and not part of the spec file.

[pmatilai]

Right. So the warning would only be seen by folks who run rpm2archive to convert legacy rpms to tar - ie a rare corner case really. A harmless warning from gnu tar in that case is quite acceptable to me at least.

[mlschroe]

Why "legacy"? Does the current code reject non-utf8 file names?

[pmatilai]

It does, by default. For many years now.

Looking closer: we turned it into an error five years ago, before that it was a warning for a similar period of time. It's still macro overridable for of course for v4 packages.

[mlschroe]

That makes things a bit easier, so we just need to teach libarchive that it should accept utf8. I'll adapt the title of this issue ;-)

[mlschroe]

I'll open a pull request for this.