ifcopenshell: 240611 -> 0.7.10, fix build and activate most tests #312381

autra · 2024-05-17T07:21:14Z

Description of changes

Fix the ifcopenshell build.

Fix #312380

Things done

Add a 👍 reaction to pull requests you find important.

pkgs/development/python-modules/ifcopenshell/default.nix

autra · 2024-05-17T20:12:42Z

Ok now it's compiling, but I have a weird problem with the runtime lib. libxml2 is missing at runtime:

 ❯ ldd result/bin/IfcConvert
        linux-vdso.so.1 (0x00007ffed5cc0000)
        libTKernel.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKernel.so.7 (0x00007f73733f4000)
        libTKMath.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKMath.so.7 (0x00007f7373000000)
        libTKBRep.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKBRep.so.7 (0x00007f73732c5000)
        libTKGeomBase.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKGeomBase.so.7 (0x00007f7372a00000)
        libTKGeomAlgo.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKGeomAlgo.so.7 (0x00007f7372400000)
        libTKG3d.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKG3d.so.7 (0x00007f7372ed8000)
        libTKG2d.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKG2d.so.7 (0x00007f73729a1000)
        libTKShHealing.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKShHealing.so.7 (0x00007f7372000000)
        libTKTopAlgo.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKTopAlgo.so.7 (0x00007f7371c00000)
        libTKMesh.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKMesh.so.7 (0x00007f73722ed000)
        libTKPrim.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKPrim.so.7 (0x00007f737293a000)
        libTKBool.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKBool.so.7 (0x00007f7371600000)
        libTKBO.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKBO.so.7 (0x00007f7371200000)
        libTKFillet.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKFillet.so.7 (0x00007f7370e00000)
        libTKSTEP.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKSTEP.so.7 (0x00007f7370a00000)
        libTKSTEPBase.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKSTEPBase.so.7 (0x00007f7370600000)
        libTKSTEPAttr.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKSTEPAttr.so.7 (0x00007f7371a39000)
        libTKXSBase.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKXSBase.so.7 (0x00007f7370200000)
        libTKSTEP209.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKSTEP209.so.7 (0x00007f7371f57000)
        libTKIGES.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKIGES.so.7 (0x00007f736fc00000)
        libTKOffset.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKOffset.so.7 (0x00007f737004c000)
        libTKHLR.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKHLR.so.7 (0x00007f73710d9000)
        libTKBin.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKBin.so.7 (0x00007f7373290000)
        libhdf5_cpp.so.310 => /nix/store/l34mr0cav6k6798gfx99hz02w5wwv1as-hdf5-cpp-1.14.3/lib/libhdf5_cpp.so.310 (0x00007f7371580000)
        libhdf5.so.310 => /nix/store/l34mr0cav6k6798gfx99hz02w5wwv1as-hdf5-cpp-1.14.3/lib/libhdf5.so.310 (0x00007f736f600000)
        libz.so.1 => /nix/store/zph9xw0drmq3rl2ik5slg0n2frw9lw5m-zlib-1.3.1/lib/libz.so.1 (0x00007f7372eba000)
        libboost_system.so.1.79.0 => /nix/store/mcj6hbf8amfq4ii6kqi510d43xgjq6as-boost-1.79.0/lib/libboost_system.so.1.79.0 (0x00007f7373289000)
        libboost_program_options.so.1.79.0 => /nix/store/mcj6hbf8amfq4ii6kqi510d43xgjq6as-boost-1.79.0/lib/libboost_program_options.so.1.79.0 (0x00007f7371eeb000)
        libboost_regex.so.1.79.0 => /nix/store/mcj6hbf8amfq4ii6kqi510d43xgjq6as-boost-1.79.0/lib/libboost_regex.so.1.79.0 (0x00007f737152f000)
        libboost_thread.so.1.79.0 => /nix/store/mcj6hbf8amfq4ii6kqi510d43xgjq6as-boost-1.79.0/lib/libboost_thread.so.1.79.0 (0x00007f737291c000)
        libboost_date_time.so.1.79.0 => /nix/store/mcj6hbf8amfq4ii6kqi510d43xgjq6as-boost-1.79.0/lib/libboost_date_time.so.1.79.0 (0x00007f7372917000)
        libxml2.so.2 => not found
        libpcre.so.1 => /nix/store/pq0cj8f364jvv16yqvkc43cz2vwcwg03-pcre-8.45/lib/libpcre.so.1 (0x00007f7370d86000)
        libstdc++.so.6 => /nix/store/c2yb135iv4maadia5f760b3xhbh6jh61-gcc-13.2.0-lib/lib/libstdc++.so.6 (0x00007f736f200000)
        libm.so.6 => /nix/store/ddwyrxif62r8n6xclvskjyy6szdhvj60-glibc-2.39-5/lib/libm.so.6 (0x00007f737091d000)
        libgcc_s.so.1 => /nix/store/c2yb135iv4maadia5f760b3xhbh6jh61-gcc-13.2.0-lib/lib/libgcc_s.so.1 (0x00007f73722c8000)
        libc.so.6 => /nix/store/ddwyrxif62r8n6xclvskjyy6szdhvj60-glibc-2.39-5/lib/libc.so.6 (0x00007f736f013000)
        /nix/store/ddwyrxif62r8n6xclvskjyy6szdhvj60-glibc-2.39-5/lib/ld-linux-x86-64.so.2 => /nix/store/anlf335xlh41yjhm114swi87406mq5pw-glibc-2.38-44/lib64/ld-linux-x86-64.so.2 (0x00007f73735e5000)
        libpthread.so.0 => /nix/store/ddwyrxif62r8n6xclvskjyy6szdhvj60-glibc-2.39-5/lib/libpthread.so.0 (0x00007f7372910000)
        librt.so.1 => /nix/store/ddwyrxif62r8n6xclvskjyy6szdhvj60-glibc-2.39-5/lib/librt.so.1 (0x00007f737290b000)
        libdl.so.2 => /nix/store/ddwyrxif62r8n6xclvskjyy6szdhvj60-glibc-2.39-5/lib/libdl.so.2 (0x00007f7372904000)
        libTKCAF.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKCAF.so.7 (0x00007f7370556000)
        libTKBinL.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKBinL.so.7 (0x00007f73708bf000)
        libTKLCAF.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKLCAF.so.7 (0x00007f736fb04000)
        libTKCDF.so.7 => /nix/store/p2y06mqk94i5sapgja08fvf9g5lzabxb-opencascade-occt-7.6.2/lib/libTKCDF.so.7 (0x00007f73704f7000)
        libicudata.so.73 => /nix/store/qwnmd1a9zxfjqgkhv3jyyf86r9jm4gsq-icu4c-73.2/lib/libicudata.so.73 (0x00007f736d000000)
        libicui18n.so.73 => /nix/store/qwnmd1a9zxfjqgkhv3jyyf86r9jm4gsq-icu4c-73.2/lib/libicui18n.so.73 (0x00007f736cc00000)
        libicuuc.so.73 => /nix/store/qwnmd1a9zxfjqgkhv3jyyf86r9jm4gsq-icu4c-73.2/lib/libicuuc.so.73 (0x00007f736c800000)

I have no idea how to fix this problem, for me buildInputs was supposed to do the job (and it does so for opencascade)... @fehnomenal any idea?

nixos-discourse · 2024-05-21T06:32:29Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/fixing-ifcopenshell-libxml2-not-found-by-linker-at-runtime/45764/1

autra · 2024-05-22T08:52:09Z

Ok so I fixed my libxml2 issue by making this a regular package and not a python package (see last temp commit). So it's something to do with the way python packages are made.

Can I - and should I - split the lib and the python packaging into 2 derivations? That would allow freecad to depend only on one of them for instance.

autra · 2024-05-22T08:55:44Z

That being said, my end goal is freecad ifc support, and apparently this needs python bindings, so I might not personnally need the split.

autra · 2024-05-23T08:27:47Z

I feel that I need mkDerivation (because it executes the patchElf phase) and buildPythonPackage. How can I combine the 2?

autra · 2024-07-23T16:46:23Z

Result of nixpkgs-review pr 312381 run on x86_64-linux 1

2 packages failed to build:

python311Packages.ifcopenshell
python312Packages.ifcopenshell

autra · 2024-07-26T14:33:35Z

Result of nixpkgs-review pr 312381 run on x86_64-linux 1

autra · 2024-07-26T14:51:07Z

pkgs/top-level/all-packages.nix

@@ -8949,8 +8949,6 @@ with pkgs;

 idle3tools = callPackage ../tools/system/idle3tools { };

- ifcopenshell = with python3Packages; toPythonApplication ifcopenshell;


my understanding is that this is useful if python provides some cli tools. This is not the case here.

There are some binaries in $out/bin though. I believe that to make things properly, these should be in another derivation in libraries/, with python build disabled. Or maybe this just uses the current default python3 version and in this case I should keep this line?

autra · 2024-07-26T14:51:34Z

pkgs/top-level/python-packages.nix

@@ -5816,7 +5816,9 @@ self: super: with self; {

 ifconfig-parser = callPackage ../development/python-modules/ifconfig-parser { };

- ifcopenshell = callPackage ../development/python-modules/ifcopenshell { };
+ ifcopenshell = callPackage ../development/python-modules/ifcopenshell {
+ inherit (pkgs) libxml2;


I have an error about libxml2 missing if I don't add this line, no ideas why.

autra · 2024-07-26T14:56:52Z

I believe this is now ready. I still have some questions though :-) The main one is whether or not I should keep it in "all-packages.nix"

Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage

For some reason, this script says "nothing to build". As the build is broken since ages, I believe we don't really need to check dependent packages ;-)

nixos-discourse · 2024-07-26T18:26:11Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/fixing-ifcopenshell-libxml2-not-found-by-linker-at-runtime/45764/3

fehnomenal

Thanks for tackling this. I know building IfcOpenShell is not easy.

nixpkgs-review pr 312381 also displayed "nothing to build for me" but dropped me inside a shell where I could build the derivation with nix build ./nixpkgs#python3.pkgs.ifcopenshell.

The executables (IfcConvert, IfcGeomServer and svgfill) seem to work or at least don't crash unexpectedly when provided no real input. I no longer work with IFC so I have no way to test with real data. This is also way I would like to step down as maintainer. Do you want to take over, @autra?

I'm a bit confused about the versioning. The fetched code seems to be from v0.7.10 but you reference a v0.8.0 Makefile in the comments (which is probably OK as it builds and the executables seem to run). The built IfcConvert also reports itself as 0.8.0. What is going on here?

Otherwise lgtm

autra · 2024-07-29T08:58:14Z

Thanks for tackling this. I know building IfcOpenShell is not easy.

indeed 😅

Do you want to take over, @autra?

I can take over indeed, as I will be working with ifc files at least for a year (maybe longer).

I'm a bit confused about the versioning. The fetched code seems to be from v0.7.10 but you reference a v0.8.0 Makefile in the comments (which is probably OK as it builds and the executables seem to run). The built IfcConvert also reports itself as 0.8.0. What is going on here?

Honestly... me too. ifcopenshell seems to have 3 parallel versioning systems: blenderbim (the blender plugin), ifcopenshell-python and ifcopenshell alone. As I need python binding, I use ifcopenshell-python tags, and in the last tag, the native part is already bumped as 0.8.0.

Their release process looks very odd to me. It seems that they are constructing a wheel by downloading the previous release build from s3 and copying the so and wrapper.py from there. This sounds like a very very bad practice security-wise (who is uploading to this s3 bucket?), and it's really asking for a xy-type of security issue if you ask me!

So my understanding is that they "propagate" the .so (and wrapper.py) from the latest v0.x.0 tag to each intermediate python release. If we want to mimic this behaviour, we should create a native ifcopenshell derivation built from this tag, then copy the .so from this to each python3xxPackages.ifcopenshell. The problem is that we then have a python version mismatch problem that I didn't manage to solve because the native part still needs python to build its .so and wrapper.py.

I guess we should suggest something saner upstream, but I'm not even sure what.

For now, I've decided to take everything from the python tag, but I can try again the split between native package and python packages if you have a solution to my python version mismatch issue.

What do you think about all that?

autra · 2024-07-30T07:41:34Z

This issue : IfcOpenShell/IfcOpenShell#3927 actually talks about that.

autra · 2024-08-02T16:21:37Z

I've started to see what I can contribute upstream to make things easier. In the meantime, I'd say we can merge this and iterate from this.

@fehnomenal I've added a commit that changes the maintainer.

By relevant release, I mean the ifcopenshell-python release, and not the blenderbim release

autra · 2024-08-02T16:25:08Z

I fixed a nixf-tidy error and added opencollada as buildInputs.

fehnomenal · 2024-08-03T14:41:12Z

Thanks for taking the time to investigate all of this and also thanks for replacing me 👍🚀

github-actions bot added the 6.topic: python label May 17, 2024

NixOSInfra added the 12. first-time contribution label May 17, 2024

ofborg bot requested a review from fehnomenal May 17, 2024 07:47

ofborg bot added 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels May 17, 2024

autra force-pushed the fix_ifcopenshell_build branch from 2ae653e to 94650c5 Compare May 17, 2024 17:20

autra commented May 17, 2024

View reviewed changes

pkgs/development/python-modules/ifcopenshell/default.nix Outdated Show resolved Hide resolved

autra force-pushed the fix_ifcopenshell_build branch from 94650c5 to 350b351 Compare May 21, 2024 06:25

ofborg bot added the 2.status: merge conflict label May 22, 2024

autra force-pushed the fix_ifcopenshell_build branch from fa560fb to 192e18c Compare June 19, 2024 06:27

autra mentioned this pull request Jun 19, 2024

python3Packages.mathutils: init at 3.3.0 #320946

Merged

13 tasks

autra force-pushed the fix_ifcopenshell_build branch from 192e18c to 689d08e Compare June 20, 2024 17:13

github-actions bot added the 8.has: maintainer-list (update) label Jun 20, 2024

ofborg bot added 8.has: package (new) and removed 2.status: merge conflict labels Jun 20, 2024

autra force-pushed the fix_ifcopenshell_build branch from 689d08e to f0b8a76 Compare July 2, 2024 09:09

github-actions bot removed the 8.has: maintainer-list (update) label Jul 2, 2024

autra mentioned this pull request Jul 2, 2024

ifcopenshell: 210410 -> 0.7.0a9 #217656

Draft

12 tasks

wegank added the 2.status: merge conflict label Jul 6, 2024

autra force-pushed the fix_ifcopenshell_build branch from f0b8a76 to af35a90 Compare July 10, 2024 17:11

ofborg bot added 8.has: clean-up and removed 2.status: merge conflict labels Jul 10, 2024

autra force-pushed the fix_ifcopenshell_build branch from af35a90 to 67d241e Compare July 10, 2024 18:28

autra force-pushed the fix_ifcopenshell_build branch from 67d241e to 640e0fc Compare July 23, 2024 16:36

ofborg bot added 10.rebuild-linux: 0 and removed 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Jul 26, 2024

autra force-pushed the fix_ifcopenshell_build branch from 918295b to aaba05e Compare July 26, 2024 14:29

autra commented Jul 26, 2024

View reviewed changes

autra mentioned this pull request Jul 26, 2024

freecad: add ifc support #330190

Draft

13 tasks

autra changed the title ~~WIP fix ifcopenshell build~~ ifcopenshell: 240611 -> 0.7.10, fix build and activate most tests Jul 26, 2024

autra marked this pull request as ready for review July 26, 2024 14:55

autra requested a review from natsukium as a code owner July 26, 2024 14:55

fehnomenal approved these changes Jul 26, 2024

View reviewed changes

autra force-pushed the fix_ifcopenshell_build branch from aaba05e to b4001e6 Compare August 2, 2024 16:20

autra requested a review from fehnomenal August 2, 2024 16:21

autra added 2 commits August 2, 2024 18:23

ifcopenshell: 240611 -> 0.7.10, fix build and activate most tests

20db8bc

By relevant release, I mean the ifcopenshell-python release, and not the blenderbim release

ifcopenshell: maintainer fehnomenal -> autra, as asked by fehnomenal

3da10a1

autra force-pushed the fix_ifcopenshell_build branch from b4001e6 to 3da10a1 Compare August 2, 2024 16:23

ofborg bot added 11.by: package-maintainer 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 and removed 10.rebuild-darwin: 0 10.rebuild-linux: 0 labels Aug 2, 2024

fehnomenal approved these changes Aug 3, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ifcopenshell: 240611 -> 0.7.10, fix build and activate most tests #312381

ifcopenshell: 240611 -> 0.7.10, fix build and activate most tests #312381

autra commented May 17, 2024 •

edited

Loading

autra commented May 17, 2024

nixos-discourse commented May 21, 2024

autra commented May 22, 2024

autra commented May 22, 2024

autra commented May 23, 2024

autra commented Jul 23, 2024

autra commented Jul 26, 2024

autra Jul 26, 2024

autra Jul 26, 2024

autra commented Jul 26, 2024

nixos-discourse commented Jul 26, 2024

fehnomenal left a comment

autra commented Jul 29, 2024

autra commented Jul 30, 2024

autra commented Aug 2, 2024

autra commented Aug 2, 2024

fehnomenal commented Aug 3, 2024

		@@ -8949,8 +8949,6 @@ with pkgs;

		idle3tools = callPackage ../tools/system/idle3tools { };

		ifcopenshell = with python3Packages; toPythonApplication ifcopenshell;

ifcopenshell: 240611 -> 0.7.10, fix build and activate most tests #312381

Are you sure you want to change the base?

ifcopenshell: 240611 -> 0.7.10, fix build and activate most tests #312381

Conversation

autra commented May 17, 2024 • edited Loading

Description of changes

Things done

autra commented May 17, 2024

nixos-discourse commented May 21, 2024

autra commented May 22, 2024

autra commented May 22, 2024

autra commented May 23, 2024

autra commented Jul 23, 2024

autra commented Jul 26, 2024

autra Jul 26, 2024

Choose a reason for hiding this comment

autra Jul 26, 2024

Choose a reason for hiding this comment

autra commented Jul 26, 2024

nixos-discourse commented Jul 26, 2024

fehnomenal left a comment

Choose a reason for hiding this comment

autra commented Jul 29, 2024

autra commented Jul 30, 2024

autra commented Aug 2, 2024

autra commented Aug 2, 2024

fehnomenal commented Aug 3, 2024

autra commented May 17, 2024 •

edited

Loading