Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Issue with opus-fast-mosestokenizer dep for ARM-macs #69

Open
rggdmonk opened this issue Feb 20, 2024 · 3 comments
Open

Issue with opus-fast-mosestokenizer dep for ARM-macs #69

rggdmonk opened this issue Feb 20, 2024 · 3 comments
Assignees
@svirpioj

Comments

@rggdmonk
Copy link

Hi!

I am currently experiencing an issue with the installation on M-series Macs (via pip install opusfilter).

Unfortunately, the installation process fails due to a problem with the opus-fast-mosestokenizer dependency.

Tested with Python 3.8/3.9/3.10/3.11

Some output:

  Building wheel for opus-fast-mosestokenizer (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [117 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.macosx-11.1-arm64-cpython-311
      creating build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer
      copying bindings/python/mosestokenizer/__init__.py -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer
      creating build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share
      copying bindings/python/mosestokenizer/share/README.txt -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share
      creating build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.hi -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.as -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.de -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.nl -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.hu -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.bn -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.kn -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.ml -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.yue -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.mni -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.pa -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.pt -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.zh -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.ta -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.en -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.lt -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.or -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.fr -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.es -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.et -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.fi -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.gu -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.mr -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.ga -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.sv -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.pl -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.sl -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.sk -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/README.txt -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.ro -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.te -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.ru -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.cs -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.it -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.is -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.ca -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.el -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      copying bindings/python/mosestokenizer/share/nonbreaking_prefixes/nonbreaking_prefix.lv -> build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/share/nonbreaking_prefixes
      running build_ext
      CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
        Compatibility with CMake < 3.5 will be removed from a future version of
        CMake.
      
        Update the VERSION argument <min> value or use a ...<max> suffix to tell
        CMake that the project does not need compatibility with older versions.
      
      
      -- The CXX compiler identification is AppleClang 15.0.0.15000100
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
      -- Found Threads: TRUE
      CMake Error at /opt/homebrew/Cellar/cmake/3.28.3/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
        Could NOT find Boost (missing: Boost_INCLUDE_DIR program_options thread)
      Call Stack (most recent call first):
        /opt/homebrew/Cellar/cmake/3.28.3/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
        /opt/homebrew/Cellar/cmake/3.28.3/share/cmake/Modules/FindBoost.cmake:2393 (find_package_handle_standard_args)
        CMakeLists.txt:70 (find_package)
      
      
      -- Configuring incomplete, errors occurred!
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/var/folders/tg/b_nprk2142s5bh07pw240f3r0000gp/T/pip-install-sd9_6q0a/opus-fast-mosestokenizer_86290359b51c412aaad6b283997c5351/setup.py", line 107, in <module>
          setup(
        File "/Users/pstepach/miniconda3/envs/clear/lib/python3.11/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/pstepach/miniconda3/envs/clear/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/Users/pstepach/miniconda3/envs/clear/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/Users/pstepach/miniconda3/envs/clear/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/Users/pstepach/miniconda3/envs/clear/lib/python3.11/site-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/Users/pstepach/miniconda3/envs/clear/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/Users/pstepach/miniconda3/envs/clear/lib/python3.11/site-packages/wheel/bdist_wheel.py", line 364, in run
          self.run_command("build")
        File "/Users/pstepach/miniconda3/envs/clear/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/Users/pstepach/miniconda3/envs/clear/lib/python3.11/site-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/Users/pstepach/miniconda3/envs/clear/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/Users/pstepach/miniconda3/envs/clear/lib/python3.11/site-packages/setuptools/_distutils/command/build.py", line 131, in run
          self.run_command(cmd_name)
        File "/Users/pstepach/miniconda3/envs/clear/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/Users/pstepach/miniconda3/envs/clear/lib/python3.11/site-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/Users/pstepach/miniconda3/envs/clear/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/private/var/folders/tg/b_nprk2142s5bh07pw240f3r0000gp/T/pip-install-sd9_6q0a/opus-fast-mosestokenizer_86290359b51c412aaad6b283997c5351/setup.py", line 53, in run
          self.build_extension(ext)
        File "/private/var/folders/tg/b_nprk2142s5bh07pw240f3r0000gp/T/pip-install-sd9_6q0a/opus-fast-mosestokenizer_86290359b51c412aaad6b283997c5351/setup.py", line 91, in build_extension
          subprocess.check_call(cmake_args, cwd=self.build_temp, env=env)
        File "/Users/pstepach/miniconda3/envs/clear/lib/python3.11/subprocess.py", line 413, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['cmake', '/private/var/folders/tg/b_nprk2142s5bh07pw240f3r0000gp/T/pip-install-sd9_6q0a/opus-fast-mosestokenizer_86290359b51c412aaad6b283997c5351', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/private/var/folders/tg/b_nprk2142s5bh07pw240f3r0000gp/T/pip-install-sd9_6q0a/opus-fast-mosestokenizer_86290359b51c412aaad6b283997c5351/build/lib.macosx-11.1-arm64-cpython-311/mosestokenizer/lib/', '-DPYTHON_EXECUTABLE=/Users/pstepach/miniconda3/envs/clear/bin/python', '-DBUILD_CLI:BOOL=OFF', '-DBUILD_PYTHON:BOOL=ON', '-DBUILD_SHARED_LIBS:BOOL=ON', '-DCMAKE_BUILD_TYPE=Release']' returned non-zero exit status 1.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for opus-fast-mosestokenizer
  Running setup.py clean for opus-fast-mosestokenizer
  Building wheel for langid (setup.py) ... done
  Created wheel for langid: filename=langid-1.1.6-py3-none-any.whl size=1941173 sha256=fb9479a4d2fdb43a32e92a248da54a802ec26086bea4c2a25ba98c612f77ac7a
  Stored in directory: /Users/pstepach/Library/Caches/pip/wheels/32/6a/b6/b7eb43a6ad55b139c15c5daa29f3707659cfa6944d3c696f5b
  Building wheel for pycld2 (setup.py) ... done
  Created wheel for pycld2: filename=pycld2-0.41-cp311-cp311-macosx_11_0_arm64.whl size=4816791 sha256=16ddfa06735a6e70169b323d9f67f81eebbee850b8fa524a5a8aa9d87140c69c
  Stored in directory: /Users/pstepach/Library/Caches/pip/wheels/9a/71/a1/89cdc6ae2ecd7bb3bf67baee27a134f761b3e2922ea7fdfbcb
Successfully built langid pycld2
Failed to build opus-fast-mosestokenizer
ERROR: Could not build wheels for opus-fast-mosestokenizer, which is required to install pyproject.toml-based projects```
@svirpioj svirpioj self-assigned this Feb 28, 2024
@svirpioj
Copy link
Member

svirpioj commented Feb 28, 2024 •
edited
Loading

There was a pending pull request on opus-fast-mosestokenizer related to this. It's now merged to the master branch, can you test if the change helps?

@rggdmonk
Copy link
Author

rggdmonk commented Mar 4, 2024

Sure. I plan to test it later this week.

@rggdmonk
Copy link
Author

rggdmonk commented Mar 11, 2024 •
edited
Loading

Still issues. I tried https://github.com/Helsinki-NLP/opus-fast-mosestokenizer/blob/master/INSTALL.md#from-source

(arm_fix) ➜  opus-fast-mosestokenizer git:(master) python -c "import pybind11; print(pybind11.get_cmake_dir())"
/Users/pstepach/miniconda3/envs/arm_fix/lib/python3.10/site-packages/pybind11/share/cmake/pybind11

opus-fast-mosestokenizer git:(master) python setup.py build_ext install
running build_ext
CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


CMake Error at CMakeLists.txt:154 (find_package):
  By not providing "Findpybind11.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "pybind11",
  but CMake did not find one.

  Could not find a package configuration file provided by "pybind11" with any
  of the following names:

    pybind11Config.cmake
    pybind11-config.cmake

  Add the installation prefix of "pybind11" to CMAKE_PREFIX_PATH or set
  "pybind11_DIR" to a directory containing one of the above files.  If
  "pybind11" provides a separate development package or SDK, be sure it has
  been installed.


-- Configuring incomplete, errors occurred!
Traceback (most recent call last):
  File "/Users/pstepach/Desktop/WORKING_FOLDER/arm_fix/opus-fast-mosestokenizer/setup.py", line 107, in <module>
    setup(
  File "/Users/pstepach/miniconda3/envs/arm_fix/lib/python3.10/site-packages/setuptools/__init__.py", line 103, in setup
    return distutils.core.setup(**attrs)
  File "/Users/pstepach/miniconda3/envs/arm_fix/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
    return run_commands(dist)
  File "/Users/pstepach/miniconda3/envs/arm_fix/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
    dist.run_commands()
  File "/Users/pstepach/miniconda3/envs/arm_fix/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
    self.run_command(cmd)
  File "/Users/pstepach/miniconda3/envs/arm_fix/lib/python3.10/site-packages/setuptools/dist.py", line 989, in run_command
    super().run_command(command)
  File "/Users/pstepach/miniconda3/envs/arm_fix/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/Users/pstepach/Desktop/WORKING_FOLDER/arm_fix/opus-fast-mosestokenizer/setup.py", line 53, in run
    self.build_extension(ext)
  File "/Users/pstepach/Desktop/WORKING_FOLDER/arm_fix/opus-fast-mosestokenizer/setup.py", line 91, in build_extension
    subprocess.check_call(cmake_args, cwd=self.build_temp, env=env)
  File "/Users/pstepach/miniconda3/envs/arm_fix/lib/python3.10/subprocess.py", line 369, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '/Users/pstepach/Desktop/WORKING_FOLDER/arm_fix/opus-fast-mosestokenizer', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/Users/pstepach/Desktop/WORKING_FOLDER/arm_fix/opus-fast-mosestokenizer/build/lib.macosx-11.1-arm64-cpython-310/mosestokenizer/lib/', '-DPYTHON_EXECUTABLE=/Users/pstepach/miniconda3/envs/arm_fix/bin/python', '-DBUILD_CLI:BOOL=OFF', '-DBUILD_PYTHON:BOOL=ON', '-DBUILD_SHARED_LIBS:BOOL=ON', '-DCMAKE_BUILD_TYPE=Release']' returned non-zero exit status 1.```

@svirpioj svirpioj mentioned this issue Jun 12, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@svirpioj svirpioj

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@svirpioj @rggdmonk