BLD Specify build time dependencies via pyproject.toml (scikit-learn#…

…16244)

* add pyproject.toml

* do it in CI

* cln

* no build isolation in the doc

* bump dependencies

* no-build-isolation everywhere needed

MANIFEST.in

@@ -5,3 +5,4 @@ recursive-include sklearn *.c *.h *.pyx *.pxd *.pxi *.tp
 recursive-include sklearn/datasets *.csv *.csv.gz *.rst *.jpg *.txt *.arff.gz *.json.gz
 include COPYING
 include README.rst
+include pyproject.toml

build_tools/azure/install.sh

@@ -90,7 +90,6 @@ elif [[ "$DISTRIB" == "conda-pip-latest" ]]; then
  # conda is still used as a convenient way to install Python and pip.
  make_conda "python=$PYTHON_VERSION"
  python -m pip install -U pip
- python -m pip install numpy scipy cython joblib
  python -m pip install pytest==$PYTEST_VERSION pytest-cov pytest-xdist
  python -m pip install pandas matplotlib pyamg
  # do not install dependencies for lightgbm since it requires scikit-learn
@@ -121,7 +120,15 @@ except ImportError:
 "
 python -m pip list
 
-# Use setup.py instead of `pip install -e .` to be able to pass the -j flag
-# to speed-up the building multicore CI machines.
-python setup.py build_ext --inplace -j 3
-python setup.py develop
+if [[ "$DISTRIB" == "conda-pip-latest" ]]; then
+ # Check that pip can automatically install the build dependencies from
+ # pyproject.toml using an isolated build environment:
+ pip install --verbose --editable .
+else
+ # Use the pre-installed build dependencies and build directly in the
+ # current environment.
+ # Use setup.py instead of `pip install -e .` to be able to pass the -j flag
+ # to speed-up the building multicore CI machines.
+ python setup.py build_ext --inplace -j 3
+ python setup.py develop
+fi

build_tools/circle/build_test_pypy.sh

@@ -34,7 +34,7 @@ export LOKY_MAX_CPU_COUNT="2"
 export OMP_NUM_THREADS="1"
 
 python setup.py build_ext --inplace -j 3
-pip install -e .
+pip install --no-build-isolation -e .
 
 # Check that Python implementation is PyPy
 python - << EOL

doc/developers/advanced_installation.rst

@@ -59,7 +59,7 @@ feature, code or documentation improvement).
 #. Install Cython_ and build the project with pip in :ref:`editable_mode`::
 
  pip install cython
- pip install --verbose --editable .
+ pip install --verbose --no-build-isolation --editable .
 
 #. Check that the installed scikit-learn has a version number ending with
  `.dev0`::
@@ -71,8 +71,11 @@ feature, code or documentation improvement).
 
 .. note::
 
- You will have to re-run the ``pip install --editable .`` command every time
- the source code of a Cython file is updated (ending in `.pyx` or `.pxd`).
+ You will have to run the ``pip install --no-build-isolation --editable .``
+ command every time the source code of a Cython file is updated
+ (ending in `.pyx` or `.pxd`). Use the ``--no-build-isolation`` flag to
+ avoid compiling the whole project each time, only the files you have
+ modified.
 
 Dependencies
 ------------
@@ -152,9 +155,9 @@ Editable mode
 
 If you run the development version, it is cumbersome to reinstall the package
 each time you update the sources. Therefore it is recommended that you install
-in with the ``pip install --editable .`` command, which allows you to edit the
-code in-place. This builds the extension in place and creates a link to the
-development directory (see `the pip docs
+in with the ``pip install --no-build-isolation --editable .`` command, which
+allows you to edit the code in-place. This builds the extension in place and
+creates a link to the development directory (see `the pip docs
 <https://pip.pypa.io/en/stable/reference/pip_install/#editable-installs>`_).
 
 This is fundamentally similar to using the command ``python setup.py develop``
@@ -207,7 +210,7 @@ environment variables in the current command prompt.
 
 Finally, build scikit-learn from this command prompt::
 
- pip install --verbose --editable .
+ pip install --verbose --no-build-isolation --editable .
 
 .. _compiler_macos:
 
@@ -240,7 +243,7 @@ scikit-learn from source::
  "conda-forge::compilers>=1.0.4" conda-forge::llvm-openmp
  conda activate sklearn-dev
  make clean
- pip install --verbose --editable .
+ pip install --verbose --no-build-isolation --editable .
 
 .. note::
 
@@ -302,7 +305,7 @@ Finally, build scikit-learn in verbose mode (to check for the presence of the
 ``-fopenmp`` flag in the compiler commands)::
 
  make clean
- pip install --verbose --editable .
+ pip install --verbose --no-build-isolation --editable .
 
 .. _compiler_linux:
 
@@ -351,7 +354,7 @@ in the user folder using conda::
 
  conda create -n sklearn-dev numpy scipy joblib cython conda-forge::compilers
  conda activate sklearn-dev
- pip install --verbose --editable .
+ pip install --verbose --no-build-isolation --editable .
 
 .. _compiler_freebsd:
 
@@ -374,7 +377,7 @@ can set the environment variables to these locations::
 
 Finally, build the package using the standard command::
 
- pip install --verbose --editable .
+ pip install --verbose --no-build-isolation --editable .
 
 For the upcoming FreeBSD 12.1 and 11.3 versions, OpenMP will be included in
 the base system and these steps will not be necessary.

doc/developers/contributing.rst

@@ -219,7 +219,7 @@ how to set up your git repository:
 
 5. Install scikit-learn in editable mode::
 
- $ pip install --editable .
+ $ pip install --no-build-isolation --editable .
 
  for more details about advanced installation, see the
  :ref:`install_bleeding_edge` section.
@@ -266,8 +266,13 @@ modifying code and submitting a PR:
 
 .. note::
 
- If you are modifying a Cython module, you have to re-run step 5 after modifications
- and before testing them.
+ If you are modifying a Cython module, you have to re-compile after
+ modifications and before testing them::
+
+ pip install --no-build-isolation -e .
+
+ Use the ``--no-build-isolation`` flag to avoid compiling the whole project
+ each time, only the files you have modified.
 
 It is often helpful to keep your local feature branch synchronized with the
 latest changes of the main scikit-learn repository::

pyproject.toml

@@ -0,0 +1,9 @@
+[build-system]
+# Minimum requirements for the build system to execute.
+requires = [
+ "setuptools",
+ "wheel",
+ "Cython>=0.28.5",
+ "numpy>=1.13.3",
+ "scipy>=0.19.1",
+]