Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add rapidsai (cudf and cuml) #595

Merged
merged 3 commits into from
Oct 16, 2019
Merged

Add rapidsai (cudf and cuml) #595

merged 3 commits into from
Oct 16, 2019

Conversation

jperez999
Copy link
Contributor

No description provided.

@jperez999
Copy link
Contributor Author

To run jupyterlabs in the container and have access to the gpu in the notebooks I had to use the following docker run command:

docker run --runtime nvidia --rm -it -e HOME=/tmp -w=/tmp -v /tmp/empty_dir:/usr/local/cuda/lib64/stubs:ro -p 8888:8888 -p 8787:8787 -p 8786:8786 kaggle/python-gpu-build

@Philmod
Copy link
Contributor

Philmod commented Oct 12, 2019

The CI tests fail with:

======================================================================
ERROR: test_cudf (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_cudf
Traceback (most recent call last):
  File "/opt/conda/lib/python3.6/unittest/loader.py", line 428, in _find_test_path
    module = self._get_module_from_name(name)
  File "/opt/conda/lib/python3.6/unittest/loader.py", line 369, in _get_module_from_name
    __import__(name)
  File "/input/tests/test_cudf.py", line 2, in <module>
    import cudf as gd
ModuleNotFoundError: No module named 'cudf'


======================================================================
ERROR: test_cuml (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_cuml
Traceback (most recent call last):
  File "/opt/conda/lib/python3.6/unittest/loader.py", line 428, in _find_test_path
    module = self._get_module_from_name(name)
  File "/opt/conda/lib/python3.6/unittest/loader.py", line 369, in _get_module_from_name
    __import__(name)
  File "/input/tests/test_cuml.py", line 4, in <module>
    from cuml import LinearRegression as cuLinearRegression
ModuleNotFoundError: No module named 'cuml'

I guess because the import at the beginning of the test files are not available for CPU tests.

@Philmod Philmod self-requested a review October 12, 2019 20:56
@jperez999
Copy link
Contributor Author

Moved GPU specific imports into the GPU tests themselves to resolve import failures on cpu container. We also cannot incorporate newer version of rapids until pandas is updated >=0.24.2, <0.25.

import unittest

from sklearn import datasets
from cuml import LinearRegression as cuLinearRegression
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may have forgotten to remove these?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your right I will update that.

@Philmod
Copy link
Contributor

Philmod commented Oct 16, 2019

We also cannot incorporate newer version of rapids until pandas is updated >=0.24.2, <0.25.

Is this blocking this PR?

@jperez999
Copy link
Contributor Author

No rapids version 0.8 works fine. However, the most recent released version is 0.9 with 0.10 coming soon. Those versions of rapids will be blocked by that pandas requirement (>=0.24.2, <0.25).

@Philmod
Copy link
Contributor

Philmod commented Oct 16, 2019

Thank you for your contribution!

@Philmod Philmod merged commit f84f305 into Kaggle:master Oct 16, 2019
@Philmod
Copy link
Contributor

Philmod commented Oct 16, 2019

Unfortunately, we are moving to tensorflow 2 and it looks like the library is not supported. https://github.com/Kaggle/docker-python/pull/573/files

rosbo added a commit that referenced this pull request Oct 16, 2019
This reverts commit f84f305, reversing
changes made to e2f1f7b.
rosbo added a commit that referenced this pull request Oct 16, 2019
Revert "Merge pull request #595 from jperez999/master"
@rosbo rosbo mentioned this pull request Jun 4, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants