Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dependencies & compilations on Google colab #76

Open
shon-otmazgin opened this issue Feb 26, 2021 · 6 comments
Open

Dependencies & compilations on Google colab #76

shon-otmazgin opened this issue Feb 26, 2021 · 6 comments

Comments

@shon-otmazgin
Copy link

Hello,
I tried to ran the following google colab:
https://colab.research.google.com/drive/19nnb2V_p434g9QfV7ILSS6lMr9Jbu8tV#scrollTo=H0xPknceFORt

And it seems like there is an error while attempting to install scikit-learn==0.19.1 and scipy==1.0.0.
I tried to use newer version of scikit-learn and scipy and is works. (except the incompatible errors)

another hack was to remove -D_GLIBCXX_USE_CXX11_ABI=0 from setup_all.sh file.

You may consider to include these changes.
@jkkummerfeld
Copy link
Contributor

Thanks for mentioning this! I've updated the colab.

@stefanhgm
Copy link

Hi @jkkummerfeld

Thanks for putting together the Colab notebook for running the SpanBERT model: https://colab.research.google.com/drive/1SlERO9Uc9541qv6yH26LJz5IM9j7YVra#scrollTo=w5n-XspBFHjD.

We tried to copy and run it today and ran into a new issue when running the pip install -r requirements.txt --log install-log.txt -q. Any idea what have changed from the previous year's version? Could this maybe be a python version issue?

Found existing installation: tensorflow 2.8.0+zzzcolab20220506162203
Uninstalling tensorflow-2.8.0+zzzcolab20220506162203:
  Successfully uninstalled tensorflow-2.8.0+zzzcolab20220506162203
     |████████████████████████████████| 99 kB 9.4 MB/s
     |████████████████████████████████| 1.2 MB 54.1 MB/s
     |████████████████████████████████| 157 kB 57.6 MB/s
     |████████████████████████████████| 6.6 MB 39.7 MB/s
     |████████████████████████████████| 543 kB 55.4 MB/s
     |████████████████████████████████| 52 kB 808 kB/s
     |████████████████████████████████| 2.2 MB 52.9 MB/s
     |████████████████████████████████| 264 kB 61.0 MB/s
     |████████████████████████████████| 889 kB 53.5 MB/s
     |████████████████████████████████| 126 kB 51.8 MB/s
     |████████████████████████████████| 147 kB 54.2 MB/s
     |████████████████████████████████| 50 kB 6.3 MB/s
     |████████████████████████████████| 41 kB 716 kB/s
     |████████████████████████████████| 87 kB 7.0 MB/s
     |████████████████████████████████| 20.3 MB 1.2 MB/s
     |████████████████████████████████| 2.1 MB 31.7 MB/s
     |████████████████████████████████| 1.2 MB 56.5 MB/s
     |████████████████████████████████| 425 kB 29.7 MB/s
     |████████████████████████████████| 71 kB 9.5 MB/s
     |████████████████████████████████| 245 kB 71.4 MB/s
     |████████████████████████████████| 70 kB 7.5 MB/s
     |████████████████████████████████| 56 kB 3.4 MB/s
     |████████████████████████████████| 194 kB 49.8 MB/s
     |████████████████████████████████| 511 kB 53.9 MB/s
     |████████████████████████████████| 59 kB 6.0 MB/s
     |████████████████████████████████| 6.7 MB 48.3 MB/s
     |████████████████████████████████| 27.4 MB 1.2 MB/s
     |████████████████████████████████| 3.1 MB 55.0 MB/s
     |████████████████████████████████| 488 kB 73.8 MB/s
     |████████████████████████████████| 377.1 MB 9.0 kB/s
     |████████████████████████████████| 748.9 MB 587 bytes/s
     |████████████████████████████████| 8.8 MB 61.3 MB/s
     |████████████████████████████████| 322 kB 51.1 MB/s
     |████████████████████████████████| 4.1 MB 52.2 MB/s
     |████████████████████████████████| 46 kB 4.7 MB/s
     |████████████████████████████████| 375 kB 50.4 MB/s
  Building wheel for absl-py (setup.py) ... done
  Building wheel for gast (setup.py) ... done
  Building wheel for h5py (setup.py) ... done
  Building wheel for html5lib (setup.py) ... done
  Building wheel for JPype1 (setup.py) ... done
  Building wheel for mmh3 (setup.py) ... done
  Building wheel for msgpack-python (setup.py) ... done
  Building wheel for psycopg2 (setup.py) ... done
  Building wheel for pycparser (setup.py) ... done
  Building wheel for pyhocon (setup.py) ... done
  Building wheel for wrapt (setup.py) ... done
  Building wheel for PyYAML (setup.py) ... done
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
kapre 0.3.7 requires tensorflow>=2.0.0, which is not installed.
yellowbrick 1.4 requires scikit-learn>=1.0.0, but you have scikit-learn 0.21.0 which is incompatible.
torchtext 0.12.0 requires torch==1.11.0, but you have torch 1.2.0 which is incompatible.
torchaudio 0.11.0+cu113 requires torch==1.11.0, but you have torch 1.2.0 which is incompatible.
tensorflow-probability 0.16.0 requires gast>=0.3.2, but you have gast 0.2.2 which is incompatible.
tensorflow-metadata 1.8.0 requires absl-py<2.0.0,>=0.9, but you have absl-py 0.7.1 which is incompatible.
tensorflow-metadata 1.8.0 requires protobuf<4,>=3.13, but you have protobuf 3.9.1 which is incompatible.
tables 3.7.0 requires numpy>=1.19.0, but you have numpy 1.17.0 which is incompatible.
pywavelets 1.3.0 requires numpy>=1.17.3, but you have numpy 1.17.0 which is incompatible.
pyasn1-modules 0.2.8 requires pyasn1<0.5.0,>=0.4.6, but you have pyasn1 0.4.2 which is incompatible.
pandas 1.3.5 requires numpy>=1.17.3; platform_machine != "aarch64" and platform_machine != "arm64" and python_version < "3.10", but you have numpy 1.17.0 which is incompatible.
pandas 1.3.5 requires python-dateutil>=2.7.3, but you have python-dateutil 2.6.1 which is incompatible.
kapre 0.3.7 requires numpy>=1.18.5, but you have numpy 1.17.0 which is incompatible.
jaxlib 0.3.7+cuda11.cudnn805 requires numpy>=1.19, but you have numpy 1.17.0 which is incompatible.
jax 0.3.8 requires numpy>=1.19, but you have numpy 1.17.0 which is incompatible.
imbalanced-learn 0.8.1 requires scikit-learn>=0.24, but you have scikit-learn 0.21.0 which is incompatible.
googleapis-common-protos 1.56.1 requires protobuf>=3.15.0, but you have protobuf 3.9.1 which is incompatible.
google-colab 1.0.0 requires astor~=0.8.1, but you have astor 0.8.0 which is incompatible.
google-colab 1.0.0 requires six~=1.15.0, but you have six 1.12.0 which is incompatible.
google-api-python-client 1.12.11 requires six<2dev,>=1.13.0, but you have six 1.12.0 which is incompatible.
google-api-core 1.31.5 requires protobuf>=3.12.0; python_version > "3", but you have protobuf 3.9.1 which is incompatible.
google-api-core 1.31.5 requires six>=1.13.0, but you have six 1.12.0 which is incompatible.
flask 1.1.4 requires Jinja2<3.0,>=2.10.1, but you have jinja2 2.10 which is incompatible.
flask 1.1.4 requires Werkzeug<2.0,>=0.15, but you have werkzeug 0.14.1 which is incompatible.
fbprophet 0.7.1 requires python-dateutil>=2.8.0, but you have python-dateutil 2.6.1 which is incompatible.
datascience 0.10.6 requires folium==0.2.1, but you have folium 0.8.3 which is incompatible.
bokeh 2.3.3 requires pillow>=7.1.0, but you have pillow 6.1.0 which is incompatible.
albumentations 0.1.12 requires imgaug<0.2.7,>=0.2.5, but you have imgaug 0.2.9 which is incompatible.
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/__init__.py", line 28, in <module>
    from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/__init__.py", line 83, in <module>
    from tensorflow.python import keras
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/__init__.py", line 26, in <module>
    from tensorflow.python.keras import activations
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/activations.py", line 24, in <module>
    from tensorflow.python.keras.utils.generic_utils import deserialize_keras_object
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/utils/__init__.py", line 34, in <module>
    from tensorflow.python.keras.utils.io_utils import HDF5Matrix
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/utils/io_utils.py", line 29, in <module>
    import h5py
  File "/usr/local/lib/python3.7/dist-packages/h5py/__init__.py", line 36, in <module>
    from ._conv import register_converters as _register_converters
  File "h5py/_conv.pyx", line 1, in init h5py._conv
  File "h5py/h5t.pyx", line 1, in init h5py.h5t
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/__init__.py", line 28, in <module>
    from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/__init__.py", line 83, in <module>
    from tensorflow.python import keras
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/__init__.py", line 26, in <module>
    from tensorflow.python.keras import activations
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/activations.py", line 24, in <module>
    from tensorflow.python.keras.utils.generic_utils import deserialize_keras_object
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/utils/__init__.py", line 34, in <module>
    from tensorflow.python.keras.utils.io_utils import HDF5Matrix
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/utils/io_utils.py", line 29, in <module>
    import h5py
  File "/usr/local/lib/python3.7/dist-packages/h5py/__init__.py", line 36, in <module>
    from ._conv import register_converters as _register_converters
  File "h5py/_conv.pyx", line 1, in init h5py._conv
  File "h5py/h5t.pyx", line 1, in init h5py.h5t
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject
coref_kernels.cc:3:10: fatal error: tensorflow/core/framework/op.h: No such file or directory
 #include "tensorflow/core/framework/op.h"
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

@jkkummerfeld
Copy link
Contributor

Some sort of version compatibility does seem like the issue, but I don't think it's a python issue as those errors all show python 3.7, which is from 2018.

I don't have time to dig into this at the moment, but in the meantime if you find a solution please share it so I can update the colab!

@LifeIsStrange
Copy link

The colab appears to be broken :/

On local python 3.7 I have psycopg2 dependency error btw

@jkkummerfeld
Copy link
Contributor

I'm happy to incorporate a fix if you are able to identify one!

@LifeIsStrange
Copy link

Sorry I'm currently trying to work with alternative libraries such as wl coref or coref hoi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@jkkummerfeld @stefanhgm @LifeIsStrange @shon-otmazgin