pip install problems

[csherwood-usgs]

I get this when I am not on the VPN:

(dashdoodler) C:\crs\src\dash_doodler>pip install -r requirements.txt
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /simple/dash-core-components/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /simple/dash-core-components/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /simple/dash-core-components/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /simple/dash-core-components/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /simple/dash-core-components/
Could not fetch URL https://pypi.org/simple/dash-core-components/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/dash-core-components/ (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)) - skipping
ERROR: Could not find a version that satisfies the requirement dash_core_components==1.10.0 (from -r requirements.txt (line 1)) (from versions: none)
ERROR: No matching distribution found for dash_core_components==1.10.0 (from -r requirements.txt (line 1))
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)) - skipping

But it works fine on the VPN. Usually it is the other way around.
Not sure this is a real issue, but is there a reason not to use conda to install these packages?

[MARDAScience]

Haven't had any reason to use conda (presented workflow worked on my linux and windows machines), so I can't answer directly. But yes you should switch to conda for the unpippables ... I suppose ssl issues might also affect conda, in which case conda config --set ssl_verify no. I don't know of a way to troubleshoor ssl errors with pip. I'm hoping the plotly/dash components have a good conda version - can you report back?

[MARDAScience]

It is possible that the cairo library could interfere with dash components installed using pip, so another thing to try would be to create the env without cairo, then see if the pip dependencies install.

[csherwood-usgs]

@rsignell-usgs (Will this loop in Rich?) I tried to set up an environment file called ddood.yml to install everything from conda (not including pydensecrf or cairo yet).

ddood.yml:

name: ddood
channels:
  - conda-forge
  - defaults
dependencies:
 - python=3.6
 - dash_core_components=1.10
 - plotly==4.7.1
 - plotly-express=0.4
 - CairoSVG==2.4.2
 - matplotlib==3.2.1
 - scipy==1.4.1
 - dash==1.12.0
 - numpy=1.18
 - scikit_image==0.16.2
 - dash_html_components=1.0
 - dash_table=4.7
 - Pillow==7.1.2
 - scikit_learn=0.23
 - scikit-image=0.16
 - gunicorn=20.0
 - pandas==1.0.3

Note that I loosened the specs for the missing packages (== to =).

I get the following error:

C:\crs\src\dash_doodler>conda env create --file ddood.yml
Collecting package metadata (repodata.json): done
Solving environment: failed

ResolvePackageNotFound:
  - dash_html_components=1.0
  - scikit_learn=0.23
  - scikit_image=0.16
  - dash_table=4.7
  - plotly-express=0.4
  - dash_core_components=1.10
  - gunicorn=20.0

I assume that means that these are not in conda-forge, although I am a little surprised about the scikit packages. How do we go about getting these into conda?

So, sticking with your pip installation for now...that worked. Can probably close this issue.

[MARDAScience]

Chris those pip packages were really specific versions because that's what worked on the ~5 computers I have at my disposal and I had learned from previous experiences .... apparently. You are the only person so far reporting issues. Stephen bosse installed this morning on his st pete machine and the install instructions worked. That's all I have to go on for now. Sorry I cant be more help. I'll try to troubleshoot this later, but for now of course I am focused on the bugs. Doodler is the hardest program I've ever written ;(

[rsignell-usgs]

@csherwood-usgs , looks like you trying this on a PC? The package versions might well be different from platform to platform. I would try unpinning everything if you want to try conda.

[csherwood-usgs]

Just to be clear - I did not have any issues installing using your instructions. It works great. My SSL issues went away when I switched VPNs, so I have a working installation. Just wondering whether conda would work, and I loosened the version numbers in case they were too restrictive. So all is good.

[MARDAScience]

You may have thrown us by the title "pip install problems" ;)

[dnowacki-usgs]

@csherwood-usgs the issue in your .yml file is the package names as they exist on conda-forge are incorrectly specified. Mostly, it's dashes instead of underscores and vice-versa. Also you have scikit-image specified twice. But in general I like the idea of sticking with conda only, since mixing pip and conda is a recipe for heartache in my experience.

This one worked for me:

name: ddood
channels:
  - conda-forge
  - defaults
dependencies:
 - python=3.6
 - dash-core-components=1.10
 - plotly==4.7.1
 - plotly_express=0.4
 - CairoSVG==2.4.2
 - matplotlib==3.2.1
 - scipy==1.4.1
 - dash==1.12.0
 - numpy=1.18
 - scikit-image==0.16.2
 - dash-html-components=1.0
 - dash-table=4.7
 - Pillow==7.1.2
 - scikit-learn=0.23
 - gunicorn=20.0
 - pandas==1.0.3

[dnowacki-usgs]

And adding

 - pydensecrf 
 - cairo

to the yaml list above would result in a conda-only setup path that could simplify the installation instructions to a single
conda env create --file environment.yml

[csherwood-usgs]

@dnowacki-usgs - Thanks! Good eyes! I copied those filenames from Dan B's. pip install .txt file...is it really possible the underlines and hyphens are different in pip v. conda channels? Also, gunicorn still can't be found, even though it should be in conda-forge. The Google intimates this might be function of my version of conda (latest) or linux v. windows. BUT...the software seems to work without gunicorn. Is it really needed?

[rsignell-usgs]

@csherwood-usgs , some packages are not available on all platforms. If you take a look at https://anaconda.org/conda-forge/gunicorn you can see that Windows is not on the list!

[csherwood-usgs]

@dbuscombe-usgs Dan - As Dan N. and Rich note, you can make an environment.yml that works for a conda install for Linux or Mac, but gunicorn is not available for Windows. However, app.py seems to work fine without gunicorn...is it needed?

[dbuscombe-usgs]

Thanks all for these insights. Gunicorn not needed until we serve this on a (*nix based) web server somewhere, so I will remove it. Thanks for the report!

[dbuscombe-usgs]

Thanks @dnowacki-usgs for the yml file suggestion. Will incorporate in the next round of docs