-
Notifications
You must be signed in to change notification settings - Fork 18
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
6 changed files
with
63 additions
and
55 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -22,7 +22,7 @@ | |
| "This is an important one! Make sure that you can download data via Python for libraries like Quandl and also to install Python. \n", | ||
| "\n", | ||
| "* Once you've done your Python installation (and have signed up for a Quandl account and API key for free), try running this code below to check. \n", | ||
| "* Your firewall should also allow you to have access to allow the downloading of packages via conda and pip (if it isn't then you can't do much else). \n", | ||
| "* Your firewall should also allow you to have access to allow the downloading of packages via conda and pip (if it isn't then you can't do much else, although in some circumstances you might be able to change the pip download URL to a site that does work)\n", | ||
| "* If opening up firewalls is too much of an issue or you don't have admin rights to install Python locally, then one easier way to get started with Python for learning purposes, is to use something Azure Notebooks (see below). \n", | ||
| "* This will however, at the very least need the website https://notebooks.azure.com/ to be accessible)" | ||
| ] | ||
|
|
@@ -174,9 +174,6 @@ | |
| "call conda create -n py36class python=3.6 anaconda\n", | ||
| "call conda activate py36class\n", | ||
| "\n", | ||
| "REM newer versions of Pandas aren't supported as well by other libraries (eg. Arctic), so stick to 0.24.2\n", | ||
| "call conda install -c anaconda pandas=0.24.2 pyarrow=0.13.0 --yes\n", | ||
| "\n", | ||
| "REM install from conda-forge\n", | ||
| "call conda install -c conda-forge ^\n", | ||
| "jupyterlab jupyter_contrib_nbextensions jupyter_nbextensions_configurator ipywidgets=7.5 ^\n", | ||
|
|
@@ -211,6 +208,12 @@ | |
| "REM to be able to plot Plotly into PNG or JPG\n", | ||
| "call conda install -c plotly plotly-orca --yes\n", | ||
| "\n", | ||
| "REM in case has been installed elsewhere by pip\n", | ||
| "call pip uninstall pandas\n", | ||
| "\n", | ||
| "REM newer versions of Pandas aren't supported as well by other libraries (eg. Arctic), so stick to 0.24.2\n", | ||
| "call conda install -c anaconda pandas=0.24.2 pyarrow=0.13.0 --yes\n", | ||
| "\n", | ||
| "REM Jupyter libraries\n", | ||
| "call jupyter contrib nbextension install --user\n", | ||
| "call jupyter nbextension enable execute_time/ExecuteTime\n", | ||
|
|
@@ -219,6 +222,8 @@ | |
| "call jupyter labextension install [email protected] --no-build\n", | ||
| "call jupyter labextension install [email protected] --no-build\n", | ||
| "call jupyter lab build\n", | ||
| "\n", | ||
| "\n", | ||
| "```" | ||
| ] | ||
| }, | ||
|
|
@@ -320,9 +325,6 @@ | |
| "conda create -n py36class python=3.6 anaconda\n", | ||
| "source activate py36class\n", | ||
| "\n", | ||
| "# newer versions of Pandas aren't supported as well by other libraries (eg. Arctic), so stick to 0.24.2\n", | ||
| "conda install -c anaconda pandas=0.24.2 pyarrow=0.13.0 --yes\n", | ||
| "\n", | ||
| "# install from conda-forge (pyarrow is a newer version!)\n", | ||
| "conda install -c conda-forge \\\n", | ||
| "jupyterlab jupyter_contrib_nbextensions jupyter_nbextensions_configurator \\\n", | ||
|
|
@@ -367,6 +369,12 @@ | |
| "# to be able to plot Plotly into PNG or JPG\n", | ||
| "conda install -c plotly plotly-orca --yes\n", | ||
| "\n", | ||
| "# in case has been installed elsewhere\n", | ||
| "pip uninstall pandas\n", | ||
| "\n", | ||
| "# newer versions of Pandas aren't supported as well by other libraries (eg. Arctic), so stick to 0.24.2\n", | ||
| "conda install -c anaconda pandas=0.24.2 pyarrow=0.13.0 --yes\n", | ||
| "\n", | ||
| "# GPU libraries\n", | ||
| "# conda install -c rapidsai -c nvidia -c conda-forge -c defaults rapids=0.10 python=3.6\n", | ||
| "\n", | ||
|
|
@@ -382,6 +390,7 @@ | |
| "jupyter labextension install [email protected] --no-build\n", | ||
| "jupyter lab build\n", | ||
| "\n", | ||
| "\n", | ||
| "```" | ||
| ] | ||
| }, | ||
|
|
@@ -552,13 +561,15 @@ | |
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "### Direct compute on a paid Azure Jupyter instance\n", | ||
| "### Direct compute on a paid Azure Jupyter instance (Data Science Virtual Machine)\n", | ||
| "\n", | ||
| "The free version of Azure notebooks has a number of limitations. In particular, that it resets every hour and only has 4GB RAM. You can instead create paid instances Data Science Virtual Machine for Linux (Ubuntu) on Azure (DSVM), which is a Linux box for a number of different CPUs/RAM which are much more powerful. You can configure the machine as want, and the configuration will be stored for as long as you have the instance. Do make sure to power down the machine, once you are finished using it. This won't wipe the hard disk, but it will mean you aren't paying for it when it's idle.\n", | ||
| "\n", | ||
| "Whilst Azure 3.6 ML from the DSVM already has many libraries installed, eg. TensorFlow etc. You will still need to install additional Python libraries, but this can be done once. Script `update_azure_conda_env.sh` will attempt to install these extra libraries such as NLTK etc. and also a version of Pandas which should be more compatible.\n", | ||
| "\n", | ||
| "You can then connect your Azure notebook to run on this server via \"Direct compute\", where you need to specify the IP of the machine. If you get a powerful machine you should be able to host multiple users. However, you should add usernames/passwords for every user if you do this." | ||
| "You can then connect your Azure notebook to run on this server via \"Direct compute\", where you need to specify the IP of the machine. At https://docs.microsoft.com/en-us/azure/notebooks/use-data-science-virtual-machine it gives details on how to do \"Direct compute\" on your Azure Notebook. If you get a powerful machine you should be able to host multiple users. However, you should add usernames/passwords for every user if you do this, so each user runs in their own instance.\n", | ||
| "\n", | ||
| "You may need to run `conda install ipykernel` to make a new conda environment appear in the Jupyter notebook. Make sure that you select the right Python kernel from the Jupyter menu (Kernel / Change Kernel)." | ||
| ] | ||
| }, | ||
| { | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3,48 +3,30 @@ | |
| # assume already has py36 Azure ML environment, which has most libraries already installed (TensorFlow, PyTorch etc.) | ||
| conda activate py36 | ||
|
|
||
| # newer versions of Pandas aren't supported as well by other libraries (eg. Arctic), so stick to 0.24.2 | ||
| # newer versions of Pandas aren't supported as well by other libraries, so stick to 0.24.2 | ||
| conda install -c anaconda pandas=0.24.2 --yes | ||
|
|
||
| # install from conda-forge (pyarrow is a newer version!) | ||
| # install from conda-forge (pyarrow is a newer version, but may get downgraded later by modin) | ||
| conda install -c conda-forge \ | ||
| jupyterlab jupyter_contrib_nbextensions jupyter_nbextensions_configurator \ | ||
| python-blosc=1.8.1 graphviz python-graphviz rise textblob wordcloud \ | ||
| python-blosc=1.8.1 graphviz python-graphviz rise \ | ||
| vaex-core vaex-viz vaex-jupyter vaex-arrow vaex-server vaex-hdf5 vaex-astro vaex-distributed \ | ||
| nltk spacy fastparquet python-snappy nodejs koalas textacy pystan fbprophet --yes | ||
| nltk spacy fastparquet python-snappy fbprophet pystan pyarrow=0.16.0 --yes | ||
|
|
||
| # celery library | ||
| pip install celery==4.3.0 celery[redis] celery[msgpack] msgpack-python | ||
| # for celery and NLP | ||
| pip install celery==4.3.0 celery[redis] celery[msgpack] msgpack-python fxcmpy alpha_vantage yfinance twython newspaper3k seasonal \ | ||
| pdfminer.six vaderSentiment gensim textblob wordcloud quandl redis | ||
|
|
||
| # for findatapy (and NLP) | ||
| pip install fxcmpy alpha_vantage yfinance twython newspaper3k seasonal pdfminer.six vaderSentiment gensim | ||
|
|
||
| # install transformers/huggingface | ||
| pip install transformers cvlib pytesseract tabula-py | ||
| # install opencv for computer vision | ||
| conda install -c conda-forge opencv --yes | ||
|
|
||
| # modin (later versions need later versions of pandas) | ||
| # may downgrade to pandas to 0.24.2 | ||
| pip install modin=0.5.0 | ||
|
|
||
| # findatapy, chartpy and finmarketpy (or can manually clone these from GitHub and add to your Python path) | ||
| # need to have Git | ||
| # pip install git+https://github.com/cuemacro/finmarketpy.git git+https://github.com/cuemacro/chartpy.git git+https://github.com/cuemacro/findatapy.git | ||
| pip install finmarketpy chartpy findatapy | ||
| # install transformers, various NLP/text libraries | ||
| pip install transformers cvlib pytesseract tabula-py koalas textacy modin==0.5.0 | ||
|
|
||
| # graphics libraries | ||
| pip install cufflinks==0.17 plotly_express==0.4.1 dash==1.8.0 dash-html-components==1.0.2 dash-core-components==1.7.0 plotly==4.5.0 dash-table==4.6.0 | ||
| # findatapy, finmarketpy, chartpy & graphics libraries | ||
| pip install finmarketpy chartpy findatapy \ | ||
| cufflinks==0.17 plotly_express==0.4.1 dash==1.8.0 dash-html-components==1.0.2 dash-core-components==1.7.0 plotly==4.5.0 dash-table==4.6.0 | ||
|
|
||
| # to be able to plot Plotly into PNG or JPG | ||
| conda install -c plotly plotly-orca --yes | ||
|
|
||
| # Jupyter and Jupyterlab extensions | ||
| jupyter contrib nbextension install --user # to activate js on Jupyter | ||
| jupyter nbextension enable execute_time/ExecuteTime | ||
| jupyter-nbextension install rise --py --sys-prefix | ||
| jupyter labextension install @jupyter-widgets/[email protected] --no-build | ||
| jupyter labextension install [email protected] --no-build | ||
| jupyter labextension install [email protected] --no-build | ||
| jupyter lab build | ||
|
|
||
|
|
||