This repository contains all the material needed for the 1st ASTERICS-OBELICS International School on "Advanced software programming for astrophysics and astroparticle physics". The time table is available at the following location.
- Important links
- Get a copy of this repository with
git
- Recommendation for Python install
- Jupyter
- IDE: PyCharm
- Cartesius machines
- Chat rooms
- Tutors
- Help
- Local copies of needed software, like Anaconda and PyCharm
- Gitter chat rooms
- Cartesius access form
Clone this repository on your personal computer.
git clone https://github.com/Asterics2020-Obelics/School2017.git
You will need it before the school to install the different tools, and during the school while attending the hands-on. For Windows, see below.
You must install Python 3.6 and a few Python libraries. The recommended way to do so is to use Anaconda. The procedures described bellow will help you install what is needed for the school.
-
Install required distribution packages :
- Ubuntu:
sudo apt-get install -y git bzip2 wget
- Fedora 25:
sudo dnf install -y git wget bzip2
- CERN Scientific Linux 6:
sudo yum install -y git tar bzip2 wget
- CERN CentOS 7:
sudo yum install -y git bzip2 wget
- Ubuntu:
-
Download (local copy) the Linux
Anaconda
installer for Python 3.6. -
Run the following command line:
bash Anaconda3-4.3.1-Linux-x86_64.sh
-
Answer
no
to the following question if you do not want to mess up your previous installs of python.Do you wish the installer to prepend the Anaconda3 install location to PATH in your /home/chotard/.bashrc ? [yes|no] [no] >>> no
-
If you haven't cloned this repository, download one of the
anaconda_setup
script (sh
orcsh
depending on your shell):wget https://raw.githubusercontent.com/Asterics2020-Obelics/School2017/master/anaconda_setup.sh or wget https://raw.githubusercontent.com/Asterics2020-Obelics/School2017/master/anaconda_setup.csh
-
If you have installed
anaconda
in a different directory than the default one suggested by theanaconda
installer (your home directory), edit the first line of theanaconda_setup
script with your selected path.ANACONDA=YOURSELECTEDPATH or setenv ANACONDA YOURSELECTEDPATH
-
Run the
anaconda_setup
script to set up the correct PATH and PYTHONPATH enabling the use of yourAnaconda
install.source anaconda_setup.sh or source anaconda_setup.csh
You can either run this command each time you need to use
Anaconda
, or add its content to your.bashrc
(or equivalent) to set it up at the opening of a new terminal. -
See next section for extra Python libraries requirements (if any).
-
Download (local copy) the Mac
Anaconda
installer for Python 3.6. -
Run the following command line:
bash Anaconda3-4.3.1-MacOSX-x86_64.sh
-
Answer
no
to the following question if you do not want to mess up your previous installs of python.Do you wish the installer to prepend the Anaconda3 install location to PATH in your /home/chotard/.bashrc ? [yes|no] [no] >>> no
-
If you haven't cloned this repository, first get the
anaconda_setup.sh
script:wget https://raw.githubusercontent.com/Asterics2020-Obelics/School2017/master/anaconda_setup.sh
-
If you have installed
anaconda
in a different directory than the default one suggested by theanaconda
installer (your home directory), edit the first line of theanaconda_setup
script with your selected path.ANACONDA=YOURSELECTEDPATH
-
Run the
anaconda_setup.sh
script to set up the correct PATH and PYTHONPATH enabling the use of yourAnaconda
install.source anaconda_setup.sh
You can either run this command each time you need to use
Anaconda
, or add its content to your.bashrc
(or equivalent) to set it up at the opening of a new terminal. -
See next section for extra Python libraries requirements (if any).
Instruction for Windows can be found
here for the installation of Anaconda (local copy). Once installed, you can run Anaconda navigator
. To run Jupyter, on the main page of the Anaconda navigator, click on Launch
on the Jupyter notebook box. This will open your favorite browser. From there, you can either load a notebook (e.g. from the Git folder) or create a new notebook by clicking new -> Python 3
.
You can also install a Git tool for Windows: Git for Windows. Launch Git GUI
or Git bash
to get started.
All the required libraries come with the Anaconda
install described
above. If you have followed the previous steps to install Python, you
can skip this section.
If you choose an other way to install Python 3.6 than the one
recommended above, you must install manually the Python libraries
listed in the requirements.txt file. To do so, we
recommend using pip
.
pip install -r requirements.txt
To launch a Jupyter notebook, simply run the following command:
jupyter notebook
On Windows, see in the above.
We strongly recommend to use pycharm, especially for the "Debugging & profiling" course. Free Community Edition: Download PyCharm (local copy) or opt for a free copy of the Professional Edition under Student License.
For the parallel and GPU sessions, we will use the Cartesius machines that are generously made available by SURFsara for the OBELICS school. You will need to connect to them through the ssh protocol (natively installed on Linux and Mac).
For Windows users, we recommend these tools to connect via ssh:
- Putty & Winscp
- Or, on Windows 10: use the native bash environment
To connect to Cartesius, please open a terminal and use the following command:
Where XXX is the account number you received via email. Type in the password you received together with your login name and press enter. NOTE that, the cursor won't move while tying in the password, this is normal, so just keep typing.
The account is made available for the school and will be valid till 30th of June. If you would like to connect to Cartesius from outside of LAPP, please use the following command:
Type in your password and select system Cartesius.
Login to Cartesius, clone the git repository and generate a key pair.
ssh [email protected]
git clone https://github.com/Asterics2020-Obelics/School2017.git
ssh-keygen -t rsa
Press Enter three times. A key pair will be generated for you in directory .ssh. Copy the contents of the public key to file authorized_keys:
cat .ssh/id_rsa.pub >> .ssh/authorized_keys
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
When the hands-on session starts, submit the following job to Cartesius:
cd School2017/gpu_programming
git pull # We made some changes to the job script recently!
sbatch job.jupyter.gpu
Open a new terminal and do the following:
ssh -L5XXX:localhost:5XXX [email protected]
Note that, you need to replace XXX with the three digits of your login account.
Use this command to check your job:
squeue –u $(whoami)
To cancel your job:
scancel JobID
If your job is running, you can open your browser and go to localhost:5XXX (replace XXX with the three digits of your own account). You should be able to see the Jupyter notebook now.
Gitter chat rooms have been created for each hands-on session and group. If you need to talk to each other during the session, share information, ask questions and get fast answer in case all the tutors are busy, you can use the corresponding chat rooms to do so. These chat rooms can also be used by the different tutors to give information or advices before or during the hands-on sessions.
Be aware that you will need to be connected to use these chat rooms, preferentially using your github account. You can also start one-to-one chat rooms.
Here is the list of tutors for the hands-on sessions. Four (or more) are needed for each session (except for GPU and parallel which are parallel sessions).
Hands-on | Main tutor(s) | Other tutors |
---|---|---|
Numpy | Tamas Gal | Axel Donath, Johannes King, Pierre Aubert, Tristan Carel, N. Chotard |
Pandas | Tamas Gal | Damian Podareanu, Karl Kosack, Johannes King, N. Chotard |
Astropy | Axel Donath, Johannes King | Karl Kosack, Hendrik Heinl, Tim Jeness, N. Chotard |
Profiling & Debugging | Karl Kosack | Axel Donath, Zheng Meyer-Zhao, Pierre Aubert, Tristan Carel |
Parallel Programming | Damian Podareanu | Tamas Gal, Pierre Aubert, Jean Jacquemier, Tristan Carel |
GPU Programming | Valeriu Codreanu | Pierre Aubert, Zheng Meyer-Zhao, Tristan Carel (?) |
The full list of tutors is available here.
Please create a new issue (you will need a github account) for each question you may have before or during the school about software install and/or about one of the classes. Of course, you should first check the existing list of issues to see if your question has already been asked and answered before creating a new one.