Everyone who has tried to mess around with their Ubuntu distro knows the pain of having to reinstall Ubuntu and set it up to their liking again
These are the scripts that I use to set my Ubuntu up as quick as possible. Feel free to fork it and create your own version, and any contributions are more than welcome :)
Every script is rock stable and runs against Travis CI to make sure everything works as expected. Note that Build-ML.sh
is not shown here as it takes >2 hours to build TF+Pytorch on the Travis systems from source, and 2 hours is the system limit for free accounts on Travis. You can, however, still see the results of it running here
1-BasicSetUp and 2-GenSoftware | 3-ML-Basic | Build-OpenCV | Build-OpenCV in a conda env |
---|---|---|---|
First download/clone this repository
Run
chmod u+x *.sh
to make all scripts executable
Then execute them in the terminal in the sequence of filenames using ./1-BasicSetUp.sh
, /2-GenSoftware.sh
, and so on.
1-BasicSetUp.sh
- Sets up terminal configuration, a download accelerator, anaconda python, and shell aliases.2-GenSoftware.sh
- Sets up tools for programming(editor, etc), and other general purpose software I use3-ML-Basic.sh
- Installs from pip commonly used DL libraries
Additional scripts to built libraries from source:
Build-ML.sh
- Compiles commonly used ML/DL libraries from source, so that it is optimized to run on your computerBuild-OpenCV.sh
- Compiles the latest tag of OpenCV+Contrib from source on your machine with focus on optimization of execution of OpenCV code.
- Default python will be changed to Anaconda, with the latest Python 3. Anaconda Python will be installed in
/opt/anaconda3/
so that it is accessible by multiple users - Default shell is changed to Zim, a zsh plugin, instead of bash. Why zsh? Because it simply has a much better autocomplete. And why zim? Because it's much faster than Oh My Zsh and Prezto
maxvol
: Will set your volume to 150%download <webpage-name>
: Download the webpage and all sub-directories linked to itserver
: Sets up a server for file sharing in your local network. Whatever is in your current directory will be visible on the ip. It will also print the possible set of IP addresses. To access from another computer, shoot up a browser and simply hitip_add:port
gpom
: Alias forgit push origin master
. Will push your current directoryjn
: Starts a jupyter notebook in that directoryjl
: Starts a jupyter lab in that directoryydl "URL"
: Downloads the song atURL
at 128kbps, 44.1kHz in m4a format with the title and song name automatically set in the metadataupdate
: Runssudo apt-get update && sudo apt-get dist-upgrade && sudo apt-get autoremove -y
tsux
: Create a tmux session with-u
(so that the icons(battery, etc) are properly displayed at the bottom), and with a window withhtop
,nvidia-smi -l 1
andlm-sensors
automatically activated.- Reason for not making this the default tmux: You cannot attach tmux sessions if you alias the
tmux
command itself
- Reason for not making this the default tmux: You cannot attach tmux sessions if you alias the
aria
: For accelerated download of files using aria2c. Runs the following command:aria2c --file-allocation=none -c -x 10 -s 10 -d aria2-downloads
-
If you are using this script to set up a computer with many users,
- You need to run these scripts using only one user, say
first_user
- We need to copy the configuration files to the new user, say
new_user
. Fromfirst_user
's account, run the following after entering the username of thenew_user
in the second line of this snippetcd ~ export NEW_USER=<username_of_new_user> sudo cp -r /opt/.zsh/zim/ /home/$NEW_USER/.zim sudo cp /opt/.zsh/bash_aliases /home/$NEW_USER/.bash_aliases sudo cp /opt/.zsh/zim/templates/zimrc /home/$NEW_USER/.zimrc sudo cp /opt/.zsh/zim/templates/zlogin /home/$NEW_USER/.zlogin sudo cp ~/.zshrc /home/$NEW_USER/.zshrc sudo cp ~/.xbindkeysrc /home/$NEW_USER sudo mkdir -p /home/$NEW_USER/.config/tilda sudo cp ~/.config/tilda/config_0 /home/$NEW_USER/.config/tilda/config_0 sudo mkdir -p /home/$NEW_USER/.config/micro sudo cp ~/.config/micro/bindings.json /home/$NEW_USER/.config/micro/bindings.json sudo cp ~/.tmux.conf* /home/$NEW_USER/ sudo chown -R $NEW_USER: /home/$NEW_USER/.zim /home/$NEW_USER/.bash_aliases /home/$NEW_USER/.zimrc /home/$NEW_USER/.zlogin /home/$NEW_USER/.zshrc /home/$NEW_USER/.xbindkeysrc /home/$NEW_USER/.config/tilda /home/$NEW_USER/.config/micro /home/$NEW_USER/.tmux.conf*
- You need to run these scripts using only one user, say
-
Make sure that your system time and date is correct and synchronized before running the scripts, otherwise this will cause failure while trying to download the packages.
-
During the
Build-OpenCV
script,yasm
package is downloaded whileffmpeg
is being built. The download link for this package fails to work on certain networks, therefore check the download link as described below before running the script. If it doesn't work, then please switch to a different network. You can switch back to the original network afteryasm
has been downloaded if required-
If you are on a browser, just open the following page - https://www.tortall.net/
If a webpage opens up, then the download will work. -
If you only have a terminal access, run this command :
curl -Is www.tortall.net | head -1
Output with Status code200 OK
means that the request has succeeded and the URL is reachable.
-
-
OpenCV is built to link to an
ffmpeg
that is built from scratch using Markus' script. Theffmpeg
that is built is stored in/opt/ffmpeg-build-script
. While the binaries are copied to/usr/local/bin
, the specific versions oflibavcodec
and other referenced libraries are still maintained at/opt/ffmpeg-build-script/workspace/lib
-
If you have Anaconda Python, OpenCV will be linked to Anaconda Python by default, not the Linux default python. If you would like to compile for the Linux default Python, remove Anaconda from your path before running the
Build-OpenCV.sh
script -
If you would like to install with OpenCV for CUDA, change the flags,
-D WITH_CUDA=0
,-D WITH_CUBLAS=0
,-D WITH_CUFFT
,-D CUDA_FAST_MATH
in the fileBuild-OpenCV.sh
toON
-
Building OpenCV with CUDA enabled can take a very long time, since it has to build the same code for all GPU architectures. If you don't need to compile for all architectures, you can specify the architecture using
CUDA_ARCH_BIN
such as 30 for Kepler, 61 for Pascal, etc. Information about your GPU can be found at Nvidia's page -
Non-free & patented algorithms in OpenCV such as SIFT & SURF have been enabled, for disabling them, set the flag
-D OPENCV_ENABLE_NONFREE=ON
to off -
OpenCV will be built without support for Python 2. If you would like to build it with Python 2 support, then add back the lines removed in this commit
-
After OpenCV installation, if you get an error of the sort
illegal hardware instructions
when you try to run a python or c++ program, that is because your CPU is an older one (Pentium/Celeron/...). You can overcome this by adding the following to the end of the cmake (just before the..
)-D ENABLE_SSE=OFF \ -D ENABLE_SSE2=OFF \ -D ENABLE_SSE3=OFF ..
If you still want to be able to receive the benefits of CPU optimization to whatever extent you can, then hit
cat /proc/cpuinfo
and see whatsse
s are available under flags -
If you run into compilation problems or something else during OpenCV installation, make sure to remove the entire
build
folder insideopencv
directory before rebuilding, otherwise strange errors can pop up. -
Building Tensorflow from source has different configuration options, info on which can be seen on Tensorflow's Build from Source page. Note that by default, 2.x version of Tensorflow will be built, to build 1.x version, add
--config=v1
to the bazel build command -
If you want to install a specific version of OpenCV or Tensorflow, i.e different from the latest release, make the following changes. The scripts should work with different versions but they haven't been tested
-
OpenCV
Comment out the line fetching the latest release tag in theBuild-OpenCV
script.
Add the line below the above commented out one specifying the OpenCV version which you want like this:latest_tag="3.4.5"
Alternatively, you could just replace$latest_tag
with the tag of the version in the following 2 lines:git checkout -f $latest_tag
Make sure that the tag of the OpenCV version you want is correct. The tags of all the releases can be checked here - https://github.com/opencv/opencv/tags -
Tensorflow
Similar to above, locate the line fetching the latest release tag of Tensorflow and replace with the tag of the version required.
The tags of all the Tensorflow releases can be checked here - https://github.com/tensorflow/tensorflow/tags
-
-
These scripts are written and tested on the following configurations -
- Ubuntu 16.04 & 18.04
- 32-bit and 64-bit Intel Processors
ML-Build.sh
- NVIDIA GPUs including but not limited to GeForce GTX 1080, 1070, 940MX, 850M, and Titan X
Although it should work on other configurations out of the box, I have not tested them
In the description of shortcuts below, if two keys are connected with a +
, then the second key is to be pressed while keeping the first key pressed. If two keys are connected with a
(space), then the previous key/keys are to be released before the next key is pressed. If two keys are separated by a /
, then it means you can choose any of the specified keys
Ctrl+b c
: Create a new window within the tmux sessionCtrl+b \
: Split existing window into two panes verticallyCtrl+b -
: Split existing window into two panes horizontallyAlt+Right/Left/Up/Down
: Switch between the panes of the same windowCtrl+b Shift+Left/Right
: Switch between the windows of the tmux sessionCtrl+b [
: Go into tmux copy mode (enable selection of text from the tmux buffer quickly using just your keyboard). Go to your preferred start point, pressSpace
to start the selection of the copy section. PressEnter
at the end point
- A docker image for this set-up (last updated Jan 30th, 2017)
- A Ubuntu customization dedicated to robotics
- CI
- configuring default wifi settings