This is a short overview of my basic ML Ubuntu 16.04 setup. It includes NVIDIA Drivers, CUDA, OpenCV as well as a basic Python virtual environment setup. For this setup I expect a PC with a (current) NVIDIA GPU and Ubuntu 16.04 installed. I use standard paths, if necessary these must be adapted according to your installation.
First of all update your system to the latest version as follows:
sudo apt update sudo apt upgrade
Install the proprietary NVIDIA drivers as well as CUDA and CUDNN to get GPU acceleration.
We install the NVIDIA drivers by using an PPA.
sudo add-apt-repository ppa:graphics-drivers/ppa sudo apt update sudo apt install nvidia-396
You may install a newer version than 396, all available versions are displayed when you type nvidia- followed by <tab>.
After the installation is finished, restart the PC.
Download the Installer as well as the patches from CUDA 9.0 Download.
Install the files via the following command (change file name for the two patches). When the installation procedure comes up a few questions will be asked about what to install where. It is important to NOT install the driver, only CUDA (optionally the CUDA examples). The default path /usr/local/cuda-9.0 is fine. Also let it create a symbolic link at /usr/local/cuda.
sudo sh cuda_9.0.176_384.81_linux-run
Add the following to your bashrc (~/.bashrc, .zshrc when zsh is used):
# Path export PATH="/usr/local/cuda/bin:$PATH" export LD_LIBRARY_PATH="/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64:$LD_LIBRARY_PATH" export LIBRARY_PATH="/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64:$LIBRARY_PATH"
Download CUDNN 7.1 for CUDA 9.0. Extract the content, this will create a cuda directory, e.g. in ~/Downloads/cuda. After extracting copy the content to the CUDA folder, as follows:
sudo rsync -rl cuda/ /usr/local/cuda
Optional step for multiple CUDNN versions
If you would like to install multiple versions of CUDNN you could just create a subfolder and add the lib64 files there and later on set the (LD_)LIBRARY_PATH accordingly to which version you need. For example:
sudo mkdir /usr/local/cuda-9.0/cudnn sudo cp -R cuda/lib64 /usr/local/cuda-9.0/cudnn/7.1
Install some prerequisites:
sudo apt install python-dev python3-dev python-pip virtualenv sudo pip install virtualenv virtualenvwrapper
Add the following to your bashrc (~/.bashrc):
# virtualenv and virtualenvwrapper export WORKON_HOME=$HOME/.virtualenvs source /usr/local/bin/virtualenvwrapper.sh
Close your terminal and reopen it (or just
A good tutorial to install OpenCV from source can be found here.
Here is the content of the tutorial in short for a installation with Python 3.5:
sudo apt-get install build-essential cmake pkg-config libjpeg8-dev libtiff5-dev libjasper-dev libpng12-dev libavcodec-dev libavformat-dev libswscale-dev libv4l-dev libxvidcore-dev libx264-dev libgtk-3-dev libatlas-base-dev gfortran mkdir ~/tmp cd ~/tmp wget -O opencv.zip https://github.com/Itseez/opencv/archive/3.4.1.zip unzip opencv.zip wget -O opencv_contrib.zip https://github.com/Itseez/opencv_contrib/archive/3.4.1.zip unzip opencv_contrib.zip mkvirtualenv cv -p python3 workon cv pip install numpy cd ~/tmp/opencv-3.4.1/ mkdir build cd build cmake -D CMAKE_BUILD_TYPE=RELEASE \ -D CMAKE_INSTALL_PREFIX=/usr/local \ -D INSTALL_PYTHON_EXAMPLES=ON \ -D INSTALL_C_EXAMPLES=OFF \ -D WITH_CUDA=OFF \ -D WITH_FFMPEG=1 \ -D OPENCV_EXTRA_MODULES_PATH=~/tmp/opencv_contrib-3.4.1/modules \ -D PYTHON_EXECUTABLE=~/.virtualenvs/cv/bin/python \ -D BUILD_EXAMPLES=ON .. make -j8 sudo make install sudo ldconfig
PyTorch Development Environment
Now it is time to setup the python environment for PyTorch development.
mkvirtualenv pytorch -p python3 --system-site-packages workon pytorch cd ~/.virtualenvs/pytorch/lib/python3.5/site-packages ln -s /usr/local/lib/python3.5/site-packages/cv2.cpython-35m-x86_64-linux-gnu.so cv2.so pip install http://download.pytorch.org/whl/cu90/torch-0.4.0-cp35-cp35m-linux_x86_64.whl pip install torchvision
Tensorflow Development Environment
Tensorflow can be setup equivalent to PyTorch by creating a new virtual environment, adding opencv and installing it via pip.
mkvirtualenv tf -p python3 --system-site-packages workon tf cd ~/.virtualenvs/tf/lib/python3.5/site-packages ln -s /usr/local/lib/python3.5/site-packages/cv2.cpython-35m-x86_64-linux-gnu.so cv2.so pip install tensorflow-gpu