Here you can find a short description about installing CUDA and cuDNN libraries.
Download CUDA from the nVidia CUDA Toolkit download page. Choose options according to your operating system, CPU architecture, distribution, version etc. After choosing the preferred options a step-by-step guide appears like:
Run this commands from the directory where you saved the CUDA package. Download the patch files and install them also.
CuDNN is a GPU Accelerated Deep Learning framework. It is optional to install, however by using it, the training time will be shorter. To install it you have to create an account on the nVidia developer site, then you can download the library. Here is the link for cuDNN: https://developer.nvidia.com/cudnn. It is free to use for everyone. The installation is very simple, only some files should be copied to the right place.
For more information about the cuDNN installation please see the cuDNN site.
Before installing TensorFlow you have to install some additional libraries and set environment variables.
For CUDA Toolkit <= 7.5
$ sudo apt-get install libcupti-dev