Step 0: Install cuda from the standard repositories.

Step 1: Register an nvidia developer account and download cudnn here (about 80 MB)

Step 2: Check where your cuda installation is. For the installation from the repository it is /usr/lib/... and /usr/include. Otherwise, it will be /urs/local/cuda/. You can check it with which nvcc

Step 3: Copy the files:

$ cd folder/extracted/contents
$ sudo cp include/cudnn.h /usr/include
$ sudo cp lib64/libcudnn* /usr/lib/x86_64-linux-gnu/
$ sudo chmod a+r /usr/lib/x86_64-linux-gnu/libcudnn*