GPUs

Using GPU enabled systems in the CS department

There are several systems available for GPU processing:

 

Blaze

Several PCs in lab 105, all machines in lab 121

Windows vlab

Lab Machines

6 desktops in lab 105 and all desktops in lab 121 are GPU enabled. The GPUs have ~300 cores each, which makes them ideal for code development and testing. However, these PCs are rebooted on a nightly basis, so jobs will not run overnight. These are Linux machines, and you will have to modify your path and install Tensorflow in your own user space.

Blaze

Blaze is the name of a Linux machine with 4 Nvidia GTX Titan X cards, each with 3072 cores.

To access blaze, run the command:

ssh username@blaze.cs.ucl.ac.uk

Blaze is a managed Linux machine, and so you will be able to access your home directory once you are connected.

Blaze is a timeshare machine, so there is no guarantee that it will be available for use. Running the command nvidia-smi will print a summary of the cards, as well as the processes that each card is working on. If all cards are occupied, unfortunately you will either have to run your jobs on a different system, or wait for the cards to become free.

 

In order to facilitate more efficient usage of Blaze’s GPUs, it is important to only run your processes on one card. To do this, navigate to the directory /usr/local/cuda/ . Then run the following command:

 

source CUDA_VISIBILITY.csh

 

To check which GPU your system is using, and to ensure that the environment variable has
been set correctly, type:

 

env | grep CUDA

 

You should the following output:

 

CUDA_VISIBLE_DEVICES=n

 

with n being a number between 0 and 3.

 

Installing Tensorflow on Linux

In order to install Tensorflow in your filespace, first you need to add Python to your path by running the following command:

setenv PATH /opt/Python/Python-3.6/bin:$PATH

Then, use pip (Python package manager) to install the tensorflow-gpu package:

pip install tensorflow-gpu --user

If you don’t add the –user flag, pip will attempt to install Tensorflow in the global filesystem, which you won’t have write permissions for.

The setenv command will add Python to your working path for the session, however if you log out of blaze you will have to re-add Python to your path. To permanently add Python to your path, you will need to copy the file .uclcs-csh-options.example (found in your home directory) to a file named .uclcs-csh-options.

cd ~

cp .uclcs-csh-options.example .uclcs-csh-options

Then add the following line to .uclcs-csh-options:

set CS_PRE_PATH = ( /opt/Python/Python-3.6/bin )

Testing Tensorflow

Once you have activated Tensorflow, you should test that it is functioning before you attempt to run any jobs.

Enter the python environment. This is done by typing python into a terminal window on Linux, or into the Miniconda TensorflowGPU prompt on Windows.

Run the following commands:

import os

os.environ['TF_CPP_MIN_LOG_LEVEL']='2'

import tensorflow as tf

hello=tf.constant('Hello, Tensorflow!')

sess=tf.Session()

print(sess.run(hello))

If you see the output b'Hello, Tensorflow!', Tensorflow is working.


 

It is also worth checking that Tensorflow is performing computation using the GPUs rather than the CPU. To do this, run the following:


import os
os.environ['TF_CPP_MIN_LOG_LEVEL']='2'
import tensorflow as tf
a=tf.constant([1.0,2.0,3.0,4.0], shape=[2,2], name='a')
b=tf.constant([1.0,2.0,3.0,4.0], shape=[2,2], name='b')
c=tf.matmul(a,b)
sess=tf.Session(config=tf.ConfigProto(log_device_placement=True))
print(sess.run(c))

Ensure that the ouput of line 7 and 8 specify GPUs.

 

 

os.environ['TF_CPP_MIN_LOG_LEVEL']='2' suppresses some warnings from Tensorflow. Simply omit the first two lines if you wish to see all warning messages.

GPU Processing on the cluster

If you are struggling with the departmental resources, there is additional GPU processing available on the cluster. However, this is not for development or testing. Once you have successfully run your code on above machines, come to room 4.20 to get an introduction to the cluster.

Windows vlab

On managed machines, access the vlab by using Citrix Receiver, which is already installed.

If you are using your own device, you can download and install the Citrix Receiver yourself. However, in order to connect to the vlab you will need to either connect to the CS wireless network or the CS VPN.

Once you have opened Citrix Receiver, connect to the address tabernus1.cs.ucl.ac.ukand provide your login details.

Once inside the vlab, you can activate the Tensorflow environment by the following instructions.

Click on the Apps shortcut on the Desktop.

 

 

Go to the Miniconda2 folder.

Open Miniconda tensorflowGPU.

 

 

This prompt is where you will be able to use Tensorflow to run your jobs.

It may take a few seconds for a prompt to appear on this screen, be patient!

In this prompt you can type python, and you’ll be entered into the python CLI. From here you can test that Tensorflow is functioning correctly.