Now that we have a basic understanding of TensorFlow and can define neural networks, it’s time to run our experiment on the GPU. In this part of the lesson, you will use C code to create an example that runs on the GPU. This solution provides a very good overview of current GPU programming basics using CUDA and OpenCL. You will do a lot of reading and configuring, but fortunately, you will not have to write any code for this lesson. However, feel free to rework this example slightly to get more comfortable with how to run TensorFlow on the GPU. If you are unfamiliar with C, the code is complex, so please don’t worry about understanding all of it.
A fantastic tool called CMake helps organize the compilation of your experiment and its dependencies. Even if you have never heard of CMake, give it a try here; it is simpler than it looks, and once you understand the basics, you will be able to modify the configuration file accordingly to run any experiment written in any language you like.
Note
CMake is a cross-platform program and library designed to build, test and package software. It is most commonly used to build C/C++ projects, but its functionality has expanded to include many additional languages including Python, Java, Lisp, Fortran and Make.
Take a look at the CMakeLists.txt file shown in Example 4-8. We will now walk you through all of the commands used in this file.
Example 4-8. Customize TensorFlow using CMake
############################################ ## Use the CUDA toolkit – mandatory ############################################ cmake_minimum_required(VERSION 2.8) ## Get the latest TensorFlow installation – mandatory ############################################ language(TensorFlow) find_package(TensorFlow REQUIRED) ## Get CMake for the GPU version ############################################ if (NOT TENSORFLOW_USE_GFLAGS) add_definitions(-DTENSORFLOW_USE_CUDA) endif(NOT TENSORFLOW_USE_GFLAGS) ## Set various paths found by TensorFlow package ############################################ find_path(PATHTODIR “”) if (NOT TENSORFLOW_FOUND) message(“TensorFlow library not found; please set TENSORFLOW_ROOT.”) endif(NOT TENSORFLOW_FOUND) ## Create the build directory ############################################ add_executable(tensorflow_gpe “${CMAKE_CURRENT_SOURCE_DIR}/tensorflow_gpe”) target_link_libraries(tensorflow_gpe “${TENSORFLOW_LIBRARIES}” “cuda” “opencl_gcc” “opencv”)
The first block sets the minimum version required to run the code, as well as where TensorFlow can be found. The latter chunk of code finds TensorFlow, makes it visible throughout the project, and sets up the environment using an extension to CMake. Finally, the last block of the code creates an executable, specifying the libraries on which the code depends.
Build CMake for the GPU
You need a specific version of CMake to compile TensorFlow library for the GPU. Download Tarball 1 of the release candidates posted by Andrei Mihailescu: https://code.google.com/archive/p/cmake/release/1.5.0rc1/cmake-1.5.0-rc1.tar.gz . On your Linux machine, gunzip and untar the file to a directory of your choice. Then, to include the development files in your build, add the following line to your /etc/profile.bash_profile (Mac OS X systems should use ~/.bash_profile):
export PATH=$PATH:/projects/cmake/files/CMake-1.5.0-rc1/bin
If you wish to run the examples in Windows, simply follow the corresponding instructions in Chapter 3.
Get TensorFlow and CUDA
To install TensorFlow, execute the following commands:
git clone https://github.com/continuumai/tensorflow.git cd tensorflow git checkout 1.3.0
Install cuDNN and GPUs
Choose one of the following options to run TensorFlow on the GPU:
Download and install NVIDIA CUDA Toolkit 5.5 from http://developer.nvidia.com/cuda-downloads . Have in mind that version 5.5 isn’t available for Mac OS X 64-bit. In addition, make sure you install