In a previous post I discussed how easy it was to setup machine learning libraries in python using virtual machines (vm) hosted in the cloud. Today, I am going to add more details to it. This post covers how to make machine learning code run faster. This post will help any user compile a tensorflow installer to make it run faster in the cloud. The speed advantages are due to installer optimizations to take advantage of processor instruction set extensions that power the cloud vm.
The steps described below will help us compile a tensorflow installer with accelerated floating point and integer operations using instruction set extensions for the AMD64 architecture . The CPU we will be dealing with today is an Intel Xeon processor running at 2.3GHz. It is a standard server CPU that supports AVX, AVX2, SSE + floating point math and SSE4.2.
In the previous post, we were using a simple pip installer for tensorflow. The pip installer is a quick and easy way to set-up tensorflow. But, this installation method for tensorflow is not optimized for the extended instruction sets that are present in these advanced CPUs powering the vm in Google Cloud Platform.
sudo pip install tensorflow
If we compile an installer optimized for these instruction set extensions, we can speed-up many of the computation tasks. When tensorflow is used as a back-end in keras for machine learning problems, the console reminds you constantly, to optimize your tensorflow installation. These instructions below will also help you get rid of those warning messages in your console.
First step to compile an optimized tensorflow installer is to complete all the linux package dependencies. Run the line below to complete the missing packages needed to compile tensorflow installer.
sudo apt-get install pkg-config zip g++ zlib1g-dev unzip
To compile tensorflow, we need a build tool called Bazel. Bazel is an opensource tool developed by Google. It is used to automate software development and testing. Since tensorflow is at the leading edge of machine learning world, features are added, bugs are fixed and progress is made at a dizzying speed than the relatively pedestrian pace of traditional software development. In this rapid development, testing and deployment environment; Bazel help users manage this process more efficiently. Here is the set of code to install Bazel.
echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key add - sudo apt-get update && sudo apt-get install bazel sudo apt-get upgrade bazel
Once Bazel is installed in your computer, next step is to clone the source files of tensorflow from GitHub.
git clone https://github.com/tensorflow/tensorflow cd tensorflow git checkout
After the source files are copied from GitHub repository to your local machine, we have to do some housekeeping. We need to ensure the python environment to run tensorflow has all the necessary libraries installed. To fulfill the library dependencies in python, we need to install numpy, development environment, pip and wheel using the code below:
sudo apt-get install python-numpy python-dev python-pip python-wheel
The next step is to configure the build process for the installer. Before we do the actual configuration, we are going to use a preview of the configuration dialog. This will help us understand what parameters we should know before hand to complete the configuration process successfully. The configuration dialog for building the installer is as follows:
Please specify the location of python. [Default is /usr/bin/python]: Please specify optimization flags to use during compilation [Default is -march=native]: Do you wish to use jemalloc as the malloc implementation? (Linux only) [Y/n] jemalloc enabled on Linux Do you wish to build TensorFlow with Google Cloud Platform support? [y/N] No Google Cloud Platform support will be enabled for TensorFlow Do you wish to build TensorFlow with Hadoop File System support? [y/N] No Hadoop File System support will be enabled for TensorFlow Do you wish to build TensorFlow with the XLA just-in-time compiler (experimental)? [y/N] No XLA support will be enabled for TensorFlow Found possible Python library paths: /usr/local/lib/python2.7/dist-packages /usr/lib/python2.7/dist-packages Please input the desired Python library path to use. Default is [/usr/local/lib/python2.7/dist-packages] Using python library path: /usr/local/lib/python2.7/dist-packages Do you wish to build TensorFlow with OpenCL support? [y/N] No OpenCL support will be enabled for TensorFlow Do you wish to build TensorFlow with CUDA support? [y/N] No CUDA support will be enabled for TensorFlow Configuration finished
We need four important pieces of information before we go ahead and configure the build process. They are: 1) location of the python installation, 2) python library path, 3) g++ location and 4) gcc location. The last two are optional and only needed to enable OpenCL. If your cloud vm supports OpenCL and CUDA, the instructions to compile tensorflow are slightly different, which I will not cover in this post. Identifying python installation location and library paths can be done using the following code. I have also included the steps for finding locations for gcc and g++ compilers in the code below:
$ python >>>import sys >>>print(sys.executable) /usr/bin/python2 # Installation location >>>print(sys.path) ['', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/home/rahul/.local/lib/python2.7/site-packages', '/usr/local/lib/python2.7/dist-packages', '/usr/local/lib/python2.7/dist-packages/Keras-2.0.1-py2.7.egg', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PILcompat'] >>>exit() $ which gcc /usr/bin/gcc $ which g++ /usr/bin/g++
We have all the information needed for configuring build process. We can proceed to configure the build process using the following line:
./configure
If you encounter the following error:
Problem with java installation: couldn't find/access rt.jar in /usr/lib/jvm/java-9-openjdk-amd64
Purge the openjdk-9 and reinstall jdk-8 version. Use the instructions below:
sudo apt-get install openjdk-8-jdk echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key add - sudo apt-get update && sudo apt-get install bazel sudo apt-get upgrade bazel
Now, try ./configure again. Once the build process is configured properly, we can go ahead and build the installer using the following commands:
sudo bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-mfpmath=both --copt=-msse4.2 -k //tensorflow/tools/pip_package:build_pip_package sudo bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg cd .. sudo pip install /tmp/tensorflow_pkg/tensorflow-1.3.0-cp35-cp35m-linux_x86_64.whl
Everything should proceed smoothly and the build process is going to take some serious time. An octacore 2.3GHz Intel Xeon powered virtual machine needs around 30 minutes to complete this process. So, plan this process ahead of time. A short notice deployment is impossible, if one is looking to build the installer from scratch.
If the last step above threw a file not found error, it can be resolved by peeking into the build directory for the correct installer name.
cd /tmp/tensorflow_pkg ls
Once you have the correct installer name, append the last line of code above with the correct installer name and the installation process should finish without any error messages. If we manage to finish all the steps above, it means: we have successfully installed an optimized installer for tensorflow. This installed tensorflow library is compiled to take advantage of the processor instruction set extensions.
Finally, to check and see if tensorflow can be imported into python, use the next few lines of code. The code follows these steps: first, we have to exit the tensorflow install directory, then invoke python and import tensorflow.
$cd ~/ $python >>>import tensorflow >>>exit()
The import tensorflow line in python environment should proceed with no errors. Hurrah, we have a working tensorflow library that can be imported in python. We will need to do some tests to ensure that everything is in order, but, no errors up to this point means easy sailing ahead.
I have described a step-by-step process of building and installing tensorflow based on my logical reasoning of progression of things. These steps have worked extremely well for me, so far. The optimized tensorflow installation has cut-down run-time for some of the tasks by a factor of ten. As an added advantage, I longer have error messages that ask me to do optimizations to tensorflow installation in the python console.
Happy machine learning in the cloud everyone.
This work is done as part of our startup project nanøveda. For continuing nanøveda’s wonderful work, we are running a crowdfunding campaign using gofundme’s awesome platform. Donation or not, please share our crowdfunding campaign and support our cause.
Donate here: gofundme page for nanøveda.