2017년 3월 16일 목요일

Ubuntu 16.04 ppc64le에서, source로부터 tensorflow를 build하는 방법

PowerAI 속에 tensorflow가 들어 있긴 합니다만, 간혹 customizing 때문에 source에서 build할 필요도 있을 수 있습니다.   여기서는 CUDA 8.0과 PowerAI가 이미 다 설치된 상태에서, build를 시작하는 것으로 하겠습니다.  먼저, 기반 OS package를 설치합니다.

test@k8002:~$ sudo apt-get install make automake autoconf gfortran-4.8-multilib-powerpc-linux-gnu libgfortran-4.8-dev-powerpc-cross gfortran libgfortran-4.8-dev gcc-powerpc-linux-gnu g++ pkg-config cmake-curses-gui zlib1g-dev zip unzip libtool openjdk-8-jdk python python-all-dev swig python-numpy g++-4.8 g++-4.8-multilib-powerpc-linux-gnu zlibc re2c libre2-dev libcupti-dev

Tensorflow를 github에서 download 받습니다.

test@k8002:~$ git clone --recurse-submodules https://github.com/tensorflow/tensorflow

이제 tensorflow에 어떤 버전의 branch들이 있는지 확인합니다.

test@k8002:~$ cd tensorflow
test@k8002:~/tensorflow$ git branch -a
* master
  remotes/origin/0.6.0
  remotes/origin/HEAD -> origin/master
  remotes/origin/master
  remotes/origin/r0.10
  remotes/origin/r0.11
  remotes/origin/r0.12
  remotes/origin/r0.7
  remotes/origin/r0.8
  remotes/origin/r0.9
  remotes/origin/r1.0

이중에서 일단은 master 버전을 build해보겠습니다.

test@k8002:~/tensorflow$ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.

여기서 일단 환경 변수 확인을 해보겠습니다.

test@k8002:~/tensorflow$ env | grep PATH
LD_LIBRARY_PATH=/usr/local/cuDNN:/usr/local/cuda-8.0/lib64
PATH=/home/test/bin:/home/test/.local/bin:/usr/local/cuda-8.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin

Github에서 download 받은 tensorflow를 build할 때도, PowerAI에 포함된 bazel을 사용하면 편리합니다.  그를 위해, 현재의 PATH 환경변수들을 PowerAI 기준으로 고쳐줘야 하는데, 그를 위해서는 아래와 같이 PowerAI에 포함된 script 하나만 수행해주면 됩니다.

test@k8002:~/tensorflow$ source /opt/DL/tensorflow/bin/tensorflow-activate

이제 아래와 같이 PATH에 /opt/DL/tensorflow/bin가 추가되고, PYTHONPATH가 추가된 것을 보실 수 있습니다.

test@k8002:~/tensorflow$ env | grep PATH
LD_LIBRARY_PATH=/usr/local/cuDNN:/usr/local/cuda-8.0/lib64
PATH=/home/test/bin:/home/test/.local/bin:/usr/local/cuda-8.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/opt/DL/tensorflow/bin
PYTHONPATH=/opt/DL/tensorflow/lib/python2.7/site-packages

여기에 추가로 bazel의 PATH도 잡아줘야 합니다.  Tensorflow의 build를 위해서는 먼저 bazel을 build해야 하는데, 사실 그것이 tensorflow build보다 더 힘듭니다.  여기서는 PowerAI에 포함된 bazel을 간편하게 사용하겠습니다.

test@k8002:~/tensorflow$ export PATH=/opt/DL/bazel/bin:$PATH

이제 configure부터 시작하여 build에 들어갑니다.

test@k8002:~/tensorflow$ ./configure
Please specify the location of python. [Default is /usr/bin/python]:
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]:
Do you wish to use jemalloc as the malloc implementation? [Y/n]
jemalloc enabled
Do you wish to build TensorFlow with Google Cloud Platform support? [y/N]
No Google Cloud Platform support will be enabled for TensorFlow
Do you wish to build TensorFlow with Hadoop File System support? [y/N]
No Hadoop File System support will be enabled for TensorFlow
Do you wish to build TensorFlow with the XLA just-in-time compiler (experimental)? [y/N]
No XLA support will be enabled for TensorFlow
Found possible Python library paths:
  /opt/DL/tensorflow/lib/python2.7/site-packages
  /usr/local/lib/python2.7/dist-packages
  /usr/lib/python2.7/dist-packages
Please input the desired Python library path to use.  Default is [/opt/DL/tensorflow/lib/python2.7/site-packages]
/usr/lib/python2.7/dist-packages

Do you wish to build TensorFlow with OpenCL support? [y/N]
No OpenCL support will be enabled for TensorFlow
Do you wish to build TensorFlow with CUDA support? [y/N] y
CUDA support will be enabled for TensorFlow
Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]:
Please specify the CUDA SDK version you want to use, e.g. 7.0. [Leave empty to use system default]: 8.0
Please specify the location where CUDA 8.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: /usr/local/cuda-8.0
Please specify the Cudnn version you want to use. [Leave empty to use system default]: 5
Please specify the location where cuDNN 5 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda-8.0]: /usr/lib/powerpc64le-linux-gnu  
Please specify a list of comma-separated Cuda compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
Please note that each additional compute capability significantly increases your build time and binary size.
[Default is: "3.5,5.2"]: 6.0    (--> P100은 6.0, K80의 경우 3.7)
Extracting Bazel installation...
.............
INFO: Starting clean (this may take a while). Consider using --expunge_async if the clean takes more than several minutes.
............
WARNING: /home/test/.cache/bazel/_bazel_test/3a51eda24c560ebddf29b66b8fa0460e/external/protobuf/protobuf.bzl:90:19: Variables HOST_CFG and DATA_CFG are deprecated in favor of strings "host" and "data" correspondingly.
WARNING: /home/test/.cache/bazel/_bazel_test/3a51eda24c560ebddf29b66b8fa0460e/external/protobuf/protobuf.bzl:96:28: Variables HOST_CFG and DATA_CFG are deprecated in favor of strings "host" and "data" correspondingly.
INFO: Downloading http://pilotfiber.dl.sourceforge.net/project/boost/boost/1.6\
1.0/boost_1_61_0.tar.gz: 104,733,800 bytes
...
INFO: All external dependencies fetched successfully.
Configuration finished

configure가 끝나면 이제 bazel build를 합니다.

test@k8002:~/tensorflow$ bazel build -c opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
...
Target //tensorflow/tools/pip_package:build_pip_package up-to-date:
  bazel-bin/tensorflow/tools/pip_package/build_pip_package
INFO: Elapsed time: 256.806s, Critical Path: 203.45s

윗 단계가 성공적으로 끝나면 다음과 같이 /tmp/tensorflow_pkg directory에 pip package를 build합니다.

test@k8002:~/tensorflow$ bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
Thu Mar 16 15:03:32 KST 2017 : === Using tmpdir: /tmp/tmp.LCBvnwu8ZZ
~/tensorflow/bazel-bin/tensorflow/tools/pip_package/build_pip_package.runfiles ~/tensorflow
~/tensorflow
/tmp/tmp.LCBvnwu8ZZ ~/tensorflow
Thu Mar 16 15:03:33 KST 2017 : === Building wheel
warning: no files found matching '*.dll' under directory '*'
warning: no files found matching '*.lib' under directory '*'
~/tensorflow
Thu Mar 16 15:03:44 KST 2017 : === Output wheel file is in: /tmp/tensorflow_pkg

이제 /tmp/tensorflow_pkg directory에 생성된 wheel file을 이용하여 pip install을 하면 tensorflow 설치가 완료됩니다.

test@k8002:~/tensorflow$ sudo pip install /tmp/tensorflow_pkg/tensorflow-1.0.1-cp27-cp27mu-linux_ppc64le.whl
...
  Downloading funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock>=2.0.0->tensorflow==1.0.1)
Installing collected packages: werkzeug, protobuf, funcsigs, mock, tensorflow
  Found existing installation: protobuf 2.6.1
    Not uninstalling protobuf at /usr/lib/python2.7/dist-packages, outside environment /usr
  Found existing installation: funcsigs 0.4
    Not uninstalling funcsigs at /usr/lib/python2.7/dist-packages, outside environment /usr
  Found existing installation: mock 1.3.0
    Not uninstalling mock at /usr/lib/python2.7/dist-packages, outside environment /usr
Successfully installed funcsigs-1.0.2 mock-2.0.0 protobuf-3.2.0 tensorflow-1.0.1 werkzeug-0.12.1
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

이제 tensorflow의 버전을 확인해보겠습니다.

test@k8002:~/tensorflow$ pip show tensorflow
---
Metadata-Version: 2.0
Name: tensorflow
Version: 0.12.0
Summary: TensorFlow helps the tensors flow
Home-page: http://tensorflow.org/
Author: Google Inc.
Author-email: opensource@google.com
Installer: pip
License: Apache 2.0
Location: /opt/DL/tensorflow/lib/python2.7/site-packages
Requires: numpy, mock, six, protobuf, wheel
Classifiers:
  Development Status :: 4 - Beta
  Intended Audience :: Developers
  Intended Audience :: Education
  Intended Audience :: Science/Research
  License :: OSI Approved :: Apache Software License
  Programming Language :: Python :: 2.7
  Topic :: Scientific/Engineering :: Mathematics
  Topic :: Software Development :: Libraries :: Python Modules
  Topic :: Software Development :: Libraries
Entry-points:
  [console_scripts]
  tensorboard = tensorflow.tensorboard.tensorboard:main
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.


위와 같이 version이 방금 설치한 v1.0이 아니라 v0.12로 나오는 것은 환경변수 PYTHONPATH가 아직 /opt/DL/tensorflow/lib/python2.7/site-packages으로 잡혀 있기 때문에, 이전에 PowerAI 설치 때 함께 설치된 tensorflow 0.12가 보이는 것입니다.  이를 configure할 때 주었던 값인 /usr/local/lib/python2.7/dist-packages으로 바꾼 뒤 다시 해보겠습니다.


test@k8002:~/tensorflow$ export PYTHONPATH=/usr/local/lib/python2.7/dist-packages:$PYTHONPATH

test@k8002:~/tensorflow$ pip show tensorflow
---
Metadata-Version: 2.0
Name: tensorflow
Version: 1.0.1
Summary: TensorFlow helps the tensors flow
Home-page: http://tensorflow.org/
Author: Google Inc.
Author-email: opensource@google.com
Installer: pip
License: Apache 2.0
Location: /usr/local/lib/python2.7/dist-packages
Requires: werkzeug, six, protobuf, wheel, numpy, mock
Classifiers:
  Development Status :: 4 - Beta
  Intended Audience :: Developers
  Intended Audience :: Education
  Intended Audience :: Science/Research
  License :: OSI Approved :: Apache Software License
  Programming Language :: Python :: 2.7
  Topic :: Scientific/Engineering :: Mathematics
  Topic :: Software Development :: Libraries :: Python Modules
  Topic :: Software Development :: Libraries
Entry-points:
  [console_scripts]
  tensorboard = tensorflow.tensorboard.tensorboard:main
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

위와 같이 v1.0.1로 제대로 보이는 것을 보실 수 있습니다.

이번에는 r0.12를 build해보겠습니다.

test@k8002:~/tensorflow$ git checkout r0.12
Switched to branch 'r0.12'
Your branch is up-to-date with 'origin/r0.12'.

동일하게, configure부터 시작하여 build에 들어갑니다.

test@k8002:~/tensorflow$ ./configure
~/tensorflow ~/tensorflow
Please specify the location of python. [Default is /usr/bin/python]:
Do you wish to build TensorFlow with Google Cloud Platform support? [y/N]
No Google Cloud Platform support will be enabled for TensorFlow
Do you wish to build TensorFlow with Hadoop File System support? [y/N]
No Hadoop File System support will be enabled for TensorFlow
Found possible Python library paths:
  /usr/local/lib/python2.7/dist-packages
  /usr/lib/python2.7/dist-packages
Please input the desired Python library path to use.  Default is [/usr/local/lib/python2.7/dist-packages]
/usr/lib/python2.7/dist-packages
Do you wish to build TensorFlow with OpenCL support? [y/N]
No OpenCL support will be enabled for TensorFlow
Do you wish to build TensorFlow with GPU support? [y/N] y
GPU support will be enabled for TensorFlow
Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]:
Please specify the Cuda SDK version you want to use, e.g. 7.0. [Leave empty to use system default]: 8.0
Please specify the location where CUDA 8.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: /usr/local/cuda-8.0
Please specify the Cudnn version you want to use. [Leave empty to use system default]: 5
Please specify the location where cuDNN 5 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda-8.0]: /usr/lib/powerpc64le-linux-gnu
Please specify a list of comma-separated Cuda compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
Please note that each additional compute capability significantly increases your build time and binary size.
[Default is: "3.5,5.2"]: 6.0    (--> P100은 6.0, K80의 경우 3.7)
............
INFO: Starting clean (this may take a while). Consider using --expunge_async if the clean takes more than several minutes.
............
INFO: All external dependencies fetched successfully.
Configuration finished



test@k8002:~/tensorflow$ bazel build -c opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
...                                                                                                                                          ^
At global scope:
cc1plus: warning: unrecognized command line option '-Wno-self-assign'
Target //tensorflow/tools/pip_package:build_pip_package up-to-date:
  bazel-bin/tensorflow/tools/pip_package/build_pip_package
INFO: Elapsed time: 306.555s, Critical Path: 273.68s



test@k8002:~/tensorflow$ bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
Fri Mar 17 16:16:24 KST 2017 : === Using tmpdir: /tmp/tmp.L1dwTgVhwv
~/tensorflow/bazel-bin/tensorflow/tools/pip_package/build_pip_package.runfiles ~/tensorflow
~/tensorflow
/tmp/tmp.L1dwTgVhwv ~/tensorflow
Fri Mar 17 16:16:25 KST 2017 : === Building wheel
~/tensorflow
Fri Mar 17 16:16:41 KST 2017 : === Output wheel file is in: /tmp/tensorflow_pkg


이제 /tmp/tensorflow_pkg를 보시면 아까 build했던 1.0.1 버전의 wheel file과 함께 새로 0.12 버전의 wheel file이 만들어진 것을 보실 수 있습니다.

test@k8002:~/tensorflow$ ls -l /tmp/tensorflow_pkg/
total 109520
-rw-rw-r-- 1 test test 61576804 Mar 17 16:16 tensorflow-0.12.1-cp27-cp27mu-linux_ppc64le.whl
-rw-rw-r-- 1 test test 50565190 Mar 16 15:03 tensorflow-1.0.1-cp27-cp27mu-linux_ppc64le.whl

그것을 pip install 명령으로 설치합니다.   아래와 같이, 기존의 tensorflow-1.0.1을 uninstall하면서 0.12가 설치됩니다.

test@k8002:~/tensorflow$ sudo pip install /tmp/tensorflow_pkg/tensorflow-0.12.1-cp27-cp27mu-linux_ppc64le.whl
The directory '/home/test/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/test/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Processing /tmp/tensorflow_pkg/tensorflow-0.12.1-cp27-cp27mu-linux_ppc64le.whl
Requirement already satisfied (use --upgrade to upgrade): numpy>=1.11.0 in /home/test/.local/lib/python2.7/site-packages (from tensorflow==0.12.1)
Requirement already satisfied (use --upgrade to upgrade): wheel in /usr/lib/python2.7/dist-packages (from tensorflow==0.12.1)
Requirement already satisfied (use --upgrade to upgrade): mock>=2.0.0 in /usr/local/lib/python2.7/dist-packages (from tensorflow==0.12.1)
Requirement already satisfied (use --upgrade to upgrade): six>=1.10.0 in /home/test/.local/lib/python2.7/site-packages (from tensorflow==0.12.1)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.1.0 in /usr/local/lib/python2.7/dist-packages (from tensorflow==0.12.1)
Requirement already satisfied (use --upgrade to upgrade): funcsigs>=1; python_version < "3.3" in /usr/local/lib/python2.7/dist-packages (from mock>=2.0.0->tensorflow==0.12.1)
Requirement already satisfied (use --upgrade to upgrade): pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock>=2.0.0->tensorflow==0.12.1)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from protobuf>=3.1.0->tensorflow==0.12.1)
Installing collected packages: tensorflow
  Found existing installation: tensorflow 1.0.1
    Uninstalling tensorflow-1.0.1:
      Successfully uninstalled tensorflow-1.0.1
Successfully installed tensorflow-0.12.1
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.


다음과 같이 tensorflow의 버전이 0.12.1로 변경된 것을 확인하실 수 있습니다.

test@k8002:~/tensorflow$ pip show tensorflow
---
Metadata-Version: 2.0
Name: tensorflow
Version: 0.12.1
Summary: TensorFlow helps the tensors flow
Home-page: http://tensorflow.org/
Author: Google Inc.
Author-email: opensource@google.com
Installer: pip
License: Apache 2.0
Location: /usr/local/lib/python2.7/dist-packages
Requires: numpy, wheel, mock, six, protobuf
Classifiers:
  Development Status :: 4 - Beta
  Intended Audience :: Developers
  Intended Audience :: Education
  Intended Audience :: Science/Research
  License :: OSI Approved :: Apache Software License
  Programming Language :: Python :: 2.7
  Topic :: Scientific/Engineering :: Mathematics
  Topic :: Software Development :: Libraries :: Python Modules
  Topic :: Software Development :: Libraries
Entry-points:
  [console_scripts]
  tensorboard = tensorflow.tensorboard.tensorboard:main
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

댓글 17개:

  1. Thank you for allowing me to read it, welcome to the next in a recent article. And thanks for sharing the nice article, keep posting or updating news article.
    Click here:
    Microsoft azure training in chennai
    Click here:
    angularjs6 Training in Chennai
    Click here:
    angularjs Training in online
    Click here:
    angularjs training in tambaram

    답글삭제
  2. Good Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging.
    Blueprism training in Chennai

    Blueprism training in Bangalore

    Blueprism training in Pune

    Blueprism online training

    Blueprism training in tambaram

    답글삭제
  3. 작성자가 댓글을 삭제했습니다.

    답글삭제
  4. This is ansuperior writing service point that doesn't always sink in within the context of the classroom. In the first superior writing service paragraph you either hook the reader's interest or lose it. Of course your teacher, who's getting paid to teach you how to write an good essay, 
    Data Science training in kalyan nagar | Data Science training in OMR
    Data Science training in chennai | Data science training in velachery
    Data science training in tambaram | Data science training in jaya nagar

    답글삭제
  5. Awesome! Education is the extreme motivation that open the new doors of data and material. So we always need to study around the things and the new part of educations with that we are not mindful.

    angularjs Training in electronic-city

    angularjs online Training

    angularjs Training in marathahalli

    angularjs interview questions and answers

    angularjs Training in bangalore

    답글삭제
  6. The knowledge of technology you have been sharing thorough this post is very much helpful to develop new idea. here by i also want to share this.

    advanced excel training in bangalore

    답글삭제
  7. I really appreciate this post. I’ve been looking all over for this! Thank goodness I found it on Bing. You’ve made my day! Thx again!

    angularjs online training

    apache spark online training

    informatica mdm online training

    devops online training

    aws online training

    답글삭제
  8. I am so proud of you and your efforts and work make me realize that anything can be done with patience and sincerity. Well I am here to say that your work has inspired me without a doubt.
    Microsoft Azure online training
    Selenium online training
    Java online training
    Python online training
    uipath online training

    답글삭제
  9. Hey, would you mind if I share your blog with my twitter group? There’s a lot of folks that I think would enjoy your content. Please let me know. Thank you.
    Java Training in Chennai | J2EE Training in Chennai | Advanced Java Training in Chennai | Core Java Training in Chennai | Java Training institute in Chennai

    답글삭제
  10. Hi,
    Good job & thank you very much for the new information, i learned something new. Very well written. It was sooo good to read and usefull to improve knowledge. Who want to learn this information most helpful. One who wanted to learn this technology IT employees will always suggest you take python training in bangalore. Because Python course in Bangalore is one of the best that one can do while choosing the course.

    답글삭제
  11. I wish to show thanks to you just for bailing me out of this particular trouble.As a result of checking through the net and meeting techniques that were not productive, I thought my life was done.


    Best PHP Training Institute in Chennai|PHP Course in chennai

    Best .Net Training Institute in Chennai
    Dotnet Training in Chennai
    Dotnet Training in Chennai
    Dotnet Training in Chennai

    답글삭제
  12. Nice content very helpful, It has a very important point which should be noted down. All points mentioned and very well written.Keep Posting & writing such content

    AWS Online Training
    Online AWS Certification Training

    답글삭제