AC922 Redhat에 Theano와 Keras 설치 및 테스트

AC922 Redhat에 python 3.6을 설치하는 것은 다음과 같이 Anaconda를 설치하는 것이 가장 간단하고 가장 편리합니다.

[user1@ac922 ~]$ wget https://repo.continuum.io/archive/Anaconda3-5.1.0-Linux-ppc64le.sh    (python 2.x을 설치하려는 경우는 Anaconda3 대신 Anaconda2로 대체)

[user1@ac922 ~]$ chmod a+x Anaconda3-5.1.0-Linux-ppc64le.sh

[user1@ac922 ~]$ ./Anaconda3-5.1.0-Linux-ppc64le.sh

이렇게 설치가 끝나면 거기에 딸린 pip를 통해서 쉽게 theano와 keras를 설치할 수 있습니다.

[user1@ac922 ~]$ which python
~/anaconda3/bin/python
[user1@ac922 ~]$ python --version
Python 3.6.4 :: Anaconda, Inc.

Keras 2.0.9를 골라서 설치하려면 다음과 같이 하시면 됩니다. 

[user1@ac922 ~]$ pip install keras==2.0.9
Collecting keras==2.0.9
  Using cached Keras-2.0.9-py2.py3-none-any.whl
Requirement already satisfied: pyyaml in ./anaconda3/lib/python3.6/site-packages (from keras==2.0.9)
Requirement already satisfied: scipy>=0.14 in ./anaconda3/lib/python3.6/site-packages (from keras==2.0.9)
Requirement already satisfied: numpy>=1.9.1 in ./anaconda3/lib/python3.6/site-packages (from keras==2.0.9)
Requirement already satisfied: six>=1.9.0 in ./anaconda3/lib/python3.6/site-packages (from keras==2.0.9)
Installing collected packages: keras
Successfully installed keras-2.0.9

Keras를 테스트해보시려면 먼저 tensorflow를 설치하셔야 합니다.  그건 이미 올려둔 아래 posting에 따라 설치하시면 됩니다.

http://hwengineer.blogspot.kr/2018/04/ac922-redhat-74-python-36-tensorflow.html

이제 아래와 같이 keras에서 GPU를 사용하는지 확인해봅니다.

[user1@ac922 ~]$ python
Python 3.6.4 |Anaconda, Inc.| (default, Feb 11 2018, 08:19:13)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

>>> from keras import backend as K
/home/user1/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
Using TensorFlow backend.
....
tensorflow/core/common_runtime/gpu/gpu_device.cc:1195] Creating TensorFlow device (/device:GPU:3) -> (device: 3, name: Tesla V100-SXM2-16GB, pci bus id: 0035:04:00.0, compute capability: 7.0)

>>> K.tensorflow_backend._get_available_gpus()
['/device:GPU:0', '/device:GPU:1', '/device:GPU:2', '/device:GPU:3']

잘 되는 것을 확인하실 수 있습니다.

이제 theano를 설치합니다.  먼저, theano 1.0.0이 GPU를 쓰기 위해서는 pygpu 0.7 이상을 설치해야 합니다.  기본적으로 설치되는 pygpu는 0.6.9이므로, 이걸 그대로 사용하면 다음과 같은 error를 겪게 됩니다.

ERROR (theano.gpuarray): pygpu was configured but could not be imported or is too old (version 0.7 or higher required)

따라서 먼저 pygpu 0.7을 설치해야 하는데, 이건 다음과 같이 libgpuarray라는 package를 설치하면 됩니다.

[user1@ac922 ~]$ git clone https://github.com/Theano/libgpuarray.git

[user1@ac922 ~]$ cd libgpuarray

[user1@ac922 libgpuarray]$ cmake3 .

[user1@ac922 libgpuarray]$ make && sudo make install

이러면 libgpuarray.so가 /usr/local/lib 밑에 설치됩니다.  이것이 우선적으로 loading되도록 LD_LIBRARY_PATH를 아래와 같이 설정합니다.

[user1@ac922 ~]$ export LD_LIBRARY_PATH=/usr/local/lib:/usr/local/cuda-9.1/lib64:/usr/lib:/usr/lib64

그리고 python binding을 위해 아래와 같이 마무리합니다.

[user1@ac922 libgpuarray]$ python setup.py install

이제 pip list를 해보면 다음과 같이 pygpu가 설치된 것을 보실 수 있습니다.

[user1@ac922 ~]$ pip list --format=columns | grep pygpu
pygpu                              0.7.5+11.g04c2892.dirty

이제 theano를 설치합니다.   최신 버전은 1.0.1입니다만 여기서는 1.0.0을 선택합니다.

[user1@ac922 ~]$ pip install theano==1.0.0
Collecting theano==1.0.0
  Downloading Theano-1.0.0.tar.gz (2.9MB)
    100% |████████████████████████████████| 2.9MB 390kB/s
Requirement already satisfied: numpy>=1.9.1 in ./anaconda3/lib/python3.6/site-packages (from theano==1.0.0)
Requirement already satisfied: scipy>=0.14 in ./anaconda3/lib/python3.6/site-packages (from theano==1.0.0)
Requirement already satisfied: six>=1.9.0 in ./anaconda3/lib/python3.6/site-packages (from theano==1.0.0)
Building wheels for collected packages: theano
  Running setup.py bdist_wheel for theano ... done
  Stored in directory: /home/user1/.cache/pip/wheels/e1/38/41/29fead4ea90d8fb9e23af0ba80d24222f8ba6effe93896ecbf
Successfully built theano
Installing collected packages: theano
Successfully installed theano-1.0.0

여기서 끝나는 것이 아니라, theano의 환경 설정을 아래와 같이 해줘야 합니다.

[user1@ac922 ~]$ vi ~/.theanorc

[global]

device = cuda0

floatX = float32

[cuda]

cuda.root = /usr/local/cuda-9.1

[dnn]

library_path = /usr/local/cuda-9.1/targets/ppc64le-linux/lib

include_path = /usr/local/cuda-9.1/targets/ppc64le-linux/include

enabled = True

확인은 다음과 같이 합니다.

[user1@ac922 ~]$ pip list --format=columns | grep -e 'Keras\|Theano'
Keras                              2.0.9
Theano                             1.0.0


확인차 아래와 같이 연습용 theano sample을 하나 돌려보겠습니다. 

[user1@ac922 files]$ vi theano_ex1.py
from theano import function, config, shared, tensor
import numpy
import time
vlen = 10 * 30 * 768  # 10 x #cores x # threads per core
iters = 1000
rng = numpy.random.RandomState(22)
x = shared(numpy.asarray(rng.rand(vlen), config.floatX))
f = function([], tensor.exp(x))
print(f.maker.fgraph.toposort())
t0 = time.time()
for i in range(iters):
    r = f()
t1 = time.time()
print("Looping %d times took %f seconds" % (iters, t1 - t0))
print("Result is %s" % (r,))
if numpy.any([isinstance(x.op, tensor.Elemwise) and
              ('Gpu' not in type(x.op).__name__)
              for x in f.maker.fgraph.toposort()]):
    print('Used the cpu')
else:
    print('Used the gpu')

여기서 "device=cpu" 또는 "device=cuda#"이라는 THEANO_FLAGS를 이용하여 CPU 또는 GPU를 쓰도록 제어할 수 있습니다.   먼저 CPU로 연산하도록 하는 예입니다.

[user1@ac922 files]$ time THEANO_FLAGS=device=cpu python theano_ex1.py
[Elemwise{exp,no_inplace}(<TensorType(float64, vector)>)]
Looping 1000 times took 6.145363 seconds
Result is [ 1.23178032  1.61879341  1.52278065 ...,  2.20771815  2.29967753
  1.62323285]
Used the cpu

real    0m6.787s
user    0m6.739s
sys     0m0.050s

이제 dev=cuda#으로 해봅니다.

[user1@ac922 files]$ time THEANO_FLAGS=device=cuda0 python theano_ex1.py

Traceback (most recent call last):
  File "__init__.pxd", line 1011, in numpy.import_array
RuntimeError: module compiled against API version 0xc but this version of numpy is 0xb
...

억, error입니다.   Googling해보면 이는 numpy의 버전이 낮아서 생기는 문제입니다.  다음과 같이 버전을 높여주면 됩니다.

[user1@ac922 ~]$ pip install numpy --upgrade
Collecting numpy
Installing collected packages: numpy
  Found existing installation: numpy 1.13.3
    Uninstalling numpy-1.13.3:
      Successfully uninstalled numpy-1.13.3
Successfully installed numpy-1.14.2

이제 다시 수행해봅니다. 

[user1@ac922 files]$ time THEANO_FLAGS=device=cuda0 python theano_ex1.py
Using cuDNN version 7005 on context None
Mapped name None to device cuda0: Tesla V100-SXM2-16GB (0004:04:00.0)
[GpuElemwise{exp,no_inplace}(<GpuArrayType<None>(float32, vector)>), HostFromGpu(gpuarray)(GpuElemwise{exp,no_inplace}.0)]
Looping 1000 times took 0.107720 seconds
Result is [1.2317803 1.6187935 1.5227807 ... 2.2077181 2.2996776 1.623233 ]
Used the gpu

real    0m4.753s
user    0m2.746s
sys     0m2.001s

잘 됩니다.  CPU를 이용하는 것보다 당연히 더 빠릅니다.  아까 $HOME/.theanorc에 device를 cuda0로 해놓았기 때문에, 사실 여기서는 THEANO_FLAGS를 따로 지정하지 않아도 default로 cuda0, 즉 GPU 0번을 이용하여 수행됩니다.

[user1@ac922 files]$ time python theano_ex1.py
Using cuDNN version 7005 on context None
Mapped name None to device cuda0: Tesla V100-SXM2-16GB (0004:04:00.0)
[GpuElemwise{exp,no_inplace}(<GpuArrayType<None>(float32, vector)>), HostFromGpu(gpuarray)(GpuElemwise{exp,no_inplace}.0)]
Looping 1000 times took 0.139874 seconds
Result is [1.2317803 1.6187935 1.5227807 ... 2.2077181 2.2996776 1.623233 ]
Used the gpu

real    0m4.725s
user    0m2.656s
sys     0m2.057s

혹시 더 최신의 Theano를 source로부터 직접 build하고 싶으시면 다음과 같이 하시면 됩니다.

[user1@ac922 files]$ pip install -U git+https://github.com/Theano/Theano.git

Collecting Theano from git+https://github.com/Theano/Theano.git#egg=Theano

  Cloning https://github.com/Theano/Theano.git to /tmp/pip-build-wpolv6wn/Theano

Collecting numpy>=1.9.1 (from Theano)

  Downloading numpy-1.14.2.zip (4.9MB)

    100% |████████████████████████████████| 4.9MB 232kB/s

Collecting scipy>=0.14 (from Theano)

  Downloading scipy-1.0.1.tar.gz (15.5MB)

    100% |████████████████████████████████| 15.5MB 75kB/s

Requirement already up-to-date: six>=1.9.0 in /home/user1/anaconda3/lib/python3.6/site-packages (from Theano)

Building wheels for collected packages: numpy, scipy

  Running setup.py bdist_wheel for numpy ... done

  Stored in directory: /home/user1/.cache/pip/wheels/7d/03/d4/050b7a6ff45c26a43b5a5b9e4ad1983b35b2aa1df42d44b5b1

  Running setup.py bdist_wheel for scipy ... done

  Stored in directory: /home/user1/.cache/pip/wheels/bc/15/5b/4b524f3adb3be2b81d59001f2b06ac758b7b73e77b4345a8b7

Successfully built numpy scipy

Installing collected packages: numpy, scipy, Theano

  Found existing installation: numpy 1.13.3

    Uninstalling numpy-1.13.3:

      Successfully uninstalled numpy-1.13.3

  Found existing installation: scipy 1.0.0

    Uninstalling scipy-1.0.0:

      Successfully uninstalled scipy-1.0.0

  Found existing installation: Theano 1.0.0

    Uninstalling Theano-1.0.0:

      Successfully uninstalled Theano-1.0.0

  Running setup.py install for Theano ... done

Successfully installed Theano-1.0.1+70.gd9b8a7c numpy-1.14.2 scipy-1.0.1

[user1@ac922 files]$ pip list --format=columns | grep -i theano

Theano                             1.0.1+70.gd9b8a7c