WML-CE (Watson Machine Learning Community Edition, 구 PowerAI) 1.6.2 설치

IBM POWER 아키텍처 (POWER8/9, 즉 ppc64le)에서 tensorflow나 caffe 등 각종 deep learning framework을 제공해주던 무료 toolkit인 기존 PowerAI는 이미 다들 아시는 바와 같이 이름을 Watson Machine Learning Community Edition (WML-CE)로 변경했습니다.  물론 여전히 무료입니다만, 기존처럼 *.rpm이나 *.deb의 형태로 제공하지 않고 아예 별도의 conda channel을 만들어서 conda에서 설치하도록 하고 있습니다.  따라서, Anaconda가 prerequsite이며, 2020년 2월 초 현재 최신 버전인 1.6.2는 Ananconda 2019.07을 prerequisite으로 하고 있습니다.  아예 모든 것이 설치된 docker image 형태로도 제공됩니다.

자세한 원본 manual은 아래 link를 참조하시면 됩니다.

https://www.ibm.com/support/knowledgecenter/SS5SF7_1.6.2/navigation/wmlce_planning.html
https://www.ibm.com/support/knowledgecenter/SS5SF7_1.6.2/navigation/wmlce_install.html

여기서는 ppc64le Ubuntu 18.04, Python 3.7.5 환경에서 WML-CE 1.6.2를 설치해보겠습니다.

cecuser@p1234-kvm1:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.2 LTS
Release:        18.04
Codename:       bionic

먼저 Anaconda 2019.07 버전을 download 받아 설치합니다.

cecuser@p1234-kvm1:~$ wget https://repo.continuum.io/archive/Anaconda3-2019.07-Linux-ppc64le.sh

cecuser@p1234-kvm1:~$ chmod a+x Anaconda3-2019.07-Linux-ppc64le.sh

cecuser@p1234-kvm1:~$ ./Anaconda3-2019.07-Linux-ppc64le.sh

설치가 끝나면 ~/.bashrc를 수행하여  conda init을 수행합니다.

cecuser@p1234-kvm1:~$ . ~/.bashrc

이제 IBM이 제공하는 WML-CE를 위한 conda channel을 conda에 추가합니다.

(base) cecuser@p1234-kvm1:~$ conda config --prepend channels https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/

Conda 가상 환경을 생성하여 거기에 WML-CE를 설치하기를 권장하므로, 먼저 python 3.7.5 환경으로 wmlce_env라는 이름의 virtual env를 만듭니다.

(base) cecuser@p1234-kvm1:~$ conda create --name wmlce_env python=3.7.5
...
The following NEW packages will be INSTALLED:

  _libgcc_mutex      pkgs/main/linux-ppc64le::_libgcc_mutex-0.1-main
  ca-certificates    pkgs/main/linux-ppc64le::ca-certificates-2020.1.1-0
  certifi            pkgs/main/linux-ppc64le::certifi-2019.11.28-py37_0
  libedit            pkgs/main/linux-ppc64le::libedit-3.1.20181209-hc058e9b_0
  libffi             pkgs/main/linux-ppc64le::libffi-3.2.1-hf62a594_5
  libgcc-ng          pkgs/main/linux-ppc64le::libgcc-ng-8.2.0-h822a55f_1
  libstdcxx-ng       pkgs/main/linux-ppc64le::libstdcxx-ng-8.2.0-h822a55f_1
  ncurses            pkgs/main/linux-ppc64le::ncurses-6.1-he6710b0_1
  openssl            pkgs/main/linux-ppc64le::openssl-1.1.1d-h7b6447c_3
  pip                pkgs/main/linux-ppc64le::pip-20.0.2-py37_1
  python             pkgs/main/linux-ppc64le::python-3.7.5-h4134adf_0
  readline           pkgs/main/linux-ppc64le::readline-7.0-h7b6447c_5
  setuptools         pkgs/main/linux-ppc64le::setuptools-45.1.0-py37_0
  sqlite             pkgs/main/linux-ppc64le::sqlite-3.30.1-h7b6447c_0
  tk                 pkgs/main/linux-ppc64le::tk-8.6.8-hbc83047_0
  wheel              pkgs/main/linux-ppc64le::wheel-0.34.1-py37_0
  xz                 pkgs/main/linux-ppc64le::xz-5.2.4-h14c3975_4
  zlib               pkgs/main/linux-ppc64le::zlib-1.2.11-h7b6447c_3


Proceed ([y]/n)? y
...
Downloading and Extracting Packages
ca-certificates-2020 | 125 KB    | ##################################### | 100%
setuptools-45.1.0    | 511 KB    | ##################################### | 100%
pip-20.0.2           | 1.7 MB    | ##################################### | 100%
sqlite-3.30.1        | 2.3 MB    | ##################################### | 100%
wheel-0.34.1         | 50 KB     | ##################################### | 100%
python-3.7.5         | 32.5 MB   | ##################################### | 100%
openssl-1.1.1d       | 3.8 MB    | ##################################### | 100%
certifi-2019.11.28   | 156 KB    | ##################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate wmlce_env
#
# To deactivate an active environment, use
#
#     $ conda deactivate

이제 conda 가상환경인 wmlce_env를 활성화합니다.

(base) cecuser@p1234-kvm1:~$ conda activate wmlce_env

이제 WML-CE를 설치합니다.  Conda package 이름은 여전히 PowerAI로 되어 있는 점에 유의하십시요.  아래와 같이 하면 tensorflow와 caffe2, pytorch 등 WML-CE에서 지원하는 모든 deep learning framework이 한꺼번에 다 설치됩니다.  혹시 WML-CE 전체를 설치하지 않고 가령 PyTorch만 설치하려 할 때는 그냥 conda install pytorch 라고 하시면 됩니다.

아래의 명령어로 어느어느 package들이 설치되는지 보여드리기 위해 긴 ouput을 일부러 다 옮겨 붙였습니다.

(wmlce_env) cecuser@p1234-kvm1:~$ conda install powerai
....
The following NEW packages will be INSTALLED:

  _py-xgboost-mutex  ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::_py-xgboost-mutex-1.0-gpu_590.g8a21f75
  _pytorch_select    ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::_pytorch_select-2.0-gpu_20238.g1faf942
  _tflow_select      ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::_tflow_select-2.1.0-gpu_840.g50de12c
  absl-py            pkgs/main/linux-ppc64le::absl-py-0.7.1-py37_0
  apex               ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::apex-0.1.0_1.6.2-py37_596.g1eb5c77
  asn1crypto         pkgs/main/linux-ppc64le::asn1crypto-1.3.0-py37_0
  astor              pkgs/main/linux-ppc64le::astor-0.7.1-py37_0
  atomicwrites       pkgs/main/linux-ppc64le::atomicwrites-1.3.0-py37_1
  attrs              pkgs/main/noarch::attrs-19.3.0-py_0
  blas               pkgs/main/linux-ppc64le::blas-1.0-openblas
  bokeh              pkgs/main/linux-ppc64le::bokeh-1.4.0-py37_0
  boost              pkgs/main/linux-ppc64le::boost-1.67.0-py37_4
  bzip2              pkgs/main/linux-ppc64le::bzip2-1.0.8-h7b6447c_0
  c-ares             pkgs/main/linux-ppc64le::c-ares-1.15.0-h7b6447c_1001
  caffe              ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::caffe-1.0_1.6.2-5184.g7b10df4
  caffe-base         ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::caffe-base-1.0_1.6.2-gpu_py37_5184.g7b10df4
  cairo              pkgs/main/linux-ppc64le::cairo-1.14.12-h8948797_3
  cffi               pkgs/main/linux-ppc64le::cffi-1.12.3-py37h2e261b9_0
  chardet            pkgs/main/linux-ppc64le::chardet-3.0.4-py37_1003
  click              pkgs/main/linux-ppc64le::click-7.0-py37_0
  cloudpickle        pkgs/main/noarch::cloudpickle-1.2.2-py_0
  coverage           pkgs/main/linux-ppc64le::coverage-5.0-py37h7b6447c_0
  cryptography       pkgs/main/linux-ppc64le::cryptography-2.8-py37h1ba5d50_0
  cudatoolkit        ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::cudatoolkit-10.1.243-616.gc122b8b
  cudnn              ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::cudnn-7.6.3_10.1-590.g5627c5e
  cycler             pkgs/main/linux-ppc64le::cycler-0.10.0-py37_0
  cytoolz            pkgs/main/linux-ppc64le::cytoolz-0.10.1-py37h7b6447c_0
  dask               pkgs/main/noarch::dask-2.3.0-py_0
  dask-core          pkgs/main/noarch::dask-core-2.3.0-py_0
  dask-cuda          ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::dask-cuda-0.9.1-py37_573.g9af8baa
  dask-xgboost       ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::dask-xgboost-0.1.7-py37_579.g8a31cf5
  ddl                ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::ddl-1.5.0-py37_1287.gc90c6f2
  ddl-tensorflow     ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::ddl-tensorflow-1.5.0-py37_1007.g8dbb51d
  decorator          pkgs/main/noarch::decorator-4.4.1-py_0
  distributed        pkgs/main/noarch::distributed-2.3.2-py_1
  ffmpeg             pkgs/main/linux-ppc64le::ffmpeg-4.0-hcdf2ecd_0
  fontconfig         pkgs/main/linux-ppc64le::fontconfig-2.13.0-h9420a91_0
  freeglut           pkgs/main/linux-ppc64le::freeglut-3.0.0-hf484d3e_5
  freetype           pkgs/main/linux-ppc64le::freetype-2.9.1-h8a8886c_0
  fsspec             pkgs/main/noarch::fsspec-0.6.2-py_0
  future             pkgs/main/linux-ppc64le::future-0.17.1-py37_0
  gast               pkgs/main/linux-ppc64le::gast-0.2.2-py37_0
  gflags             ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::gflags-2.2.2-1624.g17209b3
  glib               pkgs/main/linux-ppc64le::glib-2.63.1-h5a9c865_0
  glog               ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::glog-0.3.5-1613.gd054598
  google-pasta       ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::google-pasta-0.1.6-py37_564.g04df2d9
  graphite2          pkgs/main/linux-ppc64le::graphite2-1.3.13-h23475e2_0
  graphsurgeon       ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::graphsurgeon-0.4.1-py37_612.gb2bf6b9
  grpcio             pkgs/main/linux-ppc64le::grpcio-1.16.1-py37hf8bcb03_1
  h5py               pkgs/main/linux-ppc64le::h5py-2.8.0-py37h8d01980_0
  harfbuzz           pkgs/main/linux-ppc64le::harfbuzz-1.8.8-hffaf4a1_0
  hdf5               pkgs/main/linux-ppc64le::hdf5-1.10.2-hba1933b_1
  heapdict           pkgs/main/noarch::heapdict-1.0.1-py_0
  hypothesis         pkgs/main/linux-ppc64le::hypothesis-3.59.1-py37h39e3cac_0
  icu                pkgs/main/linux-ppc64le::icu-58.2-h64fc554_1
  idna               pkgs/main/linux-ppc64le::idna-2.8-py37_0
  imageio            pkgs/main/linux-ppc64le::imageio-2.6.1-py37_0
  importlib_metadata pkgs/main/linux-ppc64le::importlib_metadata-1.4.0-py37_0
  jasper             pkgs/main/linux-ppc64le::jasper-2.0.14-h07fcdf6_1
  jinja2             pkgs/main/noarch::jinja2-2.10.3-py_0
  joblib             pkgs/main/linux-ppc64le::joblib-0.13.2-py37_0
  jpeg               pkgs/main/linux-ppc64le::jpeg-9b-hcb7ba68_2
  keras-applications pkgs/main/noarch::keras-applications-1.0.8-py_0
  keras-preprocessi~ pkgs/main/noarch::keras-preprocessing-1.1.0-py_1
  kiwisolver         pkgs/main/linux-ppc64le::kiwisolver-1.1.0-py37he6710b0_0
  leveldb            pkgs/main/linux-ppc64le::leveldb-1.20-hf484d3e_1
  libboost           pkgs/main/linux-ppc64le::libboost-1.67.0-h46d08c1_4
  libgfortran-ng     pkgs/main/linux-ppc64le::libgfortran-ng-7.3.0-h822a55f_1
  libglu             pkgs/main/linux-ppc64le::libglu-9.0.0-hf484d3e_1
  libopenblas        pkgs/main/linux-ppc64le::libopenblas-0.3.6-h5a2b251_1
  libopencv          ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::libopencv-3.4.7-725.g92aa195
  libopus            pkgs/main/linux-ppc64le::libopus-1.3-h7b6447c_0
  libpng             pkgs/main/linux-ppc64le::libpng-1.6.37-hbc83047_0
  libprotobuf        ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::libprotobuf-3.8.0-577.g45759bb
  libtiff            pkgs/main/linux-ppc64le::libtiff-4.1.0-h2733197_0
  libuuid            pkgs/main/linux-ppc64le::libuuid-1.0.3-h1bed415_2
  libvpx             pkgs/main/linux-ppc64le::libvpx-1.7.0-hf484d3e_0
  libxcb             pkgs/main/linux-ppc64le::libxcb-1.13-h1bed415_0
  libxgboost-base    ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::libxgboost-base-0.90-gpu_590.g8a21f75
  libxml2            pkgs/main/linux-ppc64le::libxml2-2.9.9-hea5a465_1
  llvmlite           pkgs/main/linux-ppc64le::llvmlite-0.29.0-py37hd408876_0
  lmdb               pkgs/main/linux-ppc64le::lmdb-0.9.22-hf484d3e_1
  locket             pkgs/main/linux-ppc64le::locket-0.2.0-py37_1
  markdown           pkgs/main/linux-ppc64le::markdown-3.1.1-py37_0
  markupsafe         pkgs/main/linux-ppc64le::markupsafe-1.1.1-py37h7b6447c_0
  matplotlib         pkgs/main/linux-ppc64le::matplotlib-3.1.2-py37_1
  matplotlib-base    pkgs/main/linux-ppc64le::matplotlib-base-3.1.2-py37h4fdacc2_1
  mock               pkgs/main/linux-ppc64le::mock-2.0.0-py37_0
  more-itertools     pkgs/main/noarch::more-itertools-8.0.2-py_0
  msgpack-python     pkgs/main/linux-ppc64le::msgpack-python-0.6.1-py37hfd86e86_1
  nccl               ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::nccl-2.4.8-586.gdba67b7
  networkx           pkgs/main/linux-ppc64le::networkx-2.2-py37_1
  ninja              pkgs/main/linux-ppc64le::ninja-1.9.0-py37hfd86e86_0
  nomkl              pkgs/main/linux-ppc64le::nomkl-3.0-0
  numactl            ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::numactl-2.0.12-573.gdf5dc62
  numba              pkgs/main/linux-ppc64le::numba-0.45.1-py37h962f231_0
  numpy              pkgs/main/linux-ppc64le::numpy-1.16.6-py37h30dfecb_0
  numpy-base         pkgs/main/linux-ppc64le::numpy-base-1.16.6-py37h2f8d375_0
  olefile            pkgs/main/linux-ppc64le::olefile-0.46-py37_0
  onnx               ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::onnx-1.5.0-py37_614.gd049fd7
  openblas           pkgs/main/linux-ppc64le::openblas-0.3.6-1
  openblas-devel     pkgs/main/linux-ppc64le::openblas-devel-0.3.6-1
  opencv             ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::opencv-3.4.7-py37_725.g92aa195
  packaging          pkgs/main/noarch::packaging-20.1-py_0
  pai4sk             ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::pai4sk-1.5.0-py37_1071.g5abf42e
  pandas             pkgs/main/linux-ppc64le::pandas-1.0.0-py37h0573a6f_0
  partd              pkgs/main/noarch::partd-1.1.0-py_0
  pbr                pkgs/main/noarch::pbr-5.4.4-py_0
  pciutils           ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::pciutils-3.6.2-571.g2316d13
  pcre               pkgs/main/linux-ppc64le::pcre-8.43-he6710b0_0
  pillow             pkgs/main/linux-ppc64le::pillow-6.2.1-py37h0d2faf8_0
  pixman             pkgs/main/linux-ppc64le::pixman-0.34.0-h1f8d8dc_3
  pluggy             pkgs/main/linux-ppc64le::pluggy-0.13.1-py37_0
  powerai            ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::powerai-1.6.2-615.g1dade79
  powerai-license    ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::powerai-license-1.6.2-716.g7081e12
  powerai-release    ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::powerai-release-1.6.2-572.gb216c2c
  powerai-tools      ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::powerai-tools-1.6.2-565.g97f2c3f
  protobuf           ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::protobuf-3.8.0-py37_587.gab45ad3
  psutil             pkgs/main/linux-ppc64le::psutil-5.5.0-py37h7b6447c_0
  py                 pkgs/main/noarch::py-1.8.1-py_0
  py-boost           pkgs/main/linux-ppc64le::py-boost-1.67.0-py37h04863e7_4
  py-opencv          ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::py-opencv-3.4.7-py37_725.g92aa195
  py-xgboost-base    ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::py-xgboost-base-0.90-gpu_py37_590.g8a21f75
  py-xgboost-gpu     ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::py-xgboost-gpu-0.90-590.g8a21f75
  pycparser          pkgs/main/linux-ppc64le::pycparser-2.19-py37_0
  pyopenssl          pkgs/main/linux-ppc64le::pyopenssl-19.1.0-py37_0
  pyparsing          pkgs/main/noarch::pyparsing-2.4.6-py_0
  pysocks            pkgs/main/linux-ppc64le::pysocks-1.7.1-py37_0
  pytest             pkgs/main/linux-ppc64le::pytest-4.4.2-py37_0
  python-dateutil    pkgs/main/noarch::python-dateutil-2.8.1-py_0
  python-lmdb        pkgs/main/linux-ppc64le::python-lmdb-0.94-py37h14c3975_0
  pytorch            ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::pytorch-1.2.0-20238.g1faf942
  pytorch-base       ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::pytorch-base-1.2.0-gpu_py37_20238.g1faf942
  pytz               pkgs/main/noarch::pytz-2019.3-py_0
  pywavelets         pkgs/main/linux-ppc64le::pywavelets-1.1.1-py37h7b6447c_0
  pyyaml             pkgs/main/linux-ppc64le::pyyaml-5.1.2-py37h7b6447c_0
  requests           pkgs/main/linux-ppc64le::requests-2.22.0-py37_1
  scikit-image       pkgs/main/linux-ppc64le::scikit-image-0.15.0-py37he6710b0_0
  scikit-learn       pkgs/main/linux-ppc64le::scikit-learn-0.21.3-py37h22eb022_0
  scipy              pkgs/main/linux-ppc64le::scipy-1.3.1-py37he2b7bc3_0
  simsearch          ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::simsearch-1.1.0-py37_764.g7c5f6cf
  six                pkgs/main/linux-ppc64le::six-1.12.0-py37_0
  snapml-spark       ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::snapml-spark-1.4.0-py37_942.gc873569
  snappy             pkgs/main/linux-ppc64le::snappy-1.1.7-h1532aa0_3
  sortedcontainers   pkgs/main/linux-ppc64le::sortedcontainers-2.1.0-py37_0
  spectrum-mpi       ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::spectrum-mpi-10.03-622.gfc88b70
  tabulate           pkgs/main/linux-ppc64le::tabulate-0.8.2-py37_0
  tblib              pkgs/main/noarch::tblib-1.6.0-py_0
  tensorboard        ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::tensorboard-1.15.0-py37_ab7f72a_3645.gf4f525e
  tensorflow         ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::tensorflow-1.15.0-gpu_py37_841.g50de12c
  tensorflow-base    ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::tensorflow-base-1.15.0-gpu_py37_590d6ee_64210.g4a039ec
  tensorflow-estima~ ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::tensorflow-estimator-1.15.1-py37_a5f60ce_1351.g50de12c
  tensorflow-gpu     ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::tensorflow-gpu-1.15.0-841.g50de12c
  tensorflow-large-~ ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::tensorflow-large-model-support-2.0.2-py37_970.gfa57a9e
  tensorflow-probab~ ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::tensorflow-probability-0.8.0-py37_b959b26_2686.g50de12c
  tensorflow-servin~ ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::tensorflow-serving-api-1.15.0-py37_748217e_5094.g89559ef
  tensorrt           ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::tensorrt-6.0.1.5-py37_612.gb2bf6b9
  termcolor          pkgs/main/linux-ppc64le::termcolor-1.1.0-py37_1
  tf_cnn_benchmarks  ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::tf_cnn_benchmarks-1.15-gpu_py37_1374.g5e94b18
  toolz              pkgs/main/noarch::toolz-0.10.0-py_0
  torchtext          ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::torchtext-0.4.0-py37_578.g5bf3960
  torchvision-base   ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::torchvision-base-0.4.0-gpu_py37_593.g80f339d
  tornado            pkgs/main/linux-ppc64le::tornado-6.0.3-py37h7b6447c_0
  tqdm               pkgs/main/noarch::tqdm-4.32.1-py_0
  typing             pkgs/main/linux-ppc64le::typing-3.6.4-py37_0
  uff                ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::uff-0.6.5-py37_612.gb2bf6b9
  urllib3            pkgs/main/linux-ppc64le::urllib3-1.25.8-py37_0
  werkzeug           pkgs/main/noarch::werkzeug-0.15.4-py_0
  wrapt              pkgs/main/linux-ppc64le::wrapt-1.11.2-py37h7b6447c_0
  yaml               pkgs/main/linux-ppc64le::yaml-0.1.7-h1bed415_2
  zict               pkgs/main/noarch::zict-1.0.0-py_0
  zipp               pkgs/main/noarch::zipp-0.6.0-py_0
  zstd               pkgs/main/linux-ppc64le::zstd-1.3.7-h0b5b093_0


Proceed ([y]/n)? y
...

아래와 같이 설치된 package들을 각각 확인하시면 됩니다.

(wmlce_env) cecuser@p1234-kvm1:~$ conda list | grep tensorflow
ddl-tensorflow            1.5.0           py37_1007.g8dbb51d    https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
tensorflow                1.15.0          gpu_py37_841.g50de12c    https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
tensorflow-base           1.15.0          gpu_py37_590d6ee_64210.g4a039ec    https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
tensorflow-estimator      1.15.1          py37_a5f60ce_1351.g50de12c    https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
tensorflow-gpu            1.15.0             841.g50de12c    https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
tensorflow-large-model-support 2.0.2           py37_970.gfa57a9e    https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
tensorflow-probability    0.8.0           py37_b959b26_2686.g50de12c    https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
tensorflow-serving-api    1.15.0          py37_748217e_5094.g89559ef    https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda

(wmlce_env) cecuser@p1234-kvm1:~$ conda list | grep caffe
caffe                     1.0_1.6.2         5184.g7b10df4    https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
caffe-base                1.0_1.6.2       gpu_py37_5184.g7b10df4    https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda

(wmlce_env) cecuser@p1234-kvm1:~$ conda list | grep pytorch
_pytorch_select           2.0             gpu_20238.g1faf942    https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
pytorch                   1.2.0            20238.g1faf942    https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
pytorch-base              1.2.0           gpu_py37_20238.g1faf942    https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda


WML-CE에서는 NCCL와 CUDNN 등 base facility도 함께 제공되어 설치됩니다.

(wmlce_env) cecuser@p1234-kvm1:~$ conda list | grep -i nccl
nccl                      2.4.8              586.gdba67b7    https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda

(wmlce_env) cecuser@p1234-kvm1:~$ conda list | grep -i dnn
cudnn                     7.6.3_10.1         590.g5627c5e    https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda

아래와 같이 python에서 import를 해보셔도 됩니다.

(wmlce_env) cecuser@p1234-kvm1:~$ python
Python 3.7.5 (default, Oct 25 2019, 16:29:01)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import caffe
>>> import tensorflow as tf
2020-02-03 21:31:31.095570: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
>>>


다만 "conda install powerai" 만으로는 RAPIDS까지 설치되지는 않기 때문에, 아래와 같이 별도로 설치하셔야 합니다.

(wmlce_env) cecuser@p1234-kvm1:~$ conda install powerai-rapids
...
The following NEW packages will be INSTALLED:

  arrow-cpp          ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::arrow-cpp-0.15.1-py37_603.g702c836
  boost-cpp          pkgs/main/linux-ppc64le::boost-cpp-1.67.0-h14c3975_4
  brotli             pkgs/main/linux-ppc64le::brotli-1.0.6-he6710b0_0
  cudf               ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::cudf-0.9.0-cuda10.1_py37_626.gddcad2d
  cuml               ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::cuml-0.9.1-cuda10.1_py37_605.gfe9e07b
  cupy               ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::cupy-6.2.0-py37_567.g0f1e2ef
  cython             pkgs/main/linux-ppc64le::cython-0.29.14-py37he6710b0_0
  dask-cudf          ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::dask-cudf-0.9.0-py37_575.g0416adf
  dlpack             ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::dlpack-0.2-562.g28dffd9
  double-conversion  ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::double-conversion-3.1.5-564.g4b43169
  fastavro           ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::fastavro-0.22.4-py37_562.g9525976
  fastrlock          pkgs/main/linux-ppc64le::fastrlock-0.4-py37he6710b0_0
  grpc-cpp           ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::grpc-cpp-1.23.0-568.g4f71a06
  libcudf            ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::libcudf-0.9.0-cuda10.1_609.g113236a
  libcuml            ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::libcuml-0.9.1-cuda10.1_576.ga304a0a
  libevent           ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::libevent-2.1.8-561.ge1d98f7
  libnvstrings       ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::libnvstrings-0.9.0-cuda10.1_570.ga04797c
  librmm             ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::librmm-0.9.0-cuda10.1_567.gff1b1a1
  lz4-c              pkgs/main/linux-ppc64le::lz4-c-1.8.1.2-h14c3975_0
  nvstrings          ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::nvstrings-0.9.0-cuda10.1_py37_580.gdbb6546
  parquet-cpp        ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::parquet-cpp-1.5.1-579.g6eecc60
  powerai-rapids     ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::powerai-rapids-1.6.2-560.ga7c5a47
  pyarrow            ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::pyarrow-0.15.1-py37_609.g3a6717a
  re2                ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::re2-2019.08.01-561.gef92448
  rmm                ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::rmm-0.9.0-cuda10.1_py37_569.g04c75fb
  thrift-cpp         ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::thrift-cpp-0.12.0-580.gf96fa62
  uriparser          ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::uriparser-0.9.3-561.g7465fef

The following packages will be DOWNGRADED:

  pandas                               1.0.0-py37h0573a6f_0 --> 0.24.2-py37he6710b0_0


Proceed ([y]/n)? y
...


만약 GPU가 없는 시스템에서 tensorflow나 pytorch 등을 사용하시고자 할 때는, 아래와 같이 CPU-only 버전의 WML-CE를 설치하시면 됩니다.

(wmlce_env) cecuser@p1234-kvm1:~$ conda install powerai-cpu