IBM POWER 아키텍처 (POWER8/9, 즉 ppc64le)에서 tensorflow나 caffe 등 각종 deep learning framework을 제공해주던 무료 toolkit인 기존 PowerAI는 이미 다들 아시는 바와 같이 이름을 Watson Machine Learning Community Edition (WML-CE)로 변경했습니다. 물론 여전히 무료입니다만, 기존처럼 *.rpm이나 *.deb의 형태로 제공하지 않고 아예 별도의 conda channel을 만들어서 conda에서 설치하도록 하고 있습니다. 따라서, Anaconda가 prerequsite이며, 2020년 2월 초 현재 최신 버전인 1.6.2는 Ananconda 2019.07을 prerequisite으로 하고 있습니다. 아예 모든 것이 설치된 docker image 형태로도 제공됩니다.
자세한 원본 manual은 아래 link를 참조하시면 됩니다.
https://www.ibm.com/support/knowledgecenter/SS5SF7_1.6.2/navigation/wmlce_planning.html
https://www.ibm.com/support/knowledgecenter/SS5SF7_1.6.2/navigation/wmlce_install.html
여기서는 ppc64le Ubuntu 18.04, Python 3.7.5 환경에서 WML-CE 1.6.2를 설치해보겠습니다.
cecuser@p1234-kvm1:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.2 LTS
Release: 18.04
Codename: bionic
먼저 Anaconda 2019.07 버전을 download 받아 설치합니다.
cecuser@p1234-kvm1:~$ wget https://repo.continuum.io/archive/Anaconda3-2019.07-Linux-ppc64le.sh
cecuser@p1234-kvm1:~$ chmod a+x Anaconda3-2019.07-Linux-ppc64le.sh
cecuser@p1234-kvm1:~$ ./Anaconda3-2019.07-Linux-ppc64le.sh
설치가 끝나면 ~/.bashrc를 수행하여 conda init을 수행합니다.
cecuser@p1234-kvm1:~$ . ~/.bashrc
이제 IBM이 제공하는 WML-CE를 위한 conda channel을 conda에 추가합니다.
(base) cecuser@p1234-kvm1:~$ conda config --prepend channels https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/
Conda 가상 환경을 생성하여 거기에 WML-CE를 설치하기를 권장하므로, 먼저 python 3.7.5 환경으로 wmlce_env라는 이름의 virtual env를 만듭니다.
(base) cecuser@p1234-kvm1:~$ conda create --name wmlce_env python=3.7.5
...
The following NEW packages will be INSTALLED:
_libgcc_mutex pkgs/main/linux-ppc64le::_libgcc_mutex-0.1-main
ca-certificates pkgs/main/linux-ppc64le::ca-certificates-2020.1.1-0
certifi pkgs/main/linux-ppc64le::certifi-2019.11.28-py37_0
libedit pkgs/main/linux-ppc64le::libedit-3.1.20181209-hc058e9b_0
libffi pkgs/main/linux-ppc64le::libffi-3.2.1-hf62a594_5
libgcc-ng pkgs/main/linux-ppc64le::libgcc-ng-8.2.0-h822a55f_1
libstdcxx-ng pkgs/main/linux-ppc64le::libstdcxx-ng-8.2.0-h822a55f_1
ncurses pkgs/main/linux-ppc64le::ncurses-6.1-he6710b0_1
openssl pkgs/main/linux-ppc64le::openssl-1.1.1d-h7b6447c_3
pip pkgs/main/linux-ppc64le::pip-20.0.2-py37_1
python pkgs/main/linux-ppc64le::python-3.7.5-h4134adf_0
readline pkgs/main/linux-ppc64le::readline-7.0-h7b6447c_5
setuptools pkgs/main/linux-ppc64le::setuptools-45.1.0-py37_0
sqlite pkgs/main/linux-ppc64le::sqlite-3.30.1-h7b6447c_0
tk pkgs/main/linux-ppc64le::tk-8.6.8-hbc83047_0
wheel pkgs/main/linux-ppc64le::wheel-0.34.1-py37_0
xz pkgs/main/linux-ppc64le::xz-5.2.4-h14c3975_4
zlib pkgs/main/linux-ppc64le::zlib-1.2.11-h7b6447c_3
Proceed ([y]/n)? y
...
Downloading and Extracting Packages
ca-certificates-2020 | 125 KB | ##################################### | 100%
setuptools-45.1.0 | 511 KB | ##################################### | 100%
pip-20.0.2 | 1.7 MB | ##################################### | 100%
sqlite-3.30.1 | 2.3 MB | ##################################### | 100%
wheel-0.34.1 | 50 KB | ##################################### | 100%
python-3.7.5 | 32.5 MB | ##################################### | 100%
openssl-1.1.1d | 3.8 MB | ##################################### | 100%
certifi-2019.11.28 | 156 KB | ##################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
# $ conda activate wmlce_env
#
# To deactivate an active environment, use
#
# $ conda deactivate
이제 conda 가상환경인 wmlce_env를 활성화합니다.
(base) cecuser@p1234-kvm1:~$ conda activate wmlce_env
이제 WML-CE를 설치합니다. Conda package 이름은 여전히 PowerAI로 되어 있는 점에 유의하십시요. 아래와 같이 하면 tensorflow와 caffe2, pytorch 등 WML-CE에서 지원하는 모든 deep learning framework이 한꺼번에 다 설치됩니다. 혹시 WML-CE 전체를 설치하지 않고 가령 PyTorch만 설치하려 할 때는 그냥 conda install pytorch 라고 하시면 됩니다.
아래의 명령어로 어느어느 package들이 설치되는지 보여드리기 위해 긴 ouput을 일부러 다 옮겨 붙였습니다.
(wmlce_env) cecuser@p1234-kvm1:~$ conda install powerai
....
The following NEW packages will be INSTALLED:
_py-xgboost-mutex ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::_py-xgboost-mutex-1.0-gpu_590.g8a21f75
_pytorch_select ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::_pytorch_select-2.0-gpu_20238.g1faf942
_tflow_select ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::_tflow_select-2.1.0-gpu_840.g50de12c
absl-py pkgs/main/linux-ppc64le::absl-py-0.7.1-py37_0
apex ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::apex-0.1.0_1.6.2-py37_596.g1eb5c77
asn1crypto pkgs/main/linux-ppc64le::asn1crypto-1.3.0-py37_0
astor pkgs/main/linux-ppc64le::astor-0.7.1-py37_0
atomicwrites pkgs/main/linux-ppc64le::atomicwrites-1.3.0-py37_1
attrs pkgs/main/noarch::attrs-19.3.0-py_0
blas pkgs/main/linux-ppc64le::blas-1.0-openblas
bokeh pkgs/main/linux-ppc64le::bokeh-1.4.0-py37_0
boost pkgs/main/linux-ppc64le::boost-1.67.0-py37_4
bzip2 pkgs/main/linux-ppc64le::bzip2-1.0.8-h7b6447c_0
c-ares pkgs/main/linux-ppc64le::c-ares-1.15.0-h7b6447c_1001
caffe ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::caffe-1.0_1.6.2-5184.g7b10df4
caffe-base ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::caffe-base-1.0_1.6.2-gpu_py37_5184.g7b10df4
cairo pkgs/main/linux-ppc64le::cairo-1.14.12-h8948797_3
cffi pkgs/main/linux-ppc64le::cffi-1.12.3-py37h2e261b9_0
chardet pkgs/main/linux-ppc64le::chardet-3.0.4-py37_1003
click pkgs/main/linux-ppc64le::click-7.0-py37_0
cloudpickle pkgs/main/noarch::cloudpickle-1.2.2-py_0
coverage pkgs/main/linux-ppc64le::coverage-5.0-py37h7b6447c_0
cryptography pkgs/main/linux-ppc64le::cryptography-2.8-py37h1ba5d50_0
cudatoolkit ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::cudatoolkit-10.1.243-616.gc122b8b
cudnn ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::cudnn-7.6.3_10.1-590.g5627c5e
cycler pkgs/main/linux-ppc64le::cycler-0.10.0-py37_0
cytoolz pkgs/main/linux-ppc64le::cytoolz-0.10.1-py37h7b6447c_0
dask pkgs/main/noarch::dask-2.3.0-py_0
dask-core pkgs/main/noarch::dask-core-2.3.0-py_0
dask-cuda ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::dask-cuda-0.9.1-py37_573.g9af8baa
dask-xgboost ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::dask-xgboost-0.1.7-py37_579.g8a31cf5
ddl ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::ddl-1.5.0-py37_1287.gc90c6f2
ddl-tensorflow ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::ddl-tensorflow-1.5.0-py37_1007.g8dbb51d
decorator pkgs/main/noarch::decorator-4.4.1-py_0
distributed pkgs/main/noarch::distributed-2.3.2-py_1
ffmpeg pkgs/main/linux-ppc64le::ffmpeg-4.0-hcdf2ecd_0
fontconfig pkgs/main/linux-ppc64le::fontconfig-2.13.0-h9420a91_0
freeglut pkgs/main/linux-ppc64le::freeglut-3.0.0-hf484d3e_5
freetype pkgs/main/linux-ppc64le::freetype-2.9.1-h8a8886c_0
fsspec pkgs/main/noarch::fsspec-0.6.2-py_0
future pkgs/main/linux-ppc64le::future-0.17.1-py37_0
gast pkgs/main/linux-ppc64le::gast-0.2.2-py37_0
gflags ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::gflags-2.2.2-1624.g17209b3
glib pkgs/main/linux-ppc64le::glib-2.63.1-h5a9c865_0
glog ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::glog-0.3.5-1613.gd054598
google-pasta ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::google-pasta-0.1.6-py37_564.g04df2d9
graphite2 pkgs/main/linux-ppc64le::graphite2-1.3.13-h23475e2_0
graphsurgeon ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::graphsurgeon-0.4.1-py37_612.gb2bf6b9
grpcio pkgs/main/linux-ppc64le::grpcio-1.16.1-py37hf8bcb03_1
h5py pkgs/main/linux-ppc64le::h5py-2.8.0-py37h8d01980_0
harfbuzz pkgs/main/linux-ppc64le::harfbuzz-1.8.8-hffaf4a1_0
hdf5 pkgs/main/linux-ppc64le::hdf5-1.10.2-hba1933b_1
heapdict pkgs/main/noarch::heapdict-1.0.1-py_0
hypothesis pkgs/main/linux-ppc64le::hypothesis-3.59.1-py37h39e3cac_0
icu pkgs/main/linux-ppc64le::icu-58.2-h64fc554_1
idna pkgs/main/linux-ppc64le::idna-2.8-py37_0
imageio pkgs/main/linux-ppc64le::imageio-2.6.1-py37_0
importlib_metadata pkgs/main/linux-ppc64le::importlib_metadata-1.4.0-py37_0
jasper pkgs/main/linux-ppc64le::jasper-2.0.14-h07fcdf6_1
jinja2 pkgs/main/noarch::jinja2-2.10.3-py_0
joblib pkgs/main/linux-ppc64le::joblib-0.13.2-py37_0
jpeg pkgs/main/linux-ppc64le::jpeg-9b-hcb7ba68_2
keras-applications pkgs/main/noarch::keras-applications-1.0.8-py_0
keras-preprocessi~ pkgs/main/noarch::keras-preprocessing-1.1.0-py_1
kiwisolver pkgs/main/linux-ppc64le::kiwisolver-1.1.0-py37he6710b0_0
leveldb pkgs/main/linux-ppc64le::leveldb-1.20-hf484d3e_1
libboost pkgs/main/linux-ppc64le::libboost-1.67.0-h46d08c1_4
libgfortran-ng pkgs/main/linux-ppc64le::libgfortran-ng-7.3.0-h822a55f_1
libglu pkgs/main/linux-ppc64le::libglu-9.0.0-hf484d3e_1
libopenblas pkgs/main/linux-ppc64le::libopenblas-0.3.6-h5a2b251_1
libopencv ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::libopencv-3.4.7-725.g92aa195
libopus pkgs/main/linux-ppc64le::libopus-1.3-h7b6447c_0
libpng pkgs/main/linux-ppc64le::libpng-1.6.37-hbc83047_0
libprotobuf ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::libprotobuf-3.8.0-577.g45759bb
libtiff pkgs/main/linux-ppc64le::libtiff-4.1.0-h2733197_0
libuuid pkgs/main/linux-ppc64le::libuuid-1.0.3-h1bed415_2
libvpx pkgs/main/linux-ppc64le::libvpx-1.7.0-hf484d3e_0
libxcb pkgs/main/linux-ppc64le::libxcb-1.13-h1bed415_0
libxgboost-base ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::libxgboost-base-0.90-gpu_590.g8a21f75
libxml2 pkgs/main/linux-ppc64le::libxml2-2.9.9-hea5a465_1
llvmlite pkgs/main/linux-ppc64le::llvmlite-0.29.0-py37hd408876_0
lmdb pkgs/main/linux-ppc64le::lmdb-0.9.22-hf484d3e_1
locket pkgs/main/linux-ppc64le::locket-0.2.0-py37_1
markdown pkgs/main/linux-ppc64le::markdown-3.1.1-py37_0
markupsafe pkgs/main/linux-ppc64le::markupsafe-1.1.1-py37h7b6447c_0
matplotlib pkgs/main/linux-ppc64le::matplotlib-3.1.2-py37_1
matplotlib-base pkgs/main/linux-ppc64le::matplotlib-base-3.1.2-py37h4fdacc2_1
mock pkgs/main/linux-ppc64le::mock-2.0.0-py37_0
more-itertools pkgs/main/noarch::more-itertools-8.0.2-py_0
msgpack-python pkgs/main/linux-ppc64le::msgpack-python-0.6.1-py37hfd86e86_1
nccl ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::nccl-2.4.8-586.gdba67b7
networkx pkgs/main/linux-ppc64le::networkx-2.2-py37_1
ninja pkgs/main/linux-ppc64le::ninja-1.9.0-py37hfd86e86_0
nomkl pkgs/main/linux-ppc64le::nomkl-3.0-0
numactl ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::numactl-2.0.12-573.gdf5dc62
numba pkgs/main/linux-ppc64le::numba-0.45.1-py37h962f231_0
numpy pkgs/main/linux-ppc64le::numpy-1.16.6-py37h30dfecb_0
numpy-base pkgs/main/linux-ppc64le::numpy-base-1.16.6-py37h2f8d375_0
olefile pkgs/main/linux-ppc64le::olefile-0.46-py37_0
onnx ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::onnx-1.5.0-py37_614.gd049fd7
openblas pkgs/main/linux-ppc64le::openblas-0.3.6-1
openblas-devel pkgs/main/linux-ppc64le::openblas-devel-0.3.6-1
opencv ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::opencv-3.4.7-py37_725.g92aa195
packaging pkgs/main/noarch::packaging-20.1-py_0
pai4sk ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::pai4sk-1.5.0-py37_1071.g5abf42e
pandas pkgs/main/linux-ppc64le::pandas-1.0.0-py37h0573a6f_0
partd pkgs/main/noarch::partd-1.1.0-py_0
pbr pkgs/main/noarch::pbr-5.4.4-py_0
pciutils ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::pciutils-3.6.2-571.g2316d13
pcre pkgs/main/linux-ppc64le::pcre-8.43-he6710b0_0
pillow pkgs/main/linux-ppc64le::pillow-6.2.1-py37h0d2faf8_0
pixman pkgs/main/linux-ppc64le::pixman-0.34.0-h1f8d8dc_3
pluggy pkgs/main/linux-ppc64le::pluggy-0.13.1-py37_0
powerai ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::powerai-1.6.2-615.g1dade79
powerai-license ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::powerai-license-1.6.2-716.g7081e12
powerai-release ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::powerai-release-1.6.2-572.gb216c2c
powerai-tools ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::powerai-tools-1.6.2-565.g97f2c3f
protobuf ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::protobuf-3.8.0-py37_587.gab45ad3
psutil pkgs/main/linux-ppc64le::psutil-5.5.0-py37h7b6447c_0
py pkgs/main/noarch::py-1.8.1-py_0
py-boost pkgs/main/linux-ppc64le::py-boost-1.67.0-py37h04863e7_4
py-opencv ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::py-opencv-3.4.7-py37_725.g92aa195
py-xgboost-base ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::py-xgboost-base-0.90-gpu_py37_590.g8a21f75
py-xgboost-gpu ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::py-xgboost-gpu-0.90-590.g8a21f75
pycparser pkgs/main/linux-ppc64le::pycparser-2.19-py37_0
pyopenssl pkgs/main/linux-ppc64le::pyopenssl-19.1.0-py37_0
pyparsing pkgs/main/noarch::pyparsing-2.4.6-py_0
pysocks pkgs/main/linux-ppc64le::pysocks-1.7.1-py37_0
pytest pkgs/main/linux-ppc64le::pytest-4.4.2-py37_0
python-dateutil pkgs/main/noarch::python-dateutil-2.8.1-py_0
python-lmdb pkgs/main/linux-ppc64le::python-lmdb-0.94-py37h14c3975_0
pytorch ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::pytorch-1.2.0-20238.g1faf942
pytorch-base ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::pytorch-base-1.2.0-gpu_py37_20238.g1faf942
pytz pkgs/main/noarch::pytz-2019.3-py_0
pywavelets pkgs/main/linux-ppc64le::pywavelets-1.1.1-py37h7b6447c_0
pyyaml pkgs/main/linux-ppc64le::pyyaml-5.1.2-py37h7b6447c_0
requests pkgs/main/linux-ppc64le::requests-2.22.0-py37_1
scikit-image pkgs/main/linux-ppc64le::scikit-image-0.15.0-py37he6710b0_0
scikit-learn pkgs/main/linux-ppc64le::scikit-learn-0.21.3-py37h22eb022_0
scipy pkgs/main/linux-ppc64le::scipy-1.3.1-py37he2b7bc3_0
simsearch ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::simsearch-1.1.0-py37_764.g7c5f6cf
six pkgs/main/linux-ppc64le::six-1.12.0-py37_0
snapml-spark ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::snapml-spark-1.4.0-py37_942.gc873569
snappy pkgs/main/linux-ppc64le::snappy-1.1.7-h1532aa0_3
sortedcontainers pkgs/main/linux-ppc64le::sortedcontainers-2.1.0-py37_0
spectrum-mpi ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::spectrum-mpi-10.03-622.gfc88b70
tabulate pkgs/main/linux-ppc64le::tabulate-0.8.2-py37_0
tblib pkgs/main/noarch::tblib-1.6.0-py_0
tensorboard ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::tensorboard-1.15.0-py37_ab7f72a_3645.gf4f525e
tensorflow ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::tensorflow-1.15.0-gpu_py37_841.g50de12c
tensorflow-base ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::tensorflow-base-1.15.0-gpu_py37_590d6ee_64210.g4a039ec
tensorflow-estima~ ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::tensorflow-estimator-1.15.1-py37_a5f60ce_1351.g50de12c
tensorflow-gpu ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::tensorflow-gpu-1.15.0-841.g50de12c
tensorflow-large-~ ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::tensorflow-large-model-support-2.0.2-py37_970.gfa57a9e
tensorflow-probab~ ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::tensorflow-probability-0.8.0-py37_b959b26_2686.g50de12c
tensorflow-servin~ ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::tensorflow-serving-api-1.15.0-py37_748217e_5094.g89559ef
tensorrt ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::tensorrt-6.0.1.5-py37_612.gb2bf6b9
termcolor pkgs/main/linux-ppc64le::termcolor-1.1.0-py37_1
tf_cnn_benchmarks ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::tf_cnn_benchmarks-1.15-gpu_py37_1374.g5e94b18
toolz pkgs/main/noarch::toolz-0.10.0-py_0
torchtext ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::torchtext-0.4.0-py37_578.g5bf3960
torchvision-base ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::torchvision-base-0.4.0-gpu_py37_593.g80f339d
tornado pkgs/main/linux-ppc64le::tornado-6.0.3-py37h7b6447c_0
tqdm pkgs/main/noarch::tqdm-4.32.1-py_0
typing pkgs/main/linux-ppc64le::typing-3.6.4-py37_0
uff ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::uff-0.6.5-py37_612.gb2bf6b9
urllib3 pkgs/main/linux-ppc64le::urllib3-1.25.8-py37_0
werkzeug pkgs/main/noarch::werkzeug-0.15.4-py_0
wrapt pkgs/main/linux-ppc64le::wrapt-1.11.2-py37h7b6447c_0
yaml pkgs/main/linux-ppc64le::yaml-0.1.7-h1bed415_2
zict pkgs/main/noarch::zict-1.0.0-py_0
zipp pkgs/main/noarch::zipp-0.6.0-py_0
zstd pkgs/main/linux-ppc64le::zstd-1.3.7-h0b5b093_0
Proceed ([y]/n)? y
...
아래와 같이 설치된 package들을 각각 확인하시면 됩니다.
(wmlce_env) cecuser@p1234-kvm1:~$ conda list | grep tensorflow
ddl-tensorflow 1.5.0 py37_1007.g8dbb51d https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
tensorflow 1.15.0 gpu_py37_841.g50de12c https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
tensorflow-base 1.15.0 gpu_py37_590d6ee_64210.g4a039ec https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
tensorflow-estimator 1.15.1 py37_a5f60ce_1351.g50de12c https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
tensorflow-gpu 1.15.0 841.g50de12c https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
tensorflow-large-model-support 2.0.2 py37_970.gfa57a9e https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
tensorflow-probability 0.8.0 py37_b959b26_2686.g50de12c https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
tensorflow-serving-api 1.15.0 py37_748217e_5094.g89559ef https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
(wmlce_env) cecuser@p1234-kvm1:~$ conda list | grep caffe
caffe 1.0_1.6.2 5184.g7b10df4 https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
caffe-base 1.0_1.6.2 gpu_py37_5184.g7b10df4 https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
(wmlce_env) cecuser@p1234-kvm1:~$ conda list | grep pytorch
_pytorch_select 2.0 gpu_20238.g1faf942 https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
pytorch 1.2.0 20238.g1faf942 https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
pytorch-base 1.2.0 gpu_py37_20238.g1faf942 https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
WML-CE에서는 NCCL와 CUDNN 등 base facility도 함께 제공되어 설치됩니다.
(wmlce_env) cecuser@p1234-kvm1:~$ conda list | grep -i nccl
nccl 2.4.8 586.gdba67b7 https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
(wmlce_env) cecuser@p1234-kvm1:~$ conda list | grep -i dnn
cudnn 7.6.3_10.1 590.g5627c5e https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
아래와 같이 python에서 import를 해보셔도 됩니다.
(wmlce_env) cecuser@p1234-kvm1:~$ python
Python 3.7.5 (default, Oct 25 2019, 16:29:01)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import caffe
>>> import tensorflow as tf
2020-02-03 21:31:31.095570: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
>>>
다만 "conda install powerai" 만으로는 RAPIDS까지 설치되지는 않기 때문에, 아래와 같이 별도로 설치하셔야 합니다.
(wmlce_env) cecuser@p1234-kvm1:~$ conda install powerai-rapids
...
The following NEW packages will be INSTALLED:
arrow-cpp ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::arrow-cpp-0.15.1-py37_603.g702c836
boost-cpp pkgs/main/linux-ppc64le::boost-cpp-1.67.0-h14c3975_4
brotli pkgs/main/linux-ppc64le::brotli-1.0.6-he6710b0_0
cudf ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::cudf-0.9.0-cuda10.1_py37_626.gddcad2d
cuml ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::cuml-0.9.1-cuda10.1_py37_605.gfe9e07b
cupy ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::cupy-6.2.0-py37_567.g0f1e2ef
cython pkgs/main/linux-ppc64le::cython-0.29.14-py37he6710b0_0
dask-cudf ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::dask-cudf-0.9.0-py37_575.g0416adf
dlpack ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::dlpack-0.2-562.g28dffd9
double-conversion ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::double-conversion-3.1.5-564.g4b43169
fastavro ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::fastavro-0.22.4-py37_562.g9525976
fastrlock pkgs/main/linux-ppc64le::fastrlock-0.4-py37he6710b0_0
grpc-cpp ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::grpc-cpp-1.23.0-568.g4f71a06
libcudf ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::libcudf-0.9.0-cuda10.1_609.g113236a
libcuml ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::libcuml-0.9.1-cuda10.1_576.ga304a0a
libevent ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::libevent-2.1.8-561.ge1d98f7
libnvstrings ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::libnvstrings-0.9.0-cuda10.1_570.ga04797c
librmm ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::librmm-0.9.0-cuda10.1_567.gff1b1a1
lz4-c pkgs/main/linux-ppc64le::lz4-c-1.8.1.2-h14c3975_0
nvstrings ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::nvstrings-0.9.0-cuda10.1_py37_580.gdbb6546
parquet-cpp ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::parquet-cpp-1.5.1-579.g6eecc60
powerai-rapids ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::powerai-rapids-1.6.2-560.ga7c5a47
pyarrow ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::pyarrow-0.15.1-py37_609.g3a6717a
re2 ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::re2-2019.08.01-561.gef92448
rmm ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::rmm-0.9.0-cuda10.1_py37_569.g04c75fb
thrift-cpp ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::thrift-cpp-0.12.0-580.gf96fa62
uriparser ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le::uriparser-0.9.3-561.g7465fef
The following packages will be DOWNGRADED:
pandas 1.0.0-py37h0573a6f_0 --> 0.24.2-py37he6710b0_0
Proceed ([y]/n)? y
...
만약 GPU가 없는 시스템에서 tensorflow나 pytorch 등을 사용하시고자 할 때는, 아래와 같이 CPU-only 버전의 WML-CE를 설치하시면 됩니다.
(wmlce_env) cecuser@p1234-kvm1:~$ conda install powerai-cpu
