2017년 8월 2일 수요일

Minsky 서버에서의 jupyter notebook 설치/사용

Minsky 등 ppc64le 환경에서도 당연히 jupyter notebook 사용이 가능합니다.    Miniconda를 이용하여 간단히 설치가 가능한데, Miniconda 자체의 설치에 대해서는 여기를 클릭하세요.

일단 Miniconda가 설치되어 conda를 쓸 수 있게 되면 아래와 같이 하면 됩니다.

mirhenge@powerai:~/miniconda2$ conda install jupyter
Fetching package metadata .........
Solving package specifications: .

Package plan for installation in environment /home/mirhenge/miniconda2:

The following NEW packages will be INSTALLED:

    backports:          1.0-py27_0
    backports_abc:      0.5-py27_0
    bleach:             1.5.0-py27_0
    configparser:       3.5.0-py27_0
    decorator:          4.0.11-py27_0
    entrypoints:        0.2.2-py27_1
    functools32:        3.2.3.2-py27_0
    get_terminal_size:  1.0.0-py27_0
    html5lib:           0.999-py27_0
    ipykernel:          4.6.1-py27_0
    ipython:            5.3.0-py27_0
    ipython_genutils:   0.2.0-py27_0
    ipywidgets:         6.0.0-py27_0
    jinja2:             2.9.6-py27_0
    jsonschema:         2.6.0-py27_0
    jupyter:            1.0.0-py27_3
    jupyter_client:     5.0.1-py27_0
    jupyter_console:    5.1.0-py27_0
    jupyter_core:       4.3.0-py27_0
    libsodium:          1.0.10-0
    markupsafe:         0.23-py27_2
    mistune:            0.7.4-py27_0
    nbconvert:          5.1.1-py27_0
    nbformat:           4.3.0-py27_0
    notebook:           5.0.0-py27_0
    pandocfilters:      1.4.1-py27_0
    path.py:            10.3.1-py27_0
    pathlib2:           2.2.1-py27_0
    pexpect:            4.2.1-py27_0
    pickleshare:        0.7.4-py27_0
    prompt_toolkit:     1.0.14-py27_0
    ptyprocess:         0.5.1-py27_0
    pygments:           2.2.0-py27_0
    python-dateutil:    2.6.0-py27_0
    pyzmq:              16.0.2-py27_0
    scandir:            1.5-py27_0
    simplegeneric:      0.8.1-py27_1
    singledispatch:     3.4.0.3-py27_0
    ssl_match_hostname: 3.4.0.2-py27_1
    terminado:          0.6-py27_0
    testpath:           0.3-py27_0
    tornado:            4.5.1-py27_0
    traitlets:          4.3.2-py27_0
    wcwidth:            0.1.7-py27_0
    widgetsnbextension: 2.0.0-py27_0
    zeromq:             4.1.5-0

The following packages will be UPDATED:

    conda:              4.3.14-py27_0  --> 4.3.21-py27_0

Proceed ([y]/n)? y
...
entrypoints-0. 100% |#########################################| Time: 0:00:00  15.14 MB/s
jupyter_core-4 100% |#########################################| Time: 0:00:00 304.20 kB/s
pickleshare-0. 100% |#########################################| Time: 0:00:00  23.47 MB/s
testpath-0.3-p 100% |#########################################| Time: 0:00:00  35.33 MB/s
tornado-4.5.1- 100% |#########################################| Time: 0:00:00 613.23 kB/s
ipython-5.3.0- 100% |#########################################| Time: 0:00:01 851.46 kB/s
jupyter_client 100% |#########################################| Time: 0:00:00 329.04 kB/s
nbformat-4.3.0 100% |#########################################| Time: 0:00:00 360.09 kB/s
terminado-0.6- 100% |#########################################| Time: 0:00:00  20.31 MB/s
conda-4.3.21-p 100% |#########################################| Time: 0:00:01 503.31 kB/s
ipykernel-4.6. 100% |#########################################| Time: 0:00:00 249.29 kB/s
nbconvert-5.1. 100% |#########################################| Time: 0:00:00 465.37 kB/s
jupyter_consol 100% |#########################################| Time: 0:00:00  39.17 MB/s
notebook-5.0.0 100% |#########################################| Time: 0:00:02   2.07 MB/s
widgetsnbexten 100% |#########################################| Time: 0:00:01 762.68 kB/s
ipywidgets-6.0 100% |#########################################| Time: 0:00:00 167.32 kB/s
jupyter-1.0.0- 100% |#########################################| Time: 0:00:00  11.56 MB/s


이제 다시 login을 하든가 ~/.bashrc를 새로 수행하면 PATH 환경변수가 잡히면서 jupyter 사용이 가능합니다.

mirhenge@powerai:~/miniconda2$ which jupyter
/home/mirhenge/miniconda2/bin/jupyter

Web browser로는 firefox를 사용하시면 됩니다.   여기서는 X 환경을 위해 tightvncviewer를 이용하므로 DISPLAY를 unix:0으로 설정합니다.

mirhenge@powerai:~/miniconda2$ export BROWSER=/usr/bin/firefox
mirhenge@powerai:~/miniconda2$ export DISPLAY=unix:0

이제 jupyter notebook를 구동하면 다음과 같은 메시지가 나옵니다.

mirhenge@powerai:~/miniconda2$ jupyter notebook
[I 13:40:32.487 NotebookApp] Serving notebooks from local directory: /home/mirhenge/miniconda2
[I 13:40:32.488 NotebookApp] 0 active kernels
[I 13:40:32.488 NotebookApp] The Jupyter Notebook is running at: http://localhost:8888/?token=9d13bb86a1a68d0c408740c877045adb2278535287737f5f
[I 13:40:32.488 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 13:40:32.488 NotebookApp]

    Copy/paste this URL into your browser when you connect for the first time,
    to login with a token:
        http://localhost:8888/?token=9d13bb86a1a68d0c408740c877045adb2278535287737f5f
[I 13:40:35.286 NotebookApp] Accepting one-time-token-authenticated connection from 127.0.0.1

이때 tightvncserver를 설정해 놓았다면 그 화면을 통해 vncviewer에서 browser가 열립니다.




이후로는 vncviewer를 통하지 않고도 remote에서도 web browser를 이용해 접속할 수 있도록 아래와 같이 설정 파일을 생성하고 이어서 수정해줍니다.

mirhenge@powerai:~$ jupyter notebook --generate-config

mirhenge@powerai:~$ vi /home/mirhenge/.jupyter/jupyter_notebook_config.py
...
## The IP address the notebook server will listen on.
#c.NotebookApp.ip = 'localhost'
c.NotebookApp.ip = '*'


이제 다시 start해주면 굳이 local server에서 띄운 browser 뿐만 아니라 외부 PC에서 띄운 browser로도 접속이 가능합니다.

mirhenge@powerai:~$ jupyter notebook
[W 13:57:17.150 NotebookApp] WARNING: The notebook server is listening on all IP addresses and not using encryption. This is not recommended.
[I 13:57:17.158 NotebookApp] Serving notebooks from local directory: /home/mirhenge
[I 13:57:17.158 NotebookApp] 0 active kernels
[I 13:57:17.158 NotebookApp] The Jupyter Notebook is running at: http://[all ip addresses on your system]:8888/?token=acde16ba9020fd7cc174d894f760772af7257c619579519d
[I 13:57:17.158 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 13:57:17.159 NotebookApp]

    Copy/paste this URL into your browser when you connect for the first time,
    to login with a token:
        http://localhost:8888/?token=acde16ba9020fd7cc174d894f760772af7257c619579519d
[I 13:57:18.245 NotebookApp] Accepting one-time-token-authenticated connection from 127.0.0.1
[I 13:57:32.011 NotebookApp] 302 GET / (211.109.178.33) 0.33ms
[I 13:57:32.165 NotebookApp] 302 GET /tree? (211.109.178.33) 0.54ms


이때 처음 접속할 때는 위에 display된 token(위의 붉은색 부분)을 copy & paste 해서 넣어야 합니다.







한번 테스트를 해봅니다.   다음과 같이 IBM lab service에서 만든 demo용 English --> French 번역 deep learning용 package를 다운로드 받습니다.

mirhenge@powerai:~$ git clone https://github.com/ppradhan01/dl-language-translation.git
Cloning into 'dl-language-translation'...
remote: Counting objects: 13, done.
remote: Total 13 (delta 0), reused 0 (delta 0), pack-reused 13
Unpacking objects: 100% (13/13), done.
Checking connectivity... done.


mirhenge@powerai:~$ cd dl-language-translation/
mirhenge@powerai:~/dl-language-translation$ ls
code.ipynb  data  helper.py  problem_unittests.py  README.md

mirhenge@powerai:~/dl-language-translation$ cd data

mirhenge@powerai:~/dl-language-translation/data$ ls
small_vocab_en  small_vocab_fr

mirhenge@powerai:~/dl-language-translation/data$ head small_vocab_en
new jersey is sometimes quiet during autumn , and it is snowy in april .
the united states is usually chilly during july , and it is usually freezing in november .
california is usually quiet during march , and it is usually hot in june .
the united states is sometimes mild during june , and it is cold in september .
your least liked fruit is the grape , but my least liked is the apple .
his favorite fruit is the orange , but my favorite is the grape .
paris is relaxing during december , but it is usually chilly in july .
new jersey is busy during spring , and it is never hot in march .
our least liked fruit is the lemon , but my least liked is the grape .
the united states is sometimes busy during january , and it is sometimes warm in november .

mirhenge@powerai:~/dl-language-translation/data$ head small_vocab_fr
new jersey est parfois calme pendant l' automne , et il est neigeux en avril .
les états-unis est généralement froid en juillet , et il gèle habituellement en novembre .
california est généralement calme en mars , et il est généralement chaud en juin .
les états-unis est parfois légère en juin , et il fait froid en septembre .
votre moins aimé fruit est le raisin , mais mon moins aimé est la pomme .
son fruit préféré est l'orange , mais mon préféré est le raisin .
paris est relaxant en décembre , mais il est généralement froid en juillet .
new jersey est occupé au printemps , et il est jamais chaude en mars .
notre fruit est moins aimé le citron , mais mon moins aimé est le raisin .
les états-unis est parfois occupé en janvier , et il est parfois chaud en novembre .


이제 browser의 notebook에서, 위에서 download 받은 dl-language-translation directory로 찾아내려가 code.ipynb를 클릭하면 됩니다.




그러면 이제 notebook이 열리고, 필요시 위쪽 메뉴 bar에 있는 play 버튼 (player의 삼각형 버튼)을 눌러서 해당 code 부분을 직접 수행해볼 수도 있습니다.







댓글 없음:

댓글 쓰기