DockerでDeepLearningの環境を作ってみる - 空飛ぶロボットのつくりかた

dockerインストール：Get Docker CE for Ubuntu | Docker Documentation
nvidia-dockerインストール：GitHub - NVIDIA/nvidia-docker: Build and run Docker containers leveraging NVIDIA GPUs
sudo なしで docker を使えるように:Dockerコマンドをsudoなしで実行する方法 - Qiita
Dockerfileの作成(作業環境の初期状態の設計図)
imageの作成(作業環境の初期状態)
containerの作成とログイン（ image を元に新しい作業環境を作る。）

ワークフロー

初回

Dockerfile 作成

nvidia-docker build ... でイメージ作成。失敗したら goto 1

nvidia-docker run ... でコンテナ作成&ログインコンテナ環境を確認してexitしてコンテナ停止

コンテナ内作業＆ホストでの作業

docker start ... でコンテナを起動してログイン

screen|tmux を起動

コンテナ内で何か作業する

ctrl-p + ctrl-q で一旦ホストへ戻る

ホストで何か作業する

docker attach ... でコンテナ内へ戻る。goto 2

作業完了したら exit でコンテナを停止する

作業再開したければ goto 1

Dockerfile

参考：

kerasのDockerfile 趣味でディープラーニングするための GPU 環境を安上がりに作る方法 - Qiita

chainerのDockerfile nvdia-docker でディープラーニング用の環境を作る – GitHub 出張所 – プログラム関係のブログはここに

例1:Dockerfile

nvidia-docker/Dockerfile at master · NVIDIA/nvidia-docker · GitHub

FROM nvidia/cuda:8.0-devel-ubuntu16.04

CMD nvidia-smi -q

例2:Dockerfile

FROM nvidia/cuda:8.0-cudnn6-devel-ubuntu16.04
ENV DEBIAN_FRONTEND "noninteractive"

RUN apt-get update
RUN apt-get -y \
    -o Dpkg::Options::="--force-confdef" \
    -o Dpkg::Options::="--force-confold" dist-upgrade

RUN apt-get install -y --no-install-recommends \
    sudo ssh \
    zsh screen cmake unzip git curl wget vim tree htop \
    python-dev python-pip python-setuptools \
    python3-dev python3-pip python3-setuptools \
    build-essential \
    graphviz \
    libatlas-base-dev libopenblas-base libopenblas-dev liblapack-dev \
    build-essential \
    libopencv-dev \
    python-numpy \
    protobuf-compiler

RUN groupadd -g 1942 ubuntu
RUN useradd -m -u 1942 -g 1942 -d /home/ubuntu ubuntu
RUN echo "ubuntu ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers
RUN chown -R ubuntu:ubuntu /home/ubuntu
RUN chsh -s /usr/bin/zsh ubuntu

USER ubuntu
WORKDIR /home/ubuntu
ENV HOME /home/ubuntu

#### zsh ####
RUN bash -c "$(curl -fsSL https://raw.githubusercontent.com/robbyrussell/oh-my-zsh/master/tools/install.sh)"

#### zsh path setting ####
RUN echo 'export PATH=/usr/local/cuda/bin:$PATH' >> /home/ubuntu/.zshrc
RUN echo 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH' >> /home/ubuntu/.zshrc
RUN echo 'export LD_LIBRARY_PATH=/usr/local/nvidia/lib64:$LD_LIBRARY_PATH' >> /home/ubuntu/.zshrc

RUN echo 'export CPATH=/usr/local/cuda-8.0/targets/x86_64-linux/include:$CPATH' >> /home/ubuntu/.zshrc

RUN echo 'export CPATH=$HOME/cuda/include:$CPATH' >> /home/ubuntu/.zshrc
RUN echo 'export LIBRARY_PATH=$HOME/cuda/lib64:$LIBRARY_PATH' >> /home/ubuntu/.zshrc
RUN echo 'export LD_LIBRARY_PATH=$HOME/cuda/lib64:$LD_LIBRARY_PATH' >> /home/ubuntu/.zshrc

RUN echo 'export LC_ALL=C.UTF-8' >> /home/ubuntu/.zshrc
RUN echo 'export LANG=C.UTF-8' >> /home/ubuntu/.zshrc

RUN echo 'shell "/usr/bin/zsh"' >>  /home/ubuntu/.screenrc


#### pyenv install ####
RUN git clone https://github.com/yyuu/pyenv.git /home/ubuntu/.pyenv
RUN echo 'export PYENV_ROOT="$HOME/.pyenv"' >> /home/ubuntu/.zshrc
RUN echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> /home/ubuntu/.zshrc
RUN echo 'eval "$(pyenv init -)"' >> /home/ubuntu/.zshrc
ENV PYENV_ROOT $HOME/.pyenv
ENV PATH $PYENV_ROOT/shims:$PYENV_ROOT/bin:$PATH
RUN eval "$(pyenv init -)"


#### anaconda ####
RUN pyenv install anaconda3-4.2.0
RUN pyenv rehash
RUN pyenv global anaconda3-4.2.0
RUN conda update --all
RUN pip install --upgrade pip


#### tensorflow ####
RUN pip install cython
RUN pip install tensorflow-gpu
RUN pip install keras


WORKDIR $HOME/

例3:Dockerfile フレームワークたくさん入り深層学習全部入りコンテナ(Keras/TensorFlow/Chainer/Pytorch/Open AI Gym/Anaconda)で、nividia-dockerを使う - Qiita

FROM nvidia/cuda:8.0-cudnn6-runtime

RUN apt-get update
RUN apt-get install -y curl git unzip imagemagick bzip2
RUN git clone git://github.com/yyuu/pyenv.git .pyenv

WORKDIR /
ENV HOME  /
ENV PYENV_ROOT /.pyenv
ENV PATH $PYENV_ROOT/shims:$PYENV_ROOT/bin:$PATH

RUN pyenv install anaconda3-4.4.0
RUN pyenv global anaconda3-4.4.0
RUN pyenv rehash

RUN pip install opencv-python tqdm h5py keras tensorflow-gpu kaggle-cli gym
RUN pip install chainer
RUN pip install http://download.pytorch.org/whl/cu80/torch-0.2.0.post3-cp36-cp36m-manylinux1_x86_64.whl 
RUN pip install torchvision
EOF

imageの作成

Dockerfileの作成
nvidia-docker build -t docker-gpu-test-workspace ./
docker images

chainerのイメージ:https://hub.docker.com/r/chainer/chainer/

pytorchのイメージ：GitHub - pytorch/pytorch: Tensors and Dynamic neural networks in Python with strong GPU acceleration

PyTorchのDockerイメージを作成してみた | SoraLab / ソララボ

https://hub.docker.com/search/?isAutomated=0&isOfficial=0&page=1&pullCount=0&q=pytorch&starCount=0

containerの作成とログイン

例1

nvidia-docker run -it docker-gpu-test-workspace

例2

nvidia-docker run \
  --name my-workspace \
  -v /data/:/data/ \
  --net="host" \
  -ti \
  docker-gpu-test-workspace\
    /usr/bin/zsh

test sample

例1:

(container)# apt install wget
(container)# wget https://raw.githubusercontent.com/fchollet/keras/master/examples/mnist_cnn.py
(container)# python3 mnist_cnn.py

結果

Using TensorFlow backend.
2017-11-12 08:16:25.823619: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2017-11-12 08:16:25.912916: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:892] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2017-11-12 08:16:25.913187: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 0 with properties: 
name: GeForce GTX 950M major: 5 minor: 0 memoryClockRate(GHz): 0.928
pciBusID: 0000:01:00.0
totalMemory: 1.96GiB freeMemory: 1.61GiB
2017-11-12 08:16:25.913216: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX 950M, pci bus id: 0000:01:00.0, compute capability: 5.0)
Downloading data from https://s3.amazonaws.com/img-datasets/mnist.npz
11493376/11490434 [==============================] - 16s 1us/step
x_train shape: (60000, 28, 28, 1)
60000 train samples
10000 test samples
Train on 60000 samples, validate on 10000 samples
Epoch 1/12
2017-11-12 08:16:43.096840: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX 950M, pci bus id: 0000:01:00.0, compute capability: 5.0)
60000/60000 [==============================] - 135s 2ms/step - loss: 0.3366 - acc: 0.8971 - val_loss: 0.0830 - val_acc: 0.9757
Epoch 2/12
60000/60000 [==============================] - 19s 316us/step - loss: 0.1126 - acc: 0.9666 - val_loss: 0.0527 - val_acc: 0.9831
Epoch 3/12
60000/60000 [==============================] - 18s 308us/step - loss: 0.0841 - acc: 0.9747 - val_loss: 0.0414 - val_acc: 0.9860
Epoch 4/12
60000/60000 [==============================] - 19s 310us/step - loss: 0.0695 - acc: 0.9792 - val_loss: 0.0402 - val_acc: 0.9864
Epoch 5/12
60000/60000 [==============================] - 18s 307us/step - loss: 0.0606 - acc: 0.9821 - val_loss: 0.0347 - val_acc: 0.9885
Epoch 6/12
60000/60000 [==============================] - 19s 314us/step - loss: 0.0548 - acc: 0.9842 - val_loss: 0.0319 - val_acc: 0.9897
Epoch 7/12
60000/60000 [==============================] - 19s 309us/step - loss: 0.0513 - acc: 0.9849 - val_loss: 0.0317 - val_acc: 0.9895
Epoch 8/12
60000/60000 [==============================] - 19s 310us/step - loss: 0.0473 - acc: 0.9861 - val_loss: 0.0319 - val_acc: 0.9892
Epoch 9/12
60000/60000 [==============================] - 19s 310us/step - loss: 0.0423 - acc: 0.9872 - val_loss: 0.0290 - val_acc: 0.9907
Epoch 10/12
60000/60000 [==============================] - 18s 307us/step - loss: 0.0411 - acc: 0.9878 - val_loss: 0.0299 - val_acc: 0.9904
Epoch 11/12
60000/60000 [==============================] - 18s 304us/step - loss: 0.0383 - acc: 0.9887 - val_loss: 0.0294 - val_acc: 0.9902
Epoch 12/12
60000/60000 [==============================] - 18s 306us/step - loss: 0.0361 - acc: 0.9887 - val_loss: 0.0354 - val_acc: 0.9890
Test loss: 0.035408690984
Test accuracy: 0.989

うごけばおけ