Open3Dのお勉強（一回目）

やりたいこと

Depthセンサで取得したデータをOpen3Dで自由自在に操りたい

Open3D – A Modern Library for 3D Data Processing

Open3Dまじでイケてる!

Intelさんありがとうございまぁぁす!!

教科書

Open3D: A Modern Library for 3D Data Processing — Open3D 0.9.0 documentation

インストール

以下、cmakeでpythonのバージョンが揃う指定している

scripts/install-deps-ubuntu.sh
mkdir build
cd build
cmake -DPYTHON_EXECUTABLE:FILEPATH=/usr/bin/python ../src
make -j

fatal error: Python.h: No such file or directory

が出たら、

make test works, but unable to run tests individually · Issue #234 · pybind/pybind11 · GitHub

ライブラリpy3dのテスト

• $Open3D/build/lib/Tutorial/Basic$ python rgbd_redwood.py

で

が出たらおけ

Getting Started

Redwood dataset

大事↓

The Redwood format stored depth in a 16-bit single channel image.

The integer value represents the depth measurement in millimeters. It is the default format for Open3D to parse depth images.

# src/Python/Tutorial/Basic/rgbd_redwood.py

import sys
sys.path.append("../..")

#conda install pillow matplotlib
from py3d import *
import matplotlib.pyplot as plt


if __name__ == "__main__":
    print("Read Redwood dataset")
    color_raw = read_image("../../TestData/RGBD/color/00000.jpg")
    depth_raw = read_image("../../TestData/RGBD/depth/00000.png")
    rgbd_image = create_rgbd_image_from_color_and_depth(
        color_raw, depth_raw);
    print(rgbd_image)
    plt.subplot(1, 2, 1)
    plt.title('Redwood grayscale image')
    plt.imshow(rgbd_image.color)
    plt.subplot(1, 2, 2)
    plt.title('Redwood depth image')
    plt.imshow(rgbd_image.depth)
    plt.show()
    pcd = create_point_cloud_from_rgbd_image(rgbd_image,
            PinholeCameraIntrinsic.prime_sense_default)
    # Flip it, otherwise the pointcloud will be upside down
    pcd.transform([[1, 0, 0, 0], [0, -1, 0, 0], [0, 0, -1, 0], [0, 0, 0, 1]])
    draw_geometries([pcd])

The default conversion function create_rgbd_image_from_color_and_depth creates an RGBDImage from a pair of color and depth image.

The color image is converted into a grayscale image, stored in float ranged in [0, 1].

The depth image is stored in float, representing the depth value in meters. print(rgbd_image) yields:

RGBDImage of size
Color image : 640x480, with 1 channels.
Depth image : 640x480, with 1 channels.
Use numpy.asarray to access buffer data.

Here we use PinholeCameraIntrinsic.prime_sense_default as default camera parameter.

データセット

有名所の

• Redwood dataset
• SUN dataset
• NYU dataset
• TUM dataset

のデータセットの読み込みなど丁寧な説明がある。

本当に素晴らしい。

Basic

python binding

def example_import_function():
    from py3d import read_point_cloud
    pcd = read_point_cloud("../../TestData/ICP/cloud_bin_0.pcd")
    print(pcd)

-> PointCloud with 198835 points.

以下のようにhelp()でクラスや関数の説明が見れる

def example_help_function():
    import py3d as py3d
    help(py3d)
    help(py3d.PointCloud)
    help(py3d.read_point_cloud)

Point Cloud

# src/Python/Tutorial/Basic/pointcloud.py

import sys
import numpy as np
sys.path.append("../..")
from py3d import *

if __name__ == "__main__":

    print("Load a ply point cloud, print it, and render it")
    pcd = read_point_cloud("../../TestData/fragment.ply")
    print(pcd)
    print(np.asarray(pcd.points))
    draw_geometries([pcd])

    print("Downsample the point cloud with a voxel of 0.05")
    downpcd = voxel_down_sample(pcd, voxel_size = 0.05)
    draw_geometries([downpcd])

    print("Recompute the normal of the downsampled point cloud")
    estimate_normals(downpcd, search_param = KDTreeSearchParamHybrid(
            radius = 0.1, max_nn = 30))
    draw_geometries([downpcd])
    print("")

    print("Load a polygon volume and use it to crop the original point cloud")
    vol = read_selection_polygon_volume("../../TestData/Crop/cropped.json")
    chair = vol.crop_point_cloud(pcd)
    draw_geometries([chair])
    print("")

    print("Paint chair")
    chair.paint_uniform_color([1, 0.706, 0])
    draw_geometries([chair])
    print("")

読込

pcdデータなど一行で読み込める

read_point_cloud reads a point cloud from a file. It tries to decode the file based on the extension name.

The supported extension names are: pcd, ply, xyz, xyzrgb, xyzn, pts.

表示

表示ももちろん一行笑

draw_geometries visualizes the point cloud. Use mouse/trackpad to see the geometry from different view point.

操作方法

-- Mouse view control --
  Left button + drag        : Rotate.
  Ctrl + left button + drag : Translate.
  Wheel                     : Zoom in/out.

-- Keyboard view control --
  [/]          : Increase/decrease field of view.
  R            : Reset view point.
  Ctrl/Cmd + C : Copy current view status into the clipboard.
  Ctrl/Cmd + V : Paste view status from clipboard.

-- General control --
  Q, Esc       : Exit window.
  H            : Print help message.
  P, PrtScn    : Take a screen capture.
  D            : Take a depth capture.
  O            : Take a capture of current rendering settings.
:

アニメーションもつくれるみたい

In addition to draw_geometries, Open3D has a set of sibling functions with more advanced functionality. draw_geometries_with_custom_animation allows the programmer to define a custom view trajectory and play an animation in the GUI. draw_geometries_with_animation_callback and draw_geometries_with_key_callback accept Python callback functions as input. The callback function is called in an automatic animation loop, or upon a key press event. See Customized visualization for details.

Customized visualization — Open3D 0.9.0 documentation

ダウンサンプリング

まさかのダウンサンプリングも一行驚

print("Downsample the point cloud with a voxel of 0.05")
downpcd = voxel_down_sample(pcd, voxel_size = 0.05)
draw_geometries([downpcd])

Vertex normal estimation

print("Recompute the normal of the downsampled point cloud")
estimate_normals(downpcd, search_param = KDTreeSearchParamHybrid(
        radius = 0.1, max_nn = 30))
draw_geometries([downpcd])
print("")

画面上で"n", "+", "-"で法線の操作が可能

estimate_normals computes normal for every point. The function finds adjacent points and calculate the principal axis of the adjacent points using covariance analysis.

The function takes an instance of KDTreeSearchParamHybrid class as an argument. The two key arguments radius = 0.1 and max_nn = 30 specifies search radius and maximum nearest neighbor. It has 10cm of search radius, and only considers up to 30 neighbors to save computation time.

KDtree: kd木 - Wikipedia

Crop point cloud

print("We load a polygon volume and use it to crop the original point cloud")
vol = read_selection_polygon_volume("../../TestData/Crop/cropped.json")
chair = vol.crop_point_cloud(pcd)
draw_geometries([chair])
print("")

Paint Point Cloud

print("Paint chair")
chair.paint_uniform_color([1, 0.706, 0])
draw_geometries([chair])
print("")

コツコツ使い方を覚えてPoint Cloudを自由に扱えるようになりたい〜