|
|
Firefly-RK3399 上使用 Kinect ：OpenCV 3 + OpenNI2 + libfreenect
发表于 2017-5-20 15:12:29
浏览：13691
|
回复：7
打印
只看该作者
[复制链接]
楼主
本帖最后由 eevee 于 2017-5-21 08:15 编辑
OS：官方固件 Xubuntu 16.04
1) 准备 Preparation
1.1) Install libfreenect
Note:
libfreenect for Kinect Xbox 360
libfreenect2 for Kinect Xbox One - # download latest code, instead of latest release v0.5.5
- $ git clone https://github.com/OpenKinect/libfreenect.git
- $ cd libfreenect/
- # fix issue with OpenCV, in line 173
- $ vi OpenNI2-FreenectDriver/src/DepthStream.hpp
- case XN_STREAM_PROPERTY_ZERO_PLANE_DISTANCE: // unsigned long long or unsigned int (for OpenNI2/OpenCV)
- if (*pDataSize != sizeof(unsigned long long) && *pDataSize != sizeof(unsigned int))
- {
- LogError("Unexpected size for XN_STREAM_PROPERTY_ZERO_PLANE_DISTANCE");
- return ONI_STATUS_ERROR;
- } else {
- if (*pDataSize != sizeof(unsigned long long)) {
- *(static_cast<unsigned long long*>(data)) = ZERO_PLANE_DISTANCE_VAL;
- } else {
- *(static_cast<unsigned int*>(data)) = (unsigned int) ZERO_PLANE_DISTANCE_VAL;
- }
- }
- return ONI_STATUS_OK;
- # build
- $ mkdir build
- $ cd build
- $ cmake -DBUILD_OPENNI2_DRIVER=ON ..
- $ make
- # preview
- $ ./bin/freenect-glview
1.2) Install OpenNI
- # download this, aarch64 platform support
- $ git clone -b develop https://github.com/joinAero/OpenNI2.git
- $ cd OpenNI2/
- # deps
- $ sudo apt-get install libudev-dev
- # or, all
- $ sudo apt-get install \
- g++ \
- python \
- libusb-1.0-0-dev \
- libudev-dev \
- openjdk-6-jdk \
- freeglut3-dev \
- doxygen \
- graphviz
- # build
- $ make
- # setup
- $ echo "export OPENNI2_INCLUDE=`pwd`/Include" >> ~/.bashrc
- $ echo "export OPENNI2_REDIST=`pwd`/Bin/AArch64-Release" >> ~/.bashrc
Then, copy the freenect driver to OpenNI2 driver repository:
- $ cd libfreenect/build/
- $ cd lib/OpenNI2-FreenectDriver/
- $ export FREENECT_DRIVER_LIB=libFreenectDriver.so
- $ export FREENECT_DRIVER_REAL_LIB=$(python - <<'EOF'
- import os
- lib = os.environ['FREENECT_DRIVER_LIB']
- while os.path.islink(lib):
- lib = os.readlink(lib)
- print(lib)
- EOF
- )
- $ ln -sf `pwd`/$FREENECT_DRIVER_REAL_LIB $OPENNI2_REDIST/OpenNI2/Drivers/$FREENECT_DRIVER_LIB
1.3) Build OpenCV with OpenNI
- $ cd opencv/
- $ mkdir build
- $ cd build/
- $ export PY_NAME=$(python -c 'from sys import version_info as v; print("python%d.%d" % v[:2])')
- $ export PY_NUMPY_DIR=$(python -c 'import os.path, numpy.core; print(os.path.dirname(numpy.core.__file__))')
- $ cmake -DCMAKE_BUILD_TYPE=RELEASE \
- -DCMAKE_INSTALL_PREFIX=/usr/local \
- \
- -DPYTHON2_EXECUTABLE=$(which python) \
- -DPYTHON_INCLUDE_DIR=/usr/include/$PY_NAME \
- -DPYTHON_INCLUDE_DIR2=/usr/include/aarch64-linux-gnu/$PY_NAME \
- -DPYTHON_LIBRARY=/usr/lib/aarch64-linux-gnu/lib$PY_NAME.so \
- -DPYTHON2_NUMPY_INCLUDE_DIRS=/usr/lib/$PY_NAME/dist-packages/numpy/core/include/ \
- \
- -DWITH_OPENNI2=ON \
- \
- -DBUILD_DOCS=OFF \
- -DBUILD_EXAMPLES=OFF \
- -DBUILD_TESTS=OFF \
- -DBUILD_PERF_TESTS=OFF \
- \
- -DOPENCV_EXTRA_MODULES_PATH=../../opencv_contrib/modules \
- ..
- $ make -j$(nproc --all)
- $ sudo make install
2) 使用 Usage
`kinect.cc`:
- #include "camera.hpp"
- int main(int argc, char const *argv[]) {
- Camera cam(cv::CAP_OPENNI2);
- if (!cam.IsOpened()) {
- std::cerr << "ERROR: Open camera failed" << std::endl;
- return 1;
- }
- std::cout << "\033[1;32mPress ESC/Q to terminate\033[0m\n\n";
- double min, max;
- cv::Mat adjmap, colormap;
- cam.Capture([&](const cv::Mat& frame, const cv::Mat& depthmap) {
- cv::minMaxIdx(depthmap, &min, &max);
- const float scale = 255 / (max-min);
- depthmap.convertTo(adjmap, CV_8UC1, scale, -min*scale);
- //applyColorMap(adjmap, colormap, cv::COLORMAP_AUTUMN);
- cam.DrawInfo(frame);
- cam.DrawInfo(adjmap);
- cv::imshow("frame", frame);
- cv::imshow("depthmap", adjmap);
- const int key = cv::waitKey(10);
- return !(key == 27 || key == 'q' || key == 'Q'); // ESC/Q
- });
- return 0;
- }
完整的 C++ 代码见这，Python 代码见这。
3) 结束 Finally
步骤 1.1：
* 直接安装 libfreenect 使用时，没有 OpenNI2 驱动，需要自己编译。
* 自己编译 libfreenect 时，请使用最新代码。当前 Release v0.5.5 未支持好 aarch64 。
* 自己编译 libfreenect 时，如果要结合 OpenCV 使用，得做个修正。
步骤 1.2：
* 直接安装 OpenNI2 使用时，运行时没能找到 Kinect ，也可能我没插好。
* 使用官方 OpenNI2 ARM 版本时，报不兼容的错误，与 aarch64 是有冲突的。
* 自己编译 OpenNI2 时， aarch64 平台未支持好， fork 了个版本。
Python 代码：
* 同时获取图像和深度时，图像是有问题的，见此 Issue 。
参考：Use Kinect on Firefly-RK3399
|
|
