Firefly-RK3399 上使用 Kinect ：OpenCV 3 + OpenNI2 + libfreenect
本帖最后由 eevee 于 2017-5-21 08:15 编辑OS：官方固件 Xubuntu 16.04
1) 准备 Preparation
1.1) Install libfreenect
Note:
libfreenectfor Kinect Xbox 360
libfreenect2 for Kinect Xbox One
# download latest code, instead of latest release v0.5.5
$ git clone https://github.com/OpenKinect/libfreenect.git
$ cd libfreenect/
# fix issue with OpenCV, in line 173
$ vi OpenNI2-FreenectDriver/src/DepthStream.hpp
case XN_STREAM_PROPERTY_ZERO_PLANE_DISTANCE: // unsigned long long or unsigned int (for OpenNI2/OpenCV)
if (*pDataSize != sizeof(unsigned long long) && *pDataSize != sizeof(unsigned int))
{
LogError("Unexpected size for XN_STREAM_PROPERTY_ZERO_PLANE_DISTANCE");
return ONI_STATUS_ERROR;
} else {
if (*pDataSize != sizeof(unsigned long long)) {
*(static_cast<unsigned long long*>(data)) = ZERO_PLANE_DISTANCE_VAL;
} else {
*(static_cast<unsigned int*>(data)) = (unsigned int) ZERO_PLANE_DISTANCE_VAL;
}
}
return ONI_STATUS_OK;
# build
$ mkdir build
$ cd build
$ cmake -DBUILD_OPENNI2_DRIVER=ON ..
$ make
# preview
$ ./bin/freenect-glview
1.2) Install OpenNI
# download this, aarch64 platform support
$ git clone -b develop https://github.com/joinAero/OpenNI2.git
$ cd OpenNI2/
# deps
$ sudo apt-get install libudev-dev
# or, all
$ sudo apt-get install \
g++ \
python \
libusb-1.0-0-dev \
libudev-dev \
openjdk-6-jdk \
freeglut3-dev \
doxygen \
graphviz
# build
$ make
# setup
$ echo "export OPENNI2_INCLUDE=`pwd`/Include" >> ~/.bashrc
$ echo "export OPENNI2_REDIST=`pwd`/Bin/AArch64-Release" >> ~/.bashrc
Then, copy the freenect driver to OpenNI2 driver repository:
$ cd libfreenect/build/
$ cd lib/OpenNI2-FreenectDriver/
$ export FREENECT_DRIVER_LIB=libFreenectDriver.so
$ export FREENECT_DRIVER_REAL_LIB=$(python - <<'EOF'
import os
lib = os.environ['FREENECT_DRIVER_LIB']
while os.path.islink(lib):
lib = os.readlink(lib)
print(lib)
EOF
)
$ ln -sf `pwd`/$FREENECT_DRIVER_REAL_LIB $OPENNI2_REDIST/OpenNI2/Drivers/$FREENECT_DRIVER_LIB
1.3) Build OpenCV with OpenNI
请参阅这篇：Firefly-RK3399 上编译安装 OpenCV 3
$ cd opencv/
$ mkdir build
$ cd build/
$ export PY_NAME=$(python -c 'from sys import version_info as v; print("python%d.%d" % v[:2])')
$ export PY_NUMPY_DIR=$(python -c 'import os.path, numpy.core; print(os.path.dirname(numpy.core.__file__))')
$ cmake -DCMAKE_BUILD_TYPE=RELEASE \
-DCMAKE_INSTALL_PREFIX=/usr/local \
\
-DPYTHON2_EXECUTABLE=$(which python) \
-DPYTHON_INCLUDE_DIR=/usr/include/$PY_NAME \
-DPYTHON_INCLUDE_DIR2=/usr/include/aarch64-linux-gnu/$PY_NAME \
-DPYTHON_LIBRARY=/usr/lib/aarch64-linux-gnu/lib$PY_NAME.so \
-DPYTHON2_NUMPY_INCLUDE_DIRS=/usr/lib/$PY_NAME/dist-packages/numpy/core/include/ \
\
-DWITH_OPENNI2=ON \
\
-DBUILD_DOCS=OFF \
-DBUILD_EXAMPLES=OFF \
-DBUILD_TESTS=OFF \
-DBUILD_PERF_TESTS=OFF \
\
-DOPENCV_EXTRA_MODULES_PATH=../../opencv_contrib/modules \
..
$ make -j$(nproc --all)
$ sudo make install
2) 使用 Usage
`kinect.cc`:
#include "camera.hpp"
int main(int argc, char const *argv[]) {
Camera cam(cv::CAP_OPENNI2);
if (!cam.IsOpened()) {
std::cerr << "ERROR: Open camera failed" << std::endl;
return 1;
}
std::cout << "\033[1;32mPress ESC/Q to terminate\033[0m\n\n";
double min, max;
cv::Mat adjmap, colormap;
cam.Capture([&](const cv::Mat& frame, const cv::Mat& depthmap) {
cv::minMaxIdx(depthmap, &min, &max);
const float scale = 255 / (max-min);
depthmap.convertTo(adjmap, CV_8UC1, scale, -min*scale);
//applyColorMap(adjmap, colormap, cv::COLORMAP_AUTUMN);
cam.DrawInfo(frame);
cam.DrawInfo(adjmap);
cv::imshow("frame", frame);
cv::imshow("depthmap", adjmap);
const int key = cv::waitKey(10);
return !(key == 27 || key == 'q' || key == 'Q'); // ESC/Q
});
return 0;
}
完整的 C++ 代码见这，Python 代码见这。
3) 结束 Finally
步骤 1.1：
* 直接安装 libfreenect 使用时，没有 OpenNI2 驱动，需要自己编译。
* 自己编译 libfreenect 时，请使用最新代码。当前 Release v0.5.5 未支持好 aarch64 。
* 自己编译 libfreenect 时，如果要结合 OpenCV 使用，得做个修正。
步骤 1.2：
* 直接安装 OpenNI2 使用时，运行时没能找到 Kinect ，也可能我没插好。
* 使用官方 OpenNI2 ARM 版本时，报不兼容的错误，与 aarch64 是有冲突的。
* 自己编译 OpenNI2 时， aarch64 平台未支持好， fork 了个版本。
Python 代码：
* 同时获取图像和深度时，图像是有问题的，见此 Issue 。
参考：Use Kinect on Firefly-RK3399
做得很好，感谢分享 好 最后的CPP和Python代码都是错的，坑了老子一天。 LZ,opencv怎么能够在python3中支持呢？安装的都是在python2中使用 等会儿去试试 感谢分享 我也安装成功了，但是实时画面很卡，你有碰到这样的现象吗？
页:
[1]