Firefly开源社区

[人工智能] RK3399pro 安装rknn_toolkit-1.7.1 报错

123

积分

0

威望

0

贡献

技术小白

积分
123
发表于 2023-5-10 14:09:30     
问题描述及复现步骤:
平台:Ubuntu 20.04 + 2GB内存 +1GB NPU
环境:
  1. # firefly @ firefly in ~ venv [10:07:53]
  2. $ pip list
  3. Package                      Version
  4. ---------------------------- --------
  5. absl-py                      1.4.0
  6. astunparse                   1.6.3
  7. cachetools                   5.3.0
  8. certifi                      2023.5.7
  9. charset-normalizer           3.1.0
  10. flatbuffers                  23.3.3
  11. gast                         0.4.0
  12. google-auth                  2.17.3
  13. google-auth-oauthlib         0.4.6
  14. google-pasta                 0.2.0
  15. graphviz                     0.8.4
  16. grpcio                       1.54.0
  17. h5py                         3.8.0
  18. idna                         3.4
  19. importlib-metadata           6.6.0
  20. keras                        2.11.0
  21. libclang                     16.0.0
  22. Markdown                     3.4.3
  23. MarkupSafe                   2.1.2
  24. mxnet                        1.9.0
  25. numpy                        1.21.6
  26. oauthlib                     3.2.2
  27. <font color="#ff0000"><b>opencv-python                4.5.3.56</b></font>
  28. opt-einsum                   3.3.0
  29. packaging                    23.1
  30. Pillow                       9.5.0
  31. pip                          23.1.2
  32. pkg_resources                0.0.0
  33. protobuf                     3.19.6
  34. psutil                       5.6.2
  35. pyasn1                       0.5.0
  36. pyasn1-modules               0.3.0
  37. requests                     2.30.0
  38. requests-oauthlib            1.3.1
  39. rknn-toolkit-lite            1.7.1
  40. rsa                          4.9
  41. ruamel.yaml                  0.15.81
  42. setuptools                   67.7.2
  43. six                          1.16.0
  44. tensorboard                  2.11.2
  45. tensorboard-data-server      0.6.1
  46. tensorboard-plugin-wit       1.8.1
  47. tensorflow                   2.11.0
  48. tensorflow-cpu-aws           2.11.0
  49. tensorflow-estimator         2.11.0
  50. tensorflow-io-gcs-filesystem 0.29.0
  51. termcolor                    2.3.0
  52. torch                        1.12.0
  53. torchvision                  0.12.0
  54. typing_extensions            4.5.0
  55. urllib3                      2.0.2
  56. Werkzeug                     2.2.3
  57. wheel                        0.40.0
  58. wrapt                        1.15.0
  59. zipp                         3.15.0
复制代码

问题:根据《Rockchip_User_Guide_RKNN_Toolkit_V1.7.1_CN》和 《Rockchip_Quick_Start_RKNN_Toolkit_V1.7.1_CN》操作文档已经安装了RKNN相关依赖,环境是python 3.7 (virtual)
  1. pip3 install --default-timeout=100 rknn_toolkit-1.7.1-cp37-cp37m-linux_aarch64.whl
复制代码
img_v2_4b9ae6ee-5035-4f83-a13d-304ecce8ef5g.jpg



安装过程中就出现以下错误,大致是无法编译通过opencv-python,之前安装opencv-python 4.7版本
img_v2_fec52c24-edb7-41e8-aa98-b78084c556cg.jpg
  1.       c++: fatal error: Killed signal terminated program cc1plus
  2.       compilation terminated.
  3.       make[2]: *** [modules/python3/CMakeFiles/opencv_python3.dir/build.make:76: modules/python3/CMakeFiles/opencv_python3.dir/__/src2/cv2.cpp.o] Error 1
  4.       make[1]: *** [CMakeFiles/Makefile2:1843: modules/python3/CMakeFiles/opencv_python3.dir/all] Error 2
  5.       make: *** [Makefile:166: all] Error 2
  6.       Traceback (most recent call last):
  7.         File "/tmp/pip-build-env-m2qzw8o0/overlay/lib/python3.7/site-packages/skbuild/setuptools_wrap.py", line 674, in setup
  8.           cmkr.make(make_args, install_target=cmake_install_target, env=env)
  9.         File "/tmp/pip-build-env-m2qzw8o0/overlay/lib/python3.7/site-packages/skbuild/cmaker.py", line 696, in make
  10.           self.make_impl(clargs=clargs, config=config, source_dir=source_dir, install_target=install_target, env=env)
  11.         File "/tmp/pip-build-env-m2qzw8o0/overlay/lib/python3.7/site-packages/skbuild/cmaker.py", line 741, in make_impl
  12.           raise SKBuildError(msg)
  13.       
  14.       An error occurred while building with CMake.
  15.         Command:
  16.           /tmp/pip-build-env-m2qzw8o0/overlay/lib/python3.7/site-packages/cmake/data/bin/cmake --build . --target install --config Release --
  17.         Install target:
  18.           install
  19.         Source directory:
  20.           /tmp/pip-install-cyfstk69/opencv-python_017be744da0a466ba9f9c8d52a87d6d4
  21.         Working directory:
  22.           /tmp/pip-install-cyfstk69/opencv-python_017be744da0a466ba9f9c8d52a87d6d4/_skbuild/linux-aarch64-3.7/cmake-build
  23.       Please check the install target is valid and see CMake's output for more information.
  24.       
  25.       [end of output]
  26.   
  27.   note: This error originates from a subprocess, and is likely not a problem with pip.
  28.   ERROR: Failed building wheel for opencv-python
  29. Failed to build opencv-python                                                                                                                                               
  30. ERROR: Could not build wheels for opencv-python, which is required to install pyproject.toml-based projects
复制代码



通过chat-gpt搜索答案后,重新安装opencv-python 4.5.3.56,仍然未解决!!! 而且安装过程,不管是有线网还是无线网,等待非常久。 X9VJzn42Ei.jpg



log_0510.tar.gz

10.59 KB, 下载次数: 0, 下载积分: 灯泡 -1 , 经验 -1

回复

使用道具 举报

899

积分

0

威望

6

贡献

技术大神

Rank: 8Rank: 8

积分
899
发表于 2023-5-10 15:07:07     
https://dev.t-firefly.com/thread-120676-1-1.html
看这篇,如果你装lite版本就好装点,而且我也没试过这么高的OpenCV版本
回复

使用道具 举报

123

积分

0

威望

0

贡献

技术小白

积分
123
发表于 2023-5-10 15:47:36     
本帖最后由 KevinWu 于 2023-5-10 19:02 编辑

大佬,您好!我这边RK3399pro 板子已经装好了rknn_toolkit_lite 1.7.1安装rknn_toolkit_lite 1.7.1,Ubuntu 20.04--x86平台也安装好rktoolkit 1.7.3,能把tf模型转换成RKNN模型,在板子上可以用rknn_toolkit_lite来进行推理;

但是我想测试一个python代码,作者提供的代码瑞芯微 TB-RK3399Pro --YOLOV3开发与优化攻略实现是需要在RK3399pro 板子安装rktoolkit环境瑞芯微 TB-RK3399Pro -- 开发板环境,才能调用接口from rknn.api import RKNN,所以我打算自己在RK3399pro 板子配置rknn_toolkit环境进行测试。

代码我已经上传到云盘yolov3_demo【提取码: 6ga6】可自行下载查看
04f2fc0bde60b6ed14421f76b208fe1.jpg


回复

使用道具 举报

123

积分

0

威望

0

贡献

技术小白

积分
123
发表于 2023-5-10 18:58:24     
895816513 发表于 2023-5-10 15:07
https://dev.t-firefly.com/thread-120676-1-1.html
看这篇,如果你装lite版本就好装点,而且我也没试过这 ...

大佬,您好!麻烦你有空帮看看我最新的回复,谢谢
回复

使用道具 举报

899

积分

0

威望

6

贡献

技术大神

Rank: 8Rank: 8

积分
899
发表于 2023-5-11 10:50:33     
板子上难装rktoolkit,我之前装了好几次没装成功过。我看那CSDN博客也没说要运行什么demo。
回复

使用道具 举报

123

积分

0

威望

0

贡献

技术小白

积分
123
发表于 2023-5-11 11:09:23     
895816513 发表于 2023-5-11 10:50
板子上难装rktoolkit,我之前装了好几次没装成功过。我看那CSDN博客也没说要运行什么demo。

大佬,请教一下,如果不在板子上安装rknn_toolkit,怎样调用API接口from rknn.api import RKNN?
我打算把opencv-python版本降低到4.0.1.23或者4.3.0.36再试试

我看csdn博主运行的代码
  1. python3 rknn_camera_416x416.py
复制代码
确实要安装rknn_toolkit,否则我这边无法调用这个接口;我原本想接入相机进行目标检测 image.jpg


  1. import numpy as np
  2. import cv2
  3. from PIL import Image
  4. from rknn.api import RKNN
  5. from timeit import default_timer as timer

  6. GRID0 = 13
  7. GRID1 = 26import numpy as np
  8. import cv2
  9. from PIL import Image
  10. from rknn.api import RKNN
  11. from timeit import default_timer as timer

  12. GRID0 = 13
  13. GRID1 = 26
  14. GRID2 = 52
  15. LISTSIZE = 85
  16. SPAN = 3
  17. NUM_CLS = 80
  18. MAX_BOXES = 500
  19. OBJ_THRESH = 0.5
  20. NMS_THRESH = 0.6

  21. CLASSES = ("person", "bicycle", "car","motorbike ","aeroplane ","bus ","train","truck ","boat","traffic light",
  22.            "fire hydrant","stop sign ","parking meter","bench","bird","cat","dog ","horse ","sheep","cow","elephant",
  23.            "bear","zebra ","giraffe","backpack","umbrella","handbag","tie","suitcase","frisbee","skis","snowboard","sports ball","kite",
  24.            "baseball bat","baseball glove","skateboard","surfboard","tennis racket","bottle","wine glass","cup","fork","knife ",
  25.            "spoon","bowl","banana","apple","sandwich","orange","broccoli","carrot","hot dog","pizza ","donut","cake","chair","sofa",
  26.            "pottedplant","bed","diningtable","toilet ","tvmonitor","laptop        ","mouse        ","remote ","keyboard ","cell phone","microwave ",
  27.            "oven ","toaster","sink","refrigerator ","book","clock","vase","scissors ","teddy bear ","hair drier", "toothbrush ")

  28. def sigmoid(x):
  29.     return 1 / (1 + np.exp(-x))


  30. def process(input, mask, anchors):

  31.     anchors = [anchors[i] for i in mask]
  32.     grid_h, grid_w = map(int, input.shape[0:2])

  33.     box_confidence = input[..., 4]
  34.     obj_thresh = -np.log(1/OBJ_THRESH - 1)
  35.     pos = np.where(box_confidence > obj_thresh)
  36.     input = input[pos]
  37.     box_confidence = sigmoid(input[..., 4])
  38.     box_confidence = np.expand_dims(box_confidence, axis=-1)

  39.     box_class_probs = sigmoid(input[..., 5:])

  40.     box_xy = sigmoid(input[..., :2])
  41.     box_wh = np.exp(input[..., 2:4])
  42.     for idx, val in enumerate(pos[2]):
  43.         box_wh[idx] = box_wh[idx] * anchors[pos[2][idx]]
  44.     pos0 = np.array(pos[0])[:, np.newaxis]
  45.     pos1 = np.array(pos[1])[:, np.newaxis]
  46.     grid = np.concatenate((pos1, pos0), axis=1)
  47.     box_xy += grid
  48.     box_xy /= (grid_w, grid_h)
  49.     box_wh /= (416, 416)
  50.     box_xy -= (box_wh / 2.)
  51.     box = np.concatenate((box_xy, box_wh), axis=-1)

  52.     return box, box_confidence, box_class_probs

  53. def filter_boxes(boxes, box_confidences, box_class_probs):
  54.     """Filter boxes with object threshold.

  55.     # Arguments
  56.         boxes: ndarray, boxes of objects.
  57.         box_confidences: ndarray, confidences of objects.
  58.         box_class_probs: ndarray, class_probs of objects.

  59.     # Returns
  60.         boxes: ndarray, filtered boxes.
  61.         classes: ndarray, classes for boxes.
  62.         scores: ndarray, scores for boxes.
  63.     """
  64.     box_scores = box_confidences * box_class_probs
  65.     box_classes = np.argmax(box_scores, axis=-1)
  66.     box_class_scores = np.max(box_scores, axis=-1)
  67.     pos = np.where(box_class_scores >= OBJ_THRESH)

  68.     boxes = boxes[pos]
  69.     classes = box_classes[pos]
  70.     scores = box_class_scores[pos]

  71.     return boxes, classes, scores

  72. def nms_boxes(boxes, scores):
  73.     """Suppress non-maximal boxes.

  74.     # Arguments
  75.         boxes: ndarray, boxes of objects.
  76.         scores: ndarray, scores of objects.

  77.     # Returns
  78.         keep: ndarray, index of effective boxes.
  79.     """
  80.     x = boxes[:, 0]
  81.     y = boxes[:, 1]
  82.     w = boxes[:, 2]
  83.     h = boxes[:, 3]

  84.     areas = w * h
  85.     order = scores.argsort()[::-1]

  86.     keep = []
  87.     while order.size > 0:
  88.         i = order[0]
  89.         keep.append(i)

  90.         xx1 = np.maximum(x[i], x[order[1:]])
  91.         yy1 = np.maximum(y[i], y[order[1:]])
  92.         xx2 = np.minimum(x[i] + w[i], x[order[1:]] + w[order[1:]])
  93.         yy2 = np.minimum(y[i] + h[i], y[order[1:]] + h[order[1:]])

  94.         w1 = np.maximum(0.0, xx2 - xx1 + 0.00001)
  95.         h1 = np.maximum(0.0, yy2 - yy1 + 0.00001)
  96.         inter = w1 * h1

  97.         ovr = inter / (areas[i] + areas[order[1:]] - inter)
  98.         inds = np.where(ovr <= NMS_THRESH)[0]
  99.         order = order[inds + 1]
  100.     keep = np.array(keep)
  101.     return keep


  102. def yolov3_post_process(input_data):
  103.     # # yolov3
  104.     # masks = [[6, 7, 8], [3, 4, 5], [0, 1, 2]]
  105.     # anchors = [[10, 13], [16, 30], [33, 23], [30, 61], [62, 45],
  106.     #            [59, 119], [116, 90], [156, 198], [373, 326]]
  107.     # yolov3-tiny
  108.     masks = [[6, 7, 8], [3, 4, 5], [0, 1, 2]]
  109.     anchors = [[10, 13], [16, 30], [33, 23], [30, 61], [62, 45], [59, 119], [116, 90], [156, 198], [373, 326]]

  110.     boxes, classes, scores = [], [], []
  111.     for input,mask in zip(input_data, masks):
  112.         b, c, s = process(input, mask, anchors)
  113.         b, c, s = filter_boxes(b, c, s)
  114.         boxes.append(b)
  115.         classes.append(c)
  116.         scores.append(s)

  117.     boxes = np.concatenate(boxes)
  118.     classes = np.concatenate(classes)
  119.     scores = np.concatenate(scores)

  120.     # # Scale boxes back to original image shape.
  121.     # width, height = 416, 416 #shape[1], shape[0]
  122.     # image_dims = [width, height, width, height]
  123.     # boxes = boxes * image_dims

  124.     nboxes, nclasses, nscores = [], [], []
  125.     for c in set(classes):
  126.         inds = np.where(classes == c)
  127.         b = boxes[inds]
  128.         c = classes[inds]
  129.         s = scores[inds]

  130.         keep = nms_boxes(b, s)

  131.         nboxes.append(b[keep])
  132.         nclasses.append(c[keep])
  133.         nscores.append(s[keep])

  134.     if not nclasses and not nscores:
  135.         return None, None, None

  136.     boxes = np.concatenate(nboxes)
  137.     classes = np.concatenate(nclasses)
  138.     scores = np.concatenate(nscores)

  139.     return boxes, classes, scores

  140. def draw(image, boxes, scores, classes):
  141.     """Draw the boxes on the image.

  142.     # Argument:
  143.         image: original image.
  144.         boxes: ndarray, boxes of objects.
  145.         classes: ndarray, classes of objects.
  146.         scores: ndarray, scores of objects.
  147.         all_classes: all classes name.
  148.     """
  149.     for box, score, cl in zip(boxes, scores, classes):
  150.         x, y, w, h = box
  151.         print('class: {}, score: {}'.format(CLASSES[cl], score))
  152.         print('box coordinate left,top,right,down: [{}, {}, {}, {}]'.format(x, y, x+w, y+h))
  153.         x *= image.shape[1]
  154.         y *= image.shape[0]
  155.         w *= image.shape[1]
  156.         h *= image.shape[0]
  157.         top = max(0, np.floor(x + 0.5).astype(int))
  158.         left = max(0, np.floor(y + 0.5).astype(int))
  159.         right = min(image.shape[1], np.floor(x + w + 0.5).astype(int))
  160.         bottom = min(image.shape[0], np.floor(y + h + 0.5).astype(int))

  161.         # print('class: {}, score: {}'.format(CLASSES[cl], score))
  162.         # print('box coordinate left,top,right,down: [{}, {}, {}, {}]'.format(top, left, right, bottom))

  163.         cv2.rectangle(image, (top, left), (right, bottom), (255, 0, 0), 2)
  164.         cv2.putText(image, '{0} {1:.2f}'.format(CLASSES[cl], score),
  165.                     (top, left - 6),
  166.                     cv2.FONT_HERSHEY_SIMPLEX,
  167.                     0.6, (0, 0, 255), 2)

  168.         # print('class: {0}, score: {1:.2f}'.format(CLASSES[cl], score))
  169.         # print('box coordinate x,y,w,h: {0}'.format(box))

  170. def load_model():
  171.         rknn = RKNN()
  172.         print('-->loading model')
  173.         #rknn.load_rknn('./yolov3_tiny.rknn')
  174.         rknn.load_rknn('./yolov3_416x416.rknn')
  175.         print('loading model done')

  176.         print('--> Init runtime environment')
  177.         ret = rknn.init_runtime()
  178.         if ret != 0:
  179.                 print('Init runtime environment failed')
  180.                 exit(ret)
  181.         print('done')
  182.         return rknn


  183. if __name__ == '__main__':
  184.     rknn = load_model()
  185.     font = cv2.FONT_HERSHEY_SIMPLEX;
  186.     #capture = cv2.VideoCapture("data/3.mp4")
  187.     capture = cv2.VideoCapture(0)
  188.     accum_time = 0
  189.     curr_fps = 0
  190.     prev_time = timer()
  191.     fps = "FPS: ??"
  192.     try:
  193.         while(True):
  194.             ret, frame = capture.read()
  195.             if ret == True:
  196.                 image = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
  197.                 image = cv2.resize(image, (416, 416))

  198.                 testtime=timer()
  199.                 out_boxes, out_boxes2, out_boxes3 = rknn.inference(inputs=[image])
  200.                 testtime2=timer()
  201.                 print("rknn use time {}", testtime2-testtime)

  202.                 out_boxes = out_boxes.reshape(SPAN, LISTSIZE, GRID0, GRID0)
  203.                 out_boxes2 = out_boxes2.reshape(SPAN, LISTSIZE, GRID1, GRID1)
  204.                 out_boxes3 = out_boxes3.reshape(SPAN, LISTSIZE, GRID2, GRID2)
  205.                 input_data = []
  206.                 input_data.append(np.transpose(out_boxes, (2, 3, 0, 1)))
  207.                 input_data.append(np.transpose(out_boxes2, (2, 3, 0, 1)))
  208.                 input_data.append(np.transpose(out_boxes3, (2, 3, 0, 1)))
  209.             
  210.                 testtime=timer()
  211.                 boxes, classes, scores = yolov3_post_process(input_data)
  212.                 testtime2=timer()
  213.                 print("process use time: {}", testtime2-testtime)
  214.             
  215.                 testtime=timer()
  216.                 if boxes is not None:
  217.                     draw(frame, boxes, scores, classes)
  218.                 curr_time = timer()
  219.                 exec_time = curr_time - prev_time
  220.                 prev_time = curr_time
  221.                 accum_time += exec_time
  222.                 curr_fps += 1
  223.                 if accum_time > 1:
  224.                     accum_time -= 1
  225.                     fps = "FPS: " + str(curr_fps)
  226.                     curr_fps = 0
  227.                 cv2.putText(frame, text=fps, org=(3, 15), fontFace=cv2.FONT_HERSHEY_SIMPLEX,
  228.                              fontScale=0.50, color=(255, 0, 0), thickness=2)
  229.                 cv2.imshow("results", frame)
  230.                 c = cv2.waitKey(5) & 0xff
  231.                 if c == 27:
  232.                     cv2.destroyAllWindows()
  233.                     capture.release()
  234.                     rknn.release()
  235.                     break;
  236.                 testtime2=timer()
  237.                 print("show image use time: {}", testtime2-testtime)
  238.     except KeyboardInterrupt:
  239.         cv2.destroyAllWindows()
  240.         capture.release()
  241.         rknn.release()

  242. SPAN = 3
  243. NUM_CLS = 80
  244. MAX_BOXES = 500
  245. OBJ_THRESH = 0.5
  246. NMS_THRESH = 0.6

  247. CLASSES = ("person", "bicycle", "car","motorbike ","aeroplane ","bus ","train","truck ","boat","traffic light",
  248.            "fire hydrant","stop sign ","parking meter","bench","bird","cat","dog ","horse ","sheep","cow","elephant",
  249.            "bear","zebra ","giraffe","backpack","umbrella","handbag","tie","suitcase","frisbee","skis","snowboard","sports ball","kite",
  250.            "baseball bat","baseball glove","skateboard","surfboard","tennis racket","bottle","wine glass","cup","fork","knife ",
  251.            "spoon","bowl","banana","apple","sandwich","orange","broccoli","carrot","hot dog","pizza ","donut","cake","chair","sofa",
  252.            "pottedplant","bed","diningtable","toilet ","tvmonitor","laptop        ","mouse        ","remote ","keyboard ","cell phone","microwave ",
  253.            "oven ","toaster","sink","refrigerator ","book","clock","vase","scissors ","teddy bear ","hair drier", "toothbrush ")

  254. def sigmoid(x):
  255.     return 1 / (1 + np.exp(-x))


  256. def process(input, mask, anchors):

  257.     anchors = [anchors[i] for i in mask]
  258.     grid_h, grid_w = map(int, input.shape[0:2])

  259.     box_confidence = input[..., 4]
  260.     obj_thresh = -np.log(1/OBJ_THRESH - 1)
  261.     pos = np.where(box_confidence > obj_thresh)
  262.     input = input[pos]
  263.     box_confidence = sigmoid(input[..., 4])
  264.     box_confidence = np.expand_dims(box_confidence, axis=-1)

  265.     box_class_probs = sigmoid(input[..., 5:])

  266.     box_xy = sigmoid(input[..., :2])
  267.     box_wh = np.exp(input[..., 2:4])
  268.     for idx, val in enumerate(pos[2]):
  269.         box_wh[idx] = box_wh[idx] * anchors[pos[2][idx]]
  270.     pos0 = np.array(pos[0])[:, np.newaxis]
  271.     pos1 = np.array(pos[1])[:, np.newaxis]
  272.     grid = np.concatenate((pos1, pos0), axis=1)
  273.     box_xy += grid
  274.     box_xy /= (grid_w, grid_h)
  275.     box_wh /= (416, 416)
  276.     box_xy -= (box_wh / 2.)
  277.     box = np.concatenate((box_xy, box_wh), axis=-1)

  278.     return box, box_confidence, box_class_probs

  279. def filter_boxes(boxes, box_confidences, box_class_probs):
  280.     """Filter boxes with object threshold.

  281.     # Arguments
  282.         boxes: ndarray, boxes of objects.
  283.         box_confidences: ndarray, confidences of objects.
  284.         box_class_probs: ndarray, class_probs of objects.

  285.     # Returns
  286.         boxes: ndarray, filtered boxes.
  287.         classes: ndarray, classes for boxes.
  288.         scores: ndarray, scores for boxes.
  289.     """
  290.     box_scores = box_confidences * box_class_probs
  291.     box_classes = np.argmax(box_scores, axis=-1)
  292.     box_class_scores = np.max(box_scores, axis=-1)
  293.     pos = np.where(box_class_scores >= OBJ_THRESH)

  294.     boxes = boxes[pos]
  295.     classes = box_classes[pos]
  296.     scores = box_class_scores[pos]

  297.     return boxes, classes, scores

  298. def nms_boxes(boxes, scores):
  299.     """Suppress non-maximal boxes.

  300.     # Arguments
  301.         boxes: ndarray, boxes of objects.
  302.         scores: ndarray, scores of objects.

  303.     # Returns
  304.         keep: ndarray, index of effective boxes.
  305.     """
  306.     x = boxes[:, 0]
  307.     y = boxes[:, 1]
  308.     w = boxes[:, 2]
  309.     h = boxes[:, 3]

  310.     areas = w * h
  311.     order = scores.argsort()[::-1]

  312.     keep = []
  313.     while order.size > 0:
  314.         i = order[0]
  315.         keep.append(i)

  316.         xx1 = np.maximum(x[i], x[order[1:]])
  317.         yy1 = np.maximum(y[i], y[order[1:]])
  318.         xx2 = np.minimum(x[i] + w[i], x[order[1:]] + w[order[1:]])
  319.         yy2 = np.minimum(y[i] + h[i], y[order[1:]] + h[order[1:]])

  320.         w1 = np.maximum(0.0, xx2 - xx1 + 0.00001)
  321.         h1 = np.maximum(0.0, yy2 - yy1 + 0.00001)
  322.         inter = w1 * h1

  323.         ovr = inter / (areas[i] + areas[order[1:]] - inter)
  324.         inds = np.where(ovr <= NMS_THRESH)[0]
  325.         order = order[inds + 1]
  326.     keep = np.array(keep)
  327.     return keep


  328. def yolov3_post_process(input_data):
  329.     # # yolov3
  330.     # masks = [[6, 7, 8], [3, 4, 5], [0, 1, 2]]
  331.     # anchors = [[10, 13], [16, 30], [33, 23], [30, 61], [62, 45],
  332.     #            [59, 119], [116, 90], [156, 198], [373, 326]]
  333.     # yolov3-tiny
  334.     masks = [[6, 7, 8], [3, 4, 5], [0, 1, 2]]
  335.     anchors = [[10, 13], [16, 30], [33, 23], [30, 61], [62, 45], [59, 119], [116, 90], [156, 198], [373, 326]]

  336.     boxes, classes, scores = [], [], []
  337.     for input,mask in zip(input_data, masks):
  338.         b, c, s = process(input, mask, anchors)
  339.         b, c, s = filter_boxes(b, c, s)
  340.         boxes.append(b)
  341.         classes.append(c)
  342.         scores.append(s)

  343.     boxes = np.concatenate(boxes)
  344.     classes = np.concatenate(classes)
  345.     scores = np.concatenate(scores)

  346.     # # Scale boxes back to original image shape.
  347.     # width, height = 416, 416 #shape[1], shape[0]
  348.     # image_dims = [width, height, width, height]
  349.     # boxes = boxes * image_dims

  350.     nboxes, nclasses, nscores = [], [], []
  351.     for c in set(classes):
  352.         inds = np.where(classes == c)
  353.         b = boxes[inds]
  354.         c = classes[inds]
  355.         s = scores[inds]

  356.         keep = nms_boxes(b, s)

  357.         nboxes.append(b[keep])
  358.         nclasses.append(c[keep])
  359.         nscores.append(s[keep])

  360.     if not nclasses and not nscores:
  361.         return None, None, None

  362.     boxes = np.concatenate(nboxes)
  363.     classes = np.concatenate(nclasses)
  364.     scores = np.concatenate(nscores)

  365.     return boxes, classes, scores

  366. def draw(image, boxes, scores, classes):
  367.     """Draw the boxes on the image.

  368.     # Argument:
  369.         image: original image.
  370.         boxes: ndarray, boxes of objects.
  371.         classes: ndarray, classes of objects.
  372.         scores: ndarray, scores of objects.
  373.         all_classes: all classes name.
  374.     """
  375.     for box, score, cl in zip(boxes, scores, classes):
  376.         x, y, w, h = box
  377.         print('class: {}, score: {}'.format(CLASSES[cl], score))
  378.         print('box coordinate left,top,right,down: [{}, {}, {}, {}]'.format(x, y, x+w, y+h))
  379.         x *= image.shape[1]
  380.         y *= image.shape[0]
  381.         w *= image.shape[1]
  382.         h *= image.shape[0]
  383.         top = max(0, np.floor(x + 0.5).astype(int))
  384.         left = max(0, np.floor(y + 0.5).astype(int))
  385.         right = min(image.shape[1], np.floor(x + w + 0.5).astype(int))
  386.         bottom = min(image.shape[0], np.floor(y + h + 0.5).astype(int))

  387.         # print('class: {}, score: {}'.format(CLASSES[cl], score))
  388.         # print('box coordinate left,top,right,down: [{}, {}, {}, {}]'.format(top, left, right, bottom))

  389.         cv2.rectangle(image, (top, left), (right, bottom), (255, 0, 0), 2)
  390.         cv2.putText(image, '{0} {1:.2f}'.format(CLASSES[cl], score),
  391.                     (top, left - 6),
  392.                     cv2.FONT_HERSHEY_SIMPLEX,
  393.                     0.6, (0, 0, 255), 2)

  394.         # print('class: {0}, score: {1:.2f}'.format(CLASSES[cl], score))
  395.         # print('box coordinate x,y,w,h: {0}'.format(box))

  396. def load_model():
  397.         rknn = RKNN()
  398.         print('-->loading model')
  399.         #rknn.load_rknn('./yolov3_tiny.rknn')
  400.         rknn.load_rknn('./yolov3_416x416.rknn')
  401.         print('loading model done')

  402.         print('--> Init runtime environment')
  403.         ret = rknn.init_runtime()
  404.         if ret != 0:
  405.                 print('Init runtime environment failed')
  406.                 exit(ret)
  407.         print('done')
  408.         return rknn


  409. if __name__ == '__main__':
  410.     rknn = load_model()
  411.     font = cv2.FONT_HERSHEY_SIMPLEX;
  412.     #capture = cv2.VideoCapture("data/3.mp4")
  413.     capture = cv2.VideoCapture(0)
  414.     accum_time = 0
  415.     curr_fps = 0
  416.     prev_time = timer()
  417.     fps = "FPS: ??"
  418.     try:
  419.         while(True):
  420.             ret, frame = capture.read()
  421.             if ret == True:
  422.                 image = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
  423.                 image = cv2.resize(image, (416, 416))

  424.                 testtime=timer()
  425.                 out_boxes, out_boxes2, out_boxes3 = rknn.inference(inputs=[image])
  426.                 testtime2=timer()
  427.                 print("rknn use time {}", testtime2-testtime)

  428.                 out_boxes = out_boxes.reshape(SPAN, LISTSIZE, GRID0, GRID0)
  429.                 out_boxes2 = out_boxes2.reshape(SPAN, LISTSIZE, GRID1, GRID1)
  430.                 out_boxes3 = out_boxes3.reshape(SPAN, LISTSIZE, GRID2, GRID2)
  431.                 input_data = []
  432.                 input_data.append(np.transpose(out_boxes, (2, 3, 0, 1)))
  433.                 input_data.append(np.transpose(out_boxes2, (2, 3, 0, 1)))
  434.                 input_data.append(np.transpose(out_boxes3, (2, 3, 0, 1)))
  435.             
  436.                 testtime=timer()
  437.                 boxes, classes, scores = yolov3_post_process(input_data)
  438.                 testtime2=timer()
  439.                 print("process use time: {}", testtime2-testtime)
  440.             
  441.                 testtime=timer()
  442.                 if boxes is not None:
  443.                     draw(frame, boxes, scores, classes)
  444.                 curr_time = timer()
  445.                 exec_time = curr_time - prev_time
  446.                 prev_time = curr_time
  447.                 accum_time += exec_time
  448.                 curr_fps += 1
  449.                 if accum_time > 1:
  450.                     accum_time -= 1
  451.                     fps = "FPS: " + str(curr_fps)
  452.                     curr_fps = 0
  453.                 cv2.putText(frame, text=fps, org=(3, 15), fontFace=cv2.FONT_HERSHEY_SIMPLEX,
  454.                              fontScale=0.50, color=(255, 0, 0), thickness=2)
  455.                 cv2.imshow("results", frame)
  456.                 c = cv2.waitKey(5) & 0xff
  457.                 if c == 27:
  458.                     cv2.destroyAllWindows()
  459.                     capture.release()
  460.                     rknn.release()
  461.                     break;
  462.                 testtime2=timer()
  463.                 print("show image use time: {}", testtime2-testtime)
  464.     except KeyboardInterrupt:
  465.         cv2.destroyAllWindows()
  466.         capture.release()
  467.         rknn.release()
复制代码

另外,我看快速上手文档和网页操作建议基于X86安装toolkit,但是很奇怪瑞芯微官方,也提供arm板子的toolkit安装包,而且博主居然安装成功!!!
image.jpg


回复

使用道具 举报

899

积分

0

威望

6

贡献

技术大神

Rank: 8Rank: 8

积分
899
发表于 2023-5-11 17:04:48     
KevinWu 发表于 2023-5-11 11:09
大佬,请教一下,如果不在板子上安装rknn_toolkit,怎样调用API接口from rknn.api import RKNN?
我打 ...

以前我们也安装成功过的,虽然过程很麻烦。但是后面随着python库更新就好久没装成功过。所以后面就推荐arm安装lite版本
回复

使用道具 举报

123

积分

0

威望

0

贡献

技术小白

积分
123
发表于 2023-5-11 19:50:19     
回复

使用道具 举报

123

积分

0

威望

0

贡献

技术小白

积分
123
发表于 2023-5-12 15:52:20     
本帖最后由 KevinWu 于 2023-5-12 15:57 编辑
895816513 发表于 2023-5-11 17:04
以前我们也安装成功过的,虽然过程很麻烦。但是后面随着python库更新就好久没装成功过。所以后面就推荐ar ...

大佬,在线急求!刚刚调用toolkit_lite接口,测试一下demo,出现无法初始化这个问题,怎么解决?

报错如下
  1. $ python3 rknn_picture_416x416.py                                                                                                                                             
  2. -->loading model
  3. loading model done
  4. --> Init runtime environment
  5. E Only support ntb mode on Linux_x64 aarch64. But can not find device with ntb mode.
  6. E Catch exception when init runtime!
  7. E Traceback (most recent call last):
  8.   File "/home/firefly/venv/lib/python3.7/site-packages/rknnlite/api/rknn_lite.py", line 145, in init_runtime
  9.     async_mode=async_mode, rknn2precompile=rknn2precompile)
  10.   File "rknnlite/api/rknn_runtime.py", line 201, in rknnlite.api.rknn_runtime.RKNNRuntime.__init__
  11.   File "rknnlite/api/rknn_runtime.py", line 637, in rknnlite.api.rknn_runtime.RKNNRuntime._connect
  12. Exception: Init runtime environment failed!
复制代码




代码如下
  1. import platform
  2. import cv2
  3. import numpy as np
  4. from rknnlite.api import RKNNLite

  5. INPUT_SIZE = 224


  6. def show_top5(result):
  7.     output = result[0].reshape(-1)
  8.     # softmax
  9.     output = np.exp(output)/sum(np.exp(output))
  10.     output_sorted = sorted(output, reverse=True)
  11.     top5_str = 'resnet18\n-----TOP 5-----\n'
  12.     for i in range(5):
  13.         value = output_sorted
  14.         index = np.where(output == value)
  15.         for j in range(len(index)):
  16.             if (i + j) >= 5:
  17.                 break
  18.             if value > 0:
  19.                 topi = '{}: {}\n'.format(index[j], value)
  20.             else:
  21.                 topi = '-1: 0.0\n'
  22.             top5_str += topi
  23.     print(top5_str)


  24. if __name__ == '__main__':
  25.     rknn_lite = RKNNLite()

  26.     # load RKNN model
  27.     print('--> Load RKNN model')
  28.     ret = rknn_lite.load_rknn('./resnet_18.rknn')
  29.     if ret != 0:
  30.         print('Load RKNN model failed')
  31.         exit(ret)
  32.     print('done')

  33.     ori_img = cv2.imread('./space_shuttle_224.jpg')
  34.     img = cv2.cvtColor(ori_img, cv2.COLOR_BGR2RGB)

  35.     # init runtime environment
  36.     print('--> Init runtime environment')
  37.     # run on RK3399Pro/RK1808 with Debian OS, do not need specify target.
  38.     if platform.machine() == 'aarch64':
  39.         target = None
  40.     else:
  41.         target = 'rk1808'
  42.     ret = rknn_lite.init_runtime(target=target)
  43.     if ret != 0:
  44.         print('Init runtime environment failed')
  45.         exit(ret)
  46.     print('done')

  47.     # Inference
  48.     print('--> Running model')
  49.     outputs = rknn_lite.inference(inputs=[img])
  50.     show_top5(outputs)
  51.     print('done')

  52.     rknn_lite.release()
复制代码







bXbXau8zKx.jpg


h0WzuqJJLD.jpg


Ni1WKlMuhh.jpg


之前还是正常的
image.png




回复

使用道具 举报

123

积分

0

威望

0

贡献

技术小白

积分
123
发表于 2023-5-15 12:30:22     
895816513 发表于 2023-5-11 17:04
以前我们也安装成功过的,虽然过程很麻烦。但是后面随着python库更新就好久没装成功过。所以后面就推荐ar ...

大佬,您好!请问可以帮忙看看吗?我昨天重新刷了固件(AIO-RK3399PRO-JD4-UBUNTU-20.04_DESKTOP-GPT-20211230-1511.img),在python 3.7 虚拟环境按照手册教程安装好了 rknn_toolkit_lite-1.7.1-cp37-cp37m-linux_aarch64.whl,但是不知道为什么无法启动ntb 模式?
image.png

我参考网上攻略宿主机链接计算棒上1.7.1版本驱动失败,尝试在~/venv/lib/python3.7/site-packages/rknnlite/3rdparty/platform-tools/ntp/linux-aarch64 替换了最新的npu_transfer_proxy(这个是从github下载的RKNPU For RK3399Pro),并且这个路径下 sudo ./npu_transfer_proxy 启动,
npu_transfer_proxy devices 没有信息输出

image.jpg


环境如下:
Package           Version
----------------- --------------
numpy             1.16.3
opencv-python     4.7.0.72
pip               23.1.2
pkg_resources     0.0.0
psutil            5.6.2
rknn-toolkit-lite 1.7.1
ruamel.yaml       0.15.81
setuptools        67.7.2
wheel             0.40.0


----------------- ------------------------- --------
(venv) firefly@firefly:~/wzf_ws/rknn-toolkit-lite/packages$ ffgo version
OS:         Ubuntu 20.04.3 LTS
MODEL:         RK3399pro-firefly-aiojd4 board
FIREFLY: v2.10-62-g087b2b2
DATE:         20211228-1443
KERNEL:         Linux version 4.4.194 (jincheng@jincheng-PC) (gcc version 6.3.1 20170404 (Linaro GCC 6.3-2017.05) ) #15 SMP Thu Dec 30 14:52:16 CST 2021





image.png
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

友情链接 : 爱板网 电子发烧友论坛 云汉电子社区 粤ICP备14022046号-2
快速回复 返回顶部 返回列表