Firefly开源社区

标题: Gsteramer使用mppvideodec硬解码依旧延迟过大 [打印本页]

作者: oranger    时间: 2024-4-10 09:45
标题: Gsteramer使用mppvideodec硬解码依旧延迟过大
本帖最后由 oranger 于 2024-4-10 09:55 编辑

我希望能够使用RKNN推理无线相机的Rtsp流,格式为H264。由于延迟过大我尝试使用Gsteramer的mppvideodec进行硬解码,针对这部分进行了尝试。
我尝试使用如下程序进行收流:
  1. import gi
  2. gi.require_version('Gst', '1.0')
  3. from gi.repository import Gst, GObject, GLib

  4. # 初始化GStreamer
  5. Gst.init(None)
  6. loop = GLib.MainLoop()

  7. pipeline = Gst.parse_launch("rtspsrc location=rtsp://192.168.0.133:554/test ! rtph264depay ! h264parse ! mppvideodec ! videorate ! video/x-raw,framerate=60/1 ! videoscale ! videoconvert ! video/x-raw,width=1280,height=720 ! autovideosink sync=false")


  8. # 设置管道状态为播放
  9. pipeline.set_state(Gst.State.PLAYING)

  10. # 开始主循环
  11. try:
  12.     loop.run()
  13. except KeyboardInterrupt:
  14.     # 捕获键盘中断事件,停止管道并退出
  15.     pipeline.set_state(Gst.State.NULL)
  16.     loop.quit()
复制代码

此时的延迟是比较小的,约为0.2秒,但是我用下面的管道和程序进行测试则有非常大的延迟,最大时延迟会有一分钟左右:
  1. import heartrate; heartrate.trace(browser=True)

  2. import gi
  3. import cv2
  4. import numpy as np
  5. from gi.repository import Gst
  6. gi.require_version('Gst', '1.0')
  7. # 初始化GStreamer
  8. Gst.init(None)
  9. import time  # 导入time模块

  10. def gst_to_opencv(sample):
  11.     start_time = time.time()  # 获取函数开始的时间
  12.     buf = sample.get_buffer()
  13.     caps = sample.get_caps()
  14.     print("图像数据格式:", caps.to_string())
  15.     array = np.ndarray(
  16.         (caps.get_structure(0).get_value('height'),
  17.          caps.get_structure(0).get_value('width'),
  18.          3),
  19.         buffer=buf.extract_dup(0, buf.get_size()),
  20.         dtype=np.uint8)
  21.     end_time = time.time()  # 获取函数结束的时间
  22.     print("gst_to_opencv耗时: {:.5f}秒".format(end_time - start_time))  # 打印函数耗时
  23.     return array

  24. def new_sample(sink, data):
  25.     sample = sink.emit('pull-sample')
  26.     start_time = time.time()  # 获取OpenCV处理开始的时间
  27.     frame = gst_to_opencv(sample)
  28.     cv2.imshow('Live Video', frame)
  29.     cv2.waitKey(1)
  30.     end_time = time.time()  # 获取显示结束的时间
  31.     print("OpenCV显示耗时: {:.5f}秒".format(end_time - start_time))  # 打印OpenCV处理+显示耗时
  32.     return Gst.FlowReturn.OK

  33. # 设置GStreamer管道
  34. pipeline = Gst.parse_launch('rtspsrc location=rtsp://192.168.0.133:554/test ! rtph264depay ! h264parse ! mppvideodec ! videorate ! video/x-raw,framerate=60/1 ! videoscale ! videoconvert ! video/x-raw,format=BGR,width=1280,height=720 ! appsink emit-signals=True name=mysink')

  35. sink = pipeline.get_by_name('mysink')
  36. sink.set_property('sync', False)
  37. sink.connect('new-sample', new_sample, None)
  38. pipeline.set_state(Gst.State.PLAYING)

  39. try:
  40.     while True:
  41.         pass  # 循环运行,直到用户关闭窗口
  42. except KeyboardInterrupt:
  43.     print("Interrupted by user")

  44. cv2.destroyAllWindows()
  45. pipeline.set_state(Gst.State.NULL)
复制代码

gst_to_opencv以及new_sample打印出的两个耗时均为毫秒级耗时。
我想请问的是两个程序比较,为什么第二个程序的延迟增加的如此严重,如何能够减小延迟?
tchip_askquestions

code.txt

2.39 KB, 下载次数: 0, 下载积分: 灯泡 -1 , 经验 -1


作者: zhangzhenzhao    时间: 2024-4-11 09:04
用RGA不要用opencv或者直接用我们的ffmedia
https://wiki.t-firefly.com/zh_CN ... ubuntu.html#ffmedia


作者: oranger    时间: 2024-5-11 15:57
zhangzhenzhao 发表于 2024-4-11 09:04
用RGA不要用opencv或者直接用我们的ffmedia
https://wiki.t-firefly.com/zh_CN/Firefly-Linux-Guide/manua ...

您好,我按照wiki文档里操作后执行demo_opencv.py报错:
Firefly FFMedia: v2.2.4
input source is a rtsp url
INFO: ff_media: connectToServer: connected to server 192.168.0.131:554
INFO: ff_media: RTPSource: RTPSock current buffer size is(425984)
INFO: 16ModuleRtspClient: init: rtsp para w h: 1280 720, ws hs: 1280 720, format: H264
wait...INFO: ff_media: parseRTSPMessage: RTSP message was fragmented
INFO: 10MppDecoder: getTimeoutSample: 0x2d66dd80 frame info changed 1 error 0 discard 0
ERROR: 10RTSPClient: getOutputDataTimeOut: wait for session to get buffer time out
ERROR: 10RTSPClient: getOutputDataTimeOut: wait for session to get buffer time out
ERROR: 12ModuleMppDec: work: wait for productor ModuleRtspClient timeout
ERROR: 10RTSPClient: getOutputDataTimeOut: wait for session to get buffer time out
ERROR: 10RTSPClient: getOutputDataTimeOut: wait for session to get buffer time out
INFO: 10RTSPClient: teardownMediaSession: teardown session success
INFO: ff_media: connectToServer: connected to server 192.168.0.131:554
INFO: ff_media: RTPSource: RTPSock current buffer size is(425984)
INFO: ff_media: parseRTSPMessage: RTSP message was fragmented
ERROR: 10RTSPClient: getOutputDataTimeOut: wait for session to get buffer time out
ERROR: 10RTSPClient: getOutputDataTimeOut: wait for session to get buffer time out
ERROR: 12ModuleMppDec: work: wait for productor ModuleRtspClient timeout
ERROR: 10RTSPClient: getOutputDataTimeOut: wait for session to get buffer time out
ERROR: 10RTSPClient: getOutputDataTimeOut: wait for session to get buffer time out
INFO: 10RTSPClient: teardownMediaSession: teardown session success
INFO: ff_media: connectToServer: connected to server 192.168.0.131:554
INFO: ff_media: RTPSource: RTPSock current buffer size is(425984)
INFO: ff_media: parseRTSPMessage: RTSP message was fragmented
ERROR: 10RTSPClient: getOutputDataTimeOut: wait for session to get buffer time out
ERROR: 10RTSPClient: getOutputDataTimeOut: wait for session to get buffer time out
ERROR: 12ModuleMppDec: work: wait for productor ModuleRtspClient timeout
ERROR: 10RTSPClient: getOutputDataTimeOut: wait for session to get buffer time out
ERROR: 10RTSPClient: getOutputDataTimeOut: wait for session to get buffer time out
INFO: 10RTSPClient: teardownMediaSession: teardown session success
INFO: ff_media: connectToServer: connected to server 192.168.0.131:554
INFO: ff_media: RTPSource: RTPSock current buffer size is(425984)
INFO: ff_media: parseRTSPMessage: RTSP message was fragmented




欢迎光临 Firefly开源社区 (https://dev.t-firefly.com/) Powered by Discuz! X3.1