How to use nvidia component to accelerate usb camera (v4l2src or nvv4l2camerasrc) on Jetson Nano with Python3 OpenCV

huangapple go评论77阅读模式
英文:

How to use nvidia component to accelerate usb camera (v4l2src or nvv4l2camerasrc) on Jetson Nano with Python3 OpenCV

问题

我的目标是使用 JetsonNano 上的 Python cv2 包加速 USB-CAMERA。尝试使用一些类似 nvv4l2decodernvjpegdec(memory:NVMM) 这样的 gstreaner 插件。

Python 代码:

import cv2

width = 1920
height = 1080

gs_pipeline = f"v4l2src device=/dev/video0 io-mode=2 " \
              f"! image/jpeg, width={width}, height={height}" \
              f"! nvv4l2decoder mjpeg=1 " \
              f"! nvvidconv " \
              f"! video/x-raw(memory:NVMM) format=BGR" \
              f"! videoconvert " \
              f"! video/x-raw, format=BGR " \
              f"! appsink"

v_cap = cv2.VideoCapture(gs_pipeline, cv2.CAP_GSTREAMER)
if not v_cap.isOpened():
    print("failed to open video capture")
    exit(-1)

while v_cap.isOpened():
    ret_val, frame = v_cap.read()
    if not ret_val:
        break

    cv2.imshow('', frame)

    input_key = cv2.waitKey(1)
    if input_key != -1:
        print(f"input key = {input_key}")

    if input_key == ord('q'):
        break

错误:

[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (711) open OpenCV | GStreamer warning: Error opening bin: could not parse caps "video/x-raw(memory:NVMM) format=BGR"
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created

另一个要提到的重点是,某些 gst-launch-1.0 命令可以工作,但这并不意味着它也可以在 Python 中工作。最终目标是使用 python cv2 来控制相机。

仅命令行有效示例:

gst-launch-1.0 v4l2src device=/dev/video0 io-mode=2 ! image/jpeg, width=1920, height=1080, framerate=30/1, format=MJPG ! nvjpegdec ! 'video/x-raw(memory:NVMM),format=I420,width=1920,height=1080,framerate=30/1' ! nvegltransform ! nveglglessink
英文:

My goal is to accelerate the USB-CAMERA on JetsonNano with Python cv2 package. Trying to use some gstreaner plugin like nvv4l2decoder or nvjpegdec or (memory:NVMM).

Python Code:

import cv2

width = 1920
height = 1080

gs_pipeline = f"v4l2src device=/dev/video0 io-mode=2 " \
              f"! image/jpeg, width={width}, height={height}" \
              f"! nvv4l2decoder mjpeg=1 " \
              f"! nvvidconv " \
              f"! video/x-raw(memory:NVMM) format=BGR" \
              f"! videoconvert " \
              f"! video/x-raw, format=BGR " \
              f"! appsink"

v_cap = cv2.VideoCapture(gs_pipeline, cv2.CAP_GSTREAMER)
if not v_cap.isOpened():
    print("failed to open video capture")
    exit(-1)

while v_cap.isOpened():
    ret_val, frame = v_cap.read()
    if not ret_val:
        break

    cv2.imshow('', frame)

    input_key = cv2.waitKey(1)
    if input_key != -1:
        print(f"input key = {input_key}")

    if input_key == ord('q'):
        break

Error:

[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (711) open OpenCV | GStreamer warning: Error opening bin: could not parse caps "video/x-raw(memory:NVMM) format=BGR"
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created

Another point to mention, some of the gst-launch-1.0 command can work, but it does not mean it can work in python as well. The ultimate goal is to use python cv2 to control the camera.

Only Command Line Worked Example:

gst-launch-1.0 v4l2src device=/dev/video0 io-mode=2 ! image/jpeg, width=1920, height=1080, framerate=30/1, format=MJPG ! nvjpegdec ! 'video/x-raw(memory:NVMM),format=I420,width=1920,height=1080,framerate=30/1' ! nvegltransform ! nveglglessink

答案1

得分: 1

nvvidconv 不支持 BGR,只支持 BGRx(因此需要使用 videoconvert 将 BGRx 转换为 BGR)。Caps 也缺少逗号。
最后,videoconvert 只支持系统内存,因此请让 nvvidconv 输出到系统内存而不是 NVMM 内存。
所以需要更改为:

gs_pipeline = f"v4l2src device=/dev/video0 io-mode=2 " \
              f"! image/jpeg, width={width}, height={height}, framerate=30/1, format=MJPG " \
              f"! nvv4l2decoder mjpeg=1 " \
              f"! nvvidconv " \
              f"! video/x-raw, format=BGRx " \
              f"! videoconvert " \
              f"! video/x-raw, format=BGR " \
              f"! appsink drop=1"
英文:

nvvidconv doesn't support BGR, only BGRx (thus the videoconvert for BGRx->BGR). Caps also lacks a comma.
Last, videoconvert only supports system memory, so have nvvidconv to output into system memory rather than NVMM memory.
So change:

gs_pipeline = f"v4l2src device=/dev/video0 io-mode=2 " \
              f"! image/jpeg, width={width}, height={height}, framerate=30/1, format=MJPG " \
              f"! nvv4l2decoder mjpeg=1 " \
              f"! nvvidconv " \
              f"! video/x-raw, format=BGRx " \
              f"! videoconvert " \
              f"! video/x-raw, format=BGR " \
              f"! appsink drop=1"

huangapple
  • 本文由 发表于 2023年2月16日 16:20:06
  • 转载请务必保留本文链接:https://go.coder-hub.com/75469492.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定