Skip to content
Advertisement

RPi tcp video streaming with opencv and gstreamer using v4l2h264enc

I am trying to stream frames using OpenCV and Gstreamer in Python. I’m on a 64 bit Bulseye Raspberry Pi 4. This is the pipeline I am using on the Raspberry:

pipeline = 'appsrc ! "video/x-raw,framerate=25/1,format=BGR,width=640,height=480" ! ' 
           'queue ! v4l2h264enc ! "video/x-h264,level=(string)4" ! h264parse ! ' 
           'rtph264pay ! gdppay ! tcpserversink host=0.0.0.0 port=7000 '
cv2.VideoWriter(pipeline, cv2.CAP_GSTREAMER, 0, args.fps, (args.width, args.height))

It seems to be some problem with v4l2h264enc. Enabling GST_DEBUG=4 gives me

0x3e39a00 ERROR           GST_PIPELINE gst/parse/grammar.y:1007:priv_gst_parse_yyparse: no source element for URI "/x-raw,framerate=25/1,format=BGR,width=640,height=480""
0:00:00.087855767 92892      0x3e39a00 ERROR           GST_PIPELINE gst/parse/grammar.y:1007:priv_gst_parse_yyparse: no source element for URI "/x-h264,level=(string)4""

These two errors look most important to me, but you can see the full log here.

Using a similar CLI pipeline the stream connects just fine (except for some encoding grayness, which isn’t the most critical to me right now).

# Stream
gst-launch-1.0 v4l2src device=/dev/video0 ! 
    'video/x-raw,framerate=30/1,format=UYVY,width=1280,height=720' ! 
    v4l2h264enc ! 'video/x-h264,level=(string)4' ! h264parse ! 
    rtph264pay config-interval=1 pt=96 ! gdppay ! tcpserversink host=0.0.0.0 port=7000
# Client
sudo gst-launch-1.0 -v tcpclientsrc host=yraspberry ip> port=7000 ! 
    gdpdepay ! rtph264depay ! avdec_h264 ! videoconvert ! 
    autovideosink sync=false

With appsrc and opencv I also tried writing to a file without success.

The opencv library is compiled with Gstream support. This is what I get from cv2.getBuildInformation():

Video I/O:
    DC1394:                      NO
    FFMPEG:                      YES
      avcodec:                   YES (58.91.100)
      avformat:                  YES (58.45.100)
      avutil:                    YES (56.51.100)
      swscale:                   YES (5.7.100)
      avresample:                NO
    GStreamer:                   YES (1.18.4)
    v4l/v4l2:                    YES (linux/videodev2.h)

Any help would be most welcome!

Advertisement

Answer

As mentioned by @SeB, the BGR frames might not be supported by v4l2h264enc. And leads to this error, which videoconvert fixes:

opencv/opencv/modules/videoio/src/cap_gstreamer.cpp (2293) writeFrame OpenCV | GStreamer warning: Error pushing buffer to GStreamer pipeline

But the main cause for the no source element for URI errors turned out to be the double quotes around video/x-raw and video/x-h264.

This is the final pipeline that works.

pipeline = 'appsrc ! videoconvert ! v4l2h264enc ! video/x-h264,level=(string)4 ! ' 
          'h264parse ! matroskamux ! tcpserversink host=0.0.0.0 port=7000 '

Also, as @SeB suggested, I also included the matroskamux instead of rtph264pay ! gdppay, since it gives better stream performance.

User contributions licensed under: CC BY-SA
7 People found this is helpful
Advertisement