GSoC 2021: Final report for multistream support in the GStreamer element¶
This is Google Summer of Code 2021 project under libcamera aimed at achieving:
- Add support to libcamera’s GStreamer element to support multistreaming.
- Add unit test to test GStreamer element functionality in libcamera.
- Name: Vedant Paranjape
- Mentors: Paul Elder, Nicolas Dufresne
- Organization: libcamera
- Code: https://git.libcamera.org/libcamera/libcamera.git/
- Project Page: https://summerofcode.withgoogle.com/projects/#6332672818806784
- Progress log: https://ve0x10.in/gsoc2021/
- Midway Blog: https://ve0x10.in/blog/2021/07/19/gsoc-libcamera.html
The GStreamer libcamera element allows libcamera to be used in GStreamer pipelines. libcamera supports simultaneous streaming, for example having one lower quality stream for preview, while another higher quality stream is used for recording. This project extends support for the GStreamer libcamera element to support this multistreaming.
Multistream is supported on IPU3, RockChip RK3399, Raspberry Pi 3 and 4 devices camera pipeline. This will only work on those devices.
Install libcamera and GStreamer using this guide: https://libcamera.org/getting-started.html
If the GStreamer element isn’t installed system wide, set the environment variable specifying the location:
export GST_PLUGIN_PATH=$BUILDPATH/build/src/gstreamer # here $BUILDPATH points to the build directory.
Install gst-plugins-bad as follows, it is required for autovideosink
sudo apt install gstreamer1.0-plugins-bad
Run the following command on the terminal
gst-launch-1.0 libcamerasrc name=src src.src ! queue ! videoconvert ! autovideosink src.src_0 ! queue ! videoconvert ! autovideosink
If it runs correctly, you should see such two windows with camera output as follows:
This functionality can be done programmatically, the example codes are here
All the patches submitted by me that merged can be seen here
- gstreamer: Add error checking in gst_libcamera_src_task_enter()
- libcamera: Add OV5647 sensor properties
- gstreamer: Added virtual functions needed to support request pads
- libcamera: Fix the V4L2 pixel format for formats::XBGR8888
- libcamera: Add pixel format BGRX8888
- gstreamer: Store group_id in GstLibcameraSrcState
- gstreamer: Update format specifier in Request Pad template
- test: gstreamer: Add test for gstreamer single stream
Use the following command to verify working of multistream on Raspberry Pi 4B+ using Raspberry Pi Cam v1
gst-launch-1.0 libcamerasrc camera-name="/base/soc/i2c0mux/i2c@1/ov5647@36" name=src src.src ! queue ! videoconvert ! autovideosink src.src_0 ! queue ! videoconvert ! autovideosink`
Use the following command to compile gst code
gcc basic-tutorial-3.c -o basic-tutorial-3 `pkg-config --cflags --libs gstreamer-1.0`
Use the following to view camera using libcamerasrc (replace “Camera 1” with the actual name of the camera)
gst-launch-1.0 libcamerasrc camera-name="Camera 1" ! videoconvert ! autovideosink
Use the following to export GStreamer libcamera element, execute this in libcamera root path
All goals that were originally proposed have been completed. I will continue to support this project in the future, take care of any bugs, issues and will continue implementing additional features whenever required.
It has been a really great experience for me. I have learned so much in these 10 weeks, which will surely help me achieve my future goals in life. I would like to thank libcamera and Google for giving me such an amazing opportunity. Furthermore, I would also like to thank my mentors Paul and Nicolas for helping me out and supporting me throughout the project, also the members of libcamera community, namely Laurent, Kieran, Umang and Jacopo.
On an ending note, if my words or actions hurt anyone, I am extremely sorry !