libcamera @ELCE 2022¶
The libcamera team is demonstrating the latest open source camera developments at Embedded Linux Conference Europe 2022 in Dublin. Two demonstractions are set up to present three different hardware platforms, all running open source 3A control algorithms.
The first setup demonstrates a video conferencing application between a RaspberryPi and a Surface Go 2 laptop, running a video conferencing call in the browser, demonstrating the recent integration of libcamera in the Pipewire framework, which make it possible to run WebRTC applications, accessing the camera through a secure portal, as required by modern desktop environments. The camera on the RaspberryPi4 and the SurfaceGo2 run the open source 3A algorithms to deliver processed images without impacting the real time requirements of video confering applications.
The second demo setup presents the latest platform supported by the library. An embedded board equipped with an i.MX8MP SoC and a RAW image sensor making full usage of the on-board ISP using only open source, accessible and reusable software components. To demonstrate how 3A algorithms and image quality tuning play a key role in delivering good images, the demo alternates a live video stream running with automatic white balance and exposure correction with a stream of images as captured directly from the image sensor.
Open-source 3A algorithm for NXP i.MX8MP SoC¶
The most recent platform libcamera has gained support for is the NXP i.MX8M Plus SoC. It is the first SoC from that vendor that contains an ISP which until today, unfortunatly had to be bypassed and remain unused when running Linux.
With the forthcoming inclusion of the support for the ISP in mainline Linux and the recent addition of platform support to libcamera for the ISP, all of the image enhancement functions that are required to make a RAW camera sensor usable can now be fully exploited using only open source software components!
The demo shows how the libcamera stack can apply corrections and enhancements to images produced by a RAW image sensor (the Sony IMX219) that make the video stream actually consumable by users.
Only a few algorithms among the many supported by the platform have currently been fully implemented, among which we can list automatic white balancing, automatic exposure and gain control and lens shading correction. More advanced features of the ISP can and will be supported, but we can already observe the difference between processed and non processed video streams.
Open Source Video call¶
A Raspberry Pi 4 with an IMX477 is presented using the new libcamera stack, created by Ideas on Board, and Raspberry Pi. This is used by PipeWire as developed by Red Hat and Collabora to provide a desktop interface to the camera. Pipewire integration in Chromium and libwebrtc has been developed by Pengutronix, and Ideas on Board have integrated all of these components to present the camera through a Video call using the online service Jitsi.
The Raspberry Pi is joined by a Microsoft Surface Go 2 using the Intel IPU3 ISP and an Omnivision OV5693 using the same software stack, through libcamera, pipewire, libwebrtc, chromium and Jitsi to connect the call.