A Grey world white balance correction algorithm. More...

Inheritance diagram for libcamera::ipa::ipu3::algorithms::Awb:

Collaboration diagram for libcamera::ipa::ipu3::algorithms::Awb:

Public Member Functions
int	configure (IPAContext &context, const IPAConfigInfo &configInfo) override
	Configure the Algorithm given an IPAConfigInfo. More...

void	prepare (IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, ipu3_uapi_params *params) override
	Fill the params buffer with ISP processing parameters for a frame. More...

void	process (IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats, ControlList &metadata) override
	Process ISP statistics, and run algorithm operations. More...

Public Member Functions inherited from libcamera::ipa::Algorithm< _Module >
virtual int	init ([[maybe_unused]] typename Module::Context &context, [[maybe_unused]] const YamlObject &tuningData)
	Initialize the Algorithm with tuning data. More...

virtual int	configure ([[maybe_unused]] typename Module::Context &context, [[maybe_unused]] const typename Module::Config &configInfo)
	Configure the Algorithm given an IPAConfigInfo. More...

virtual void	queueRequest ([[maybe_unused]] typename Module::Context &context, [[maybe_unused]] const uint32_t frame, [[maybe_unused]] typename Module::FrameContext &frameContext, [[maybe_unused]] const ControlList &controls)
	Provide control values to the algorithm. More...

virtual void	prepare ([[maybe_unused]] typename Module::Context &context, [[maybe_unused]] const uint32_t frame, [[maybe_unused]] typename Module::FrameContext &frameContext, [[maybe_unused]] typename Module::Params *params)
	Fill the params buffer with ISP processing parameters for a frame. More...

virtual void	process ([[maybe_unused]] typename Module::Context &context, [[maybe_unused]] const uint32_t frame, [[maybe_unused]] typename Module::FrameContext &frameContext, [[maybe_unused]] const typename Module::Stats *stats, [[maybe_unused]] ControlList &metadata)
	Process ISP statistics, and run algorithm operations. More...

Additional Inherited Members
Public Types inherited from libcamera::ipa::Algorithm< _Module >
using	Module = _Module
	The IPA module type for this class of algorithms.

Detailed Description

A Grey world white balance correction algorithm.

The Grey World algorithm assumes that the scene, in average, is neutral grey. Reference: Lam, Edmund & Fung, George. (2008). Automatic White Balancing in Digital Photography. 10.1201/9781420054538.ch10.

The IPU3 generates statistics from the Bayer Down Scaler output into a grid defined in the ipu3_uapi_awb_config_s structure.

Cells are defined in Pixels

Zones are defined in Cells

                      80 cells
     /───────────── 1280 pixels ───────────\
                      16 zones
      16
    ┌────┬────┬────┬────┬────┬─  ──────┬────┐   \
    │Cell│    │    │    │    │    |    │    │   │
 16 │ px │    │    │    │    │    |    │    │   │
    ├────┼────┼────┼────┼────┼─  ──────┼────┤   │
    │    │    │    │    │    │    |    │    │
    │    │    │    │    │    │    |    │    │   7
    │ ── │ ── │ ── │ ── │ ── │ ──  ── ─┤ ── │ 1 2 4
    │    │    │    │    │    │    |    │    │ 2 0 5

    │    │    │    │    │    │    |    │    │ z p c
    ├────┼────┼────┼────┼────┼─  ──────┼────┤ o i e
    │    │    │    │    │    │    |    │    │ n x l
    │                        │    |    │    │ e e l
    ├───                  ───┼─  ──────┼────┤ s l s
    │                        │    |    │    │   s
    │                        │    |    │    │
    ├───   Zone of Cells  ───┼─  ──────┼────┤   │
    │        (5 x 4)         │    |    │    │   │
    │                        │    |    │    │   │
    ├──                   ───┼─  ──────┼────┤   │
    │                   │    │    |    │    │   │
    │    │    │    │    │    │    |    │    │   │
    └────┴────┴────┴────┴────┴─  ──────┴────┘   /

In each cell, the ImgU computes for each colour component the average of all unsaturated pixels (below a programmable threshold). It also provides the ratio of saturated pixels in the cell.

The AWB algorithm operates on a coarser grid, made by grouping cells from the hardware grid into zones. The number of zones is fixed to kAwbStatsSizeX x kAwbStatsSizeY. For example, a frame of 1280x720 is divided into 80x45 cells of [16x16] pixels and 16x12 zones of [5x4] cells each (kAwbStatsSizeX=16 and kAwbStatsSizeY=12). If the number of cells isn't an exact multiple of the number of zones, the right-most and bottom-most cells are ignored. The grid configuration is computed by IPAIPU3::calculateBdsGrid().

Before calculating the gains, the algorithm aggregates the cell averages for each zone in generateAwbStats(). Cells that have a too high ratio of saturated pixels are ignored, and only zones that contain enough non-saturated cells are then used by the algorithm.

The Grey World algorithm will then estimate the red and blue gains to apply, and store the results in the metadata. The green gain is always set to 1.

Member Function Documentation

◆ configure()

int libcamera::ipa::ipu3::algorithms::Awb::configure	(	IPAContext &	context,
		const IPAConfigInfo &	configInfo
	)

override

Configure the Algorithm given an IPAConfigInfo.

Parameters

[in]	context	The shared IPA context
[in]	configInfo	The IPA configuration data, received from the pipeline handler

Algorithms may implement a configure operation to pre-calculate parameters prior to commencing streaming.

Configuration state may be stored in the IPASessionConfiguration structure of the IPAContext.

Returns: 0 if successful, an error code otherwise

◆ prepare()

void libcamera::ipa::ipu3::algorithms::Awb::prepare	(	IPAContext &	context,
		const uint32_t	frame,
		IPAFrameContext &	frameContext,
		ipu3_uapi_params *	params
	)

override

Fill the params buffer with ISP processing parameters for a frame.

Parameters

[in]	context	The shared IPA context
[in]	frame	The frame context sequence number
[in]	frameContext	The FrameContext for this frame
[out]	params	The ISP specific parameters

This function is called for every frame when the camera is running before it is processed by the ISP to prepare the ISP processing parameters for that frame.

Algorithms shall fill in the parameter structure fields appropriately to configure the ISP processing blocks that they are responsible for. This includes setting fields and flags that enable those processing blocks.

◆ process()

void libcamera::ipa::ipu3::algorithms::Awb::process	(	IPAContext &	context,
		const uint32_t	frame,
		IPAFrameContext &	frameContext,
		const ipu3_uapi_stats_3a *	stats,
		ControlList &	metadata
	)

override

Process ISP statistics, and run algorithm operations.

Parameters

[in]	context	The shared IPA context
[in]	frame	The frame context sequence number
[in]	frameContext	The current frame's context
[in]	stats	The IPA statistics and ISP results
[out]	metadata	Metadata for the frame, to be filled by the algorithm

This function is called while camera is running for every frame processed by the ISP, to process statistics generated from that frame by the ISP. Algorithms shall use this data to run calculations, update their state accordingly, and fill the frame metadata.

Processing shall not take an undue amount of time, and any extended or computationally expensive calculations or operations must be handled asynchronously in a separate thread.

Algorithms can store state in their respective IPAFrameContext structures, and reference state from the IPAFrameContext of other algorithms.

Todo:: Historical data may be required as part of the processing. Either the previous frame, or the IPAFrameContext state of the frame that generated the statistics for this operation may be required for some advanced algorithms to prevent oscillations or support control loops correctly. Only a single IPAFrameContext is available currently, and so any data stored may represent the results of the previously completed operations.

Care shall be taken to ensure the ordering of access to the information such that the algorithms use up to date state as required.

The documentation for this class was generated from the following files:

src/ipa/ipu3/algorithms/awb.h
src/ipa/ipu3/algorithms/awb.cpp

Public Member Functions

Additional Inherited Members

Detailed Description

Member Function Documentation

◆ configure()

◆ prepare()

◆ process()