Milan ZamazalandKieran Bingham
e2b4000dc9
This patch applies color correction matrix (CCM) in debayering if the CCM is specified. Not using CCM must still be supported for performance reasons. The CCM is applied as follows: [r1 g1 b1] [r] [r2 g2 b2] * [g] [r3 g3 b3] [b] The CCM matrix (the left side of the multiplication) is constant during single frame processing, while the input pixel (the right side) changes. Because each of the color channels is only 8-bit in software ISP, we can make 9 lookup tables with 256 input values for multiplications of each of the r_i, g_i, b_i values. This way we don't have to multiply each pixel, we can use table lookups and additions instead. Gamma (which is non-linear and thus cannot be a part of the 9 lookup tables values) is applied on the final values rounded to integers using another lookup table. Because the changing part is the pixel value with three color elements, only three dynamic table lookups are needed. We use three lookup tables to represent the multiplied matrix values, each of the tables corresponding to the given matrix column and pixel color. We use int16_t to store the precomputed multiplications. This seems to be noticeably (>10%) faster than `float' for the price of slightly less accuracy and it covers the range of values that sane CCMs produce. The selection and structure of data is performance critical, for example using bytes would add significant (>10%) speedup but would be too short to cover the value range. The color lookup tables can be represented either as unions, accommodating tables for both the CCM and non-CCM cases, or as separate tables for each of the cases, leaving the tables for the other case unused. The latter is selected as a matter of preference. The tables are copied (as before), which is not elegant but also not a big problem. There are patches posted that use shared buffers for parameters passing in software ISP (see software ISP TODO #5) and they can be adjusted for the new parameter format. Color gains from white balance are supposed not to be a part of the specified CCM. They are applied on it using matrix multiplication, which is simple and in correspondence with future additions in the form of matrix multiplication, like saturation adjustment. With this patch, the reported per-frame slowdown when applying CCM is about 45% on Debix Model A and about 75% on TI AM69 SK. Using std::clamp in debayering adds some performance penalty (a few percent). The clamping is necessary to eliminate out of range values possibly produced by the CCM. If it could be avoided by adjusting the precomputed tables some way then performance could be improved a bit. Signed-off-by: Milan Zamazal <mzamazal@redhat.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
180 lines
4.6 KiB
C++
180 lines
4.6 KiB
C++
/* SPDX-License-Identifier: LGPL-2.1-or-later */
|
|
/*
|
|
* Copyright (C) 2023, Linaro Ltd
|
|
* Copyright (C) 2023-2025 Red Hat Inc.
|
|
*
|
|
* Authors:
|
|
* Hans de Goede <hdegoede@redhat.com>
|
|
*
|
|
* debayer base class
|
|
*/
|
|
|
|
#include "debayer.h"
|
|
|
|
namespace libcamera {
|
|
|
|
/**
|
|
* \struct DebayerParams
|
|
* \brief Struct to hold the debayer parameters.
|
|
*/
|
|
|
|
/**
|
|
* \var DebayerParams::kRGBLookupSize
|
|
* \brief Size of a color lookup table
|
|
*/
|
|
|
|
/**
|
|
* \struct DebayerParams::CcmColumn
|
|
* \brief Type of a single column of a color correction matrix (CCM)
|
|
*
|
|
* When multiplying an input pixel, columns in the CCM correspond to the red,
|
|
* green or blue component of input pixel values, while rows correspond to the
|
|
* red, green or blue components of the output pixel values. The members of the
|
|
* CcmColumn structure are named after the colour components of the output pixel
|
|
* values they correspond to.
|
|
*/
|
|
|
|
/**
|
|
* \var DebayerParams::CcmColumn::r
|
|
* \brief Red (first) component of a CCM column
|
|
*/
|
|
|
|
/**
|
|
* \var DebayerParams::CcmColumn::g
|
|
* \brief Green (second) component of a CCM column
|
|
*/
|
|
|
|
/**
|
|
* \var DebayerParams::CcmColumn::b
|
|
* \brief Blue (third) component of a CCM column
|
|
*/
|
|
|
|
/**
|
|
* \typedef DebayerParams::LookupTable
|
|
* \brief Type of the lookup tables for single lookup values
|
|
*/
|
|
|
|
/**
|
|
* \typedef DebayerParams::CcmLookupTable
|
|
* \brief Type of the CCM lookup tables for red, green, blue values
|
|
*/
|
|
|
|
/**
|
|
* \var DebayerParams::red
|
|
* \brief Lookup table for red color, mapping input values to output values
|
|
*/
|
|
|
|
/**
|
|
* \var DebayerParams::green
|
|
* \brief Lookup table for green color, mapping input values to output values
|
|
*/
|
|
|
|
/**
|
|
* \var DebayerParams::blue
|
|
* \brief Lookup table for blue color, mapping input values to output values
|
|
*/
|
|
|
|
/**
|
|
* \var DebayerParams::redCcm
|
|
* \brief Lookup table for the CCM red column, mapping input values to output values
|
|
*/
|
|
|
|
/**
|
|
* \var DebayerParams::greenCcm
|
|
* \brief Lookup table for the CCM green column, mapping input values to output values
|
|
*/
|
|
|
|
/**
|
|
* \var DebayerParams::blueCcm
|
|
* \brief Lookup table for the CCM blue column, mapping input values to output values
|
|
*/
|
|
|
|
/**
|
|
* \var DebayerParams::gammaLut
|
|
* \brief Gamma lookup table used with color correction matrix
|
|
*/
|
|
|
|
/**
|
|
* \class Debayer
|
|
* \brief Base debayering class
|
|
*
|
|
* Base class that provides functions for setting up the debayering process.
|
|
*/
|
|
|
|
LOG_DEFINE_CATEGORY(Debayer)
|
|
|
|
Debayer::~Debayer()
|
|
{
|
|
}
|
|
|
|
/**
|
|
* \fn int Debayer::configure()
|
|
* \brief Configure the debayer object according to the passed in parameters
|
|
* \param[in] inputCfg The input configuration
|
|
* \param[in] outputCfgs The output configurations
|
|
* \param[in] ccmEnabled Whether a color correction matrix is applied
|
|
*
|
|
* \return 0 on success, a negative errno on failure
|
|
*/
|
|
|
|
/**
|
|
* \fn Size Debayer::patternSize(PixelFormat inputFormat)
|
|
* \brief Get the width and height at which the bayer pattern repeats
|
|
* \param[in] inputFormat The input format
|
|
*
|
|
* Valid sizes are: 2x2, 4x2 or 4x4.
|
|
*
|
|
* \return Pattern size or an empty size for unsupported inputFormats
|
|
*/
|
|
|
|
/**
|
|
* \fn std::vector<PixelFormat> Debayer::formats(PixelFormat inputFormat)
|
|
* \brief Get the supported output formats
|
|
* \param[in] inputFormat The input format
|
|
*
|
|
* \return All supported output formats or an empty vector if there are none
|
|
*/
|
|
|
|
/**
|
|
* \fn std::tuple<unsigned int, unsigned int> Debayer::strideAndFrameSize(const PixelFormat &outputFormat, const Size &size)
|
|
* \brief Get the stride and the frame size
|
|
* \param[in] outputFormat The output format
|
|
* \param[in] size The output size
|
|
*
|
|
* \return A tuple of the stride and the frame size, or a tuple with 0,0 if
|
|
* there is no valid output config
|
|
*/
|
|
|
|
/**
|
|
* \fn void Debayer::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output, DebayerParams params)
|
|
* \brief Process the bayer data into the requested format
|
|
* \param[in] frame The frame number
|
|
* \param[in] input The input buffer
|
|
* \param[in] output The output buffer
|
|
* \param[in] params The parameters to be used in debayering
|
|
*
|
|
* \note DebayerParams is passed by value deliberately so that a copy is passed
|
|
* when this is run in another thread by invokeMethod().
|
|
*/
|
|
|
|
/**
|
|
* \fn virtual SizeRange Debayer::sizes(PixelFormat inputFormat, const Size &inputSize)
|
|
* \brief Get the supported output sizes for the given input format and size
|
|
* \param[in] inputFormat The input format
|
|
* \param[in] inputSize The input size
|
|
*
|
|
* \return The valid size ranges or an empty range if there are none
|
|
*/
|
|
|
|
/**
|
|
* \var Signal<FrameBuffer *> Debayer::inputBufferReady
|
|
* \brief Signals when the input buffer is ready
|
|
*/
|
|
|
|
/**
|
|
* \var Signal<FrameBuffer *> Debayer::outputBufferReady
|
|
* \brief Signals when the output buffer is ready
|
|
*/
|
|
|
|
} /* namespace libcamera */
|