The existing RkISP1Params helper class allows the RkISP1 IPA to handle
both the extensible parameters format and the legacy fixed-size format.
With the introduction of v4l2-isp.h in the Linux kernel the part of
the RkISP1Params helper class that handles extensible parameters can
be generalized so that other IPA modules can use the same helpers
to populate a parameters buffer compatible with v4l2-isp.h.
Generalize the RkISP1Params class to a new libipa component named
V4L2Params and derive the existing RkISP1Params from it, leaving
in the RkISP1-specific implementation the handling of the legacy format.
Deriving RkISP1Params from V4L2Params requires changing the size
associated to each block to include the size of v4l2_params_block_header
in the ipa:rkisp1::kBlockTypeInfo map as the V4L2Params::block()
implementation doesn't account for that as RkIS1Params::block()
implementation did.
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Tested-by: Antoine Bouyer <antoine.bouyer@nxp.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The yTarget loading code is broken and works neither for plain values
nor for arrays of values to form a PWL. Fix this by supporting both
cases. If a list is provided in the tuning file construct a PWL,
otherwise construct a single point PWL with the given value.
Fixes: 24247a12c7 ("ipa: libipa: Add AgcMeanLuminance base class")
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
In some situations it is necessary to specify the target brightness
value depending on the overall lux level. Replace the float
relativeLuminanceTraget by a PWL. As the PWL loading code loads a plain
value as single point PWL, backwards compatibility to existing tuning
files is ensured.
While at it, order the class members in reverse xmas tree notation.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In the tuning files, it is useful to specify some values as PWL that
produces a different output value depending on an input value. For
example it is useful to specify the relativeLuminanceTarget depending on
the lux level, so that the regulation regulates a bit darker in low
light scenes. For simple setups this is not necessary and a single value
is sufficient.
This patch extends the yaml loading code, so that a single point PWL can
also be specified as a plain value. This way the following yaml
expressions are all valid:
yTarget: [ 1000, 0.15, 2000, 0.17 ] # Regular PWL
yTarget: [ 0, 0.17 ] # Single point PWL
yTarget: 0.17 # Same as above
For cases (I'm not aware of any) where a single point Pwl is not allowed
there is no change as that must be checked externally anyways.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
In commit f077c58e08 ("libipa: exposure_mode_helper: Take
exposure/gain quantization into account") calculation of the
quantization gain was added to ExposureModeHelper::clampGain(). This
works as expected when a sensor helper is configured but the gain is not
reset to 1.0 in case the sensor helper is not configured. This leads to
incorrect gain calculations in ExposureModeHelper::splitExposure() as
that expects the quantization gain to be valid in any case. Fix that by
setting the quantization gain to 1.0 in case no sensor helper is
configured.
Fixes: f077c58e08 ("libipa: exposure_mode_helper: Take exposure/gain quantization into account")
Closes: https://gitlab.freedesktop.org/camera/libcamera/-/issues/292
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Tested-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
It is beneficial to have the option during development to disable and
enable algorithms via the tuning file without having to delete their
entries.
Add support for an optional "enabled" parameter to accomplish this.
Usage example:
version: 1
algorithms:
- Agc:
enabled: true
- Awb:
enabled: false
This will enable AGC, and disable AWB. If the enabled flag is not
present, the algorithm will be enabled.
Signed-off-by: Isaac Scott <isaac.scott@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
[Kieran: Reflow text]
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The imx708 sensor driver has long been available, especially in raspberry
pi kernels; and the raspberry pi ipa module has had the corresponding
helper class since 2023 (952ef94ed7). The camera sensor properties
database also has an entry for it (2fb0f25019), but the camera sensor
helper class is missing from the common libipa component. So add it, with
the same gain formula present in the raspberry pi ipa module, and the black
level taken from the rpi tuning files.
Handling the raspberry pi specific "wide" / "noir" suffixes is omitted.
They are not present in the camera sensor properties database either.
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
[Add black level, extend commit message.]
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Calculate the error introduced by quantization as "quantization gain"
and return it separately from splitExposure(). It is not included in the
digital gain, to not silently ignore the limits imposed by the AGC
configuration.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In ExposureModeHelper::splitExposure() the quantization of exposure time
and gain is not taken into account. This can lead to visible flicker
when the quantization steps are too big. As a preparation to fixing
that, add a function to set the sensor line length and the current
sensor mode helper and extend the clampXXX functions to return the
quantization error.
By default the exposure time quantization is assumed to be 1us and gain
is assumed to not be quantized at all.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add a small utility function that calculates the quantized gain that
gets applied by a sensor when the gain code is set to gainCode(gain).
This is needed by algorithms to calculate a digital correction gain that
gets applied to mitigate the error introduce by quantization.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Top-level `const` qualifiers are not useful, so avoid them. This is done
either by simply removing the top-level `const`, or making the function
return a reference to const where that is appropriate.
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Exposure compensation allows to over- or under-expose an
image by a given value. Add support for that in agc_mean_luminance.
The added exposure compensation can lead to luminance target values that
are close or above saturation and are therefore never reachable. Add a
fix for that by limiting the maximum luminance target to 0.95.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Fix the eval() function for Pwls that contain only a single point.
Remove the should_fail from the corresponding test case.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
If a Pwl is evaluated outside of the domain, it gets extrapolated
linearly. Fix the documentation as it states otherwise.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In the grey world AWB case, if no colour gains are contained in the
tuning file, the colour gains get reset to 1 when the colour temperature
is set manually. This is unexpected and undesirable. Allow the
gainsFromColourTemp() function to return a std::nullopt to handle that
case.
While at it, remove an unnecessary import from rkisp1/algorithms/awb.h.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The interQuantileMean() is supposed to return a weighted mean value
between two quantiles. This works for fine histograms, but fails for
coarse histograms and small quantile ranges because the weight is always
taken from the lower border of the bin.
Fix that by rewriting the algorithm to calculate a lower and upper bound
for every (partial) bin that goes into the mean calculation and weight
the bins by the middle of these bounds.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The calculation of the frac variable is based solely on integers and
therefore results in the fractional part being either 0 or 1.
In the original code from RaspberryPi this is mitigated by casting the
nominator to a double. This works for most cases, but fails when q is
very small because of the quantization introduced by item being an
integer.
Fix both issues by doing the full calculation in double and remove the
should_fail tag.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
In a proper system it never happens that the effectiveExposureValue
drops to zero. If that still happens due to a bug outside of
agc_mean_luminance, the calculated gain goes towards infinity but the
newExposureValue is still 0 because it is the result of multiplying the
effectiveExposureTime with the gain, leading to wild oscillations.
Catch that condition, print an error message and set the new effective
exposure value to an arbitrary 10ms.
Note that in any case the underlying problem must be fixed. The
important change is the added error message to be able to detect such a
situation.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The ExposureModeHelper::splitExposures() runs through the configured
stages to find the best gain/exposure time pair. It first raises the
exposure time until it reaches the limit of the current stage. Then it
raises the gain until that also reaches the limit of the current stage.
After that it continues with the next stage until a match is found.
Due to a slight mistake in the initial code, the second step doesn't
work as expected because the exposure time gets divided by the gain of
the current stage, effectively leading to a jump of the gain value from
the maximum gain of the last stage to the maximum gain of the current
stage instead of gradually increasing the gain value.
Depending on the tuning file this leads to very visible oscillations and
jumps in the brightness.
Fix by clamping the exposure time in the second step to the maximum
exposure time of the current stage.
While at it, add two comments for easier understanding.
Fixes: 34c9ab6282 ("ipa: libipa: Add ExposureModeHelper")
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The AwbStats structure has virtual functions but a publicly accessible
non-virtual destructors. This can cause undefined behaviour if deleting
a derived class instance through a pointer to the base AwbStats class.
The problem is theoretical only as no code in libcamera is expected to
perform such deletion, but compilers can't know that and will emit a
warning if the -Wnon-virtual-dtor option is enabled.
Fixing this can be done by declaring a virtual public destructor in the
AwbStats class. A more efficient alternative is to declare a protected
non-virtual destructor, ensuring that instances can't be deleted through
a pointer to the base class. Do so, and mark the derived RkISP1AwbStats
as final to avoid the same warning.
Fixes: 6f663990a0 ("libipa: Add AWB algorithm base class")
Reported-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Tested-by: Milan Zamazal <mzamazal@redhat.com>
The doxygen \todo directive doesn't need to be followed by a colon. Drop
it. While at it, turn one 'todo:' into '\todo'.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Fix the top-level file description to mention the file contains an AWB
grey world implementation, not a base class, and fix a grammar mistake
in a documentation block.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Drop unneeded headers and add missing ones.
The yaml_parser.h header is dropped from awb_grey.h as the classes it
provides are only used in virtual functions defined by the base class,
so any required definitions are guaranteed to be available already.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
The convention in libcamera is not to prefix getters with a 'get'
prefix. Rename the AwbStats::getRGBMeans() function accordingly.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Function names are followed by parentheses in doxygen documentation
blocks as convention in libcamera. Add missing parentheses in the
AwbAlgorithm documentation.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Sort the documentation of the class members in the same order as the
member declaration in the class definition, as is customary in
libcamera.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
The original code used to specify the probabilities in log space and
scaled for the RaspberryPi hardware with 192 AWB measurement points.
This is reasonable as the whole algorithm makes use of unitless numbers
to prefer some colour temperatures based on a lux level. These numbers
are then hand tuned with the specific device in mind.
This has two shortcomings:
1. The linear interpolation of PWLs in log space is mathematically
incorrect. The outcome might still be ok, as both spaces (log and
linear) are monotonic, but it is still not "right".
2. Having unitless numbers gets more error prone when we try to
harmonize the behavior over multiple platforms.
Change the algorithm to interpret the numbers as being in linear space.
This makes the interpolation mathematically correct at the expense of a
few log operations.
To account for that change, update the numbers in the tuning example
file with the linear counterparts scaled to one AWB zone measurement.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Logging every search step is too verbose even with debug messages
enabled and it hides the more important messages (min max values of
errors and likelihoods). Remove the debug messages in a separate commit,
so that it can easily be reverted if needed.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
When tuning the AWB algorithm it is more helpful to get a feeling for
the value ranges than to get verbose output of every single step. Add a
small utility class to track the limits and log them.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
The bayesian AWB algorithm is an AWB algorithm that takes prior
probabilities for a given light source dependent on the current lux
level into account.
The biggest improvement compared to the grey world model comes from the
search of the ideal white point on the CT curve. The algorithm walks the
CT curve to minimize the colour error for a given statistics. After the
minimium is found it additionally tries to search the area around that
spot and also off the curve. So even without defined prior probabilities
this algorithm provides much better results than the grey world
algorithm.
The logic for this code was taken from the RaspberryPi implementation.
The logic was only minimally adjusted for usage with the rkisp1 and a
few things were left out (see doxygen doc for the AwbBayes class). The
code is refactored to better fit the libcamera code style and to make
use of the syntactic sugar provided by the Interpolator and Vector
classes.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
The AWB modes are specified in the libcamera core controls. It is
therefore quite likely that every AWB algorithm will implement them. Add
helper functions for parsing and storing the configured modes in the
AwbAlgorithm base class.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Add a class to provide a generic interface for auto white balance
algorithms. Concrete AWB algorithms are expected to subclass the
AwbAlgorithm class to implement their functionality.
IPAs are expected to subclass the AwbStats class and implement the
necessary functions to give the algorithm access to the hardware
specific statistics data.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Jacopo Mondi
Stefan Klug
Isaac ScottandKieran Bingham
Barnabás Pőcze
Daniel ScallyandKieran Bingham
Matthias FendandKieran Bingham
Benjamin MugnierandKieran Bingham
Laurent Pinchart