The yTarget loading code is broken and works neither for plain values
nor for arrays of values to form a PWL. Fix this by supporting both
cases. If a list is provided in the tuning file construct a PWL,
otherwise construct a single point PWL with the given value.
Fixes: 24247a12c7 ("ipa: libipa: Add AgcMeanLuminance base class")
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
In some situations it is necessary to specify the target brightness
value depending on the overall lux level. Replace the float
relativeLuminanceTraget by a PWL. As the PWL loading code loads a plain
value as single point PWL, backwards compatibility to existing tuning
files is ensured.
While at it, order the class members in reverse xmas tree notation.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
With the upcoming regulation rework the processing order in the IPA is
updated a bit so that process() updates the active state with new
measurements and fills the metadata with the data from the corresponding
frame context. In prepare() all parameters for one frame are tied
together using the most up to date values from active state.
Change the lux algorithm to support that order of events. Also prepare
for cases where stats can be null which can happen with the upcoming
regulation rework.
While at it fix a formatting issue reported by checkstyle and drop a
unnecessary local variable.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In the tuning files, it is useful to specify some values as PWL that
produces a different output value depending on an input value. For
example it is useful to specify the relativeLuminanceTarget depending on
the lux level, so that the regulation regulates a bit darker in low
light scenes. For simple setups this is not necessary and a single value
is sufficient.
This patch extends the yaml loading code, so that a single point PWL can
also be specified as a plain value. This way the following yaml
expressions are all valid:
yTarget: [ 1000, 0.15, 2000, 0.17 ] # Regular PWL
yTarget: [ 0, 0.17 ] # Single point PWL
yTarget: 0.17 # Same as above
For cases (I'm not aware of any) where a single point Pwl is not allowed
there is no change as that must be checked externally anyways.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
In commit f077c58e08 ("libipa: exposure_mode_helper: Take
exposure/gain quantization into account") calculation of the
quantization gain was added to ExposureModeHelper::clampGain(). This
works as expected when a sensor helper is configured but the gain is not
reset to 1.0 in case the sensor helper is not configured. This leads to
incorrect gain calculations in ExposureModeHelper::splitExposure() as
that expects the quantization gain to be valid in any case. Fix that by
setting the quantization gain to 1.0 in case no sensor helper is
configured.
Fixes: f077c58e08 ("libipa: exposure_mode_helper: Take exposure/gain quantization into account")
Closes: https://gitlab.freedesktop.org/camera/libcamera/-/issues/292
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Tested-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
It is beneficial to have the option during development to disable and
enable algorithms via the tuning file without having to delete their
entries.
Add support for an optional "enabled" parameter to accomplish this.
Usage example:
version: 1
algorithms:
- Agc:
enabled: true
- Awb:
enabled: false
This will enable AGC, and disable AWB. If the enabled flag is not
present, the algorithm will be enabled.
Signed-off-by: Isaac Scott <isaac.scott@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
[Kieran: Reflow text]
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Software CPU ISP computes a gamma lookup table. The table incorporates
black level and contrast. All entries in the table below the black
level are set to 0. This is not necessarily correct all the time.
Let's consider this case: The CCM is
[1 0 0]
[0 1 0]
[0 0 0]
and contrast is set to zero. The gamma table has all the entries above
the black level set to 186 (due to zero contrast) and all the entries
below the black level set to 0. CCM is applied before gamma, a
non-black level pixel has the blue component set to 0 with the CCM
above. Now, when the gamma lookup is applied, the red and green
components are set to 186, while the blue component is set to 0. The
resulting pixel is then yellow rather than grey (as it should be with
zero contrast).
There are two ways to fix this: Either clamping pixel colour channels to
the black level in debayering or setting the below black level entries
in the gamma lookup table to the lowest value of the gamma table rather
than 0. Both should have the same effect. Let's opt for the latter for
its simplicity.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Hans de Goede <johannes.goede@oss.qualcomm.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The filter strength configuration block was previously only enabled on the
first frame (frame == 0). Apply the strength values when denoise is active.
This prevents the strength config from being disabled on subsequent frames.
Signed-off-by: Rui Wang <rui.wang@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
The imx708 sensor driver has long been available, especially in raspberry
pi kernels; and the raspberry pi ipa module has had the corresponding
helper class since 2023 (952ef94ed7). The camera sensor properties
database also has an entry for it (2fb0f25019), but the camera sensor
helper class is missing from the common libipa component. So add it, with
the same gain formula present in the raspberry pi ipa module, and the black
level taken from the rpi tuning files.
Handling the raspberry pi specific "wide" / "noir" suffixes is omitted.
They are not present in the camera sensor properties database either.
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
[Add black level, extend commit message.]
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
We're going to use a "floating statistics region" to store a full
image Y sum. The VC4 platform actually has no floating region for
this, but we can synthesize such a region as follows in software.
We know that the 15 AGC regions that we do have are arranged to cover
the whole image, and they cannot be changed. Adding up the R, G and B
values here will get us most of the way to Y. But we do also need to
know the most recent colour gains, so code must also be added to
remember the last AWB status.
With this change, algorithms can now look at the first floating region
on both VC4 and PiSP platforms to get a full image Y average value.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Floating regions are currently unused on the PiSP platform, so we can
use one of them to get an average Y value for the whole image.
If an algorithm (such as the "lux" algorithm) wants a scene-referred
estimate of absolute brightness, then this would be a better choice
than a value from Y histogram as the latter is not invariant to the
metering mode (e.g. centre-weighted, spot etc.). (So note that for the
AGC/AEC algorithm, where the metering mode is relevant, the Y
histogram is the appropriate choice.)
We also fix the loop that copies the hardware's floating region values
into our statistics structure; it was using the wrong limit which was
causing it not to do anything.
A future commit will update the lux algorithm to use this feature.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In the current code, decompand will only set a curve in the prepare
phase, which will only run after 1-2 frames pass through the FE. This
is fixed by adding an initialValues() member function to the decompand
algorithm, which will be called in the IPA before we start the hardware
streaming.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Don't let applyDecompand() directly change the FE global enables.
Instead pass the global mask into the function and set it in the caller.
This will be needed when a future commit will add setting the decompand
initial values.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This patch integrates a new decompand algorithm that utilizes the PiSP
FE hardware block available on Raspberry Pi 5. The implementation
enables conversion of companded sensor data into linear format prior to
ISP processing.
Changes include:
- Implementation of decompand logic for controller and pipe interfaces
- Enabling decompand block by "rpi.decompand" in tuning.json
Signed-off-by: Sena Asotani <aso.fam429@gmail.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
If the histogram size is non-zero but lower than the number of bins,
yHistValsPerBin is zero and then the AGC processing crashes on division
by it. Let's check yHistValsPerBin for being zero and stop AGC
processing in such a case. The condition also covers the cases where
histogramSize or yHistValsPerBinMod are zero.
Tested-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Hans de Goede <hansg@kernel.org>
Tested-by: Hans de Goede <hansg@kernel.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The result type of the std::accumulate() call is uint64_t; let's use the
same type for the initial value (rather than the default int) to prevent
summing up the values in a different integer type.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Hans de Goede <hansg@kernel.org>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Tested-by: Hans de Goede <hansg@kernel.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The R/G/B sums computed in AWB simple IPA may be zero or perhaps even
negative. Let's make sure the sums are always positive, to prevent
division by zero or completely nonsense results.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Hans de Goede <hansg@kernel.org>
Tested-by: Hans de Goede <hansg@kernel.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
At the moment the blc code uses 255 as starting blacklevel for sensors
where there is no blacklevel info in the sensor-helper.
There are a number of issues with this:
1. When the first frame is bad (e.g. mostly white) which happens sometimes
the initial blacklevel will be kept leading to a divide by zero problem
in the AGC code (this divide by 0 problem is avoided in the AGC code by
not running the AGC algorithm).
2. Not runnning the AGC algorithm means that the gain/exposure do not
change, which causes the BLC algorithm to not run on the next frames,
so we keep the bad 255 blacklevel which stops AGC from running which
stops BLC from running. Leaving things stuck at a 255 blacklevel
resulting in an unusuable image.
3. Sometimes the auto-blc code leads to an unrealistic high
blacklevel detection which results in lower image quality.
To fix this start with a blacklevel of 16, which is the highest
(4096 / 256) blacklevel used for any sensor listing a blackLevel_
value in the sensor-helper class.
Note 2. could alternatively be fixed by disabling the check for
exposure/gain changes when the blacklevel is unrealistic high,
but that still leaves the other problems.
Signed-off-by: Hans de Goede <hansg@kernel.org>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Generating statistics for every single frame is not really necessary.
However a roundtrip through ipa_->processStats() still need to be done
every frame, even if there are no stats to make the IPA generate metadata
for every frame.
Add a valid flag to the statistics struct to let the IPA know when there
are no statistics for the frame being processed and modify the IPA to
only generate metadata for frames without valid statistics.
This is a preparation patch for skipping statistics generation for some
frames.
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Hans de Goede <hansg@kernel.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
At the moment when the overall image brightness is considered too high
the AGC code will lower the gain all the way down to againMin before
considering lowering the exposure.
What should happen instead is lower the gain no lower than 1.0 and after
that lower the exposure instead of lowering the gain.
Otherwise there might be a heavily overexposed image (e.g. all white)
which then is made less white by a gain < 1.0 which is no good.
When there is no sensor-helper, assume the driver reported default-gain
value is close to a gain of 1.0 .
While at it also remove the weird limitation to only lower the gain
when exposure is set to the maximum. As long as the gain is higher
than the default gain, the gain should be lowered first.
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
Signed-off-by: Hans de Goede <hansg@kernel.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The code was only allowing the exposure mode to be updated when both
exposure/gain were in manual mode, which is a mistake. It needs to be
updatable precisely when the auto modes are enabled.
The fix is to ignore the enabled/disabled status of AEC/AGC, matching
our other controls (metering mode, constraint mode etc.). While there
might be a debate to be had about what the actual behaviour of all
these controls should be, for the time being we'll just match
everything else.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add a WDR algorithm to do global tone mapping. Global tone mapping is
used to increase the perceived dynamic range of an image. The typical
effect is that in areas that are normally overexposed, additional
structure becomes visible.
The overall idea is that the algorithm applies an exposure value
correction to underexpose the image to the point where only a small
number of saturated pixels is left. This artificial underexposure is
then mitigated by applying a tone mapping curve.
This algorithm implements 4 tone mapping strategies:
- Linear
- Power
- Exponential
- Histogram equalization
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Query the params device for RKISP1_CID_SUPPORTED_PARAMS_BLOCKS and
inject the information into the IPA hardware context for use by the
algorithms.
To be able to modify the hardware configuration at runtime, replace the
pointer with an instance and create a copy of the static hardware
specific data.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The Y mode of the histogram gets captured at the ISP output, before the
output formatter. This has the side effect that the first and the last
bins are empty in case of limited YUV range. Another side effect is
that gamma and GWDR processing is included in the histogram which makes
algorithm development very difficult. In RGB mode the histogram is taken
after xtalk (CCM) and is therefore independent of gamma and WDR. The
limited range issue also does not apply. In the ISP reference it is
however stated that "it is not possible to calculate a luminance or
grayscale histogram from an RGB histogram since the position information
is lost during its generation".
During testing the RGB histogram provided good data and better
algorithmic stability at a possible (but not measured) inaccuracy.
Another option would be to pass the color space information into the IPA
and strip the histogram accordingly. For ease of implementation switch to
the RGB mode.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
There are several occasions where quantization can lead to visible
effects.
In WDR mode it can happen that exposure times get set to very low values
(Sometimes 2-3 lines). This intentionally introduced underexposure is
corrected by the GWDR module. As exposure time is quantized by lines,
the smallest possible change in exposure time now results in a quite
visible change in perceived brightness.
On some sensors the possible gain steps are also quite large leading to
visible jumps if e.g. if the exposure time is fixed.
Mitigate that by applying a global gain to account for the error
introduced by the exposure quantization.
ToDo: This needs perfect frame synchronous control of the sensor to work
properly which is not guaranteed in all cases. It still improves the
behavior with the current regulation and can easily be skipped, be
removing the compress algorithm from the tuning file.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Calculate the error introduced by quantization as "quantization gain"
and return it separately from splitExposure(). It is not included in the
digital gain, to not silently ignore the limits imposed by the AGC
configuration.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In ExposureModeHelper::splitExposure() the quantization of exposure time
and gain is not taken into account. This can lead to visible flicker
when the quantization steps are too big. As a preparation to fixing
that, add a function to set the sensor line length and the current
sensor mode helper and extend the clampXXX functions to return the
quantization error.
By default the exposure time quantization is assumed to be 1us and gain
is assumed to not be quantized at all.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add a small utility function that calculates the quantized gain that
gets applied by a sensor when the gain code is set to gainCode(gain).
This is needed by algorithms to calculate a digital correction gain that
gets applied to mitigate the error introduce by quantization.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The i.MX8 M Plus has a compression curve inside the compand block. This
curve is necessary to process HDR stitched data and is useful for other
aspects like applying a digital gain to the incoming sensor data.
Add a basic algorithm for the compression curve. This algorithm has a
hardcoded input width of 20bit and output width of 12bit which matches
the imx8mp pipeline. Only a static gain is supported in this version.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
For both vc4 and pisp, vd55g1.json has been generated using ctt with
rpi.dpc algorithm removed as this is already handled in the sensor's
ISP. vd55g1_mono.json has been adapted from vd55g1.json by removing
color correction related algorithms.
Adding Cyril Liotard as co-developer for providing the base vd55g1.json
tuning files for both vc4 and pisp. Thank you.
Co-Developed-by: Cyril Liotard <cyril.liotard@st.com>
Signed-off-by: Cyril Liotard <cyril.liotard@st.com>
Signed-off-by: Benjamin Mugnier <benjamin.mugnier@foss.st.com>
Acked-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
