In the current code, decompand will only set a curve in the prepare
phase, which will only run after 1-2 frames pass through the FE. This
is fixed by adding an initialValues() member function to the decompand
algorithm, which will be called in the IPA before we start the hardware
streaming.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Don't let applyDecompand() directly change the FE global enables.
Instead pass the global mask into the function and set it in the caller.
This will be needed when a future commit will add setting the decompand
initial values.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This patch integrates a new decompand algorithm that utilizes the PiSP
FE hardware block available on Raspberry Pi 5. The implementation
enables conversion of companded sensor data into linear format prior to
ISP processing.
Changes include:
- Implementation of decompand logic for controller and pipe interfaces
- Enabling decompand block by "rpi.decompand" in tuning.json
Signed-off-by: Sena Asotani <aso.fam429@gmail.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
If the histogram size is non-zero but lower than the number of bins,
yHistValsPerBin is zero and then the AGC processing crashes on division
by it. Let's check yHistValsPerBin for being zero and stop AGC
processing in such a case. The condition also covers the cases where
histogramSize or yHistValsPerBinMod are zero.
Tested-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Hans de Goede <hansg@kernel.org>
Tested-by: Hans de Goede <hansg@kernel.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The result type of the std::accumulate() call is uint64_t; let's use the
same type for the initial value (rather than the default int) to prevent
summing up the values in a different integer type.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Hans de Goede <hansg@kernel.org>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Tested-by: Hans de Goede <hansg@kernel.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The R/G/B sums computed in AWB simple IPA may be zero or perhaps even
negative. Let's make sure the sums are always positive, to prevent
division by zero or completely nonsense results.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Hans de Goede <hansg@kernel.org>
Tested-by: Hans de Goede <hansg@kernel.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
At the moment the blc code uses 255 as starting blacklevel for sensors
where there is no blacklevel info in the sensor-helper.
There are a number of issues with this:
1. When the first frame is bad (e.g. mostly white) which happens sometimes
the initial blacklevel will be kept leading to a divide by zero problem
in the AGC code (this divide by 0 problem is avoided in the AGC code by
not running the AGC algorithm).
2. Not runnning the AGC algorithm means that the gain/exposure do not
change, which causes the BLC algorithm to not run on the next frames,
so we keep the bad 255 blacklevel which stops AGC from running which
stops BLC from running. Leaving things stuck at a 255 blacklevel
resulting in an unusuable image.
3. Sometimes the auto-blc code leads to an unrealistic high
blacklevel detection which results in lower image quality.
To fix this start with a blacklevel of 16, which is the highest
(4096 / 256) blacklevel used for any sensor listing a blackLevel_
value in the sensor-helper class.
Note 2. could alternatively be fixed by disabling the check for
exposure/gain changes when the blacklevel is unrealistic high,
but that still leaves the other problems.
Signed-off-by: Hans de Goede <hansg@kernel.org>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Generating statistics for every single frame is not really necessary.
However a roundtrip through ipa_->processStats() still need to be done
every frame, even if there are no stats to make the IPA generate metadata
for every frame.
Add a valid flag to the statistics struct to let the IPA know when there
are no statistics for the frame being processed and modify the IPA to
only generate metadata for frames without valid statistics.
This is a preparation patch for skipping statistics generation for some
frames.
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Hans de Goede <hansg@kernel.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
At the moment when the overall image brightness is considered too high
the AGC code will lower the gain all the way down to againMin before
considering lowering the exposure.
What should happen instead is lower the gain no lower than 1.0 and after
that lower the exposure instead of lowering the gain.
Otherwise there might be a heavily overexposed image (e.g. all white)
which then is made less white by a gain < 1.0 which is no good.
When there is no sensor-helper, assume the driver reported default-gain
value is close to a gain of 1.0 .
While at it also remove the weird limitation to only lower the gain
when exposure is set to the maximum. As long as the gain is higher
than the default gain, the gain should be lowered first.
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
Signed-off-by: Hans de Goede <hansg@kernel.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The code was only allowing the exposure mode to be updated when both
exposure/gain were in manual mode, which is a mistake. It needs to be
updatable precisely when the auto modes are enabled.
The fix is to ignore the enabled/disabled status of AEC/AGC, matching
our other controls (metering mode, constraint mode etc.). While there
might be a debate to be had about what the actual behaviour of all
these controls should be, for the time being we'll just match
everything else.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add a WDR algorithm to do global tone mapping. Global tone mapping is
used to increase the perceived dynamic range of an image. The typical
effect is that in areas that are normally overexposed, additional
structure becomes visible.
The overall idea is that the algorithm applies an exposure value
correction to underexpose the image to the point where only a small
number of saturated pixels is left. This artificial underexposure is
then mitigated by applying a tone mapping curve.
This algorithm implements 4 tone mapping strategies:
- Linear
- Power
- Exponential
- Histogram equalization
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Query the params device for RKISP1_CID_SUPPORTED_PARAMS_BLOCKS and
inject the information into the IPA hardware context for use by the
algorithms.
To be able to modify the hardware configuration at runtime, replace the
pointer with an instance and create a copy of the static hardware
specific data.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The Y mode of the histogram gets captured at the ISP output, before the
output formatter. This has the side effect that the first and the last
bins are empty in case of limited YUV range. Another side effect is
that gamma and GWDR processing is included in the histogram which makes
algorithm development very difficult. In RGB mode the histogram is taken
after xtalk (CCM) and is therefore independent of gamma and WDR. The
limited range issue also does not apply. In the ISP reference it is
however stated that "it is not possible to calculate a luminance or
grayscale histogram from an RGB histogram since the position information
is lost during its generation".
During testing the RGB histogram provided good data and better
algorithmic stability at a possible (but not measured) inaccuracy.
Another option would be to pass the color space information into the IPA
and strip the histogram accordingly. For ease of implementation switch to
the RGB mode.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
There are several occasions where quantization can lead to visible
effects.
In WDR mode it can happen that exposure times get set to very low values
(Sometimes 2-3 lines). This intentionally introduced underexposure is
corrected by the GWDR module. As exposure time is quantized by lines,
the smallest possible change in exposure time now results in a quite
visible change in perceived brightness.
On some sensors the possible gain steps are also quite large leading to
visible jumps if e.g. if the exposure time is fixed.
Mitigate that by applying a global gain to account for the error
introduced by the exposure quantization.
ToDo: This needs perfect frame synchronous control of the sensor to work
properly which is not guaranteed in all cases. It still improves the
behavior with the current regulation and can easily be skipped, be
removing the compress algorithm from the tuning file.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Calculate the error introduced by quantization as "quantization gain"
and return it separately from splitExposure(). It is not included in the
digital gain, to not silently ignore the limits imposed by the AGC
configuration.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In ExposureModeHelper::splitExposure() the quantization of exposure time
and gain is not taken into account. This can lead to visible flicker
when the quantization steps are too big. As a preparation to fixing
that, add a function to set the sensor line length and the current
sensor mode helper and extend the clampXXX functions to return the
quantization error.
By default the exposure time quantization is assumed to be 1us and gain
is assumed to not be quantized at all.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add a small utility function that calculates the quantized gain that
gets applied by a sensor when the gain code is set to gainCode(gain).
This is needed by algorithms to calculate a digital correction gain that
gets applied to mitigate the error introduce by quantization.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The i.MX8 M Plus has a compression curve inside the compand block. This
curve is necessary to process HDR stitched data and is useful for other
aspects like applying a digital gain to the incoming sensor data.
Add a basic algorithm for the compression curve. This algorithm has a
hardcoded input width of 20bit and output width of 12bit which matches
the imx8mp pipeline. Only a static gain is supported in this version.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
For both vc4 and pisp, vd55g1.json has been generated using ctt with
rpi.dpc algorithm removed as this is already handled in the sensor's
ISP. vd55g1_mono.json has been adapted from vd55g1.json by removing
color correction related algorithms.
Adding Cyril Liotard as co-developer for providing the base vd55g1.json
tuning files for both vc4 and pisp. Thank you.
Co-Developed-by: Cyril Liotard <cyril.liotard@st.com>
Signed-off-by: Cyril Liotard <cyril.liotard@st.com>
Signed-off-by: Benjamin Mugnier <benjamin.mugnier@foss.st.com>
Acked-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The CCM algorithm will now let an explicit colour matrix be set when
AWB is in manual mode.
We must handle any controls that can cause the AWB to be enabled or
disabled first, so that we know the AWB's state correctly when we come
to set the CCM.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
[Kieran: Remove duplicated Matrix3x3 from ccm.cpp]
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Sharpening is reduced slightly for official Raspberry Pi cameras, and
exposure profiles made a bit more consistent.
Denoise is reduced for the imx708 where it appears too strong.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Noise and detail tuning is improved for all official Raspberry Pi
cameras.
The old tunings left too much noise in and even sharpened some of
it. The new tunings remove more noise, and no longer sharpen it. Some
of the more general over-sharpening is also removed. Note that lost
detail can be recovered well using TDN (temporal denoise), which is
the recommended method to get the best results.
There are some minor adjustments to the CDN deviation, now that this
gets backed-off as TDN ramps up.
The contrast in the gamma in the bright areas is also toned down just
a little.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The CDN (colour denoise) deviation gets gradually reduced frame by
frame as TDN (temporal denoise) comes in and has more effect. CDN is
more harmful to image detail than TDN, so ramping it down in favour of
TDN is beneficial.
The tuning file parameters are chosen so that existing tuning files
don't have to be updated and will carry on working "mostly like they
did" (apart from the new back-off).
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add an imx415 tuning file for both the VC4 and PiSP. This tuning file
has been created and supplied by Arducam to support the B0569 module.
Note that this conflicts with an already existing imx415.json and
as such is provided as imx415_b0459.json.
More work will be required to support module specific tuning file
parsing.
Acked-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add an imx335 tuning file for both VC4 and PiSP. This tuning file
has been created and supplied by Arducam to support the B0568 module.
Acked-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Top-level `const` qualifiers are not useful, so avoid them. This is done
either by simply removing the top-level `const`, or making the function
return a reference to const where that is appropriate.
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
For both vc4 and pisp, vd56g3.json has been generated using ctt with
rpi.dpc algorithm removed as this is already handled in the sensor's
ISP. vd56g3_mono.json has been adapted from vd56g3.json by removing
color correction related algorithms.
Adding Cyril Liotard and Jean Poire as co-developers for providing the
base vd56g3.json tuning files for vc4 and pisp respectively. Thank you.
Signed-off-by: Benjamin Mugnier <benjamin.mugnier@foss.st.com>
Co-developed-by: Cyril Liotard <cyril.liotard@st.com>
Signed-off-by: Cyril Liotard <cyril.liotard@st.com>
Co-developed-by: Jean Poire <jean.poire@st.com>
Signed-off-by: Jean Poire <jean.poire@st.com>
Reviewed-by: Naushir Patuck <naush@rasbperrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Header comment blocks used to contain the file name. Considered as
useless information, the names have been removed, with the last ones
supposed to be dropped in commit d3bf27180e ("libcamera: Drop
remaining file names from header comment blocks"). A few have however
been forgotten, and more crept back since. Remove them.
While at it, fix one typo in a header comment block by replacing
'MaliC55 with Mali-C55', and add a missing blank line in
src/ipa/rpi/pisp/pisp.cpp.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
At the moment the mali-c55 pipeline handler sets bytesused for a
buffer to be the maximum possible size (i.e. the size of a struct
mali_c55_params_buffer). This is not really in keeping with the goal
of the extensible parameters formats, and will not work with the new
framework for those formats. Update the IPA module and pipeline
handler to set bytesused to the size of the parameters that were
actually supplied rather than the maximum possible size.
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
