At the moment when the overall image brightness is considered too high
the AGC code will lower the gain all the way down to againMin before
considering lowering the exposure.
What should happen instead is lower the gain no lower than 1.0 and after
that lower the exposure instead of lowering the gain.
Otherwise there might be a heavily overexposed image (e.g. all white)
which then is made less white by a gain < 1.0 which is no good.
When there is no sensor-helper, assume the driver reported default-gain
value is close to a gain of 1.0 .
While at it also remove the weird limitation to only lower the gain
when exposure is set to the maximum. As long as the gain is higher
than the default gain, the gain should be lowered first.
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
Signed-off-by: Hans de Goede <hansg@kernel.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The code was only allowing the exposure mode to be updated when both
exposure/gain were in manual mode, which is a mistake. It needs to be
updatable precisely when the auto modes are enabled.
The fix is to ignore the enabled/disabled status of AEC/AGC, matching
our other controls (metering mode, constraint mode etc.). While there
might be a debate to be had about what the actual behaviour of all
these controls should be, for the time being we'll just match
everything else.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add a WDR algorithm to do global tone mapping. Global tone mapping is
used to increase the perceived dynamic range of an image. The typical
effect is that in areas that are normally overexposed, additional
structure becomes visible.
The overall idea is that the algorithm applies an exposure value
correction to underexpose the image to the point where only a small
number of saturated pixels is left. This artificial underexposure is
then mitigated by applying a tone mapping curve.
This algorithm implements 4 tone mapping strategies:
- Linear
- Power
- Exponential
- Histogram equalization
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Query the params device for RKISP1_CID_SUPPORTED_PARAMS_BLOCKS and
inject the information into the IPA hardware context for use by the
algorithms.
To be able to modify the hardware configuration at runtime, replace the
pointer with an instance and create a copy of the static hardware
specific data.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The Y mode of the histogram gets captured at the ISP output, before the
output formatter. This has the side effect that the first and the last
bins are empty in case of limited YUV range. Another side effect is
that gamma and GWDR processing is included in the histogram which makes
algorithm development very difficult. In RGB mode the histogram is taken
after xtalk (CCM) and is therefore independent of gamma and WDR. The
limited range issue also does not apply. In the ISP reference it is
however stated that "it is not possible to calculate a luminance or
grayscale histogram from an RGB histogram since the position information
is lost during its generation".
During testing the RGB histogram provided good data and better
algorithmic stability at a possible (but not measured) inaccuracy.
Another option would be to pass the color space information into the IPA
and strip the histogram accordingly. For ease of implementation switch to
the RGB mode.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
There are several occasions where quantization can lead to visible
effects.
In WDR mode it can happen that exposure times get set to very low values
(Sometimes 2-3 lines). This intentionally introduced underexposure is
corrected by the GWDR module. As exposure time is quantized by lines,
the smallest possible change in exposure time now results in a quite
visible change in perceived brightness.
On some sensors the possible gain steps are also quite large leading to
visible jumps if e.g. if the exposure time is fixed.
Mitigate that by applying a global gain to account for the error
introduced by the exposure quantization.
ToDo: This needs perfect frame synchronous control of the sensor to work
properly which is not guaranteed in all cases. It still improves the
behavior with the current regulation and can easily be skipped, be
removing the compress algorithm from the tuning file.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Calculate the error introduced by quantization as "quantization gain"
and return it separately from splitExposure(). It is not included in the
digital gain, to not silently ignore the limits imposed by the AGC
configuration.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In ExposureModeHelper::splitExposure() the quantization of exposure time
and gain is not taken into account. This can lead to visible flicker
when the quantization steps are too big. As a preparation to fixing
that, add a function to set the sensor line length and the current
sensor mode helper and extend the clampXXX functions to return the
quantization error.
By default the exposure time quantization is assumed to be 1us and gain
is assumed to not be quantized at all.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add a small utility function that calculates the quantized gain that
gets applied by a sensor when the gain code is set to gainCode(gain).
This is needed by algorithms to calculate a digital correction gain that
gets applied to mitigate the error introduce by quantization.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The i.MX8 M Plus has a compression curve inside the compand block. This
curve is necessary to process HDR stitched data and is useful for other
aspects like applying a digital gain to the incoming sensor data.
Add a basic algorithm for the compression curve. This algorithm has a
hardcoded input width of 20bit and output width of 12bit which matches
the imx8mp pipeline. Only a static gain is supported in this version.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
For both vc4 and pisp, vd55g1.json has been generated using ctt with
rpi.dpc algorithm removed as this is already handled in the sensor's
ISP. vd55g1_mono.json has been adapted from vd55g1.json by removing
color correction related algorithms.
Adding Cyril Liotard as co-developer for providing the base vd55g1.json
tuning files for both vc4 and pisp. Thank you.
Co-Developed-by: Cyril Liotard <cyril.liotard@st.com>
Signed-off-by: Cyril Liotard <cyril.liotard@st.com>
Signed-off-by: Benjamin Mugnier <benjamin.mugnier@foss.st.com>
Acked-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The CCM algorithm will now let an explicit colour matrix be set when
AWB is in manual mode.
We must handle any controls that can cause the AWB to be enabled or
disabled first, so that we know the AWB's state correctly when we come
to set the CCM.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
[Kieran: Remove duplicated Matrix3x3 from ccm.cpp]
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Sharpening is reduced slightly for official Raspberry Pi cameras, and
exposure profiles made a bit more consistent.
Denoise is reduced for the imx708 where it appears too strong.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Noise and detail tuning is improved for all official Raspberry Pi
cameras.
The old tunings left too much noise in and even sharpened some of
it. The new tunings remove more noise, and no longer sharpen it. Some
of the more general over-sharpening is also removed. Note that lost
detail can be recovered well using TDN (temporal denoise), which is
the recommended method to get the best results.
There are some minor adjustments to the CDN deviation, now that this
gets backed-off as TDN ramps up.
The contrast in the gamma in the bright areas is also toned down just
a little.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The CDN (colour denoise) deviation gets gradually reduced frame by
frame as TDN (temporal denoise) comes in and has more effect. CDN is
more harmful to image detail than TDN, so ramping it down in favour of
TDN is beneficial.
The tuning file parameters are chosen so that existing tuning files
don't have to be updated and will carry on working "mostly like they
did" (apart from the new back-off).
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add an imx415 tuning file for both the VC4 and PiSP. This tuning file
has been created and supplied by Arducam to support the B0569 module.
Note that this conflicts with an already existing imx415.json and
as such is provided as imx415_b0459.json.
More work will be required to support module specific tuning file
parsing.
Acked-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add an imx335 tuning file for both VC4 and PiSP. This tuning file
has been created and supplied by Arducam to support the B0568 module.
Acked-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Top-level `const` qualifiers are not useful, so avoid them. This is done
either by simply removing the top-level `const`, or making the function
return a reference to const where that is appropriate.
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
For both vc4 and pisp, vd56g3.json has been generated using ctt with
rpi.dpc algorithm removed as this is already handled in the sensor's
ISP. vd56g3_mono.json has been adapted from vd56g3.json by removing
color correction related algorithms.
Adding Cyril Liotard and Jean Poire as co-developers for providing the
base vd56g3.json tuning files for vc4 and pisp respectively. Thank you.
Signed-off-by: Benjamin Mugnier <benjamin.mugnier@foss.st.com>
Co-developed-by: Cyril Liotard <cyril.liotard@st.com>
Signed-off-by: Cyril Liotard <cyril.liotard@st.com>
Co-developed-by: Jean Poire <jean.poire@st.com>
Signed-off-by: Jean Poire <jean.poire@st.com>
Reviewed-by: Naushir Patuck <naush@rasbperrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Header comment blocks used to contain the file name. Considered as
useless information, the names have been removed, with the last ones
supposed to be dropped in commit d3bf27180e ("libcamera: Drop
remaining file names from header comment blocks"). A few have however
been forgotten, and more crept back since. Remove them.
While at it, fix one typo in a header comment block by replacing
'MaliC55 with Mali-C55', and add a missing blank line in
src/ipa/rpi/pisp/pisp.cpp.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
At the moment the mali-c55 pipeline handler sets bytesused for a
buffer to be the maximum possible size (i.e. the size of a struct
mali_c55_params_buffer). This is not really in keeping with the goal
of the extensible parameters formats, and will not work with the new
framework for those formats. Update the IPA module and pipeline
handler to set bytesused to the size of the parameters that were
actually supplied rather than the maximum possible size.
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Here we update the digital gain handling to use the value computed by
process() in the AgcStatus, not the version that was previously in the
AgcPrepareStatus.
Because we apply this digital gain directly with no further
modification, we have to update it to reflect any exposure/gain
quantisation that happens (in IpaBase::applyAGC).
We must also run the new platformPrepareAgc() even when we're skipping
platformPrepareIsp(), which has been split out of the previous
platformPrepareIsp() implementation.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Previously we let prepare() do the work by comparing the desired total
exposure against the shutter time and analogue gain. This can cause
the image to "wink" at high framerates because we may skip running
prepare() to get the new digital gain even when the delayed AGC status
(which came out of an earlier call to process()) shows that a change
was required.
Now we're taking explicit control of the digital gain by calculating
it ourselves so that we can output it in the standard AgcStatus
object. This means that whenever the delayed AGC status changes, we
have the correct digital gain to go with it.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The delay context counter must be advanced even when we decide not to
run prepare/process. Otherwise, when we skip them at higher
framerates, the current IPA context counter will catch up and
overwrite the delay context.
It's safe to advance the counter because the metadata is always copied
forward a slot when we decide not to run the IPAs.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Much of the time we use the term "analogue gain" where we really mean
the combined analogue and digital gain (because the digital gain will
make up whatever the analogue gain can't deliver).
This commit replaces the use of "analogue gain" by just "gain" in
places where we really mean the combined gain. There are a couple of
principle areas:
1. Where we previously talked about the "fixedAnalaogueGain"
(including setting it "manually") this is now just the "fixedGain"
(because it always encompassed both analogue and digital gain).
Along with this, the setfixedExposureTime/Gain functions no longer
update the output status directly. Applications should wait in the
usual way for AGC/AEC changes to take effect, and this "shortcut"
actually doesn't fit well with the gain being the combined gain.
2. The divideUpExposure method is adjusted to be clearer that it's
setting the combined gain, and it's prepare() that will discover later
what the analogue gain actually delivered.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Previously these were handled in the AGC/AEC exposure update
calculations by explicitly driving a higher digital gain to "cancel
out" any colour gains that were less than 1.
Now we're ignoring this in the AGC and leaving it to the IPA code to
normalise all the gains so that the smallest is 1. We don't regard
this as a "real" increase because one of the colour channels (just not
necessarily the green one) still gets the minimum gain possible.
We do, however, update the statistics calculations so that they
reflect any such digital gain increase, so that images are driven to
the correct level.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Exposure compensation allows to over- or under-expose an
image by a given value. Add support for that in agc_mean_luminance.
The added exposure compensation can lead to luminance target values that
are close or above saturation and are therefore never reachable. Add a
fix for that by limiting the maximum luminance target to 0.95.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
