In preparation for switching to a derivation of AgcMeanLuminance, add
a function to parse and store the statistics for easy retrieval in an
overriding estimateLuminance() function.
Now that we have a MeanLuminanceAgc class that centralises our AEGC
algorithm, derive the IPU3's Agc class from it and plumb in the
necessary framework to enable it to be used. For simplicity's sake
this commit switches the algorithm to use the derived class, but
does not remove the bespoke functions at this time.
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The Agc algorithms for the RkIsp1 and IPU3 IPAs do the same thing in
very large part; following the Rpi IPA's algorithm in spirit with a
few tunable values in that IPA being hardcoded in the libipa ones.
Add a new base class for AgcMeanLuminance which implements the same
algorithm and additionally parses yaml tuning files to inform an IPA
module's Agc algorithm about valid constraint and exposure modes and
their associated bounds.
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Provide the Sony IMX415 camera sensor properties and registration
with libipa for the gain code helpers.
The test patterns exposed by the IMX415 do not map well to the current
set of test pattern controls supplied by libcamera. These are left
intentionally unimplemented.
Signed-off-by: Alexander Stein <alexander.stein@ew.tq-group.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Black may not be represented as 0 pixel value for given hardware, it may
be higher. If this is not compensated then various problems may occur
such as low contrast or suboptimal exposure.
The black pixel value can be either retrieved from a tuning file for the
given hardware, or automatically on the fly. The former is the right
and correct method, while the latter can be used when a tuning file is
not available for the given hardware. Since there is currently no
support for tuning files in software ISP, the automatic, hardware
independent way, is always used. Support for tuning files should be
added in future but it will require more work than this patch.
The patch looks at the image histogram and assumes that black starts
when pixel values start occurring on the left. A certain amount of the
darkest pixels is ignored; it doesn't matter whether they represent
various kinds of noise or are real, they are better to omit in any case
to make the image looking better. It also doesn't matter whether the
darkest pixels occur around the supposed black level or are spread
between 0 and the black level, the difference is not important.
An arbitrary threshold of 2% darkest pixels is applied; there is no
magic about that value.
The patch assumes that the black values for different colors are the
same and doesn't attempt any other non-primitive enhancements. It
cannot completely replace tuning files and simplicity, while providing
visible benefit, is its goal. Anything more sophisticated is left for
future patches.
A possible cheap enhancement, if needed, could be setting exposure +
gain to minimum values temporarily, before setting the black level. In
theory, the black level should be fixed but it may not be reached in all
images. For this reason, the patch updates black level only if the
observed value is lower than the current one; it should be never
increased.
The purpose of the patch is to compensate for hardware properties.
General image contrast enhancements are out of scope of this patch.
Stats are still gathered as an uncorrected histogram, to avoid any
confusion and to represent the raw image data. Exposure must be
determined after the black level correction -- it has no influence on
the sub-black area and must be correct after applying the black level
correction. The granularity of the histogram is increased from 16 to 64
to provide a better precision (there is no theory behind either of those
numbers).
Reviewed-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Define the Soft IPA main and event interfaces, add the Soft IPA
implementation.
The current src/ipa/meson.build assumes the IPA name to match the
pipeline name. For this reason "-Dipas=simple" is used for the
Soft IPA module.
Auto exposure/gain and AWB implementation by Dennis, Toon and Martti.
Auto exposure/gain targets a Mean Sample Value of 2.5 following
the MSV calculation algorithm from:
https://www.araa.asn.au/acra/acra2007/papers/paper84final.pdf
Use CameraSensorHelper to convert the analogue gain code read from the
camera sensor into real analogue gain value. In the future this makes
it possible to use faster AE/AGC algorithm. Right now the CameraSensorHelper
lets us use the full range of analogue gain values.
If there is no CameraSensorHelper for the camera sensor in use, a
warning log message is printed.
Tested-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org> # sc8280xp Lenovo x13s
Tested-by: Pavel Machek <pavel@ucw.cz>
Reviewed-by: Pavel Machek <pavel@ucw.cz>
Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org>
Co-developed-by: Dennis Bonke <admin@dennisbonke.com>
Signed-off-by: Dennis Bonke <admin@dennisbonke.com>
Co-developed-by: Marttico <g.martti@gmail.com>
Signed-off-by: Marttico <g.martti@gmail.com>
Co-developed-by: Toon Langendam <t.langendam@gmail.com>
Signed-off-by: Toon Langendam <t.langendam@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Handle the RKISP1_V_IMX8MP version in the rkisp1 IPA. This enables basic
support of the i.MX8MP, excluding the processing blocks specific to the
new ISP version. Those will be supported later.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
The histogram weights are initialized to hardcoded 1's for each
histogram grid cell. The code uses the wrong variable for the grid size,
resulting in some weights having a 0 value. Fix it.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Versions of the ISP differ in the processing blocks they include, as
well as in the implementation of some of those blocks. In particular,
they have different numbers of histogram bins oe AE statistics cells.
The algorithms take these differences into account by checking the ISP
version reported by the driver.
These checks are currently scattered in multiple places. Centralize them
in the IPARkISP1::init() function, and store the version-dependent
hardware parameters in the IPA context, accessible by all algorithms.
While at it, drop the IPASessionConfiguration::hw member that stores the
revision number, unused by the algorithms. It can be added back laer to
the IPAHwSettings structure if needed.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
The RkISP1 statistics structure contains multiple arrays whose length
varies depending on the hardware revision. Accessing those arrays is
error-prone, wrap them in spans at the top level to reduce risks of
out-of-bound accesses.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
The CameraSensorHelperFactory::createInstance() function overrides a
virtual function from CameraSensorHelperFactoryBase. The function
declaration doesn't mark it with the override keyword. This could cause
issues in the future if the base class' function changes, as the
compiler will not issue any warning in that case. Fix it.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
All the Raspberry Pi official camera tuning files are updated for
HDR. As stated previously, there is no mechanism in the hardware for
combining images so all this does is enable multi-channel AGC to
produce short and long exposure frames. It will be up to the
application to deal with them.
The changes are identical in every tuning file.
1. The existing AGC tuning is duplicated twice so that we have 3 AGC
channels.
2. The first is left alone (the default AGC channel), the second is
tweaked to under-expose significantly (ev -3) and the final one is
tweaked to over-exposure slightly (ev +0.5)
3. Control parameters are provided to the "rpi.hdr" algorithm to
associate these AGC channels correctly with the HDR modes.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Sufficient plumbing is added so that the HDR mode control can be used
to engage HDR modes on platforms that support them. On the vc4
platform, this allows multi-channel AGC to run, though there is no
image merging.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Implement the StatsOutputEnable control for the VC4 IPA. When set,
this outputs the ISP statistics as a uint8_t span through the Bcm2835StatsOutput
metadata control.
To get this working, IpaBase::libcameraMetadata_ is moved from a private
to a protected member variable. This makes it accessable to the VC4
derived IPA class.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This allows the IPA to get reasonable default colour gains before AWB
has run. This is particularly important on the PiSP platform where
these numbers are helpful in programming the Front End statistics
block in advance.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This allows the IPA to discover the correct black level values even
before any frames have been processed. This is important on the PiSP
platform where the front end black level blocks must be programmed in
advance.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Advertise hardware constraints on the pixel processing rate through the
Controller::HardwareConfig structure. When calculating the minimum line
length during a configure() operation, ensure that we don't exceed this
constraint.
If we do exceed the hardware constraints, increase the modes's minimum
line length so the pixel processing rate falls below the hardware limit.
If this is not possible, throw a loud error message in the logs.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Meson uses tags to sort installed files in categories, and makes it
possible to install a subset of the files using the '--tags' argument to
'meson install'. This is typically used by distributions to split the
runtime, development and documentation files into separate packages.
By default, meson tries to guess the correct tag for installed files,
but can't always do so properly. Mark the install targets that meson
can't guess with the correct install_tag.
As the feature has been introduced in meson 0.60, bump the minimum meson
version. The latest LTS release of all major distributions that
libcamera currently targets ship a recent enough meson version.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
We make a few small improvements to the code:
* The arrayToSet method is prevented from overwriting the end of the
array if there are too many values in the input table. If you supply
a table, it will force you to put the correct number of elements in
it.
* The arrayToSet and setStrength member functions are turned into
static functions. (There may be a different public setStrength
member function in future.)
* When no tables at all are given, the configuration is flagged as
being disabled, so that we can avoid copying tables full of zeroes
around. As a consequence, the pipeline handler too will disable this
hardware block rather than run it needlessly. (Note that the tuning
tool will put in a completely empty "rpi.cac" block if no CAC tuning
images are supplied, benefiting from this behaviour.)
* The initialise member function is removed as it does nothing.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The recent change where time-filtering is done before sorting out the
digital gain means that the target exposure without digital gain is no
longer set, breaking the 'AeLocked' calculation.
We can use the regular (full) target exposure instead.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Fixes: 84b6327789 ("ipa: rpi: agc: Filter exposures before dealing with digital gain")
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Label draft controls and properties through the "draft" vendor tag
and deprecate the existing "draft: true" mechanism. This uses the new
vendor tags mechanism to place draft controls in the same
libcamera::controls::draft namespace and provide a defined control id
range for these controls. This requires moving all draft controls from
control_ids.yaml to control_ids_draft.yaml.
One breaking change in this commit is that draft control ids also move
to the libcamera::controls::draft namespace from the existing
libcamera::controls namespace. This is desirable to avoid API breakages
when adding new libcamera controls. So, for example, the use of
controls::NOISE_REDUCTION_MODE will need to be replaced with
controls::draft::NOISE_REDUCTION_MODE.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The algorithm computes R/G and B/G colour ratio statistics which we
should not allow to go to zero because there is clearly no gain you
could apply to R or B to equalise them. Instead flag such regions as
having "insufficient data" in the normal manner.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
We need to be able to do things like enable/disable AGC for all the
channels, so most of the AGC controls are updated to be applied to all
channels. There are a couple of exceptions, such as setting explicit
shutter/gain values, which apply only to channel 0.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
AWB writes this out during prepare, so we may as well read it in AGC
prepare as well. Reading it in process is wrong on the PiSP platform
because process runs before prepare, so the AWB status won't be there
(on vc4 it made no difference).
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Since noise control handling differs between the VC4 and PiSP IPAs,
move the current denoise control handler from ipa base into the vc4 IPA
derived class.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
"Fast desaturation" is a technique that can help the AGC algorithm to
desaturate images more quickly when they are very
over-exposed. However, it uses digital gain to do this which can
confuse our HDR techniques. Therefore make it optional.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This was being re-read in order to determine what LSC gains had been
applied. We can just retrieve these numbers from the prevAsyncResults_
instead.
This will also enable other future algorithms to manipulate the LSC
tables in the alsc.status, without it breaking the core ALSC algorithm
here.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
We can perform some of the local contrast adjustment using global
gains in the LSC table. We can vary the amount of gain according to
the measured brightness of that image region.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Whenever the AGC active channels are changed, start with the first
channel listed. This allows applications to rely on a particular channel
being generated first. For example, multi-exposure HDR always wants the
short channel first.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The code was inadvertently overwriting the caller's StatisticsPtr,
meaning that subsequent algorithms would get the wrong image
statistics when AGC channels changed.
This could be fix using std::ref, though I find the C-style pointer
fix easier to understand!
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Some use cases may require stronger, or different, denosie settings to
others. For example, the way frames are accumulated during single
exposure HDR means that we may want stronger denoise.
This commit adds such support for different configurations that can be
defined in the tuning file.
Older tuning files, or files where there is only a single
configuration, load only the "normal" denoise configuration.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The enableCe() function enables or disables adaptive contrast
enhancement and the restoreCe() function sets it back to its normal
state (which is what was read from the tuning file).
In future, algorithms like HDR might want to take over tonemapping
functions, so any dynamic behaviour here would upset them.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add a small "stable region" parameter (defaulting to 2%) within which
the AGC will not adjust the exposure it requests. It allows
applications to configure the AGC to avoid continual micro-adjustments
of exposure values if they are somehow sensitive to it.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add new CAC, HDR, Saturation and Tonemapping algorithms.
Add a new Denoise algorithm that handles spatial/temporal/colour denoise
through one interface. With this change, the old SDN algorithm is now
considered deprecated and a warning message will be displayed if it is
enabled.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Prepare the AWB algorithm to support the PiSP hardware. The key change
is to factor in the LS correction in the AWB zone statistics. This is
different from VC4 where the LS correction happens before statistics
gathering.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Prepare the ALSC algorithm to support the PiSP hardware. The key change
is to avoid factoring out the WB correction in the AWB zone statistics.
Add the ALSC correction to the global metadata so that AWB can use it to
factor the gains back in for the AWB calculations.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add a new boolean field (statsInline) to Controller::HardwareConfigMap.
This field indicates where the statistics are generated in the hardware
ISP pipeline. For statsInline == true, statistics are generated before
the frame is processed (e.g. the PiSP case), and statsInline == false
indicates statistics are generated after the frame is processed (e.g.
the VC4 case).
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add the PiSP Frontend and Backend SharedMemObject file descriptors to
the ipa::init() call. This will allow the pipeline handler to pass these
objects to the IPA after construction.
Add a flag to indicate if buffer swaps are needed when starting the
ISP for the stitch block.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
