Handle the RKISP1_V_IMX8MP version in the rkisp1 IPA. This enables basic
support of the i.MX8MP, excluding the processing blocks specific to the
new ISP version. Those will be supported later.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
The histogram weights are initialized to hardcoded 1's for each
histogram grid cell. The code uses the wrong variable for the grid size,
resulting in some weights having a 0 value. Fix it.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Versions of the ISP differ in the processing blocks they include, as
well as in the implementation of some of those blocks. In particular,
they have different numbers of histogram bins oe AE statistics cells.
The algorithms take these differences into account by checking the ISP
version reported by the driver.
These checks are currently scattered in multiple places. Centralize them
in the IPARkISP1::init() function, and store the version-dependent
hardware parameters in the IPA context, accessible by all algorithms.
While at it, drop the IPASessionConfiguration::hw member that stores the
revision number, unused by the algorithms. It can be added back laer to
the IPAHwSettings structure if needed.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
The RkISP1 statistics structure contains multiple arrays whose length
varies depending on the hardware revision. Accessing those arrays is
error-prone, wrap them in spans at the top level to reduce risks of
out-of-bound accesses.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
The CameraSensorHelperFactory::createInstance() function overrides a
virtual function from CameraSensorHelperFactoryBase. The function
declaration doesn't mark it with the override keyword. This could cause
issues in the future if the base class' function changes, as the
compiler will not issue any warning in that case. Fix it.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
All the Raspberry Pi official camera tuning files are updated for
HDR. As stated previously, there is no mechanism in the hardware for
combining images so all this does is enable multi-channel AGC to
produce short and long exposure frames. It will be up to the
application to deal with them.
The changes are identical in every tuning file.
1. The existing AGC tuning is duplicated twice so that we have 3 AGC
channels.
2. The first is left alone (the default AGC channel), the second is
tweaked to under-expose significantly (ev -3) and the final one is
tweaked to over-exposure slightly (ev +0.5)
3. Control parameters are provided to the "rpi.hdr" algorithm to
associate these AGC channels correctly with the HDR modes.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Sufficient plumbing is added so that the HDR mode control can be used
to engage HDR modes on platforms that support them. On the vc4
platform, this allows multi-channel AGC to run, though there is no
image merging.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Implement the StatsOutputEnable control for the VC4 IPA. When set,
this outputs the ISP statistics as a uint8_t span through the Bcm2835StatsOutput
metadata control.
To get this working, IpaBase::libcameraMetadata_ is moved from a private
to a protected member variable. This makes it accessable to the VC4
derived IPA class.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This allows the IPA to get reasonable default colour gains before AWB
has run. This is particularly important on the PiSP platform where
these numbers are helpful in programming the Front End statistics
block in advance.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This allows the IPA to discover the correct black level values even
before any frames have been processed. This is important on the PiSP
platform where the front end black level blocks must be programmed in
advance.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Advertise hardware constraints on the pixel processing rate through the
Controller::HardwareConfig structure. When calculating the minimum line
length during a configure() operation, ensure that we don't exceed this
constraint.
If we do exceed the hardware constraints, increase the modes's minimum
line length so the pixel processing rate falls below the hardware limit.
If this is not possible, throw a loud error message in the logs.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Meson uses tags to sort installed files in categories, and makes it
possible to install a subset of the files using the '--tags' argument to
'meson install'. This is typically used by distributions to split the
runtime, development and documentation files into separate packages.
By default, meson tries to guess the correct tag for installed files,
but can't always do so properly. Mark the install targets that meson
can't guess with the correct install_tag.
As the feature has been introduced in meson 0.60, bump the minimum meson
version. The latest LTS release of all major distributions that
libcamera currently targets ship a recent enough meson version.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
We make a few small improvements to the code:
* The arrayToSet method is prevented from overwriting the end of the
array if there are too many values in the input table. If you supply
a table, it will force you to put the correct number of elements in
it.
* The arrayToSet and setStrength member functions are turned into
static functions. (There may be a different public setStrength
member function in future.)
* When no tables at all are given, the configuration is flagged as
being disabled, so that we can avoid copying tables full of zeroes
around. As a consequence, the pipeline handler too will disable this
hardware block rather than run it needlessly. (Note that the tuning
tool will put in a completely empty "rpi.cac" block if no CAC tuning
images are supplied, benefiting from this behaviour.)
* The initialise member function is removed as it does nothing.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The recent change where time-filtering is done before sorting out the
digital gain means that the target exposure without digital gain is no
longer set, breaking the 'AeLocked' calculation.
We can use the regular (full) target exposure instead.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Fixes: 84b6327789 ("ipa: rpi: agc: Filter exposures before dealing with digital gain")
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Label draft controls and properties through the "draft" vendor tag
and deprecate the existing "draft: true" mechanism. This uses the new
vendor tags mechanism to place draft controls in the same
libcamera::controls::draft namespace and provide a defined control id
range for these controls. This requires moving all draft controls from
control_ids.yaml to control_ids_draft.yaml.
One breaking change in this commit is that draft control ids also move
to the libcamera::controls::draft namespace from the existing
libcamera::controls namespace. This is desirable to avoid API breakages
when adding new libcamera controls. So, for example, the use of
controls::NOISE_REDUCTION_MODE will need to be replaced with
controls::draft::NOISE_REDUCTION_MODE.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The algorithm computes R/G and B/G colour ratio statistics which we
should not allow to go to zero because there is clearly no gain you
could apply to R or B to equalise them. Instead flag such regions as
having "insufficient data" in the normal manner.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
We need to be able to do things like enable/disable AGC for all the
channels, so most of the AGC controls are updated to be applied to all
channels. There are a couple of exceptions, such as setting explicit
shutter/gain values, which apply only to channel 0.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
AWB writes this out during prepare, so we may as well read it in AGC
prepare as well. Reading it in process is wrong on the PiSP platform
because process runs before prepare, so the AWB status won't be there
(on vc4 it made no difference).
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Since noise control handling differs between the VC4 and PiSP IPAs,
move the current denoise control handler from ipa base into the vc4 IPA
derived class.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
"Fast desaturation" is a technique that can help the AGC algorithm to
desaturate images more quickly when they are very
over-exposed. However, it uses digital gain to do this which can
confuse our HDR techniques. Therefore make it optional.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This was being re-read in order to determine what LSC gains had been
applied. We can just retrieve these numbers from the prevAsyncResults_
instead.
This will also enable other future algorithms to manipulate the LSC
tables in the alsc.status, without it breaking the core ALSC algorithm
here.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
We can perform some of the local contrast adjustment using global
gains in the LSC table. We can vary the amount of gain according to
the measured brightness of that image region.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Whenever the AGC active channels are changed, start with the first
channel listed. This allows applications to rely on a particular channel
being generated first. For example, multi-exposure HDR always wants the
short channel first.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The code was inadvertently overwriting the caller's StatisticsPtr,
meaning that subsequent algorithms would get the wrong image
statistics when AGC channels changed.
This could be fix using std::ref, though I find the C-style pointer
fix easier to understand!
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Some use cases may require stronger, or different, denosie settings to
others. For example, the way frames are accumulated during single
exposure HDR means that we may want stronger denoise.
This commit adds such support for different configurations that can be
defined in the tuning file.
Older tuning files, or files where there is only a single
configuration, load only the "normal" denoise configuration.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The enableCe() function enables or disables adaptive contrast
enhancement and the restoreCe() function sets it back to its normal
state (which is what was read from the tuning file).
In future, algorithms like HDR might want to take over tonemapping
functions, so any dynamic behaviour here would upset them.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add a small "stable region" parameter (defaulting to 2%) within which
the AGC will not adjust the exposure it requests. It allows
applications to configure the AGC to avoid continual micro-adjustments
of exposure values if they are somehow sensitive to it.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add new CAC, HDR, Saturation and Tonemapping algorithms.
Add a new Denoise algorithm that handles spatial/temporal/colour denoise
through one interface. With this change, the old SDN algorithm is now
considered deprecated and a warning message will be displayed if it is
enabled.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Prepare the AWB algorithm to support the PiSP hardware. The key change
is to factor in the LS correction in the AWB zone statistics. This is
different from VC4 where the LS correction happens before statistics
gathering.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Prepare the ALSC algorithm to support the PiSP hardware. The key change
is to avoid factoring out the WB correction in the AWB zone statistics.
Add the ALSC correction to the global metadata so that AWB can use it to
factor the gains back in for the AWB calculations.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add a new boolean field (statsInline) to Controller::HardwareConfigMap.
This field indicates where the statistics are generated in the hardware
ISP pipeline. For statsInline == true, statistics are generated before
the frame is processed (e.g. the PiSP case), and statsInline == false
indicates statistics are generated after the frame is processed (e.g.
the VC4 case).
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add the PiSP Frontend and Backend SharedMemObject file descriptors to
the ipa::init() call. This will allow the pipeline handler to pass these
objects to the IPA after construction.
Add a flag to indicate if buffer swaps are needed when starting the
ISP for the stitch block.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add a virtual IpaBase::platformStart() member function that is called
at the end of IpaBase::start(). For the IpaVc4 derived class, this
function does nothing, but will be used in the PiSP derived class to
reset internal state on startup.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
We avoid skipping the IPAs while frameCount_ is less than
dropFrameCount_, indicating that these frames will not be sent to the
application. This means that when these numbers are equal then this is
the first frame the application will get, so again, we must avoid
skipping the IPAs. Consequently the test here must avoid the case of
equality.
Fixes: 51533fecae ("ipa: rpi: Fix frame count logic when running algorithms")
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The frame counter test to determine if we run the IPA algorithms has a
logic bug where it treats dropFrameCount_ and mistrustCount_ as frame
numbers, not counts of frames (which it is). The implication is that
startup convergence and initial settings take one extra frame to apply.
Fix this.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
If the json file parsing failed due to a malformed file, the root
pointer would be null. This was not tested and caused a segfault when
trying to use the pointer to retrieve the version key.
Fix this by bailing out early if the parser returns a null pointer.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Whenever we run Agc::process(), we store the most recent total
exposure requested for each channel.
With these values we can apply the channel constraints after
time-filtering the requested total exposure, but before working out
how much digital gain is needed.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
A channel constraint is somewhat similar to the upper/lower bound
constraints that we use elsewhere, but these constraints apply between
multiple AGC channels. For example, it lets you say things like "don't
let the channel 1 total exposure be more than 8x that of channel 0",
and so on. By using both an upper and lower bound constraint, you
could fix one AGC channel always to be a fixed ratio of another.
Also read a vector of them (if present) when loading the tuning file.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
The switchMode, prepare and process methods are updated to implement
multi-channel AGC correctly:
* switchMode now invokes switchMode on all the channels (whether
active or not).
* prepare must find what channel the current frame is, and run on
behalf of that channel.
* process updates the most recent DeviceStatus and statistics for the
channel of the frame that has just arrived, but generates updated
values working through the active channels in round-robin fashion.
One minor detail in process is that we don't want to change the
DeviceStatus metadata of the current frame, so we now pass this to the
AgcChannel's process method, rather than letting it find the
DeviceStatus in the metadata.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
This commit does the basic reorganisation of the code in order to
implement multi-channel AGC. The main changes are:
* The previous Agc class (in agc.cpp) has become the AgcChannel class
in (agc_channel.cpp).
* A new Agc class is introduced which is a wrapper round a number of
AgcChannels.
* The basic plumbing from ipa_base.cpp to Agc is updated to include a
channel number. All the existing controls are hardwired to talk
directly to channel 0.
There are a couple of limitations which we expect to apply to
multi-channel AGC. We're not allowing different frame durations to be
applied to the channels, nor are we allowing separate metering
modes. To be fair, supporting these things is not impossible, but
there are reasons why it may be tricky so they remain "TBD" for now.
This patch only includes the basic reorganisation and plumbing. It
does not yet update the important methods (switchMode, prepare and
process) to implement multi-channel AGC properly. This will appear in
a subsequent commit. For now, these functions are hard-coded just to
use channel 0, thereby preserving the existing behaviour.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Add a new helper function Histogram::interBinMean() that essentially
replaces the existing Histogram::interQuantileMean() logic but working on
bins instead.
Rework the interQuantileMean() to call into interBinMean() with the
appropriate convertion from quatiles to bins.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Laurent Pinchart
Paul Elder
David Plowman
Lee Jackson
Jacopo Mondi
Naushir Patuck
Nick Hollinghurst