The rkisp1 driver supports two formats for the ISP parameters buffer,
the legacy fixed format and the new extensible format. In preparation of
support for the new format, pass the parameters buffer format from the
pipeline handler to the IPA module and store it.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The frame context is used to store data used for processing that frame.
It is later used to either act as input for other algorithms or to fill
the metadata. For the colour temperature this is not needed, as the
meatadata shall not contain the value that was active when the image was
processed, but the value that was calculated based on the statistics for
that image. This is no functional change.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The AGC active state and frame context both contain a variable named
maxShutterSpeed. The variable is used to limit the maximum shutter speed
when computing the exposure time and gains, but stores the maximum frame
duration, not clamped by the sensor's maximum shutter speed. Rename it
to maxFrameDuration.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Plumb controls for setting metering mode, exposure mode, constraint
mode, and frame duration limits. Also report them as available controls,
as well as in metadata.
While at it, add the missing #include for tuple, as a std::tie is used.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add a gamma algorithm for the rkisp1. It defaults to a gamma of 2.2 which
closely resembles sRGB. No seperate sRGB mode was implemented because the pwl
that models the gamma curve is so coarse that there is basically no difference
between srgb and gamma=2.2. The default can be overridden within the tuning
file or set at runtime using the gamma control.
The gamma algorithm is not enabled by default. This will be done in future
tuning file updates.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Source files in libcamera start by a comment block header, which
includes the file name and a one-line description of the file contents.
While the latter is useful to get a quick overview of the file contents
at a glance, the former is mostly a source of inconvenience. The name in
the comments can easily get out of sync with the file name when files
are renamed, and copy & paste during development have often lead to
incorrect names being used to start with.
Readers of the source code are expected to know which file they're
looking it. Drop the file name from the header comment block.
The change was generated with the following script:
----------------------------------------
dirs="include/libcamera src test utils"
declare -rA patterns=(
['c']=' \* '
['cpp']=' \* '
['h']=' \* '
['py']='# '
['sh']='# '
)
for ext in ${!patterns[@]} ; do
files=$(for dir in $dirs ; do find $dir -name "*.${ext}" ; done)
pattern=${patterns[${ext}]}
for file in $files ; do
name=$(basename ${file})
sed -i "s/^\(${pattern}\)${name} - /\1/" "$file"
done
done
----------------------------------------
This misses several files that are out of sync with the comment block
header. Those will be addressed separately and manually.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Now that we have a AgcMeanLuminance class that centralises our AEGC
algorithm, derive the RkISP1's Agc class from it and plumb in the
necessary framework to enable it to be used. For simplicities sake
this commit switches the algorithm to use the derived class, but
does not remove the bespoke functions at this time.
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Versions of the ISP differ in the processing blocks they include, as
well as in the implementation of some of those blocks. In particular,
they have different numbers of histogram bins oe AE statistics cells.
The algorithms take these differences into account by checking the ISP
version reported by the driver.
These checks are currently scattered in multiple places. Centralize them
in the IPARkISP1::init() function, and store the version-dependent
hardware parameters in the IPA context, accessible by all algorithms.
While at it, drop the IPASessionConfiguration::hw member that stores the
revision number, unused by the algorithms. It can be added back laer to
the IPAHwSettings structure if needed.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
The RkISP1 can capture raw frames by bypassing the ISP. In that mode,
the ISP will not produce statistics nor consume parameters. Most
algorithms will thus be unable to run, with one exception: the AGC will
still be able to configure the sensor exposure time and analog gain in
manual mode.
To prepare for this, add the ability to disable algorithms for the
duration of the capture session based on the camera configuration.
Individual algorithms report whether they support raw formats at
construction time, and the IPA module disables algorithms in configure()
based on the stream configurations.
Disabled algorithms are skipped during the capture session in the
processStatsBuffer() operation. As the ISP doesn't produce statistics,
don't try to access the stats buffer. There is no need for similar logic
in fillParamsBuffer() as that operation won't be called for raw capture.
All algorithms report not supporting raw capture by default. Raw support
in AGC will be added separately.
The feature is implemented in the RkISP1 module without any support from
libipa at this point to avoid designing a generic API based on a single
user. This may be changed in the future.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
The limits for the shutter speed and analogue gain are stored in
IPASessionConfiguration::agc. While they're related to the AGC, they are
properties of the sensor, and are stored in the session configuration by
the IPA module, not the AGC algorithm. Move them to the
IPASessionConfiguration::sensor structure where they belong.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
When the RGB means are too small, gains and color temperature can't be
meaningfully calculated. Freeze the AWB in that case, using the
previously calculated values.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
The color temperature doesn't need floating point precision, and is
calculated by Awb::estimateCCT() as an unsigned integer. Store it with
the same data type in the frame context.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Rework the algorithm's usage of the active state, to store the value of
controls for the last queued request in the queueRequest() function, and
store a copy of the values in the corresponding frame context. The
latter is used in the prepare() function to populate the ISP parameters
with values corresponding to the right frame.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Rework the algorithm's usage of the active state, to store the value of
controls for the last queued request in the queueRequest() function, and
store a copy of the values in the corresponding frame context. The
latter is used in the prepare() function to populate the ISP parameters
with values corresponding to the right frame.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Rework the algorithm's usage of the active state, to store the value of
controls for the last queued request in the queueRequest() function, and
store a copy of the values in the corresponding frame context. The
latter is used in the prepare() function to populate the ISP parameters
with values corresponding to the right frame.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Rework the algorithm's usage of the active state and frame context to
store data in the right place.
The active state stores two distinct categories of information:
- The consolidated value of all algorithm controls. Requests passed to
the queueRequest() function store values for controls that the
application wants to modify for that particular frame, and the
queueRequest() function updates the active state with those values.
The active state thus contains a consolidated view of the value of all
controls handled by the algorithm.
- The value of parameters computed by the algorithm when running in auto
mode. Algorithms running in auto mode compute new parameters every
time statistics buffers are received (either synchronously, or
possibly in a background thread). The latest computed value of those
parameters is stored in the active state in the process() function.
The frame context also stores two categories of information:
- The value of the controls to be applied to the frame. These values are
typically set in the queueRequest() function, from the consolidated
control values stored in the active state. The frame context thus
stores values for all controls related to the algorithm, not limited
to the controls specified in the corresponding request, but
consolidated from all requests that have been queued so far.
For controls that can be specified manually or computed by the
algorithm depending on the operation mode (such as the colour gains),
the control value will be stored in the frame context in
queueRequest() only when operating in manual mode. When operating in
auto mode, the values are computed by the algorithm and stored in the
frame context in prepare(), just before being stored in the ISP
parameters buffer.
The queueRequest() function can also store ancillary data in the frame
context, such as flags to indicate if (and what) control values have
changed compared to the previous request.
- Status information computed by the algorithm for a frame. For
instance, the colour temperature estimated by the algorithm from ISP
statistics calculated on a frame is stored in the frame context for
that frame in the process() function.
The active state and frame context thus both contain identical members
for most control values, but store values that have a different meaning.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Rework the algorithm's usage of the active state to store the value of
controls for the last queued request in the queueRequest() function, and
store a copy of the values in the corresponding frame context.
The frame context is used in the prepare() function to populate the ISP
parameters with values corresponding to the right frame.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Inherit from the base FrameContext class in the RkISP1 IPAFrameContext.
As the IPAFrameContext is currently unused, this change is a no-op, but
it prepares the RkISP1 IPA module for frame context queue support.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
The RkISP1 IPA module creates a single instance of its IPAFrameContext
structure, effectively using it more as an active state than a per-frame
context. To prepare for the introduction of a real per-frame context,
move all the members of the IPAFrameContext structure to a new
IPAActiveState structure. The IPAFrameContext becomes effectively
unused at runtime, and will be populated back with per-frame data after
converting the RkISP1 IPA module to using a frame context queue.
The IPAActiveState structure will slowly morph into a different entity
as individual algorithm get later ported to the frame context API.
While at it, fix a typo in the documentation of the
Agc::computeExposure() function that incorrectly refers to the frame
context instead of the global context.
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Add support for manually controlling the color gains on the rkisp1 IPA.
To that end, add and plumb the AwbEnable and ColourGains controls. As
per-frame controls aren't supported yet in the rkisp1 IPA, simply apply
and perform checks on the controls immediately.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Denoise and Sharpness filters will be applied by RkISP1 during the
demosaicing step. The denoise filter is responsible for removing noise
from the image, while the sharpness filter will enhance its acutance.
Add filter algorithm with denoise and sharpness values based on user
controls.
Signed-off-by: Florian Sylvestre <fsylvestre@baylibre.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The Lens Shading Correction algorithm applies multipliers to all pixels
to compensate for the lens shading effect. The coefficients are
specified in a downscaled table in the YAML tuning file.
Signed-off-by: Florian Sylvestre <fsylvestre@baylibre.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The RkISP1 ISP calculates a mean value for Y, Cr and Cb at each frame.
There is a RGB mode which could theoretically give us the values for R,
G and B directly, but it seems to be failing right now.
Convert those values into R, G and B and estimate the gain to apply in a
grey world.
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Tested-by: Peter Griffin <peter.griffin@linaro.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
As for the IPU3, we can estimate the histogram of the luminance. The
RkISP1 can estimate multiple ones, the R, G and B ones, the Y only one
and a combination of RGB. The one we are interested by in AGC is the Y
histogram.
Use the hardware revision to determine the number of bins of the
produced histogram.
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Tested-by: Peter Griffin <peter.griffin@linaro.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Introduce a frameCount variable in the IPAFrameContext which increments
each time a request is queued. It is reset at configure call, when the
camera is started.
This will allow the frameCount to be used by other algorithms, without
having to keep multiple private frame counters.
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Now that we have IPAContext and Algorithm, we can implement a simple AGC
based on the IPU3 one. It is very similar, except that there is no
histogram used for an inter quantile mean. The RkISP1 is returning a 5x5
array (for V10) of luminance means. Estimating the relative luminance is
thus a simple mean of all the blocks already calculated by the ISP.
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Before using any algorithm, we want the IPAContext to be ready for
those. Introduce the IPAContext following the existing design from
IPA::IPU3. Each algorithm will then introduce the needed fields.
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Laurent Pinchart
Stefan Klug
Paul Elder
Daniel Scally
Kieran Bingham
Florian Sylvestre
Jean-Michel Hautbois