The RkISP1 can capture raw frames by bypassing the ISP. In that mode,
the ISP will not produce statistics nor consume parameters. Most
algorithms will thus be unable to run, with one exception: the AGC will
still be able to configure the sensor exposure time and analog gain in
manual mode.
To prepare for this, add the ability to disable algorithms for the
duration of the capture session based on the camera configuration.
Individual algorithms report whether they support raw formats at
construction time, and the IPA module disables algorithms in configure()
based on the stream configurations.
Disabled algorithms are skipped during the capture session in the
processStatsBuffer() operation. As the ISP doesn't produce statistics,
don't try to access the stats buffer. There is no need for similar logic
in fillParamsBuffer() as that operation won't be called for raw capture.
All algorithms report not supporting raw capture by default. Raw support
in AGC will be added separately.
The feature is implemented in the RkISP1 module without any support from
libipa at this point to avoid designing a generic API based on a single
user. This may be changed in the future.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Currently, the Android HAL does not work on rkisp1-based devices because
required FrameDurationLimits metadata is missing from the IPA
implementation.
This change sets FrameDurationLimits for rkisp1 based on the existing
ipu3 implementation, using the sensor's reported range of vertical
blanking intervals with the minimum reported horizontal blanking
interval.
Signed-off-by: Nicholas Roth <nicholas@rothemail.net>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The RkISP1 pipeline and IPA module allows for the CameraSensorInfo to be
empty, probably to accommodate some sensor used in a test platform that
does not provide the mandatory libcamera requirements.
As the \todo item in the IPA reports, there is a possibility that the
received CameraSensorInfo is empty and it should be checked before
accessing it, but currently such requirement is not enforced in the
code.
This allows to assume all the test platforms in use have now
successfully moved their sensor driver to comply with the minimum
requirements and provide a populated CameraSensorInfo to the IPA.
As the safety check is not enforced, and as we don't want to allow
faulty sensors to send empty CameraSensorInfo to the IPA, remove the
\todo item in the IPA and fail hard in the pipeline handler if the
sensor does not comply with libcamera requirements.
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The RkISP1 implementation of IPA::configure() still uses the legacy
interface where sensor controls (and eventually lens controls) are
passed from the pipeline handler to the IPA in a map.
Since the introduction of mojom-based IPA interface definition, it is
possible to define custom data types and use them in the interface
definition between the pipeline handler and the IPA.
Align the RkISP1 IPA::configure() implementation with the one in the
IPU3 IPA module by using a custom data type instead of relying on a map
to pass controls to the IPA.
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The CameraSensor class validates that the sensor driver in use supports
the controls required for IPA modules to work correctly.
For in-tree IPA modules, whose pipeline handlers already use
CameraSensor there's no need to validate such controls again.
Remove controls validation from the IPU3 and RkISP1 IPA modules and rely
on CameraSensor doing that at initialization time.
The list of mandatory controls is expanded to add V4L2_CID_ANALOGUE_GAIN
without which IPA modules cannot function.
The new requirement only applies to RAW sensors, platforms like UVC and
Simple are not impacted by this change.
While at it, expand the sensor driver requirements documentation to
include V4L2_ANALOGUE_GAIN in the list of mandatory controls a sensor
driver has to support.
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The BufferMask enum provides a way of identifying which stream a frame buffer
belongs to. This enum is defined in the raspberrypi.mojom interface file.
However, the IPA does not need these enum definitions to mmap buffers that it
uses.
Move this enum out of the raspberrypi.mojom interface file and put it into
the RPi namespace visible only to the pipeline handler. This removes the
need to include the auto-generated IPA interface header in the RPi::Stream
definition.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
No existing Algorithm used the base pause(), resume() functions
or the paused_ flag, nor is there a need for a generic pause API.
Remove these. The AGC and AWB algorithms now have methods named
disableAuto(), enableAuto() which better describe their functionality.
Signed-off-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Add coefficients sets in the YAML tuning file to allow using different set
depending of the image color temperature (provided by AWB algorithm).
During processing, LSC algorithm computes coefficients by doing a linear
interpolation between the two closer set.
Signed-off-by: Florian Sylvestre <fsylvestre@baylibre.com>
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The limits for the shutter speed and analogue gain are stored in
IPASessionConfiguration::agc. While they're related to the AGC, they are
properties of the sensor, and are stored in the session configuration by
the IPA module, not the AGC algorithm. Move them to the
IPASessionConfiguration::sensor structure where they belong.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reorder functions in the base ipa::Algorithm and its derived classes to
match the calling order: queueRequest(), prepare() and process(). This
makes the code flow easier to read. No functional change intended.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Previously we only did this when the system starts (on the first
switch_mode). Now we do it whenever the manual colour gains are
updated. To facilitate this, this R/B vs. colour temperature inverse
functions are stored persistently in the AwbConfig.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
pwl.cpp uses abs() instead of std::abs(), which causes unexpected
behavior in the Clang compiler used for Android. Replace with
C++-standard absolute value function std::abs(), which supports
double-precision absolute values in a standard way.
Signed-off-by: Nicholas Roth <nicholas@rothemail.net>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The raspberrypi IPA is missing thread-safety annotations, which breaks
the build.
Add required thread-safety annotations.
../src/ipa/raspberrypi/controller/metadata.h:108:31: error: mutex
'mutex_' is still held at the end of function [-Werror,-Wthread-safety-analysis]
void lock() { mutex_.lock(); }
^
../src/ipa/raspberrypi/controller/metadata.h:108:23: note: mutex
acquired here
void lock() { mutex_.lock(); }
^
../src/ipa/raspberrypi/controller/metadata.h:109:25: error: releasing
mutex 'mutex_' that was not held [-Werror,-Wthread-safety-analysis]
void unlock() { mutex_.unlock(); }
^
Signed-off-by: Nicholas Roth <nicholas@rothemail.net>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Fill the frame metadata in the AGC and AWB algorithm's prepare()
function. This removes the need to fill metadata manually in the IPA
module's processStatsBuffer() function.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Extend the Algorithm::process() function with a metadata control list,
to be filled by individual algorithms with frame metadata. Update the
rkisp1 and ipu3 IPA modules accordingly, and drop the dead code in the
IPARkISP1::prepareMetadata() function while at it.
This only creates the infrastructure, filling metadata in individual
algorithms will be handled separately.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
The recent change to the naming scheme for the camera tuning files failed to
rename ov9281.json, causing the sensor to fail to load. Since this is a mono
sensor module, rename this file to ov9281_mono.json.
Fixes: 820607b195: ("pipeline: raspberrypi: Update naming convention for tuning files")
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
There's no need to print the exposure and gain control ranges as an Info
message. Downgrade it to Debug. While at it, print the ranges using the
"[min, max]" syntax.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
For the purpose of testing serializing/deserializing Flags in function
parameters, add an enum class TestFlags and Flags<TestFlags> to some
function parameters, both for input and output and Signals.
While at it, update the ipa_interface_test.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
For the purpose of testing serializing/deserializing enums in function
parameters, add IPAOperationCode to the parameter list of init().
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Rename CamHelper::getVBlanking to CamHelper::getBlanking, and update the
calculations in that function to return both horizontal and vertical
blanking values for a given exposure time and frame duration limits. The
calculations are setup such that vertical blanking is extended to the
maximum allowable value, and any remainder gets put into horizontal
blanking.
The calculated horizontal blanking value is now returned to the pipeline
handler to pass into DelayedControls to program into the sensor.
Update the IPA to now specify the maximum frame duration from the
maximum horizontal + vertical blanking values provided by the sensor
mode. Additionally, the IPA now uses the frame specific horizontal
blanking value (as returned by DelayedControls) in all instances.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Add CamHelper::hblankToLineLength() to calculate the line length
duration from the horizontal blanking (in pixels) value.
Add CamHelper::lineLengthToHblank() to calculate the horizontal blanking
(in pixels) value from the line length duration.
Add CamHelper::lineLengthPckToDuration() to calculate the line length
duration from the line length in pixels.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Update CamHelper::getDelays() to return the sensor HBLANK delay. The
HBLANK delay is set to the same value as VBLANK delay for all sensors in
the Raspberry Pi IPA.
Return the HBLANK gain delay from the IPA to the pipeline handler, and
initialise DelayedControls to handle V4L2_CID_HBLANK with this delay
value.
As a drive-by, check that the V4L2_CID_HBLANK control is available when
calling IPARPi::configure().
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Update CamHelper::exposureLines() and CamHelper::exposure() to take a
line length duration parameter for use in the exposure calculations.
For now, only use the minimum line length for all the calculations to
match the existing IPA behavior.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Add fields for minimum and maximum line length duration to the
CameraMode structure. This replaces the existing lineLength field.
Any use of the existing lineLength field is replaced by the new
minLineLength field, as logically we always want to use the fastest
sensor readout by default.
As a drive-by cosmetic change, split all fields in the CameraMode
structure into separate lines.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Add fields for minimum and maximum line length (in units of pixels) to
the IPACameraSensorInfo structure. This replaces the existing lineLength
field.
Update the ipu3, raspberrypi and rkisp1 IPAs to use
IPACameraSensorInfo::minLineLength instead of
IPACameraSensorInfo::lineLength, as logically we will always want to use
the fastest sensor readout by default.
Since the IPAs now use minLineLength for their calculations, set the
starting value of the V4L2_CID_HBLANK control to its minimum in
CameraSensor::init().
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
utils::defopt causes compilation issues on gcc 8.0.0 to gcc 8.3.0,
likely due to bug https://gcc.gnu.org/bugzilla/show_bug.cgi?id=86521
that was fixed in gcc 8.4.0. gcc 8.3.0 may be considered old (libcamera
requires gcc-8 or newer), but it is shipped by Debian 10 that has LTS
support until mid-2024.
As no workaround has been found to fix compilation on gcc 8.3.0 while
still retaining the functionality of utils::defopt, stop using it in the
RkISP1 IPA module.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Commit 09c1b081ba ("libcamera: controls: Generate and use fixed-sized
Span types") added explicit Span casts for fixed extent spans that were
required due to the ControlList::set() function that takes an
std::initializer_list not being able to infer a control size from
template arguments. This has now been fixed, so the casts are not needed
anymore. Drop them.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Christian Rauch <Rauch.Christian@gmx.de>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
The REGISTER_CAMERA_SENSOR_HELPER() macro defines a class type that
inherits from the CameraSensorHelperFactory class, and implements a
constructor and createInstance() function. Replace the code generation
through macro with the C++ equivalent, a class template, as done by the
Algorithm factory.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Xavier Roumegue <xavier.roumegue@oss.nxp.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Avoid naked pointer with memory allocation by returning a unique_ptr
from CameraSensorHelperFactory::createInstance(), in order to increase
memory allocation safety.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Xavier Roumegue <xavier.roumegue@oss.nxp.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
The CameraSensorHelperFactory registerType() and createInstance()
functions are called by the CameraSensorHelperFactory class only. Make
them private.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Xavier Roumegue <xavier.roumegue@oss.nxp.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
The IPAFrameContext documentation has a spurious Doxygen \struct
statement that is not needed, and uses a \struct instead of a \var in
another location. Fix both issues. This doesn't cause any change in the
generated documentation.
Note that the Doxygen output for IPAFrameContext is incorrect,
documentation is missing for all members. This is caused by using "."
instead of "::" in our documentation, which we currently do because the
correct syntax produces Doxygen warnings (and still incorrect output).
See https://github.com/doxygen/doxygen/issues/9343.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Append "_mono" to the sensor name when generating the tuning filename for
monochrome sensor variants. So the new naming convention is as follows:
<sensor_name>.json - Standard colour sensor variant
<sensor_name>_mono.json - Monochrome sensor variant
Rename the existing imx296.json file to imx296_mono.json as this tuning file
is based on the monochrome variant of the IMX296.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The introduction of the FCQueue in the IPU3 inadvertently introduced a
bug which cleared the initialisation of the session configuration
immediately after some parameters had been set.
Furthermore, it cleared and never re-initialised the sensor line
duration property, which was previously only set during the call to
init().
Move the clearing of the contexts from the updateSessionConfiguration()
call to the earliest opportunity in configure(), and immediately
re-initialise the sensor parameters.
Bug: https://bugs.libcamera.org/show_bug.cgi?id=151
Fixes: 85c5c47325 ("ipa: ipu3: Use the FCQueue")
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Multiple algorithms have an initialized_ flag that they set to true at
the end of the init() function, and check at the beginning of prepare()
to skip preparation. This serves no real purpose, as the flag can only
be false if init() fails, in which case the IPA module initialization as
a whole will fail.
Drop the initialized_ flags.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The red and blue gains are computed by dividing the green mean by the
red and blue means respectively. An offset of 1 is added to the dividers
to avoid divisions by zero. This introduces a bias in the gain values.
Fix it by clamping the divisors to a minimum of 1.0 instead of adding an
offset.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
