The imx708 must drop a single frame on startup - but only when in HDR
mode. Non-HDR modes do not need to drop frames. Fix the logic in
hideFramesModeSwitch() which currently unconditionally advertises to
drop one frame.
Unfortunately there is no clear way to tell if the sensor is in the HDR
mode. So for now, look the resolution and framerate to deduce this.
Additionally ensure we override hideFramesStartup() and return the same
number as hideFramesModeSwitch().
Bug: https://github.com/raspberrypi/libcamera-apps/issues/524
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In commit 0ee9339331, a default metering/exposure/constraint mode is
used if a control sets a mode that is not listed in the camera tuning
file.
Setting a default mode may be undesirable in these cases, so instead
keep the agc mode unchanged. This also matches the behaviour for other
IPA controls where no changes are made in error conditions.
Fixes: 0ee9339331 ("ipa: rpi: agc: Gracefully handle missing agc modes")
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
In addition to sensor controls, return lens controls from
IpaBase::configure() back to the pipeline handler. If there are lens
controls present in the ControlList, action them in the pipeline handler
as part of the configure routine.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Commit a3178dd039 ("ipa: rkisp1: agc: drop hard-coded analogue gain range")
removed both minimum and maximum limits for the analogue gain value.
However, as some sensors can potentially have a minimum gain lower than
1.0, restore the check for the minimum limit.
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Do not advertise colour related controls (i.e. [A]WB, colour saturation)
in the ControlInfoMap of available controls returned out to the
application.
Silently ignore these controls in the control handler in case applications
don't use the advertised ControlInfoMap to validate controls.
As a drive-by fix, don't advertise controls::ColourCorrectionMatrix in
the ControlInfoMap as it is not handled by the IPA.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
As the sensor's analogue gain range is known (read-out in
IPARkISP1::configure()), drop the limiting hard-coded range.
This enables better performance in low-light conditions for sensors with
a higher gain (e.g. the imx327).
Signed-off-by: Benjamin Bara <benjamin.bara@skidata.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When iterating over enabled pipelines and IPA modules, libcamera
descends into subdirectories in a recursive manner, which involves
nested loops. Both the outer and inner loops use the same loop variable
named 'pipeline'. As the outer loop uses the variable after descending
into the inner directory, it ends up using an incorrect value. Fix it by
moving all use of the variable before the subdir() call, and add a
comment that warns about the issue to avoid reintroducing it.
Fixes: e8526c0c2b ("ipa: meson: Allow nested IPA directory structures")
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Create a new PipelineHandlerBase class that handles general purpose
housekeeping duties for the Raspberry Pi pipeline handler. Code the
implementation of new class is essentially pulled from the existing
pipeline/rpi/vc4/raspberrypi.cpp file with a small amount of
refactoring.
Create a derived PipelineHandlerVc4 class from PipelineHandlerBase that
handles the VC4 pipeline specific tasks of the pipeline handler. Again,
code for this class implementation is taken from the existing
pipeline/rpi/vc4/raspberrypi.cpp with a small amount of
refactoring.
The goal of this change is to allow third parties to implement their own
pipeline handlers running on the Raspberry Pi without duplicating all of
the pipeline handler housekeeping tasks.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Create a new IpaBase class that handles general purpose housekeeping
duties for the Raspberry Pi IPA. The implementation of the new class
is essentially pulled from the existing ipa/rpi/vc4/raspberrypi.cpp file
with a minimal amount of refactoring.
Create a derived IpaVc4 class from IpaBase that handles the VC4 pipeline
specific tasks of the IPA. Again, code for this class implementation is
taken from the existing ipa/rpi/vc4/raspberrypi.cpp with a
minimal amount of refactoring.
The goal of this change is to allow third parties to implement their own
IPA running on the Raspberry Pi without duplicating all of the IPA
housekeeping tasks.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Restructure the IPA mojom interface to be more consistent in the use
of the API. Function parameters are now grouped into *Params structures
and results are now returned in *Results structures.
The following pipeline -> IPA interfaces have been removed:
signalQueueRequest(libcamera.ControlList controls);
signalIspPrepare(ISPConfig data);
signalStatReady(uint32 bufferId, uint32 ipaContext);
and replaced with:
prepareIsp(PrepareParams params);
processStats(ProcessParams params);
signalQueueRequest() is now encompassed within prepareIsp().
The following IPA -> pipeline interfaces have been removed:
runIsp(uint32 bufferId);
embeddedComplete(uint32 bufferId);
statsMetadataComplete(uint32 bufferId, libcamera.ControlList controls);
and replaced with the following async calls:
prepareIspComplete(BufferIds buffers);
processStatsComplete(BufferIds buffers);
metadataReady(libcamera.ControlList metadata);
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Split the Raspberry Pi pipeline handler and IPA source code into common
and VC4/BCM2835 specific file structures.
For the pipeline handler, the common code files now live in
src/libcamera/pipeline/rpi/common/
and the VC4-specific files in src/libcamera/pipeline/rpi/vc4/.
For the IPA, the common code files now live in
src/ipa/rpi/{cam_helper,controller}/
and the vc4 specific files in src/ipa/rpi/vc4/. With this change, the
camera tuning files are now installed under share/libcamera/ipa/rpi/vc4/.
To build the pipeline and IPA, the meson configuration options have now
changed from "raspberrypi" to "rpi/vc4":
meson setup build -Dipas=rpi/vc4 -Dpipelines=rpi/vc4
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The current IPA build files require a flat directory structure for the
IPAs. Modify the build files to remove this restriction and allow a
directory structure such as:
src/ipa
|- raspberrypi
|- common
|- cam_helpers
|- controller
|- vc4
|- rkisp1
|- ipu3
where each subdir (e.g. raspberrypi/common, raspberrypi/cam_helper) has
its own meson.build file. Such a directory structure will be introduced
for the Raspberry Pi IPA in a future commit.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Add to the RkISP1 data configuration files for the ov2685 and ov5695
sensor found on Google DRU "Scarlett" tablet.
The tuning files contain LSC tables extracted from the Google Chrome
Android HAL configuration files, available at
src/overlays/overlay-scarlet/media-libs/cros-camera-hal-configs-scarlet/files/IQ/
at revision "stabilize-14790.B".
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Clarify IPA's acronym by specifying what "IPA" stands for as part of the
ipa namespaces' "brief" doxygen-generated description. This allows
visitors to the docs to immediately have an idea of the purpose of the
IPA namespace at a glance. Because of the prevalence and importance of
the IPA namespace and functionality, the fact that it stands for "Image
Processing Algorithm" should be accessible to even casual perusers of
the docs.
Signed-off-by: Gabby George <gabbymg94@gmail.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Files opened internally in libcamera without the O_CLOEXEC file will
remain open upon a call to one of the exec(3) functions. As exec()
doesn't destroy local or global objects, this can lead to various side
effects. Avoid this by opening file descriptors with O_CLOEXEC for all
internal files.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Use the generalised focus statistics structure to compute the centre
window focus FoM value. This avoids needed to hard-code a specific
grid size.
Remove the focus reporting algorithm as the functionality is duplicated
by this bit of IPA code. Remove focus_status.h as it is no longer needed.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Remove any hard-coded assumptions about the target hardware platform
from the autofocus algorithm. Instead, use the "target" string provided
by the camera tuning config and generalised statistics structures to
determing parameters such as grid and region sizes.
Additionally, PDAF statistics are represented by a generalised region
statistics structure to be device agnostic.
These changes also require the autofocus algorithm to initialise
region weights on the first frame's prepare()/process() call rather
than during initialisation.
Signed-off-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Remove any hard-coded assumptions about the target hardware platform
from the AGC algorithm. Instead, use the "target" string provided by
the camera tuning config and generalised statistics structures to
determing parameters such as grid and region sizes.
This change replaces all hard-coded arrays with equivalent std::vector
types.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Generalise the contrast algorithm code by removing any hard-coded
assumptions about the target hardware platform. Instead, the algorithm
code creates a generic Pwl that gets returned to the IPA, where it gets
converted to the bcm2835 hardware specific lookup table.
As a drive-by, remove an unused mutex.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The algorithm uses the data type std::vector<std::array<double, 4>> to
represent the large sparse matrices that are XY (X, Y being the ALSC
grid size) high but with only 4 non-zero elements on each row.
Replace this slightly long type name by SparseArray<double>.
No functional changes.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The Array2D class is a very thin wrapper round std::vector that can be
used almost identically in the code, but it carries its 2D size with
it so that we aren't passing it around all the time.
All the std::vectors that were X * Y in size (X and Y being the ALSC
grid size) have been replaced. The sparse matrices that are XY * 4 in
size have not been as they are somewhat different, are used
differently, require more code changes, and actually make things more
confusing if everything looks like an Array2D but are not the same.
There should be no change in algorithm behaviour at all.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Remove any hard-coded assumptions about the target hardware platform
from the ALSC algorithm. Instead, use the "target" string provided by
the camera tuning config and generalised statistics structures to
determing parameters such as grid and region sizes.
The ALSC calculations use run-time allocated arrays/vectors on every
frame. Allocating these might add a non-trivial run-time penalty.
Replace these dynamic allocations with a set of reusable pre-allocated
vectors during the init phase.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add a new Controller::HardwareConfig structure that captures the
hardware statistics grid/histogram sizes and pipeline widths. This
ensures there is a single centralised places for these parameters.
Add a getHardwareConfig() helper function to retrieve these values for a
given hardware target.
Update the statistics populating routine in the IPA to use the values
from this structure instead of the hardcoded numbers.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The target string may be used by algorithms to determine the running
hardware target.
Store the target string provided by the camera tuning files in the
controller state. Add a getTarget() member function to retrieve this
string.
Validate the correct hardware target ("bcm2835") during the IPA
initialisation phase.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Make a copy of the CameraMode structure on a switch mode call. This
replaces the existing lastSensitivity_ field.
Limit the AGC gain calculations to the minimum value given by the
CameraMode structure. The maximum value remains unclipped as any gain
over the sensor maximum will be made up by digital gain.
Rename clipShutter to limitShutter for consistency, and have the latter
limit the shutter speed to both upper and lower bounds.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Use the new analogue gain and shutter speed limit fields in the ipa
code when reporting back the control value limits and calculating the
analogue gain code to use. This also replaces the now unused (and
removed) maxSensorGainCode_ field.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add fields in the CameraMode structure to capture the mode specific
limits for analogue gain and shutter speed. For convenience, also add
fields for minimum and maximum frame durations.
Populate these new fields when setting up the CameraMode structure.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When compiled with LTO (the default on Ubuntu), the global static
objects camHelpers and algorithms cause a crash in raspberrypi_ipa_proxy
at runtime as they're not allocated by the time the registration
routines execute.
This is a fairly crude fix which just converts the global static objects
into local static objects inside an equivalently named function.
Signed-off-by: Dave Jones <dave.jones@canonical.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Some updates to the tuning for the imx296 sensors.
For the colour variant:
* Minor change to the AWB curve, making things a little less green.
* Updated CCMs that reduce colour saturation to a more accurate level.
Thanks to Dr. Rolf Henkel for these measurements and calculations.
* Sharpening has been toned down quite a lot.
* rpi.focus algorithm added so that the focus measure can be accessed.
The sharpening and focus changes are applied to the mono version of
the sensor too as we expect similar characteristics.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The maxVal variable in the computeInitialY function needs to be a
uint64_t, otherwise the subsequent multiplications in the function
can overflow on relatively high resolution images (when the counts in
the regions go over 16 bits).
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The existing mechanism of setting a timeout value simply uses the
maximum possible frame length advertised by the sensor mode. This can be
problematic when, for example, the IMX477 sensor can use a frame length
of over 600 seconds. However, for typical usage the frame length will
never go over several 100s of milliseconds, making the timeout very
impractical.
Store a list of the last 10 frame length values requested by the AGC. On
startup, and at the end of every frame, take the maximum frame length
value from this list and return that to the pipeline handler through the
setCameraTimeoutValue() signal. This allows the timeout value to better
track the actual sensor usage.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add an explicit helper function setCameraTimeout() in the pipeline
handler to set the Unicam timeout value. This function is signalled from
the IPA to set up an appropriate timeout. This replaces the
maxSensorFrameLengthMs value parameter returned back from
IPARPi::start().
Adjust the timeout to be 5x the maximum frame duration reported by the
IPA.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The RkISP1 implementation of the LensShadingCorrection algorithm has been
made adaptive to the scene color temperature in commit 14c869c00f ("ipa:
rkisp1: Take into account color temperature during LSC algorithm").
The LSC algorithm interpolates the correction factors using the
table's reference color temperatures. When calculating the interpolation
coefficients, an unintended integer division makes both coefficient
zeros resulting in a completely black image.
Fix this by type casting to double one of the division operands.
Fixes: 14c869c00f ("ipa: rkisp1: Take into account color temperature during LSC algorithm")
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The VC4 ISP uses a pipeline bit-depth of 13-bits. The AGC algorithm needs to
know this bit-depth when computing the Y value for the image.
Instead of hardcoding the VC4 bit-depth in the AGC source code, normalise all
region sums to 16-bits when filling the Statistics structure. AWB and ALSC are
agnostic about pipeline depth, so do not need changing.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
