In the current code, decompand will only set a curve in the prepare
phase, which will only run after 1-2 frames pass through the FE. This
is fixed by adding an initialValues() member function to the decompand
algorithm, which will be called in the IPA before we start the hardware
streaming.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This patch integrates a new decompand algorithm that utilizes the PiSP
FE hardware block available on Raspberry Pi 5. The implementation
enables conversion of companded sensor data into linear format prior to
ISP processing.
Changes include:
- Implementation of decompand logic for controller and pipe interfaces
- Enabling decompand block by "rpi.decompand" in tuning.json
Signed-off-by: Sena Asotani <aso.fam429@gmail.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The CCM algorithm will now let an explicit colour matrix be set when
AWB is in manual mode.
We must handle any controls that can cause the AWB to be enabled or
disabled first, so that we know the AWB's state correctly when we come
to set the CCM.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
[Kieran: Remove duplicated Matrix3x3 from ccm.cpp]
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The CDN (colour denoise) deviation gets gradually reduced frame by
frame as TDN (temporal denoise) comes in and has more effect. CDN is
more harmful to image detail than TDN, so ramping it down in favour of
TDN is beneficial.
The tuning file parameters are chosen so that existing tuning files
don't have to be updated and will carry on working "mostly like they
did" (apart from the new back-off).
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Header comment blocks used to contain the file name. Considered as
useless information, the names have been removed, with the last ones
supposed to be dropped in commit d3bf27180e ("libcamera: Drop
remaining file names from header comment blocks"). A few have however
been forgotten, and more crept back since. Remove them.
While at it, fix one typo in a header comment block by replacing
'MaliC55 with Mali-C55', and add a missing blank line in
src/ipa/rpi/pisp/pisp.cpp.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Previously we let prepare() do the work by comparing the desired total
exposure against the shutter time and analogue gain. This can cause
the image to "wink" at high framerates because we may skip running
prepare() to get the new digital gain even when the delayed AGC status
(which came out of an earlier call to process()) shows that a change
was required.
Now we're taking explicit control of the digital gain by calculating
it ourselves so that we can output it in the standard AgcStatus
object. This means that whenever the delayed AGC status changes, we
have the correct digital gain to go with it.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Much of the time we use the term "analogue gain" where we really mean
the combined analogue and digital gain (because the digital gain will
make up whatever the analogue gain can't deliver).
This commit replaces the use of "analogue gain" by just "gain" in
places where we really mean the combined gain. There are a couple of
principle areas:
1. Where we previously talked about the "fixedAnalaogueGain"
(including setting it "manually") this is now just the "fixedGain"
(because it always encompassed both analogue and digital gain).
Along with this, the setfixedExposureTime/Gain functions no longer
update the output status directly. Applications should wait in the
usual way for AGC/AEC changes to take effect, and this "shortcut"
actually doesn't fit well with the gain being the combined gain.
2. The divideUpExposure method is adjusted to be clearer that it's
setting the combined gain, and it's prepare() that will discover later
what the analogue gain actually delivered.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Previously these were handled in the AGC/AEC exposure update
calculations by explicitly driving a higher digital gain to "cancel
out" any colour gains that were less than 1.
Now we're ignoring this in the AGC and leaving it to the IPA code to
normalise all the gains so that the smallest is 1. We don't regard
this as a "real" increase because one of the colour channels (just not
necessarily the green one) still gets the minimum gain possible.
We do, however, update the statistics calculations so that they
reflect any such digital gain increase, so that images are driven to
the correct level.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
A previous change introduced a bug in which it reported AfStateIdle
when idle in Auto mode, when it should continue to report the most
recent AF cycle's outcome (AfStateFocused or AfStateFailed).
Also fix the Pause method so it won't reset state to AfStateIdle
when paused in Continuous AF mode (to match documented behaviour).
Fixes: ea5f451c56 ("ipa: rpi: controller: AutoFocus bidirectional scanning")
Signed-off-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
To reduce unnecessary lens movements, allow the CDAF-based
search procedure to start from either end of the range;
or if not near an end, from the current lens position.
This sometimes requires a second coarse scan, if the first
one started in the middle and did not find peak contrast.
Shorten the fine scan from 5 steps to 3 steps; allow fine scan
to be omitted altogether when "step_fine": 0 in the tuning file.
Move updateLensPosition() out of startProgrammedScan() to avoid
calling it more than once per iteration.
Signed-off-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Analyse AWB statistics: used both for scene change detection
and to detect IR lighting (when a flag is set in the tuning file).
Option to suppress PDAF altogether when IR lighting is detected.
Rather than being based solely on PDAF "dropout", allow a scan to
be (re-)triggered whenever the scene changes and then stabilizes,
based on contrast and average RGB statistics within the AF window.
Signed-off-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Increase threshold for ETBP, from "confEpsilon" to "confThresh".
Correct sign test to take account of pdafGain sign (typically -ve).
Reduce allowed extrapolation range, but relax the check in the
case of Continuous AF, when we go back into the PDAF closed loop.
Signed-off-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In getPhase(), stop using different weights for sumWc and sumWcp.
This should improve linearity e.g. in earlyTerminationByPhase().
Phases are slightly larger but confidence values slightly reduced.
Signed-off-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This fixes two small bugs:
We previously populated LensPosition's ControlInfo with hard-coded
values, ignoring the tuning file. Now we query the AfAlgorithm to
get limits (over all AF ranges) and default (for AfRangeNormal).
We previously sent a default position to the lens driver, even when
a user-specified starting position would follow. Defer doing this,
to reduce unnecessary lens movement at startup (for some drivers).
Bug: https://bugs.libcamera.org/show_bug.cgi?id=258
Signed-off-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The "fast" parameter has not been used since it first appeared in the
source code. And not only is it not used, but its retrieval from
the configuration since c1597f9896 ("ipa: raspberrypi: Use YamlParser
to replace dependency on boost") has been incorrect. So remove it.
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
The newly introduced controls to drive the AEGC algorithm allow to
control the computation of the exposure time and analogue gain
separately.
The RPi AEGC implementation already computes the shutter and gain values
separately but does not expose separate functions to control them.
Augment the AgcAlgorithm interface to allow pausing/resuming the shutter
and gain automatic computations separately and plumb them to the newly
introduced controls.
Add safety checks to ignore ExposureTime and AnalogueGain values if the
algorithms are not paused, and report the correct AeState value by
checking if both algorithms have been paused or if they have converged.
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Acked-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
This property (dataBufferStrided) indicates if the CSI-2 hardware writes
to the embedded/metadata buffer directly, or if it treats the buffer
like an image buffer and strides the metadata lines.
Unicam writes this buffer strided, while the PiSP Frontend writes to it
directly. This information will be relevant to data parsers in the
helpers where the data is structured in lines.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
ColourTemperature is now exported as a writable control so that
applications can set it directly. The AWB algorithm class now requires
a method to be provided to perform this operation. The method should
clamp the passed value to the calibrated range known to the algorithm.
The default range is set very wide to cover all conceivable future AWB
calibrations. It will always be clamped before use.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Tested-by: Stefan Klug <stefan.klug@ideasonboard.com>
The RaspberryPi IPA contains a private Matrix3x3 class inside the ccm
algorithm. Replace it with the Matrix class available in
libcamera/internal.
While at it, mark the matrices RGB2Y and Y2RGB as static const.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Naushir Patuck <naush@raspberrypi.com>
The upcoming patches will introduce a Matrix class into
libcamera/internal. That name clashes with the Matrix class from the
RaspberryPi ccm implementation. Rename the rpi version to Matrix3x3 to
prevent the name clash. Matrix3x3 will be replaced by the generic
implementation later.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The rec601LuminanceFromRGB() and estimateCCT() functions take RGB
triplets as three variables. Replace them with instances of the RGB
class and adapt the users accordingly. Only variables passed directly to
these functions are converted to RGB instances, further conversion of
IPA modules to the RGB class will be performed separately.
While at it, fix a typo in the documentation of the estimateCCT()
function.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
The terms "shutter" and "shutter speed" are used through libcamera to
mean "exposure time". This is confusing, both due to "speed" being used
as "time" while it should be the inverse (i.e. a maximum speed should
correspond to the minimum time), and due to "shutter speed" and
"exposure time" being used in different places with the same meaning.
To improve clarity of the code base and the documentation, use "exposure
time" consistently to replace "shutter speed".
This rename highlighted another vocabulary issue in libcamera. The
ExposureModeHelper::splitExposure() function used to document that it
splits "exposure time into shutter time and gain". It has been reworded
to "split exposure into exposure time and gain". That is not entirely
satisfactory, as "exposure" has a defined meaning in photography (see
https://en.wikipedia.org/wiki/Exposure_(photography)) that is not
expressed as a duration. This issue if left to be addressed separately.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In the case of an AWB search failure, the current algorithm logic will
return a point on the CT curve closest to where the search finisned.
This can be quite undesirable. Instead, add some bias params to the AWB
algorithm which will direct the search to a set CT value in the case
where statistics become unreliable causing the search to fail.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
As described in the coding style document, libcamera favours <cmath>
over <math.h>. Replace the last few occurrences of the latter with the
former in the Raspberry Pi IPA and adapt the code accordingly. In some
cases, the <math.h> include is simply dropped as it isn't needed.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
As explained in the coding style document, usage of std::abs() is
preferred over abs() or fabs() as it picks the correct function based on
the argument type. Replace calls to abs() and fabs() with std::abs() in
the Raspberry Pi algorithms.
This fixes a reported warning from clang:
../src/ipa/rpi/controller/rpi/awb.cpp:508:6: error: using integer absolute value function 'abs' when argument is of floating point type [-Werror,-Wabsolute-value]
if (abs(denominator) > eps) {
^
../src/ipa/rpi/controller/rpi/awb.cpp:508:6: note: use function 'std::abs' instead
if (abs(denominator) > eps) {
^~~
std::abs
Reported-by: Maarten Lankhorst <dev@lankhorst.se>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When a user is taking control of exposure and gain, setting them
manually, we set the AGC "stable region" to zero. This means that any
user changes, however small, will be applied, and they won't be
regarded as "too small to bother with".
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Unlike in C where they have been standardized since C99, variable-length
arrays in C++ are an extension supported by gcc and clang. Clang started
warning about this with -Wall in version 18:
src/libcamera/ipc_unixsocket.cpp:250:11: error: variable length arrays in C++ are a Clang extension [-Werror,-Wvla-cxx-extension]
250 | char buf[CMSG_SPACE(num * sizeof(uint32_t))];
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
One simple option is to disable the warning. However, usage of VLAs in
C++ is discouraged by some, usually due to security reasons, based on
the rationale that developers are often unaware of unintentional use of
VLAs and how they may affect the security of the code when the array
size is not properly validated.
This rationale may sound dubious, as the most commonly proposed fix is
to replace VLAs with vectors (or just arrays dynamically allocated with
new() wrapped in unique pointers), without adding any size validation.
This will not produce much better results. However, keeping the VLA
warning and converting the code to dynamic allocation may still be
slightly better, as it can prompt developers to notice VLAs and check if
size validation is required.
For these reasons, convert all VLAs to std::vector. Most of the VLAs
don't need extra size validation, as the size is bound through different
constraints (e.g. image width for line buffers). An arguable exception
may be the buffers in IPCUnixSocket::sendData() and
IPCUnixSocket::recvData() as the number of fds is not bound-checked
locally, but we will run out of file descriptors before we could
overflow the buffer size calculation.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Acked-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Multiple local functions are defined in the global namespace without the
static keyword. This compiles fine for now, but will cause a missing
declaration warning when we enable them. To prepare for that, move the
function declaration to an anonymous namespace.
While at it, for consistency, include an existing static function in the
namespace and drop the static keyword.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Add support for the following HDR modes in the Raspberry Pi IPA:
- Night mode
- Single exposure mode
- Multi-exposure (merged and unmerged)
The algorithm is updated to expect the HDR short channel to meter
explicitly for highlights. This means that it will not in general
under-expose the short channel more than is actually necessary.
When images don't have much saturation, it's good to detect this so
that some of the boost we want to apply to the dark areas can be
implemented as regular gain. This means we can then adjust the tone
curve less, leading to less flat looking images.
The impact on the HDR algorithm is then that this determines how we
build tonemaps dynamically. The highlights are more-or-less correct
now, so we have to build a power-type curve that gives us the
appropriately configured targets in the lower part of the histogram.
We allow the tuning file to supply the maximum spatial gain value,
rather than the whole curve (though it can supply this if it
wants). Some parameter defaults are tweaked to be generally better
across the range of our cameras.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Acked-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Source files in libcamera start by a comment block header, which
includes the file name and a one-line description of the file contents.
While the latter is useful to get a quick overview of the file contents
at a glance, the former is mostly a source of inconvenience. The name in
the comments can easily get out of sync with the file name when files
are renamed, and copy & paste during development have often lead to
incorrect names being used to start with.
Readers of the source code are expected to know which file they're
looking it. Drop the file name from the header comment blocks in all
remaining locations that were not caught by the automated script as they
are out of sync with the file name.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Source files in libcamera start by a comment block header, which
includes the file name and a one-line description of the file contents.
While the latter is useful to get a quick overview of the file contents
at a glance, the former is mostly a source of inconvenience. The name in
the comments can easily get out of sync with the file name when files
are renamed, and copy & paste during development have often lead to
incorrect names being used to start with.
Readers of the source code are expected to know which file they're
looking it. Drop the file name from the header comment block.
The change was generated with the following script:
----------------------------------------
dirs="include/libcamera src test utils"
declare -rA patterns=(
['c']=' \* '
['cpp']=' \* '
['h']=' \* '
['py']='# '
['sh']='# '
)
for ext in ${!patterns[@]} ; do
files=$(for dir in $dirs ; do find $dir -name "*.${ext}" ; done)
pattern=${patterns[${ext}]}
for file in $files ; do
name=$(basename ${file})
sed -i "s/^\(${pattern}\)${name} - /\1/" "$file"
done
done
----------------------------------------
This misses several files that are out of sync with the comment block
header. Those will be addressed separately and manually.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
