As described in the coding style document, libcamera favours <cmath>
over <math.h>. Replace the last few occurrences of the latter with the
former in the Raspberry Pi IPA and adapt the code accordingly. In some
cases, the <math.h> include is simply dropped as it isn't needed.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
As explained in the coding style document, usage of std::abs() is
preferred over abs() or fabs() as it picks the correct function based on
the argument type. Replace calls to abs() and fabs() with std::abs() in
the Raspberry Pi algorithms.
This fixes a reported warning from clang:
../src/ipa/rpi/controller/rpi/awb.cpp:508:6: error: using integer absolute value function 'abs' when argument is of floating point type [-Werror,-Wabsolute-value]
if (abs(denominator) > eps) {
^
../src/ipa/rpi/controller/rpi/awb.cpp:508:6: note: use function 'std::abs' instead
if (abs(denominator) > eps) {
^~~
std::abs
Reported-by: Maarten Lankhorst <dev@lankhorst.se>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The black level is likely to get updated, if ever, only after exposure
or gain changes. Don't compute its possible updates if exposure and
gain are unchanged.
It's probably not worth trying to implement something more
sophisticated. Better to spend the effort on supporting tuning files.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This is the last step to fully convert software ISP to Algorithm-based
processing.
The newly introduced frameContext.sensor parameters are set, and the
updated code moved, before calling Algorithm::process() to have the
values up-to-date in stats processing.
Resolves software ISP TODO #10.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Use the standard libcamera mechanism to report the "current" controls
rather than delaying updates by counting from the last update.
A problem is that with software ISP we cannot be sure about the sensor
delay. The original implementation simply skips exposure updates for 2
frames, which should be enough in most cases. After this change, we
assume the delay being exactly 2 frames, which may or may not be correct
and may work with outdated values if the delay is shorter.
According to Kieran, the wrong parts are also wrong on the
IPU3/RKISP1/Mali pipelines and only RPi have this correct. We need to
fix this, by correctly specifying the gains in the libipa camera sensor
helpers. The sooner the better because this change could introduce a
risk of increasing oscillations.
This patch also prepares moving exposure+gain to an algorithm module
where the original delay mechanism would be a (possibly unnecessary)
complication.
Resolves software ISP TODO #11 + #12.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
It's more natural to represent color gains as floating point numbers
rather than using a particular pixel-related representation.
double is used rather than float because it's a more common floating
point type in libcamera algorithms. Otherwise there is no obvious
reason to select one over the other here.
The constructed color tables still use integer representation for
efficiency.
Black level still uses pixel (integer) values, for consistency with
other libcamera parts.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
After black level handling has been moved to an algorithm module, white
balance and the construction of color tables can be moved to algorithm
modules too.
This time, the moved code is split between stats processing and
parameter construction methods. It is also split to two algorithm
modules:
- White balance computation.
- Gamma table computation and color lookup tables construction. While
this applies the color gains computed by the white balance algorithm,
it is not directly related to white balance. And we may want to
modify the color lookup tables in future according to other parameters
than just gamma and white balance gains.
Gamma table computation and color lookup tables construction could be
split to separate algorithms too. But there is no big need for that now
so they are kept together for simplicity.
This is the only part of the software ISP algorithms that sets the
parameters so emitting setIspParams can be moved to prepare() method.
A more natural representation of the gains (and black level) would be
floating point numbers. This is not done here in order to minimize
changes in code movements. It will be addressed in a followup patch.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Acked-by: Umang Jain <umang.jain@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The black level determination, already present as a separate class, can
be moved to the prepared Algorithm processing structure. It is the
first of the current software ISP algorithms that is called in the stats
processing sequence, which means it is also the first one that should be
moved to the new structure. Stats processing starts with calling
Algorithm-based processing so the call order of the algorithms is
retained.
Movement of this algorithm is relatively straightforward because all it
does is processing stats.
The comment about getting black level from the tuning files is dropped.
The black level will be taken from CameraSensorHelper if available and
that will be implemented in one of the followup patches.
Black level is now recomputed on each stats processing. In a future
patch, after DelayedControls are used, this will be changed to recompute
the black level only after exposure/gain changes.
The black level depends on the sensor used, the computed value can be
reused in a followup capture sessions with the same sensor. Thus it is
sufficient to (re)set the initial value in BlackLevel::init.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This patch adds Algorithm::process call for the defined algorithms.
This is preparation only since there are currently no Algorithm based
algorithms defined.
As software ISP currently doesn't produce any metadata, a dummy and
unused metadata instance is created to satisfy Algorithm::process API.
This should be changed in future.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This patch adds Algorithm::queueRequest call for the defined algorithms.
As there are currently no control knobs in software ISP nor the
corresponding queueRequest call chain, the patch also introduces the
queueRequest methods and calls from the pipeline to the IPA.
This is preparation only since there are currently no Algorithm based
algorithms defined and no current software ISP algorithms support
control knobs.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This patch adds Algorithm::configure call for the defined algorithms.
This is preparation only since there are currently no Algorithm based
algorithms defined.
A part of this change is passing IPAConfigInfo instead of ControlInfoMap
to configure() calls as this is what Algorithm::configure expects.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This patch adds frame and bufferId arguments to stats related calls.
Although the parameters are currently unused, because frame ids are not
tracked and used and the stats buffer is passed around directly rather
than being referred by its id, they bring the internal APIs closer to
their counterparts in hardware pipelines.
It serves as a preparation for followup patches that will introduce:
- Frame number tracking in order to switch to DelayedControls
(software ISP TODO #11 + #12).
- A ring buffer for stats in order to improve passing the stats
(software ISP TODO #2).
Frame and buffer ids are unrelated for the given purposes but since they
are passed together at the same places, the change is implemented as a
single patch rather than two, basically the same, patches.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Software ISP image processing algorithms are currently defined in a
simplified way, different from other libcamera pipelines. This is not
good for several reasons:
- It makes the software ISP code harder to understand due to its
different structuring.
- Adding more algorithms may make the code harder to understand
generally.
- Mass libcamera code changes may not be easily applicable to software
ISP.
- Algorithm sharing with other pipelines is not easily possible.
This patch introduces basic software ISP IPA skeletons structured
similarly to the other pipelines. The newly added files are currently
not used or compiled and the general skeleton structures don't contain
anything particular. It is just a preparation step for a larger
refactoring and the code will be actually used and extended as needed in
followup patches.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
IPA modules use custom namespaces for all their internal components to
avoid namespace clashes. The simple IPA module for the software ISP uses
libcamera::ipa::soft for this purpose. It however defines an internal
class named BlackLevel in the root of the libcamera namespace, making it
prone to clashes. Move it to the ipa::soft namespace along with the rest
of the code.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add a loader that is capable of loading polynomial coefficients from the
tuning files. The polynomial is sampled at load time to reduce the
computational overhead at runtime.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
For the LSC algorithm to dynamically calculate the LSC tables based on
the sensor size and the crop rectangle it needs access to that data.
Provide access to it by adding the sensorInfo object to the context.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Now, that the generic interpolator is available, use it to do the
interpolation of the lens shading tables. This makes the algorithm
easier to read and remove some duplicate code.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The MatrixInterpolator is great for interpolation of matrices for
different color temperatures. It has however one limitation - it can
only handle matrices. For LSC it would be great to interpolate the LSC
tables (or even polynomials) using the same approach. Add a generic
Interpolator class based on the existing MatrixInterpolator. This class
can be adapted to any other type using partial template specialization.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
When a user is taking control of exposure and gain, setting them
manually, we set the AGC "stable region" to zero. This means that any
user changes, however small, will be applied, and they won't be
regarded as "too small to bother with".
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The ipa_interface.h file includes a number of headers that are not
directly used. Remove them, and add them to the source files that
include ipa_interface.h as required.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
The LSP autoformatter doesn't like some of the current formatting, let's
make it happier. Note that not all of its suggestions were accepted
because readability is preferred and adjusting .clang-format may not be
easy or possible.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The LSP autoformatter doesn't like some of the current formatting, let's
make it happier. Note that not all of its suggestions were accepted
because readability is preferred and adjusting .clang-format may not be
easy or possible.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Extend the RkISP1 BLC algorithm to use the ISP 'companding' block for
versions of the ISP (such as the one on the i.MX8MP) that lack the
dedicated BLS block but implement BLS as part of the companding block.
As access to the companding block requires the extensible parameters
format, disable BLC when using the legacy parameters format on i.MX8MP
to avoid crashes at runtime with older kernels.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Use the new ISP parameters abstraction class RkISP1Params to access the
ISP parameters in the IPA algorithms. The class replaces the pointer to
the rkisp1_params_cfg structure passed to the algorithms' prepare()
function, and is used to access individual parameters blocks.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Individual algorithms of the rkisp1 IPA module access their
corresponding ISP parameters through the top-level structure
rkisp1_params_cfg. This will not work anymore with the new parameters
format. In order to ease the transition to the new format, abstract the
ISP parameters in a new RkISP1Params class that offers the same
interface regardless of the format.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
The ISP parameters buffer currently has a fixed payload size, which is
hardcoded in the pipeline handler. To prepare for support of the
extensible parameters format that has a variable payload size, pass the
size from the IPA module to the pipeline handler explicitly. Keep the
size hardcoded to sizeof(struct rkisp1_params_cfg) for now, this will be
udpated later.
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The rkisp1 driver supports two formats for the ISP parameters buffer,
the legacy fixed format and the new extensible format. In preparation of
support for the new format, pass the parameters buffer format from the
pipeline handler to the IPA module and store it.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The libcamera coding style groups the C and C++ standard library headers
in a single group. Fix the few offenders in the source tree.
While at it, add a missing blank line between header groups in a
separate location.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
The libcamera_generated_ipa_headers variable, containing the list of
generated IPA headers, is listed in the sources of IPA modules, as well
as IPA tests. This was done to ensure that the modules and tests get
rebuilt when the generate IPA headers change. However, the dependency is
already handled through the libcamera_private dependency object,
specified for all those modules and tests. There's no need to list the
IPA generated headers as sources. Drop them.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The documentation for the blackLevel_ member is very terse. Reference
the more complete documentation of the sibling blackLevel() member
function to provide more information.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Unlike in C where they have been standardized since C99, variable-length
arrays in C++ are an extension supported by gcc and clang. Clang started
warning about this with -Wall in version 18:
src/libcamera/ipc_unixsocket.cpp:250:11: error: variable length arrays in C++ are a Clang extension [-Werror,-Wvla-cxx-extension]
250 | char buf[CMSG_SPACE(num * sizeof(uint32_t))];
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
One simple option is to disable the warning. However, usage of VLAs in
C++ is discouraged by some, usually due to security reasons, based on
the rationale that developers are often unaware of unintentional use of
VLAs and how they may affect the security of the code when the array
size is not properly validated.
This rationale may sound dubious, as the most commonly proposed fix is
to replace VLAs with vectors (or just arrays dynamically allocated with
new() wrapped in unique pointers), without adding any size validation.
This will not produce much better results. However, keeping the VLA
warning and converting the code to dynamic allocation may still be
slightly better, as it can prompt developers to notice VLAs and check if
size validation is required.
For these reasons, convert all VLAs to std::vector. Most of the VLAs
don't need extra size validation, as the size is bound through different
constraints (e.g. image width for line buffers). An arguable exception
may be the buffers in IPCUnixSocket::sendData() and
IPCUnixSocket::recvData() as the number of fds is not bound-checked
locally, but we will run out of file descriptors before we could
overflow the buffer size calculation.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Acked-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Naushir Patuck
Milan Zamazal
Laurent Pinchart
André Apitzsch
Stefan Klug
Kieran Bingham
will whang
David Plowman
Paul Elder
Jacopo Mondi