This patch adds configuration options for environment variables used in
the IPA proxy.
The configuration snippet:
configuration:
ipa:
config_paths:
- config path 1
- config path 2
- ...
module_paths:
- module path 1
- module path 2
- ...
proxy_paths:
- proxy path 1
- proxy path 2
- ...
force_isolation: BOOL
LIBCAMERA_<IPA_NAME>_TUNING_FILE remains configurable only via the
environment variable; this is supposed to be used only for testing and
debugging and it's not clear what to do about IPA names like "rpi/vc4"
and "rpi/pisp" exactly.
There are two ways to pass the configuration to the places where it is
needed: Either to pass it as an argument to the method calls that need
it, or to pass it to the class constructors and extract the needed
configuration from there. This patch uses the second method as it is
less polluting the code.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Let's add some helpers to make accessing simple configuration values
simpler. The helpers are used in the followup patches.
GlobalConfiguration::option ensures that no value is returned rather
than a value of YamlObject::empty.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Global configuration is accessed via a GlobalConfiguration instance.
The instance is conceptually a singleton, but singletons are not welcome
in libcamera so we must store the (preferably single) instance
somewhere.
This patch creates a GlobalConfiguration instance in CameraManager and
defines the corresponding access method. CameraManager is typically
instantiated only once or a few times, it is accessible in many places
in libcamera and the configuration can be retrieved from it and passed
to other places if needed (it's read-only once created). Using
CameraManager for the purpose is still suboptimal and we use it only due
to lack of better options. An alternative could be Logger, which is
still a singleton and it's accessible from everywhere. But with Logger,
we have a chicken and egg problem -- GlobalConfiguration should log
contingent problems with the configuration when it's loaded but if it is
created in the logger then there are mutual infinite recursive calls.
One possible way to deal with this is to look at the environment
variables only during logging initialisation and apply the logging
configuration when a CameraManager is constructed. Considering there
are intentions to remove the Logger singleton, let's omit logging
configuration for now.
If there are multiple CameraManager instances, there are also multiple
GlobalConfiguration instances, each CameraManager instance is meant to
be fully independent, including configuration. They may or may not
contain the same data, depending on whether the global configuration
file in the file system was changed in the meantime.
The configuration is stored in the private CameraManager. It's
accessible within libcamera (via CameraManager) but it's not meant to be
accessed by applications.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Currently, libcamera can be configured in runtime using several
environment variables. With introducing more and more variables, this
mechanism reaches its limits. It would be simpler and more flexible if
it was possible to configure libcamera in a single file.
For example, there was a request to define pipeline precedence in
runtime. We want to compile in multiple pipelines, in order to have
them accessible within single packages in distributions. And then being
able to select among the pipelines manually as needed based on the
particular hardware or operating system environment. Having the
configuration file then allows easy switching between hardware, GPU or
CPU IPAs. The configuration file can also be used to enable or disable
experimental features and avoid the need to track local patches changing
configuration options hard-wired in the code when working on new
features.
This patch introduces basic support for configuration files.
GlobalConfiguration class reads and stores the configuration. Its
instance can be used by other libcamera objects to access the
configuration. A GlobalConfiguration instance is supposed to be stored
in a well-defined place, e.g. a CameraManager instance. It is possible
to have multiple GlobalConfiguration instances, which may or may not
make sense.
libcamera configuration can be specified using a system-wide
configuration file or a user configuration file. The user configuration
file takes precedence if present. There is currently no way to merge
multiple configuration files, the one found is used as the only
configuration file. If no configuration file is present, nothing
changes to the current libcamera behavior (except for some log
messages related to configuration file lookup).
The configuration file is a YAML file. We already have a mechanism for
handling YAML configuration files in libcamera and the given
infrastructure can be reused for the purpose. However, the
configuration type is abstracted to make contingent future change of the
underlying class easier while retaining (most of) the original API.
The configuration is versioned. This has currently no particular
meaning but is likely to have its purpose in future, especially once
configuration validation is introduced.
The configuration YAML file looks as follows:
---
version: 1
configuration:
WHATEVER CONFIGURATION NEEDED
This patch introduces just the basic idea. Actually using the
configuration in the corresponding places (everything what is currently
configurable via environment variables should be configurable in the
file configuration) and other enhancements are implemented in the
followup patches.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Add a WDR algorithm to do global tone mapping. Global tone mapping is
used to increase the perceived dynamic range of an image. The typical
effect is that in areas that are normally overexposed, additional
structure becomes visible.
The overall idea is that the algorithm applies an exposure value
correction to underexpose the image to the point where only a small
number of saturated pixels is left. This artificial underexposure is
then mitigated by applying a tone mapping curve.
This algorithm implements 4 tone mapping strategies:
- Linear
- Power
- Exponential
- Histogram equalization
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Query the params device for RKISP1_CID_SUPPORTED_PARAMS_BLOCKS and
inject the information into the IPA hardware context for use by the
algorithms.
To be able to modify the hardware configuration at runtime, replace the
pointer with an instance and create a copy of the static hardware
specific data.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The Y mode of the histogram gets captured at the ISP output, before the
output formatter. This has the side effect that the first and the last
bins are empty in case of limited YUV range. Another side effect is
that gamma and GWDR processing is included in the histogram which makes
algorithm development very difficult. In RGB mode the histogram is taken
after xtalk (CCM) and is therefore independent of gamma and WDR. The
limited range issue also does not apply. In the ISP reference it is
however stated that "it is not possible to calculate a luminance or
grayscale histogram from an RGB histogram since the position information
is lost during its generation".
During testing the RGB histogram provided good data and better
algorithmic stability at a possible (but not measured) inaccuracy.
Another option would be to pass the color space information into the IPA
and strip the histogram accordingly. For ease of implementation switch to
the RGB mode.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When the extensible parameters queued to the kernel contain an unknown
block type it fails with -EINVAL. This should not happen as user land is
supposed to check for the supported parameter types. But it took a while
to figure out where things went wrong. Add a error statement when
queuing of the parameter buffer fails for whatever reason.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
There are several occasions where quantization can lead to visible
effects.
In WDR mode it can happen that exposure times get set to very low values
(Sometimes 2-3 lines). This intentionally introduced underexposure is
corrected by the GWDR module. As exposure time is quantized by lines,
the smallest possible change in exposure time now results in a quite
visible change in perceived brightness.
On some sensors the possible gain steps are also quite large leading to
visible jumps if e.g. if the exposure time is fixed.
Mitigate that by applying a global gain to account for the error
introduced by the exposure quantization.
ToDo: This needs perfect frame synchronous control of the sensor to work
properly which is not guaranteed in all cases. It still improves the
behavior with the current regulation and can easily be skipped, be
removing the compress algorithm from the tuning file.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Calculate the error introduced by quantization as "quantization gain"
and return it separately from splitExposure(). It is not included in the
digital gain, to not silently ignore the limits imposed by the AGC
configuration.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In ExposureModeHelper::splitExposure() the quantization of exposure time
and gain is not taken into account. This can lead to visible flicker
when the quantization steps are too big. As a preparation to fixing
that, add a function to set the sensor line length and the current
sensor mode helper and extend the clampXXX functions to return the
quantization error.
By default the exposure time quantization is assumed to be 1us and gain
is assumed to not be quantized at all.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add a small utility function that calculates the quantized gain that
gets applied by a sensor when the gain code is set to gainCode(gain).
This is needed by algorithms to calculate a digital correction gain that
gets applied to mitigate the error introduce by quantization.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The i.MX8 M Plus has a compression curve inside the compand block. This
curve is necessary to process HDR stitched data and is useful for other
aspects like applying a digital gain to the incoming sensor data.
Add a basic algorithm for the compression curve. This algorithm has a
hardcoded input width of 20bit and output width of 12bit which matches
the imx8mp pipeline. Only a static gain is supported in this version.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
For both vc4 and pisp, vd55g1.json has been generated using ctt with
rpi.dpc algorithm removed as this is already handled in the sensor's
ISP. vd55g1_mono.json has been adapted from vd55g1.json by removing
color correction related algorithms.
Adding Cyril Liotard as co-developer for providing the base vd55g1.json
tuning files for both vc4 and pisp. Thank you.
Co-Developed-by: Cyril Liotard <cyril.liotard@st.com>
Signed-off-by: Cyril Liotard <cyril.liotard@st.com>
Signed-off-by: Benjamin Mugnier <benjamin.mugnier@foss.st.com>
Acked-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Take the unique pointer to the `Fence` object by rvalue reference
so that it is not destroyed if the function returns an error code
and does not take ownership of the unique pointer.
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
In commit b8d332cdcc ("libcamera: framebuffer: Replace vector with
span in constructor") the FrameBuffer::planes() function was modified to
return a Span instead of a vector. This leads to the following runtime
exception in the python binding:
TypeError: Unregistered type : libcamera::Span<libcamera::FrameBuffer::Plane const, 18446744073709551615ul>
Fix that by manually converting the Span to a vector.
Note: The best solution would be to implement a Span type caster for
pybind11. But implementing and testing that properly is a bit more
involved than expected. As we don't need bidirectional mapping, use the
same workaround as for FrameMetadata::planes() for now.
While at it, update the lambda for pyFrameMetadata.planes() to call the
inner planes() only once.
Fixes: b8d332cdcc ("libcamera: framebuffer: Replace vector with span in constructor")
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
GPU ISP can produce only 32-bit output. Let's add support to the PPM
writer for all the common RGB image formats so that we can store GPU ISP
output as PPM files. Contingent alpha values are ignored as there is no
support for the alpha channel in PPM.
There is no obvious performance penalty in my environment compared to
output in the raw format.
Signed-off-by: Pavel Machek <pavel@ucw.cz>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Pavel Machek <pavel@ucw.cz>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In the near future we will add a SyncAdjustment control for adjusting
the frame duration via the sync algorithm. This control needs to be able
to take on a negative value, since the frame duration can be shortened
in addition to being extended. While the control is an int, it would be
convenient to be able to clamp it to frame duration limits, which are
usually handled as utils::Duration values internally. To allow this
using utils::Duration, add a unary negation operation to
utils::Duration. Also add a test for the operator.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The CCM algorithm will now let an explicit colour matrix be set when
AWB is in manual mode.
We must handle any controls that can cause the AWB to be enabled or
disabled first, so that we know the AWB's state correctly when we come
to set the CCM.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
[Kieran: Remove duplicated Matrix3x3 from ccm.cpp]
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
If the currently selected camera disappears as reported by the `cameraRemoved`
signal, then `MainWindow::camera_` is reset to `nullptr`. In this case,
pressing the start/stop button will try to start streaming, leading to
a nullptr derefence in `MainWindow::startCapture()` when the configuration
is generated for the camera.
Fix that by returning from `MainWindow::toggleCapture()` if no camera is set.
While this will cause the "checked" status of `startStopAction_` to go out of
sync, this should not be an issue because when a new camera is selected, the
state is synchronized.
Bug: https://bugs.libcamera.org/show_bug.cgi?id=177
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Sharpening is reduced slightly for official Raspberry Pi cameras, and
exposure profiles made a bit more consistent.
Denoise is reduced for the imx708 where it appears too strong.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Noise and detail tuning is improved for all official Raspberry Pi
cameras.
The old tunings left too much noise in and even sharpened some of
it. The new tunings remove more noise, and no longer sharpen it. Some
of the more general over-sharpening is also removed. Note that lost
detail can be recovered well using TDN (temporal denoise), which is
the recommended method to get the best results.
There are some minor adjustments to the CDN deviation, now that this
gets backed-off as TDN ramps up.
The contrast in the gamma in the bright areas is also toned down just
a little.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The CDN (colour denoise) deviation gets gradually reduced frame by
frame as TDN (temporal denoise) comes in and has more effect. CDN is
more harmful to image detail than TDN, so ramping it down in favour of
TDN is beneficial.
The tuning file parameters are chosen so that existing tuning files
don't have to be updated and will carry on working "mostly like they
did" (apart from the new back-off).
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add an imx415 tuning file for both the VC4 and PiSP. This tuning file
has been created and supplied by Arducam to support the B0569 module.
Note that this conflicts with an already existing imx415.json and
as such is provided as imx415_b0459.json.
More work will be required to support module specific tuning file
parsing.
Acked-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add an imx335 tuning file for both VC4 and PiSP. This tuning file
has been created and supplied by Arducam to support the B0568 module.
Acked-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In current implementation, the sink pad counter of the crossbar is not
incremented if the pad is not connected to any subdevice. This would lead
to incorrect routing and format configuration if CSI is not connected
to first sink pad.
To avoid such issue, every sink pads must be taken into account. Then if
CSI and sensor are present, current counter is used for routing at match(),
and stored in camera data to be reused during configure().
Signed-off-by: Antoine Bouyer <antoine.bouyer@nxp.com>
Tested-by: Pavel Löbl <pavel@loebl.cz>
Tested-by: Julien Vuillaumier <julien.vuillaumier@nxp.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
libcamera uses the meson python module to handle native compilation of
Python extension modules, and uses the shared_module() function when
cross-compiling due to an issue in the python module. The difference
between native and cross compilation also extends to the installation
path: native compilation lets the python module handle the paths, while
cross compilation constructs a path manually using a heuristic based on
the Python version and hardcoded components.
This manually-constructed installation path is problematic for cross
compilation for the same reason it caused issue when used for native
compilation: it is not guaranteed to be right, and it can't be
overridden by users.
Switch to obtaining the installation path from the meson python module
for cross-compilation as well. This also prepares for usage of
py.extension_module() once the file suffix issue will be fixed in meson.
On Debian 13, this change replaces the incorrect path
/usr/local/lib/python3.12/site-packages/libcamera with the still (but
differently) incorrect /usr/local/lib/python3/dist-packages/libcamera.
Future fixes in meson to address this issue will make the path correct
by default.
When the path calculated by the python module is not correct, it can now
be overridden by the user through the meson python.platlibdir
configuration variable.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
libcamera uses the meson python module to handle native compilation of
Python extension modules.
When cross-compiling, the module uses the build machine suffix instead
of the host machine suffix in some enviroments (for instance naming the
shared object file _libcamera.cpython-313-x86_64-linux-gnu.so instead of
_libcamera.cpython-313-aarch64-linux-gnu.so when cross-compiling from
x86_64 to aarch64). This prevents using the python module in that case,
and libcamera uses the normal dependency() function to locate the Python
libraries, and the shared_module() function to build the module.
Not using the meson python module to get the Python dependency prevents
selecting a specific Python interpreter, the same way as it does for
native builds. While having multiple Python interpreter versions in a
cross-build environment is likely less common, different behaviours and
features between native and cross-compilation are still not optimal.
Improve this situation by getting the dependency from the python module
for cross-compilation as well. This also prepares for usage of
py.extension_module() once the file suffix issue will be fixed in meson.
The user will need to ensure that the Python interpreter for the build
machine matches the version of the interpreter in the cross-compilation
environment for the host machine. Otherwise, meson will fail to find the
Python dependency. Cross-compilation environment provided by Linux
distributions (such as Debian multi-arch support) should work out of the
box, but compiling libcamera manually against a cross-compilation
environment provided by Buildroot or Yocto may require manual
configuration.
When the interpreters versions do not match, meson needs to be pointed
to the build ùachine interpreter from the cross-compilation environment
using the cross file. For instance, assuming a 'br_host_dir' variable
pointing to the host directory from Buildroot, the cross file should
contain
[binaries]
python = br_host_dir / 'bin/python3'
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
The existing meson.build file installs the bindings to a manually
constructed directory that is not included in the Python path in most
distributions. For instance, on a Debian 12 system, the modules is
intalled in /usr/lib/x86_64-linux-gnu/python3.11/site-packages/, while
the Python interpreter looks for site packages in
/usr/lib/python3/dist-packages/.
It also always builds the bindings using the system Python, as it
searches for the Python library using the standard dependency()
function. This prevents build the Python bindings for a different
interpreter version without changing the system default interpreter.
Modify the build process to use the meson python module to build the
Python bindings targets, so it installs them to the correct directories
for Python. This also allows specifying a different target Python
interpreter through the '[binaries]' section of a meson native file.
The behaviour is not changed for cross-compilation, as the meson python
module has known issues in that case.
Signed-off-by: William Vinnicombe <william.vinnicombe@raspberrypi.com>
Co-developed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
The planes variable in the Device::createFrameBuffer() function is a
reference to buffer.planes() that is only used as a range initializer in
a range-based for loop. Use buffer.planes() directly and drop the
variable.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Milan Zamazal
Stefan Klug
Matthias Fend
Benjamin Mugnier
Barnabás Pőcze
Laurent Pinchart
Paul Elder
David Plowman
Hans de Goede
Naushir Patuck
Kieran Bingham
Antoine Bouyer
William Vinnicombe