At the moment when the overall image brightness is considered too high
the AGC code will lower the gain all the way down to againMin before
considering lowering the exposure.
What should happen instead is lower the gain no lower than 1.0 and after
that lower the exposure instead of lowering the gain.
Otherwise there might be a heavily overexposed image (e.g. all white)
which then is made less white by a gain < 1.0 which is no good.
When there is no sensor-helper, assume the driver reported default-gain
value is close to a gain of 1.0 .
While at it also remove the weird limitation to only lower the gain
when exposure is set to the maximum. As long as the gain is higher
than the default gain, the gain should be lowered first.
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
Signed-off-by: Hans de Goede <hansg@kernel.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Currently the virtual pipeline generates the images synchronously. This is not
ideal because it blocks the camera manager's internal thread, and because its
behaviour is different from other existing pipeline handlers, all of which
complete requests asynchronously.
So move the image generation to a separate thread by deriving `VirtualCameraData`
from `Thread`, as well as `Object` and using the existing asynchronous signal
and method call mechanism.
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
While optional, libunwind integration is enabled when meson finds it without
having a way to disable it. This is the case for Debian where libunwind is
installed by build dependencies. Since we want to reduce dependencies on
libunwind in Debian due to several issues with it[0], we need an option to
control its activation.
[0]: https://bugs.debian.org/1093688
Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
[Longer commit message lines, reworded meson option description.]
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
The code was only allowing the exposure mode to be updated when both
exposure/gain were in manual mode, which is a mistake. It needs to be
updatable precisely when the auto modes are enabled.
The fix is to ignore the enabled/disabled status of AEC/AGC, matching
our other controls (metering mode, constraint mode etc.). While there
might be a debate to be had about what the actual behaviour of all
these controls should be, for the time being we'll just match
everything else.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Software ISP performs performance measurement on certain part of initial
frames. Let's make this range configurable.
For this purpose, this patch introduces new configuration options
software_isp.measure.skip and software_isp.measure.number. Setting the
latter one to 0 disables the measurement.
Instead of the last frame, the class member and its configuration
specify the number of frames to measure. This is easier to use for
users and doesn't require to adjust two configuration parameters when
the number of the initially skipped frames is changed.
The patch also changes the names of the class members to make them more
accurate.
Completes software ISP TODO #7.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
On some platforms, working directly on the input buffer is very slow due
to disabled caching. This is why we copy the input buffer into standard
(cached) memory. This is an unnecessary overhead on platforms with
cached buffers.
Let's make input buffer copying configurable. The default is still
copying, as its overhead is much lower than contingent operations on
non-cached memory. Ideally, we should improve this in future to set the
default to non-copying if we can be sure under observable circumstances
that we are working with cached buffers.
Completes software ISP TODO #6.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
This patch allows enabling or disabling software ISP via config file in
addition to compile time. This can be useful for software ISP testing
on various platforms as well as for overriding the defaults in case the
defaults don't work well (e.g. hardware ISP may or may not work on
i.MX8MP depending on the kernel and libcamera patches present in the
given system).
The configuration is specified as follows:
configuration:
pipelines:
simple:
supported_devices:
- driver: DRIVER-NAME
software_isp: BOOLEAN
- ...
For example:
configuration:
pipelines:
simple:
supported_devices:
- driver: mxc-isi
software_isp: true
The overall configuration of enabling or disabling software ISP may get
dropped in future but this patch is still useful in the meantime.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
This patch adds configuration options for environment variables used in
the IPA proxy.
The configuration snippet:
configuration:
ipa:
config_paths:
- config path 1
- config path 2
- ...
module_paths:
- module path 1
- module path 2
- ...
proxy_paths:
- proxy path 1
- proxy path 2
- ...
force_isolation: BOOL
LIBCAMERA_<IPA_NAME>_TUNING_FILE remains configurable only via the
environment variable; this is supposed to be used only for testing and
debugging and it's not clear what to do about IPA names like "rpi/vc4"
and "rpi/pisp" exactly.
There are two ways to pass the configuration to the places where it is
needed: Either to pass it as an argument to the method calls that need
it, or to pass it to the class constructors and extract the needed
configuration from there. This patch uses the second method as it is
less polluting the code.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Let's add some helpers to make accessing simple configuration values
simpler. The helpers are used in the followup patches.
GlobalConfiguration::option ensures that no value is returned rather
than a value of YamlObject::empty.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Global configuration is accessed via a GlobalConfiguration instance.
The instance is conceptually a singleton, but singletons are not welcome
in libcamera so we must store the (preferably single) instance
somewhere.
This patch creates a GlobalConfiguration instance in CameraManager and
defines the corresponding access method. CameraManager is typically
instantiated only once or a few times, it is accessible in many places
in libcamera and the configuration can be retrieved from it and passed
to other places if needed (it's read-only once created). Using
CameraManager for the purpose is still suboptimal and we use it only due
to lack of better options. An alternative could be Logger, which is
still a singleton and it's accessible from everywhere. But with Logger,
we have a chicken and egg problem -- GlobalConfiguration should log
contingent problems with the configuration when it's loaded but if it is
created in the logger then there are mutual infinite recursive calls.
One possible way to deal with this is to look at the environment
variables only during logging initialisation and apply the logging
configuration when a CameraManager is constructed. Considering there
are intentions to remove the Logger singleton, let's omit logging
configuration for now.
If there are multiple CameraManager instances, there are also multiple
GlobalConfiguration instances, each CameraManager instance is meant to
be fully independent, including configuration. They may or may not
contain the same data, depending on whether the global configuration
file in the file system was changed in the meantime.
The configuration is stored in the private CameraManager. It's
accessible within libcamera (via CameraManager) but it's not meant to be
accessed by applications.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Currently, libcamera can be configured in runtime using several
environment variables. With introducing more and more variables, this
mechanism reaches its limits. It would be simpler and more flexible if
it was possible to configure libcamera in a single file.
For example, there was a request to define pipeline precedence in
runtime. We want to compile in multiple pipelines, in order to have
them accessible within single packages in distributions. And then being
able to select among the pipelines manually as needed based on the
particular hardware or operating system environment. Having the
configuration file then allows easy switching between hardware, GPU or
CPU IPAs. The configuration file can also be used to enable or disable
experimental features and avoid the need to track local patches changing
configuration options hard-wired in the code when working on new
features.
This patch introduces basic support for configuration files.
GlobalConfiguration class reads and stores the configuration. Its
instance can be used by other libcamera objects to access the
configuration. A GlobalConfiguration instance is supposed to be stored
in a well-defined place, e.g. a CameraManager instance. It is possible
to have multiple GlobalConfiguration instances, which may or may not
make sense.
libcamera configuration can be specified using a system-wide
configuration file or a user configuration file. The user configuration
file takes precedence if present. There is currently no way to merge
multiple configuration files, the one found is used as the only
configuration file. If no configuration file is present, nothing
changes to the current libcamera behavior (except for some log
messages related to configuration file lookup).
The configuration file is a YAML file. We already have a mechanism for
handling YAML configuration files in libcamera and the given
infrastructure can be reused for the purpose. However, the
configuration type is abstracted to make contingent future change of the
underlying class easier while retaining (most of) the original API.
The configuration is versioned. This has currently no particular
meaning but is likely to have its purpose in future, especially once
configuration validation is introduced.
The configuration YAML file looks as follows:
---
version: 1
configuration:
WHATEVER CONFIGURATION NEEDED
This patch introduces just the basic idea. Actually using the
configuration in the corresponding places (everything what is currently
configurable via environment variables should be configurable in the
file configuration) and other enhancements are implemented in the
followup patches.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Add a WDR algorithm to do global tone mapping. Global tone mapping is
used to increase the perceived dynamic range of an image. The typical
effect is that in areas that are normally overexposed, additional
structure becomes visible.
The overall idea is that the algorithm applies an exposure value
correction to underexpose the image to the point where only a small
number of saturated pixels is left. This artificial underexposure is
then mitigated by applying a tone mapping curve.
This algorithm implements 4 tone mapping strategies:
- Linear
- Power
- Exponential
- Histogram equalization
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Query the params device for RKISP1_CID_SUPPORTED_PARAMS_BLOCKS and
inject the information into the IPA hardware context for use by the
algorithms.
To be able to modify the hardware configuration at runtime, replace the
pointer with an instance and create a copy of the static hardware
specific data.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The Y mode of the histogram gets captured at the ISP output, before the
output formatter. This has the side effect that the first and the last
bins are empty in case of limited YUV range. Another side effect is
that gamma and GWDR processing is included in the histogram which makes
algorithm development very difficult. In RGB mode the histogram is taken
after xtalk (CCM) and is therefore independent of gamma and WDR. The
limited range issue also does not apply. In the ISP reference it is
however stated that "it is not possible to calculate a luminance or
grayscale histogram from an RGB histogram since the position information
is lost during its generation".
During testing the RGB histogram provided good data and better
algorithmic stability at a possible (but not measured) inaccuracy.
Another option would be to pass the color space information into the IPA
and strip the histogram accordingly. For ease of implementation switch to
the RGB mode.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When the extensible parameters queued to the kernel contain an unknown
block type it fails with -EINVAL. This should not happen as user land is
supposed to check for the supported parameter types. But it took a while
to figure out where things went wrong. Add a error statement when
queuing of the parameter buffer fails for whatever reason.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
There are several occasions where quantization can lead to visible
effects.
In WDR mode it can happen that exposure times get set to very low values
(Sometimes 2-3 lines). This intentionally introduced underexposure is
corrected by the GWDR module. As exposure time is quantized by lines,
the smallest possible change in exposure time now results in a quite
visible change in perceived brightness.
On some sensors the possible gain steps are also quite large leading to
visible jumps if e.g. if the exposure time is fixed.
Mitigate that by applying a global gain to account for the error
introduced by the exposure quantization.
ToDo: This needs perfect frame synchronous control of the sensor to work
properly which is not guaranteed in all cases. It still improves the
behavior with the current regulation and can easily be skipped, be
removing the compress algorithm from the tuning file.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Calculate the error introduced by quantization as "quantization gain"
and return it separately from splitExposure(). It is not included in the
digital gain, to not silently ignore the limits imposed by the AGC
configuration.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In ExposureModeHelper::splitExposure() the quantization of exposure time
and gain is not taken into account. This can lead to visible flicker
when the quantization steps are too big. As a preparation to fixing
that, add a function to set the sensor line length and the current
sensor mode helper and extend the clampXXX functions to return the
quantization error.
By default the exposure time quantization is assumed to be 1us and gain
is assumed to not be quantized at all.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add a small utility function that calculates the quantized gain that
gets applied by a sensor when the gain code is set to gainCode(gain).
This is needed by algorithms to calculate a digital correction gain that
gets applied to mitigate the error introduce by quantization.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The i.MX8 M Plus has a compression curve inside the compand block. This
curve is necessary to process HDR stitched data and is useful for other
aspects like applying a digital gain to the incoming sensor data.
Add a basic algorithm for the compression curve. This algorithm has a
hardcoded input width of 20bit and output width of 12bit which matches
the imx8mp pipeline. Only a static gain is supported in this version.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
For both vc4 and pisp, vd55g1.json has been generated using ctt with
rpi.dpc algorithm removed as this is already handled in the sensor's
ISP. vd55g1_mono.json has been adapted from vd55g1.json by removing
color correction related algorithms.
Adding Cyril Liotard as co-developer for providing the base vd55g1.json
tuning files for both vc4 and pisp. Thank you.
Co-Developed-by: Cyril Liotard <cyril.liotard@st.com>
Signed-off-by: Cyril Liotard <cyril.liotard@st.com>
Signed-off-by: Benjamin Mugnier <benjamin.mugnier@foss.st.com>
Acked-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Take the unique pointer to the `Fence` object by rvalue reference
so that it is not destroyed if the function returns an error code
and does not take ownership of the unique pointer.
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
In commit b8d332cdcc ("libcamera: framebuffer: Replace vector with
span in constructor") the FrameBuffer::planes() function was modified to
return a Span instead of a vector. This leads to the following runtime
exception in the python binding:
TypeError: Unregistered type : libcamera::Span<libcamera::FrameBuffer::Plane const, 18446744073709551615ul>
Fix that by manually converting the Span to a vector.
Note: The best solution would be to implement a Span type caster for
pybind11. But implementing and testing that properly is a bit more
involved than expected. As we don't need bidirectional mapping, use the
same workaround as for FrameMetadata::planes() for now.
While at it, update the lambda for pyFrameMetadata.planes() to call the
inner planes() only once.
Fixes: b8d332cdcc ("libcamera: framebuffer: Replace vector with span in constructor")
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
GPU ISP can produce only 32-bit output. Let's add support to the PPM
writer for all the common RGB image formats so that we can store GPU ISP
output as PPM files. Contingent alpha values are ignored as there is no
support for the alpha channel in PPM.
There is no obvious performance penalty in my environment compared to
output in the raw format.
Signed-off-by: Pavel Machek <pavel@ucw.cz>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Pavel Machek <pavel@ucw.cz>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In the near future we will add a SyncAdjustment control for adjusting
the frame duration via the sync algorithm. This control needs to be able
to take on a negative value, since the frame duration can be shortened
in addition to being extended. While the control is an int, it would be
convenient to be able to clamp it to frame duration limits, which are
usually handled as utils::Duration values internally. To allow this
using utils::Duration, add a unary negation operation to
utils::Duration. Also add a test for the operator.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The CCM algorithm will now let an explicit colour matrix be set when
AWB is in manual mode.
We must handle any controls that can cause the AWB to be enabled or
disabled first, so that we know the AWB's state correctly when we come
to set the CCM.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
[Kieran: Remove duplicated Matrix3x3 from ccm.cpp]
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
If the currently selected camera disappears as reported by the `cameraRemoved`
signal, then `MainWindow::camera_` is reset to `nullptr`. In this case,
pressing the start/stop button will try to start streaming, leading to
a nullptr derefence in `MainWindow::startCapture()` when the configuration
is generated for the camera.
Fix that by returning from `MainWindow::toggleCapture()` if no camera is set.
While this will cause the "checked" status of `startStopAction_` to go out of
sync, this should not be an issue because when a new camera is selected, the
state is synchronized.
Bug: https://bugs.libcamera.org/show_bug.cgi?id=177
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Hans de Goede
Barnabás Pőcze
Dylan Aïssi
David Plowman
Milan Zamazal
Stefan Klug
Matthias Fend
Benjamin Mugnier
Laurent Pinchart
Paul Elder
Naushir Patuck