We need a separate control to report the nominal frame duration, but
it's also useful to report the min/max frame duration values that will
be used. Split the FrameDurations control into FrameDuration and
FrameDurationLimits respectively to support both of these.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Hirokazu Honda <hiroh@chromium.org>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
There can be multiple IPAs per pipeline-handler or platform.
They can live in-tree or externally linked. It is up to the vendor
whether to use in-tree IPA for a platform or provide it externally.
In the case of IPAs being external, building in-tree IPAs might be
futile hence, provide a clear and explicit mechanism to choose whether
to build the in-tree IPA for the platform or not.
By default, all in-tree IPAs are built when a matching Pipeline handler
is also enabled.
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
CameraSensorInfo structure is designed to pass in camera sensor related
information from pipeline-handler to IPA. Since the pipeline-handler
and IPA are connected via mojom IPC IPA interface, the interface
itself provides a more suitable placement of CameraSensorInfo,
instead of camera_sensor.h (which is a libcamera internal header
ultimately, at this point).
As CameraSensorInfo is already defined in core.mojom, it is just
a matter of removing [skipHeader] tag to allow code-generation
of CameraSensorInfo.
Finally, update header paths to include CameraSensorInfo definition
from IPA interfaces instead of "libcamera/internal/camera_sensor.h".
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The controller algorithms currently run on every frame provided to the
IPA by the pipeline handler. This may be undesirable for very fast fps
operating modes where it could significantly increase the computation
cycles (per unit time) without providing any significant changes to the
IQ parameters. The added latencies could also cause dropped frames.
Pass the FrameBuffer timestamp to the IPA through the controls. This
timestamp will be used to rate-limit the controller algorithms to run
with a minimum inter-frame time given by a compile time constant,
currently set to 16.66ms. On startup, we don't rate-limit the algorithms
until after the number of frames required for convergence.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Replace std::lock_guard with std::scoped_lock. When locking a single
mutex, both are functionally the same. When locking two mutexes in the
operator= overload, the scoped_lock uses a deadlock avoidance algorithm
to avoid deadlock.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Rearrange header includes to be in alphabetical order.
Add whitespace to class definition to match libcamera coding guidelines.
Fix a typo in the comment showing an example of scoped locks.
There are no functional changes in this commit.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
CamHelpers get virtual Prepare() and Process() methods, running just
before and just after the ISP, just like Raspberry Pi Algorithms.
The Prepare() method is able to parse the register dumps in embedded
data buffers, and can be specialised to perform custom processing when
necessary.
The IPA itself only needs to call the new Prepare() and Process()
methods. It must fill in the DeviceStatus from the controls first, in
case the Prepare() method does not supply its own values.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Define libcamera specific system paths for use within the library code
base. These can be used to identify system configuration files and
shared data.
Also, make use of the new system path variables and map the ipa paths
upon the libcamera system path, making use of the new shorter syntax for
join_paths().
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Implement basic auto-exposure (AE) and auto-gain (AG) correction functions.
The functions computeTargetExposure() and computeGain() are adapted from
the Raspberry Pi AGC implementation to suit the IPU3 structures, and
filtering is added to reduce visible stepsize when there are large
exposure changes.
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Tested-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The IPA will locally modify the parameters before they are passed down
to the ImgU. Use a local parameter object to give a reference to those
algorithms.
Inherit from the Algorithm class to implement basic AWB functions.
The configure() call will set exposure and gain to their minimum value,
so while AGC is not there, the frames will be dark.
Once AWB is done, a color temperature is estimated and a default CCM matrix
will be used (yet to be tuned).
Implement a basic "grey-world" AWB algorithm just for demonstration purpose.
The BDS output size is passed by the pipeline handler to the IPA.
The best grid is then calculated to maximize the number of pixels taken
into account in each cells.
As commented in the source code, it can be improved, as it has (at least)
one limitation: if a cell is big (say 128 pixels wide) and indicated as
saturated, it won't be taken into account at all.
Maybe is it possible to have a smaller one, at the cost of a few pixels
to lose, in which case we can center the grid using the x_start and
y_start parameters.
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Tested-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This class will be used at least by AGC algorithm when quantiles are
needed for example. It stores a cumulative frequency histogram. Going from
cumulative frequency back to per-bin values is a single subtraction, while
going the other way is a loop.
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Tested-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Do not send events back to the pipeline handler, while the IPA thread
has not been started.
If information needs to be returned from configure it should be returned
directly.
Move the initial call to setControls() out of configure() and into the
start() method which is called after the IPA running_ state is updated.
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Sebastian Fricke <sebastian.fricke@posteo.net>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The configure operation is synchronous and should not send events back
to the pipeline handler.
If information needs to be returned from configure it should be handled
through the interface directly.
Move the initial call to setControls() out of configure() and into the
start() method which is called after the IPA running_ state is updated.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Rename ConfigInput to IPAConfig to be more consistent with the naming,
and remove ConfigInput::op, as it is never used.
Replace ConfigOutput with a ControlList type, as that is the only return
type from ipa::configure().
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Separate out the in and out parameters in ipa::start() as they are not
the same. This function now takes in a ControlList and returns out a
struct StartConfig which holds a ControlList and drop frame count for
the pipeline handler to action.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Move the opening of the CamHelper from ipa::configure() to ipa::init().
This allows the pipeline handler to get the sensor specific parameters
in pipeline_handler::match() where the ipa is initialised.
Having the sensor parameters available earlier will allow selective
use of the embedded data node in a future change.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The IPU3 IPA will need the BDS configuration when the AWB/AGC algorithm
will be integrated.
In order to do that, the configure() interface needs to be modified.
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
This patch adds support for the Soho Enterprises SE327M12 module. The
sensor is an imx327 which therefore uses the imx290 kernel driver and
CamHelper.
To use this module and camera tuning, place the following in the
/boot/config.txt file:
dtoverlay=imx290,clock-frequency=37125000
and then the existing imx290.json file must be over-written with
se327m12.json
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
In kernel 5.11 the rkisp1 uapi had changed to support different hardware
revisions. Currently only revision 10 is supported by the rkisp1 IPA and
therefore 'init' should fail if the revision is not 10.
This changes depends on the kernel driver reporting the hardware
revision, and thus requires the rkisp1 driver from v5.11 or newer.
Signed-off-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The IPA of rkisp1 relies on some of the camera's controls.
Therefore it can't work if those controls are not given.
Return -EINVAL from 'configure' in that case.
Also return error from the pipeline's 'configure' method
if the IPA configure fails.
Signed-off-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Now that we support returning int directly in addition to other output
parameters, improve the configure() function in the raspberrypi IPA
interface.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
imx290 and imx327 share the same kernel driver (imx290.c) and are
therefore both recognised here as "imx290". We add the necessary
CamHelper for these sensors, as well as a camera tuning file.
The tuning was done with an Innomaker STARVIS IMX327LQR module. These
have no IR cut filter so there is no proper colour tuning. However,
you should obtain reasonable results for most modules using this
sensor. Specific tunings for further modules can always be added
subsequently.
To use this sensor on the Raspberry Pi platform, please add
dtoverlay=imx290,clock-frequency=74250000
into your /boot/config.txt file.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
For some sensors (e.g. imx477) we need to update the vblanking on the
frame before the exposure. For this reason the GetDelays method must
also return the number of frame delays for the vblanking control.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
With the recent change to pass a ControlList to the IPA with exposure
and gain values used for a frame, RPiController::MdParserRPi is not
needed any more. Remove all traces of it.
The derived CamHelper objects now pass nullptr values for the parser to
the base CamHelper class when sensors do not use metadata.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
When running with sensors that had no embedded data, the pipeline handler
would fill a dummy embedded data buffer with gain/exposure values, and
pass this buffer to the IPA together with the bayer buffer. The IPA would
extract these values for use in the controller algorithms.
Rework this logic entirely by having a new RPiCameraData::BayerFrame
queue to replace the existing bayer queue. In addition to storing the
FrameBuffer pointer, this also stores all the controls tracked by
DelayedControls for that frame in a ControlList. This includes include
exposure and gain values. On signalling RPi::IPA_EVENT_SIGNAL_ISP_PREPARE
IPA event, the pipeline handler now passes this ControlList from the
RPiCameraData::BayerFrame queue.
The IPA now extracts the gain and exposure values from the ControlList
instead of using RPiController::MdParserRPi to parse the embedded data
buffer.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The rkisp1 driver has received support for newer ISP versions, which
changes its userspace API and ABI. Adapt to the API change. This
requires kernel v5.11 or newer, or backporting the corresponding rkisp1
changes to older kernels.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Regions of the image where the statistics are not trusted should not
be put in the zones_ list.
This also means that the AWB may fail to run initially, so
async_results_ needs to be initialised to some kind of default.
Finally, the RGB zone structures can be zero-initialised safely now,
the previous value of -1 would seem unusual.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Jean-Michel Hautbois
Paul Elder
Umang Jain
Naushir Patuck
David Plowman
Kieran Bingham
Sebastian Fricke
Dafna Hirschfeld
Laurent Pinchart