The IPU3 IPA will need the BDS configuration when the AWB/AGC algorithm
will be integrated.
In order to do that, the configure() interface needs to be modified.
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
When the SimplePipeline is setting up its data and media pipeline in the
SimpleCameraData constructor, it merely tries to find the first valid
pad and link to the next entity, starting from the camera sensor.
Following this path may not always lead to a valid capture device and
therefore the setup will fail on some machines. This is for example an
issue when using the SimplePipeline on an i.MX-6Q with its i.MX IPU.
This commit implements a different approach to setup the media-pipeline
by finding the shortest path to a valid capture device, using the
breadth-first search algorithm. On i.MX6Q, the shortest path has a good
chance to be the path from the sensor to the CSI capture device, as
other paths may involve image converters, encoders or other IPU blocks
and will have therefore more nodes.
Signed-off-by: Marian Cichy <m.cichy@pengutronix.de>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
If the ISP::Output0 stream has not been configured, we must enable it
with a default format and resolution for internal use. This is to allow
the pipeline handler data flow to be consistent, and allow the IPA to
receive statistics for the frame.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
It is possible for the application to call pipeline_handler::configure()
multiple times, which would attempt to open the Unicam embedded data
node on every call. This would cause a warning message as the node
would have already been opened. Avoid this by tracking if the node
has previously been opened.
Note that this is a temporary fix since the open call for the Unicam
embedded data node will be moved from pipeline_handler::configure() to
pipeline_handler::match().
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
If an exposure time change adjusts the vblanking limits, and we set both
VBLANK and EXPOSURE controls through the VIDIOC_S_EXT_CTRLS ioctl, the
latter may fail if the value is outside of the limits calculated by the
old VBLANK value. This is a limitation in V4L2 and cannot be fixed by
setting VBLANK before EXPOSURE in a single VIDIOC_S_EXT_CTRLS ioctl.
The workaround here is to have the DelayedControls object mark the
VBLANK control as "priority write", which then write VBLANK separately
from (and ahead of) any other controls. This way, the sensor driver will
update the EXPOSURE control with new limits before the new values is
presented, and will thus be seen as valid.
To support this, a new struct DelayedControls::ControlParams is used in
the constructor to provide the control delay value as well as the
priority write flag.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
[Kieran: Fix up trivial comments, merge conflicts]
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In kernel 5.11 the rkisp1 uapi had changed to support different hardware
revisions. Currently only revision 10 is supported by the rkisp1 IPA and
therefore 'init' should fail if the revision is not 10.
This changes depends on the kernel driver reporting the hardware
revision, and thus requires the rkisp1 driver from v5.11 or newer.
Signed-off-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The IPA of rkisp1 relies on some of the camera's controls.
Therefore it can't work if those controls are not given.
Return -EINVAL from 'configure' in that case.
Also return error from the pipeline's 'configure' method
if the IPA configure fails.
Signed-off-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Register the FrameDurations control in the IPU3 pipeline handler
computed using the vertical blanking limits and the sensor
pixel rate as parameters.
The FrameDurations control limits should be updated everytime a new
configuration is applied to the sensor.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
The controls' limits initialized by the IPU3 pipeline handler depend
on the sensor configuration. In order to compute controls using a known
state apply to the sensor a configuration equal to its own resolution.
Move the \todo note regarding the controls' limits dependency on the
sensor configuration at the beginning of the function and remove the
other redundant ones in the function body.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Now that we support returning int directly in addition to other output
parameters, improve the configure() function in the raspberrypi IPA
interface.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
When the IPU3Frames completes, it deletes the internal info storage.
This storage contains the pointer to the Request, but in some cases the
pointer was being accessed after the info structure was removed.
Ensure that the Request is obtained before attempting to complete to
obtain a valid pointer.
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Camera sensors can include an ISP. For instance, the AP1302 external ISP
can be connected to up to two raw camera sensors, and the combination of
the sensors and ISP is considered as a (smart) camera sensor from
libcamera's point of view.
The CameraSensor class has limited support for this already. Extend the
simple pipeline handler to support such sensors, by using the media
entity corresponding to the ISP instead of the raw camera sensor's
entity.
We don't need to handle the case where an entity in the SoC would expose
the MEDIA_ENT_F_PROC_VIDEO_ISP function, as pipeline containing an ISP
would have a dedicated pipeline handler.
The implementation is limited as it won't support other multi-entity
camera sensors (such as CCS). While this would be worth supporting, we
don't have a test platform with a CCS-compatible sensor at this point,
so let's not over-engineer the solution. Extending support to CCS (and
possibly other sensor topologies) will likely involve helpers that can
be used by other pipeline handlers (such as generic graph walk helpers
for instance) and extensions to the CameraSensor class.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
The pipeline handler would enable and use the Unicam embedded data stream
even if the sensor did not support it. This was to allow a means to
pass exposure and gain values for the frame to the IPA in a synchronised
way.
The recent changes to get the pipeline handler to pass a ControlList
with exposure and gain values means this is no longer required. Disable
the use of the embedded data stream when a sensor does not support it.
This change also removes the mappedEmbeddedBuffers_ map as it is no
longer used.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
When running with sensors that had no embedded data, the pipeline handler
would fill a dummy embedded data buffer with gain/exposure values, and
pass this buffer to the IPA together with the bayer buffer. The IPA would
extract these values for use in the controller algorithms.
Rework this logic entirely by having a new RPiCameraData::BayerFrame
queue to replace the existing bayer queue. In addition to storing the
FrameBuffer pointer, this also stores all the controls tracked by
DelayedControls for that frame in a ControlList. This includes include
exposure and gain values. On signalling RPi::IPA_EVENT_SIGNAL_ISP_PREPARE
IPA event, the pipeline handler now passes this ControlList from the
RPiCameraData::BayerFrame queue.
The IPA now extracts the gain and exposure values from the ControlList
instead of using RPiController::MdParserRPi to parse the embedded data
buffer.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Allow support for multiple streams on a per-device basis. The decision
should be made based on the ability of the converter to run multiple
times within the duration of one frame. Hardcode it in
SimplePipelineInfo for now.
We may later compute the number of supported streams dynamically based
on the requested configuration, using converter bandwidth information
instead of a hardcoded fixed value.
All platforms are currently limited to a single stream until they get
successfully tested with multiple streams.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Extend the SimpleCameraConfiguration to support multiple streams, using
the multi-stream capability of the SimpleConverter class. Wiring up
multi-stream support in the other pipeline handler operations will come
in further commits.
To keep the code simple, require all streams to use the converter if any
stream needs it. It would be possible to generate one stream without
conversion (provided the format and size match what the capture device
can generate), and this is left as a future optimization.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Store the list of converter output formats in the Configuration
structure, to be used to implement multi-stream support. As the
Configuration structure grows bigger, avoid duplicating it in the
formats_ map for each supported pixel format by storing it in a configs_
vector instead, and storing pointers only in the map.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
As the pipeline configuration is selected in
SimpleCameraConfiguration::validate() already, cache it in the
SimpleCameraConfiguration instead of looking it up in
SimplePipelineHandler::configure(). This makes little difference at the
moment, but will save duplication of more complex logic between
validate() and configure() when adding support for multiple streams.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The simple pipeline handler has grown over time, and isn't that simple
anymore that it can easily be understood by an unfamiliar reader.
Document the design to explicitly state the expectations of the pipeline
handler, and to explain how it operates.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
While the M2M device backing the converter doesn't support multiple
streams natively, it can be run once per stream to produce multiple
outputs from the same input, with different output formats and sizes.
To support this, create a class to model a stream and move control of
the M2M device to the Stream class. The SimpleConverter class then
creates stream instances and iterates over them. Each stream needs its
own instance of the V4L2M2MDevice, to support different output
configurations. The SimpleConverter class retains a device instance to
support the query operations.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The SimpleConverter API signals completion of input and output buffer
pairs. This unnecessarily delays requeueing the input buffer to the
video capture queue until the output buffer completes, and also delays
signalling request completion until the input buffer completes. While
this shouldn't cause large delays in practice, it will also not scale
when multi-stream support will be added to the converter class.
To address the current issue and prepare for the future, decouple
signalling of input and output buffers completion.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Simplify the SimpleConverter interface by opening the M2M device in the
constructor. The explicit call to open() is replaced by a check through
a new isValid() function, and the unused close() function is removed.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The number of buffers on the input and output of the converter don't
necessarily need to match. Use the buffer count from the input and
output configuration respectively. This removes the need to pass the
buffer count to the start() function, which brings it closer to the
pipeline handler API.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Group the configuration parameters for the converter input in a
StreamConfiguration instance. This makes the configure() function
signature cleaner, and will allow passing additional parameters (such as
stride and buffer count).
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The SimpleConverter class has different sets of functions, related to
static queries, device configuration and runtime operation. Group the
query functions together. While at it, swap the arguments to the
strideAndFrameSize() function to match the order in which pixel format
and size are usually specified.
No functional change is included.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The stride (and frame size) calculation has been moved from configure
time to configuration validate time by commit 89fb1efac2 ("libcamera: simple:
Fill stride and frameSize at config validation"). This change has
however left one stray setting of the stride when configuring the
converter. Fix it.
While at it, turn the SimpleConverter::configure() output configuration
argument to a const reference to emphasize it can't be null and isn't
modified by the function, and rename it from cfg to outputCfg to make
its purpose clearer.
Fixes: 89fb1efac2 ("libcamera: simple: Fill stride and frameSize at config validation")
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The flag that is used to record if the sensor driver supports the H/V flip
controls is initialized at camera creation time, but it was not
initialized by the IPU3Camera constructor, resulting in erroneous
values that might break capture for sensor that do not support flipping.
Fix this by initializing the flag in the class constructor.
Fixes: 6c4ce7de30 ("libcamera: ipu3: Add rotation to ipu3 pipeline")
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Fabian Wüthrich <me@fabwu.ch>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
The addition of the new IPA IPC mechanism compiles pipeline-specific
headers to define the interface between the pipeline and the IPA.
This was optimised in 08ce394465 ("meson: ipa, proxy: Only build
proxies for enabled pipelines") to only build for enabled pipelines,
however the tests directly use the VIMC pipeline handler, and require
it to be built.
Create a local variable to store the requested pipelines from the user
configuration and extend the enabled pipelines to ensure that VIMC is
always enabled if the tests are also enabled
Fixes: 08ce394465 ("meson: ipa, proxy: Only build proxies for enabled pipelines")
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Tested-by: Sebastian Fricke<sebastian.fricke@posteo.net>
Use the same transformation logic as in the raspberry pipeline to
implement rotations in the ipu3 pipeline.
Tested on a Surface Book 2 with an experimental driver for OV5693.
Signed-off-by: Fabian Wüthrich <me@fabwu.ch>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Acked-by: Fabian Wüthrich <me@fabwu.ch>
Usage of std::move() on a const lvalue will produce a const rvalue,
which the compiler will bind to the const lvalue reference of the
constructor or assignement operator. This results in the object being
copied, not moved. Drop std::move() in that case as it's misleading.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
When a Request is completed upon receiving the IPA produced metadata,
the metadata associated with the Request were over-written, deleting
the information set, in example, at ImgU output buffer completion.
If any additional Request metadata should be registered by inspecting
the IPA produced metadata it has to be done without deleting the already
registered entry.
Fix this by replacing the metadata over-write with a todo entry.
This change fixes CTS which was broken due to missing metadata in
the completed requests.
Fixes: 9708f49fec ("libcamera: ipu3: Share parameter and statistic buffers with IPA")
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
In order to get the stats back, the imgu subdev needs to have the
V4L2_CID_INTEL_IPU3_MODE control set.
Set it to video mode by default to get the stats at each frame.
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
[Kieran: Updated message to match Laurent's suggestion]
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Jean-Michel Hautbois
Marian Cichy
Naushir Patuck
Dafna Hirschfeld
Jacopo Mondi
Paul Elder
David Plowman
Kieran Bingham
Laurent Pinchart
Fabian Wüthrich