The V4l2 buffer cache allows us to map incoming buffers to an instance
of the V4L2 buffer required to actually queue.
If the cache_ is not available, then the buffers required to allow
queuing to a device have been released, and this indicates an issue at
the pipeline handler.
This could be a common mistake, as it could happen if a pipeline handler
always requeues buffers to the device after they complete, without
checking if they are cancelled.
Catch any invalid queueing of buffers to the V4L2 video device when
resources have been released by adding a Fatal log message to highlight
the error during development.
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Ensure that the buffers are associated with the request even if they are
used internally to be able to correctly map back to the resources they
are being used to fulfil.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Ensure that checks on resource availability are handled first, and then
operate on the queues only after the resources are confirmed as
available.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When the imguOutputBufferReady() detects a cancelled frame, it is
reporting that the metadata has been processed in order to be able to
complete the cancelled request.
This causes the FrameInfo to be completed and deleted early, but then an
active buffer on the IMGU can complete and be unable to find the
FrameInfo for it to complete correctly.
Do not mark metadataProcessed early on the event that a single buffer is
detected as cancelled. The stopping of the V4L2 devices will ensure
that all queued buffers are returned to us and we can follow the normal
and expected shutdown sequence.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The IPA should be stopped before the hardware devices to ensure that
all asynchronous actions have completed within the IPA before resources
are removed and released.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Rename ConfigInput to IPAConfig to be more consistent with the naming,
and remove ConfigInput::op, as it is never used.
Replace ConfigOutput with a ControlList type, as that is the only return
type from ipa::configure().
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Separate out the in and out parameters in ipa::start() as they are not
the same. This function now takes in a ControlList and returns out a
struct StartConfig which holds a ControlList and drop frame count for
the pipeline handler to action.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Move the opening of the CamHelper from ipa::configure() to ipa::init().
This allows the pipeline handler to get the sensor specific parameters
in pipeline_handler::match() where the ipa is initialised.
Having the sensor parameters available earlier will allow selective
use of the embedded data node in a future change.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
There are std::unique_ptr rvalue reference arguments. They are
intended to pass the ownership to the functions. In the case,
it is right to let the argument be std::unique_ptr value and
pass by std::move().
Signed-off-by: Hirokazu Honda <hiroh@chromium.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
After commit 4671911df0 ("pipeline: simple: Use breadth-first search
to setup media pipeline"), the explanation in the SimplePipeline
documentation how the handler tries to find a valid path to capture
device does not reflect the reality anymore. Update the text to the new
situation.
Fixes: 4671911df0 ("pipeline: simple: Use breadth-first search to setup media pipeline")
Signed-off-by: Marian Cichy <m.cichy@pengutronix.de>
Reviewed-by: Sebastian Fricke <sebastian.fricke@posteo.net>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The documentation of the controls() method refers to the methods
ControlList::operator[]() and ControlList::update(), which do not exist.
Instead refer to ControlList::get() and ControlList::set() to achieve a
similar documentation.
Fixes: a8c40942b9 ("libcamera: controls: Improve the API towards applications")
Signed-off-by: Marian Cichy <m.cichy@pengutronix.de>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The IPU3 IPA will need the BDS configuration when the AWB/AGC algorithm
will be integrated.
In order to do that, the configure() interface needs to be modified.
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
When the SimplePipeline is setting up its data and media pipeline in the
SimpleCameraData constructor, it merely tries to find the first valid
pad and link to the next entity, starting from the camera sensor.
Following this path may not always lead to a valid capture device and
therefore the setup will fail on some machines. This is for example an
issue when using the SimplePipeline on an i.MX-6Q with its i.MX IPU.
This commit implements a different approach to setup the media-pipeline
by finding the shortest path to a valid capture device, using the
breadth-first search algorithm. On i.MX6Q, the shortest path has a good
chance to be the path from the sensor to the CSI capture device, as
other paths may involve image converters, encoders or other IPU blocks
and will have therefore more nodes.
Signed-off-by: Marian Cichy <m.cichy@pengutronix.de>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
If the ISP::Output0 stream has not been configured, we must enable it
with a default format and resolution for internal use. This is to allow
the pipeline handler data flow to be consistent, and allow the IPA to
receive statistics for the frame.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
It is possible for the application to call pipeline_handler::configure()
multiple times, which would attempt to open the Unicam embedded data
node on every call. This would cause a warning message as the node
would have already been opened. Avoid this by tracking if the node
has previously been opened.
Note that this is a temporary fix since the open call for the Unicam
embedded data node will be moved from pipeline_handler::configure() to
pipeline_handler::match().
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
There was an off-by-one error in DelayedControls::get() when picking
controls from the queue to return back to the pipeline handler.
This is only noticeable as small oscillations in brightness when closely
viewing frame while AGC is running. The old StaggeredCtrl did not show
this error as the startup queuing mechanism has changed in
DelayedControls.
Fix this by indexing to the correct position in the queue.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reported-by: David Plowman <david.plowman@raspberrypi.com>
Fixes: 3d4b7b0059 ("libcamera: delayed_controls: Add helper for controls that apply with a delay")
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In DelayedControls::applyControls(), the controls queue would be
extended via a no-op control push to fill the intermittent slots with
unchanged control values. This is needed so that we read back unchanged
control values correctly on every frame.
However, there was one additional no-op performed on every frame that is
not required, meaning that any controls queued by the pipeline handler
would have their write delayed by one further frame. The original
StaggeredCtrl did not do this, as it only had one index to manage,
whereas DelayedControls uses two.
Remove this last no-op push so that the pipeline_handler provided
controls would be written on the very next frame if possible. As a
consequence, we must mark the control update as completed within the
DelayedControls::applyControls() loop, otherwise it might get reused
when cycling through the ring buffer.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reported-by: David Plowman <david.plowman@raspberrypi.com>
Fixes: 3d4b7b0059 ("libcamera: delayed_controls: Add helper for controls that apply with a delay")
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
On DelayedControls::reset(), the values retrieved from the sensor device
were added to the queues with the updated flag set to true. This would
cause the helper to write out the value to the device again on the first
DelayedControls::applyControls() call. This is unnecessary, as the
controls written are identical to what is stored in the device driver.
Fix this by explicitly setting the update flag to false in
DelayedControls::reset() when adding the controls to the queue.
Additionally, use the Info() constructor when adding items to the queue
for consistency.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Fixes: 3d4b7b0059 ("libcamera: delayed_controls: Add helper for controls that apply with a delay")
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
If an exposure time change adjusts the vblanking limits, and we set both
VBLANK and EXPOSURE controls through the VIDIOC_S_EXT_CTRLS ioctl, the
latter may fail if the value is outside of the limits calculated by the
old VBLANK value. This is a limitation in V4L2 and cannot be fixed by
setting VBLANK before EXPOSURE in a single VIDIOC_S_EXT_CTRLS ioctl.
The workaround here is to have the DelayedControls object mark the
VBLANK control as "priority write", which then write VBLANK separately
from (and ahead of) any other controls. This way, the sensor driver will
update the EXPOSURE control with new limits before the new values is
presented, and will thus be seen as valid.
To support this, a new struct DelayedControls::ControlParams is used in
the constructor to provide the control delay value as well as the
priority write flag.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
[Kieran: Fix up trivial comments, merge conflicts]
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In kernel 5.11 the rkisp1 uapi had changed to support different hardware
revisions. Currently only revision 10 is supported by the rkisp1 IPA and
therefore 'init' should fail if the revision is not 10.
This changes depends on the kernel driver reporting the hardware
revision, and thus requires the rkisp1 driver from v5.11 or newer.
Signed-off-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The IPA of rkisp1 relies on some of the camera's controls.
Therefore it can't work if those controls are not given.
Return -EINVAL from 'configure' in that case.
Also return error from the pipeline's 'configure' method
if the IPA configure fails.
Signed-off-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Register the FrameDurations control in the IPU3 pipeline handler
computed using the vertical blanking limits and the sensor
pixel rate as parameters.
The FrameDurations control limits should be updated everytime a new
configuration is applied to the sensor.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
The controls' limits initialized by the IPU3 pipeline handler depend
on the sensor configuration. In order to compute controls using a known
state apply to the sensor a configuration equal to its own resolution.
Move the \todo note regarding the controls' limits dependency on the
sensor configuration at the beginning of the function and remove the
other redundant ones in the function body.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Now that we support returning int directly in addition to other output
parameters, improve the configure() function in the raspberrypi IPA
interface.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Since commit 96aecfe365 ("libcamera: camera_sensor: Use active area
size as resolution") the CameraSensor::resolution() method returned the
sensor's active pixel area size.
As the CameraSensor::resolution() method is widely used in the library
code base to retrieve the maximum frame size the sensor can produce,
in case it is smaller than the pixel area size the returned size cannot
be used to configure the sensor correctly.
Fix this by returning the maximum frame resolution the sensor can
produce, or the pixel area size in case the sensor embeds and ISP that
can upscale and the supported maximum frame size is thus larger that
the pixel array size.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
When the IPU3Frames completes, it deletes the internal info storage.
This storage contains the pointer to the Request, but in some cases the
pointer was being accessed after the info structure was removed.
Ensure that the Request is obtained before attempting to complete to
obtain a valid pointer.
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add tracing to the base pipeline handler class to track when requests are queued.
Tracing is already available for other Request operations, but queuing a Request
is not an operation handled by the Request itself.
Add the tracepoint to the PipelineHandler::queueRequest() so the lifetime of a
Request can be viewed when tracing.
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When a pipeline handler completes a request, the request itself is not
deleted by libcamera, and the application regains control over the
object. It may choose to delete the Request, or re-use it.
Clarify this in the comment by removing the declaration that the Request
is deleted, but state that it is no longer managed by the pipeline
handler and must not be accessed further after this function returns.
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Umang Jain <email@uajain.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Requests should only be completed from the RequestPending state.
Requests which are completed from the RequestCancelled, or RequestComplete
state, will indicate that a double-complete has been called on the Request,
or that it has been used internally after it has been given back to the
application.
Ensure that this can be caught early if it occurs by enforcing the state
required with an assert.
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Umang Jain <email@uajain.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Camera sensors can include an ISP. For instance, the AP1302 external ISP
can be connected to up to two raw camera sensors, and the combination of
the sensors and ISP is considered as a (smart) camera sensor from
libcamera's point of view.
The CameraSensor class has limited support for this already. Extend the
simple pipeline handler to support such sensors, by using the media
entity corresponding to the ISP instead of the raw camera sensor's
entity.
We don't need to handle the case where an entity in the SoC would expose
the MEDIA_ENT_F_PROC_VIDEO_ISP function, as pipeline containing an ISP
would have a dedicated pipeline handler.
The implementation is limited as it won't support other multi-entity
camera sensors (such as CCS). While this would be worth supporting, we
don't have a test platform with a CCS-compatible sensor at this point,
so let's not over-engineer the solution. Extending support to CCS (and
possibly other sensor topologies) will likely involve helpers that can
be used by other pipeline handlers (such as generic graph walk helpers
for instance) and extensions to the CameraSensor class.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
The pipeline handler would enable and use the Unicam embedded data stream
even if the sensor did not support it. This was to allow a means to
pass exposure and gain values for the frame to the IPA in a synchronised
way.
The recent changes to get the pipeline handler to pass a ControlList
with exposure and gain values means this is no longer required. Disable
the use of the embedded data stream when a sensor does not support it.
This change also removes the mappedEmbeddedBuffers_ map as it is no
longer used.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
When running with sensors that had no embedded data, the pipeline handler
would fill a dummy embedded data buffer with gain/exposure values, and
pass this buffer to the IPA together with the bayer buffer. The IPA would
extract these values for use in the controller algorithms.
Rework this logic entirely by having a new RPiCameraData::BayerFrame
queue to replace the existing bayer queue. In addition to storing the
FrameBuffer pointer, this also stores all the controls tracked by
DelayedControls for that frame in a ControlList. This includes include
exposure and gain values. On signalling RPi::IPA_EVENT_SIGNAL_ISP_PREPARE
IPA event, the pipeline handler now passes this ControlList from the
RPiCameraData::BayerFrame queue.
The IPA now extracts the gain and exposure values from the ControlList
instead of using RPiController::MdParserRPi to parse the embedded data
buffer.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Camera sensors can include an ISP, which may be reported as a separate
entity from the pixel array in the media graph.
Support such sensors by accepting MEDIA_ENT_F_PROC_VIDEO_ISP as a valid
entity type. This allows using sensors that can be fully (or at least
meaningfully) configured through the ISP's source pad only. Sensors that
require further configuration, on the ISP sink pad and/or on the pixel
array's source pad, will require further extension to the CameraSensor
class.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
When a sensor can upscale the image, the native sensor resolution isn't
equal to the largest size reported by the sensor. Use the active area
size instead, and default it to the largest enumerated size if the crop
rectangle targets are not supported.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Implement and expose the symbol and functions that the new cros camera
API requires. Since we don't actually need them, leave them empty.
Update meson accordingly.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Allow support for multiple streams on a per-device basis. The decision
should be made based on the ability of the converter to run multiple
times within the duration of one frame. Hardcode it in
SimplePipelineInfo for now.
We may later compute the number of supported streams dynamically based
on the requested configuration, using converter bandwidth information
instead of a hardcoded fixed value.
All platforms are currently limited to a single stream until they get
successfully tested with multiple streams.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Extend the SimpleCameraConfiguration to support multiple streams, using
the multi-stream capability of the SimpleConverter class. Wiring up
multi-stream support in the other pipeline handler operations will come
in further commits.
To keep the code simple, require all streams to use the converter if any
stream needs it. It would be possible to generate one stream without
conversion (provided the format and size match what the capture device
can generate), and this is left as a future optimization.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
