For all frame indexes, use the same sequence number as generated
by the Request object.
This allows clear matching of what operations occurred to which request.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When the camera is being stop()ped, active requests will complete. These
may trigger an application to re-queue those requests to the camera but
that is not permitted, and is an error in the application.
Extend the camera state to include a stopping state which is entered as
soon as a call to stop() is made. At this point, any request queued will
be rejected with a warning, while any pending requests are either
successfully completed or cancelled.
When the pipeline handler has finished stopping, the camera state will
transition to the CameraConfigured state where it can begin to accept
requests again, and be restarted.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The camera object has a state machine to ensure calls are only made
when in the correct state. It isn't easy to identify where things happen
when assertions fail so add extra information to make this clearer.
The error level of the isAccessAllowed is raised from Debug to Error as
this is important information for applications to know if they have made
a request in an invalid state.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
All requests must have completed before the Camera has fully stopped.
Requests completing when the camera is not running represent an internal
pipeline handler bug.
Trap this event with a fatal error.
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Provide a sequence number on Requests which are added by the pipeline
handler.
Each pipeline handler keeps a requestSequence per CameraData and
increments everytime a request is queued on that camera.
The sequence number is associated with the Request and can be utilised
for assisting with debugging, and printing the queueing sequence of in
flight requests.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The FrameBuffer class is only friends with Request so that the request
can be associated with the buffers.
FrameBuffer already has a helper to setRequest(), so let's use that
directly instead.
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Asynchronous tasks can only be submitted while the IPA is running.
Further more, the shutdown sequence can not be tracked with a simple
running flag. We can also be in the state 'Stopping' where we have not
yet completed all events, but we must not commence anything new.
Refactor the running_ boolean into a stateful enum to track this.
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
A range-based for loop whose range expression is an array of char
pointers and range variable declaration is a const reference to a
std::string creates a temporary string from the char pointer and binds
the range variable reference to it. This creates a const reference to a
temporary, which is valid in C++, and extends the lifetime of the
temporary to the lifetime of the reference.
However, lifetime extension in range-based for loops is considered as a
sign of a potential issue, as a temporary is created for every
iteration, which can be costly, and the usage of a reference in the
range declaration doesn't make it obvious that the code isn't simply
binding a reference to an existing object. gcc 11, with the
-Wrange-loop-construct option, flags this:
uvcvideo.cpp:432:33: error: loop variable 'name' of type 'const string&' {aka 'const std::__cxx11::basic_string<cha
r>&'} binds to a temporary constructed from type 'const char* const' [-Werror=range-loop-construct]
| 432 | for (const std::string &name : { "idVendor", "idProduct" }) {
| | ^~~~
To please the compiler, make the range variable a const char *. This may
bring a tiny performance improvement, as the name is only used once, in
a location where the compiler can use
operator+(const std::string &, const char *)
instead of
operator+(const std::string &, const std::string &)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Use a const char * type instead of auto, and update the commit message
accordingly.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The V4l2 buffer cache allows us to map incoming buffers to an instance
of the V4L2 buffer required to actually queue.
If the cache_ is not available, then the buffers required to allow
queuing to a device have been released, and this indicates an issue at
the pipeline handler.
This could be a common mistake, as it could happen if a pipeline handler
always requeues buffers to the device after they complete, without
checking if they are cancelled.
Catch any invalid queueing of buffers to the V4L2 video device when
resources have been released by adding a Fatal log message to highlight
the error during development.
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Ensure that the buffers are associated with the request even if they are
used internally to be able to correctly map back to the resources they
are being used to fulfil.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Ensure that checks on resource availability are handled first, and then
operate on the queues only after the resources are confirmed as
available.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When the imguOutputBufferReady() detects a cancelled frame, it is
reporting that the metadata has been processed in order to be able to
complete the cancelled request.
This causes the FrameInfo to be completed and deleted early, but then an
active buffer on the IMGU can complete and be unable to find the
FrameInfo for it to complete correctly.
Do not mark metadataProcessed early on the event that a single buffer is
detected as cancelled. The stopping of the V4L2 devices will ensure
that all queued buffers are returned to us and we can follow the normal
and expected shutdown sequence.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The IPA should be stopped before the hardware devices to ensure that
all asynchronous actions have completed within the IPA before resources
are removed and released.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Rename ConfigInput to IPAConfig to be more consistent with the naming,
and remove ConfigInput::op, as it is never used.
Replace ConfigOutput with a ControlList type, as that is the only return
type from ipa::configure().
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Separate out the in and out parameters in ipa::start() as they are not
the same. This function now takes in a ControlList and returns out a
struct StartConfig which holds a ControlList and drop frame count for
the pipeline handler to action.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Move the opening of the CamHelper from ipa::configure() to ipa::init().
This allows the pipeline handler to get the sensor specific parameters
in pipeline_handler::match() where the ipa is initialised.
Having the sensor parameters available earlier will allow selective
use of the embedded data node in a future change.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
There are std::unique_ptr rvalue reference arguments. They are
intended to pass the ownership to the functions. In the case,
it is right to let the argument be std::unique_ptr value and
pass by std::move().
Signed-off-by: Hirokazu Honda <hiroh@chromium.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
After commit 4671911df0 ("pipeline: simple: Use breadth-first search
to setup media pipeline"), the explanation in the SimplePipeline
documentation how the handler tries to find a valid path to capture
device does not reflect the reality anymore. Update the text to the new
situation.
Fixes: 4671911df0 ("pipeline: simple: Use breadth-first search to setup media pipeline")
Signed-off-by: Marian Cichy <m.cichy@pengutronix.de>
Reviewed-by: Sebastian Fricke <sebastian.fricke@posteo.net>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The documentation of the controls() method refers to the methods
ControlList::operator[]() and ControlList::update(), which do not exist.
Instead refer to ControlList::get() and ControlList::set() to achieve a
similar documentation.
Fixes: a8c40942b9 ("libcamera: controls: Improve the API towards applications")
Signed-off-by: Marian Cichy <m.cichy@pengutronix.de>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The IPU3 IPA will need the BDS configuration when the AWB/AGC algorithm
will be integrated.
In order to do that, the configure() interface needs to be modified.
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
When the SimplePipeline is setting up its data and media pipeline in the
SimpleCameraData constructor, it merely tries to find the first valid
pad and link to the next entity, starting from the camera sensor.
Following this path may not always lead to a valid capture device and
therefore the setup will fail on some machines. This is for example an
issue when using the SimplePipeline on an i.MX-6Q with its i.MX IPU.
This commit implements a different approach to setup the media-pipeline
by finding the shortest path to a valid capture device, using the
breadth-first search algorithm. On i.MX6Q, the shortest path has a good
chance to be the path from the sensor to the CSI capture device, as
other paths may involve image converters, encoders or other IPU blocks
and will have therefore more nodes.
Signed-off-by: Marian Cichy <m.cichy@pengutronix.de>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
If the ISP::Output0 stream has not been configured, we must enable it
with a default format and resolution for internal use. This is to allow
the pipeline handler data flow to be consistent, and allow the IPA to
receive statistics for the frame.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
It is possible for the application to call pipeline_handler::configure()
multiple times, which would attempt to open the Unicam embedded data
node on every call. This would cause a warning message as the node
would have already been opened. Avoid this by tracking if the node
has previously been opened.
Note that this is a temporary fix since the open call for the Unicam
embedded data node will be moved from pipeline_handler::configure() to
pipeline_handler::match().
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
There was an off-by-one error in DelayedControls::get() when picking
controls from the queue to return back to the pipeline handler.
This is only noticeable as small oscillations in brightness when closely
viewing frame while AGC is running. The old StaggeredCtrl did not show
this error as the startup queuing mechanism has changed in
DelayedControls.
Fix this by indexing to the correct position in the queue.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reported-by: David Plowman <david.plowman@raspberrypi.com>
Fixes: 3d4b7b0059 ("libcamera: delayed_controls: Add helper for controls that apply with a delay")
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In DelayedControls::applyControls(), the controls queue would be
extended via a no-op control push to fill the intermittent slots with
unchanged control values. This is needed so that we read back unchanged
control values correctly on every frame.
However, there was one additional no-op performed on every frame that is
not required, meaning that any controls queued by the pipeline handler
would have their write delayed by one further frame. The original
StaggeredCtrl did not do this, as it only had one index to manage,
whereas DelayedControls uses two.
Remove this last no-op push so that the pipeline_handler provided
controls would be written on the very next frame if possible. As a
consequence, we must mark the control update as completed within the
DelayedControls::applyControls() loop, otherwise it might get reused
when cycling through the ring buffer.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reported-by: David Plowman <david.plowman@raspberrypi.com>
Fixes: 3d4b7b0059 ("libcamera: delayed_controls: Add helper for controls that apply with a delay")
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
On DelayedControls::reset(), the values retrieved from the sensor device
were added to the queues with the updated flag set to true. This would
cause the helper to write out the value to the device again on the first
DelayedControls::applyControls() call. This is unnecessary, as the
controls written are identical to what is stored in the device driver.
Fix this by explicitly setting the update flag to false in
DelayedControls::reset() when adding the controls to the queue.
Additionally, use the Info() constructor when adding items to the queue
for consistency.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Fixes: 3d4b7b0059 ("libcamera: delayed_controls: Add helper for controls that apply with a delay")
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
If an exposure time change adjusts the vblanking limits, and we set both
VBLANK and EXPOSURE controls through the VIDIOC_S_EXT_CTRLS ioctl, the
latter may fail if the value is outside of the limits calculated by the
old VBLANK value. This is a limitation in V4L2 and cannot be fixed by
setting VBLANK before EXPOSURE in a single VIDIOC_S_EXT_CTRLS ioctl.
The workaround here is to have the DelayedControls object mark the
VBLANK control as "priority write", which then write VBLANK separately
from (and ahead of) any other controls. This way, the sensor driver will
update the EXPOSURE control with new limits before the new values is
presented, and will thus be seen as valid.
To support this, a new struct DelayedControls::ControlParams is used in
the constructor to provide the control delay value as well as the
priority write flag.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Tested-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
[Kieran: Fix up trivial comments, merge conflicts]
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In kernel 5.11 the rkisp1 uapi had changed to support different hardware
revisions. Currently only revision 10 is supported by the rkisp1 IPA and
therefore 'init' should fail if the revision is not 10.
This changes depends on the kernel driver reporting the hardware
revision, and thus requires the rkisp1 driver from v5.11 or newer.
Signed-off-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The IPA of rkisp1 relies on some of the camera's controls.
Therefore it can't work if those controls are not given.
Return -EINVAL from 'configure' in that case.
Also return error from the pipeline's 'configure' method
if the IPA configure fails.
Signed-off-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Register the FrameDurations control in the IPU3 pipeline handler
computed using the vertical blanking limits and the sensor
pixel rate as parameters.
The FrameDurations control limits should be updated everytime a new
configuration is applied to the sensor.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
The controls' limits initialized by the IPU3 pipeline handler depend
on the sensor configuration. In order to compute controls using a known
state apply to the sensor a configuration equal to its own resolution.
Move the \todo note regarding the controls' limits dependency on the
sensor configuration at the beginning of the function and remove the
other redundant ones in the function body.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Now that we support returning int directly in addition to other output
parameters, improve the configure() function in the raspberrypi IPA
interface.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Since commit 96aecfe365 ("libcamera: camera_sensor: Use active area
size as resolution") the CameraSensor::resolution() method returned the
sensor's active pixel area size.
As the CameraSensor::resolution() method is widely used in the library
code base to retrieve the maximum frame size the sensor can produce,
in case it is smaller than the pixel area size the returned size cannot
be used to configure the sensor correctly.
Fix this by returning the maximum frame resolution the sensor can
produce, or the pixel area size in case the sensor embeds and ISP that
can upscale and the supported maximum frame size is thus larger that
the pixel array size.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
When the IPU3Frames completes, it deletes the internal info storage.
This storage contains the pointer to the Request, but in some cases the
pointer was being accessed after the info structure was removed.
Ensure that the Request is obtained before attempting to complete to
obtain a valid pointer.
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add tracing to the base pipeline handler class to track when requests are queued.
Tracing is already available for other Request operations, but queuing a Request
is not an operation handled by the Request itself.
Add the tracepoint to the PipelineHandler::queueRequest() so the lifetime of a
Request can be viewed when tracing.
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When a pipeline handler completes a request, the request itself is not
deleted by libcamera, and the application regains control over the
object. It may choose to delete the Request, or re-use it.
Clarify this in the comment by removing the declaration that the Request
is deleted, but state that it is no longer managed by the pipeline
handler and must not be accessed further after this function returns.
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Umang Jain <email@uajain.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Requests should only be completed from the RequestPending state.
Requests which are completed from the RequestCancelled, or RequestComplete
state, will indicate that a double-complete has been called on the Request,
or that it has been used internally after it has been given back to the
application.
Ensure that this can be caught early if it occurs by enforcing the state
required with an assert.
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Umang Jain <email@uajain.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Sebastian Fricke
Kieran Bingham
Laurent Pinchart
Khem Raj
Naushir Patuck
Hirokazu Honda
Jacopo Mondi
Marian Cichy
Jean-Michel Hautbois
Dafna Hirschfeld
Paul Elder
David Plowman