In kernel 5.11 the rkisp1 uapi had changed to support different hardware
revisions. Currently only revision 10 is supported by the rkisp1 IPA and
therefore 'init' should fail if the revision is not 10.
This changes depends on the kernel driver reporting the hardware
revision, and thus requires the rkisp1 driver from v5.11 or newer.
Signed-off-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The IPA of rkisp1 relies on some of the camera's controls.
Therefore it can't work if those controls are not given.
Return -EINVAL from 'configure' in that case.
Also return error from the pipeline's 'configure' method
if the IPA configure fails.
Signed-off-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Register the FrameDurations control in the IPU3 pipeline handler
computed using the vertical blanking limits and the sensor
pixel rate as parameters.
The FrameDurations control limits should be updated everytime a new
configuration is applied to the sensor.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
The controls' limits initialized by the IPU3 pipeline handler depend
on the sensor configuration. In order to compute controls using a known
state apply to the sensor a configuration equal to its own resolution.
Move the \todo note regarding the controls' limits dependency on the
sensor configuration at the beginning of the function and remove the
other redundant ones in the function body.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Now that we support returning int directly in addition to other output
parameters, improve the configure() function in the raspberrypi IPA
interface.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Since commit 96aecfe365 ("libcamera: camera_sensor: Use active area
size as resolution") the CameraSensor::resolution() method returned the
sensor's active pixel area size.
As the CameraSensor::resolution() method is widely used in the library
code base to retrieve the maximum frame size the sensor can produce,
in case it is smaller than the pixel area size the returned size cannot
be used to configure the sensor correctly.
Fix this by returning the maximum frame resolution the sensor can
produce, or the pixel area size in case the sensor embeds and ISP that
can upscale and the supported maximum frame size is thus larger that
the pixel array size.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
When the IPU3Frames completes, it deletes the internal info storage.
This storage contains the pointer to the Request, but in some cases the
pointer was being accessed after the info structure was removed.
Ensure that the Request is obtained before attempting to complete to
obtain a valid pointer.
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add tracing to the base pipeline handler class to track when requests are queued.
Tracing is already available for other Request operations, but queuing a Request
is not an operation handled by the Request itself.
Add the tracepoint to the PipelineHandler::queueRequest() so the lifetime of a
Request can be viewed when tracing.
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When a pipeline handler completes a request, the request itself is not
deleted by libcamera, and the application regains control over the
object. It may choose to delete the Request, or re-use it.
Clarify this in the comment by removing the declaration that the Request
is deleted, but state that it is no longer managed by the pipeline
handler and must not be accessed further after this function returns.
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Umang Jain <email@uajain.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Requests should only be completed from the RequestPending state.
Requests which are completed from the RequestCancelled, or RequestComplete
state, will indicate that a double-complete has been called on the Request,
or that it has been used internally after it has been given back to the
application.
Ensure that this can be caught early if it occurs by enforcing the state
required with an assert.
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Umang Jain <email@uajain.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Camera sensors can include an ISP. For instance, the AP1302 external ISP
can be connected to up to two raw camera sensors, and the combination of
the sensors and ISP is considered as a (smart) camera sensor from
libcamera's point of view.
The CameraSensor class has limited support for this already. Extend the
simple pipeline handler to support such sensors, by using the media
entity corresponding to the ISP instead of the raw camera sensor's
entity.
We don't need to handle the case where an entity in the SoC would expose
the MEDIA_ENT_F_PROC_VIDEO_ISP function, as pipeline containing an ISP
would have a dedicated pipeline handler.
The implementation is limited as it won't support other multi-entity
camera sensors (such as CCS). While this would be worth supporting, we
don't have a test platform with a CCS-compatible sensor at this point,
so let's not over-engineer the solution. Extending support to CCS (and
possibly other sensor topologies) will likely involve helpers that can
be used by other pipeline handlers (such as generic graph walk helpers
for instance) and extensions to the CameraSensor class.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
The pipeline handler would enable and use the Unicam embedded data stream
even if the sensor did not support it. This was to allow a means to
pass exposure and gain values for the frame to the IPA in a synchronised
way.
The recent changes to get the pipeline handler to pass a ControlList
with exposure and gain values means this is no longer required. Disable
the use of the embedded data stream when a sensor does not support it.
This change also removes the mappedEmbeddedBuffers_ map as it is no
longer used.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
When running with sensors that had no embedded data, the pipeline handler
would fill a dummy embedded data buffer with gain/exposure values, and
pass this buffer to the IPA together with the bayer buffer. The IPA would
extract these values for use in the controller algorithms.
Rework this logic entirely by having a new RPiCameraData::BayerFrame
queue to replace the existing bayer queue. In addition to storing the
FrameBuffer pointer, this also stores all the controls tracked by
DelayedControls for that frame in a ControlList. This includes include
exposure and gain values. On signalling RPi::IPA_EVENT_SIGNAL_ISP_PREPARE
IPA event, the pipeline handler now passes this ControlList from the
RPiCameraData::BayerFrame queue.
The IPA now extracts the gain and exposure values from the ControlList
instead of using RPiController::MdParserRPi to parse the embedded data
buffer.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Camera sensors can include an ISP, which may be reported as a separate
entity from the pixel array in the media graph.
Support such sensors by accepting MEDIA_ENT_F_PROC_VIDEO_ISP as a valid
entity type. This allows using sensors that can be fully (or at least
meaningfully) configured through the ISP's source pad only. Sensors that
require further configuration, on the ISP sink pad and/or on the pixel
array's source pad, will require further extension to the CameraSensor
class.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
When a sensor can upscale the image, the native sensor resolution isn't
equal to the largest size reported by the sensor. Use the active area
size instead, and default it to the largest enumerated size if the crop
rectangle targets are not supported.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Implement and expose the symbol and functions that the new cros camera
API requires. Since we don't actually need them, leave them empty.
Update meson accordingly.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Allow support for multiple streams on a per-device basis. The decision
should be made based on the ability of the converter to run multiple
times within the duration of one frame. Hardcode it in
SimplePipelineInfo for now.
We may later compute the number of supported streams dynamically based
on the requested configuration, using converter bandwidth information
instead of a hardcoded fixed value.
All platforms are currently limited to a single stream until they get
successfully tested with multiple streams.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Extend the SimpleCameraConfiguration to support multiple streams, using
the multi-stream capability of the SimpleConverter class. Wiring up
multi-stream support in the other pipeline handler operations will come
in further commits.
To keep the code simple, require all streams to use the converter if any
stream needs it. It would be possible to generate one stream without
conversion (provided the format and size match what the capture device
can generate), and this is left as a future optimization.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Store the list of converter output formats in the Configuration
structure, to be used to implement multi-stream support. As the
Configuration structure grows bigger, avoid duplicating it in the
formats_ map for each supported pixel format by storing it in a configs_
vector instead, and storing pointers only in the map.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
As the pipeline configuration is selected in
SimpleCameraConfiguration::validate() already, cache it in the
SimpleCameraConfiguration instead of looking it up in
SimplePipelineHandler::configure(). This makes little difference at the
moment, but will save duplication of more complex logic between
validate() and configure() when adding support for multiple streams.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The simple pipeline handler has grown over time, and isn't that simple
anymore that it can easily be understood by an unfamiliar reader.
Document the design to explicitly state the expectations of the pipeline
handler, and to explain how it operates.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
While the M2M device backing the converter doesn't support multiple
streams natively, it can be run once per stream to produce multiple
outputs from the same input, with different output formats and sizes.
To support this, create a class to model a stream and move control of
the M2M device to the Stream class. The SimpleConverter class then
creates stream instances and iterates over them. Each stream needs its
own instance of the V4L2M2MDevice, to support different output
configurations. The SimpleConverter class retains a device instance to
support the query operations.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The SimpleConverter API signals completion of input and output buffer
pairs. This unnecessarily delays requeueing the input buffer to the
video capture queue until the output buffer completes, and also delays
signalling request completion until the input buffer completes. While
this shouldn't cause large delays in practice, it will also not scale
when multi-stream support will be added to the converter class.
To address the current issue and prepare for the future, decouple
signalling of input and output buffers completion.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Simplify the SimpleConverter interface by opening the M2M device in the
constructor. The explicit call to open() is replaced by a check through
a new isValid() function, and the unused close() function is removed.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The number of buffers on the input and output of the converter don't
necessarily need to match. Use the buffer count from the input and
output configuration respectively. This removes the need to pass the
buffer count to the start() function, which brings it closer to the
pipeline handler API.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Group the configuration parameters for the converter input in a
StreamConfiguration instance. This makes the configure() function
signature cleaner, and will allow passing additional parameters (such as
stride and buffer count).
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The SimpleConverter class has different sets of functions, related to
static queries, device configuration and runtime operation. Group the
query functions together. While at it, swap the arguments to the
strideAndFrameSize() function to match the order in which pixel format
and size are usually specified.
No functional change is included.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The stride (and frame size) calculation has been moved from configure
time to configuration validate time by commit 89fb1efac2 ("libcamera: simple:
Fill stride and frameSize at config validation"). This change has
however left one stray setting of the stride when configuring the
converter. Fix it.
While at it, turn the SimpleConverter::configure() output configuration
argument to a const reference to emphasize it can't be null and isn't
modified by the function, and rename it from cfg to outputCfg to make
its purpose clearer.
Fixes: 89fb1efac2 ("libcamera: simple: Fill stride and frameSize at config validation")
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The flag that is used to record if the sensor driver supports the H/V flip
controls is initialized at camera creation time, but it was not
initialized by the IPU3Camera constructor, resulting in erroneous
values that might break capture for sensor that do not support flipping.
Fix this by initializing the flag in the class constructor.
Fixes: 6c4ce7de30 ("libcamera: ipu3: Add rotation to ipu3 pipeline")
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Fabian Wüthrich <me@fabwu.ch>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Print a warning when the orientation of a sensor is unknown. The
location property is still defaulted to external.
Also add a recommended controls list, similar to the optional and
mandatory controls list, to handle controls in a similar situation in
the future.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The addition of the new IPA IPC mechanism compiles pipeline-specific
headers to define the interface between the pipeline and the IPA.
This was optimised in 08ce394465 ("meson: ipa, proxy: Only build
proxies for enabled pipelines") to only build for enabled pipelines,
however the tests directly use the VIMC pipeline handler, and require
it to be built.
Create a local variable to store the requested pipelines from the user
configuration and extend the enabled pipelines to ensure that VIMC is
always enabled if the tests are also enabled
Fixes: 08ce394465 ("meson: ipa, proxy: Only build proxies for enabled pipelines")
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Tested-by: Sebastian Fricke<sebastian.fricke@posteo.net>
Use the same transformation logic as in the raspberry pipeline to
implement rotations in the ipu3 pipeline.
Tested on a Surface Book 2 with an experimental driver for OV5693.
Signed-off-by: Fabian Wüthrich <me@fabwu.ch>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Acked-by: Fabian Wüthrich <me@fabwu.ch>
Usage of std::move() on a const lvalue will produce a const rvalue,
which the compiler will bind to the const lvalue reference of the
constructor or assignement operator. This results in the object being
copied, not moved. Drop std::move() in that case as it's misleading.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
When a Request is completed upon receiving the IPA produced metadata,
the metadata associated with the Request were over-written, deleting
the information set, in example, at ImgU output buffer completion.
If any additional Request metadata should be registered by inspecting
the IPA produced metadata it has to be done without deleting the already
registered entry.
Fix this by replacing the metadata over-write with a todo entry.
This change fixes CTS which was broken due to missing metadata in
the completed requests.
Fixes: 9708f49fec ("libcamera: ipu3: Share parameter and statistic buffers with IPA")
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
In order to get the stats back, the imgu subdev needs to have the
V4L2_CID_INTEL_IPU3_MODE control set.
Set it to video mode by default to get the stats at each frame.
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
[Kieran: Updated message to match Laurent's suggestion]
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
