With the recent change in the bayer/embedded buffer matching code,
a condition would make the bayer buffer be requeued back to the device,
even though it could potentially be queued for matching. This would
cause unnecessary frame drops as sync would be lost.
The fix is to ensure the bayer buffer only gets requeued if the embedded
data buffer queue is not empty, i.e. the buffer truly is orphaned.
Additionally, we do this test before deciding to flush any of the two
queues of all their buffers.
Fixes: 909882b (pipeline: raspberrypi: Rework bayer/embedded data buffer matching)
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Crop rectangle was not being configured on the isp output pad nor in the
resizer input pad, causing an unecessary crop in the image and an
unecessary scaling by the resizer when streaming with a higher
resolution then the default 800x600.
Example:
cam -c 1 -C -s width=3280,height=2464
In the pipeline:
sensor->isp->resizer->dma_engine
isp output crop is set to 800x600, which limits the output format to
800x600, which is propagated to the resizer input format set to 800x600,
and the resizer output format is set to the desired end resolution
3280x2464.
Signed-off-by: Helen Koike <helen.koike@collabora.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
There is a condition that would cause the buffers to never be in sync
when we using only a single buffer in the bayer and embedded data streams.
This occurred because even though both streams would get flushed to resync,
one stream's only buffer was already queued in the device, and would end
up never matching.
Rework the buffer matching logic by combining updateQueue() and
tryFlushQueue() into a single function findMatchingBuffers(). This would
allow us to flush the queues at the same time as we match buffers, avoiding
the the above condition.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
When configuring the converter, the format is first set on the output
side based on the format of the camera pipeline output, and then the
format is set on the capture side to match the desired stream
configuration. The format parameter passed to
V4L2VideoDevice::setFormat() uses the same variable for both calls,
which has the unwanted side effect of carrying plane configuration from
the output side to the capture side of the converter. In particular, the
stride or plane size requested on the capture side can become
unnecessarily large when converting to a format with a lower number of
bits per pixel (for instance converting YUYV to NV12).
Fix this by resetting the format variable before using it to configure
the capture side.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The V4L2DeviceFormat class now has default initializers for all members,
explicit initialization isn't needed anymore.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
If converter_->open() fails, the code deletes the converter_ but then
happily goes on, and at the very next lines will use converter_ to
connect the bufferReady signal.
Ensure the converter is only connected when successfully opened by
extending the conditional and connecting in an else scope instead.
Reported-by: Tomi Valkeinen <tomi.valkeinen@iki.fi>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
During configure() we update the ScalerCropMaximum to the correct
value for this camera mode and work out the minimum crop size allowed
by the ISP.
Whenever a new ScalerCrop request is received we check it's valid and
apply it to the ISP V4L2 device. When the IPA returns its metadata to
us we add the ScalerCrop information, rescaled to sensor native
pixels.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Register controls for the IPU3 pipeline handler. The only supported
Camera control is currently the pipeline depth control.
Report the minimum and maximum values the pipeline handler supports for
the pipeline processing stages and report for each request the pipeline
depth describing the number of processing steps the frames went through.
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
The rkisp1 and simple pipeline handlers can fail to register any camera,
if initialization of all the detected cameras fail. In that case, they
still return success from their match function. As no camera gets
registered, the pipeline handler is immediately destroyed, releasing the
acquired media devices, and the camera manager immediately tries to
match the same pipeline handler with the same media device, causing an
endless loop.
Fix it by returning false from the match function if no camera gets
registered.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Tested-by: Andrey Konovalov <andrey.konovalov@linaro.org>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
The FrameBuffer used to track any addition to the request in an
under-run event was shadowed against the returned buffer, being placed
back on the availableBuffers_ queue.
Rename the shadowed variable to be more explicit that it is a separate
Buffer.
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
qcom-camss driver supports MIPI CSI2 receivers on APQ8016 and APQ8096
Application processors from Qualcomm, which are used on e.g. DragonBoard
410c and DragonBoard 820c dev boards respectively.
The simple pipeline handler provides basic support for these devices.
Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The current simple pipeline handler refuses to work with capture devices
which have V4L2_CAP_VIDEO_CAPTURE_MPLANE flag set in the device capabilities
field. This is too restrictive, as devices supporting the multi-planar API
can be using contiguous memory for semi-planar and planar formats, and this
would just work without any changes to libcamera.
Drop the guard against MPLANE devices, and replace it with the check of
the number of planes in the format the simple pipeline handler is going to
use for capture. This will let MPLANE devices which don't use non-contiguous
memory for frame buffers to work with the simple pipeline handler.
The following code in SimpleCameraData::init() filters out the pixel formats
libcamera doesn't support:
PixelFormat pixelFormat = videoFormat.first.toPixelFormat();
if (!pixelFormat)
continue;
So the check for the number of memory planes this patch adds would not
trigger until non-contiguous planar formats becomes supported in libcamera,
and video devices using these formats are enabled in the simple pipeline
handler. Then this check will remind one to review the simple pipeline
handler code.
Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The UVCCameraData::init() is the only consumer of the default entry in
the media graph, move the lookup of it into the init() function and pass
it the MediaDevice. This is done in preparation to extend the CameraData
initialization to consume more information from the MediaDevice.
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The upstream driver has changed how the link formats are validated when
starting to stream [1]. This revealed that libcamera did not adjust the
media bus format from the link between {main,self} resizer source pad
and the capture video device as expected by the driver.
The media bus code YUYV8_2X8 was hardcoded to MEDIA_BUS_FMT_YUYV8_2X8
for all pixel formats while it must be adjusted to YUYV8_1_5X8 for NV12
and NV21, fix this.
1. 6803a9e0e1e43e9e ("media: staging: rkisp1: cap: simplify link validation by comparing media bus code")
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The upstream driver has dropped support for YVYU and VYUY [1], remove
support from the pipeline handler.
1. 3acb3e06baf64e28 ("media: staging: rkisp1: cap: remove unsupported formats")
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Instead of manually tracking if a path is enable or not use the media
graph link status. There is no functional change.
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Move the start and stop of a path to RkISP1Path. This allows the
importing of buffers to be moved closer the path start/stop simplifying
the code. Also by adding a simple running tracker the error logic in
PipelineHandlerRkISP1 can be simplified as stop() can always be called.
This also removes all external users of RkISP1Path::video_ so it can be
made private.
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
As a step to be able to make RkISP1Path::video_ private add simple
wrappers for buffer handling. There is no functional change.
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Move the path configuration generation and validation to RkISP1Path.
This is done to increase code reuse and to encapsulate the main and self
path differences inside the RkISP1Path class. There is no functional
change.
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Move the path configuration to RkISP1Path to increase code reuse and
make the V4L2 subdevice resizer private to the path. There is no
functional change.
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The self and main paths are very similar and the introduction of support
for two simultaneous streams have made it clear their handling could be
abstracted in a separate class.
This is the first step to create such a class by breaking out the
initialization and storage of the video and subdevices. There is no
functional change in this patch.
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Be more precise in commenting why the ImgU shall not be configured
if only the RAW stream is requested.
As an example, if the ImgU gets unecessary configured:
cam -srole=viewfinder -c2 -C -> WORKS
cam -srole=stillraw -c2 -C -> WORKS
cam -srole=viewfinder -c2 -C -> Failed to queue buffer 0: Invalid argument
Since commit ("libcamera: ipu3: Fix RAW+YUV capture") the ImgU
configuration procedure also correctly implements the assumption that at
least one of the YUV output is being operated. If that's not the case,
as in the RAW-only capture use case, the code tries to access a
non-existing configuration. One more reason to exit early if no YUV
stream is requested.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Since the re-implementation of the IPU3 pipeline handler
configuration procedure, the main output is always assigned in
case any YUV stream is requested.
Remove a dead code block that checks for the main output to be
valid.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
When requesting one RAW stream and one YUV stream the
StreamConfiguration assigned to the RAW stream is the first one
added to the CameraConfiguration, while the YUV stream gets assigned to
the main output.
At configure() time the viewfinder output needs to be configured with
the same format as the main output, but since the introduction of RAW
capture support, the pipeline has not been updated and still assumes
the main output configuration is the first one in the
CameraConfiguration. This causes the viewfinder to be configured
with the same format as the raw stream, breaking capture operations.
Before this commit the following command fails and the ImgU does not
produce frames:
cam -srole=stillraw -srole=viewfinder -c2 -C
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Change variable names to camel case to be consistent with the rest of
the source files. Remove #define consts and replace with constexpr.
Add some newlines to make the code more readable.
There are no functional changes in this commit.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
[Kieran: Rebase merge conflicts resolved]
[Kieran: Fix checkstyle line under 80 chars]
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This commit plumbs the user transform from the Raspberry Pi pipeline
handler through to the IPA. Note that the transform is actually
handled in the sensor (by setting the h/v flip bits), so the IPAs need
to understand the orientation of the image they receive.
Once in the IPA we add it to the CameraMode description, so that it
becomes automatically available to all the individual control
algorithms.
The IPA configure method has to be reordered just a little so as to
fill in the transform in the camera mode before calling SwitchMode.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The Raspberry Pi pipeline handler allows all transforms except those
involving a transpose. The user transform is combined with any
inherent rotation of the camera, and the camera's H and V flip bits
are set accordingly.
Note that the validate() method has to work out what the final Bayer
order of any raw streams will be, before configure() actually applies
the transform to the sensor. We make a note of the "native"
(untransformed) Bayer order when the system starts, so that we can
deduce transformed Bayer orders more easily.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add a field to the CameraConfiguration (including the necessary
documentation) to represent a 2D transform requested by the
application. All pipeline handlers are amended to coerce this to the
Identity, marking the configuration as "adjusted" if something
different had been requested.
Pipeline handlers that support Transforms can be amended subsequently.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This reverts commit 1e8c91b656.
Now that we shall be implementing application-defined 2D transforms
it's no longer possible to set the sensor orientation so early on. We
have to wait until we have the CameraConfiguration object as that's
where the application puts its choice of transform.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Expose the self stream to applications and prefers it for the viewfinder
and video roles as it can be extended to produce RGB. Keep preferring
the main path for still capture as it could be extended to support RAW
formats which makes most sense for still capture.
With this change the self path becomes available to applications and a
camera backed by this pipeline can produce two streams simultaneously.
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Extend the format validation to work with both main and self paths. The
heuristics honors that the first stream in the configuration has the
highest priority while still examining both streams for a best match.
It is not possible to capture from the self path as the self stream is
not yet exposed to applications.
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Naushir Patuck
Helen Koike
Laurent Pinchart
Kieran Bingham
Tomi Valkeinen
David Plowman
Jacopo Mondi
Hirokazu Honda
Andrey Konovalov
Niklas Söderlund