As a step to be able to make RkISP1Path::video_ private add simple
wrappers for buffer handling. There is no functional change.
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Move the path configuration generation and validation to RkISP1Path.
This is done to increase code reuse and to encapsulate the main and self
path differences inside the RkISP1Path class. There is no functional
change.
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Move the path configuration to RkISP1Path to increase code reuse and
make the V4L2 subdevice resizer private to the path. There is no
functional change.
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The self and main paths are very similar and the introduction of support
for two simultaneous streams have made it clear their handling could be
abstracted in a separate class.
This is the first step to create such a class by breaking out the
initialization and storage of the video and subdevices. There is no
functional change in this patch.
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Be more precise in commenting why the ImgU shall not be configured
if only the RAW stream is requested.
As an example, if the ImgU gets unecessary configured:
cam -srole=viewfinder -c2 -C -> WORKS
cam -srole=stillraw -c2 -C -> WORKS
cam -srole=viewfinder -c2 -C -> Failed to queue buffer 0: Invalid argument
Since commit ("libcamera: ipu3: Fix RAW+YUV capture") the ImgU
configuration procedure also correctly implements the assumption that at
least one of the YUV output is being operated. If that's not the case,
as in the RAW-only capture use case, the code tries to access a
non-existing configuration. One more reason to exit early if no YUV
stream is requested.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Since the re-implementation of the IPU3 pipeline handler
configuration procedure, the main output is always assigned in
case any YUV stream is requested.
Remove a dead code block that checks for the main output to be
valid.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
When requesting one RAW stream and one YUV stream the
StreamConfiguration assigned to the RAW stream is the first one
added to the CameraConfiguration, while the YUV stream gets assigned to
the main output.
At configure() time the viewfinder output needs to be configured with
the same format as the main output, but since the introduction of RAW
capture support, the pipeline has not been updated and still assumes
the main output configuration is the first one in the
CameraConfiguration. This causes the viewfinder to be configured
with the same format as the raw stream, breaking capture operations.
Before this commit the following command fails and the ImgU does not
produce frames:
cam -srole=stillraw -srole=viewfinder -c2 -C
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Change variable names to camel case to be consistent with the rest of
the source files. Remove #define consts and replace with constexpr.
Add some newlines to make the code more readable.
There are no functional changes in this commit.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
[Kieran: Rebase merge conflicts resolved]
[Kieran: Fix checkstyle line under 80 chars]
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This commit plumbs the user transform from the Raspberry Pi pipeline
handler through to the IPA. Note that the transform is actually
handled in the sensor (by setting the h/v flip bits), so the IPAs need
to understand the orientation of the image they receive.
Once in the IPA we add it to the CameraMode description, so that it
becomes automatically available to all the individual control
algorithms.
The IPA configure method has to be reordered just a little so as to
fill in the transform in the camera mode before calling SwitchMode.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The Raspberry Pi pipeline handler allows all transforms except those
involving a transpose. The user transform is combined with any
inherent rotation of the camera, and the camera's H and V flip bits
are set accordingly.
Note that the validate() method has to work out what the final Bayer
order of any raw streams will be, before configure() actually applies
the transform to the sensor. We make a note of the "native"
(untransformed) Bayer order when the system starts, so that we can
deduce transformed Bayer orders more easily.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add a field to the CameraConfiguration (including the necessary
documentation) to represent a 2D transform requested by the
application. All pipeline handlers are amended to coerce this to the
Identity, marking the configuration as "adjusted" if something
different had been requested.
Pipeline handlers that support Transforms can be amended subsequently.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This reverts commit 1e8c91b656.
Now that we shall be implementing application-defined 2D transforms
it's no longer possible to set the sensor orientation so early on. We
have to wait until we have the CameraConfiguration object as that's
where the application puts its choice of transform.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Expose the self stream to applications and prefers it for the viewfinder
and video roles as it can be extended to produce RGB. Keep preferring
the main path for still capture as it could be extended to support RAW
formats which makes most sense for still capture.
With this change the self path becomes available to applications and a
camera backed by this pipeline can produce two streams simultaneously.
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Extend the format validation to work with both main and self paths. The
heuristics honors that the first stream in the configuration has the
highest priority while still examining both streams for a best match.
It is not possible to capture from the self path as the self stream is
not yet exposed to applications.
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Allow for both the main and self path streams to be configured. This
change adds the self path as an internal stream to the pipeline handler.
It is not exposed as a Camera stream so it can not yet be used.
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Create RkISP1Frames from camera data instead of picking information out
from it. This is done to prepare for multi stream support where more
information from the camera data will be needed.
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The buffer ready handlers are designed for a single application facing
stream from the main path. To prepare for multiple application facing
streams from main and/or self path the handlers need to be prepared.
The data keeping track of the frame number and advancing the timeline
can be moved from the application facing buffer ready handler to the
statistics handler. For each request processed there will always be a
statistic buffer and as the ISP is inline and is the source of both
main, self and statistic paths there is no change in behavior.
The application facing handler no longer needs a special case for
cancelled frames and can be made simpler. With this change the handlers
are ready to deal with any combinations of application facing streams.
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
In preparation of supporting both the main and self path configure all
the media graph links as a part of the configuration step. Before this
change the link between ISP and DMA engine was setup at match time as
the only supported path was the main path and only the link between
sensor and ISP was updated at part of the configuration step.
The main path is still the only path between ISP and DMA engine that is
possible to enable.
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
There is no need to check if Request contains a buffer belonging the
RkISP1 Camera as this is already done in Camera::queueRequest(), remove
the redundant check.
Reported-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
The RkISP1 pipeline originally only supported NV formats which have 2
planes. When support for YUV formats was added the plane count on the
output format was not made to reflect this. Instead of hard coding the
plane count to 2 fetch the number of planes from the format information.
Reported-by: Jacopo Mondi <jacopo@jmondi.org>
Fixes: 2b1a908b52 ("libcamera: camera: Add a validation API to the CameraConfiguration class")
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Handle the case where a FrameBuffer that has been externally allocated
(i.e. not through the v4l2 video device) is passed into a Request.
We must store the buffer pointer in the stream internal buffer list to
identify when used.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
By using a set container, we can easily insert/remove buffer ids that have
been mmaped by the IPA. This will be required to track buffers allocated
externally and passed to the pipeline handler through a Request.
Move the IPA buffer mapping code into a function to remove duplicated
code.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
By using a map container, we can easily insert/remove buffers from the
buffer list. This will be required to track buffers allocated externally
and passed to the pipeline handler through a Request.
Replace the buffer index tracking with an id generated internally by the
stream object.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Add further queueing into the RPiStream object to ensure that we always
follow the buffer ordering (be it internal or external) given by
incoming Requests.
This is essential, otherwise we risk dropping frames that are meant to
be part of a Request, and can cause the pipeline to stall indefinitely.
This also prevents any possibility of mismatched frame buffers going
through the pipeline and out to the application.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Stop using v4l2_videodevice::allocateBuffer() for internal buffers and
instead export/import all buffers. This allows the pipeline to return
any stream buffer requested by the application as zero-copy.
Advertise the Unicam Image stream as the RAW capture stream now.
The RPiStream object now maintains a new list of buffers that are
available to queue into a device. This is needed to distinguish between
FrameBuffers allocated for internal use vs externally provided buffers.
When a Request comes in, if a buffer is not provided for an exported
stream, we re-use a buffer from this list.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
When calculating the pipeline configuration for the IPU3 platform,
libcamera tries to be smart and select the smallest sensor frame
resolution large enough to accommodate the stream sizes
requested by the application.
While this makes a lot of sense, in practice optimizing the
selected sensor resolution makes the pipeline configuration calculation
process fail in multiple occasions, or results in stalls during capture.
As a trivial example, capturing with cam with the following command
line results in a stall:
$ cam -swidth=1280,height=720 -swidth=640,height=480 -c1 -C
Likewise, the Android HAL supported format enumeration fails in
reporting smaller resolutions as supported when used with the OV5670
sensor.
320x240:
DEBUG IPU3 ipu3.cpp:192 CIO2 configuration: 648x486-SGRBG10_IPU3
ERROR IPU3 imgu.cpp:408 Failed to calculate pipe configuration
ERROR IPU3 ipu3.cpp:299 Failed to calculate pipe configuration: unsupported resolutions.
640x480:
DEBUG IPU3 ipu3.cpp:192 CIO2 configuration: 320x240-SGRBG10_IPU3
ERROR IPU3 imgu.cpp:408 Failed to calculate pipe configuration
ERROR IPU3 ipu3.cpp:299 Failed to calculate pipe configuration: unsupported resolutions.
Furthermore the reference xml files used for the IPU3 camera
configuration on the ChromeOS platform restricts the number of sensor
resolution to be used for the OV5670 sensor to 2 from the 6 supported by
the driver [1].
The selection criteria of the correct CIO2 mode are not specified, and
for the time being, as a workaround, always use the sensor maximum
resolution at the expense of frame rate and bus bandwidth to allow the
pipeline to successfully support smaller modes for the OV5670 sensor and
solve pipeline stalls when capturing with both sensors.
[1] See the <sensor_modes> enumeration in:
https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/master/baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/files/gcss/graph_settings_ov5670.xml
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Improve the device opening at class construction time by testing all
the available device paths and printout the appropriate error message.
While at it, initialize dmaHeapHandle_ to -1 and check for its value
at destruction time.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
The default sensor orientation must be set early on in match() to ensure
generateConfiguration() and configure() return out the correct Bayer
ordering to the application. This is particularly important for RAW
capture streams.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
We build libcamera with -Wno-unused-parameter and this doesn't cause
much issue internally. However, it prevents catching unused parameters
in inline functions defined in public headers. This can lead to
compilation warnings for applications compiled without
-Wno-unused-parameter.
To catch those issues, remove -Wno-unused-parameter and fix all the
related warnings with [[maybe_unused]].
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
