Allow support for multiple streams on a per-device basis. The decision
should be made based on the ability of the converter to run multiple
times within the duration of one frame. Hardcode it in
SimplePipelineInfo for now.
We may later compute the number of supported streams dynamically based
on the requested configuration, using converter bandwidth information
instead of a hardcoded fixed value.
All platforms are currently limited to a single stream until they get
successfully tested with multiple streams.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Extend the SimpleCameraConfiguration to support multiple streams, using
the multi-stream capability of the SimpleConverter class. Wiring up
multi-stream support in the other pipeline handler operations will come
in further commits.
To keep the code simple, require all streams to use the converter if any
stream needs it. It would be possible to generate one stream without
conversion (provided the format and size match what the capture device
can generate), and this is left as a future optimization.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Store the list of converter output formats in the Configuration
structure, to be used to implement multi-stream support. As the
Configuration structure grows bigger, avoid duplicating it in the
formats_ map for each supported pixel format by storing it in a configs_
vector instead, and storing pointers only in the map.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
As the pipeline configuration is selected in
SimpleCameraConfiguration::validate() already, cache it in the
SimpleCameraConfiguration instead of looking it up in
SimplePipelineHandler::configure(). This makes little difference at the
moment, but will save duplication of more complex logic between
validate() and configure() when adding support for multiple streams.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The simple pipeline handler has grown over time, and isn't that simple
anymore that it can easily be understood by an unfamiliar reader.
Document the design to explicitly state the expectations of the pipeline
handler, and to explain how it operates.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
While the M2M device backing the converter doesn't support multiple
streams natively, it can be run once per stream to produce multiple
outputs from the same input, with different output formats and sizes.
To support this, create a class to model a stream and move control of
the M2M device to the Stream class. The SimpleConverter class then
creates stream instances and iterates over them. Each stream needs its
own instance of the V4L2M2MDevice, to support different output
configurations. The SimpleConverter class retains a device instance to
support the query operations.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The SimpleConverter API signals completion of input and output buffer
pairs. This unnecessarily delays requeueing the input buffer to the
video capture queue until the output buffer completes, and also delays
signalling request completion until the input buffer completes. While
this shouldn't cause large delays in practice, it will also not scale
when multi-stream support will be added to the converter class.
To address the current issue and prepare for the future, decouple
signalling of input and output buffers completion.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Simplify the SimpleConverter interface by opening the M2M device in the
constructor. The explicit call to open() is replaced by a check through
a new isValid() function, and the unused close() function is removed.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The number of buffers on the input and output of the converter don't
necessarily need to match. Use the buffer count from the input and
output configuration respectively. This removes the need to pass the
buffer count to the start() function, which brings it closer to the
pipeline handler API.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Group the configuration parameters for the converter input in a
StreamConfiguration instance. This makes the configure() function
signature cleaner, and will allow passing additional parameters (such as
stride and buffer count).
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The SimpleConverter class has different sets of functions, related to
static queries, device configuration and runtime operation. Group the
query functions together. While at it, swap the arguments to the
strideAndFrameSize() function to match the order in which pixel format
and size are usually specified.
No functional change is included.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The stride (and frame size) calculation has been moved from configure
time to configuration validate time by commit 89fb1efac2 ("libcamera: simple:
Fill stride and frameSize at config validation"). This change has
however left one stray setting of the stride when configuring the
converter. Fix it.
While at it, turn the SimpleConverter::configure() output configuration
argument to a const reference to emphasize it can't be null and isn't
modified by the function, and rename it from cfg to outputCfg to make
its purpose clearer.
Fixes: 89fb1efac2 ("libcamera: simple: Fill stride and frameSize at config validation")
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
There is no need to pass the Camera pointer to queueRequest(),
completeBuffer() and completeRequest() as the Request also passed
contains the same information. Remove the Camera argument to avoid
situations where the information in the Request and the argument differ.
There is no functional change and no public API change as the interface
is only used between the Camera and PipelineHandler.
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Applications now have the ability to pass in controls that need to be
applied on startup, rather than doing it through Request where there might
be some frames of delay in getting the controls applied.
This commit adds the ability to pass in a set of libcamera controls into
the pipeline handlers through the pipeline_handler::start() method. These
controls are provided by the application through the camera::start()
public API.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The V4L2DeviceFormat class now has default initializers for all members,
explicit initialization isn't needed anymore.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
If converter_->open() fails, the code deletes the converter_ but then
happily goes on, and at the very next lines will use converter_ to
connect the bufferReady signal.
Ensure the converter is only connected when successfully opened by
extending the conditional and connecting in an else scope instead.
Reported-by: Tomi Valkeinen <tomi.valkeinen@iki.fi>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The rkisp1 and simple pipeline handlers can fail to register any camera,
if initialization of all the detected cameras fail. In that case, they
still return success from their match function. As no camera gets
registered, the pipeline handler is immediately destroyed, releasing the
acquired media devices, and the camera manager immediately tries to
match the same pipeline handler with the same media device, causing an
endless loop.
Fix it by returning false from the match function if no camera gets
registered.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Tested-by: Andrey Konovalov <andrey.konovalov@linaro.org>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
qcom-camss driver supports MIPI CSI2 receivers on APQ8016 and APQ8096
Application processors from Qualcomm, which are used on e.g. DragonBoard
410c and DragonBoard 820c dev boards respectively.
The simple pipeline handler provides basic support for these devices.
Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The current simple pipeline handler refuses to work with capture devices
which have V4L2_CAP_VIDEO_CAPTURE_MPLANE flag set in the device capabilities
field. This is too restrictive, as devices supporting the multi-planar API
can be using contiguous memory for semi-planar and planar formats, and this
would just work without any changes to libcamera.
Drop the guard against MPLANE devices, and replace it with the check of
the number of planes in the format the simple pipeline handler is going to
use for capture. This will let MPLANE devices which don't use non-contiguous
memory for frame buffers to work with the simple pipeline handler.
The following code in SimpleCameraData::init() filters out the pixel formats
libcamera doesn't support:
PixelFormat pixelFormat = videoFormat.first.toPixelFormat();
if (!pixelFormat)
continue;
So the check for the number of memory planes this patch adds would not
trigger until non-contiguous planar formats becomes supported in libcamera,
and video devices using these formats are enabled in the simple pipeline
handler. Then this check will remind one to review the simple pipeline
handler code.
Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Add a field to the CameraConfiguration (including the necessary
documentation) to represent a 2D transform requested by the
application. All pipeline handlers are amended to coerce this to the
Identity, marking the configuration as "adjusted" if something
different had been requested.
Pipeline handlers that support Transforms can be amended subsequently.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Use the CameraSensor ID as the camera name in pipelines that uses a
CameraSensors, this is done in preparation of turning the camera name
into an ID. The CameraSensor ID meets the requirements that will be put
on camera ID.
Before this change example of camera names:
* OF based systems
ov5695 7-0036
ov2685 7-003c
* ACPI based systems
ov13858 8-0010
ov5670 10-0036
* VIMC
VIMC Sensor B
After this change the same cameras are:
* OF based systems
/base/i2c@ff160000/camera@36
/base/i2c@ff160000/camera@36
* ACPI based systems
\_SB_.PCI0.I2C2.CAM0
\_SB_.PCI0.I2C4.CAM1
* VIMC
platform/vimc.0 Sensor B
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Now SimpleCameraData::setupFormats() can fail if the camera sensor
supports media bus code which some entities down the pipeline don't.
When this happens continue with the next media bus code instead of
aborting the enumeration of the possible pipeline configurations.
Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The SimpleCameraData::init() function needs to setup links along the
pipeline, but doesn't need to repeat that operation for each media bus
code it tries. Move the link setup before the loop.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Fill the stride and frameSize fields of the StreamConfiguration at
configuration validation time instead of at camera configuration time.
This allows applications to get the stride when trying a configuration
without modifying the active configuration of the camera.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Extend the SimpleConverter to support scaling, with reporting of the
minimum and maximum output sizes supported for a given input size.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Report the stride when configuring the camera. The stride is retrieved
from the capture device first, and overridden by the converter if used.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
The libcamera internal headers are located in src/libcamera/include/.
The directory is added to the compiler headers search path with a meson
include_directories() directive, and internal headers are included with
(e.g. for the internal semaphore.h header)
#include "semaphore.h"
All was well, until libcxx decided to implement the C++20
synchronization library. The __threading_support header gained a
#include <semaphore.h>
to include the pthread's semaphore support. As include_directories()
adds src/libcamera/include/ to the compiler search path with -I, the
internal semaphore.h is included instead of the pthread version.
Needless to say, the compiler isn't happy.
Three options have been considered to fix this issue:
- Use -iquote instead of -I. The -iquote option instructs gcc to only
consider the header search path for headers included with the ""
version. Meson unfortunately doesn't support this option.
- Rename the internal semaphore.h header. This was deemed to be the
beginning of a long whack-a-mole game, where namespace clashes with
system libraries would appear over time (possibly dependent on
particular system configurations) and would need to be constantly
fixed.
- Move the internal headers to another directory to create a unique
namespace through path components. This causes lots of churn in all
the existing source files through the all project.
The first option would be best, but isn't available to us due to missing
support in meson. Even if -iquote support was added, we would need to
fix the problem before a new version of meson containing the required
support would be released.
The third option is thus the only practical solution available. Bite the
bullet, and do it, moving headers to include/libcamera/internal/.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Jacopo Mondi <jacopo@jmondi.org>
The simple pipeline handler rejects devices that have multiple capture
video nodes. There's no real reason to do so, a more dynamic approach is
possible as the pipeline handler already locates the video device by
walking the media graph.
Rework the match sequence by skipping any check on the video nodes, and
create the V4L2VideoDevice for the media entity at the end of the
pipeline when initializing the camera data. The V4L2VideoDevice
instances are managed by the pipeline handler itself, to avoid creating
separate instances in the camera data if multiple sensors are routed to
the same video device.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Andrey Konovalov <andrey.konovalov@linaro.org>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Add support for an optional format converter, supported by the
SimpleConverter class. If a converter is available for the pipeline, it
will be used to expose additional pixel formats.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
This new pipeline handler aims at supporting any simple device without
requiring any device-specific code. Simple devices are currently defined
as a graph made of one or multiple camera sensors and a single video
node, with each sensor connected to the video node through a linear
pipeline.
The simple pipeline handler will automatically parse the media graph,
enumerate sensors, build supported stream configurations, and configure
the pipeline, without any device-specific knowledge. It doesn't support
configuration of any processing in the pipeline at the moment, but may
be extended to support simple processing such as format conversion or
scaling in the future.
The only device-specific information in the pipeline handler is the list
of supported drivers, required for device matching. We may be able to
remove this in the future by matching with the simple pipeline handler
as a last resort option, after all other pipeline handlers have been
tried.
Signed-off-by: Martijn Braam <martijn@brixit.nl>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Andrey Konovalov <andrey.konovalov@linaro.org>
Laurent Pinchart
Niklas Söderlund
Phi-Bang Nguyen
Jacopo Mondi
Naushir Patuck
Kieran Bingham
Hirokazu Honda
Andrey Konovalov
David Plowman
Paul Elder
Martijn Braam