The V4L2VideoDevice::toV4L2PixelFormat() function is incorrectly
implemented, as it will pick a multi-planar format if the device
supports the multi-planar API, even if only single-planar formats are
supported. This currently works because the implementation calls
V4L2PixelFormat::fromPixelFormat(), which ignores the multiplanar
argument and always returns a single-planar format.
Fixing this isn't trivial. As we don't need to support multi-planar V4L2
formats at this point, drop the function instead of pretending
everything is fine, and call V4L2PixelFormat::fromPixelFormat() directly
from pipeline handlers. As the single-planar case is the most common,
set the multiplanar argument to false by default to avoid long lines.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Hirokazu Honda <hiroh@chromium.org>
To use multiple cameras at the same time, a per-camera buffer ready
handler is needed. Move the bufferReady() function connected to the
V4L2VideoDevice bufferReady signal from the SimplePipelineHandler class
to the SimpleCameraData class.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
To use multiple cameras at the same time, each camera instance will need
its own converter. Store the converter in SimpleCameraData, and open it
in init().
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
The cameras created by the same pipeline handler instance may share
hardware resources, prohibiting usage of multiple cameras concurrently.
Implement a heuristic to reserve resources and handle mutual exclusiong
in a generic way.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
Move opening of video devices at match() time, the same way as subdevs
are opened, to make the handling of V4L2 video devices and subdevices
more consistent.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
Merge the SimplePipelineHandler videos_ and subdevs_ maps, which
respectively store V4L2 video devices and subdevices associated with
entities, into a single entities_ map that contains an EntityData
structure. This gathers all data about entities in a single place,
allowing for easy extension of entity data in the future.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
Store the entity corresponding to the video node at the end of the
pipeline in the SimpleCameraData::entities_ list. This requires special
handling of the video node in the loops that iterate over all entities,
but will be useful to implement mutually exclusive access to entities
for concurrent camera usage.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
The video device is currently opened in the SimpleCameraData
constructor. This requires opening the video devices on-demand the first
time they're accessed, which gets in the way of refactoring of
per-entity data storage in the pipeline handler.
Move opening of the video device to the SimpleCameraData::init()
function. The on-demand open mechanism isn't touched yet, it will be
refactored later.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
Record the sink and source pads through which an entity is traversed in
the list of entities stored in the camera data. This prepares for
implementing mutually exclusive access to entities between cameras.
The debug message that displays detected pipelines now prints pads to
describe the pipeline more precisely:
[0:00:35.901275750] [260] DEBUG SimplePipeline simple.cpp:404 Found pipeline: [imx290 2-001a|0] -> [0|csis-32e40000.csi|1] -> [0|mxc_isi.0|1] -> [0|mxc_isi.0.capture]
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
Replace manual static casts from the PipelineHandler pointer to a
derived class pointer with helper functions in the camera data classes.
This simplifies code accessing the pipeline from the camera data.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
As part of the effort to remove the CameraData class, migrate the
pipeline handler-specific camera data from CameraData to the
Camera::Private class.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In order to allow subclassing Camera::Private in pipeline handlers, pass
the pointer to the private data to the Camera constructor, and to the
Camera::createCamera() function.
The Camera::Private id_ and streams_ members now need to be initialized
by the Camera constructor instead of the Camera::Private constructor, to
allow storage of the streams in a pipeline handler-specific subclass of
Camera::Private.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
When the configuration of the converter fails due to format mismatch,
the error messages only indicates that a failure occurred. Improve it to
ease debugging by printing the requested and obtained formats.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Move the functionality for the following components to the new
base support library:
- BoundMethod
- EventDispatcher
- EventDispatcherPoll
- Log
- Message
- Object
- Signal
- Semaphore
- Thread
- Timer
While it would be preferable to see these split to move one component
per commit, these components are all interdependent upon each other,
which leaves us with one big change performing the move for all of them.
Reviewed-by: Hirokazu Honda <hiroh@chromium.org>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Commit 922833f774 ("libcamera: simple: Report sensor timestamp")
unconditionally tries to access the request through the capture buffer
to store the capture timestamp in the metadata. This causes a null
pointer dereference when using a converter, as the capture buffers are
free-wheeling in that case, and not associated with a request.
Fix this by getting the request from the user-facing buffer, which can
be the capture buffer when no converter is used.
Fixes: 922833f774 ("libcamera: simple: Report sensor timestamp")
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
The supportedDevices may contain entries which have the same driver
but different converters. For example, if we add these two entries:
{ "mtk-seninf", "mtk-mdp", 3 },
{ "mtk-seninf", "mtk-mdp3", 3 },
the simple pipeline handler will always take the first one where it
can acquire the driver and skip the rest.
So, make the changes to support this usecase.
Signed-off-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Report the sensor's timestamp in the Request metadata using the
completed buffer timestamp.
The buffer's timestamp is recorded at DMA-transfer time, and it does not
theoretically matches the 'start of exposure' definition. Record this with
a \todo entry.
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Hirokazu Honda <hiroh@chromium.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
When seting up the pipeline, the latest entity in the queue is
taken but the oldest one is poped. This is a mistake. Fix it.
Fixes: 4671911df0 ("pipeline: simple: Use breadth-first search to setup media pipeline")
Signed-off-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
After commit 4671911df0 ("pipeline: simple: Use breadth-first search
to setup media pipeline"), the explanation in the SimplePipeline
documentation how the handler tries to find a valid path to capture
device does not reflect the reality anymore. Update the text to the new
situation.
Fixes: 4671911df0 ("pipeline: simple: Use breadth-first search to setup media pipeline")
Signed-off-by: Marian Cichy <m.cichy@pengutronix.de>
Reviewed-by: Sebastian Fricke <sebastian.fricke@posteo.net>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
When the SimplePipeline is setting up its data and media pipeline in the
SimpleCameraData constructor, it merely tries to find the first valid
pad and link to the next entity, starting from the camera sensor.
Following this path may not always lead to a valid capture device and
therefore the setup will fail on some machines. This is for example an
issue when using the SimplePipeline on an i.MX-6Q with its i.MX IPU.
This commit implements a different approach to setup the media-pipeline
by finding the shortest path to a valid capture device, using the
breadth-first search algorithm. On i.MX6Q, the shortest path has a good
chance to be the path from the sensor to the CSI capture device, as
other paths may involve image converters, encoders or other IPU blocks
and will have therefore more nodes.
Signed-off-by: Marian Cichy <m.cichy@pengutronix.de>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Camera sensors can include an ISP. For instance, the AP1302 external ISP
can be connected to up to two raw camera sensors, and the combination of
the sensors and ISP is considered as a (smart) camera sensor from
libcamera's point of view.
The CameraSensor class has limited support for this already. Extend the
simple pipeline handler to support such sensors, by using the media
entity corresponding to the ISP instead of the raw camera sensor's
entity.
We don't need to handle the case where an entity in the SoC would expose
the MEDIA_ENT_F_PROC_VIDEO_ISP function, as pipeline containing an ISP
would have a dedicated pipeline handler.
The implementation is limited as it won't support other multi-entity
camera sensors (such as CCS). While this would be worth supporting, we
don't have a test platform with a CCS-compatible sensor at this point,
so let's not over-engineer the solution. Extending support to CCS (and
possibly other sensor topologies) will likely involve helpers that can
be used by other pipeline handlers (such as generic graph walk helpers
for instance) and extensions to the CameraSensor class.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Allow support for multiple streams on a per-device basis. The decision
should be made based on the ability of the converter to run multiple
times within the duration of one frame. Hardcode it in
SimplePipelineInfo for now.
We may later compute the number of supported streams dynamically based
on the requested configuration, using converter bandwidth information
instead of a hardcoded fixed value.
All platforms are currently limited to a single stream until they get
successfully tested with multiple streams.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Extend the SimpleCameraConfiguration to support multiple streams, using
the multi-stream capability of the SimpleConverter class. Wiring up
multi-stream support in the other pipeline handler operations will come
in further commits.
To keep the code simple, require all streams to use the converter if any
stream needs it. It would be possible to generate one stream without
conversion (provided the format and size match what the capture device
can generate), and this is left as a future optimization.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Store the list of converter output formats in the Configuration
structure, to be used to implement multi-stream support. As the
Configuration structure grows bigger, avoid duplicating it in the
formats_ map for each supported pixel format by storing it in a configs_
vector instead, and storing pointers only in the map.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
As the pipeline configuration is selected in
SimpleCameraConfiguration::validate() already, cache it in the
SimpleCameraConfiguration instead of looking it up in
SimplePipelineHandler::configure(). This makes little difference at the
moment, but will save duplication of more complex logic between
validate() and configure() when adding support for multiple streams.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The simple pipeline handler has grown over time, and isn't that simple
anymore that it can easily be understood by an unfamiliar reader.
Document the design to explicitly state the expectations of the pipeline
handler, and to explain how it operates.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
While the M2M device backing the converter doesn't support multiple
streams natively, it can be run once per stream to produce multiple
outputs from the same input, with different output formats and sizes.
To support this, create a class to model a stream and move control of
the M2M device to the Stream class. The SimpleConverter class then
creates stream instances and iterates over them. Each stream needs its
own instance of the V4L2M2MDevice, to support different output
configurations. The SimpleConverter class retains a device instance to
support the query operations.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The SimpleConverter API signals completion of input and output buffer
pairs. This unnecessarily delays requeueing the input buffer to the
video capture queue until the output buffer completes, and also delays
signalling request completion until the input buffer completes. While
this shouldn't cause large delays in practice, it will also not scale
when multi-stream support will be added to the converter class.
To address the current issue and prepare for the future, decouple
signalling of input and output buffers completion.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Simplify the SimpleConverter interface by opening the M2M device in the
constructor. The explicit call to open() is replaced by a check through
a new isValid() function, and the unused close() function is removed.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The number of buffers on the input and output of the converter don't
necessarily need to match. Use the buffer count from the input and
output configuration respectively. This removes the need to pass the
buffer count to the start() function, which brings it closer to the
pipeline handler API.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Group the configuration parameters for the converter input in a
StreamConfiguration instance. This makes the configure() function
signature cleaner, and will allow passing additional parameters (such as
stride and buffer count).
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The SimpleConverter class has different sets of functions, related to
static queries, device configuration and runtime operation. Group the
query functions together. While at it, swap the arguments to the
strideAndFrameSize() function to match the order in which pixel format
and size are usually specified.
No functional change is included.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The stride (and frame size) calculation has been moved from configure
time to configuration validate time by commit 89fb1efac2 ("libcamera: simple:
Fill stride and frameSize at config validation"). This change has
however left one stray setting of the stride when configuring the
converter. Fix it.
While at it, turn the SimpleConverter::configure() output configuration
argument to a const reference to emphasize it can't be null and isn't
modified by the function, and rename it from cfg to outputCfg to make
its purpose clearer.
Fixes: 89fb1efac2 ("libcamera: simple: Fill stride and frameSize at config validation")
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
There is no need to pass the Camera pointer to queueRequest(),
completeBuffer() and completeRequest() as the Request also passed
contains the same information. Remove the Camera argument to avoid
situations where the information in the Request and the argument differ.
There is no functional change and no public API change as the interface
is only used between the Camera and PipelineHandler.
Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Laurent Pinchart
Kieran Bingham
Phi-Bang Nguyen
Jacopo Mondi
Marian Cichy
Dafna Hirschfeld
Niklas Söderlund