Some kernel drivers give their entities names that will differ from
implementation to implementation; for example the drivers for the
Camera Receiver Unit and CSI-2 receiver in the RZ/V2H(P) SoC give their
entities names that include their memory address, in the format
"csi-16000400.csi2". Passing that entity name to
V4L2Subdevice::fromEntityName() is too inflexible given it would only
then work if that specific CSI-2 receiver were the one being used.
Add an overload for V4L2Subdevice::fromEntityName() to instead allow
users to pass a std::basic_regex, and use std::regex_search() instead
of a direct string comparison to find a matching entity. Ths allows
us to form regular expressions like "csi-[0-9a-f]{8}.csi2" to find
the entities.
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Some kernel drivers give their entities names that will differ from
implementation to implementation; for example the drivers for the
Camera Receiver Unit and CSI-2 receiver in the RZ/V2H(P) SoC give their
entities names that include their memory address, in the format
"csi-16000400.csi2". Passing that entity name to
MediaDevice::getEntityByName() is too inflexible given it would only
then work if that specific CSI-2 receiver were the one being used.
Add an overload for MediaDevice::getEntityByName() that accepts a
std::basic_regex instead of a string, and use std::regex_search()
instead of a direct string comparison to find a matching entity. This
allows us to search for entites using regex patterns like
"csi-[0-9a-f]{8}.csi2".
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Some entities in a media graph have names that might differ from
implementation to implementation; for example the Camera Receiver
Unit and CSI-2 receiver on the RZ/V2H(P) SoC have entities with names
that include their address, in the form "csi-16000400.csi2". Passing
that entity name to DeviceMatch is too inflexible given it would only
work if that specific CSI-2 receiver were the one being used.
Add an overload for DeviceMatch::add() such that users can pass in a
std::regex instead of a string. Update DeviceMatch::match() to check
for entities that are matched by the regular expressions added with
the new overload after checking for any exact matches from the vector
of strings. This allows us to use regex to match on patterns like
"csi-[0-9a-f]{8}.csi2".
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
This change integrates the MediaPipeline class into the imx8-isi
pipeline handler. Purpose is to allow a dynamic discovery and
configuration of the actual subdevices graph between the sensor and
the ISI crossbar. This brings support for more complex topologies and
simplifies the implementation.
Signed-off-by: Andrei Gansari <andrei.gansari@nxp.com>
Signed-off-by: Antoine Bouyer <antoine.bouyer@nxp.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Exposes internal MediaEntity::Entity list to help extracting more
information regarding linked entities.
For example, when the pad index of the last device in the list need to be
retrieved from the media pipeline user.
Exposes as const to with a dedicated access to prevent any corruption from
user. Then it is still protected so as when the list was private.
Since MediaPipeline::Entity needs also to be moved to public, then need to
add some documentation in cpp source. Existing documentation from header
file is applied when available.
Signed-off-by: Andrei Gansari <andrei.gansari@nxp.com>
Signed-off-by: Antoine Bouyer <antoine.bouyer@nxp.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Array controls (eg. ColourCorrectionMatrix, FrameDurationLimits,
ColourGains) are serialized properly by the ControlSerializer, but are
not deserialized properly. This is because their arrayness and size are
not considered during deserialization.
Fix this by adding arrayness and size to the serialized form of all
ControlValues. This is achieved by fully serializing the min/max/def
ControlValue's metadata associated with each ControlInfo entry in the
ControlInfoMap.
While at it, clean up the serialization format of ControlValues and
ControlLists:
- ControlValue's id is only used by ControlList, so add a new struct for
ControlList entries to contain it, and remove id from ControlValue
- Remove offset from ControlInfo's entry, as it is no longer needed,
since the serialized data of a ControlInfo has now been converted to
simply three serialized ControlValues
- Remove the type from the serialized data of ControlValue, as it is
already in the metadata entry
The issue regarding array controls was not noticed before because the
default value of the ControlInfo of other array controls had been set to
scalar values similar to how min/max are set, and ColourCorrectionMatrix
was the first control to properly define a non-scalar default value.
Bug: https://bugs.libcamera.org/show_bug.cgi?id=285
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Tested-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com> # rkisp1
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When the dewarper is present it can handle arbitrary orientations
specified in the requested camera configuration. In that case handle all
transformations inside the dewarper (even if the sensor supports some of
them) because that makes it easier to handle coordinates for lens
dewarping inside the dewarper.
This complicates the path selection a bit, as for transformations that
include a transpose, the format before the dewarper has swapped
width/height.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add a transpose() function to size that applies the
Transformation::Transpose operation in the size. This is useful when
handling orientation adjustments.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
To do actual lens dewarping, the dewarper will be configured based on
the tuning file.
As a first step implement the basic loading of the
tuning file and enable/disable the dewarper for the given camera based
on the existence of the "Dewarp" entry under a new top level element
'modules' in the tuning file.
Note: This is an backwards incompatible change in that the dewarper is
currently included in the chain unconditionally. Some users may want to
not use the dewarper, so it is sensible to make that configurable.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In raw mode the number of configurations is actively limited to 1. It is
therefore safe to move isRaw up one level to simplify the code and
prepare for later use.
During that rework it was noticed that the old code actually has a bug
in that it reduces the number of configurations to 1 in case a raw
config is found, but it doesn't reduce the config vector to that raw
config, but the first config.
Change that behavior to check the first config and either remove all
remaining configs if the first is raw or drop all raw configs if the
first is non-raw.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The dewarper integration into the rkisp1 pipeline is quite complicated.
Simplify that by switching to the now available ConverterDW100Module. As
there is no other known converter in combination with the rkisp1 ISP this
is a safe step to do.
This change also paves the way to implement dw100 specific features later.
The input crop implemented in the dw100 kernel driver is quite limited
in that it doesn't allow arbitrary crop rectangles but only scale
factors quantized to the underlying fixed point representation and only
aspect ratio preserving crops.
The vertex map based implementation allows for pixel perfect crops. The
only downside is that ScalerCrop can no longer be set dynamically on
older kernels. A corresponding warning is already implemented in the
converter module.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The DW100 Dewarp engine is present on i.MX8MP SoC and possibly others.
This patch provides a dedicated converter module that allows easy
integration of such a dewarper into a pipeline handler.
In this patch only the ScalerCrop control is implemented. Support for
additional functionality will be added in later patches.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Using a custom vertex map the dw100 dewarper is capable of doing
complex and useful transformations on the image data. This class
implements a pipeline featuring:
- Arbitrary ScalerCrop
- Full transform support (Flip, 90deg rotations)
- Arbitrary move, scale, rotate
ScalerCrop and Transform is implemented to provide a interface that is
standardized libcamera wide. The rest is implemented on top for more
flexible dw100 specific features.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When the dewarper is used, an addition buffer loop with
kRkISP1MinBufferCount buffers is created between ISP and dewarper. When
the dewarper is configured, it stores the bufferCount value of the
requested stream configurations. This number of buffers is then imported
when the dewarper is started.
On the input stream of the dewarper the bufferCount is currently left
unchanged, meaning it carries the bufferCount as supplied by the user
instead of the bufferCount of the additional loop. Fix that by setting
the bufferCount to kRkISP1MinBufferCount.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In order to allow digital zooming, scale down in the dewarper instead of
the resizer. That means forwarding the full sensor size data to the
dewarper. The ScalerCrop rectangle will also be applied at the dewarper.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The decision if the dewarper shall be used is not per pipeline but per
camera and per configuration (raw streams can't use it). Move the
corresponding flag into the camera data class. Rename the flag to
"usesDewarper" which is easier understand when we later add the ability
to enable/disable the dewarper on a per camera basis which will be
expressed using a "canUseDewarper" flag.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In case the buffer returned from the ISP was canceled, the upstream
buffer was not correctly marked as canceled. Move the cancel
functionality into an own helper function and correctly cancel the
upstream buffers. Add missing cancellation in case queuing to the
dewarper fails.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
To actually use requests with the m2m device, requests need to be
allocated on the underlying media device. This can only be done if the
media device is opened which means acquiring it. Add a function to check
if the m2m device supports requests by acquiring the media device,
asking it and then releasing it again. Also add a function to allocate
requests that acquires the internal media device and releases it after
allocating the requests.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add V4L2 request support to the V4L2M2MConverter class. Extend the
functions related to buffer queuing with an optional request parameter
that gets passed to the lower layers.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
The V4L2 requests API provides support to atomically tie controls to a
set of buffers. This is especially common for m2m devices. Such a
request is represented by an fd that is allocated via
MEDIA_IOC_REQUEST_ALLOC and then passed to the various V4L2 functions.
Implement a V4L2Request class to wrap such an fd and add the
corresponding utility functions.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
With the upcoming addition of V4L2 requests support, the converters need
to keep a handle to the corresponding media device.
Prepare for that by changing the constructor parameter from a raw
pointer to a shared pointer.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
Adapt the PipelineHandler::acquireMediaDevice() support function to
return a shared pointer instead of the underlying raw pointer.
Propagate this update to all pipeline handlers that use the MediaDevice
and store a std::shared_ptr<MediaDevice> accordingly.
This is required to support media devices that are potentially shared
among multiple pipeline handlers, like a dewarper implemented as v4l2
m2m device.
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
Virtual pipeline handler should provide colorSpace in
generateConfiguration() and validate the colorspace in validate().
It is mandatory for a pipeline handler to set the colorspace if it
is unset in the stream configuration, during validate().
For choosing the colorspace for the generated NV12 frames, following
points have been taken into account:
- The transfer function should be Rec.709 for NV12
- The YCbCr encoding has been chosen Rec.709 as it is the most common
than Rec.601/Rec.2020
- Range should be 'Limited' as with the NV12 pixel format.
Hence, the closest colorspace match is ColorSpace::Rec709 which is
set for the virtual pipeline handler.
Signed-off-by: Umang Jain <uajain@igalia.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Tested-by: Robert Mader <robert.mader@collabora.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Bayer pattern on the sensor can change while configuring it with the
intended capture format. This is due to the transform being applied on
the sensor which supports [v/h]flips.
During configure(), the simple pipeline handler does not detect any
bayer pattern changes that can arise due to the transformations being
applied via SimpleCameraData:setupFormats(). In such cases, the video
node will be configured in-correctly, without realising the bayer
pattern has changed on the sensor, for the given capture format.
This patch detects the bayer pattern change after the sensor has
been configured and retrieves the corresponding V4L2 pixel format
to correctly configure the video node and the input to converter or
Soft-ISP.
Signed-off-by: Umang Jain <uajain@igalia.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
StreamConfiguration's should have colorSpace set. This is not the case
in the simple pipeline. Let's set it there. This also fixes a crash in
`cam' due to accessing an unset colorSpace.
We set the colour spaces according to the pixel format. This is not
completely correct because pixel formats and colour spaces are
different, although not completely independent, things. But for the
lack of a better practical option to determine the colour space, we use
this.
Closes: https://gitlab.freedesktop.org/camera/libcamera/-/issues/294
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Simplify a bit by storing the `EventNotifier` objects directly in the
`std::map` instead of wrapping them in unique_ptr. An other advantage
is that it removes one allocation per fence.
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Use `std::memory_order_acquire` everywhere the dispatcher is loaded
to guarantee synchronization with the release-store that places
the pointer there.
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The requestComplete signal is not emitted when the camera is stopped and
the request is still in the waitingRequests_ queue. Fix that by calling
doQueueRequest() on the waiting requests after marking them as
cancelled. This ensures that the requests gets a proper sequence number
and are added to the queuedRequests_ list. This list is then iterated in
completeRequest() and leads to the requestComplete signal.
Closes: https://gitlab.freedesktop.org/camera/libcamera/-/issues/281
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
The Simple Pipeline is designed to support a wide variety of pipeline
configurations and attached devices and will enumerate the pixel formats
of the connected sensors to map these to libcamera formats where
available.
In fixed pipelines, where the pixel format is not mapped correctly it is
a desired behaviour to express this warning so that the pixelformat can
be added, while in the simple-pipeline case we do not expect warnings
for every discovered pixel format which is not supported by libcamera.
This currently manifests itself as very highly verbose warnings about
unsupported pixel formats are not desired when there are working formats
that have already been enumerated.
Fortunately in commit 434edb7b44 ("libcamera: formats: Fix warning for
unknown V4L2 pixfmt") we have a mechanism to disable the warning for
occasions where we wish to ignore unsupported formats.
Use this feature to disable the warning in the core V4L2PixelFormat and
instead report only a debug level print from the simple pipeline
handler.
On devices such as the Pinephone, this removes overly verbose warnings
for tiled YUV formats:
[0:06:39.291083146] [1922] ERROR SimplePipeline simple.cpp:1600 No valid pipeline for sensor 'gc2145 0-003c', skipping
[0:06:39.302229740] [1922] WARN V4L2 v4l2_pixelformat.cpp:346 Unsupported V4L2 pixel format HM12
[0:06:39.302779117] [1922] WARN V4L2 v4l2_pixelformat.cpp:346 Unsupported V4L2 pixel format HM12
[0:06:39.303417578] [1922] WARN V4L2 v4l2_pixelformat.cpp:346 Unsupported V4L2 pixel format HM12
[0:06:39.303928998] [1922] WARN V4L2 v4l2_pixelformat.cpp:346 Unsupported V4L2 pixel format HM12
[0:06:39.304615751] [1922] WARN V4L2 v4l2_pixelformat.cpp:346 Unsupported V4L2 pixel format HM12
Closes: https://gitlab.freedesktop.org/camera/libcamera/-/issues/291
Suggested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Fixes behavior when calling 'cam -l' during a live stream from a camera
in another process.
Issue is that multiple process should be able to list (match procedure)
the camera supported. But only the unique process that lock the media
devices in order to be able to configure then start the pipeline should
setup the routes, graphs, etc.
Thus, the setRouting() is to be moved to a PipelineHandlerISI::acquireDevice()
implementation to override the default Pipeline::acquireDevice() function.
Fixes: 92df79112f ("pipeline: imx8-isi: Add multicamera support")
Signed-off-by: Andrei Gansari <andrei.gansari@nxp.com>
Signed-off-by: Antoine Bouyer <antoine.bouyer@nxp.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
For debugging purposes, threads can be assigned a name, which eases
distinguishing between them in e.g. htop or gdb. This uses a
Linux-specific API for now which is limited to 15 characters (+ null
terminator), so truncation is done and names for existing thread
instantiations were chosen to be consise.
[Kieran: Apply checkstyle suggestions, rebase on proxy rework]
Signed-off-by: Schulz, Andreas <andreas.schulz2@karlstorz.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The DelayedControls class works around a limitation of the V4L2 controls
API by assigning to controls that modify the limits of other controls a
'priority' flag.
'Priority' controls are written to the device before others to make sure
the limits of dependent controls are correctly updated.
A typical example of a priority control is VBLANK, whose value changes the
limits of the EXPOSURE control. This doesn't apply to a specific hardware
platform but to all V4L2 sensors.
The RkISP1 pipeline handler doesn't mark VBLANK as a priority control, an
issue which might result in the exposure being clamped to an outdated frame
length.
Fix the rkisp1 pipeline by marking VBLANK as a priority control.
Fixes: f72c76eb6e ("rkisp1: Honor the FrameDurationLimits control")
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
The libcamera hex string adaptor specifies and casts each type
specifically to map the size of each type.
This needlessly repeats itself for each type and further more has a bug
with signed integer extension which causes values such as 0x80 to be
printed as 0xffffffffffffff80 instead.
Remove the template specialisations for each type, and unify with a
single templated constructor of the struct hex trait.
Suggested-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Bryan O'Donoghue
Milan Zamazal
Daniel Scally
Jacopo Mondi
Andrei Gansari
Paul Elder
Stefan Klug
Umang Jain
Kieran Bingham
Umang Jain
Barnabás Pőcze
Antoine Bouyer
Schulz, Andreas