Pass the path name of the YAML IPA tuning file to the IPA module. The
file name is derived from the sensor name ("${sensor_name}.yaml"), with
a fallback to "uncalibrated.yaml".
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Currently the pipeline handler advertises controls handled by the IPA
from a ControlInfoMap it manually constructs. This is wrong, as the IPA
module is the component that knows what controls it supports. Fix this
by moving the ControlInfoMap construction to the IPA module, and pass it
to the pipeline handler as a return value from the IPA init() function.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Florian Sylvestre <fsylvestre@baylibre.com>
Several NXP i.MX8 SoCs (such as the i.MX8MN and i.MX8MP) contain a
camera pipeline made of sensor interfaces (with parallel and/or CSI-2
receivers) and an image processing engine named ISI. The ISI contains an
input crossbar switch and one or more processing pipelines capable of
format conversion and scaling.
This is a good candidate for the simple pipeline handler as a first
step. The i.MX8MP should eventually graduate to having its own pipeline
handler as it also contains two ISP instances (supported by the rkisp1
driver).
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Tested-by: Jacopo Mondi <jacopo@jmondi.org>
If a subdev supports the internal routing API, pads unrelated to the
pipeline for a given camera sensor may carry streams for other cameras.
The link setup logic is updated to take this into account, by avoiding
disabling links to unrelated pads.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
When traversing the media graph to discover a pipeline from the camera
sensor to a video node, all sink-to-source paths inside subdevs are
considered. This can lead to invalid paths being followed, when a subdev
has restrictions on its internal routing.
The V4L2 API supports exposing subdev internal routing to userspace.
Make use of this feature, when implemented by a subdev, to restrict the
internal paths to the currently active routes. If a subdev doesn't
implement the internal routing operations, all source pads are
considered, as done today.
This change is needed to properly support multiple sensors with devices
such as the NXP i.MX8 ISI or the MediaTek i350 and i500 SENINF. Support
for changing routes dynamically will be added later when required.
Signed-off-by: Phi-Bang Nguyen <pnguyen@baylibre.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Tested-by: Jacopo Mondi <jacopo@jmondi.org>
To setup links the pipeline handler iterates over all entities in the
pipeline and disables all links but the ones we want to enable. Some
entities don't allow multiple sink links to be enabled, so the iteration
needs to be based on sink entities, disabling all their links first and
then enabling the sink link that is part of the pipeline.
The loop implementation iterates over all Entity instances, and uses
their source link to locate the MediaEntity for the connected sink. The
sink entity is then processed in the context of the source's loop
iteration. This prevents the code from being able to access extra
information about the sink entity, as we only have access to the
MediaEntity, not the Entity.
To prepare for subdev routing support that will require accessing
additional entity information, refactor the loop to process the sink
entity in the context of its Entity instance.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Tested-by: Jacopo Mondi <jacopo@jmondi.org>
Reset the routing table of subdevices supporting the V4L2 streams API to
its default state when initializing the pipeline handler. This avoids
issues caused by usage of leftover state.
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Tested-by: Jacopo Mondi <jacopo@jmondi.org>
This is a partial revert of commit 395d43d6d7 ("libcamera:
v4l2_videodevice: Drop toV4L2PixelFormat()")
The function was removed because it incorrectly maps non-contiguous V4L2
format variants (ie V4L2_PIX_FMT_YUV420M) to the API version supported
by the video device (singleplanar API and multiplanar API). It was
decided at the time to remove the function and let its users call
directly V4L2PixelFormat::fromPixelFormat() which accepts a
'multiplanar' flags.
As we aim to associate multiple V4L2PixelFormat to a single libcamera
format, the next patches will verify which of them is actually supported
by the video device. For now, return the contiguous version
unconditionally.
Re-introduce V4L2VideoDevice::toV4L2PixelFormat() and convert all
the V4L2PixelFormat::fromPixelFormat() users to use it.
The V4L2 compatibility layer is the only outlier as it doesn't have a
video device to poke, hence it still uses
V4L2PixelFormat::fromPixelFormat().
Next patches will implement the device format matching logic and handle
the non-contiguous plane issue in V4L2VideoDevice::toV4L2PixelFormat().
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Paul Elder <paul.elder@ideasonboard.com>
Denoise and Sharpness filters will be applied by RkISP1 during the
demosaicing step. The denoise filter is responsible for removing noise
from the image, while the sharpness filter will enhance its acutance.
Add filter algorithm with denoise and sharpness values based on user
controls.
Signed-off-by: Florian Sylvestre <fsylvestre@baylibre.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The logic used to match asynchronous image and embedded buffers was being overly
aggressive by possibly allowing an unmatched image buffer to be sent to the IPA
if the matching embedded buffer had not yet been dequeued. This condition only
occurs when the system is heavily loaded and dropping frames.
Fix this by holding image buffer in the queue during these conditions until the
next embedded buffer dequeue event.
Reported-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com>
Tested-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Some hardware supported by the rkisp1 driver, such as the ISP in the
i.MX8MP, don't have a self path. Although at the moment the driver still
exposes the self path, prepare the rkisp1 pipeline handler for when the
self path will be removed for devices that don't support it.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Now that the ControlList::get() function returns an instance of
std::optional<>, we can replace the ControlList::contains() calls with a
nullopt check on the return value of get(). This avoids double lookups
of controls through the code base.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Previously, ControlList::get<T>() would use default constructed objects to
indicate that a ControlList does not have the requested Control. This has
several disadvantages: 1) It requires types to be default constructible,
2) it does not differentiate between a default constructed object and an
object that happens to have the same state as a default constructed object.
std::optional<T> additionally stores the information if the object is valid
or not, and therefore is more expressive than a default constructed object.
Signed-off-by: Christian Rauch <Rauch.Christian@gmx.de>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
If the pixel format gets adjusted to the default NV12 format during
validation, it gets currently ignored and the user-supplied format is
used nonetheless.
Fix it by using the adjusted pixel format.
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The controls::ScalerCrop in the ControlInfoMap was advertised based on
the ISP output Rectangle. This is incorrect, it needs to be set based on
the sensor analogue crop Rectangle. Fix this.
Additionally, do not use emplace to be consistent with the other
controls set in the ControlInfoMap.
Fixes: 9dacde0d65 (pipeline: raspberrypi: Advertise ScalerCrop from the pipeline handler)
Reported-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The rkisp1 hardware supports both a CSI-2 input and a parallel input,
where the sensor is connected directly to the ISP. On RK3399, the CSI-2
receiver is internal, but on the i.MX8MP, the CSI-2 receiver is a
separate IP core, connected to the parallel input of the ISP, and gets
exposed to userspace as a V4L2 subdev. To prepare for this, handle an
optional CSI-2 receiver subdev in the pipeline.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Jacopo Mondi <jacopo@jmondi.org>
Pass the path name of the YAML IPA tuning file to the IPA module. The
file name is derived from the sensor name ("${sensor_name}.yaml"), with
a fallback to "uncalibrated.yaml".
The tuning file name can be manually overridden with the
LIBCAMERA_RKISP1_TUNING_FILE environment variable.
Signed-off-by: Florian Sylvestre <fsylvestre@baylibre.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The ScalerCrop control is handled directly by the pipeline handler. Remove the
control from the IPA's static ControlInfoMap, and let the pipeline handler add
it to the ControlInfoMap advertised to the application, ensuring the limits
are set appropriately based on the current sensor mode.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The ipa currently advertises a static ControlInfoMap to specify the available
controls and their limits. Fix this limitation by having the IPA populate a new
ControlInfoMap with updated limits for ExposureTime, AnalogueGain, and
FrameDurationLimits controls based on the current sensor mode. This new
ControlInfoMap is then returned back to the pipeline handler and available to
the application after a successful Camera::configure() call.
Before the first call to Camera::configure(), this ControlInfoMap provides some
reasonable default limits for ExposureTime, AnalogueGain, and
FrameDurationLimits. However, applications must not rely on these values, but
obtain the correct limits after the call to Camera::configure().
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Currently the pipeline handler advertises controls handled by the IPA from a
static ControlInfoMap defined in the raspberrypi.h header. This change removes
this header file, and instead the IPA returns the ControlInfoMap to the pipeline
handler from the ipa::init() function. This is done to allow the IPA to adjust
the limits of the controls based on the sensor mode in a subsequent change.
Bug: https://bugs.libcamera.org/show_bug.cgi?id=83
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
As the simple pipeline handler targets simple pipelines on the SoC side,
it often gets used with platforms that have a YUV sensor capable of
outputting different sizes. Extend the heuristics used for pipeline
discovery and configuration to scale as much as possible on the sensor
side, in order to minimize the required bandwidth.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Dorota Czaplejewicz <dorota.czaplejewicz@puri.sm>
Reviewed-by: Dorota Czaplejewicz <dorota.czaplejewicz@puri.sm>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When enumerating the supported configurations, store the corresponding
sensor resolution in the SimpleCameraData::Configuration structure and
use it when configuring the camera, instead of hardcoding the sensor
full resolution. This prepares for support of downscaling in the camera
sensor.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Dorota Czaplejewicz <dorota.czaplejewicz@puri.sm>
Reviewed-by: Dorota Czaplejewicz <dorota.czaplejewicz@puri.sm>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Commit 8a845ab078 ("libcamera: Replace toString with operator<<() for
format classes") incorrectly converted some of the toString() usages,
resulting in pointer values being printed instead of formats. Fix it.
Fixes: 8a845ab078 ("libcamera: Replace toString with operator<<() for format classes")
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Use the removable attribute exposed by the USB device in sysfs to infer
if the camera is an external (when removable) or front (when
non-removable) camera. This is likely not perfect as the removable
attribute is derived from ACPI data which may not always be accurate,
and non-ACPI platforms likely report all devices as removable. Still,
this is a first step upon which a better method could be built.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The freeBuffers() cleanup code calls into the IPA to unmap and free
shared buffers. However, this function could be called before the IPA
has opened (via registerCamera()), causing a segmentation fault. Fix
this by guarding against calling the IPA if it has not been opened.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
These changes retrieve the correct value for sensitivity of the mode
selected for the sensor. This value is known to the CamHelper which
passes it across to the pipeline handler so that it can be set
correctly in the camera properties.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The IPAIPU3 interface currently uses event-type based structures in
order to communicate with the pipeline-handler (and vice-versa).
Replace the event based structures with dedicated functions associated
to each operation.
The translated naming scheme of actions to signals are:
ActionSetSensorControls => setSensorControls
ActionParamFilled => paramsBufferReady
ActionMetadataReady => metadataReady
The translated naming scheme of events to dedicated functions are:
EventProcessControls => queueRequest()
EventStatReady => processStatsBuffer()
EventFillParams => fillParamsBuffer()
The dedicated functions are called from pipeline-handler to the IPA
using IPC. These functions run asynchronously and when completed,
the IPA emits the respective signals as stated above in the translated
naming scheme.
The EventProcessControls is translated to queueRequest() to bring
symmetry to the IPU3 interface with other IPA interfaces.
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Enable the V4L2VideoDevice dequeue timeout for the Unicam Image node, and
connect the timeout signal to a slot in the pipeline handler. This slot will
log an error message informing the user of a possible hardware stall.
The timeout is calculated as 2x the maximum frame length possible for a given
mode, returned by the IPA.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Queuing of request (i.e. passing of controls to the IPA)
and filling of the parameters buffer are two separate operations.
Treat them as such by splitting them into two functions in
the rkisp1 IPA interface.
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The original use of RPi::Stream::reset() was to clear the external flag state
and free/clear out the framebuffers for the stream. However, the latter is now
done through PipelineHandlerRPi::configure(). Rework
PipelineHandlerRPi::configure() to call RPi::Stream::setExternal() instead of
RPi::Stream::reset() to achieve the same thing.
Repurpose RPi::Stream::reset() to instead reset the state of the buffer handling
logic, where all internally allocated buffers are put back into the queue of
available buffers. As such, rename the function to RPi::Stream::resetbuffers().
This resetbuffers() is now called from PipelineHandlerRPi::start(), allowing the
pipeline handler to correctly deal with start()/stop()/start() sequences and
reusing the buffers allocated on the first start().
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Currently, all framebuffer allocations get freed and cleared on a stop in
PipelineHandlerRPi::stopDevice(). If PipelineHandlerRPi::start() is then called
without an intermediate PipelineHandlerRPi::configure(), it will re-allocate and
prepare all the buffers again, which is unnecessary.
Fix this by not freeing the buffer in PipelineHandlerRPi::stopDevice(), but
insted doing it in PipelineHandlerRPi::configure(), as the buffers might have
to be resized.
Add a flag to indicate that buffer allocations need to be done on the next
call to PipelineHandlerRPi::start().
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The V4L2DeviceFormat structure for the ISP Output 1 node was a copy of what is
used ISP Output 0 node, but with the size changed. However, the plane size and
stride values were not updated. So there is a possibility that the buffer might
be over-sized for the requested resolution.
Fix this by only copying the relevant fields from the ISP Output 0
V4L2DeviceFormat structure, and let the device driver size the planes as needed.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The RkISP1Path::generateConfiguration() function limits the maximum
resolution to the sensor resolution, to avoid upscaling. It however
doesn't take the sensor aspect ratio into account, which leads to a
maximum (and default) resolution of 1920x1920 when using the self path
with a sensor that has a higher resolution.
Fix it by constraining the minimum and maximum resolutions to match the
sensor's aspect ratio.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The IPARkISP1Interface currently uses event-type based structures in
order to communicate with the pipeline-handler (and vice-versa).
Replace the event based structures with dedicated functions associated
to each operation.
The translated naming scheme of operations to dedicated functions:
ActionV4L2Set => setSensorControls
ActionParamFilled => paramsBufferReady
ActionMetadata => metdataReady
EventSignalStatBuffer => processStatsBuffer()
EventQueueRequest => queueRequest()
The lexical of IPARkISP1::metadataReady() will now conflict with the
metadataReady Signal being introduced in this patch as part of the
interface change. Hence, rename IPARkISP1::metadataReady() to
IPARkISP1::prepareReady() to prevent the conflict.
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Tested-by: Paul Elder <paul.elder@ideasonboard.com>
Laurent Pinchart
Phi-Bang Nguyen
Jacopo Mondi
Florian Sylvestre
Naushir Patuck
Xavier Roumegue
Paul Elder
Christian Rauch
David Plowman
Dorota Czaplejewicz
Umang Jain
Daniel Scally
Jean-Michel Hautbois