When a converter or the software ISP is used, output sizes do not equal
input sizes - they notably can be smaller.
Previous to this patch only capture sizes were considered, in some cases
resulting in configs with too small maximum output sizes being selected,
such as 1912x1080 for stream sizes of 1920x1080.
Check that the maximum output sizes are big enough instead, while continuing
to minimize capture sizes.
Closes: https://gitlab.freedesktop.org/camera/libcamera/-/issues/236
Signed-off-by: Robert Mader <robert.mader@collabora.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Acked-by: Umang Jain <uajain@igalia.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Using auto variables for simple types reduces readability. Spell out
unsigned int explicitly here, and replace the <= 0 check with a zero
check now that the explicit type shows the value can't be negative.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Umang Jain <uajain@igalia.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
While a default value of 4 buffers appears to be a good default that is
used by other pipelines as well, allowing both higher and lower values
can be desirable, notably for:
1. Video encoding, e.g. encoding multiple buffers in parallel.
2. Clients requesting a single buffer - e.g. in multi-stream scenarios.
Thus allow buffer counts between 1 and 32 buffers - following the default
maximum from vb2 core - while keeping the default to the previous 4.
While on it mark the config as adjusted when appropriate.
Signed-off-by: Robert Mader <robert.mader@collabora.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The Simple Pipeline handler supports a variety of hardware with
different capabilities and performances.
To improve performance and reliability of the cameras across the
supported range, increase the number of internal buffers to 4.
This allows lower performance devices more opportunity to process the
frames and increases stability.
Align the Simple Pipeline handler and Soft ISP buffering with the other
hardware based platforms and use 4 internal buffers.
Signed-off-by: Robert Mader <robert.mader@collabora.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Currently the virtual pipeline generates the images synchronously. This is not
ideal because it blocks the camera manager's internal thread, and because its
behaviour is different from other existing pipeline handlers, all of which
complete requests asynchronously.
So move the image generation to a separate thread by deriving `VirtualCameraData`
from `Thread`, as well as `Object` and using the existing asynchronous signal
and method call mechanism.
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This patch allows enabling or disabling software ISP via config file in
addition to compile time. This can be useful for software ISP testing
on various platforms as well as for overriding the defaults in case the
defaults don't work well (e.g. hardware ISP may or may not work on
i.MX8MP depending on the kernel and libcamera patches present in the
given system).
The configuration is specified as follows:
configuration:
pipelines:
simple:
supported_devices:
- driver: DRIVER-NAME
software_isp: BOOLEAN
- ...
For example:
configuration:
pipelines:
simple:
supported_devices:
- driver: mxc-isi
software_isp: true
The overall configuration of enabling or disabling software ISP may get
dropped in future but this patch is still useful in the meantime.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Query the params device for RKISP1_CID_SUPPORTED_PARAMS_BLOCKS and
inject the information into the IPA hardware context for use by the
algorithms.
To be able to modify the hardware configuration at runtime, replace the
pointer with an instance and create a copy of the static hardware
specific data.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When the extensible parameters queued to the kernel contain an unknown
block type it fails with -EINVAL. This should not happen as user land is
supposed to check for the supported parameter types. But it took a while
to figure out where things went wrong. Add a error statement when
queuing of the parameter buffer fails for whatever reason.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In current implementation, the sink pad counter of the crossbar is not
incremented if the pad is not connected to any subdevice. This would lead
to incorrect routing and format configuration if CSI is not connected
to first sink pad.
To avoid such issue, every sink pads must be taken into account. Then if
CSI and sensor are present, current counter is used for routing at match(),
and stored in camera data to be reused during configure().
Signed-off-by: Antoine Bouyer <antoine.bouyer@nxp.com>
Tested-by: Pavel Löbl <pavel@loebl.cz>
Tested-by: Julien Vuillaumier <julien.vuillaumier@nxp.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
There is no reason to allocate the frame info objects dynamically,
and then store raw pointers in the `std::map` in the rkisp1
and ipu3 pipeline handler.
Instead, store the objects directly in the map. This removes
the need for manually calling new/delete, simplifies the code,
and eliminates one memory allocation per frame.
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Top-level `const` qualifiers are not useful, so avoid them. This is done
either by simply removing the top-level `const`, or making the function
return a reference to const where that is appropriate.
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Header comment blocks used to contain the file name. Considered as
useless information, the names have been removed, with the last ones
supposed to be dropped in commit d3bf27180e ("libcamera: Drop
remaining file names from header comment blocks"). A few have however
been forgotten, and more crept back since. Remove them.
While at it, fix one typo in a header comment block by replacing
'MaliC55 with Mali-C55', and add a missing blank line in
src/ipa/rpi/pisp/pisp.cpp.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
The match() function currently reports that it is not possible to create
any cameras if it encounters an empty media graph.
Fix this by looping over all media graphs and only returning false when
all of them fail to create a camera.
It is worth noting that an issue does exist when on a partial match that
ends in an invalid match, any media devices that were acquired will stay
acquired. This is not a new issue though, as any acquired media devices
in general are not released until pipeline handler deconstruction. This
requires a rework of how we do matching and pipeline handler
construction, so it is captured in a comment.
In the meantime, this fix fixes a problem without increasing the net
number of problems.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Tested-by: Hui Fang <hui.fang@nxp.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
ISI can be used to support multiple cameras at the same time in the media
device. A dedicated pad is assigned to each camera by HW (dts). We then
assign one (or more) source pad(s) depending on ISI constraints in the SoC.
Since ISI may have different number of pipes depending on SoC, the number
of cameras is computed _before_ going through all pipeline's components,
to limit number of pipes that could be assigned to a camera.
For instance, 3 is targeted as maximum amount of streams per camera as
defined by 'kNumStreams' constant. If 2 cameras are connected, with
only 5 ISI pipes, this target cannot be achieved. In such case, 2 streams
are assigned to each camera.
On the other hand, if ISI has 8 source pads (from index 6 to 13) and 2
cameras, first three source pads [6:8] are assigned to first camera, and
the three next source pads [9:11] are assigned to the second camera. All
these pads (sink and sources) must have same format to prevent configuration
mismatch during start.
However, since ISI routing must be performed _before_ any stream becomes
active, 'setRouting' is now executed during the 'match' step, instead of
'configure'. Indeed, it is up to the application to decide when a camera
'start' is executed: this could happen before or after the other camera
configuration is executed. Moving routing into the 'match' step makes sure
the routing is correctly applied before any 'start' is executed.
Signed-off-by: Antoine Bouyer <antoine.bouyer@nxp.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This patch adds number of streams per camera as constructor parameter, and
limits stream count to 3. Some applications may need up to 3 streams for
preview + capture + video record. Currently, imx8-isi pipeline only supports
up to 2. Increase constant parameter to 3 to match these applications'
requirements.
Minimum value between default value 3, and total amount of ISI's pipes is
now applied. For SOCs which only have 1 ISI pipe (ie i.MX93), available
stream count becomes 1.
Signed-off-by: Antoine Bouyer <antoine.bouyer@nxp.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The mali-c55 pipeline handler currently looks for a media entity with
the function MEDIA_ENT_F_IO_V4L to recognise a memory input subdevice.
This is apparently intended for video device entities, and we should
be looking for MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER. Correct the
entity function that the pipeline handler looks for.
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
At the moment the mali-c55 pipeline handler sets bytesused for a
buffer to be the maximum possible size (i.e. the size of a struct
mali_c55_params_buffer). This is not really in keeping with the goal
of the extensible parameters formats, and will not work with the new
framework for those formats. Update the IPA module and pipeline
handler to set bytesused to the size of the parameters that were
actually supplied rather than the maximum possible size.
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Currently the rkisp1 pipeline handler relies on bufferCount to decide on
the number of parameter and statistics buffers to allocate internally.
Instead, the number of internal buffers should be the minimum required
by the pipeline to keep the requests flowing, in order to avoid wasting
memory.
Stop relying on bufferCount for these numbers and instead set them to
kRkISP1MinBufferCount.
Signed-off-by: Nícolas F. R. A. Prado <nfraprado@collabora.com>
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Sven Püschel <s.pueschel@pengutronix.de>
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Umang Jain <uajain@igalia.com>
The StreamConfiguration::bufferCount is reset to a hardcoded value of 4
in RkISP1Path::validate(). Keep the minimum value of 4 but do not reset
it, if it was set to a larger value. This allows the user to set
bufferCount to an arbitrary number of buffers which then can be
allocated for example by the FrameBufferAllocator. If the bufferCount is
set to a smaller value it gets reset to kRkISP1MinBufferCount again and
the configuration is marked as adjusted.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Tested-by: Sven Püschel <s.pueschel@pengutronix.de>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Umang Jain <uajain@igalia.com>
To keep the regulation of the algorithms as fast as possible and at the
same time allow more buffers to be allocated, limit the amount of
buffers that get queued into the device to the pipeline depth plus a
tiny margin.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Tested-by: Sven Püschel <s.pueschel@pengutronix.de>
Reviewed-by: Umang Jain <uajain@igalia.com>
Since 6cf9c4d34f ("pipeline: ipa: rpi: Split RPiCameraData::dropFrameCount_")
the initial n frames are not dropped anymore. So clearing the request metadata
should not be necessary anymore, remove it.
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
`SensorTimestamp` and `FrameWallClock` should always be available. However,
if that ever changes or they are not available for some unforeseen reason,
setting them to 0 is not ideal. That makes it more complicated for the
application to detect these cases (since they have to check the existence
either way), and if an application blindly assumes e.g. that `SensorTimestamp`
is monotonically increasing, then receiving a timestamp of 0 will likely
cause issues.
So simply omit them from the request metadata if they are not available.
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Use nanoseconds for the FrameWallClock control to match the units for
other timestamp controls, including SensorTimestamp.
Update the RPi pipeline handlers to match the new nanoseconds units when
converting from SensorTimestamp to FrameWallClock.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When there are multiple entities between the sensor and CFE device (e.g.
a serialiser and deserialiser or multiple mux devices), the media graph
enumeration would work incorrectly and report that the frontend entity
was not found. This is because the found flag was stored locally in a
boolean and got lost in the recursion.
Fix this by explicitly tracking and returning the frontend found flag
through the return value of enumerateVideoDevices(). This ensures the
flag does not get lost through nested recursion.
This flag can also be used to fail a camera registration if the frontend
is not found.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Umang Jain <uajain@igalia.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The `AeEnable` control is handled in `Camera::queueRequest()` but it
still reaches the pipeline handler because a single element cannot be
removed from a `ControlList`. So ignore it silently.
Fixes: ffcecda4d5 ("libcamera: pipeline: uvcvideo: Report new AeEnable control as available")
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
A ClockRecovery object is added for derived classes to use, and
wallclock timestamps are copied into the request metadata for
applications.
Wallclock timestamps are derived corresponding to the sensor
timestamp, and made available to the base pipeline handler class and
to IPAs, for both vc4 and pisp platforms.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
So far, imx8-isi pipeline supports _symetrical_ crossbar, with same
amount of sink and source pads.
But for some other imx SoCs, such as i.MX8QM or i.MX95, crossbar is not
symetric anymore.
Since each crossbar source is already captured as a pipes_ vector entry,
we use pipes_ vector's size to compute 1st source index.
"1st source index" = "total number of crossbar pads" - pipes_.count()
Signed-off-by: Antoine Bouyer <antoine.bouyer@nxp.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
With the previous change to not drop frames in the pipeline handler,
the "disable_startup_frame_drops" pipeline config option is not used.
Remove it, and throw a warning if the option is present in the YAML
config file.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Split the pipeline handler drop frame tracking into startup frames and
invalid frames, as reported by the IPA.
Remove the drop buffer handling logic in the pipeline handler. Now all
image buffers are returned out with the appropriate FrameStatus set
for startup or invalid frames.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Replace the dropFrameCount parameter returned from ipa::start() to the
pipeline handler by startupFrameCount and invalidFrameCount. The former
counts the number of frames required for AWB/AGC to converge, and the
latter counts the number of invalid frames produced by the sensor when
starting up.
In the pipeline handler, use the sum of these 2 values to replicate the
existing dropFrameCount behaviour.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In the MaliC55CameraData::init() function there are two places that
return values they shouldn't; the ret variable is returned after
checking a pointer is not null instead of an explicit -ENODEV and later
the boolean value false is returned on failure instead of the error
value returned by V4L2Subdevice::open() - fix both problems.
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Robert Mader
Laurent Pinchart
Barnabás Pőcze
Milan Zamazal
Stefan Klug
Hans de Goede
Antoine Bouyer
Umang Jain
Paul Elder
Daniel Scally
Nícolas F. R. A. Prado
Naushir Patuck
David Plowman
Niklas Söderlund
Kieran Bingham