This reverts commit e85c7ddd38.
Linux kernel predating 6.4 (specifically commit 7cfb35d3a800 ("media:
rkisp1: Implement ENUM_FRAMESIZES") do not have the ioctl in rkisp1
driver required to dynamically query the resizer limits.
Because of that, maxResolution and minResolution are both {0, 0}
(default value for Size objects) which means filterSensorResolution()
will create an entry for the sensor in sensorSizesMap_ but because the
sensor resolution cannot fit inside the min and max resolution of the
rkisp1, no size is put into this entry in sensorSizesMap_.
On the next call to filterSensorResolution(),
sensorSizesMap_.find(sensor) will return the entry but when attempting
to call back() on iter->second, it'll trigger an assert because the size
array is empty.
Linux kernel 6.1 is supported until December 2027, so it seems premature
to get rid of those hard-coded resizer limits before this happens.
Let's restore the hard-coded resizer limits as fallbacks, actual limits
are still queried from the driver on recent enough kernels.
Fixes: 761545407c ("pipeline: rkisp1: Filter out sensor sizes not supported by the pipeline")
Signed-off-by: Quentin Schulz <quentin.schulz@cherry.de>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The DelayedControls instance for the camera sensor is created in
SimplePipelineHandler::configure(). Constant deletion and reconstruction
of a new object is unnecessary, as the control delays are an intrinsic
property of the sensor and are known at initialization time. Move the
DelayedControls creation to the SimpleCameraData class constructor.
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com> # v6
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Stanislaw Gruszka <stanislaw.gruszka@linux.intel.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The simple pipeline handler uses frame start events to apply sensor
controls through the DelayedControls class. The setSensorControls()
function applies the controls directly, which would result in controls
being applied twice, if it wasn't for the fact that the pipeline handler
forgot to enable the frame start events in the first place. Those two
issues cancel each other, but cause controls to always be applied
directly.
Fix the issue by only applying controls directly in setSensorControls()
if no frame start event emitter is available, and by enabling the frame
start events in startDevice() otherwise. Disable them in stopDevice()
for symmetry.
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com> # v6
Co-developed-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Co-developed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Stanislaw Gruszka <stanislaw.gruszka@linux.intel.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
If `!target`, then `*target` is undefined behaviour, so check if the optional
is empty when printing the error message. Simplify the check as well.
Fixes: 6c71ee1f15 ("pipeline: raspberrypi: Introduce PipelineHandlerBase class")
Fixes: 841ef2b4bb ("pipeline: rpi: Add support for Raspberry Pi 5")
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
The mapping in `UVCCameraData::processControl()` is not entirely correct
because the control value is retrieved as a `bool` instead of `int32_t`.
Additionally, the available modes are not taken into account.
Retrieve the control value with the right type, `int32_t`, check if the
requested mode is available, and if so, set the appropriate V4L2 control
value selected by `addControl()` earlier.
Fixes: bad8d591f8 ("libcamera: uvcvideo: Register ExposureTimeMode control")
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
`ControlInfo(Span<const int32_t>{...})` calls the incorrect constructor
of `ControlInfo`. The intended constructor to be called is
`ControlInfo(Span<const ControlValue>, ...)` however that is not called
because a span of `const int32_t` is passed. Instead, the constructor
`ControlInfo(const ControlValue &min, const ControlValue &max, ...)`
will be called.
Furthermore, since `values.back()` is used, only the last element of
the array is actually set.
To fix this, convert the array to contain `ControlValue` objects and use
a separate variable to keep track of which element to set next.
For each of `ExposureTimeMode{Auto,Manual}` save the V4L2 control value
that is to be used when the libcamera control is set.
Fixes: bad8d591f8 ("libcamera: uvcvideo: Register ExposureTimeMode control")
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Extend the Simple IPA IPC to support returning a metadata ControlList
when the process call has completed.
A new signal from the IPA is introduced to report the metadata,
similarly to what the hardware pipelines do.
Merge the metadata reported by the ISP into any completing request to
provide to the application. Completion of a request is delayed until
this is done; this doesn't apply to canceled requests.
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Hardware pipelines track requests and other information related to
particular frames. This hasn't been needed in software ISP so far. But
in order to be able to track metadata corresponding to a given frame,
frame-request tracking mechanism starts being useful. This patch
introduces the basic tracking structure, actual metadata handling is
added in the following patch.
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com> # Lenovo X13s
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
gcc 13.3.0 from buildroot 2024.11.1 complains about an uninitialized
variable. This is a false positive as the cfe_ array can't be empty.
Nonetheless, it breaks builds, so initialize the variable to work around
the issue.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add the following new stream flags:
Needs16bitEndianSwap - Indicates that a 16-bit endian swap needs to be
performed on the framebuffer in software.
Needs14bitUnpack - Indicates that a CSI-2 14-bit unpacking (to 16-bits)
needs to be performed on the framebuffer in software.
These are to workaround hardware restrictions in the CFE hardware that
will be supported in a future commit.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The doxygen \todo directive doesn't need to be followed by a colon. Drop
it. While at it, turn one 'todo:' into '\todo'.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Just return an `std::unique_ptr` constructed from an empty
initializer instead of doing a `reset()` on the existing
`config` variable and returning that. This is simpler.
Signed-off-by: Barnabás Pőcze <pobrn@protonmail.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
inputBufferReady ready signal in the simple pipeline is handled in the
pipeline handler thread. outputBufferReady and ispStatsReady signals
should be handled there too.
Rather than relying on the user of the SoftwareIsp instance, let
SoftwareIsp inherits Object. SoftwareIsp serves as a signal proxy, the
signals above are emitted from signal handlers. This means that if
SoftwareIsp inherits Object then the slots are invoked in SoftwareIsp
thread. Which is the camera manager thread because the SoftwareIsp
instance is created there.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The rkisp1 and rpi pipeline handlers duplicate code to handle the
LIBCAMERA_RKISP1_TUNING_FILE and LIBCAMERA_RPI_TUNING_FILE environment
variables that override tuning file selection. Move the common code to
IPAProxy::configurationFile() to avoid the duplication, and make the
feature available to all pipeline handlers with the same behaviour.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Port the UVC pipeline handler to use the new ExposureTimeMode control
when processing Camera controls in place of the AeEnable control.
The V4L2_CID_EXPOSURE_AUTO control allows 4 possible values, which
map to ExposureTimeModeAuto and ExposureTimeModeManual.
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Install the example configuration file of the virtual pipeline
handler as it serves documentation purposes, and to make the
virtual pipeline handler easily usable in CI.
Nonetheless, the file is installed with the ".example" suffix
so that it will not be used by default, to avoid cluttering
the camera lists of users whose distributions decide to
enable the virtual pipeline handler.
The file is installed in the proper location for convenience:
(1) is is easier to use it in the CI;
(2) users need not browse documentation to determine where
they should place the file.
Signed-off-by: Barnabás Pőcze <pobrn@protonmail.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Harvey Yang <chenghaoyang@chromium.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The virtual pipeline handler prints an error message when its
configuration file can't be opened. Not providing a configuration file
is the default method to disable virtual cameras, so this error is
confusing for users. Replace it with a debug message when the
configuration file is not found, and keep an error message when it
exists but can't be opened.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Julien Vuillaumier <julien.vuillaumier@nxp.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
After the initial generation, when a frame is requested,
the test pattern generator rotates the image left by 1 column.
The current approach has two shortcomings:
(1) it allocates a temporary buffer to hold one column;
(2) it swaps two columns at a time.
The test patterns are simple ARGB images, in row-major order,
so doing (2) works against memory prefetching. This can be
addressed by doing the rotation one row at a time as that way
the image is addressed in a purely linear fashion. Doing so also
eliminates the need for a dynamically allocated temporary buffer,
as the required buffer now only needs to hold one sample,
which is 4 bytes in this case.
In an optimized build, this results in about a 2x increase in the
number of frames per second as reported by `cam`. In an unoptimized,
ASAN and UBSAN intrumented build, the difference is even bigger,
which is useful for running lc-compliance in CI in a reasonable time.
Signed-off-by: Barnabás Pőcze <pobrn@protonmail.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
`PixelFormatInfo::planes.size()` always returns 3 since `planes` is
an array, but that is not the number of planes of the pixel format.
Use the `numPlanes()` getter instead.
Signed-off-by: Barnabás Pőcze <pobrn@protonmail.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Plumb the Pipeline-IPA loop in.
Load the IPA module at camera creation time and create the loop between
the pipeline and the IPA.
When a new Request is queued the IPA is asked to prepare the parameters
buffer, once ready it notifies the pipeline which queues the parameters
to the ISP along with a buffer for statistics and frames,
Once statistics are ready they get passed to the IPA which upates its
settings for the next frame.
Driveby fix an error message in the Pipeline Handler's ::freeBuffers()
function which reported a problem with the wrong video device in an
error path.
Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Acquire the mali-c55 3a stats and parameters video devices during
::match() and plumb them in.
For this commit we simply allocate and release buffers for the
statistics and parameters. Statistics buffers are queue and dequeued
from the stats video device but their contents are for now untouched.
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Acked-by: Nayden Kanchev <nayden.kanchev@arm.com>
Co-developed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Now that we have camera sensor control application delay values in
the CameraSensorProperties class, remove the duplicated definitions
in the RPi IPA's CameraSensorHelpers and update the pipeline handler
to use the values from CameraSensorProperties.
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In RkISPPath::validate() the sensor sizes are correctly filtered to the
ones supported by the ISP. But later in RkISPPipeline::configure() the
configured stream size is passed to sensor->getFormat() and the
CameraSensor class chooses the best sensor format for the requested
stream size. This can result in a sensor format that is too big for the
ISP. Fix that by supplying the maximum resolution supported by the ISP
to setFormat().
Fixes: 761545407c ("pipeline: rkisp1: Filter out sensor sizes not supported by the pipeline")
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
When the dewarper is used, config->validate() needs to take the
restrictions of the dewarper into account. Add the corresponding checks.
As the useDewarper_ variable is now accessed earlier in
PipelineHandlerRkISP1::configure(), ensure it gets set early enough.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
For the validate() implementation, the RkISP1CameraConfiguration needs
access to dewarper related members of the PipelineHandlerRkISP1 object.
Allow this access by adding const versions of the pipe accessors and
making RkISP1CameraConfiguration a friend of PipelineHandlerRkISP1.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
In configure() and in the future in generateConfiguration() the
calculated stream sizes and crop rectangles depend on the dewarper being
available or not. It is therefore not possible to disable the dewarper
later if configuration fails. Error out in that case.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Refactor validation code to prepare for extensions in the upcoming
patches. Code duplication is reduced by moving parts of the validation
logic into a lambda function. This patch does not include any functional
changes.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
ScalerCrop is specified as being in sensor coordinates. The current
dewarper implementation on the imx8mp handles ScalerCrop in dewarper
coordinates. This leads to unexpected results and an unusable ScalerCrop
control in camshark. Fix that by transforming back and forth between
sensor coordinates and dewarper coordinates.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Query the crop bounds on the dewarper instead of the stream in case the
camera was not yet configured when updateControls() gets called. This
provides sane defaults for the controls.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
The sensorInfo (specifically the crop rectangle of the selected sensor
mode) is collected to be passed to the IPA later. In an upcoming patch
that data will also be needed for correct ScalerCrop handling. Move the
collection of the sensorInfo before the dewarper configuration step and
refactor the code a bit.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
One Rectangle instance is used to calculate the inputCrop and the
outputCrop of the ISP in the rkisp1 pipeline. Split that into two
distinct variables, because both values will be needed in the upcoming
patches. This patch does not contain any functional changes.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
In the current code, the input stage of the image resizer is used to
apply a crop to keep the aspect ratio in cases where the requested
output aspect ratio differs from the one of the selected sensor mode. On
the imx8mp the resizer hardware is not capable of cropping (for
reference see also rkisp1-resizer.c:rkisp1_rsz_set_sink_crop() in the
linux kernel v6.10).
Therefore apply the necessary cropping on the output of the ISP (on the
image stabilization block). The cropping code on the image resizer
doesn't need modifications as the requested crop gets ignored by the
kernel.
While at it, remove a todo comment that is no longer needed.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
This patch introduces support for applying runtime controls to software
ISP. It enables the contrast control as the first control that can be
used.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Implement support for the ScalerCrop control that allows to apply a
digital zoom to the captured streams.
Initialize the camera controls at camera registration time and update
them at configure time as the sensor's analogue crop size might change
depending on the desired Camera configuration.
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
The Mali C55 ISP has a resizing pipeline that allows to crop and scale
images.
So far the mali-c55 pipeline has only supported cropping without using
the scaling functionalities.
Now that the kernel has gained support for the scaling operations, make
the libcamera pipeline use it by combining it with a first cropping step
to align the input and output images FOV ratio, and then scale to the
desired output size.
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Barnabás Pőcze
Kieran Bingham
Paul Elder
Quentin Schulz
Stanislaw Gruszka
Laurent Pinchart
Milan Zamazal
Naushir Patuck
Barnabás Pőcze
Jacopo Mondi
Jacopo Mondi
Daniel Scally
Stefan Klug