Commit Graph

88 Commits

Author SHA1 Message Date
Robert Mader
0c2ed9ebf9 egl: Use the Mesa surfaceless platform instead of GBM
Mesa surfaceless platform appears to be a better fit for the use-case at hand:
1. Like GBM it is Mesa specific, so no change in supported setups is
   expected. If ever required, a fallback to the generic device platform
   could be added on top.
2. It leaves the complexity of selecting a renderer device to the
   driver, reducing code and dependencies.
3. It allows to use llvmpipe / software drivers without dri device,
   which can be useful on CI or debugging (with LIBGL_ALWAYS_SOFTWARE=1).

Signed-off-by: Robert Mader <robert.mader@collabora.com>
Reviewed-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Tested-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org> # sm8250/rb5, x1e/Dell Insprion14p
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Milan Zamazal <mzamazal@redhat.com> # TI AM69
Tested-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com> # ThinkPad X1 Yoga Gen 7 + ov2740
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2026-01-21 12:05:00 +00:00
Robert Mader
22b15ff683 egl: Remove bpp variable
It's unused and, in one case, was set wrongly.

Signed-off-by: Robert Mader <robert.mader@collabora.com>
Reviewed-by: Hans de Goede <johannes.goede@oss.qualcomm.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Tested-by: Hans de Goede <johannes.goede@oss.qualcomm.com> # Thinkpad X1 Carbon G13 IPU7 ov08x40
Tested-by: Milan Zamazal <mzamazal@redhat.com> # TI AM69
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2026-01-19 10:25:44 +00:00
Robert Mader
7c1527cd44 egl: Remove unused functions
They are left-overs from older iterations of the GPU-ISP.

Signed-off-by: Robert Mader <robert.mader@collabora.com>
Reviewed-by: Hans de Goede <johannes.goede@oss.qualcomm.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Tested-by: Hans de Goede <johannes.goede@oss.qualcomm.com> # Thinkpad X1 Carbon G13 IPU7 ov08x40
Tested-by: Milan Zamazal <mzamazal@redhat.com> # TI AM69
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2026-01-19 10:25:05 +00:00
Bryan O'Donoghue
aaa48bb276 libcamera: software_isp: Add a gpuisp todo list
List the series of things to do in GPU ISP in perceived order of
difficulty.

Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Robert Mader <robert.mader@collabora.com>
Tested-by: Hans de Goede <johannes.goede@oss.qualcomm.com> # ThinkPad T14s gen 6 (arm64) ov02c10 + X1c gen 12 ov08x40
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com> # Lenovo X13s
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2026-01-07 17:02:58 +00:00
Bryan O'Donoghue
2cf92d00c2 libcamera: software_isp: debayer_egl: Make gpuisp default softisp mode
In some cases the GPU can deliver 15x performance in Debayer with the
CCM on, reference hardware Qualcomm RB5 with IMX512 sensor.

Given this large performance difference it makes sense to make GPUISP
the default for the Software ISP.

If LIBCAMERA_SOFTISP_MODE is omitted gpu will be the default. If
libcamera is compiled without gpuisp support, CPU Debayer will be used.

It is still possible to select CPU mode with LIBCAMERA_SOFISP_MODE=cpu.

Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Robert Mader <robert.mader@collabora.com>
Tested-by: Hans de Goede <johannes.goede@oss.qualcomm.com> # ThinkPad T14s gen 6 (arm64) ov02c10 + X1c gen 12 ov08x40
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com> # Lenovo X13s
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2026-01-07 17:02:58 +00:00
Bryan O'Donoghue
fe9e143702 ipa: simple: Add a flag to indicate gpuIspEnabled
Flag gpuIspEnabled in the simple IPA context. This flag will allow to
selectively avoid some calculations or to generate a default CCM.

Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Robert Mader <robert.mader@collabora.com>
Tested-by: Hans de Goede <johannes.goede@oss.qualcomm.com> # ThinkPad T14s gen 6 (arm64) ov02c10 + X1c gen 12 ov08x40
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com> # Lenovo X13s
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2026-01-07 17:02:57 +00:00
Bryan O'Donoghue
a92cc12fb5 libcamera: software_isp: debayer_egl: Make DebayerEGL an environment option
If GPUISP support is available make it so an environment variable can
switch it on.

Given we don't have full feature parity with CPUISP just yet on pixel
format output, we should default to CPUISP mode giving the user the option
to switch on GPUISP by setting LIBCAMERA_SOFTISP_MODE=gpu

Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Robert Mader <robert.mader@collabora.com>
Tested-by: Hans de Goede <johannes.goede@oss.qualcomm.com> # ThinkPad T14s gen 6 (arm64) ov02c10 + X1c gen 12 ov08x40
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com> # Lenovo X13s
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2026-01-07 17:02:57 +00:00
Bryan O'Donoghue
6a478459f3 libcamera: software_isp: debayer_cpu: Make getInputConfig and getOutputConfig static
Make getInputConfig and getOutputConfig static so as to allow for
interrogation of the supported pixel formats prior to object instantiation.
Do this so as to allow the higher level logic make an informed choice
between CPU and GPU ISP based on which pixel formats are supported.

Currently CPU ISP supports more diverse input and output schemes.

Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Robert Mader <robert.mader@collabora.com>
Tested-by: Hans de Goede <johannes.goede@oss.qualcomm.com> # ThinkPad T14s gen 6 (arm64) ov02c10 + X1c gen 12 ov08x40
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com> # Lenovo X13s
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2026-01-07 17:02:57 +00:00
Bryan O'Donoghue
f520b29fe9 libcamera: software_isp: debayer_egl: Add an eGL Debayer class
Add a class to run the existing glsl debayer shaders on a GBM surface.

Signed-off-by: Robert Mader <robert.mader@collabora.com>
Co-developed-by: Robert Mader <robert.mader@collabora.com>
[bod: took scaling and buffer size fixes from Robert]
[bod: took fix for center byte calculation from Hans]
[bod: took formatting fixes from Milan]
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Co-developed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Robert Mader <robert.mader@collabora.com>
Tested-by: Robert Mader <robert.mader@collabora.com>
Tested-by: Hans de Goede <johannes.goede@oss.qualcomm.com> # ThinkPad T14s gen 6 (arm64) ov02c10 + X1c gen 12 ov08x40
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com> # Lenovo X13s
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2026-01-07 17:02:57 +00:00
Bryan O'Donoghue
db09fcd8b6 libcamera: software_isp: Add a Size_ member variable to pass to eGL later
We will be initialising eGL from Debayer::start(). We need a copy of the
size data passed in configure() to tell the fragment shaders the right
output size.

Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Robert Mader <robert.mader@collabora.com>
Tested-by: Hans de Goede <johannes.goede@oss.qualcomm.com> # ThinkPad T14s gen 6 (arm64) ov02c10 + X1c gen 12 ov08x40
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com> # Lenovo X13s
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2026-01-07 17:02:57 +00:00
Bryan O'Donoghue
7e72e84f3d libcamera: software_isp: Add member variables to track selected input/output pixelFormat
In order to have Debayer::start() tell the eGL shader compilation routine what
the input and output pixel format is, we need to have a copy of the
selected format available. Add variables to the inputConfig and
outputConfig structures to allow tracking of this data for later use.

Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Robert Mader <robert.mader@collabora.com>
Tested-by: Hans de Goede <johannes.goede@oss.qualcomm.com> # ThinkPad T14s gen 6 (arm64) ov02c10 + X1c gen 12 ov08x40
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com> # Lenovo X13s
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2026-01-07 17:02:57 +00:00
Bryan O'Donoghue
a1a6253ff9 libcamera: software_isp: debayer: Latch contrastExp not contrast to debayer parameters
Pass contrastExp as calculated in lut to debayer params not the raw
contrast. This way we calculate contrastExp once per frame in lut and pass
the calculated value into the shaders, instead of passing contrast and
calculating contrastExp once per pixel in the shaders.

Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Robert Mader <robert.mader@collabora.com>
Tested-by: Hans de Goede <johannes.goede@oss.qualcomm.com> # ThinkPad T14s gen 6 (arm64) ov02c10 + X1c gen 12 ov08x40
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com> # Lenovo X13s
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2026-01-07 17:02:57 +00:00
Bryan O'Donoghue
5f51e2e545 libcamera: software_isp: lut: Make contrast available in debayer params
Provide the contrast used in IPA to Bayer parameters. Similar to the
calculated Gamma value we will pass this value into the debayer fragment
shader for further consumption.

Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-12-12 23:32:53 +00:00
Bryan O'Donoghue
5bfb96661f libcamera: software_isp: lut: Make gamma from lut.cpp available in debayer params
Provide the gamma used in IPA to Bayer parameters. We will pass Gamma into
the shader via a uniform and can then tweak that value from outside at
will.

Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-12-12 23:32:53 +00:00
Bryan O'Donoghue
724a29e3cd libcamera: software_isp: blacklevel: Make black level available in debayer params
Populate black level gain in blacklevel::prepare(). A copy is made of the gain
value in the DebayerParams structure.

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-12-12 23:32:53 +00:00
Bryan O'Donoghue
3d7ef342b7 libcamera: software_isp: lut: Make CCM available in debayer params
Provide the CCM calculated in LUT to the debayer params structure for
consumption in the debayer shaders.

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-12-12 23:32:52 +00:00
Bryan O'Donoghue
0640f5ac11 libcamera: software_isp: debayer: Make the debayer_ object of type class Debayer not DebayerCpu
Make the type of object Debayer not DebayerCpu thus allowing us to assign
the object to either DebayerCpu or DebayerEGL.

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-12-12 23:32:52 +00:00
Bryan O'Donoghue
4cb388a1eb libcamera: software_isp: debayer: Introduce a start() / stop() methods to the debayer object
In order to initialise and deinitialise gpuisp we need to be able to setup
EGL in the same thread as Debayer::process() happens in.

This requires extending the Debayer object to provide start and stop
methods which are triggered through invokeMethod in the same way as
process() is.

Introduce start() and stop() methods to the Debayer class. Trigger those
methods as described above via invokeMethod. The debayer_egl class will
take care of initialising and de-initialising as necessary. Debayer CPU
sees no functional change.

Per feedback from Barnabas the stop method is using blocking
synchronisation and thus we drop ispWorkerThread_.removeMessages().

[bod: Made method blocking not queued per Robert's bugfixes]
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-12-12 23:32:52 +00:00
Bryan O'Donoghue
364886fb13 libcamera: software_isp: Move isStandardBayerOrder to base class
isStandardBayerOrder is useful to both CPU and GPU debayer logic and
reusable as-is for both.

Move to shared location in base class.

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-12-12 23:32:52 +00:00
Bryan O'Donoghue
c9f3e09634 libcamera: software_isp: Make output DMA sync contingent
The DMA sync output buffer from the GPU need only have its cache
invalidated if the CPU is going to modify the buffer. Right now this is
not required for gpuisp so only act on the output buffer if it is
non-null.

Suggested-by: Robert Mader <robert.mader@collabora.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-12-12 23:32:52 +00:00
Bryan O'Donoghue
6b224d91a9 libcamera: software_isp: Move DMA Sync code to Debayer base class
We can reuse the DMA Sync code in the GPUISP. Move the code we need to
the base class.

Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-12-12 23:32:51 +00:00
Bryan O'Donoghue
e3c74bbc89 libcamera: software_isp: Move param select code to Debayer base class
Move the parameter selection code into the Debayer base class in-order to
facilitate reuse of the lookup tables in the eGL shaders.

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-12-12 23:32:51 +00:00
Bryan O'Donoghue
cafb39b257 libcamera: software_isp: Move Bayer params init from DebayerCpu to Debayer
Move the initialisation of Bayer params and CCM to a new constructor in the
Debayer class.

Ensure we call the base class constructor from DebayerCpu's constructor in
the expected constructor order Debayer then DebayerCpu.

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-12-12 23:32:51 +00:00
Bryan O'Donoghue
d635cd884e libcamera: software_isp: Move useful items from DebayerCpu to Debayer base class
The DebayerCpu class has a number of variables, embedded structures and
methods which are useful to DebayerGpu implementation.

Move relevant variables and methods to base class.

Since we want to call setParams() from the GPUISP and reuse the code in
the existing CPUISP as a first step, we need to move all of the
dependent variables in DebayerCPU to the Debayer base class including
LookupTable and redCcm_.

The DebayerEGL class will ultimately be able to consume both the CCM and
non-CCM data.

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-12-12 23:32:51 +00:00
Hans de Goede
134f926346 libcamera: swstats_cpu: Add processFrame() method
Add a method to the SwstatsCpu class to process a whole Framebuffer in
one go, rather then line by line. This is useful for gathering stats
when debayering is not necessary or is not done on the CPU.

Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
[bod: various rebase splats fixed]
[bod: Added constructor Doxygen header]
[bod: Squashed a fix from Hans to calculate stats on every 4th frame]
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-12-12 23:32:51 +00:00
Hans de Goede
bf51f39f3b libcamera: software_isp: Move benchmark code to its own class
Move the code for the builtin benchmark to its own small
Benchmark class.

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
[bod: Fixed up some drift in this patch since initial propostion]
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-12-12 23:32:50 +00:00
Hans de Goede
4da17de043 libcamera: swstats_cpu: Move header to libcamera/internal/software_isp
Move the swstats_cpu.h file to include/libcamera/internal/software_isp/
so that it can be used outside the src/libcamera/software_isp/ directory.

Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-12-12 23:07:01 +00:00
Hans de Goede
85cba34493 libcamera: swstats_cpu: Drop patternSize_ documentation
patternSize_ is a private variable and its meaning is already documented
in the patternSize() getter documentation.

Move the list of valid sizes to the patternSize() getter documentation
and drop the patternSize_ documentation.

While at it also add 1x1 as valid size for use with future support
of single plane non Bayer input data.

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-12-12 23:07:01 +00:00
Hans de Goede
7092566187 libcamera: swstats_cpu: Update statsProcessFn() / processLine0() documentation
Update the documentation of the statsProcessFn() / processLine0() src[]
pointer argument to take into account that swstats_cpu may also be used
with planar input data or with non Bayer single plane input data.

The statsProcessFn typedef is private, so no documentation is generated
for it. Move the new updated src[] pointer argument documentation to
processLine0() so that it gets included in the generated docs.

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-12-12 23:07:01 +00:00
Schulz, Andreas
559128b1f1 Thread: Add name parameter
For debugging purposes, threads can be assigned a name, which eases
distinguishing between them in e.g. htop or gdb. This uses a
Linux-specific API for now which is limited to 15 characters (+ null
terminator), so truncation is done and names for existing thread
instantiations were chosen to be consise.

[Kieran: Apply checkstyle suggestions, rebase on proxy rework]
Signed-off-by: Schulz, Andreas <andreas.schulz2@karlstorz.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-11-05 11:31:10 +00:00
Milan Zamazal
06aee9135f libcamera: software_isp: Apply CCM swap also on green
When CPU ISP is asked to apply the CCM matrix

  [0 1 0]
  [0 0 0]
  [0 0 0]

for a format that requires swapping red and blue channels, the resulting
image has a wrong colour.  The CCM matrix above should take green from
pixels and make it red.  Instead, the image is blue.

The problem is that the lookup tables setup in CPU debayering swaps red
and blue in the lookup tables for red and blue, but not for green.  The
colours must be swapped also in the lookup table for green, which this
patch adds.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-10-20 09:57:35 +01:00
Barnabás Pőcze
8c30369119 libcamera: software_isp: Clear pending async work
Debayering is carried out on `ispWorkerThread_`. When stopping, the queued
work needs to be flushed or cancelled to ensure that the next time it starts,
it won't process stale data. So remove all messages targeting the `Debayer`
object on the worker thread.

Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Tested-by: Robert Mader <robert.mader@collabora.com>
2025-10-17 10:05:19 +02:00
Milan Zamazal
6a48f382e0 libcamera: software_isp: Pass correct y-coordinate to stats
The window set by SwStatsCpu::setWindow is relative to the processed
image area.  But debayering passes the processed line y-coordinate to
the stats relative to the whole image area.  This can result in
gathering stats from a wrong image area or in not gathering stats at
all.

Let's pass the correct y-coordinate to the stats processing methods.

Bug: https://bugs.libcamera.org/show_bug.cgi?id=280
Signed-off-by: Maciej S. Szmigiero <mail@maciej.szmigiero.name>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Hans de Goede <hansg@kernel.org>
Tested-by: Hans de Goede <hansg@kernel.org>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-10-02 22:07:20 +01:00
Milan Zamazal
4ec1d75fdd libcamera: software_isp: Clarify SwStatsCpu::setWindow use
The window coordinates passed to SwStatsCpu::setWindow are confusing.
Let's clarify what the coordinates should be.

A source of confusion is that the specified window is relative to the
processed area.  Debayering adjusts line pointers for its processed area
and this is what's also passed to stats processing.  The window passed
to SwStatsCpu::setWindow should either specify the size of the whole
processed (not image) area, or its cropping in case the stats shouldn't
be gathered over the whole processed area.  This patch should clarify
this in the code.

Reviewed-by: Maciej S. Szmigiero <mail@maciej.szmigiero.name>
Reviewed-by: Hans de Goede <hansg@kernel.org>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Tested-by: Hans de Goede <hansg@kernel.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-10-02 22:07:15 +01:00
Milan Zamazal
e79cec02fb libcamera: software_isp: Fix width adjustment in SwStatsCpu::setWindow
SwStatsCpu::setWindow reduces the window width by the added x-offset, to
prevent exceeding image bounds.  But if the window width is smaller than
the x-offset, we get unsigned integer underflow.  Fix it by setting the
window width to 0 in such a case.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Hans de Goede <hansg@kernel.org>
Tested-by: Hans de Goede <hansg@kernel.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-10-02 22:07:08 +01:00
Hans de Goede
c28bb6a6a4 libcamera: software_isp: Run sw-statistics once every 4th frame
Run sw-statistics once every 4th frame, instead of every frame. There are
2 reasons for this:

1. There really is no need to have statistics for every frame and only
doing this every 4th frame helps save some CPU time.

2. The generic nature of the simple pipeline-handler, so no information
about possible CSI receiver frame-delays. In combination with the software
ISP often being used with sensors without sensor info in the sensor-helper
code, so no reliable control-delay information means that the software ISP
is prone to AGC oscillation. Skipping statistics gathering also means
skipping running the AGC algorithm slowing it down, avoiding this
oscillation.

Note ideally the AGC oscillation problem would be fixed by adding sensor
metadata support all through the stack so that the exact gain and exposure
used for a specific frame are reliably provided by the sensor metadata.

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Tested-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Hans de Goede <hansg@kernel.org>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-10-01 16:45:17 +01:00
Hans de Goede
9b441cf198 libcamera: software_isp: Add valid flag to struct SwIspStats
Generating statistics for every single frame is not really necessary.

However a roundtrip through ipa_->processStats() still need to be done
every frame, even if there are no stats to make the IPA generate metadata
for every frame.

Add a valid flag to the statistics struct to let the IPA know when there
are no statistics for the frame being processed and modify the IPA to
only generate metadata for frames without valid statistics.

This is a preparation patch for skipping statistics generation for some
frames.

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Hans de Goede <hansg@kernel.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-10-01 16:45:17 +01:00
Milan Zamazal
79a75b9088 libcamera: software_isp: Make measurement configurable
Software ISP performs performance measurement on certain part of initial
frames.  Let's make this range configurable.

For this purpose, this patch introduces new configuration options
software_isp.measure.skip and software_isp.measure.number.  Setting the
latter one to 0 disables the measurement.

Instead of the last frame, the class member and its configuration
specify the number of frames to measure.  This is easier to use for
users and doesn't require to adjust two configuration parameters when
the number of the initially skipped frames is changed.

The patch also changes the names of the class members to make them more
accurate.

Completes software ISP TODO #7.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2025-09-21 09:42:14 +03:00
Milan Zamazal
6e1d889cfe libcamera: software_isp: Make input buffer copying configurable
On some platforms, working directly on the input buffer is very slow due
to disabled caching.  This is why we copy the input buffer into standard
(cached) memory.  This is an unnecessary overhead on platforms with
cached buffers.

Let's make input buffer copying configurable.  The default is still
copying, as its overhead is much lower than contingent operations on
non-cached memory.  Ideally, we should improve this in future to set the
default to non-copying if we can be sure under observable circumstances
that we are working with cached buffers.

Completes software ISP TODO #6.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2025-09-21 09:42:13 +03:00
Milan Zamazal
cc137b1c6d libcamera: software_isp: Remove type casts in statistics computation
Type casting from unsigned int to int performed in stats computation is
unnecessary, window_.width is unsigned and the array index is always
non-negative.  Let's simply use unsigned int in all the
SwStatsCpu:stats* methods.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Umang Jain <uajain@igalia.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-07-21 12:27:09 +01:00
Milan Zamazal
485a807dcb ipa: simple: Report exposure in metadata
Report exposure and gain in metadata.

This is more complicated than it could be expected because the exposure
value should be in microseconds but it's handled using V4L2_CID_EXPOSURE
control, which doesn't specify the unit, see
https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/control.html.
So the unit conversion is done in the way rkisp1 IPA uses.

This requires getting and passing IPACameraSensorInfo around.  To avoid
naming confusion and to improve consistency with rkisp1 IPA,
sensorCtrlInfoMap parameter is renamed to sensorControls.

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2025-03-28 02:09:16 +02:00
Kieran Bingham
fb99081586 ipa: simple: softisp: Extend to pass metadata
Extend the Simple IPA IPC to support returning a metadata ControlList
when the process call has completed.

A new signal from the IPA is introduced to report the metadata,
similarly to what the hardware pipelines do.

Merge the metadata reported by the ISP into any completing request to
provide to the application.  Completion of a request is delayed until
this is done; this doesn't apply to canceled requests.

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2025-03-28 02:09:13 +02:00
Milan Zamazal
e2b4000dc9 libcamera: software_isp: Apply CCM in debayering
This patch applies color correction matrix (CCM) in debayering if the
CCM is specified.  Not using CCM must still be supported for performance
reasons.

The CCM is applied as follows:

  [r1 g1 b1]   [r]
  [r2 g2 b2] * [g]
  [r3 g3 b3]   [b]

The CCM matrix (the left side of the multiplication) is constant during
single frame processing, while the input pixel (the right side) changes.
Because each of the color channels is only 8-bit in software ISP, we can
make 9 lookup tables with 256 input values for multiplications of each
of the r_i, g_i, b_i values.  This way we don't have to multiply each
pixel, we can use table lookups and additions instead.  Gamma (which is
non-linear and thus cannot be a part of the 9 lookup tables values) is
applied on the final values rounded to integers using another lookup
table.

Because the changing part is the pixel value with three color elements,
only three dynamic table lookups are needed.  We use three lookup tables
to represent the multiplied matrix values, each of the tables
corresponding to the given matrix column and pixel color.

We use int16_t to store the precomputed multiplications.  This seems to
be noticeably (>10%) faster than `float' for the price of slightly less
accuracy and it covers the range of values that sane CCMs produce.  The
selection and structure of data is performance critical, for example
using bytes would add significant (>10%) speedup but would be too short
to cover the value range.

The color lookup tables can be represented either as unions,
accommodating tables for both the CCM and non-CCM cases, or as separate
tables for each of the cases, leaving the tables for the other case
unused.  The latter is selected as a matter of preference.

The tables are copied (as before), which is not elegant but also not a
big problem.  There are patches posted that use shared buffers for
parameters passing in software ISP (see software ISP TODO #5) and they
can be adjusted for the new parameter format.

Color gains from white balance are supposed not to be a part of the
specified CCM.  They are applied on it using matrix multiplication,
which is simple and in correspondence with future additions in the form
of matrix multiplication, like saturation adjustment.

With this patch, the reported per-frame slowdown when applying CCM is
about 45% on Debix Model A and about 75% on TI AM69 SK.

Using std::clamp in debayering adds some performance penalty (a few
percent).  The clamping is necessary to eliminate out of range values
possibly produced by the CCM.  If it could be avoided by adjusting the
precomputed tables some way then performance could be improved a bit.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-03-26 10:45:01 +00:00
Milan Zamazal
ac30686556 libcamera: software_isp: Track whether CCM is enabled
Applying color correction matrix (CCM) in software ISP is optional due
to performance reasons.  CCM is applied if and only if `Ccm' algorithm
is present in the tuning file.

Software ISP debayering is a performance critical piece of code and we
do not want to use dynamic conditionals there.  Therefore we pass
information about CCM application to debayering configuration and let it
select the right versions of debayering functions using templates.  This
is a trick similar to the previously used one for adding or not adding
an alpha channel to the output.

Debayering gets this information but it ignores it in this patch.
Actual processing with CCM is added in the followup patch.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-03-26 10:45:01 +00:00
Milan Zamazal
23dfd69081 libcamera: software_isp: Use a macro to assign debayering methods
Assignments of the debayering methods to be used is a repetitive pattern
that can be (arguably) better expressed by using a macro.  This removes
some duplication and also makes easier to introduce more complex
assignment patterns.  This will be useful once color correction matrix
support is added.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-03-26 10:45:01 +00:00
Milan Zamazal
f1955a0058 libcamera: software_isp: Use common code to store debayered pixels
The debayering macros use the same pattern, let's extract it to a common
macro.  This reduces code duplication a bit now and it'll make changes
of debayering easier when color correction matrix is introduced.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-03-26 10:45:01 +00:00
Milan Zamazal
86ffaf936d libcamera: software_isp: Dispatch messages on stop
There may be pending messages in SoftwareIsp message queue when
SoftwareIsp stops.  The call to IPAProxySoft::stop() will dispatch them
before SoftwareIsp::stop() finishes.  But this is dependent on
IPAProxySoft::stop() implementation, let's break this dependency and
dispatch messages to SoftwareIsp explicitly in SoftwareIsp::stop().

This also allows dropping `running_' flag.  Since the SoftwareIsp
messages get processed and invoke IPA calls before the IPA proxy is set
to ProxyStopping state and the SoftwareIsp worker thread is no longer
running, it's guaranteed that no new messages come to SoftwareIsp and
attempt to call the stopped IPA proxy.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-03-01 23:10:05 +00:00
Milan Zamazal
cd32e069ec libcamera: software_isp: Handle queued input buffers on stop
When SoftwareIsp stops, input and output buffers queued to it may not
yet be fully processed.  They will be eventually returned but stop means
stop, there should be no processing related actions invoked afterwards.

Let's stop forwarding processed input buffers from SoftwareIsp slots
when SoftwareIsp is stopped.  Let's track the queued input buffers and
return them back for capture in SoftwareIsp::stop().

The returned input buffers are marked as cancelled.  This is not
necessary at the moment but it gives the pipeline handlers chance to
deal with this if they need to.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-03-01 23:09:56 +00:00
Milan Zamazal
ba4715ffed libcamera: software_isp: Handle queued output buffers on stop
When SoftwareIsp stops, input and output buffers queued to it may not
yet be fully processed.  They will be eventually returned but stop means
stop, there should be no processing related actions invoked afterwards.

Let's stop forwarding processed output buffers from the SoftwareIsp
slots once SoftwareIsp is stopped.  Let's track the queued output
buffers and mark those still pending as cancelled in SoftwareIsp::stop
and return them to the pipeline handler.

Dealing with input buffers is addressed in a separate patch.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-03-01 23:07:15 +00:00
Milan Zamazal
b72d789475 libcamera: software_isp: Emit ispStatsReady only if IPA is running
Software ISP runs debayering in a separate thread and debayering may
emit statsReady when software ISP (including the IPA) is being stopped.
The signal waits in a queue and gets invoked later, resulting in an
assertion error when attempting to invoke a method on the stopped IPA:

  FATAL default soft_ipa_proxy.cpp:456 assertion
  "state_ == ProxyRunning" failed in processStatsThread()

Let's prevent this problem by forwarding the ISP stats signal from
software ISP only when the IPA is running.  To track this,
SoftwareISP::running_ variable is introduced.

Making processing of the other signals in SoftwareISP more robust is
addressed in the followup patches.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reported-by: Stanislaw Gruszka <stanislaw.gruszka@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-03-01 23:06:12 +00:00