Commit Graph

40 Commits

Author SHA1 Message Date
Hans de Goede
5f770f748c libcamera: Make converter/softISP configure()'s outputCfgs argument really const
The intent of the outputCfgs argument to the configure() function of
converter classes and the softISP is to allow the passed in stream-configs
to not be changed.

But only the vector is const, the reference inside the vector are not
const, which allows modifying the stream-configs as can be seen inside
DebayerEGL::configure() which was using a non const reference outputCfg
helper variable.

Fix this by making the references inside the vector const.

Signed-off-by: Hans de Goede <johannes.goede@oss.qualcomm.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2026-04-01 14:59:38 +01:00
Hans de Goede
6e53e72e96 software_isp: swstats_cpu: Prepare for multi-threading support
Make the storage used to accumulate the RGB sums and the Y histogram
value a vector of SwIspStats objects instead of a single object so
that when using multi-threading every thread can use its own storage to
collect intermediate stats to avoid cache-line bouncing.

Benchmarking with the GPU-ISP which does separate swstats benchmarking,
on the Arduino Uno-Q which has a weak CPU which is good for performance
testing, shows 20ms to generate stats for a 3272x2464 frame both before
and after this change.

Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Hans de Goede <johannes.goede@oss.qualcomm.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2026-04-01 12:04:42 +01:00
Milan Zamazal
0d3e543b4d libcamera: software_isp: Set initial values of DebayerParams
Debayer parameters and processing are currently run asynchronously.
This can lead to assertion errors in case the processing tries to use
not yet computed debayer parameters.  To prevent this situation, specify
some default values for DebayerParams members.  This doesn't make
correct parameters but prevents crashes or other crazy behaviours at
least.

Note this patch is just a workaround.  The mutually asynchronous
parameters computation and processing can cause more problems, like
using parameters computed for a different frame.  But it is non-trivial
to fix that; in the meantime, setting the default values solves the
worst problem.

Closes: https://gitlab.freedesktop.org/camera/libcamera/-/issues/311
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Hans de Goede <johannes.goede@oss.qualcomm.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2026-02-26 16:37:41 +00:00
Hans de Goede
7bcc2168bf software_isp: benchmark: Print what is being benchmarked
With the GPU accelerated softISP 2 separate benchmark results are printed,
1 for the generation of the output images on the GPU and a separate one
for generating the statistics on the CPU.

Add a new name argument to the Benchmark class descriptor and print this
out when printing the benchmark result.

Signed-off-by: Hans de Goede <johannes.goede@oss.qualcomm.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2026-02-18 10:48:52 +00:00
Hans de Goede
f0a79dd68b software_isp: benchmark: Add missing _ postfix to measure data member
All class data members should have a _ postifx, add the missing _ postfix
to the Benchmark::measure_ data member.

Signed-off-by: Hans de Goede <johannes.goede@oss.qualcomm.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2026-02-18 10:48:52 +00:00
Milan Zamazal
3ffaa4c0e2 libcamera: ipa: simple: Remove Lut algorithm
The Lut algorithm is not really an algorithm.  Moreover, algorithms may
be enabled or disabled but with Lut disabled, nothing will work.

Let's move the construction of lookup tables to CPU debayering, where it
is used.  The implied and related changes are:

- DebayerParams is changed to contain the real params rather than lookup
  tables.
- contrastExp parameter introduced by GPU ISP is used for CPU ISP too.
- The params must be initialised so that debayering gets meaningful
  parameter values even when some algorithms are disabled.
- combinedMatrix must be put to params everywhere where it is modified.
- Matrix changes needn't be tracked in the algorithms any more.
- CPU debayering must watch for changes of the corresponding parameters
  to update the lookup tables when and only when needed.
- Swapping red and blue is integrated into lookup table constructions.
- gpuIspEnabled flags are removed as they are not needed any more.

Reviewed-by: Robert Mader <robert.mader@collabora.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2026-01-28 15:20:34 +00:00
Milan Zamazal
14d340a27b libcamera: ipa: simple: Use float type for adjustment controls
control_ids.h defines the contrast type as float, let's use the same in
simple IPA, instead of double.  Saturation and gamma already use float,
except for the knobs initializers, let's use float for the knobs too.

Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Robert Mader <robert.mader@collabora.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2026-01-28 15:20:33 +00:00
Milan Zamazal
07240afa12 libcamera: simple: Fix black level offsets in AWB
The black level offset subtracted in AWB is wrong.  It assumes that the
stats contain sums of the individual colour pixels.  But they actually
contain sums of the colour channels of larger "superpixels" consisting
of the individual colour pixels.  Each of the RGB colour values and the
computed luminosity (a histogram entry) are added once to the stats per
such a superpixel.  This means the offset computed from the black level
and the number of pixels should be used as it is, not divided.

The patch fixes the subtracted offset.  Since the evaluation is the same
for all the three colours now, the individual class variables are
replaced with a single RGB variable.

Fixes: 4e13c6f55b ("Honor black level in AWB")
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Robert Mader <robert.mader@collabora.com>
Tested-by: Robert Mader <robert.mader@collabora.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2026-01-28 12:08:21 +00:00
Bryan O'Donoghue
a1a6253ff9 libcamera: software_isp: debayer: Latch contrastExp not contrast to debayer parameters
Pass contrastExp as calculated in lut to debayer params not the raw
contrast. This way we calculate contrastExp once per frame in lut and pass
the calculated value into the shaders, instead of passing contrast and
calculating contrastExp once per pixel in the shaders.

Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Robert Mader <robert.mader@collabora.com>
Tested-by: Hans de Goede <johannes.goede@oss.qualcomm.com> # ThinkPad T14s gen 6 (arm64) ov02c10 + X1c gen 12 ov08x40
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com> # Lenovo X13s
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2026-01-07 17:02:57 +00:00
Bryan O'Donoghue
5f51e2e545 libcamera: software_isp: lut: Make contrast available in debayer params
Provide the contrast used in IPA to Bayer parameters. Similar to the
calculated Gamma value we will pass this value into the debayer fragment
shader for further consumption.

Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-12-12 23:32:53 +00:00
Bryan O'Donoghue
5bfb96661f libcamera: software_isp: lut: Make gamma from lut.cpp available in debayer params
Provide the gamma used in IPA to Bayer parameters. We will pass Gamma into
the shader via a uniform and can then tweak that value from outside at
will.

Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-12-12 23:32:53 +00:00
Bryan O'Donoghue
724a29e3cd libcamera: software_isp: blacklevel: Make black level available in debayer params
Populate black level gain in blacklevel::prepare(). A copy is made of the gain
value in the DebayerParams structure.

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-12-12 23:32:53 +00:00
Bryan O'Donoghue
3d7ef342b7 libcamera: software_isp: lut: Make CCM available in debayer params
Provide the CCM calculated in LUT to the debayer params structure for
consumption in the debayer shaders.

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-12-12 23:32:52 +00:00
Bryan O'Donoghue
0640f5ac11 libcamera: software_isp: debayer: Make the debayer_ object of type class Debayer not DebayerCpu
Make the type of object Debayer not DebayerCpu thus allowing us to assign
the object to either DebayerCpu or DebayerEGL.

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-12-12 23:32:52 +00:00
Hans de Goede
134f926346 libcamera: swstats_cpu: Add processFrame() method
Add a method to the SwstatsCpu class to process a whole Framebuffer in
one go, rather then line by line. This is useful for gathering stats
when debayering is not necessary or is not done on the CPU.

Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
[bod: various rebase splats fixed]
[bod: Added constructor Doxygen header]
[bod: Squashed a fix from Hans to calculate stats on every 4th frame]
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-12-12 23:32:51 +00:00
Hans de Goede
bf51f39f3b libcamera: software_isp: Move benchmark code to its own class
Move the code for the builtin benchmark to its own small
Benchmark class.

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
[bod: Fixed up some drift in this patch since initial propostion]
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-12-12 23:32:50 +00:00
Hans de Goede
4da17de043 libcamera: swstats_cpu: Move header to libcamera/internal/software_isp
Move the swstats_cpu.h file to include/libcamera/internal/software_isp/
so that it can be used outside the src/libcamera/software_isp/ directory.

Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-12-12 23:07:01 +00:00
Hans de Goede
9b441cf198 libcamera: software_isp: Add valid flag to struct SwIspStats
Generating statistics for every single frame is not really necessary.

However a roundtrip through ipa_->processStats() still need to be done
every frame, even if there are no stats to make the IPA generate metadata
for every frame.

Add a valid flag to the statistics struct to let the IPA know when there
are no statistics for the frame being processed and modify the IPA to
only generate metadata for frames without valid statistics.

This is a preparation patch for skipping statistics generation for some
frames.

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Hans de Goede <hansg@kernel.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-10-01 16:45:17 +01:00
Kieran Bingham
fb99081586 ipa: simple: softisp: Extend to pass metadata
Extend the Simple IPA IPC to support returning a metadata ControlList
when the process call has completed.

A new signal from the IPA is introduced to report the metadata,
similarly to what the hardware pipelines do.

Merge the metadata reported by the ISP into any completing request to
provide to the application.  Completion of a request is delayed until
this is done; this doesn't apply to canceled requests.

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2025-03-28 02:09:13 +02:00
Milan Zamazal
e2b4000dc9 libcamera: software_isp: Apply CCM in debayering
This patch applies color correction matrix (CCM) in debayering if the
CCM is specified.  Not using CCM must still be supported for performance
reasons.

The CCM is applied as follows:

  [r1 g1 b1]   [r]
  [r2 g2 b2] * [g]
  [r3 g3 b3]   [b]

The CCM matrix (the left side of the multiplication) is constant during
single frame processing, while the input pixel (the right side) changes.
Because each of the color channels is only 8-bit in software ISP, we can
make 9 lookup tables with 256 input values for multiplications of each
of the r_i, g_i, b_i values.  This way we don't have to multiply each
pixel, we can use table lookups and additions instead.  Gamma (which is
non-linear and thus cannot be a part of the 9 lookup tables values) is
applied on the final values rounded to integers using another lookup
table.

Because the changing part is the pixel value with three color elements,
only three dynamic table lookups are needed.  We use three lookup tables
to represent the multiplied matrix values, each of the tables
corresponding to the given matrix column and pixel color.

We use int16_t to store the precomputed multiplications.  This seems to
be noticeably (>10%) faster than `float' for the price of slightly less
accuracy and it covers the range of values that sane CCMs produce.  The
selection and structure of data is performance critical, for example
using bytes would add significant (>10%) speedup but would be too short
to cover the value range.

The color lookup tables can be represented either as unions,
accommodating tables for both the CCM and non-CCM cases, or as separate
tables for each of the cases, leaving the tables for the other case
unused.  The latter is selected as a matter of preference.

The tables are copied (as before), which is not elegant but also not a
big problem.  There are patches posted that use shared buffers for
parameters passing in software ISP (see software ISP TODO #5) and they
can be adjusted for the new parameter format.

Color gains from white balance are supposed not to be a part of the
specified CCM.  They are applied on it using matrix multiplication,
which is simple and in correspondence with future additions in the form
of matrix multiplication, like saturation adjustment.

With this patch, the reported per-frame slowdown when applying CCM is
about 45% on Debix Model A and about 75% on TI AM69 SK.

Using std::clamp in debayering adds some performance penalty (a few
percent).  The clamping is necessary to eliminate out of range values
possibly produced by the CCM.  If it could be avoided by adjusting the
precomputed tables some way then performance could be improved a bit.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-03-26 10:45:01 +00:00
Milan Zamazal
ac30686556 libcamera: software_isp: Track whether CCM is enabled
Applying color correction matrix (CCM) in software ISP is optional due
to performance reasons.  CCM is applied if and only if `Ccm' algorithm
is present in the tuning file.

Software ISP debayering is a performance critical piece of code and we
do not want to use dynamic conditionals there.  Therefore we pass
information about CCM application to debayering configuration and let it
select the right versions of debayering functions using templates.  This
is a trick similar to the previously used one for adding or not adding
an alpha channel to the output.

Debayering gets this information but it ignores it in this patch.
Actual processing with CCM is added in the followup patch.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-03-26 10:45:01 +00:00
Milan Zamazal
86ffaf936d libcamera: software_isp: Dispatch messages on stop
There may be pending messages in SoftwareIsp message queue when
SoftwareIsp stops.  The call to IPAProxySoft::stop() will dispatch them
before SoftwareIsp::stop() finishes.  But this is dependent on
IPAProxySoft::stop() implementation, let's break this dependency and
dispatch messages to SoftwareIsp explicitly in SoftwareIsp::stop().

This also allows dropping `running_' flag.  Since the SoftwareIsp
messages get processed and invoke IPA calls before the IPA proxy is set
to ProxyStopping state and the SoftwareIsp worker thread is no longer
running, it's guaranteed that no new messages come to SoftwareIsp and
attempt to call the stopped IPA proxy.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-03-01 23:10:05 +00:00
Milan Zamazal
cd32e069ec libcamera: software_isp: Handle queued input buffers on stop
When SoftwareIsp stops, input and output buffers queued to it may not
yet be fully processed.  They will be eventually returned but stop means
stop, there should be no processing related actions invoked afterwards.

Let's stop forwarding processed input buffers from SoftwareIsp slots
when SoftwareIsp is stopped.  Let's track the queued input buffers and
return them back for capture in SoftwareIsp::stop().

The returned input buffers are marked as cancelled.  This is not
necessary at the moment but it gives the pipeline handlers chance to
deal with this if they need to.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-03-01 23:09:56 +00:00
Milan Zamazal
ba4715ffed libcamera: software_isp: Handle queued output buffers on stop
When SoftwareIsp stops, input and output buffers queued to it may not
yet be fully processed.  They will be eventually returned but stop means
stop, there should be no processing related actions invoked afterwards.

Let's stop forwarding processed output buffers from the SoftwareIsp
slots once SoftwareIsp is stopped.  Let's track the queued output
buffers and mark those still pending as cancelled in SoftwareIsp::stop
and return them to the pipeline handler.

Dealing with input buffers is addressed in a separate patch.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-03-01 23:07:15 +00:00
Milan Zamazal
b72d789475 libcamera: software_isp: Emit ispStatsReady only if IPA is running
Software ISP runs debayering in a separate thread and debayering may
emit statsReady when software ISP (including the IPA) is being stopped.
The signal waits in a queue and gets invoked later, resulting in an
assertion error when attempting to invoke a method on the stopped IPA:

  FATAL default soft_ipa_proxy.cpp:456 assertion
  "state_ == ProxyRunning" failed in processStatsThread()

Let's prevent this problem by forwarding the ISP stats signal from
software ISP only when the IPA is running.  To track this,
SoftwareISP::running_ variable is introduced.

Making processing of the other signals in SoftwareISP more robust is
addressed in the followup patches.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reported-by: Stanislaw Gruszka <stanislaw.gruszka@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-03-01 23:06:12 +00:00
Milan Zamazal
9bc8b6a573 libcamera: software_isp: Handle signals in the proper thread
inputBufferReady ready signal in the simple pipeline is handled in the
pipeline handler thread.  outputBufferReady and ispStatsReady signals
should be handled there too.

Rather than relying on the user of the SoftwareIsp instance, let
SoftwareIsp inherits Object.  SoftwareIsp serves as a signal proxy, the
signals above are emitted from signal handlers.  This means that if
SoftwareIsp inherits Object then the slots are invoked in SoftwareIsp
thread.  Which is the camera manager thread because the SoftwareIsp
instance is created there.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2025-02-01 10:09:46 +00:00
Milan Zamazal
3930b94021 libcamera: software_isp: Add contrast control
This patch introduces support for applying runtime controls to software
ISP.  It enables the contrast control as the first control that can be
used.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2024-12-06 12:34:48 +00:00
Milan Zamazal
04d171e6b2 libcamera: software_isp: Call Algorithm::queueRequest
This patch adds Algorithm::queueRequest call for the defined algorithms.
As there are currently no control knobs in software ISP nor the
corresponding queueRequest call chain, the patch also introduces the
queueRequest methods and calls from the pipeline to the IPA.

This is preparation only since there are currently no Algorithm based
algorithms defined and no current software ISP algorithms support
control knobs.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2024-09-27 15:01:57 +01:00
Milan Zamazal
ecbc05c4c5 libcamera: software_isp: Call Algorithm::configure
This patch adds Algorithm::configure call for the defined algorithms.
This is preparation only since there are currently no Algorithm based
algorithms defined.

A part of this change is passing IPAConfigInfo instead of ControlInfoMap
to configure() calls as this is what Algorithm::configure expects.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2024-09-27 15:01:57 +01:00
Milan Zamazal
f06c344bd5 libcamera: software_isp: Track and pass frame ids
A previous preparation patch implemented passing frame ids to stats
processing but without actual meaningful frame id value passed there.
This patch extends that by actually providing the frame id and passing
it through to the stats processor.

The frame id is taken from the request sequence number, the same as in
hardware pipelines.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2024-09-27 15:01:57 +01:00
Milan Zamazal
c64f2f197a libcamera: software_isp: Make stats frame and buffer aware
This patch adds frame and bufferId arguments to stats related calls.
Although the parameters are currently unused, because frame ids are not
tracked and used and the stats buffer is passed around directly rather
than being referred by its id, they bring the internal APIs closer to
their counterparts in hardware pipelines.

It serves as a preparation for followup patches that will introduce:

- Frame number tracking in order to switch to DelayedControls
  (software ISP TODO #11 + #12).
- A ring buffer for stats in order to improve passing the stats
  (software ISP TODO #2).

Frame and buffer ids are unrelated for the given purposes but since they
are passed together at the same places, the change is implemented as a
single patch rather than two, basically the same, patches.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2024-09-27 15:01:57 +01:00
Umang Jain
cc3a3c46a5 libcamera: converter: Replace usage of stream index by Stream pointer
The converter interface uses the unsigned int output stream index to map
to the output frame buffers. This is cumbersome to implement new
converters because one has to keep around additional book keeping
to track the streams with their correct indexes.

The v4l2_converter_m2m and simple pipeline handler are adapted to
use the new interface. This work roped in software ISP as well,
which also seems to use indexes (although it doesn't implement converter
interface) because of a common conversionQueue_ queue used for
converter_ and swIsp_.

The logPrefix is no longer able to generate an index from a stream, and
is updated to be more expressive by reporting the stream configuration
instead, for example, reporting "1920x1080-MJPEG" in place of
"stream0".

Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Tested-by: Andrei Konovalov <andrey.konovalov.ynk@gmail.com> # sm8250 RB5
2024-07-03 10:36:55 +05:30
Hans de Goede
447da4a11f libcamera: Rename DmaHeap class to DmaBufAllocator
Users of the DmaHeap class really just want some way to allocate
dma-buffers from userspace. This can also be done by using /dev/udmabuf
instead of using /dev/dma_heap/*.

Rename DmaHeap class to DmaBufAllocator in preparation of adding
/dev/udmabuf support.

And update the DmaHeap class docs to match including replacing references
to "dma-heap type" with "dma-buf provider".

This is a pure automated rename on the code ('s/DmaHeap/DmaBufAllocator/')
+ file renames + doc updates. There are no functional changes.

The DmaBufAllocator objects in vc4.cpp and software_isp.cpp are left named
dmaHeap_ to keep the changes to those 2 files to a minimum.

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Tested-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org> # Lenovo-x13s
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2024-06-04 16:19:33 +01:00
Milan Zamazal
f5b7921e0a libcamera: software_isp: Remove DebayerParams::kGain10
The constant is used in a single place internally and doesn't belong to
DebayerParams anymore.  Let's use 256 directly.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Andrei Konovalov <andrey.konovalov.ynk@gmail.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2024-06-02 01:59:24 +03:00
Milan Zamazal
539c62ff8e libcamera: software_isp: Move color mappings out of debayering
Constructing the color mapping tables is related to stats rather than
debayering, where they are applied.  Let's move the corresponding code
to stats processing.

The same applies to the auxiliary gamma table.  As the gamma value is
currently fixed and used in a single place, with the temporary exception
mentioned below, there is no need to share it anywhere anymore.

It's necessary to initialize SoftwareIsp::debayerParams_ to default
values.  These initial values are used for the first two frames, before
they are changed based on determined stats.  To avoid sharing the gamma
value constant in artificial ways, we use 0.5 directly in the
initialization.  This all is not a particularly elegant thing to do,
such a code belongs conceptually to the similar code in stats
processing, but doing better is left for larger refactoring.

This is a preliminary step towards building this functionality on top of
libipa/algorithm.h, which should follow.

Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Andrei Konovalov <andrey.konovalov.ynk@gmail.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2024-06-02 01:54:20 +03:00
Laurent Pinchart
626172a16b libcamera: Drop file name from header comment blocks
Source files in libcamera start by a comment block header, which
includes the file name and a one-line description of the file contents.
While the latter is useful to get a quick overview of the file contents
at a glance, the former is mostly a source of inconvenience. The name in
the comments can easily get out of sync with the file name when files
are renamed, and copy & paste during development have often lead to
incorrect names being used to start with.

Readers of the source code are expected to know which file they're
looking it. Drop the file name from the header comment block.

The change was generated with the following script:

----------------------------------------

dirs="include/libcamera src test utils"

declare -rA patterns=(
	['c']=' \* '
	['cpp']=' \* '
	['h']=' \* '
	['py']='# '
	['sh']='# '
)

for ext in ${!patterns[@]} ; do
	files=$(for dir in $dirs ; do find $dir -name "*.${ext}" ; done)
	pattern=${patterns[${ext}]}

	for file in $files ; do
		name=$(basename ${file})
		sed -i "s/^\(${pattern}\)${name} - /\1/" "$file"
	done
done
----------------------------------------

This misses several files that are out of sync with the comment block
header. Those will be addressed separately and manually.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
2024-05-08 22:39:50 +03:00
Milan Zamazal
b2ef255295 libcamera: software_isp: Apply black level compensation
Black may not be represented as 0 pixel value for given hardware, it may
be higher.  If this is not compensated then various problems may occur
such as low contrast or suboptimal exposure.

The black pixel value can be either retrieved from a tuning file for the
given hardware, or automatically on the fly.  The former is the right
and correct method, while the latter can be used when a tuning file is
not available for the given hardware.  Since there is currently no
support for tuning files in software ISP, the automatic, hardware
independent way, is always used.  Support for tuning files should be
added in future but it will require more work than this patch.

The patch looks at the image histogram and assumes that black starts
when pixel values start occurring on the left.  A certain amount of the
darkest pixels is ignored; it doesn't matter whether they represent
various kinds of noise or are real, they are better to omit in any case
to make the image looking better.  It also doesn't matter whether the
darkest pixels occur around the supposed black level or are spread
between 0 and the black level, the difference is not important.

An arbitrary threshold of 2% darkest pixels is applied; there is no
magic about that value.

The patch assumes that the black values for different colors are the
same and doesn't attempt any other non-primitive enhancements.  It
cannot completely replace tuning files and simplicity, while providing
visible benefit, is its goal.  Anything more sophisticated is left for
future patches.

A possible cheap enhancement, if needed, could be setting exposure +
gain to minimum values temporarily, before setting the black level.  In
theory, the black level should be fixed but it may not be reached in all
images.  For this reason, the patch updates black level only if the
observed value is lower than the current one; it should be never
increased.

The purpose of the patch is to compensate for hardware properties.
General image contrast enhancements are out of scope of this patch.

Stats are still gathered as an uncorrected histogram, to avoid any
confusion and to represent the raw image data.  Exposure must be
determined after the black level correction -- it has no influence on
the sub-black area and must be correct after applying the black level
correction.  The granularity of the histogram is increased from 16 to 64
to provide a better precision (there is no theory behind either of those
numbers).

Reviewed-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2024-04-16 13:00:21 +01:00
Andrey Konovalov
6362bd46ca libcamera: Introduce SoftwareIsp
Doxygen documentation by Dennis Bonke.

Tested-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org> # sc8280xp Lenovo x13s
Tested-by: Pavel Machek <pavel@ucw.cz>
Reviewed-by: Pavel Machek <pavel@ucw.cz>
Co-developed-by: Dennis Bonke <admin@dennisbonke.com>
Signed-off-by: Dennis Bonke <admin@dennisbonke.com>
Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2024-04-16 13:00:21 +01:00
Hans de Goede
783f92c018 libcamera: software_isp: Add Debayer base class
Add a base class for debayer implementations. This is intended to be
suitable for both GPU (or otherwise) accelerated debayer implementations
as well as CPU based debayering.

Doxygen documentation by Dennis Bonke.

Tested-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org> # sc8280xp Lenovo x13s
Tested-by: Pavel Machek <pavel@ucw.cz>
Reviewed-by: Pavel Machek <pavel@ucw.cz>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Co-developed-by: Dennis Bonke <admin@dennisbonke.com>
Signed-off-by: Dennis Bonke <admin@dennisbonke.com>
Co-developed-by: Andrey Konovalov <andrey.konovalov@linaro.org>
Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2024-04-16 13:00:21 +01:00
Hans de Goede
c683e81947 libcamera: software_isp: Add SwStatsCpu class
Add a CPU based SwStats implementation for SoftwareISP / SoftIPA use.

This implementation offers a configure function + functions to gather
statistics on a line by line basis. This allows CPU based software
debayering to call into interleave debayering and statistics gathering
on a line by line basis while the input data is still hot in the cache.

This implementation also allows specifying a window over which to gather
statistics instead of processing the whole frame.

Doxygen documentation by Dennis Bonke.

Tested-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org> # sc8280xp Lenovo x13s
Tested-by: Pavel Machek <pavel@ucw.cz>
Reviewed-by: Pavel Machek <pavel@ucw.cz>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Co-developed-by: Andrey Konovalov <andrey.konovalov@linaro.org>
Signed-off-by: Andrey Konovalov <andrey.konovalov@linaro.org>
Co-developed-by: Pavel Machek <pavel@ucw.cz>
Signed-off-by: Pavel Machek <pavel@ucw.cz>
Co-developed-by: Dennis Bonke <admin@dennisbonke.com>
Signed-off-by: Dennis Bonke <admin@dennisbonke.com>
Co-developed-by: Marttico <g.martti@gmail.com>
Signed-off-by: Marttico <g.martti@gmail.com>
Co-developed-by: Toon Langendam <t.langendam@gmail.com>
Signed-off-by: Toon Langendam <t.langendam@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2024-04-16 13:00:21 +01:00