A `Process` object has address identity because a pointer to it is
stored inside the `ProcessManager`. However, copy/move special
methods are still generated by the compiler. So disable them to
avoid potential issues and confusion.
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
`appendPOD()` does a single insertion, so if only a single `appendPOD()`
will be called on a vector before returning, then calling `reserve()`
is not that useful, so remove it.
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
It is useful to multiply matrices and vectors of heterogeneous types, for
instance float and double. Extend the multiplication operator to support
this, avoiding the need to convert one of the operations. The type of the
returned vector is selected automatically to avoid loosing precision.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
It is useful to multiply matrices of heterogneous types, for instance
float and double. Extend the multiplication operator to support this,
avoiding the need to convert one of the matrices. The type of the
returned matrix is selected automatically to avoid loosing precision.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
By zero-initializing the data_ member we can make most functions
constexpr which will come in handy in upcoming patches. Note that this
is due to C++17. In C++20 we will be able to leave data_ uninitialized
for constexpr. The Matrix(std::array) version of the constructor can
not be constexpr because std::copy only became constexpr in C++20.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The `signature()` getter can just return a reference to the private vector
member variable, and let the caller make a copy if needed. Since the
return type is const qualified, this was likely the original intention.
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Extend the Simple IPA IPC to support returning a metadata ControlList
when the process call has completed.
A new signal from the IPA is introduced to report the metadata,
similarly to what the hardware pipelines do.
Merge the metadata reported by the ISP into any completing request to
provide to the application. Completion of a request is delayed until
this is done; this doesn't apply to canceled requests.
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
This patch applies color correction matrix (CCM) in debayering if the
CCM is specified. Not using CCM must still be supported for performance
reasons.
The CCM is applied as follows:
[r1 g1 b1] [r]
[r2 g2 b2] * [g]
[r3 g3 b3] [b]
The CCM matrix (the left side of the multiplication) is constant during
single frame processing, while the input pixel (the right side) changes.
Because each of the color channels is only 8-bit in software ISP, we can
make 9 lookup tables with 256 input values for multiplications of each
of the r_i, g_i, b_i values. This way we don't have to multiply each
pixel, we can use table lookups and additions instead. Gamma (which is
non-linear and thus cannot be a part of the 9 lookup tables values) is
applied on the final values rounded to integers using another lookup
table.
Because the changing part is the pixel value with three color elements,
only three dynamic table lookups are needed. We use three lookup tables
to represent the multiplied matrix values, each of the tables
corresponding to the given matrix column and pixel color.
We use int16_t to store the precomputed multiplications. This seems to
be noticeably (>10%) faster than `float' for the price of slightly less
accuracy and it covers the range of values that sane CCMs produce. The
selection and structure of data is performance critical, for example
using bytes would add significant (>10%) speedup but would be too short
to cover the value range.
The color lookup tables can be represented either as unions,
accommodating tables for both the CCM and non-CCM cases, or as separate
tables for each of the cases, leaving the tables for the other case
unused. The latter is selected as a matter of preference.
The tables are copied (as before), which is not elegant but also not a
big problem. There are patches posted that use shared buffers for
parameters passing in software ISP (see software ISP TODO #5) and they
can be adjusted for the new parameter format.
Color gains from white balance are supposed not to be a part of the
specified CCM. They are applied on it using matrix multiplication,
which is simple and in correspondence with future additions in the form
of matrix multiplication, like saturation adjustment.
With this patch, the reported per-frame slowdown when applying CCM is
about 45% on Debix Model A and about 75% on TI AM69 SK.
Using std::clamp in debayering adds some performance penalty (a few
percent). The clamping is necessary to eliminate out of range values
possibly produced by the CCM. If it could be avoided by adjusting the
precomputed tables some way then performance could be improved a bit.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Applying color correction matrix (CCM) in software ISP is optional due
to performance reasons. CCM is applied if and only if `Ccm' algorithm
is present in the tuning file.
Software ISP debayering is a performance critical piece of code and we
do not want to use dynamic conditionals there. Therefore we pass
information about CCM application to debayering configuration and let it
select the right versions of debayering functions using templates. This
is a trick similar to the previously used one for adding or not adding
an alpha channel to the output.
Debayering gets this information but it ignores it in this patch.
Actual processing with CCM is added in the followup patch.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The `V4L2BufferCache` type is not thread-safe. Its `lastUsedCounter_`
member is not used in contexts where its atomicity would matter.
So it does not need to be have an atomic type.
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
`MediaEntity::ancillaryEntities()` can just return a const lvalue
reference to the underlying array, a copy need not be made. That
was likely the original intention.
Fixes: 9490c664b5 ("libcamera: Add members to MediaEntity to support ancillary entities")
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
There may be pending messages in SoftwareIsp message queue when
SoftwareIsp stops. The call to IPAProxySoft::stop() will dispatch them
before SoftwareIsp::stop() finishes. But this is dependent on
IPAProxySoft::stop() implementation, let's break this dependency and
dispatch messages to SoftwareIsp explicitly in SoftwareIsp::stop().
This also allows dropping `running_' flag. Since the SoftwareIsp
messages get processed and invoke IPA calls before the IPA proxy is set
to ProxyStopping state and the SoftwareIsp worker thread is no longer
running, it's guaranteed that no new messages come to SoftwareIsp and
attempt to call the stopped IPA proxy.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When SoftwareIsp stops, input and output buffers queued to it may not
yet be fully processed. They will be eventually returned but stop means
stop, there should be no processing related actions invoked afterwards.
Let's stop forwarding processed input buffers from SoftwareIsp slots
when SoftwareIsp is stopped. Let's track the queued input buffers and
return them back for capture in SoftwareIsp::stop().
The returned input buffers are marked as cancelled. This is not
necessary at the moment but it gives the pipeline handlers chance to
deal with this if they need to.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When SoftwareIsp stops, input and output buffers queued to it may not
yet be fully processed. They will be eventually returned but stop means
stop, there should be no processing related actions invoked afterwards.
Let's stop forwarding processed output buffers from the SoftwareIsp
slots once SoftwareIsp is stopped. Let's track the queued output
buffers and mark those still pending as cancelled in SoftwareIsp::stop
and return them to the pipeline handler.
Dealing with input buffers is addressed in a separate patch.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Software ISP runs debayering in a separate thread and debayering may
emit statsReady when software ISP (including the IPA) is being stopped.
The signal waits in a queue and gets invoked later, resulting in an
assertion error when attempting to invoke a method on the stopped IPA:
FATAL default soft_ipa_proxy.cpp:456 assertion
"state_ == ProxyRunning" failed in processStatsThread()
Let's prevent this problem by forwarding the ISP stats signal from
software ISP only when the IPA is running. To track this,
SoftwareISP::running_ variable is introduced.
Making processing of the other signals in SoftwareISP more robust is
addressed in the followup patches.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reported-by: Stanislaw Gruszka <stanislaw.gruszka@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Prepare the move of the Vector class from libipa to libcamera by copying
the relevant files into the corresponding libcamera directories. The
files are copied without modification.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add a data() function to the Matrix class to access the internal data.
This is useful for code that needs to use the matrix contents as a
linear array, as shown by the RkISP1::Ccm::process() function that needs
to copy the matrix data to a local variable. Simplify that function by
using the new accessor.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
inputBufferReady ready signal in the simple pipeline is handled in the
pipeline handler thread. outputBufferReady and ispStatsReady signals
should be handled there too.
Rather than relying on the user of the SoftwareIsp instance, let
SoftwareIsp inherits Object. SoftwareIsp serves as a signal proxy, the
signals above are emitted from signal handlers. This means that if
SoftwareIsp inherits Object then the slots are invoked in SoftwareIsp
thread. Which is the camera manager thread because the SoftwareIsp
instance is created there.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The PipelineHandler::configurationFile() function prints an error
message when no configuration file is found. It can be useful for
pipeline handlers to silence the lookup operation and handle errors
themselves. Add a silent parameter to the function to enable this.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Julien Vuillaumier <julien.vuillaumier@nxp.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Without the change the build fails on upcoming `gcc-15` as:
In file included from ../src/libcamera/dma_buf_allocator.cpp:9:
../include/libcamera/internal/dma_buf_allocator.h:66:19: error: 'uint64_t' has not been declared
66 | void sync(uint64_t step);
| ^~~~~~~~
Signed-off-by: Sergei Trofimovich <slyich@gmail.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Some sensors support producing and transmitting embedded data over a
stream separate from the image stream. Add support for this feature in
the CameraSensor interface, and implement it for the CameraSensorRaw
class. The CameraSensorLegacy uses the default stub implementation, as
the corresponding kernel drivers don't support embedded data.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
With support for metadata in the streams API, the v4l2_meta_format
structure has been extended with width, height and bytesperline fields.
Support them in the V4L2VideoDevice getFormat() and setFormat()
functions is the video device is meta capture device and if the
pixel format is one of the generic line-based metadata formats.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
As DmaSyncer does sync start/end in the c'tor/d'tor, copying a DmaSyncer
instance would trigger sync end earlier than expected. This patch makes
it non-copyable to avoid the issue.
Fixes: 39482d59fe ("DmaBufAllocator: Add Dma Buffer synchronization function & helper class")
Signed-off-by: Harvey Yang <chenghaoyang@chromium.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The getFormat function takes the aspect ratio and the area of the
requested size into account when choosing the best sensor size. In case
the sensor is connected to an rkisp1 the maximum supported frame size of
the ISP is another constraining factor for the selection of the best
format. Add a maxSize parameter to support such a constraint.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Add to the Converter interface two functions used by pipeline handlers
to validate and adjust the converter input and output configurations
by specifying the desired alignment for the adjustment.
Add the adjustInputSize() and adjustOutputSize() functions that allows
to adjust the converter input/output sizes with the desired alignment.
Add a validateOutput() function meant to be used by the pipeline
handler implementations of validate(). The function adjusts a
StreamConfiguration to a valid configuration produced by the Converter.
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Add a isConfigured() function to be able to check if a given stream was
configured in the converter. This is useful in pipelines to either query
device or stream specific crop bounds depending on whether the stream is
configured or not.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The inputCropBounds_ member of the V4L2M2MConverter::Stream class is
only initialized after a V4L2M2MConverter::configure() call, when the
streams are initialized.
However, the converter has crop limits that do not depend on the
configured Streams, and which are currently not accessible from the
class interface.
Add a new inputCropBounds() function to the V4L2M2MConverter class
that allows to retrieve the converter crop limits before any stream
is configured.
This is particularly useful for pipelines to initialize controls and
properties and to implement validation before the Camera gets
configured.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
In preparation for adding new functions and overrides to the
V4L2M2MDevice class, add an override specifier to all overridden
functions in the class. This prevents -Winconsistent-missing-override
warnings.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
This patch introduces support for applying runtime controls to software
ISP. It enables the contrast control as the first control that can be
used.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The Matrix constructor that takes a std::vector is meant and only used
to initialize a Matrix from an initializer list. Using a std::vector is
problematic for two reasons. First, it requires constructing a vector,
copying the data from the initializer list, which is an expensive
operation. Then, the vector size can't be verified at compile time,
making the constructor unsafe.
The first issue could be solved by replacing the vector with a
std::initializer_list or a Span. The second issue would require checking
the initializer list size with a static assertion, or restricting usage
of the constructor to fixed-extent spans. Unfortunately, even if the
size of initializer lists is always known at compile time, the
std::initializer_list::size() function is a compile-time constant only
for constant initializer lists. Using a span would work better, but
construction of a fixed extent span from an initializer list must be
explicit, making the API cumbersome.
We can solve all those issues by passing an std::array to the
constructor. Construction of an array from an initializer list can be
implicit and doesn't involve a copy, and the array size is a template
parameter and therefore guaranteed to be a compile-time constant.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
In preparation to moving the matrix implementation from libipa to
libcamera copy the corresponding files to the new location. The files
are copied without modification to make upcoming integration changes
easier to see. The new files are not included in the build and therefore
have no negative side effects on the build.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Add properties covering the sensor control application delays to both
the static CameraSensorProperties definitions. The values used are
taken from Raspberry Pi's CamHelper class definitions. Where no more
specific values are known the delay struct is defined as empty and
defaults supplied through the getter function.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
std::from_chars(), introduced in C++17, is a fast, locale-independent
string-to-arithmetic conversion function. The C++ standard library
provides overloads for all integer types, making it a prime candidate to
replace the manual handling of integer sizes in the YamlParser string to
integer conversion.
Compared to std::strtol(), std::from_chars() doesn't recognize the '0x'
prefix or '+' prefix, and doesn't ignore leading white space. As the
YamlParser doesn't require those features, std::from_chars() can be used
safely, reducing the amount of code.
C++17 also requires the standard C++ library to provide overloads for
floating-point types, but libc++ does not implement those. The float and
bool implementations of YamlParser::Getter::get() are therefore kept
as-is.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Debug metadata often occurs in places where the metadata control list is
not available e.g. in queueRequest() or processStatsBuffer() or even in
a class far away from the metadata handling code. It is therefore
difficult to add debug metadata without adding lots of boilerplate
code. This can be mitigated by recording the metadata and forwarding it
to the metadata control list when it becomes available. To solve the
issue of code that is far away from the metadata context, add a chaining
mechanism to allow loose coupling at runtime.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
