When injecting input tensors, the output results are expected to be made
available at a specific frame. This change exports tensors' frameCount
so that an appropriate comparison can be made by the controlling
application.
Signed-off-by: Richard Oliver <richard.oliver@raspberrypi.com>
If the decimation width is greater than the max stats width, add a
centre crop on the image to allow the stats to be gathered correctly.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Look for the RAW mandatory stream flag in the pipeline handler config
file. If this flag is set, it guarantees that the application will
provide buffers for Unicam Image, so override the minUnicamBuffers and
minTotalUnicamBuffers config parameters in the following way:
- If startup drop frames are required, allocate at least 1 internal buffer.
- If no startup drop frames are required, do not allocate any internal buffers.
Look for the Output 0 mandatory stream flag in in the pipeline handler
config file. If this flag is set, it guarantees that the application
will provide buffers for the ISP, do not allocate any internal buffers
for the device.
Add a new rpi_apps.yaml pipeline handler config file that enables both
these flags. To use the file, set the following env variable for a
custom build:
export LIBCAMERA_RPI_CONFIG_FILE=/usr/local/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml
or for a packaged install:
export LIBCAMERA_RPI_CONFIG_FILE=/usr/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
This warning is issued by the compiler because the strncpy copies across
the whole string array, leaving no place for the nul terminator. Fix it
by copying sizeof(str) - 1 elements.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Add a Sony IMX500 camera helper to the IPA. This also includes support
for the on-chip CNN hardware accelerator and parsing of the neural
network data stream returned in the metadata buffer.
Add tuning files for both VC4 and PiSP platforms.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Add code to handle the new CNN vendor controls in the Raspberry Pi IPA.
The value of CnnInputTensorInfo is cached as it is the only stateful
input control.
All other controls are output controls, and the values are copied into
directly from the rpiMetadata object if present. The camera helpers
populate the rpiMetadata object if the sensor supports on-board CNN
processing, such as the IMX500.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Add the follwing RPi vendor controls to handle Convolutional Neural
Network processing:
CnnOutputTensor
CnnOutputTensorInfo
CnnEnableInputTensor
CnnInputTensor
CnnInputTensorInfo
CnnKpiInfo
These controls will be used to support the new Raspberry Pi AI Camera,
using an IMX500 sensor with on-board neural network processing.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
libcamera supports several environment variables that can override
configuration options. This requires components that use such overrides
to call the special envOption() and envListOption() functions, spreading
knowledge of the overrides through the code base. This will hinder
future enhancements to the global configuration, such as implementing
per-camera options that will override pipeline handler-level options. To
prepare for that, move handling of the environment variables to the
GlobalConfiguration class.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
The GlobalConfiguration::Option type was introduced to decouple the
configuration option storage from its users, and make changes to the
underlying implementation easier. While the type could indeed be changed
to map to a different class, the API would need to remain the same. The
Option type therefore brings little value. Drop it.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
global_configuration.h uses std::initializer_list, include the
corresponding header. Add a comment in global_configuration.cpp to close
an anonymous namespace, as mandated by the coding style.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
Member functions should be implemented in the .cpp file in the same
order as in the class definition, within each access group. Move them to
the right location. While at it, move load() before loadFile() in the
class definition to match execution order, making the code easier to
read.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
The GlobalConfiguration is explicitly passed around through constructors
of various objects that need access to the configuration. This ad-hoc
solution works for the specific use cases it was meant to support, but
isn't very generic. We have a top-level object in libcamera, the
CameraManager, that also needs to be accessed from various locations and
is passed to object constructors. Standardize on passing the
CameraManager everywhere, and access the global configuration through
it.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
There will be a need to erase child nodes when implementing support for
overriding configuration options. Add two erase() functions to the
ValueNode class, mimicking the add() API.
More erase() overloads could be added, for instance taking iterators as
parameters, to improve efficiency. This should be considered later, when
usage of the ValueNode class will expand, based on the actual usage
patterns.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
The GlobalConfiguration class will need to add nested children to a
ValueNode. Add a new overload to the add() function for this purpose.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Add two at() functions that return mutable pointer to child nodes, based
on an index for lists and a key for dictionaries.
The API differs from const children accessors that use operator[] and
return a reference in order to allow better error handling: while the
const accessors return a reference to a global instance of an empty
ValueNode when the requested child doesn't exist, a mutable accessor
can't do the same without allowing modifying the empty global instance.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
The ValueNode class was initially designed to store read-only property
trees. It is useful to also provide mutable access. Add non-const
adapters and iterators and adapters. This allows obtaining a mutable
ValueNode instance when traversing a tree.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
The YamlObject class is now a generic data container to model trees of
values. Rename it to ValueNode and expand the class documentation.
While at it, drop the unneeded libcamera:: namespace prefix when using
the ValueNode class.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Fix a few documentation issues:
- Drop unneeded \fn
- Sort \brief and \tparam correctly
- Add missing \tparam and \param
- Explain the T and U template parameters for get()
- Standardize the naming of dictionary keys as "key"
- Fix typo
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
The YamlObject class was designed to represent data parsed from YAML
files. It is outgrowing the initial needs. Move it to a separate file to
prepare for a rename.
Most files that include yaml_parser.h only need access to a YamlObject.
Switch them to yaml_object.h. pipeline_base.cpp was including
yaml_parser.h indirectly through pipeline_base.h, include it directly
now.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
The YamlObject::Accessor structure is designed to extend the YamlObject
class with new getter and setter types without modifying
yaml_parser.cpp. This feature is used for various libcamera classes,
while standard C++ types are handled in yaml_parser.cpp.
The Size class is an outlier: it is a libcamera class, but is handled in
yaml_parser.cpp. Move it to geometry.cpp. Drop the std::vector<Size>
specialization as it is currently unused.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
YamlParserContext is a friend of the YamlObject class to access private
member variables. Now that YamlObject exposes functions to set a value
and add children, this is not needed anymore. Decouple the two classes.
The YamlParserContext::readValue() function now takes a const reference
to the EventPtr as the YamlParserContext::parseNextYamlObject() function
needs to access it in the error handler.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Add YamlObject::add() functions to add children to a list or dictionary
object. This will be used by the YamlParserContext to replace direct
access to YamlObject member variables to decouple the two classes.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
The YamlObject class has two member function templates to get values:
the get() function gets a scalar value, while the getList() function
gets a vector of scalar values.
As get() is a function template, we can provide specializations for
vector types. This makes the code more systematic, and therefore more
readable. Replace all getList() occurrences, and drop the getList()
function.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Plumb in the MaliC55 pipeline handler support for capturing frames
from memory using the CRU.
Introduce a data flow which uses the CRU to feed the ISP through
the IVC.
In detail:
- push incoming request to a pending queue until a buffer from the CRU
is available
- delay the call to ipa_->fillParams() to the CRU buffer ready event
- once the IPA has computed parameters feed the ISP through the IVC
with buffers from the CRU, params and statistics
- Handle completion of in-flight requests: in m2m mode buffers are
queued earlier to the ISP capture devices. If the camera is stopped
these buffers are returned earlier in error state: complete the
Request bypassing the IPA operations
- As cancelled Requests might now be completed earlier then IPA events,
modify the IPA event handler to ignore Requests that have been
completed already
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Do not rely on bufferCount for the allocation of parameters and stats
buffers but add instead kMaliC55BufferCount which is also used to limit
the number of in-flight requests supported by the pipeline handler.
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Add support for memory-to-memory Camera to the
PipelineHandlerMaliC55::configure() function.
Start by enabling the IVC links, then configure the CRU and
propagate the format to the IVC.
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Add support to Mali C55 pipeline handler for memory-to-memory camera
mode.
The Mali-C55, as integrated in the Renesas RZ/V2H(P) SoC, is fed with
images saved to memory by the CRU unit and read on behalf of the ISP
by the IVC.
Update the pipeline handler's match() function to search for the CRU
media entity necessary to register a memory input camera. Create and
initialize a CameraData instance for the memory camera use case if a
valid CRU is found and initialize the IVC components used to feed the
ISP with images read from memory.
Do not implement camera configuration for the time being to reuce
the patch size. It will be provided in the following patches.
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In order to prepare to support memory input cameras, split the handling of
the TPG and Inline camera cases.
The Mali C55 pipeline handler uses the entity and subdevice stored in the
CameraData to support both the handling of the TPG and of the Inline (CSI-2
+ sensor) use cases. Adding support for memory cameras by using the CRU unit
would add yet-another special case making the code harder to follow and
more prone to errors.
Split the handling of the TPG and Inline cameras by introducing an
std::variant<> variable and to deflect the functions called on the
camera data to the correct type by using overloaded std::visit<>().
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
std::visit() allows quite elegant type-matching implementation of the
visitor pattern.
The 'overloaded' type helpers allow to define a hierarchy of overloaded
operator() implementations which can be used by std::visit().
Currently only the Virtual pipeline handler uses this type-matching
implementation of std::visit(). To prepare to add another user in the
Mali C55 pipeline handler move the 'overloaded' helper type to
libcamera::utils for easier re-use.
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Add a class allowing us to control the RZ/G2L Camera Receiver Unit.
The class is named after the media device it handles, which is called
"rzg2l_cru" but the CRU is actually found in other SoCs than the RZ/G2L
one, such as the RZ/V2H(P).
The RZG2LCRU class models the CSI-2 receiver found on those SoCs and
allows to add support for memory-to-memory operations on the RZ/V2H(P)
SoC which integrates a Mali-C55 ISP and a specialized DMA engine named
IVC that feeds images from memory, where the CRU has saved them.
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The IPAManager is currently instantiated when constructing the
CameraManager constructor, in the CameraManager::Private constructor.
This is the only sizeable object constructed with the CameraManager, all
other components are constructed when starting the manager.
Early construction of the IPAManager isn't wrong per-se, but prevents
accessing the CameraManager from the IPAManager constructor (as the
former isn't fully constructed). It also results in internal objects
constructed in different threads, which isn't an issue at the moment as
none of the objects constructed by the IPAManager are thread-bound, but
could cause issues later. Finally, it prevents influencing the
IPAManager creation with parameters passed to the CameraManager::start()
function once we implement this, which would be useful for applications
to override configuration parameters related to IPA modules.
Move the instantiation of the IPAManager to start time to fix all those
issues.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
The ipa_interface unit test uses a FIFO to communicate with the vimc IPA
module. FIFOs are named pipes, created in the file system. The path to
the FIFO is hardcoded so that both the unit test and IPA module know
where to locate the file.
If the ipa_interface crashes for any reason, the FIFO will not be
removed, and subsequent usage of the vimc IPA module will hang when
trying to write to the FIFO in the IPA module.
Fix this by replacing the FIFO with a pipe. Pipes are unidirectional
data channels that are represented by a pair of file descriptors,
without any presence in the file system. The write end of the pipe is
passed to the vimc IPA module init() function, and then used the same
way as the FIFO.
While at it, use a std::unique_ptr to manage the notifier in the unit
test instead of manually allocating and deleting the object.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
IPA proxies are created with a call to IPAManager::createIPA(), which is
a static member function. This requires a complicated dance in the
createIPA() function to retrieve the IPAManager instance through the
camera manager, itself retrieved from the pipeline handler. Simplify the
code by turning IPAManager::createIPA() into a non-static member
function, and providing a wrapper in the PipelineHandler class to keep
instantiation of IPA proxies easy in pipeline handlers.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
libcamera uses std::unique_ptr<> to simplify life time management of
objects and avoid leaks. For historical reasons there are a fair number
of plain pointers with manual memory management. Replace them with
std::unique_ptr<> when the conversion is simple.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
`RkISP1Frames::{clear,destroy}()` always push all buffers from the
`RkISP1FrameInfo` object to the corresponding `available*Buffers_`
queues. This is not entirely correct.
If no dewarper is used, then `availableMainPathBuffers_` is never
consumed, so it will grow with each request. Fix it by only queueing
the buffer if `RkISP1CameraData::usesDewarper_`.
Additionally, in raw capture mode, no parameter or statistics buffers are
used, so those queues will be filled up with `nullptr`s without limit.
Fix that by only queueing them if they are not null.
Fixes: c8f63760e5 ("pipeline: rkisp1: Support raw Bayer capture at runtime")
Fixes: 12b553d691 ("libcamera: rkisp1: Plumb the dw100 dewarper as V4L2M2M converter")
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Add the missing `override` specifiers in the derived classes of `Debayer`,
so that the compiler can ensure that the functions in the derived classes
really override the functions in the base class.
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Bryan O'Donoghue <bod.linux@nxsw.ie>
Richard OliverandKonsta
Naushir PatuckandKonsta
Laurent Pinchart
Jacopo MondiandKieran Bingham
Daniel ScallyandJacopo Mondi
Barnabás Pőcze