Commit Graph

599 Commits

Author SHA1 Message Date
Naushir Patuck 5acc21fd04 pipeline: raspberrypi: Apply sensor flips at the start of configure()
Sensor flips might change the Bayer order of the requested format. The existing
code would set a sensor format along with the appropriate Unicam and ISP input
formats, but reset the latter two on start() once the flips had been requested.

We can now set the sensor flips just before we set the sensor mode in
configure(), thereby not needing the second pair of format sets in start().

Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-02 12:08:45 +00:00
Jacopo Mondi 4ed22985a8 ipa: ipu3: Update camera controls in configure()
When a new CameraConfiguration is applied to the Camera the IPA is
configured as well, using the newly applied sensor configuration and its
updated V4L2 controls.

Also update the Camera controls at IPA::configure() time by re-computing
the controls::ExposureTime and controls::FrameDurationLimits limits and
update the controls on the pipeline handler side after having configured
the IPA.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-15 09:32:30 +02:00
Jacopo Mondi 2a938efc8c libcamera: ipu3: Split controls init/update
In order to prepare for updating the Camera controls limits when a new
camera configuration is applied, split the initControls() function in
two:
- updateControls() to actually compute controls values
- initControls() to initialize the sensor configuration and call
  updateControls

Update the functions documentation accordingly.

No functional changes intended.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2021-10-15 09:32:30 +02:00
Jacopo Mondi 2dab504a9b libcamera: ipu3: Rationalize constant expressions names
Following the previous patch that moved all the ImgU-related contants in
the ImgUDevice class namespace and that aligned their naming scheme to
the 'kNameOfConstant' scheme, apply the same changes to the other
components of the IPU3 pipeline handler.

Cosmetic change, no functional changes intended.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-15 09:32:28 +02:00
Jacopo Mondi 3bd3b95a09 libcamera: ipu3: Centralize ImgU sizes definition
The definition of several constants that describe the ImgU
characteristics are spread between two files: ipu3.cpp and imgu.cpp.

As the ipu3.cpp uses definitions from the imgu.cpp file, in order to
remove the usage of magic numbers, it is required to move the
definitions to a common header file where they are accessible to the
other .cpp modules.

Move all the definitions of the ImgU sizes and alignments to the
ImgUDevice class as static constexpr and update their users accordingly.

Cosmetic changes, no functional changes intended.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-15 09:30:55 +02:00
Jacopo Mondi 1e9826ff71 libcamera: ipu3: Use the optimal sensor size
As reported by commit 7208e70211 ("libcamera: ipu3: Always use sensor
full frame size") the current implementation of the IPU3 pipeline
handler always uses the sensor resolution as the ImgU input frame size in
order to work around an issue with the ImgU configuration procedure.

Now that the frame selection policy has been modified in the CIO2Device
class implementation to comply with the requirements of the ImgU
configuration script we can remove the workaround and select the most
opportune sensor size to feed the ImgU with.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-15 09:25:04 +02:00
Laurent Pinchart ca5fb99409 libcamera: pipeline: ipu3: Use new Size grownBy() and shrunkBy() helpers
The Size class has new helpers that can simplify the code in the IPU3
pipeline handler. Use them.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-10-15 05:05:19 +03:00
David Plowman 962df634bd pipeline: raspberrypi: Create empty control lists correctly
When the pipeline handler start() method is supplied with a NULL list
of controls, we send an empty control list to the IPA. When the IPA is
running in isolated mode the control list goes through the data
serializer, for which it must be marked correctly as a list of
"controls::controls", otherwise the IPA process will abort.

The IPA has a similar problem returning a control list in its
configure() method. We must be careful to initialise it properly even
when empty.

Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-07 01:05:29 +03:00
Jacopo Mondi 4c1fc33d8a libcamera: ipu3: Drop entityControls map
The IPA::configure() function has an IPAConfigInfo parameters which
contains a map of numerical indexes to ControlInfoMap instances.

This is a leftover of the old IPA protocol, where it was not possible to
specify a rich interface as it is possible today and each entity
ControlInfoMap was indexed by a numerical id and stored in a map.

Now that the IPA interface allows to specify parameters by name, drop the
map and send the sensor's control info map only.

If we'll need more ControlInfoMap to be shared with the IPA, a new parameter
can be added to IPAConfigInfo.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-09-27 14:35:51 +02:00
Laurent Pinchart 9a8c0365f7 libcamera: Use V4L2PixelFormat::fromPixelFormat()
Replace manual searches for V4L2 pixel format in the PixelFormatInfo
with the V4L2PixelFormat::fromPixelFormat() helper function. This
prepares for multi-planar support that will modify how V4L2 pixel
formats are stored in PixelFormatInfo.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Hirokazu Honda <hiroh@chromium.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-09-07 19:17:49 +03:00
Laurent Pinchart 395d43d6d7 libcamera: v4l2_videodevice: Drop toV4L2PixelFormat()
The V4L2VideoDevice::toV4L2PixelFormat() function is incorrectly
implemented, as it will pick a multi-planar format if the device
supports the multi-planar API, even if only single-planar formats are
supported. This currently works because the implementation calls
V4L2PixelFormat::fromPixelFormat(), which ignores the multiplanar
argument and always returns a single-planar format.

Fixing this isn't trivial. As we don't need to support multi-planar V4L2
formats at this point, drop the function instead of pretending
everything is fine, and call V4L2PixelFormat::fromPixelFormat() directly
from pipeline handlers. As the single-planar case is the most common,
set the multiplanar argument to false by default to avoid long lines.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Hirokazu Honda <hiroh@chromium.org>
2021-09-07 19:17:46 +03:00
Laurent Pinchart 29b372a56a libcamera: pipeline: simple: Remove SimplePipelineHandler::activeCamera_
The SimplePipelineHandler activeCamera_ member pointer is set but never
used. Drop it.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
2021-08-31 22:44:11 +03:00
Laurent Pinchart 33914c4cee libcamera: pipeline: simple: Move bufferReady handler to SimpleCameraData
To use multiple cameras at the same time, a per-camera buffer ready
handler is needed. Move the bufferReady() function connected to the
V4L2VideoDevice bufferReady signal from the SimplePipelineHandler class
to the SimpleCameraData class.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
2021-08-31 22:44:11 +03:00
Laurent Pinchart d6e48e5e7c libcamera: pipeline: simple: Move converter to SimpleCameraData
To use multiple cameras at the same time, each camera instance will need
its own converter. Store the converter in SimpleCameraData, and open it
in init().

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
2021-08-31 22:44:11 +03:00
Laurent Pinchart 8c9e1926b9 libcamera: pipeline: simple: Add pipeline pad reservation mechanism
The cameras created by the same pipeline handler instance may share
hardware resources, prohibiting usage of multiple cameras concurrently.
Implement a heuristic to reserve resources and handle mutual exclusiong
in a generic way.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
2021-08-31 22:44:11 +03:00
Laurent Pinchart f69b19667f libcamera: pipeline: simple: Open all video devices at match() time
Move opening of video devices at match() time, the same way as subdevs
are opened, to make the handling of V4L2 video devices and subdevices
more consistent.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
2021-08-31 22:44:11 +03:00
Laurent Pinchart 2b31458763 libcamera: pipeline: simple: Store all entity devices in common map
Merge the SimplePipelineHandler videos_ and subdevs_ maps, which
respectively store V4L2 video devices and subdevices associated with
entities, into a single entities_ map that contains an EntityData
structure. This gathers all data about entities in a single place,
allowing for easy extension of entity data in the future.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
2021-08-31 22:44:11 +03:00
Laurent Pinchart 41a8f8a9cc libcamera: pipeline: simple: Store video node entity in camera data
Store the entity corresponding to the video node at the end of the
pipeline in the SimpleCameraData::entities_ list. This requires special
handling of the video node in the loops that iterate over all entities,
but will be useful to implement mutually exclusive access to entities
for concurrent camera usage.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
2021-08-31 22:44:11 +03:00
Laurent Pinchart 0e8d6903b2 libcamera: pipeline: simple: Delay opening of video device until init()
The video device is currently opened in the SimpleCameraData
constructor. This requires opening the video devices on-demand the first
time they're accessed, which gets in the way of refactoring of
per-entity data storage in the pipeline handler.

Move opening of the video device to the SimpleCameraData::init()
function. The on-demand open mechanism isn't touched yet, it will be
refactored later.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
2021-08-31 22:44:11 +03:00
Laurent Pinchart 0f65875baf libcamera: pipeline: simple: Add sink and source pads to entity data
Record the sink and source pads through which an entity is traversed in
the list of entities stored in the camera data. This prepares for
implementing mutually exclusive access to entities between cameras.

The debug message that displays detected pipelines now prints pads to
describe the pipeline more precisely:

[0:00:35.901275750] [260] DEBUG SimplePipeline simple.cpp:404 Found pipeline: [imx290 2-001a|0] -> [0|csis-32e40000.csi|1] -> [0|mxc_isi.0|1] -> [0|mxc_isi.0.capture]

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Tested-by: Martin Kepplinger <martin.kepplinger@puri.sm>
2021-08-31 22:44:10 +03:00
Umang Jain 5fc426fbfe ipu3: cio2: Tweak sensor size selection policy
Do not compare higher precision of the ratios, as it might lead to
absurd selection of sensor size for a relatively low requested
resolution size.

For example:
The imx258 driver supports the following sensor resolutions:

 - 4208x3118 = 1.349583066
 - 2104x1560 = 1.348717949
 - 1048x780  = 1.343589744

It can be inferred that, that the aspect ratio only differs by a small
mantissa with each other. It does not makes sense to select a 4208x3118
for a requested size of say 640x480 or 1280x720, which is what is
happening currently.
($) cam -c1 -swidth=640,height=480,role=raw
    - CIO2 configuration: 4208x3118-SGRBG10_IPU3 [*]

In order to address this constraint, only compare the ratio with single
precision to make a better decision on the sensor resolution policy
selection.

($) cam -c1 -srole=raw,width=640,height=480
    - CIO2 configuration: 1048x780-SGRBG10_IPU3 [*]

[*] Please revert 0536a9aa7189("ipu3: Disallow raw only camera configuration")
    if the configuration is reported as invalid.

Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Tested-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-31 12:39:50 +05:30
Umang Jain 58b053c021 ipu3: cio2: Change sensor size selection policy
The current implementation of getSensorFormat() prioritizes sensor
sizes that match the output size FOV ratio.

Modify the frame size selection procedure to prioritize resolutions
with the same FOV as the sensor's native one, to avoid cropping in the
sensor pixel array.

For example:

- on a Soraka device equipped with ov13858 as back sensor, with a
  native resolution of 4224x3136 (4:3), when requested to select the
  sensor output size to produce 1080p (16:9) a frame size of 2112x1188
  (16:9) is selected causing the ImgU configuration procedure to fail.
  If a resolution with the same FOV as the sensor's native size, such
  as 2112x1568 (4:3), is selected the pipeline works correctly.

Suggested-by:: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Tested-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-31 12:27:02 +05:30
Umang Jain fd9068ce44 ipu3: cio2: Replicate CameraSensor::getFormats() to a member function
This prepares a base to introduce custom selection of sensor format
based on platform(Soraka or Nautilus) constraints. The changes in
selection policy will be introduced in a subsequent patch.

The function is copied from CameraSensor::getFormat() in the IPU3
pipeline handler code to be later changed on top.

The copy is not verbatim and has a minor change as follows:
CameraSensor::getFormats() has access to a V4L2Subdevice::Formats
internally and use it directly to iterate over supported camera sensor
frame sizes. The copy is adapted to use the CameraSensor::sizes(mbusCode)
instead, to enumerate over the supported frame sizes as per
mbusCodesToPixelFormat map.

No functional changes in this patch.

Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Tested-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-31 12:26:58 +05:30
Umang Jain a18c9f0490 libcamera: camera_sensor: Transform CameraSensor::sizes()
In CameraSensor, the mbusCodes() and sizes() accessor functions
retrieves all the supported media bus codes and the supported sizes
respectively. However, this is quite limiting since the caller
probably isn't in a position to match which range of sizes are
supported for a particular mbusCode.

Hence, the caller is most likely interested to know about the sizes
supported for a particular media bus code. This patch transforms the
existing CameraSensor::sizes() to CameraSensor::sizes(mbuscode) to
achieve that goal.

The patch also transforms existing CIO2Device::sizes() in IPU3 pipeline
handler to CIO2Device::sizes(PixelFormat) on a similar principle. The
function is then plumbed to CameraSensor::sizes(mbusCode) to enumerate
the per-format sizes as required in
PipelineHandlerIPU3::generateConfiguration().

Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Tested-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-31 12:26:54 +05:30
Umang Jain c0417c4d52 pipeline: ipu3: Skip recording timestamp for cancelled buffers
There is no point in recording sensor's timestamp when V4L2VideoDevice
has marked the frame buffers with FrameMetadata::FrameCancelled
(happens when the streams are stopped). The metadata is mostly invalid
for cancelled buffers  hence, setting timestamp on invalid metadata
does not make sense (however some metadata can be considered valid
for e.g. returning cause of failure through metadata on cancelled
buffers).

Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-17 12:30:00 +05:30
Laurent Pinchart cb63fc75fb libcamera: pipeline: Cast to derived pipeline handler with helpers
Replace manual static casts from the PipelineHandler pointer to a
derived class pointer with helper functions in the camera data classes.
This simplifies code accessing the pipeline from the camera data.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-17 00:24:16 +03:00
Laurent Pinchart ddd267c348 libcamera: pipeline_handler: Drop CameraData class
The CameraData class isn't used anymore. Drop it.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-08-17 00:24:16 +03:00
Laurent Pinchart 42808665a0 libcamera: pipeline: ipu3: Migrate to Camera::Private
As part of the effort to remove the CameraData class, migrate the
pipeline handler-specific camera data from CameraData to the
Camera::Private class.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-08-17 00:24:15 +03:00
Laurent Pinchart 746feade70 libcamera: pipeline: raspberrypi: Migrate to Camera::Private
As part of the effort to remove the CameraData class, migrate the
pipeline handler-specific camera data from CameraData to the
Camera::Private class.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-08-17 00:24:15 +03:00
Laurent Pinchart 211b91c284 libcamera: pipeline: rkisp1: Migrate to Camera::Private
As part of the effort to remove the CameraData class, migrate the
pipeline handler-specific camera data from CameraData to the
Camera::Private class.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-08-17 00:24:15 +03:00
Laurent Pinchart de2668d8a7 libcamera: pipeline: vimc: Migrate to Camera::Private
As part of the effort to remove the CameraData class, migrate the
pipeline handler-specific camera data from CameraData to the
Camera::Private class.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-08-17 00:24:10 +03:00
Laurent Pinchart aae303c5aa libcamera: pipeline: uvcvideo: Migrate to Camera::Private
As part of the effort to remove the CameraData class, migrate the
pipeline handler-specific camera data from CameraData to the
Camera::Private class.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-08-16 23:57:28 +03:00
Laurent Pinchart fb1bcb9c38 libcamera: pipeline: simple: Migrate to Camera::Private
As part of the effort to remove the CameraData class, migrate the
pipeline handler-specific camera data from CameraData to the
Camera::Private class.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-16 23:57:28 +03:00
Laurent Pinchart bcab244b1f libcamera: pipeline_handler: Move CameraData members to Camera::Private
With pipeline handlers now being able to subclass Camera::Private, start
the migration from CameraData to Camera::Private by moving the members
of the base CameraData class. The controlInfo_, properties_ and pipe_
members are duplicated for now, to allow migrating pipeline handlers one
by one.

The Camera::Private class is now properly documented, don't exclude it
from documentation generation.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-08-16 23:57:28 +03:00
Laurent Pinchart 32b8832e38 libcamera: camera: Pass Private pointer to Camera constructor
In order to allow subclassing Camera::Private in pipeline handlers, pass
the pointer to the private data to the Camera constructor, and to the
Camera::createCamera() function.

The Camera::Private id_ and streams_ members now need to be initialized
by the Camera constructor instead of the Camera::Private constructor, to
allow storage of the streams in a pipeline handler-specific subclass of
Camera::Private.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-08-16 23:57:28 +03:00
Umang Jain 5420e359f2 pipeline: vimc: Force complete of request on cancelled buffers
When the stream is stopped, the V4L2VideoDevice sends back all
the queued buffers with FrameMetadata::FrameCancelled status.
It is the responsibility of the pipeline handler to handle
these buffers with FrameMetadata::FrameCancelled. VIMC is
currently missing this handling path.

As the FrameMetadata::FrameCancelled is set when the stream is
stopped, we can be sure that no more queued and re-use of request
shall happen.  Hence, cancel all the requests' buffers force a
complete with completeBuffer().

The issue is caught by the gstreamer_single_stream_test.cpp running
with vimc. During the check with meson built-in option
	'-Db_sanitize=address,undefined'
it was observed:

==118003==ERROR: AddressSanitizer: heap-use-after-free on address 0x60e000037108 at pc 0x7f225160c9ac bp 0x7f224a47b620 sp 0x7f224a47b618
READ of size 4 at 0x60e000037108 thread T1
    #0 0x7f225160c9ab in libcamera::Request::sequence() const ../include/libcamera/request.h:55
    #1 0x7f22518297aa in libcamera::VimcCameraData::bufferReady(libcamera::FrameBuffer*) ../src/libcamera/pipeline/vimc/vimc.cpp:577
    #2 0x7f225183b1ef in libcamera::BoundMethodMember<libcamera::VimcCameraData, void, libcamera::FrameBuffer*>::activate(libcamera::FrameBuffer*, bool) ../include/libcamera/base/bound_method.h:194
    #3 0x7f22515cc91f in libcamera::Signal<libcamera::FrameBuffer*>::emit(libcamera::FrameBuffer*) ../include/libcamera/base/signal.h:126
    #4 0x7f22515c3305 in libcamera::V4L2VideoDevice::streamOff() ../src/libcamera/v4l2_videodevice.cpp:1605
    #5 0x7f225181f345 in libcamera::PipelineHandlerVimc::stop(libcamera::Camera*) ../src/libcamera/pipeline/vimc/vimc.cpp:365

The VimcCameraData::bufferReady seems to emit even after the stream
is stopped. It's primarily due to vimc's lack of handling
FrameMetadata::FrameCancelled in its pipeline handler.

Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-16 20:04:11 +05:30
Umang Jain c2437e8cde ipa: vimc: Send and retrieve FrameBuffers from IPA
Plumb through VIMC mojo interface to enable buffers passing.
VIMC does not have parameters or statistics buffers but we can
mimick the typical case of passing IPA buffers from pipeline
handler to IPA using mock buffers. The mock IPA buffers are
FrameBuffers which are dmabuf backed (in other words, mmap()able
through MappedFramebuffer inside the IPA).

This commits shows:
- Passing the parameter buffer from the pipeline handler to
  the IPA through functions defined in mojom interface.
- Passing request controls ControlList to the IPA.

Any tests using VIMC will now loop in the IPA paths. Any tests running
in isolated mode will help us to test IPA IPC code paths especially
around (de)serialization of data passing from pipeline handlers to the
IPA. Future IPA interface tests can simply extend the vimc mojom
interface to achieve/test a specific use case as required.

Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-16 13:22:11 +05:30
Umang Jain 3c5732d04a ipa: vimc: Map and unmap buffers
VIMC pipeline handler has dmabuf-backed mock FrameBuffers which are
specifically targetted mimicking IPA buffers (parameter and statistics).
Map these mock buffers to the VIMC IPA that would enable exercising IPA
IPC code paths. This will provide leverage to our test suite to test
IPA IPC code paths, which are common to various platforms.

Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-16 13:22:11 +05:30
Umang Jain aad68099e5 pipeline: vimc: Allocate mock IPA buffers
VIMC is a virtual test driver that doesn't have statistics or
parameters buffers that are typically passed from a pipeline
handler to its platform IPA. To increase the test coverage going
forward, we can at least mimick the typical interaction of how
a pipeline handler and IPA interact, and use it to increase the
test coverage.

Hence, create simple (single plane) dmabuf-backed FrameBuffers,
which can act as mock IPA buffers and can be memory mapped (mmap)
to VIMC IPA. To create these buffers, temporarily hijack the output
video node and configure it with a V4L2DeviceFormat. Buffers then
can be exported from the output video node using
V4L2VideoDevice::exportBuffers(). These buffers will be mimicked as
IPA buffers in subsequent commits.

Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-08-16 13:11:49 +05:30
Laurent Pinchart 568c4049bc ipa: vimc: Add configure() function
As part of an effort to make the vimc IPA usable for testing, extend it
with a configure function. The configuration is currently ignored by the
IPA.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
2021-08-16 12:01:18 +05:30
Jacopo Mondi 11fe4333c5 libcamera: ipu3: Initialize controls in the IPA
All the IPU3 Camera controls are currently initialized by the pipeline
handler which initializes them using the camera sensor configuration and
platform specific requirements.

However, some controls are better initialized by the IPA, which might,
in example, cap the exposure times and frame duration to the constraints
of its algorithms implementation.

Also, moving forward, the IPA should register controls to report its
capabilities, in example the ability to enable/disable 3A algorithms on
request.

Move the existing controls initialization to the IPA, by providing
the sensor configuration and its controls to the IPU3IPA::init()
function, which initializes controls and returns them to the pipeline
through an output parameter.

The existing controls initialization has been copied verbatim from the
pipeline handler to the IPA, if not a for few line breaks adjustments
and the resulting Camera controls values are not changed.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2021-08-12 10:08:32 +02:00
Jacopo Mondi b48db3c489 libcamera: controls: Create ControlInfoMap with ControlIdMap
ControlInfoMap does not have a ControlId map associated, but rather
creates one with the generateIdMap() function at creation time.

As a consequence, when in the need to de-serialize a ControlInfoMap all
the ControlId it contains are created by the deserializer instance, not
being able to discern if the controls the ControlIdMap refers to are the
global libcamera controls (and properties) or instances local to the
V4L2 device that has first initialized the controls.

As a consequence the ControlId stored in a de-serialized map will always
be newly created entities, preventing lookup by ControlId reference on a
de-serialized ControlInfoMap.

In order to make it possible to use globally available ControlId
instances whenever possible, create ControlInfoMap with a reference to
an externally allocated ControlIdMap instead of generating one
internally.

As a consequence the class constructors take and additional argument,
which might be not pleasant to type in, but enforces the concepts that
ControlInfoMap should be created with controls part of the same id map.

As the ControlIdMap the ControlInfoMap refers to needs to be allocated
externally:
- Use the globally available controls::controls (or
  properties::properties) id map when referring to libcamera controls
- The V4L2 device that creates ControlInfoMap by parsing the device's
  controls has to allocate a ControlIdMap
- The ControlSerializer that de-serializes a ControlInfoMap has to
  create and store the ControlIdMap the de-serialized info map refers to

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-12 10:06:25 +02:00
Kieran Bingham aa1972529b src: Remove all unused sys/mman.h inclusions
Remove leftover inclusions of the sys/mman header file.

Reviewed-by: Hirokazu Honda <hiroh@chromium.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-10 10:53:43 +01:00
Laurent Pinchart a48a000a33 libcamera: Rename 'method' to 'function'
Usage of 'method' to refer to member functions comes from Java. The C++
standard uses the term 'function' only. Replace 'method' with 'function'
or 'member function' through the whole code base and documentation.
While at it, fix two typos (s/backeng/backend/).

The BoundMethod and Object::invokeMethod() are left as-is here, and will
be addressed separately.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-09 15:40:32 +03:00
Umang Jain 0536a9aa71 ipu3: Disallow raw only camera configuration
To capture raw frames, the ImgU isn't needed. However, to implement
auto-exposure, we do need to configure the IPA since it shall setup
the sensor controls (exposure, vblank and so on) for the capture.
One cannot simply configure the IPA, without the ImgU as the
parameters and statistics buffer passed to the IPA are actually
managed by the ImgU.

Until we prepare and setup the ImgU to run an internal queue for
raw-only camera configuration, disallow this configuration and
report it as invalid.

Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-09 11:02:14 +05:30
Naushir Patuck e87fb20f8f pipeline: raspberrypi: Fix a bug when clearing out Request buffers on stop
When RPiCameraData::clearIncompleteRequests() clears out the request
queue during a stop condition, it unconditionally calls completeBuffer()
on all buffers in each request.  This is wrong, as a buffer could have
already been completed as part of the current request, but the request
itself may not yet have completed.

Fix this by checking if the buffers in the request have been completed
before cancelling them.

Fixes: d372aaa10d ("pipeline: raspberrypi: Simplify RPiCameraData::clearIncompleteRequests()")
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-07-28 09:55:24 +03:00
David Plowman eb9a539938 libcamera: raspberrypi: Allow the tuning file to be set by an environment variable
The configuration (camera tuning) file used by the Raspberry Pi comes
by default from the sensor name. However, we now allow this to be
overridden by the LIBCAMERA_RPI_TUNING_FILE environment variable.

Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@posteo.net>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-07-12 20:45:42 +03:00
Laurent Pinchart d6d4710d04 libcamera: framebuffer: Make FrameBuffer class Extensible
Implement the D-Pointer design pattern in the FrameBuffer class to allow
changing internal data without affecting the public ABI.

Move the request_ field and the setRequest() function to the
FrameBuffer::Private class. This allows hiding the setRequest() function
from the public API, removing one todo item. More fields may be moved
later.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-07-12 19:59:22 +03:00
Naushir Patuck 9c0f6a22a2 pipeline: raspberrypi: Use priority write for vblank when writing sensor ctrls
When directly writing controls to the sensor device, ensure that VBLANK is
written ahead of and before the EXPOSURE control. This is the same priority
write mechanism used in DelayedControls.

Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-07-12 15:57:12 +03:00
Laurent Pinchart 6a31a8d8e2 libcamera: buffer: Rename buffer.h to framebuffer.h
libcamera names header files based on the classes they define. The
buffer.h file is an exception. Rename it to framebuffer.h.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Hirokazu Honda <hiroh@chromium.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-07-11 17:42:02 +03:00