A ClockRecovery object is added for derived classes to use, and
wallclock timestamps are copied into the request metadata for
applications.
Wallclock timestamps are derived corresponding to the sensor
timestamp, and made available to the base pipeline handler class and
to IPAs, for both vc4 and pisp platforms.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
New controls are added to control the camera "sync" algorithm, which
allows different cameras to synchronise their frames. For the time
being, the controls are Raspberry Pi specific, though this is expected
to change in future.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The ClockRecovery class takes pairs of timestamps from two different
clocks, and models the second ("output") clock from the first ("input")
clock.
We can use it, in particular, to get a good wallclock estimate for a
frame's SensorTimestamp.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Whenever a downstream element queries latency, libcamerasrc will always reply,
even though it has not yet determined the latency.
However some downstream elements (e.g. glvideomixer/aggregator) will query the
latency before libcamerasrc sets the caps. When these elements get the latency,
they will start the caps negotiation. Since libcamerasrc has not yet determined
caps, invalid negotiation is performed and workflow is disrupted.
So, set latency to 'GST_CLOCK_TIME_NONE' during initialization, and reply to the
query after libcamerasrc confirms the latency. At this time, libcamerasrc has also
completed caps negotiation and downstream elements work fine.
In addition, every time the src pad task stops, we reset the latency to
GST_CLOCK_TIME_NONE to ensure that when next time task starts, the downstream
elements can generate out buffers after receiving the effective latency.
Signed-off-by: Hou Qi <qi.hou@nxp.com>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This is how uvcvideo and rkisp1 do it. See ee918b370a
("ipa: rkisp1: agc: Initialize enum controls with a list of values")
for the motivation. In summary, having a list of values is used as a sign
that the control is an enum in multiple places (e.g. `cam`, `camshark`).
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The caps object passeed to the gst_libcamera_create_video_pool()
function is managed as a g_autoptr() in the caller. The function doesn't
acquire any new reference, so it shouldn't call gst_caps_unref(). Fix
it.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
The gst_libcamera_create_video_pool() function leaks a GstQuery instance
and a GstBufferPool instance in an error path. Fix the leaks with
g_autoptr().
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Now that the code is isolated in a function, the video_pool variable in
gst_libcamera_create_video_pool() can be renamed to pool without
clashing with another local variable. Do so to reduce line length.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Now that video pool creation is handled by a dedicated function, the
logic can be simplified by returning early instead of nesting scopes. Do
so to decrease indentation and improve readability, and document the
implementation of the function with comments.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
The gst_libcamera_src_negotiate() function uses 5 indentation levels,
causing long lines. Move video pool creation to a separate function to
increase readability.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
A const_cast<> was recently added to fix a compilation issue with older
GStreamer versions. Add a comment to indicate it can be removed when
bumping the minimum GStreamer version requirement. While at it, also
document a possible future improvement in the same function, and wrap
long lines.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
So far, imx8-isi pipeline supports _symetrical_ crossbar, with same
amount of sink and source pads.
But for some other imx SoCs, such as i.MX8QM or i.MX95, crossbar is not
symetric anymore.
Since each crossbar source is already captured as a pipes_ vector entry,
we use pipes_ vector's size to compute 1st source index.
"1st source index" = "total number of crossbar pads" - pipes_.count()
Signed-off-by: Antoine Bouyer <antoine.bouyer@nxp.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
With the previous change to not drop frames in the pipeline handler,
the "disable_startup_frame_drops" pipeline config option is not used.
Remove it, and throw a warning if the option is present in the YAML
config file.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Split the pipeline handler drop frame tracking into startup frames and
invalid frames, as reported by the IPA.
Remove the drop buffer handling logic in the pipeline handler. Now all
image buffers are returned out with the appropriate FrameStatus set
for startup or invalid frames.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Replace the dropFrameCount parameter returned from ipa::start() to the
pipeline handler by startupFrameCount and invalidFrameCount. The former
counts the number of frames required for AWB/AGC to converge, and the
latter counts the number of invalid frames produced by the sensor when
starting up.
In the pipeline handler, use the sum of these 2 values to replicate the
existing dropFrameCount behaviour.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add a new status enum, FrameStartup, used to denote that even though
the frame has been successfully captured, the IQ parameters set by the
IPA will cause the frame to be unusable and applications are advised to
not consume this frame. An example of this would be on a cold-start of
the 3A algorithms, and there will be large oscillations to converge to
a stable state quickly.
Additional, update the definition of the FrameError state to cover the
usage when the sensor is known to produce a number of invalid/error
frames after stream-on.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Returning 0 when a running process is already managed can be confusing
since the parameters might be completely different, causing the caller
to mistakenly assume that the program it specified has been started.
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Firstly, get the number of arguments first, and use that to determine the
size of the allocation instead of retrieving it twice.
Secondly, use `const_cast` instead of a C-style cast when calling `execv()`.
Third, use `size_t` to match the type of `args.size()`.
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Saturation control is added on top of the colour correction matrix. A
method of saturation adjustment that can be fully integrated into the
colour correction matrix is used. The control is available only if Ccm
algorithm is enabled.
The control uses 0.0-2.0 value range, with 1.0 being unmodified
saturation, 0.0 full desaturation and 2.0 quite saturated.
The saturation is adjusted by converting to Y'CbCr colour space,
applying the saturation value on the colour axes, and converting back to
RGB. ITU-R BT.601 conversion is used to convert between the colour
spaces, for no particular reason.
The colour correction matrix is applied before gamma and the given
matrix is suitable for such a case. Alternatively, the transformation
used in libcamera rpi ccm.cpp could be used.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In the MaliC55CameraData::init() function there are two places that
return values they shouldn't; the ret variable is returned after
checking a pointer is not null instead of an explicit -ENODEV and later
the boolean value false is returned on failure instead of the error
value returned by V4L2Subdevice::open() - fix both problems.
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Device::openCard() in the cam DRM helpers looks for a /dev/dri/card*
device that can be opened and that doesn't fail when asked about
DRM_CAP_DUMB_BUFFER capability (regardless whether the capability is
supported by the device).
There can be matching devices that are not display devices. This can
lead to selection of such a device and inability to use KMS output with
the `cam' application. The ultimate goal is to display something on the
device and later the KMS sink will fail if there is no connector
attached to the device (although it can actually fail earlier, when
trying to set DRM_CLIENT_CAP_ATOMIC capability if this is not
supported).
Let's avoid selecting devices without connectors, CRTCs or encoders.
The added check makes the original check for DRM_CAP_DUMB_BUFFER API
most likely unnecessary, let's remove it.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Mattijs Korpershoek <mkorpershoek@kernel.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Even without AGC definition in the tuning file, the application would
still dereference agc unconditionally, leading to a segmentation fault
if AGC is absent.
This is relevant for sensors already providing AGC/AEC by themselves.
Check if AGC is present prior to setting maximum exposure time.
Signed-off-by: Benjamin Mugnier <benjamin.mugnier@foss.st.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Tested-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com> # RPi4 + imx708_wide
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When a child process is started from Process::start(), the file
descriptors inherited from the parent process are closed, except
the ones explicitly listed in the fds[] argument.
One issue is that the file descriptors for stdin, stdout and stderr
being closed, the subsequent file descriptors created by the child
process will reuse the values 0, 1 and 2 that are now available.
Thus, usage of printf(), assert() or alike may direct its output
to the new resource bound to one of these reused file descriptors.
The other issue is that the child process can no longer log on
the console because stderr has been closed.
To address the 2 issues, Process:start() is amended as below:
- Child process inherits from parent's stderr fd in order to share
the same logging descriptor
- Child process stdin, stdout and stderr fds are bound to /dev/null
if not inherited from parent. That is to prevent those descriptors
to be reused for any other resource, that could be corrupted by
the presence of printf(), assert() or alike.
Signed-off-by: Julien Vuillaumier <julien.vuillaumier@nxp.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
`appendPOD()` does a single insertion, so if only a single `appendPOD()`
will be called on a vector before returning, then calling `reserve()`
is not that useful, so remove it.
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Currently, modifying `controls.py` does not make those build targets dirty
that use a script that includes it (e.g. `gen-controls.py`) because meson
has no knowledge of this dependency. Add `depend_files` to each
`custom_target()` invocation to fix this.
Ideally it would be possible to attach this dependency to `gen_controls`,
`gen_gst_controls`, etc. objects themselves, so that repetition is
avoided, but this does not seem possible at the moment.
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Acked-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The current `ControlValue` mechanism does not support arrays
of strings, the assignment in the removed snippet will in fact
trigger an assertion failure in `ControlValue::set()` because
`sizeof(std::string) != ControlValueSize[ControlTypeString]`.
Fixes: b35f04b3c1 ("cam: capture_script: Support parsing array controls")
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The two overloads of `find()` and `at()` have the same behaviour
regardless of the argument type: `unsigned int` or `const ControlId *`.
However, `count()` is not so because `count(unsigned int)` only checks
the `ControlIdMap`, and it does not check if the given id is actually
present in the map storing the `ControlInfo` objects.
So `count()` returns 1 for every control id that is present in the
associated `ControlIdMap` regardless of whether there is an actual
entry for the `ControlId` associated with the given numeric id.
Fix that by simply using `find()` to determine the return value.
Fixes: 76b9923e55 ("libcamera: controls: Avoid exception in ControlInfoMap count() and find()")
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The AWB measurements are taken after the CCM. This can be seen by
enabling debug logging on AWB, disabling AWB (stats will still be
processed) and manually chaning the CCM.
This means that the estimated colour temperature and the corresponding
CCM also lead to changed rgbMeans which in turn leads to oscillations.
Fix that by applying the inverse transform on the rgbMeans.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
