The GPU processing supports 8-bit sensor formats and 10/12-bit packed
formats. Support for 10/12-bit unpacked formats is missing, let's add
it.
10/12-bit unpacked formats use two adjacent bytes to store the value.
This means the 8-bit shaders can be used if we can modify them for
additional support of 16-bit addressing. This requires the following
modifications:
- Using GL_RG (two bytes per pixel) instead of GL_LUMINANCE (one byte
per pixel) as the texture format for the given input formats.
- Setting the texture width to the number of pixels rather than the
number of bytes.
- Making the definition of `fetch' macro variable, according to the
pixel format.
- Using only `fetch' for accessing the texture.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Hans de Goede <johannes.goede@oss.qualcomm.com>
Tested-by: Robert Mader <robert.mader@collabora.com>
Tested-by: Hans de Goede <johannes.goede@oss.qualcomm.com> # ThinkPad T14s gen 6 (arm64) ov02c10 + X1c gen 12 ov08x40
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com> # Lenovo X13s
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Make getInputConfig and getOutputConfig static so as to allow for
interrogation of the supported pixel formats prior to object instantiation.
Do this so as to allow the higher level logic make an informed choice
between CPU and GPU ISP based on which pixel formats are supported.
Currently CPU ISP supports more diverse input and output schemes.
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Robert Mader <robert.mader@collabora.com>
Tested-by: Hans de Goede <johannes.goede@oss.qualcomm.com> # ThinkPad T14s gen 6 (arm64) ov02c10 + X1c gen 12 ov08x40
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com> # Lenovo X13s
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In order to have Debayer::start() tell the eGL shader compilation routine what
the input and output pixel format is, we need to have a copy of the
selected format available. Add variables to the inputConfig and
outputConfig structures to allow tracking of this data for later use.
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Robert Mader <robert.mader@collabora.com>
Tested-by: Hans de Goede <johannes.goede@oss.qualcomm.com> # ThinkPad T14s gen 6 (arm64) ov02c10 + X1c gen 12 ov08x40
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com> # Lenovo X13s
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When bayer_unpacked.vert is calculating the center and x/yCoord values
stride != width is taken into account for x/yCoord deltas since it is taken
into account by debayer_egl when setting the x part of tex_step uniform.
But it is not taken into account for the center.x which is just directly
copied from textureIn, leading to the input width sampling covering
the entire input stride instead of just covering the input width.
Use the existing and currently unused stride_factor uniform to pass
the width/stride ratio and correct center.x for this. This fixes
the misrendering seen on x86 laptops which is caused by the CSI2 receiver
there requiring a stride which is a multiple of 32 often leading to
stride != width.
Signed-off-by: Hans de Goede <johannes.goede@oss.qualcomm.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Robert Mader <robert.mader@collabora.com>
Tested-by: Hans de Goede <johannes.goede@oss.qualcomm.com> # ThinkPad T14s gen 6 (arm64) ov02c10 + X1c gen 12 ov08x40
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com> # Lenovo X13s
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Populate qcam viewfinder_gl to set default Bayer values so that the
shaders can be used in their original mode without conditional compilation.
Set an identity CCM, identity Black Level and set Gamma and Contrast to
1.0f respectively.
Once this change is made we can use the Bayer shaders in their original
format in qcam with raw streams.
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Tested-by: Robert Mader <robert.mader@collabora.com>
Tested-by: Hans de Goede <johannes.goede@oss.qualcomm.com> # ThinkPad T14s gen 6 (arm64) ov02c10 + X1c gen 12 ov08x40
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com> # Lenovo X13s
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Introduce an eGL base helper class which provides an eGL context based on a
passed width and height.
The initGLContext function could be overloaded to provide an interface to a
real display.
A set of helper functions is provided to compile and link GLSL shaders.
linkShaderProgram currently compiles vertex/fragment pairs but could be
overloaded or passed a parameter to link a compute shader instead.
Breaking the eGL interface away from debayering - allows to use the eGL
context inside of a dma-buf heap cleanly, reuse that context inside of a
debayer layer and conceivably reuse the context in a multi-stage shader
pass.
Small note the image_attrs[] array doesn't pass checkstyle.py however the
elements of the array are in pairs.
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
[bod: Takes fix from Hans for constructor stride bpp]
[bod: Drops eglClientWaitSync in favour of glFinish Robert/Milan]
Co-developed-by: Hans de Goede <johannes.goede@oss.qualcomm.com>
Signed-off-by: Hans de Goede <johannes.goede@oss.qualcomm.com>
Co-developed-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Robert Mader <robert.mader@collabora.com>
Tested-by: Robert Mader <robert.mader@collabora.com>
Tested-by: Hans de Goede <johannes.goede@oss.qualcomm.com> # ThinkPad T14s gen 6 (arm64) ov02c10 + X1c gen 12 ov08x40
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com> # Lenovo X13s
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Pass contrastExp as calculated in lut to debayer params not the raw
contrast. This way we calculate contrastExp once per frame in lut and pass
the calculated value into the shaders, instead of passing contrast and
calculating contrastExp once per pixel in the shaders.
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Tested-by: Robert Mader <robert.mader@collabora.com>
Tested-by: Hans de Goede <johannes.goede@oss.qualcomm.com> # ThinkPad T14s gen 6 (arm64) ov02c10 + X1c gen 12 ov08x40
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com> # Lenovo X13s
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Replace the `LIBCAMERA_NO_LOG_COLOR` env variable with another environment
variable that recognizes the "auto", "yes", "no" values. When set to "auto",
the messages are only colored if the standard error is a tty (as determined
by `isatty()`).
"auto" is the default value. This ensures that the ansi escape codes won't
litter the output if the stderr is redirected to a file, `tee`, etc.
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
In order to clear the V4L2VideoDevice cache, we must call
V4L2VideoDevice::releaseBuffers() when stopping the camera. To do this
without releasing the allocated buffers, we have to rework the internal
buffer allocation logic.
Remove calls to V4L2VideoDevice::importBuffers() and releaseBuffers()
from within the RPi::Stream class. Instead, move these calls to the
PipelineHandlerBase class, where we can call releaseBuffers(), i.e. clear
the V4L2VideoDevice cache unconditionally on stop without de-allocating
the internal buffers.
The code in Stream::clearBuffers() can be then moved into releaseBuffers()
which will actually de-allocate the internal buffers when required.
Closes: https://gitlab.freedesktop.org/camera/libcamera/-/issues/265
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com> # rpi4
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
If the first frame of a stream is bad, the IPA will not get called with
frame == 0, leaving activeState.agc expo/again uninitialized. This causes
the agc algorithm to set a very low gain and exposure on the next run
(where it will hit the if (!stats->valid) {} path) resulting in starting
with a black image.
Fix this by using a valid flag instead of checking for frame == 0.
The entire activeState gets cleared to 0 on configure() resetting the new
valid flag.
Signed-off-by: Hans de Goede <johannes.goede@oss.qualcomm.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
On most imx sensors the formula seems to be:
again_float = 512.0 / (512.0 - again_ctrl_value)
So the minimum again of 0 makes sense and actually translates to a gain of 1.0.
And the max gain of 400 leads to 512.0 / 112.0 = 4.57 which is about typical for
a maximum again. Since a minimum again value of 0 is actually normal, the special
handling of againMin == 0 is undesirable and this is actually causing problems
on these IMX sensors. again10 correctly gets set to 0 which is less than the
adjusted againMin which causes the AGC code to not work properly.
Fix this by dropping the special handling of againMin == 0.
Signed-off-by: Vasiliy Doylov <nekocwd@mainlining.org>
Reviewed-by: Hans de Goede <johannes.goede@oss.qualcomm.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Save the result of `Camera::generateConfiguration()` in a local variable,
and only commit it to the member variable `config_` if it passes all checks.
This removes the need for explicit `config_.reset()` calls in error paths,
hence making things simpler.
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
The ControlList associated with a Request are created with
the libcamera::controls::controls id map.
The cam application, when parsing controls from a script
assigns to the Request's control list a newly created one, which is
initialized without a valid idmap or info map.
This causes IPA that run in isolated mode to fail, as the IPC
serialization/deserialization code requires all ControlList to have
a valid idmap or info map associated.
Instead of overwriting the Request::controls() list, merge it with
the one created by cam when parsing the capture script.
Closes: https://gitlab.freedesktop.org/camera/libcamera/-/issues/295
Fixes: 568865a6c1 ("cam: Use script parser to set controls")
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The list of controls part of a Request is initialized with the
ControlInfoMap of the Camera the Request is created from.
Applications can re-assign the controls list in a Request
which could cause issues during serialization.
Validate that the ControlList in a Request is valid when
the Request is queued to the Camera by inspecting the
ControlInfoMap pointer.
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Address a long standing \todo item that suggested to implement a
read-only interface for the Request::metadata() accessor and deflect to
the internal implementation for the read-write accessor used by pipeline
handlers.
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Acked-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
The control lists associated with a Request is created with the global
libcamera::controls::controls id map.
This is fine as the idmap/info map used to contruct a ControlList are not
used for any control validation purposes by the libcamera code.
However creating a ControlList with the camera ControlInfoMap has two
advantages:
1) The idmap can be extracted from the info map, but not the other way
around
2) The control list is constructed with a valid map of info to the
controls it can actually supports, instead of the global map of
libcamera controls
Initialize the ControlList part of a Request with the Camera control
info map, as the association between a Request and a Camera is permanent
and valid for the whole lifetime of a Request.
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
The existing RkISP1Params helper class allows the RkISP1 IPA to handle
both the extensible parameters format and the legacy fixed-size format.
With the introduction of v4l2-isp.h in the Linux kernel the part of
the RkISP1Params helper class that handles extensible parameters can
be generalized so that other IPA modules can use the same helpers
to populate a parameters buffer compatible with v4l2-isp.h.
Generalize the RkISP1Params class to a new libipa component named
V4L2Params and derive the existing RkISP1Params from it, leaving
in the RkISP1-specific implementation the handling of the legacy format.
Deriving RkISP1Params from V4L2Params requires changing the size
associated to each block to include the size of v4l2_params_block_header
in the ipa:rkisp1::kBlockTypeInfo map as the V4L2Params::block()
implementation doesn't account for that as RkIS1Params::block()
implementation did.
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Tested-by: Antoine Bouyer <antoine.bouyer@nxp.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
It is possible that same device is processed multiple times, leading to
multiple `MediaDevice`s being instantiated, mostly likely leading to
a fatal error:
Trying to register a camera with a duplicated ID xyzw...
There is a time window after the `udev_monitor` has been created in `init()`
and the first (and only) enumeration done in `enumerate()`. If e.g. a UVC
camera is connected in this time frame, then it is possible that it will be
processed both by the `udevNotify()` and the initial `enumerate()`, leading
to the fatal error. This can be reproduced as follows:
1. $ gdb --args cam -m
2. (gdb) break libcamera::DeviceEnumeratorUdev::enumerate
3. (gdb) run
4. when the breakpoint is hit, connect a usb camera
5. (gdb) continue
6. observe fatal error
To address this, keep track of the devnums of all devices reported by
udev, and reject devices with already known devnums. This ensures that
the same device won't be reported multiple times (assuming that udev
reports "add" / "remove" events in the correct order).
Closes: https://gitlab.freedesktop.org/camera/libcamera/-/issues/293
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
The AWB gains reported in the metadata are currently scaled against the
maximum gain, but in fact the gain itself is already stored as a
floating point gain value.
Remove the incorrect scaling and report the gains directly.
Fixes: a0b97475b1 ("ipa: simple: Report the ColourGains in metadata")
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Meson already takes care of passing the proper absolute or relative
paths to commands. There is no need do more path manipulation.
So simplify the script by using the paths as-is. This also fixes the
path manipulation issue that prevented libcamera from building as a
subproject.
Fixes: 19371dee41 ("utils: gen-shader-headers: Add a utility to generate headers from shaders")
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The current definition of the stream-role property of libcamerasrc
element does not allow for the Raw (0) value to be assigned.
That property is used to query the pipeline handlers through
generateConfiguration(roles) to retrieve the formats exposed to
the user. Not being able to specify the raw stream-role, bayer
formats are not reported so can not be used for caps negotiation.
Adding value stream-role=raw enables usage of bayer GStreamer pipelines
like:
`gst-launch-1.0 libcamerasrc camera-name=<name> src::stream-role=raw !
video/x-bayer, format=bggr16le ! ...`
Signed-off-by: Julien Vuillaumier <julien.vuillaumier@nxp.com>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When a raw stream is requested, whether alone or together with a
processed stream, its buffers must be handled outside the software ISP
machinery. They serve as output buffers, even when a processed stream
is produced. But when both processed and raw streams are requested, the
buffer can be completed only after the processing is finished, to make
sure it's untouched by the application as long as the processing runs.
At most one raw stream and at most one processed stream are supported
and can be combined. An example of producing both raw and processed
files using `cam' application:
cam -c1 -C100 -Ffile# \
-s role=viewfinder,width=1920,height=1080,pixelformat=RGB888 \
-s role=raw,width=3280,height=2464,pixelformat=SRGGB8
Note the difference in viewfinder and raw stream sizes due to the fact
that debayering requires enlarging the image width, which enforces
selecting a larger sensor resolution in this case.
In order to track whether a raw stream is requested and which one it is,
SimpleCameraData::rawStream_ member variable is introduced.
This is the final step to make raw streams working.
Reviewed-by: Umang Jain <uajain@igalia.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
If software ISP is enabled then metadata is required in the simple
pipeline. But this doesn't apply if the software ISP is not actually
used, for example when only a raw stream is produced. Then the pipeline
waits for metadata that never comes.
This patch fixes the problem by requiring metadata only when software
ISP is used.
Reviewed-by: Umang Jain <uajain@igalia.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When software ISP is enabled, we want to be able to provide a raw stream
in addition to the processed stream. For this purpose, we need two
streams. If only the processed stream is requested, it doesn't harm to
allocate two.
The number of streams is determined as a camera property in the pipeline
matching. To be able to produce both raw and processed output, two
streams must be provided. The actual number of streams needed (one or
two) is determined only in SimplePipelineHandler::validate().
In theory, software ISP could produce multiple processed streams but
this is out of scope of this patch series. Hence two streams are
sufficient at the moment.
When software ISP is not enabled, the camera won't be able to produce
multiple streams (assuming there's no hardware converter) and only
single stream should be allocated as before. The simple pipeline
handler assumes there's a linear pipeline from the camera sensor to a
video capture device, and only supports a single stream. Branches in
the hardware pipeline that would allow capturing multiple streams from
the same camera sensor are not supported. We have no plan to change
that, as a device that can produce multiple streams will likely be
better supported by a dedicated pipeline handler.
Reviewed-by: Umang Jain <uajain@igalia.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When a raw stream is requested, either alone or together with a
processed stream, it can be produced without conversion. Let's amend
the corresponding check on the number of configurations, so that the
mere presence of a raw stream doesn't enforce conversion.
Reviewed-by: Umang Jain <uajain@igalia.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
SimpleCameraConfiguration::validate() looks for the best configuration.
As part of enabling raw stream support, the method must consider raw
streams in addition to the processed streams.
Raw streams are adjusted from the capture format and size.
Configuration validation computes the maximum size of all the requested
streams and compares it to the output sizes. When e.g. only a raw
stream is requested then this may result in an invalid adjustment of its
size. This is because the output sizes are computed for processed
streams and may be smaller than capture sizes. If a raw stream with the
capture size is requested, it may then be wrongly adjusted to a larger
size because the output sizes, which are irrelevant for raw streams
anyway, are smaller than the requested capture size. The problem is
resolved by tracking raw and processed streams maximum sizes separately
and comparing raw stream sizes against capture rather than output sizes.
Note that with both processed and raw streams, the requested sizes must
be mutually matching, including resizing due to debayer requirements.
For example, the following `cam' setup is valid for imx219
cam -s role=viewfinder,width=1920,height=1080 \
-s role=raw,width=3280,height=2464
rather than
cam -s role=viewfinder,width=1920,height=1080 \
-s role=raw,width=1920,height=1080
due to the resolution of 1924x1080 actually selected for debayering to
1920x1080. If the resolutions don't match mutually or don't match the
available sizes, validation adjusts them.
Setting up the right configurations is still not enough to make the raw
streams working. Buffer handling must be changed in the simple
pipeline, which is addressed in followup patches.
Co-developed-by: Umang Jain <uajain@igalia.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Umang Jain <uajain@igalia.com>
Signed-off-by: Umang Jain <uajain@igalia.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Let's handle both processed and/or raw output configurations. In
addition to the already handled processed formats and sizes, this patch
adds handling of raw formats and sizes, which correspond to the capture
formats and sizes.
When creating stream configurations, raw or processed formats are
selected according to the requested stream roles.
This is another preparatory patch without making raw outputs working.
Reviewed-by: Umang Jain <uajain@igalia.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
In order to support raw streams, we need to add raw formats to software
ISP configurations. In this preparatory patch, the raw formats are
excluded from output configurations for conversions.
Reviewed-by: Umang Jain <uajain@igalia.com>
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Milan Zamazal
Bryan O'Donoghue
Hans de Goede
Barnabás Pőcze
Naushir Patuck
Vasiliy Doylov
Jacopo Mondi
Michael Olbrich
Kieran Bingham
Julien Vuillaumier