A max_gain parameter is added to the config file which we pass to the
lens shading calibration. This clamps the maximum luminance gain that
gets written into the tuning files so as to prevent overflows.
It is particularly useful for lenses that cut off the light completely
from the sensor corners, and allows usable tables to be generated for
them.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Naushir Patuck <naush@raspberrypi.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The various boilerplate parts of the tuning file are extended to
include the necessary extra bits for HDR, specifically:
* rpi.denoise has different configurations for HDR modes
* rpi.agc now has extra channels for HDR
* rpi.hdr parameters are added.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Naushir Patuck <naush@raspberrypi.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Some V4L2 formats translate to the same pixel format, e.g. YU12 and
YM12 both produce YUV420. In this case our ISP driver advertises the
same size range for both, but we must not record the same thing twice
for the same pixel format (which will cause a failure later on).
Instead, ignore the V4l2 format if the pixel format has already been
seen.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When validating a stream, and no valid packed pixel format can be
found, see if an unpacked format can be used instead.
This is particularly helpful for 8 (and 16) bit raw formats, where
asking for a packed format would previously have failed. Now the
configuration will be adjusted to give you a format (in fact, the only
format) that will work.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
All the Raspberry Pi official camera tuning files are updated for
HDR. As stated previously, there is no mechanism in the hardware for
combining images so all this does is enable multi-channel AGC to
produce short and long exposure frames. It will be up to the
application to deal with them.
The changes are identical in every tuning file.
1. The existing AGC tuning is duplicated twice so that we have 3 AGC
channels.
2. The first is left alone (the default AGC channel), the second is
tweaked to under-expose significantly (ev -3) and the final one is
tweaked to over-exposure slightly (ev +0.5)
3. Control parameters are provided to the "rpi.hdr" algorithm to
associate these AGC channels correctly with the HDR modes.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Sufficient plumbing is added so that the HDR mode control can be used
to engage HDR modes on platforms that support them. On the vc4
platform, this allows multi-channel AGC to run, though there is no
image merging.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Previously the code would brighten up images in case the Macbeth Chart
is slightly dark, and also zoom in on sections of it to look for
charts occupying less of the field of view. But it would not do both
together.
This change makes the search for smaller charts also repeat that
search for the brightened up images that it made earlier, thereby
increasing the chances of success for non-optimal tuning images.
There are also a couple of very small drive-by typo fixes.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This allows the IPA to get reasonable default colour gains before AWB
has run. This is particularly important on the PiSP platform where
these numbers are helpful in programming the Front End statistics
block in advance.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This allows the IPA to discover the correct black level values even
before any frames have been processed. This is important on the PiSP
platform where the front end black level blocks must be programmed in
advance.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
We make a few small improvements to the code:
* The arrayToSet method is prevented from overwriting the end of the
array if there are too many values in the input table. If you supply
a table, it will force you to put the correct number of elements in
it.
* The arrayToSet and setStrength member functions are turned into
static functions. (There may be a different public setStrength
member function in future.)
* When no tables at all are given, the configuration is flagged as
being disabled, so that we can avoid copying tables full of zeroes
around. As a consequence, the pipeline handler too will disable this
hardware block rather than run it needlessly. (Note that the tuning
tool will put in a completely empty "rpi.cac" block if no CAC tuning
images are supplied, benefiting from this behaviour.)
* The initialise member function is removed as it does nothing.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The recent change where time-filtering is done before sorting out the
digital gain means that the target exposure without digital gain is no
longer set, breaking the 'AeLocked' calculation.
We can use the regular (full) target exposure instead.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Fixes: 84b6327789 ("ipa: rpi: agc: Filter exposures before dealing with digital gain")
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The algorithm computes R/G and B/G colour ratio statistics which we
should not allow to go to zero because there is clearly no gain you
could apply to R or B to equalise them. Instead flag such regions as
having "insufficient data" in the normal manner.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
We add an HdrMode control (to enable and disable HDR processing)
and an HdrChannel, which indicates what kind of HDR frame (short, long
or medium) has just arrived.
Currently the HdrMode supports the following values:
* Off - no HDR processing at all.
* MultiExposureUnmerged - frames at multiple different exposures are
produced, but not merged together. They are returned "as is".
* MultiExposure - frames at multiple different exposures are merged
to create HDR images.
* SingleExposure - multiple frames all at the same exposure are
merged to create HDR images.
* Night - multiple frames will be combined to create "night mode"
images.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
We need to be able to do things like enable/disable AGC for all the
channels, so most of the AGC controls are updated to be applied to all
channels. There are a couple of exceptions, such as setting explicit
shutter/gain values, which apply only to channel 0.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
AWB writes this out during prepare, so we may as well read it in AGC
prepare as well. Reading it in process is wrong on the PiSP platform
because process runs before prepare, so the AWB status won't be there
(on vc4 it made no difference).
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
"Fast desaturation" is a technique that can help the AGC algorithm to
desaturate images more quickly when they are very
over-exposed. However, it uses digital gain to do this which can
confuse our HDR techniques. Therefore make it optional.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This was being re-read in order to determine what LSC gains had been
applied. We can just retrieve these numbers from the prevAsyncResults_
instead.
This will also enable other future algorithms to manipulate the LSC
tables in the alsc.status, without it breaking the core ALSC algorithm
here.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
We can perform some of the local contrast adjustment using global
gains in the LSC table. We can vary the amount of gain according to
the measured brightness of that image region.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The code was inadvertently overwriting the caller's StatisticsPtr,
meaning that subsequent algorithms would get the wrong image
statistics when AGC channels changed.
This could be fix using std::ref, though I find the C-style pointer
fix easier to understand!
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Some use cases may require stronger, or different, denosie settings to
others. For example, the way frames are accumulated during single
exposure HDR means that we may want stronger denoise.
This commit adds such support for different configurations that can be
defined in the tuning file.
Older tuning files, or files where there is only a single
configuration, load only the "normal" denoise configuration.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The enableCe() function enables or disables adaptive contrast
enhancement and the restoreCe() function sets it back to its normal
state (which is what was read from the tuning file).
In future, algorithms like HDR might want to take over tonemapping
functions, so any dynamic behaviour here would upset them.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Add a small "stable region" parameter (defaulting to 2%) within which
the AGC will not adjust the exposure it requests. It allows
applications to configure the AGC to avoid continual micro-adjustments
of exposure values if they are somehow sensitive to it.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
We avoid skipping the IPAs while frameCount_ is less than
dropFrameCount_, indicating that these frames will not be sent to the
application. This means that when these numbers are equal then this is
the first frame the application will get, so again, we must avoid
skipping the IPAs. Consequently the test here must avoid the case of
equality.
Fixes: 51533fecae ("ipa: rpi: Fix frame count logic when running algorithms")
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Whenever we run Agc::process(), we store the most recent total
exposure requested for each channel.
With these values we can apply the channel constraints after
time-filtering the requested total exposure, but before working out
how much digital gain is needed.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
A channel constraint is somewhat similar to the upper/lower bound
constraints that we use elsewhere, but these constraints apply between
multiple AGC channels. For example, it lets you say things like "don't
let the channel 1 total exposure be more than 8x that of channel 0",
and so on. By using both an upper and lower bound constraint, you
could fix one AGC channel always to be a fixed ratio of another.
Also read a vector of them (if present) when loading the tuning file.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
The switchMode, prepare and process methods are updated to implement
multi-channel AGC correctly:
* switchMode now invokes switchMode on all the channels (whether
active or not).
* prepare must find what channel the current frame is, and run on
behalf of that channel.
* process updates the most recent DeviceStatus and statistics for the
channel of the frame that has just arrived, but generates updated
values working through the active channels in round-robin fashion.
One minor detail in process is that we don't want to change the
DeviceStatus metadata of the current frame, so we now pass this to the
AgcChannel's process method, rather than letting it find the
DeviceStatus in the metadata.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
This commit does the basic reorganisation of the code in order to
implement multi-channel AGC. The main changes are:
* The previous Agc class (in agc.cpp) has become the AgcChannel class
in (agc_channel.cpp).
* A new Agc class is introduced which is a wrapper round a number of
AgcChannels.
* The basic plumbing from ipa_base.cpp to Agc is updated to include a
channel number. All the existing controls are hardwired to talk
directly to channel 0.
There are a couple of limitations which we expect to apply to
multi-channel AGC. We're not allowing different frame durations to be
applied to the channels, nor are we allowing separate metering
modes. To be fair, supporting these things is not impossible, but
there are reasons why it may be tricky so they remain "TBD" for now.
This patch only includes the basic reorganisation and plumbing. It
does not yet update the important methods (switchMode, prepare and
process) to implement multi-channel AGC properly. This will appear in
a subsequent commit. For now, these functions are hard-coded just to
use channel 0, thereby preserving the existing behaviour.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
The Agc::process() function returns an AgcStatus object in the
metadata as before, but Agc::prepare() is changed to return the values
it computes in a separate AgcPrepareStatus object (under the new tag
"agc.prepare_status").
The "digitalGain" and "locked" fields are moved from AgcStatus to
AgcPrepareStatus.
This will be useful going forward as we can be more flexible about the
order in which prepare() and process() are called, without them
trampling on each other's results.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
We now time-filter the exposure before sorting out how much digital
gain is required. This is actually a little more natural and
simplifies the code. It also prepares us for some future work where
this arrangement will be helpful.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrpyi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
prepare() doesn't use the AWB status, so fetching it in process() is
probably better. This change is preparatory to other changes, where we
may find ourselves calling process() without having called prepare()
previously.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
We handle the flicker modes by passing the correct period to the
AEC/AGC algorithm which already contains the necessary code.
The "Auto" mode, as well as reporting the detected flicker period via
the "AeFlickerDetected" metadata, are unsupported for now.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Flicker is the term used to describe brightness banding or oscillation
of images caused typically by artificial lighting driven by a 50 or
60Hz mains supply. We add three controls intended to be used by
AEC/AGC algorithms:
AeFlickerMode to enable flicker avoidance.
AeFlickerPeriod to set the flicker period "manually".
AeFlickerDetected to report any flicker that is currently detected.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
The algorithm uses the data type std::vector<std::array<double, 4>> to
represent the large sparse matrices that are XY (X, Y being the ALSC
grid size) high but with only 4 non-zero elements on each row.
Replace this slightly long type name by SparseArray<double>.
No functional changes.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The Array2D class is a very thin wrapper round std::vector that can be
used almost identically in the code, but it carries its 2D size with
it so that we aren't passing it around all the time.
All the std::vectors that were X * Y in size (X and Y being the ALSC
grid size) have been replaced. The sparse matrices that are XY * 4 in
size have not been as they are somewhat different, are used
differently, require more code changes, and actually make things more
confusing if everything looks like an Array2D but are not the same.
There should be no change in algorithm behaviour at all.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Some updates to the tuning for the imx296 sensors.
For the colour variant:
* Minor change to the AWB curve, making things a little less green.
* Updated CCMs that reduce colour saturation to a more accurate level.
Thanks to Dr. Rolf Henkel for these measurements and calculations.
* Sharpening has been toned down quite a lot.
* rpi.focus algorithm added so that the focus measure can be accessed.
The sharpening and focus changes are applied to the mono version of
the sensor too as we expect similar characteristics.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The maxVal variable in the computeInitialY function needs to be a
uint64_t, otherwise the subsequent multiplications in the function
can overflow on relatively high resolution images (when the counts in
the regions go over 16 bits).
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Commit 613d540267 ("pipeline: raspberrypi: Fix handling of colour
spaces") adjusts the colorspace to ColorSpace::Raw for raw streams.
However, if the colorspace is not requested for raw streams(nullopt),
we should still set the colorspace to ColorSpace::Raw, for raw streams.
Fixes: 613d540267 ("pipeline: raspberrypi: Fix handling of colour spaces")
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
We implement a custom validateColorSpaces method that forces all
(non-raw) streams to same colour space, whilst distinguishing RGB
streams from YUV ones, as the former must have the YCbCr encoding and
range over-written.
When we apply the colour space, we always send the full YUV version as
that gets converted correctly to what our hardware drivers expect. It
is also careful to check what comes back as the YCbCr information gets
overwritten again on the way back.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Previously the x,y offsets in the min/max ScalerCrop control values
were zero. Here we make them the same as the sensor's analogue crop
offset which is I think less misleading.
With this change, it also seems reasonable to advertise the default
scaler crop value to be the true default that you will get. This makes
it possible for applications to see what that value will be without
having to start the camera and wait for frames.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The intention is that the "main" colour space is the colour space of
the largest non-raw stream. Unfortunately the use of "config_[i].size"
is clearly incorrect, and has been copied from prior versions of the
code. This patch corrects the error.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
The Python bindings will now accept, or return, a list or tuple of
libcamera.Rectangle objects for such controls.
This had previously been omitted, but now we have, for example, the
AfWindows control which requires this feature.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The latest camera modules have a very slightly different IR filter, so
the tuning file is slightly revised to give best results with both old
and new camera modules.
The original tuning file is retained as imx477_v1.json in case anyone
should wish to continue using it.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This reverts commit 30d704732b.
It turns out that this commit causes some regressions and is in fact
unnecessary because the related commit "libcamera: v4l2_videodevice:
Guard against releasing unallocated buffers"
(a2bdff6d0b) fixes the problem
completely (if the buffers were never allocated, the video device
avoids trying to free them even if the pipeline handler asks).
The reason for the regressions is that in this new (broken) scheme we
would never call clearBuffers() on all the streams if the internal
buffers were never allocated (i.e. buffersAllocated_ is never
set). This causes the stream's bufferMap_ list to get longer and
longer if there are multiple back-to-back calls to configure, and
dev_->importBuffers() will ultimately to fail.
So either we need to think more carefully about how to stop the
pipeline handler from freeing buffers that it doesn't own, or we just
leave it as the other commit resolves the problem on its own. In the
interim, simply reverting this commit certainly seems like the best
solution.
Fixes: 30d704732b ("pipeline: raspberrypi: Do not unconditionally free buffers on close")
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Previously we only did this when the system starts (on the first
switch_mode). Now we do it whenever the manual colour gains are
updated. To facilitate this, this R/B vs. colour temperature inverse
functions are stored persistently in the AwbConfig.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The "R" pixel formats (R8, R10, R10_CSI2P etc.) record the associated
colour space as being YUV rather than RAW, meaning that the code was
not detecting them as being raw formats.
In the case of Raspberry Pi, we deal only with raw formats, so the
revised test must work correctly for both these and the standard Bayer
formats.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
This patch describes a series of controls that allow applications to
drive AF algorithms:
AfMode - manual, auto or continuous
AfRange - full, macro or normal
AfSpeed - fast or slow
AfMetering - how to choose where to measure focus
AfWindows - AF window locations
AfTrigger - start (trigger) an AF scan or cancel
AfPause - pause continuous AF
LensPosition - set or retrieve position of lens
AfState - reports whether scanning/success/failure
AfPauseState - reports whether continuous AF paused or not
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
These changes retrieve the correct value for sensitivity of the mode
selected for the sensor. This value is known to the CamHelper which
passes it across to the pipeline handler so that it can be set
correctly in the camera properties.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
We must calculate the initial scaler crop when the camera is
configured, otherwise the metadata will report this rectangle as being
all zeroes.
Because the calculation is identical to that performed later in handling
the scaler crop control, we factor it into a small helper function,
RPiCameraData::scaleIspCrop.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
We now handle disabling ("pausing") AWB in the same way as
AEC/AGC. Instead of letting the pause flag be set so that the code
never runs at all, we instead fix the manual settings to the current
values (but continue to be called).
The algorithm does not restart any calculations in this state, but
continues to add AWB metadata to every frame. Therefore certain other
algorithms that want to know it (CCM and ALSC, for example) can still
find it.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Now that the sensor is defined to list mbus codes using its native
(untransformed) Bayer order, the method of obtaining it can be
simplified. We don't have to try and reset the flips, we just use the
value directly from the sensor.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
We clear the V4L2_CID_HFLIP and V4L2_CID_VFLIP controls immediately
after opening the camera device. This means the camera's Bayer format
and mbus codes will be in the sensor's "native" order, and we document
this to be the case so that it can be relied upon.
Clearing the flips is harmless where sensor flips do not affect the
Bayer order.
This also fixes a bug in the Raspberry Pi pipeline handler where the
native Bayer order was being computed wrongly, but the new behaviour
here will be helpful to other pipeline handlers too. A subsequent
commit will tidy up the Raspberry Pi pipeline handler in this area as
it can now be simplified.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Fixes: 83a5128161 (pipeline: raspberrypi: Convert the pipeline handler to use media controller)
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
If the sensor exposes the V4L2_CID_NOTIFY_GAINS control, assume it
means the sensor wants to be told the latest colour gains.
We store whether the control exists and if so its default value, to
save us checking for it on every frame.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The Raspberry Pi pipeline handler now sets color spaces correctly.
In generateConfiguration() it sets them to reasonable default values
based on the stream role.
validate() now calls validateColorSpaces() to ensure that the
requested color spaces are sensible, before proceeding to check what
the hardware can deliver.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
This function forces raw streams to have the "raw" color space, and
also optionally makes all non-raw output streams to share the same
color space as some platforms may require this.
When sharing color spaces we take the shared value to be the one from
the largest of these streams. This choice is ultimately arbitrary, but
can be appropriate if smaller output streams are used for image
analysis rather than human consumption, when the precise colours may
be less important.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
This adds a ColorSpace field to the V4L2SubdeviceFormat so that we can
set and request particular color spaces from V4L2.
This commit simply adds the field and fixes some occurrences of brace
initializers that would otherwise be broken. A subsequent commit will
pass and retrieve the value correctly to/from V4l2 itself.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Add functions to the V4L2Device class to convert to and from
libcamera ColorSpace.
These functions are added to the base V4L2Device class so that they can
be shared both by the video device class and subdevices.
With the ColorSpace class, the color space and related other fields
are stored together, corresponding to a number of fields in the
various different V4L2 format structures. Template functions are
therefore a convenient implementation, and we must explicitly
instantiate the templates that will be needed.
Note that unset color spaces are converted to requests for the
device's "default" color space.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The code was reducing the number of raw stream buffers allocated when
the application is providing some of its own. However, it was not
taking account of the fact that the application cannot supply embedded
data buffers, so it must always allocate a reasonable minimum number
of these buffers (possibly more than the number of raw stream buffers)
to prevent frame drops.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When the pipeline handler start() method is supplied with a NULL list
of controls, we send an empty control list to the IPA. When the IPA is
running in isolated mode the control list goes through the data
serializer, for which it must be marked correctly as a list of
"controls::controls", otherwise the IPA process will abort.
The IPA has a similar problem returning a control list in its
configure() method. We must be careful to initialise it properly even
when empty.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The cause is that we read out delayed values using a frame's sequence
number (DelayedControls::get). But we fill the values up
(DelayedControls::applyControls) incrementing writeCount by only one
even if the sequence number has jumped by several since last
time. This is exactly what happens when frames are being dropped.
So the fix is to increment writeCount by "as much as the sequence
number has jumped since last time", which means that we just follow
the sequence number directly.
Bug: https://bugs.libcamera.org/show_bug.cgi?id=74
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
"NOIR" modules are ones that have had the IR filters removed but are
otherwise identical. The same tuning can be used as for the regular
version except that the colour calibration supplied to the AWB
algorithm no longer works. Instead we need to switch the algorithm to
its basic "grey world" method.
Users with "NOIR" modules can switch to the matching "xxx_noir.json"
tuning file by using the LIBCAMERA_RPI_TUNING_FILE environment
variable.
Signed-off-by: David Plowman <david.plowman@raspberypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Starting in version 0.22, the NearestCentroid function is only available
in the sklearn.neighbors namespace, when it was previously available in
both the sklearn.neighbors.nearest_centroid and sklearn.neighbors
namespaces. Use sklearn.neighbors as it works on all versions of
sklearn.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
This commit adds a tuning file for the 12MP imx378 sensor. The sensor
actually shares the same driver (and CamHelper) as the imx477 so only
a new tuning file is required. The default choice of imx477.json can
be overridden by pointing LIBCAMERA_RPI_TUNING_FILE at a version of
the new imx378.json file.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
This commit adds support for monochrome (greyscale) raw sensors. These
are sensors that have no colour filter array, so all pixels are the
same and there are no distinct colour channels.
These sensors still require many of an ISP's processing stages, such
as denoise, tone mapping, but not those that involve colours (such as
demosaic, or colour matrices).
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The minimum and maximum vblanking can change when a new format is
applied to the sensor subdevice, so be sure to retrieve up-to-date
values.
The V4L2Device acquires the new updateControlInfo() method to perform
this function, and which the CameraSensor calls automatically if its
setFormat method is used to update the sensor.
However, not all pipeline handlers invoke the setFormat method
directly, so the new method must be made publicly available for
pipeline handlers to call if they need to.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
CamHelpers get virtual Prepare() and Process() methods, running just
before and just after the ISP, just like Raspberry Pi Algorithms.
The Prepare() method is able to parse the register dumps in embedded
data buffers, and can be specialised to perform custom processing when
necessary.
The IPA itself only needs to call the new Prepare() and Process()
methods. It must fill in the DeviceStatus from the controls first, in
case the Prepare() method does not supply its own values.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
This patch adds support for the Soho Enterprises SE327M12 module. The
sensor is an imx327 which therefore uses the imx290 kernel driver and
CamHelper.
To use this module and camera tuning, place the following in the
/boot/config.txt file:
dtoverlay=imx290,clock-frequency=37125000
and then the existing imx290.json file must be over-written with
se327m12.json
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
imx290 and imx327 share the same kernel driver (imx290.c) and are
therefore both recognised here as "imx290". We add the necessary
CamHelper for these sensors, as well as a camera tuning file.
The tuning was done with an Innomaker STARVIS IMX327LQR module. These
have no IR cut filter so there is no proper colour tuning. However,
you should obtain reasonable results for most modules using this
sensor. Specific tunings for further modules can always be added
subsequently.
To use this sensor on the Raspberry Pi platform, please add
dtoverlay=imx290,clock-frequency=74250000
into your /boot/config.txt file.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
For some sensors (e.g. imx477) we need to update the vblanking on the
frame before the exposure. For this reason the GetDelays method must
also return the number of frame delays for the vblanking control.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Regions of the image where the statistics are not trusted should not
be put in the zones_ list.
This also means that the AWB may fail to run initially, so
async_results_ needs to be initialised to some kind of default.
Finally, the RGB zone structures can be zero-initialised safely now,
the previous value of -1 would seem unusual.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
