Files
external_libcamera/test/ipa
Stefan Klug 8936e81e3f libipa: histogram: Fix quantile() calculation for fractional results
The calculation of the frac variable is based solely on integers and
therefore results in the fractional part being either 0 or 1.

In the original code from RaspberryPi this is mitigated by casting the
nominator to a double. This works for most cases, but fails when q is
very small because of the quantization introduced by item being an
integer.

Fix both issues by doing the full calculation in double and remove the
should_fail tag.

Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2025-04-03 12:36:07 +02:00
..