pipeline: raspberrypi: Fix rounding issue in findBestFormat()

The aspect ratio calculation divides two integer values then casts to a double.
This might reduce precision when scoring for aspect rato differences.

Fix this by casting the integer to a double before the division.

Reported-by: Coverity CID=361652
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This commit is contained in:
Naushir Patuck
2021-11-02 12:46:57 +00:00
committed by Kieran Bingham
parent 5004d8a969
commit 06008c6e81
@@ -141,7 +141,7 @@ V4L2SubdeviceFormat findBestFormat(const SensorFormats &formatsMap, const Size &
for (const Size &size : iter.second) {
double reqAr = static_cast<double>(req.width) / req.height;
double fmtAr = size.width / size.height;
double fmtAr = static_cast<double>(size.width) / size.height;
/* Score the dimensions for closeness. */
score = scoreFormat(req.width, size.width);