PawletOS/external_libcamera
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

ipa: libipa: fixedpoint: Fix unsigned usage

Browse Source 
The fixedToFloatingPoint does not support unsigned Q types, and
incorrectly sign-extends all values which have the top most bit set in
the quantized values.

Fix this by ensuring that only signed types perform sign extension, and
simplify the calculation for unsigned types.

Convert the storage of the test cases to signed types to correctly
represent their intended purpose, to prevent test failures.

Reviewed-by: Isaac Scott <isaac.scott@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This commit is contained in:
Kieran Bingham
2026-02-19 15:05:04 +00:00
parent cb40945db6
commit cd3149a7fa
2 changed files with 6 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
test/ipa/libipa/fixedpoint.cpp
View File

@@ -68,7 +68,7 @@ protected:
* The second 7.992 test is to test that unused bits don't
* affect the result.
*/
std::map<double, uint16_t> testCases = {
std::map<double, int16_t> testCases = {
{ 7.992, 0x3ff },
{ 0.2, 0x01a },
{ -0.2, 0x7e6 },
@@ -81,14 +81,14 @@ protected:
int ret;
for (const auto &testCase : testCases) {
ret = testSingleFixedPoint<4, 7, uint16_t>(testCase.first,
ret = testSingleFixedPoint<4, 7, int16_t>(testCase.first,
testCase.second);
if (ret != TestPass)
return ret;
}
/* Special case with a superfluous one in the unused bits */
ret = testFixedToFloat<4, 7, uint16_t, double>(0xbff, 7.992);
ret = testFixedToFloat<4, 7, int16_t, double>(0xbff, 7.992);
if (ret != TestPass)
return ret;