Commit Graph

21 Commits

Author SHA1 Message Date
Naushir Patuck
5322b7ba61 libcamera: yaml: Increase the YAML parser limit
Increase the maximum list size to 2000 elements. This allows, for
example, larger lens shading config structures to be parsed correctly
without throwing any errors.

Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2023-07-28 18:34:02 +03:00
Laurent Pinchart
0e3b8d71f5 base: utils: Add and use strtod() helper
The strtod() function is locale-dependent, and thus ill-suited to parse
numbers coming from, for instance, YAML files. The YamlObject class uses
strtod_l() to fix that issue, but that function is not available with
all libc implementations. Correctly handling this problem is becoming
out of scope for the YamlObject class.

As a first step, add a strtod() helper function in the utils namespace
that copies the implementation from YamlObject, and use it in
YamlObject. The core issue will then be fixed in utils::strtod().

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2023-01-10 15:39:18 +02:00
Kieran Bingham
e8ae254970 libcamera: yaml_parser: Use C locale
When parsing configuration files on systems with differing locales, the
use of strtod can produce different results, or in the worst case - fail
to parse expected values.

Fix this by using strtod_l() instead. To avoid constructing and
destructing a locale_t instance for every use of strtod_l(), create an
RAII class that wraps the locale_t and use it to provide a global "C"
locale.

Bug: https://bugs.libcamera.org/show_bug.cgi?id=174
Bug: https://github.com/raspberrypi/libcamera/issues/29
Reported-by: https://github.com/kralo
Reported-by: Hannes Winkler <hanneswinkler2000@web.de>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2023-01-03 13:09:52 +00:00
Kieran Bingham
1a082a3e95 libcamera: yaml_parser: Report filename on failures
It can be helpful to know 'which' file failed to parse if there is a
failure.

Report it to the user in the error message.

Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2022-09-05 17:34:49 +01:00
Laurent Pinchart
acf7213230 libcamera: yaml_parser: De-duplicate common code in YamlObject::get()
The specializations of the YamlObject::get() function template for
integer types duplicate code that doesn't directly depend on the
template type argument. Move it to separate helper functions to reduce
the object size.

While at it, rephrase the comment about unsigned integer parsing.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2022-08-21 21:50:10 +03:00
Laurent Pinchart
a69958fcd6 libcamera: yaml_parser: Enable YamlObject::get() for int8_t and uint8_t
The YamlObject::get() function template is implemented for 16-bit and
32-bit integers. Add an 8-bit specialization that will be used in the
rkisp1 IPA module, and extend the unit tests accordingly.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2022-08-21 21:47:19 +03:00
Laurent Pinchart
dc688f1d88 libcamera: yaml_parser: Fix bounds checking for 16-bit YamlObject::get()
The YamlObject::get() function specializations for 16-bit integers cast
the return value of strto(u)l() to a 16-bit integer, rendering the
bounds checking useless. Fix them.

Fixes: c7d260c03a ("libcamera: yaml_parser: Add get() specializations for 16-bit integers")
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2022-08-21 21:47:18 +03:00
Laurent Pinchart
1a6c7477fd libcamera: yaml_parser: Return nullopt on error from YamlObject::get()
The YamlParser::get<>() function returns an std::optional<> to indicate
when YAML parsing failed.

The current implementation returns a default constructed std::optional
in case of errors with

        return {};

This has been reported as generating compiler warnings with a gcc 9.3.0
arm64 cross-compiler:

../src/libcamera/yaml_parser.cpp:184:11: error: ‘<anonymous>’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
  184 |   return {};
      |           ^

Replace this with an explicit

	return std::nullopt;

which fixes the warnings and conveys the purpose more explicitly.

Reported-by: Christian Rauch <Rauch.Christian@gmx.de>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2022-08-09 22:22:17 +03:00
Jacopo Mondi
2e77ccbb93 libcamera: yaml_parser: Return nullopt on error
The YamlParser::getList<>() function returns an std::optional<> to allow
callers to identify cases where parsing the .yaml file failed from cases
where the parsed list is just empty.

The current implementation returns a default constructed std::optional
in case of errors with

	return {};

The returned value is thus equal to std::nullopt, but the code can be
easily misinterpreted as returning an empty vector by a reader.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2022-08-03 15:07:20 +02:00
Laurent Pinchart
1715b7ed08 libcamera: Drop unnecessary typename keyword used with std::enable_if_t
Usage of the std::enable_if_t type doesn't need to be prefixed by
typename. Drop the unnecessary keyword.

Reported-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2022-07-28 18:47:17 +03:00
Florian Sylvestre
02e387e7b6 libcamera: yaml_parser: Add getList() function
Allow to retrieve a YAML list of any already supported types in a
std::vector.

Signed-off-by: Florian Sylvestre <fsylvestre@baylibre.com>
Tested-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2022-07-28 13:48:54 +03:00
Laurent Pinchart
38987e165c libcamera: yaml_parser: Preserve order of items in dictionary
The std::map container used to store dictionary items in YamlObject
doesn't preserve the YAML data order, as maps are ordered by key, not by
insertion order. While this is compliant with the YAML specification
which doesn't guarantee ordering of mappings, the Raspberry Pi IPA
relies on elements being ordered as in the YAML data. To replace the
dependency on boost with the YamlParser class, we thus need to guarantee
that the order is preserved.

Preserve the order by storing items in list_ unconditionally. Turn the
list_ vector from storing YamlObject unique pointers to storing
key-value pairs, with the key being absent when the object is a list,
not a dictionary.

The YamlObject implementation is updated to preserve the existing API,
with the only difference being that YamlObject::memberNames() now
returns member names in the same order as in the YAML file.

The ordering is an implementation detail, so changing it doesn't violate
the YAML specification. The documentation is not updated to reflect
this, as we don't want any new user to rely on a particular ordering.
This commit could be reverted if desired when the Raspberry Pi IPA
updates to a new tuning data format and drops support for the old
format.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Naushir Patuck <naush@raspberrypi.com>
2022-07-28 13:47:48 +03:00
Laurent Pinchart
feb8c9be78 libcamera: yaml_parser: Replace ok flag to get() with std::optional
The YamlObject::get() function takes a default value and an optional
bool ok flag to handle parsing errors. This ad-hoc mechanism complicates
error handling in callers.

A better API is possible by dropping the default value and ok flag and
returning an std::optional. Not only does it simplify the calls, it also
lets callers handle errors through the standard std::optional class
instead of the current ad-hoc mechanism.

Provide a get() wrapper around std::optional::value_or() to further
simplify callers that don't need any specific error handling.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2022-07-28 13:47:13 +03:00
Laurent Pinchart
c7d260c03a libcamera: yaml_parser: Add get() specializations for 16-bit integers
Extend the YamlObject::get() function template to support 16-bit
integers.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Han-Lin Chen <hanlinchen@chromium.org>
2022-06-20 15:12:27 +03:00
Laurent Pinchart
839c4a5a48 libcamera: yaml_parser: Fix range checks for 32-bit integers
The strtol() and strtoul() functions return long integers, which may be
larger than 32-bit integers. Add manual range checks.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Han-Lin Chen <hanlinchen@chromium.org>
2022-06-20 15:12:26 +03:00
Laurent Pinchart
9dacead615 libcamera: yaml_parser: Remove memberNames() function
Now that YamlObject supports iteration, the memberNames() function isn't
useful anymore as it can be implemented using utils::map_keys() if
really needed. Drop it.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Han-Lin Chen <hanlinchen@chromium.org>
2022-06-20 15:12:26 +03:00
Laurent Pinchart
3a18ad1607 libcamera: yaml_parser: Add iterator API
Allow using range-based for loops over YamlObject instances by
implementing iterators. New YamlObject::DictAdapter and
YamlObject::ListAdapter adapter classes are introduced to provide
different iterators depending on the object type.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Han-Lin Chen <hanlinchen@chromium.org>
2022-06-20 15:12:11 +03:00
Laurent Pinchart
d6d0a675bf libcamera: yaml_parser: Switch from FILE to File
THe FILE object isn't very user-friendly as it requires manual close.
Replace it with File to provide RAII-style resource management in the
YamlParser API.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2022-06-16 02:43:47 +03:00
Laurent Pinchart
27fb47f70b libcamera: yaml_parser: Extend YamlObject::size() to dictionaries
Dictionaries have a size too, extend the size() function to support
them.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2022-06-16 02:43:47 +03:00
Laurent Pinchart
27483e971f libcamera: yaml_object: Turn Type into an enum class
Turn the Type enum into an enum class to force qualifying 'List' and
'Dictionary' in the YamlObject namespace scope. This will help avoiding
ambiguities when adding iterator support.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2022-06-16 02:43:46 +03:00
Han-Lin Chen
fcb0ea001a libcamera: Introduce YamlParser as a helper to parse yaml files
Introduce YamlParser as a helper to convert contents of a yaml file to
a tree based structure for easier reading, and to avoid writing parser
with raw yaml tokens. The class is based on libyaml, and only support
reading but not writing a yaml file.

The interface is inspired by Json::Value class from jsoncpp:
http://jsoncpp.sourceforge.net/class_json_1_1_value.html

Signed-off-by: Han-Lin Chen <hanlinchen@chromium.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2022-05-10 00:22:36 +03:00