Stefan Klug
aec2d99e3a
The libcamera Sphinx documentation contains three toctrees: a main toctree that contains all documentation pages in a flat hierarchy, and two hidden toctrees that point to the introduction and API pages. This architecture is mostly meant to support publishing the documentation on the libcamera.org website. The process recreates a hybrid documentation tree mixing content specific to the website and content extracted from libcamera. The hidden toctrees are used to prevent Sphinx from warning about unreferenced pages when the documentation is built as part of libcamera. This set of hacks work, but produce unorganized documentation in the build directory, as well as when installed to the system. Furthermore, they make it difficult to host multiple versions of the libcamera documentation on the website, which we will eventually want to do as the API stabilizes. It would be generally better to host on libcamera.org the documentation built as part of libcamera with the same structure of documents. To prepare for that change, reorganize the toctrees in libcamera with three visible trees: a toctree for users, a toctree for developers, and a toctree for integrators. Include the public and internal API pages in the first two trees respectively. This mimics the structure of the documentation as currently organized on the website. The resulting documentation becomes easier to navigate in the build and installation directories. Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
219 lines
11 KiB
ReStructuredText
219 lines
11 KiB
ReStructuredText
.. SPDX-License-Identifier: CC-BY-SA-4.0
|
|
|
|
.. include:: documentation-contents.rst
|
|
|
|
************
|
|
Introduction
|
|
************
|
|
|
|
What is libcamera?
|
|
==================
|
|
|
|
libcamera is an open source complex camera support library for Linux, Android
|
|
and ChromeOS. The library interfaces with Linux kernel device drivers and
|
|
provides an intuitive API to developers in order to simplify the complexity
|
|
involved in capturing images from complex cameras on Linux systems.
|
|
|
|
What is a "complex camera"?
|
|
===========================
|
|
|
|
A modern "camera" tends to infact be several different pieces of hardware which
|
|
must all be controlled together in order to produce and capture images of
|
|
appropriate quality. A hardware pipeline typically consists of a camera sensor
|
|
that captures raw frames and transmits them on a bus, a receiver that decodes
|
|
the bus signals, and an image signal processor that processes raw frames to
|
|
produce usable images in a standard format. The Linux kernel handles these
|
|
multimedia devices through the 'Linux media' subsystem and provides a set of
|
|
application programming interfaces known collectively as the
|
|
V4L2 (`Video for Linux 2`_) and the `Media Controller`_ APIs, which provide an
|
|
interface to interact and control media devices.
|
|
|
|
.. _Video for Linux 2: https://www.linuxtv.org/downloads/v4l-dvb-apis-new/userspace-api/v4l/v4l2.html
|
|
.. _Media Controller: https://www.linuxtv.org/downloads/v4l-dvb-apis-new/userspace-api/mediactl/media-controller.html
|
|
|
|
Included in this subsystem are drivers for camera sensors, CSI2 (Camera
|
|
Serial Interface) receivers, and ISPs (Image Signal Processors).
|
|
|
|
The usage of these drivers to provide a functioning camera stack is a
|
|
responsibility that lies in userspace, and is commonly implemented separately
|
|
by vendors without a common architecture or API for application developers. This
|
|
adds a lot of complexity to the task, particularly when considering that the
|
|
differences in hardware pipelines and their representation in the kernel's APIs
|
|
often necessitate bespoke handling.
|
|
|
|
What is libcamera for?
|
|
======================
|
|
|
|
libcamera provides a complete camera stack for Linux-based systems to abstract
|
|
the configuration of hardware and image control algorithms required to obtain
|
|
desirable results from the camera through the kernel's APIs, reducing those
|
|
operations to a simple and consistent method for developers. In short instead of
|
|
having to deal with this:
|
|
|
|
.. graphviz:: mali-c55.dot
|
|
|
|
you can instead simply deal with:
|
|
|
|
.. code-block:: python
|
|
|
|
>>> import libcamera as lc
|
|
>>> camera_manager = lc.CameraManager.singleton()
|
|
[0:15:59.582029920] [504] INFO Camera camera_manager.cpp:313 libcamera v0.3.0+182-01e57380
|
|
>>> for camera in camera_manager.cameras:
|
|
... print(f' - {camera.id}')
|
|
...
|
|
- mali-c55 tpg
|
|
- imx415 1-001a
|
|
|
|
The library handles the rest for you. These documentary pages give more
|
|
information on the internal workings of libcamera (and the kernel camera stack
|
|
that lies behind it) as well as guidance on using libcamera in an application or
|
|
extending the library with support for your hardware (through the pipeline
|
|
handler and IPA module writer's guides).
|
|
|
|
How should I use it?
|
|
====================
|
|
|
|
There are a few ways you might want to use libcamera, depending on your
|
|
application. It's always possible to use the library directly, and you can find
|
|
detailed information on how to do so in the
|
|
:doc:`application writer's guide <guides/application-developer>`.
|
|
|
|
It is often more appropriate to use one of the frameworks with libcamera
|
|
support. For example an application powering an embedded media device
|
|
incorporating capture, encoding and streaming of both video and audio would
|
|
benefit from using `GStreamer`_, for which libcamera provides a plugin.
|
|
Similarly an application for user-facing devices like a laptop would likely
|
|
benefit accessing cameras through the XDG camera portal and `pipewire`_, which
|
|
brings the advantages of resource sharing (multiple applications accessing the
|
|
stream at the same time) and access control.
|
|
|
|
.. _GStreamer: https://gstreamer.freedesktop.org/
|
|
.. _pipewire: https://pipewire.org/
|
|
|
|
Camera Stack
|
|
============
|
|
|
|
::
|
|
|
|
a c / +-------------+ +-------------+ +-------------+ +-------------+
|
|
p a | | Native | | Framework | | Native | | Android |
|
|
p t | | V4L2 | | Application | | libcamera | | Camera |
|
|
l i | | Application | | (gstreamer) | | Application | | Framework |
|
|
i o \ +-------------+ +-------------+ +-------------+ +-------------+
|
|
n ^ ^ ^ ^
|
|
| | | |
|
|
l a | | | |
|
|
i d v v | v
|
|
b a / +-------------+ +-------------+ | +-------------+
|
|
c p | | V4L2 | | Camera | | | Android |
|
|
a t | | Compat. | | Framework | | | Camera |
|
|
m a | | | | (gstreamer) | | | HAL |
|
|
e t \ +-------------+ +-------------+ | +-------------+
|
|
r i ^ ^ | ^
|
|
a o | | | |
|
|
n | | | |
|
|
/ | ,................................................
|
|
| | ! : Language : !
|
|
l f | | ! : Bindings : !
|
|
i r | | ! : (optional) : !
|
|
b a | | \...............................................'
|
|
c m | | | | |
|
|
a e | | | | |
|
|
m w | v v v v
|
|
e o | +----------------------------------------------------------------+
|
|
r r | | |
|
|
a k | | libcamera |
|
|
| | |
|
|
\ +----------------------------------------------------------------+
|
|
^ ^ ^
|
|
Userspace | | |
|
|
------------------------ | ---------------- | ---------------- | ---------------
|
|
Kernel | | |
|
|
v v v
|
|
+-----------+ +-----------+ +-----------+
|
|
| Media | <--> | Video | <--> | V4L2 |
|
|
| Device | | Device | | Subdev |
|
|
+-----------+ +-----------+ +-----------+
|
|
|
|
The camera stack comprises four software layers. From bottom to top:
|
|
|
|
* The kernel drivers control the camera hardware and expose a
|
|
low-level interface to userspace through the Linux kernel V4L2
|
|
family of APIs (Media Controller API, V4L2 Video Device API and
|
|
V4L2 Subdev API).
|
|
|
|
* The libcamera framework is the core part of the stack. It
|
|
handles all control of the camera devices in its core component,
|
|
libcamera, and exposes a native C++ API to upper layers. Optional
|
|
language bindings allow interfacing to libcamera from other
|
|
programming languages.
|
|
|
|
Those components live in the same source code repository and
|
|
all together constitute the libcamera framework.
|
|
|
|
* The libcamera adaptation is an umbrella term designating the
|
|
components that interface to libcamera in other frameworks.
|
|
Notable examples are a V4L2 compatibility layer, a gstreamer
|
|
libcamera element, and an Android camera HAL implementation based
|
|
on libcamera.
|
|
|
|
Those components can live in the libcamera project source code
|
|
in separate repositories, or move to their respective project's
|
|
repository (for instance the gstreamer libcamera element).
|
|
|
|
* The applications and upper level frameworks are based on the
|
|
libcamera framework or libcamera adaptation, and are outside of
|
|
the scope of the libcamera project.
|
|
|
|
V4L2 Compatibility Layer
|
|
V4L2 compatibility is achieved through a shared library that traps all
|
|
accesses to camera devices and routes them to libcamera to emulate high-level
|
|
V4L2 camera devices. It is injected in a process address space through
|
|
``LD_PRELOAD`` and is completely transparent for applications.
|
|
|
|
The compatibility layer exposes camera device features on a best-effort basis,
|
|
and aims for the level of features traditionally available from a UVC camera
|
|
designed for video conferencing.
|
|
|
|
Android Camera HAL
|
|
Camera support for Android is achieved through a generic Android camera HAL
|
|
implementation on top of libcamera. The HAL implements features required by
|
|
Android and out of scope from libcamera, such as JPEG encoding support.
|
|
|
|
This component is used to provide support for ChromeOS platforms.
|
|
|
|
GStreamer element (gstlibcamerasrc)
|
|
A `GStreamer element`_ is provided to allow capture from libcamera supported
|
|
devices through GStreamer pipelines, and connect to other elements for further
|
|
processing.
|
|
|
|
Native libcamera API
|
|
Applications can make use of the libcamera API directly using the C++
|
|
API. An example application and walkthrough using the libcamera API can be
|
|
followed in the :doc:`Application writer's guide </guides/application-developer>`
|
|
|
|
.. _GStreamer element: https://gstreamer.freedesktop.org/documentation/application-development/basics/elements.html
|
|
|
|
Licensing
|
|
=========
|
|
|
|
The libcamera core is covered by the `LGPL-2.1-or-later`_ license. Pipeline
|
|
Handlers are a part of the libcamera code base and need to be contributed
|
|
upstream by device vendors. IPA modules included in libcamera are covered by a
|
|
free software license, however third-parties may develop IPA modules outside of
|
|
libcamera and distribute them under a closed-source license, provided they do
|
|
not include source code from the libcamera project.
|
|
|
|
The libcamera project itself contains multiple libraries, applications and
|
|
utilities. Licenses are expressed through SPDX tags in text-based files that
|
|
support comments, and through the .reuse/dep5 file otherwise. A copy of all
|
|
licenses are stored in the LICENSES directory, and a full summary of the
|
|
licensing used throughout the project can be found in the COPYING.rst document.
|
|
|
|
Applications which link dynamically against libcamera and use only the public
|
|
API are an independent work of the authors and have no license restrictions
|
|
imposed upon them from libcamera.
|
|
|
|
.. _LGPL-2.1-or-later: https://spdx.org/licenses/LGPL-2.1-or-later.html
|