Yocto: Difference between revisions

From KDE Community Wiki
 
(21 intermediate revisions by the same user not shown)
Line 1: Line 1:
= Yocto [DRAFT] =
= KDE in Yocto =
 
If you never heard about Yocto, a very short and somewhat precise description is: Yocto is a framework to build very hardware tailored device images. It is the currently most used technology in automotive as well as in many embedded projects. In order to make device creation simply it both provides a build system called "BitBake" and hundreds of recipes for libraries, applications, hardware adaptations and images, whereas most of those recipes do not come from the "Yocto Project" itself but from the "OpenEmbedded" project. Those recipes are organized in layers and KDE provides the layers as described below.
 
== First Steps ==
The following links are meant for first-time users and contributors with little Yocto experience who want to know more about what KDE Embedded is and how to start using and contributing:
 
* [[Yocto/Introduction|Introduction]] : What we do and what our goals are
* [[Yocto/GettingStarted|Getting Started]] : How to start contributing
 
== What KDE Provides ==


KDE currently ships two Yocto meta layers that are supposed to ease the use of KDE Frameworks, Plasma and Applications for device creators.
KDE currently ships two Yocto meta layers that are supposed to ease the use of KDE Frameworks, Plasma and Applications for device creators.
* meta-kf6
** Repository: https://invent.kde.org/packaging/yocto-meta-kf6
** Yocto Index: http://layers.openembedded.org/layerindex/branch/master/layer/meta-kf6/
* meta-kde (Qt6/KF6 based packaging currently in branch work/qt6 until first official release based on Qt6)
** Repository: https://invent.kde.org/packaging/yocto-meta-kde
** Yocto Index: http://layers.openembedded.org/layerindex/branch/master/layer/meta-kde/
For the Qt5/KF5 based KDE releases, the following layer provides integration (in maintenance mode):


* meta-kf5
* meta-kf5
** Repository: https://invent.kde.org/packaging/yocto-meta-kf5
** Repository: https://invent.kde.org/packaging/yocto-meta-kf5
** Yocto Index: http://layers.openembedded.org/layerindex/branch/master/layer/meta-kf5/
** Yocto Index: http://layers.openembedded.org/layerindex/branch/master/layer/meta-kf5/
* meta-kde
** Repository: https://invent.kde.org/packaging/yocto-meta-kde
** Yocto Index: http://layers.openembedded.org/layerindex/branch/master/layer/meta-kde/


== Usage ==
== Usage ==
The KDE community does not distribute any full system images in terms of a distributor but provides device manufacturers to integrate released KDE software into their products and or even to build their software products on top of KDE frameworks. Our goal is to follow the Yocto best practices in order to behave like any other Yocto layer.
=== manifest Repository ===
The manifest repository tracks compatible combinations of Yocto meta layers and is supposed to ease the setup of a build environment. All manifest files in this repository are [https://gerrit.googlesource.com/git-repo Google repo tool manifests].
See https://invent.kde.org/packaging/yocto-meta-kde-demo/-/blob/master/README.md for how to use the manifest files to configure a Bitbake environment and start the Yocto Build.
Note that there are two types of manifests in the repository:
# Yocto branch manifests: They have the goal to ease layer development for a specific Yocto branch by fixing all layers external of KDE to defined revisions. These branches are called "default", "kirkstone", "dunfell"... For the KDE meta layers, these manifests simply point to the git HEAD revision.
# Release manifests: These manifests use fixed versions of all layers and they are supposed to document compatible combinations when important KDE releases are integrated and when demo images are created.
=== meta-kf5 / meta-kf6 ===
For using meta-kf5 or meta-kf6, simply clone the repository and add it as layer to bblayers. Our current approach with this layer is that there is only the master branch, which targets the following Yocto upstream branches. Dependencies are documented in the layer.conf file:
* meta-kf5 supported Yocto releases:
** dunfell (old Yocto LTS): minimal requirement is dunfell patch version that supports new BitBake colon syntax
** kirkstone (latest Yocto LTS) : supported
** nanbield (current Yocto latest release) : supported
* meta-kf6 supported Yocto releases:
** kirkstone (latest Yocto LTS) : supported
** nanbield (current Yocto latest release) : supported


== Layer Development ==
The layer provides several recipes that follow exactly the repository names of the existing KDE Frameworks recipes. They are internally organized by "tiers" in the same way [https://api.kde.org/frameworks/index.html as the KDE Frameworks itself are organized].


=== CI Approach [DRAFT] ===
Note that some of these recipes also bring tools that are required at compile time. Examples for those are "ki18n" or "kdoctools". In general, for those cases always there is a BitBake class provided by the layer that allows you for an recipe, which require those tools at runtime, to inherit it. That will configure the environment that cross-building will correctly pick up the right version of the tool.


=== Branch / Merge Strategy [DRAFT] ===
=== meta-kde ===
For meta-kde, it is mandatory to also use meta-kf5. At the moment, the only Yocto branch in scope is:
 
* kirkstone (latest Yocto LTS)
* mickledore (current Yocto latest release) : supported
 
= Layer Development =
All changes to the meta-layers are supposed to be proposed as merge-requests in the GitLab repositories and be reviewed before being merged. The main reasons for defaulting to code-reviews is the complexity of the Bitbake language together with the absence of static code check tooling as well as the quality demands of users of the Yocto layers.
 
== CI Approach ==
=== Scope ===
* The goal of the CI tooling is to automatically check the compatibility of the layers with regard to a certain release of the Yocto project in combination with a certain release in relation to a certain release of meta-qt{5|6}
* The CI shall ensure that there is a documented compatible combination of Yocto meta-layers
* The CI shall ensure that backports to older Yocto layers are compatible with older Bitbake versions and recipes
 
=== Technical Implementation ===
Mostly follow the approach done in meta-qt{5|6} in contrast to the Yocto approach, because the goal is to integrate CI builds into the KDE invent.kde.org infrastructure.
 
* yocto-manifest : new repo to track repo XMl files that provide defined combinations of meta-layers that are compatible for a certain CI branch or a specific release tag:
* both meta-kf5 and meta-kde shall contain a core-image-minimal derived test-image, which then can be used as a CI build target
* nativesdk build is in scope, because that is needed for developers to bootstrap their cross-building integrations
* target hardware for the CI builds is always qemu and not a specific target device
 
== Branch / Merge Strategy ==
* application releases are updated in the master branch
* application releases are updated in the master branch
** exception is if the tracking branch for an old Yocto LTS release shall be updated to version older than in the master branch
** exception is if the tracking branch for an old Yocto LTS release shall be updated to version older than in the master branch
* change from the master branch that shall be used for older Yocto releases shall be cherry-picked
* change from the master branch that shall be used for older Yocto releases shall be cherry-picked
== Recipe & Release Updates ==
The big strength of the meta-kf5 and meta-kde layers is that they directly come from the KDE community and thus very well integrate with release cycles and tooling. Our goal is to keep the layers for the most recently supported Yocto version as close as possible to the latest releases of KDE software. Updating of the Yocto layers is done by release scripts that are contained in the <code>scripts/</code> folder of each layer.
* KDE Frameworks 5: shipped by meta-kf5
* KDE Applications: shipped by meta-kde
* KDE Plasma: shipped by meta-kde
* KDE Plasma Mobile: shipped by meta-kde
== Demo Setups ==
The Yocto based KDE setups are regularly displayed at conferences. Examples are:
* [[Yocto/Demos/FOSDEM2024|FOSDEM 2024]]
* [[Yocto/Demos/FOSDEM2023|FOSDEM 2023]]

Latest revision as of 19:25, 31 January 2024

KDE in Yocto

If you never heard about Yocto, a very short and somewhat precise description is: Yocto is a framework to build very hardware tailored device images. It is the currently most used technology in automotive as well as in many embedded projects. In order to make device creation simply it both provides a build system called "BitBake" and hundreds of recipes for libraries, applications, hardware adaptations and images, whereas most of those recipes do not come from the "Yocto Project" itself but from the "OpenEmbedded" project. Those recipes are organized in layers and KDE provides the layers as described below.

First Steps

The following links are meant for first-time users and contributors with little Yocto experience who want to know more about what KDE Embedded is and how to start using and contributing:

What KDE Provides

KDE currently ships two Yocto meta layers that are supposed to ease the use of KDE Frameworks, Plasma and Applications for device creators.

For the Qt5/KF5 based KDE releases, the following layer provides integration (in maintenance mode):

Usage

The KDE community does not distribute any full system images in terms of a distributor but provides device manufacturers to integrate released KDE software into their products and or even to build their software products on top of KDE frameworks. Our goal is to follow the Yocto best practices in order to behave like any other Yocto layer.

manifest Repository

The manifest repository tracks compatible combinations of Yocto meta layers and is supposed to ease the setup of a build environment. All manifest files in this repository are Google repo tool manifests.

See https://invent.kde.org/packaging/yocto-meta-kde-demo/-/blob/master/README.md for how to use the manifest files to configure a Bitbake environment and start the Yocto Build.

Note that there are two types of manifests in the repository:

  1. Yocto branch manifests: They have the goal to ease layer development for a specific Yocto branch by fixing all layers external of KDE to defined revisions. These branches are called "default", "kirkstone", "dunfell"... For the KDE meta layers, these manifests simply point to the git HEAD revision.
  2. Release manifests: These manifests use fixed versions of all layers and they are supposed to document compatible combinations when important KDE releases are integrated and when demo images are created.

meta-kf5 / meta-kf6

For using meta-kf5 or meta-kf6, simply clone the repository and add it as layer to bblayers. Our current approach with this layer is that there is only the master branch, which targets the following Yocto upstream branches. Dependencies are documented in the layer.conf file:

  • meta-kf5 supported Yocto releases:
    • dunfell (old Yocto LTS): minimal requirement is dunfell patch version that supports new BitBake colon syntax
    • kirkstone (latest Yocto LTS) : supported
    • nanbield (current Yocto latest release) : supported
  • meta-kf6 supported Yocto releases:
    • kirkstone (latest Yocto LTS) : supported
    • nanbield (current Yocto latest release) : supported

The layer provides several recipes that follow exactly the repository names of the existing KDE Frameworks recipes. They are internally organized by "tiers" in the same way as the KDE Frameworks itself are organized.

Note that some of these recipes also bring tools that are required at compile time. Examples for those are "ki18n" or "kdoctools". In general, for those cases always there is a BitBake class provided by the layer that allows you for an recipe, which require those tools at runtime, to inherit it. That will configure the environment that cross-building will correctly pick up the right version of the tool.

meta-kde

For meta-kde, it is mandatory to also use meta-kf5. At the moment, the only Yocto branch in scope is:

  • kirkstone (latest Yocto LTS)
  • mickledore (current Yocto latest release) : supported

Layer Development

All changes to the meta-layers are supposed to be proposed as merge-requests in the GitLab repositories and be reviewed before being merged. The main reasons for defaulting to code-reviews is the complexity of the Bitbake language together with the absence of static code check tooling as well as the quality demands of users of the Yocto layers.

CI Approach

Scope

  • The goal of the CI tooling is to automatically check the compatibility of the layers with regard to a certain release of the Yocto project in combination with a certain release in relation to a certain release of meta-qt{5|6}
  • The CI shall ensure that there is a documented compatible combination of Yocto meta-layers
  • The CI shall ensure that backports to older Yocto layers are compatible with older Bitbake versions and recipes

Technical Implementation

Mostly follow the approach done in meta-qt{5|6} in contrast to the Yocto approach, because the goal is to integrate CI builds into the KDE invent.kde.org infrastructure.

  • yocto-manifest : new repo to track repo XMl files that provide defined combinations of meta-layers that are compatible for a certain CI branch or a specific release tag:
  • both meta-kf5 and meta-kde shall contain a core-image-minimal derived test-image, which then can be used as a CI build target
  • nativesdk build is in scope, because that is needed for developers to bootstrap their cross-building integrations
  • target hardware for the CI builds is always qemu and not a specific target device

Branch / Merge Strategy

  • application releases are updated in the master branch
    • exception is if the tracking branch for an old Yocto LTS release shall be updated to version older than in the master branch
  • change from the master branch that shall be used for older Yocto releases shall be cherry-picked

Recipe & Release Updates

The big strength of the meta-kf5 and meta-kde layers is that they directly come from the KDE community and thus very well integrate with release cycles and tooling. Our goal is to keep the layers for the most recently supported Yocto version as close as possible to the latest releases of KDE software. Updating of the Yocto layers is done by release scripts that are contained in the scripts/ folder of each layer.

  • KDE Frameworks 5: shipped by meta-kf5
  • KDE Applications: shipped by meta-kde
  • KDE Plasma: shipped by meta-kde
  • KDE Plasma Mobile: shipped by meta-kde

Demo Setups

The Yocto based KDE setups are regularly displayed at conferences. Examples are: