Infrastructure/Project Metadata

KDE Project Metadata

Metadata describing the Git repositories that make up KDE software, and the relationships between those repositories, are contained in two different sources.

1. A KDE Projects Management website, where various data about individual repos can be altered by git repository maintainers, including which branches are considered 'stable' and 'development' branches for i18n purposes.
2. Metadata about the relationships between individual repositories are kept in a separate git repository, kde-build-metadata.

kde-build-metadata

The kde-build-metadata repository contains several files which can be used by scripts and automated programs to properly handle the KDE git repositories. As of this writing there are several files that make up this repository:

• build-script-ignore: This file contains a list of git repositories that should be ignored by scripts used to build the KDE source repositories. Empty lines and comments (prefixed by a #) are permitted. Each other line should be the full kde-project path of a module to ignore. Most examples are for modules that simply have nothing to build and install, but other uses include convenience modules that duplicate functionality handled in other source code modules.
• dependency-data: This file contains a list of dependencies between KDE git repositories. It is used by the kdesrc-build build script, and the continuous integration infrastructure.

Logical module grouping

Note

This section documents a proposed addition. Nothing actually uses this at this point, although it has been reviewed by some of the sysadmins.

In order to make it easy for the various groups building KDE software to get the version they wish, there is a proposal to add the concept of logical module groups so that scripts may automatically select the most appropriate branch for a given individual git repository.

The current proposal is to use a JSON file, with the following structure:

A top-level object, with the following key-value pairs:

version

Will be set to the version supported by conforming scripts. This documentation documents version 0 (the number, not a string). It is intended that the version is only increased for changes that cannot be made in a backward-compatible fashion. Scripts should check that the version is set to a supported version and fail if not.

layers

Will be set to an array of the logical module groupings that are available for use. Currently this is stable-qt4, latest-qt4, kf5-qt5, but this can change as needed. Scripts should allow groupings only from this array.

groups

This is set to an object describing the group layout of the layers described above. See Grouping syntax for more details.

dependencies

This contains and supersedes the information defined in dependency-data. See Specifying dependencies for more details.

Grouping syntax

As described above, the groups key has an object as its value, which itself contains key/value pairs, where each key describes the kde-project module path to operate on, with wildcards being acceptable. The value for this path is another object, describing the layers : branch name mappings for repositories included in the given kde-project module path.

Note

The kde-project module path used as the key should be the full module path in this case. The same is true for all usages of a module path within this file.

For example:

{
/* .... */
        "kde/*": {
            "stable-qt4": "KDE/4.11",
            "latest-qt4": "master",
            "kf5-qt5": "frameworks"
        },
        "kde/kde-workspace" : {
            "stable-qt4": "KDE/4.11",
            "latest-qt4": "KDE/4.11",
            "kf5-qt5": "master"
        }
/* .... */
}

This shows that for kde/kde-workspace (and only for this module, since there are no wildcards), that that stable-qt4 and latest-qt4 logical groupings both pull from the remote git branch called KDE/4.11, while kf5-qt5 developers / scripts should use the remote git branch called master

Selecting a logical group

A logical group may apply to multiple independent git repositories. However, each individual git repository will only match a single logical group (or none at all). In other words, there is no "cascading", so if a layer is not defined within that logical group there is no fallback to a more-generic logical group.

When deciding how to select a logical group, the rule is that the most specific possible match is selected. A * wildcard may be used to stand in for a path component, and all remaining components on the module path. In other words, kde/* would find all kde-project modules that begin with kde/, even those with many intermediate path components, e.g. kde/kdelibs/nepomuk-core.

Note

The wildcard standing in for a path component means that it does not make sense to see except for directly after a path separator, and the wildcard should always be the last part of the specifier. 'kde/*' makes sense, but 'kde*/libs' does not!

An easy implementation of the wildcard logic (once we've failed to find an exact match) is to take the list of logical group specifiers, and sort them all by length in descending order. Search each logical group specifier in that order, seeing if the module path being considered starts with the full specifier (except the wildcard). The first specifier where this is true is chosen. Do not forget to handle the case of a specifier consisting of nothing but '*', which will match everything (though, this is probably a bad idea to include in the file itself!)

If a logical group is matched but does not define a branch name to use for a layer, the script should assume a branch name to use by default. This normally means master, but the idea is that a failure to find a module name in a logical group will not completely stop a build, since many kde-project repositories will not differentiate between different development layers in this fashion.

Using the example given above, the kde/kde-workspace repository would match its own group, kde/kdeutils/kcalc would match the kde/* group, while kdesupport/phonon/phonon would not match any group at all (and therefore would get default branch names, whatever 'default' means for that script).

Specifying dependencies

TODO: Flesh out