Latest revision as of 11:56, 21 December 2020

This page describes some basics in taking a package from https://invent.kde.org/neon/ and building it locally. This can be useful for doing local testing before committing changes.

Prerequisites

A pre-existing KDE development environment.
Some knowledge of the content in the Debian New Maintainers' Guide.
dpkg-dev and xbuilder packages will include the standard Debian toolchain for building packages.

Setting up local files for building package

Clone the package repository repository, i.e. https://invent.kde.org/neon/kde/libkdegames
Copy the source tree from your kde sources to where the debian/ path resides.
Create a usr/share/locale to simulate CI locale injection, mkdir -p usr/share/locale && touch usr/share/locale/stub
Remove symbols to simulate CI removing the symbol files, rm debian/*.symbols
Change debian/source/format to use native instead of quilt, sed -i 's/quilt/native/' debian/source/format
Apply patches, QUILT_PATCHES=debian/patches quilt push -a

Or, in one command:

mkdir -p usr/share/locale && touch usr/share/locale/stub && rm -f debian/*.symbols && sed -i 's/quilt/native/' debian/source/format && QUILT_PATCHES=debian/patches quilt push -a

Building the package

Use your favorite tool:

dpkg-buildpackage
debuild

Known false negatives

Complaints about dsc and missing key.
Lintian returns malformed-debian-changelog-version.

Speeding up compilation

A compiler cache can help speed things up immensely, particularly when it comes to packages that take a while to compile.

Install ccache. It might be worth also setting these options:

ccache -o max_size="20G" - Increases the cache size to 20G, roughly kdesrc-build uses about 12G currently and the default is 10G.
ccache -o compression="true" - Enable compression, reduces cache size by compressing objects.
ccache -o compression_level="9" - Highest level of compression, helps utilize your space significantly better at a relatively minimal performance cost.

Configure ~/.devscripts to use ccache when using debuild:

echo 'DEBUILD_PREPEND_PATH="/usr/lib/ccache"' | tee -a ~/.devscripts

Now you should be able to use debuild with ccache. You may want to validate this by checking its' usage statistics while building, use ccache -s.

Using pbuilder

pbuilder is a useful tool when you want to build in a clean environment, and to validate that required development packages are correctly configured in the debian/control file.

pbuilder requires the definition of the Neon repositories when creating the base image:

sudo pbuilder create --distribution focal --othermirror "deb http://archive.neon.kde.org/user focal main|deb-src http://archive.neon.kde.org/user focal main" --keyring /usr/share/keyrings/neon-archive-keyring.gpg

Using ccache

Create a ccache path, such as: sudo mkdir -p /cache/pbuilder

Now configure pbuilder to use ccache, with similar settings we used above

echo 'CCACHEDIR=/cache/pbuilder EXTRAPACKAGES=ccache export CCACHE_MAXSIZE="20G" export CCACHE_COMPRESS"="true" export CCACHE_COMPRESSLEVEL="9"' | sudo tee -a /etc/pbuilderrc

Deduplicating ccache caches

Very likely, you will want to share the same ccache data as to not duplicate effort, to do this will make one central location for cache files and produce mount binds for each user.

Create our generic ccache path: sudo mkdir -p /cache/ccache && sudo chown 65534:65534 /cache/ccache

We will be storing all files under this as nobody:nogroup.

Create our user ccache path: sudo mkdir -p /cache/user && sudo chmod 0000 /cache/user

Create our pbuilder ccache path: sudo mkdir -p /cache/pbuilder && sudo chmod 0000 /cache/pbuilder

Add bindfs mount points (make sure bindfs is installed):

echo 'bindfs#/cache/ccache /cache/user fuse force-user=1000,force-group=1000,perms=u+rwX,create-for-user=65534,create-for-group=65534 0 0 bindfs#/cache/ccache /cache/pbuilder fuse force-user=1234,force-group=1234,perms=u+rwX,create-for-user=65534,create-for-group=65534 0 0' |sudo tee -a /etc/fstab

mount the new binds, sudo mount -a

Set your new user cache directory:

ccache -o cache_dir=/cache/user

@@ Line 3: / Line 3: @@
 = Prerequisites =
-* A pre-existing KDE [https://community.kde.org/Get_Involved/development development environment]
+* A pre-existing KDE [https://community.kde.org/Get_Involved/development development environment].
-* <code>dpkg-dev</code> package will include the standard debian toolchain for building packages.
+* Some knowledge of the content in the [https://www.debian.org/doc/manuals/maint-guide/ Debian New Maintainers' Guide].
+* <code>dpkg-dev</code> and <code>xbuilder</code> packages will include the standard Debian toolchain for building packages.
 = Setting up local files for building package =
@@ Line 11: / Line 12: @@
 # Create a <code>usr/share/locale</code> to simulate CI locale injection, <code>mkdir -p usr/share/locale && touch usr/share/locale/stub</code>
 # Remove symbols to simulate CI removing the symbol files, <code>rm debian/*.symbols</code>
+# Change debian/source/format to use native instead of quilt, <code>sed -i 's/quilt/native/' debian/source/format</code>
+# Apply patches, <code>QUILT_PATCHES=debian/patches quilt push -a</code>
+Or, in one command:
+<code>mkdir -p usr/share/locale && touch usr/share/locale/stub && rm -f debian/*.symbols && sed -i 's/quilt/native/' debian/source/format && QUILT_PATCHES=debian/patches quilt push -a</code>
 = Building the package =
-Because a tarball does not exist, you will need to tell the package builder to not clean the environment before build <code>-nc</code>. It's helpful to further clean the environment after <code>-tc</code> for easier git usage for committing content. To do this, run:
+Use your favorite tool:
+* <code>dpkg-buildpackage</code>
+* <code>debuild</code>
+= Known false negatives =
+* Complaints about dsc and missing key.
+* Lintian returns <code>malformed-debian-changelog-version</code>.
+= Speeding up compilation =
+A compiler cache can help speed things up immensely, particularly when it comes to packages that take a while to compile.
+Install <code>ccache</code>. It might be worth also setting these options:
+* <code>ccache -o max_size="20G"</code> - Increases the cache size to 20G, roughly kdesrc-build uses about 12G currently and the default is 10G.
+* <code>ccache -o compression="true"</code> - Enable compression, reduces cache size by compressing objects.
+* <code>ccache -o compression_level="9"</code> - Highest level of compression, helps utilize your space significantly better at a relatively minimal performance cost.
+Configure <code>~/.devscripts</code> to use <code>ccache</code> when using <code>debuild</code>:
+<code>echo 'DEBUILD_PREPEND_PATH="/usr/lib/ccache"' | tee -a ~/.devscripts</code>
+Now you should be able to use <code>debuild</code> with ccache. You may want to validate this by checking its' usage statistics while building, use <code>ccache -s</code>.
+= Using pbuilder =
+pbuilder is a useful tool when you want to build in a clean environment, and to validate that required development packages are correctly configured in the <code>debian/control</code> file.
+pbuilder requires the definition of the Neon repositories when creating the base image:
+<code>sudo pbuilder create --distribution focal --othermirror "deb http://archive.neon.kde.org/user focal main|deb-src http://archive.neon.kde.org/user focal main" --keyring /usr/share/keyrings/neon-archive-keyring.gpg</code>
+== Using ccache ==
+Create a ccache path, such as:
+<code>sudo mkdir -p /cache/pbuilder</code>
+Now configure pbuilder to use ccache, with similar settings we used above
+<code>echo 'CCACHEDIR=/cache/pbuilder
+EXTRAPACKAGES=ccache
+export CCACHE_MAXSIZE="20G"
+export CCACHE_COMPRESS"="true"
+export CCACHE_COMPRESSLEVEL="9"' | sudo tee -a /etc/pbuilderrc</code>
+=== Deduplicating ccache caches ===
+Very likely, you will want to share the same ccache data as to not duplicate effort, to do this will make one central location for cache files and produce mount binds for each user.
+Create our generic ccache path:
+<code>sudo mkdir -p /cache/ccache && sudo chown 65534:65534 /cache/ccache</code>
+We will be storing all files under this as nobody:nogroup.
+Create our user ccache path:
+<code>sudo mkdir -p /cache/user && sudo chmod 0000 /cache/user</code>
+Create our pbuilder ccache path:
+<code>sudo mkdir -p /cache/pbuilder && sudo chmod 0000 /cache/pbuilder</code>
+Add bindfs mount points (make sure <code>bindfs</code> is installed):
+<code>echo 'bindfs#/cache/ccache /cache/user fuse force-user=1000,force-group=1000,perms=u+rwX,create-for-user=65534,create-for-group=65534 0 0
+bindfs#/cache/ccache /cache/pbuilder fuse force-user=1234,force-group=1234,perms=u+rwX,create-for-user=65534,create-for-group=65534 0 0' |sudo tee -a /etc/fstab</code>
+mount the new binds, <code>sudo mount -a</code>
-<code>dpkg-buildpackage -nc -tc</code>
+Set your new user cache directory:
-Alternatively, change the contents of <code>debian/source/format</code> to <code>native</code> and the tooling can be used without specifying options like <code>-nc</code>.
+<code>ccache -o cache_dir=/cache/user</code>