Guidelines and HOWTOs/Snap: Difference between revisions

From KDE Community Wiki
No edit summary
(33 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{Construction}}
= Put Your App in the Snap Store =


Want to run your application binaries on any Linux distribution? Snap makes that possible.
[[File:Screenshot 20210318 150131.png|thumb|Snap Store KDE Page]]


For general purpose information about snap, snapcraft and how to use them please have a look at their documentation  as it excellently teaches you most generic information about snaps https://docs.snapcraft.io
It is a KDE goal to be [https://kde.org/goals/ All About the Apps] to deliver our apps directly to our users.  Snaps is one of the ways to do this.  Snaps are Linux app packages that can run on pretty much any Linux distro.  There is a single centralised Snap store that delivers them to users.  Take a look at the [https://snapcraft.io/publisher/kde KDE page on the Snap Store] to see what's available.
Even so this page will teach you a lot of the basics of snaps.


For a primer on YAML, the format used to describe snap building,  have a look here https://docs.ansible.com/ansible/latest/reference_appendices/YAMLSyntax.html
== Snap intro ==


TBD separate page specifically about how to use the content snap
A Snap package typically contains all the files, including libraries and data files, to run the app.  There are also Content Snaps which contain reuseble libraries.  In KDE land we have the [https://snapcraft.io/kde-frameworks-5-qt-5-15-core20 KDE Frameworks Content Snap] which includes recent Qt and KDE Frameworks and this is shared between all KDE apps so we do not have to waste disk space and build resources.
{{Construction}}


== How it Works ==
Give it a try by installing a package or two on your system
{{Input|1=<nowiki>snap install kcalc</nowiki>}}
And run kcalc from your apps menu.


Snaps work very much like DMG files on OS X. A snap is a compressed squashfs file containing the entire file tree of the program. A system-level helper called snapd manages the snap file and mounts it into the file system. Snapd also manages host-level integration (e.g. creation of .desktop files). Running an application from the snap needs to go through snapd (e.g. snap run foo) which needs to be installed on the target system. Snap, the format, could be used by various daemon implementations, but as a matter of fact the only viable implementation is snapd and maintained by Canonical. Snapd frowns upon side-loaded snaps. You may still install a snap downloaded manually, but the preferred way to use snaps is through snapd directly which in turn will go to the associated [https://snapcraft.io/store store] as a remote source. The store too is run by Canonical. Most aspects of snaps may be inspected/changed through the CLI for snapd. It's simply called "snap" <code>snap help</code> for help.
This will have downloaded the kcalc Snap package from the Snap store into e.g. <code>/var/lib/snapd/snaps/kcalc_73.snap</code> and mounted it into e.g. <code>/snap/kcalc/current/</code>. You can also just download it to a local directory with  <code>snap download kcalc</code>, use <code>lesspipe kcalc*snap</code> to see what it inside it.


== Binary Sources ==
<code>snap list</code> will show your currently installed snaps and it will now show that you have <code>kcalc</code> and the content snap <code>kde-frameworks-5-qt-5-15-core20</code> as well as the <code>core20</code> content snap installed.


Building a snap we'll lovingly call "snapping". At the time of writing you can snapcraft on two core systems: one is Ubuntu 16.04, the other is Ubuntu 18.04. Both of them are the LTS version of Ubuntu and therefore supported for what seems like forever. Snapcraft has native support to pull binary packages (i.e. debs) into the snap so that you might build against them without having to build the binaries yourselves. For example you could use Qt from Ubuntu directly without the need to build it in your snap. However, since the base systems are LTS the software is usually very dated. Seeing as you may need newer dependencies there's a bunch of ways you can get them besides pre-built Ubuntu debs:
Snaps are containers, similar to Docker. From inside the Snap container access to the file system and system resources are limited. This is good for inter-app security but means the app sees your system quite differently from how it might expect. You can "log" into the container with <code>snap run --shell kcalc</code> to have a look at how the Snapped kcalc app sees your system.


=== As Snapcraft Parts ===
To give the app controlled permissions to the system it plugs connections into resources such as the network or container snaps.  Run <code>snap connections kcalc</code> to see what it gets given access to.  The connections are controlled by the store and app maintainers need to ask the store to apply the auto-connections.  They can also be overridden locally.


You can add any number of additional parts that build dependencies. For example you might build your own qtbase as part of snap. How much work this is can be vastly different between software. Building all of Qt can get quickly slow, at the same time relying on an ancient Qt may not be practical either. To gain access to newer foundations software you may instead want to get them ...
You can take a look at the snap package with <code>snap download kcalc</code> which will download files such as <code>kcalc_73.assert</code> and <code>kcalc_73.snap</code>.  The .assert has the checksums and signatures for the package. The .snap has the (non-store) metadata and all the files of the package. <code>lesspipe kcalc_73.snap</code> to take a look.


=== From KDE neon ===
== Concepts ==


We are in luck and KDE neon already builds packages for Ubuntu LTS, so you may choose to simply use neon debs on top of Ubuntu debs. This gives access to the latest Qt, KDE frameworks and other related libraries. This can be a huge time saver. Unfortunately Snapcraft's support for adding additional repositories is non-existent and so if you choose to go this route you may need to somewhat manually manage the build environment yourself e.g. using an especially prepared LXD container.
Snaps are usually one app per Snap package. The Snap package contains all the libraries and resources it needs to run except those in the shared content kde-frameworks Snap.


=== From Content Snaps ===
In practice this means all of Qt and KF5 including Breeze icons and themes are in the kde-frameworks content Snap and your app Snap only needs to compile its own sources.  If you apps needs other libraries it can either install these as Apt packages from the Ubuntu or KDE neon or other repository, or it can compile them from source as well.  You will need to manually list the build-packages (all the -dev packages) and the stage-packages used in the final package, it'll warn you if any final libraries it expects are missing.


Content snaps are other snaps you can "include" in your snap to get access to pre-existing binary (or data) assets so you don't need to ship them. Notable advantage is that one content snap may be shared across multiple consumers and thus reduce the disk and network footprint. For example there is a read to use KF5 content snap which you can use to get access to all
'''Snapcraft''' is used to build snaps. It can be installed as a snap with <code>snap install snapcraft --classic</code>.  Snap packages are defined with snapcraft.yaml files. Snapcraft will build them inside a virtual machine, we use LXD to build the KDE ones (the default is to use Multipass another virtual machine manager but that has problems on cloud machines). Using a virtual machine makes it reliable to build the Snaps on any system with identical results.  
of KF5 and Qt5. Content snaps may be combined with the other means of sourcing binaries.


=== From PPAs ===
snapcraft.yaml files are kept in either [https://invent.kde.org/neon KDE neon git repositories] or in the apps repository.  They can be built on the [https://build.neon.kde.org/view/Snaps/ KDE neon Jenkins CI].  All KDE Developers have access to all these git repos and use of the Jenkins CI.  Stable versions are kept in the Neon/release branches or the stable KDE git branch of your app.  Unstable versions in Neon/unstable or the unstable branch of your app (usually master).


Much like KDE neon, this too will require you to manage the build environment manually. Also, when using PPAs beware that they may not be compatible with neon (so, ideally you should use either-or) and that many of them are not particularly trustworthy or well maintained (important vis-a-vis security).
Our Snaps read metadata from AppStream metadata files so it is important the metadata is up to date including current release versions.


== Snapcraft ==
The [https://snapcraft.io/ Snap Store] is the centralised app store by Canonical.  There is no practical way to use other stores or repositories with Snaps.  It is what Snapcraft uploads built snaps to and what your local snapd will download and install snaps from.  It also says what permissions those snaps should have.  As an app developer if you want your app to have extra permissions (for example [https://invent.kde.org/neon/kde/kdf/-/blob/Neon/unstable/snapcraft.yaml kdf uses mount-observe]) then you need to ask for it on the [https://forum.snapcraft.io/t/request-for-connection-kdf-mount-observe/10953 snapcraft forum].


The tool to build snaps is called snapcraft. The definition for how to craft a snap is written down in snapcraft.yaml. Generally speaking a snapcraft.yaml will contain global metadata of the snap, a list of applications provided by the snap, and lastly a list of parts that when put together result in the snap. Access to the host system is controlled through a plug-and-slot system which is also used in the snapcraft.yaml. Each application may have one or multiple plugs it uses to get access to resources of the host system or other snaps. For example the 'desktop' plug gives access to host fonts and icons. Also see https://docs.snapcraft.io/interface-management/6154
A Classic containment Snap has no restrictions on what files it can see on your system or what external executable can be run.  This is useful for IDEs and similar apps such as [https://forum.snapcraft.io/t/kate-as-classic-snap/23514 Kate] which runs external programs.  Again this needs to be set in your [https://invent.kde.org/neon/kde/kate/-/blob/Neon/release/snapcraft.yaml#L3 snapcraft.yaml] then you need to ask on the Snap forum for the store to set it to classic. The Store will then tell snapd for anyone installing the Snap to have it installed as a Classic confinement Snap.


== Types of Snap==
There is a KDE account on the Snap store which is run by the KDE neon developers Jonathan Riddell and Harald Sitter.  One Snap on the store can be shared between more than one account so app maintainers can also create a separate account if they want to have more control over when their app is released and what the store says about it.


With a broad overview of abilities and shortcomings let's dive right in and look at types of snaps we might build.
The store has four channels for different levels of stability.  Our stable branch builds get uploaded to the Candidate channel and can be moved to the Stable channel once tested.


=== Standalone  ===
== Example ==


A standalone snap is a snap which solely relies on a "core" but no other snaps. This is generally speaking the most reliable type of snap as everything the snap needs is inside the snap (except libc and friends which are in the core).
[https://apps.kde.org/blinken/ Blinken] is an exciting memory game from KDE.  It's [https://snapcraft.io/blinken available on the Snap store].  The Snap package is defined by a <code>snapcraft.yaml</code> file which is in the <code>Neon/release</code> branch of [https://invent.kde.org/neon/kde/blinken/-/blob/Neon/release/snapcraft.yaml KDE neon's Blinken packging].  Any update to that branch triggest a build of the [https://build.neon.kde.org/view/Snaps/job/focal_stable_kde_blinken_snap_amd64/ Blinken Snap job] in KDE neon's Jenkins builder.  If the build is successful it will be uploaded to the <code>Candidate channel</code> of the Snap store ready for review.


It is also the best supported way of building a snap since it's been around since the very beginning.
The <code>snapcraft.yaml</code> file looks like this:
 
Advantages:
 
* Very reliable
* Easy to build and test
* You are always on your own and unrelated changes rarely if ever can impair your snap
 
Disadvantages:
 
* Huge in size (each standalone snap needs to ship their own Qt/l10n and necessary kf5 and other dependencies)
* You need to take care of setting up your execution environment yourself.
* You are always on your own and unrelated changes rarely if ever can improve your snap
 
==== Example ====
 
{{Input|<syntaxhighlight lang="yaml" line>
name: qtnetsample
version: '0'  # the version of the snap. has no semantic meaning
summary: This is my-snap's summary  # 79 char long summary
description: This is my-snap's description  # a longer description for the snap
confinement: strict  # use "strict" to enforce system access only via declared interfaces
grade: devel # use "stable" to assert the snap quality
base: core18 # the core this snap depends on
 
apps:
    qtnetsample:
        command: launcher qtnetsample # the launcher will setup the environment for qtnetsample to find libraries/plugins/data etc
        plugs: [x11, network, network-bind] # this snap will be able to act as xclient and talk over the network
 
parts:
    qtnetsample:
        build-packages: [qt5-default]
        plugin: cmake
        stage-packages: [libqt5network5, libqt5core5a]
        source: .
</syntaxhighlight>}}
 
=== Shared Snap ===
 
A snap may also choose to use one or more Content Snaps (see glossary) to share part of the binaries or UI assets with other snaps. As shared content will generally be in the content snap, the ultimate size of the snap can be fairly small. Think of this as an approach more akin to how traditional binary package dependencies work. Albeit with many of the same complexities surrounding it.
 
For example KDE neon builds the kde-frameworks-5 content snap. It contains all of Qt and all (not-deprecated) KDE frameworks along with Plasma integration rigging.
 
Advantages:
 
* Application snap is super small
* You don't need to care of setting up the execution environment
* Integration and international improvements are all in one place (shared environment setup etc)
* Generally speaking when using the KF5 content snap SDK you can get access to KDE neon's Qt and Frameworks without having to actually add the deb sources.
 
Disadvantages:
 
* Up-front "cost" of a single application may be higher. e.g. if the application only uses QtCore, the content snap will still bring in all of Qt and all of KF5 through the content snap. It's like a shared library, the more it is used the smaller the cost per-user.
* Somewhat harder to build and test because of added complexity. Also managing deb build dependencies in addition to content snap SDKs is problematic '''TBD link to forum post'''
* Unrelated changes in the content snaps may impair your snap
* Since this type was introduced a while after snap initially came into being you still can feel rough edges when working with content snaps.
 
==== Example ====


{{Input|<syntaxhighlight lang="yaml" line>
{{Input|<syntaxhighlight lang="yaml" line>
---
---
name: kbruch
name: blinken
version: 18.12.1
confinement: strict
confinement: strict
grade: stable
grade: stable
base: core18
base: core20
adopt-info: kbruch # part to adopt appstream data from (first in parse list)
adopt-info: blinken
apps:
apps:
     kbruch:
     blinken:
         command: kf5-launch kbruch
        extensions:
        - kde-neon
        common-id: org.kde.blinken.desktop
         command: usr/bin/blinken
         plugs:
         plugs:
        - kde-frameworks-5-plug
         - home
         - home
        - x11
        - opengl
         - network
         - network
         - network-bind
         - network-bind
         - unity7
         - audio-playback
        - pulseaudio
         - removable-media
        - desktop
         - desktop-legacy
        common-id: org.kde.kbruch.desktop
        desktop: "usr/share/applications/org.kde.kbruch.desktop"
slots:
slots:
     session-dbus-interface:
     session-dbus-interface:
         interface: dbus
         interface: dbus
         name: org.kde.kbruch
         name: org.kde.blinken
         bus: session
         bus: session
plugs:
package-repositories:
    kde-frameworks-5-plug:
-   type: apt
        content: kde-frameworks-5-core18-all
    components:
        interface: content
    - main
        default-provider: kde-frameworks-5-core18
    suites:
        target: kf5 # target directory where the content is mounted i.e. $SNAP/kf5/
    - focal
    key-id: 444DABCF3667D0283F894EDDE6D4736255751E5D
    url: http://origin.archive.neon.kde.org/user
    key-server: keyserver.ubuntu.com
parts:
parts:
     kbruch:
     blinken:
        build-snaps:
        - kde-frameworks-5-core18-sdk
        after:
        - kde-frameworks-5-env
         plugin: cmake
         plugin: cmake
         source: src
        build-packages:
         configflags:
        - libkf5doctools-dev
        - libphonon4qt5-dev
        - libphonon4qt5experimental-dev
         source: http://download.kde.org/stable/release-service/20.12.3/src/blinken-20.12.3.tar.xz
         cmake-parameters:
         - "-DKDE_INSTALL_USE_QT_SYS_PATHS=ON"
         - "-DKDE_INSTALL_USE_QT_SYS_PATHS=ON"
         - "-DCMAKE_INSTALL_PREFIX=/usr"
         - "-DCMAKE_INSTALL_PREFIX=/usr"
Line 158: Line 97:
         - "-DBUILD_TESTING=OFF"
         - "-DBUILD_TESTING=OFF"
         - "-DKDE_SKIP_TEST_SETTINGS=ON"
         - "-DKDE_SKIP_TEST_SETTINGS=ON"
         parse-info: [usr/share/metainfo/org.kde.kbruch.appdata.xml]
        - "-DCMAKE_FIND_ROOT_PATH=/usr\\;/root/stage\\;/snap/kde-frameworks-5-qt-5-15-core20-sdk/current"
    kde-frameworks-5-env:
         parse-info:
        plugin: dump
        - usr/share/metainfo/org.kde.blinken.appdata.xml
        source: https://github.com/apachelogger/kf5-snap-env.git
        filesets:
            exclusion:
            - "-usr/lib/*/cmake/*"
            - "-usr/include/*"
            - "-usr/share/ECM/*"
            - "-usr/share/doc/*"
            - "-usr/share/man/*"
            - "-usr/share/icons/breeze-dark*"
            - "-usr/bin/X11"
            - "-usr/lib/gcc/x86_64-linux-gnu/6.0.0"
            - "-usr/lib/aspell/*"
        prime:
        - "$exclusion"
</syntaxhighlight>}}
</syntaxhighlight>}}


== Execution Environment and Launchers ==
Check [https://snapcraft.io/docs/snapcraft-yaml-reference Snapcraft YAML reference] if unsure.
 
When binaries inside snaps get executed they only get a super minimal environment set up by snapd. The snap itself needs to take care of most of the higher level spin up of the environment.
 
Inside a confined snap the <code>/</code> will be the core snap, while the actual snap will be in <code>SNAP=/snap/name/rev/...</code>. As a result for example icons, which usually would be expected in <code>$XDG_DATA_DIRS/icons</code> meaning <code>/usr/share/icons</code>, will need to actually be looked for in <code>$SNAP/usr/share/icons</code>. The same applies to pretty much all XDG_* variables, LD_LIBRARY_PATH, various QT_* variables and so on and so forth.
 
Simply put: a snap's tree is not "merged" with the core's tree, rather it is "mounted" inside the core tree under $SNAP and so each snap needs to set up an environment which redirects or adds $SNAP to all lookup locations you can possibly imagine. As general assumptions about where things are on a Linux system no longer hold true. One the one hand that technically allows you to create a snap which entirely does away with the [https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard FHS], on the other it means someone needs to actually mangle the environment so files may be located properly.
 
That's why most, if not all, desktop application snaps will need a launch helper. The launcher will set up all the general purpose path variables so they point to $SNAP. A standard desktop launcher implementation is available here https://github.com/ubuntu/snapcraft-desktop-helpers. Obviously you can also write your own, but since there are lots of things to consider, even for simple applications, it's probably not a good idea to do so.
 
You can have a look at the standard environment by running
 
{{Input|1=<nowiki>
snap install hello-world
snap run --shell hello-world
env
</nowiki>}}
 
This will drop you on a minimal shell inside the confined snap, where you can have a look around to see what the snap sees.
 
== A Snap from Scratch ==
 
We'll create a snap bundle from scratch using LXD, the KDE neon repositories, and will also look at how to make use of the KDE Frameworks 5 content snap. Using LXD and managing the environment manually  allows us to use the KDE neon repositories, it does however also mean that we need to take care of more things ourselves. It also means that snapcraft will need to be run with <code>--destructive-mode</code> to instruct it that it may install dependencies and the like into the actual system.


To follow along you'll need a working LXD setup. A KDE neon VM would do as well. Docker however is pretty unsuitable as we need a working systemd, which is hard to get with docker. To get started with LXD, you need to run <code>lxd init</code> on most distributions.
=== Top Level ===


{{Input|<nowiki>
* name: blinken ← the snap name registered on the snap store
sudo lxc launch --ephemeral ubuntu:18.04 mycontainer # start an ephemeral container. it will be deleted once stopped
* confinement: strict ← Snaps are a containerised format and can't see the outside system from inside their container. Strict is the normal container method. Classic is also possible which allows it to see the outside system and is used by e.g. Kate because Kate needs to run external programs like Git.  It can only be Classic on request.  Can also be devmode for testing.
sudo lxc exec mycontainer -- bash
* grade: stable ← It must be stable to be in a released channel, can also be devel.
</nowiki>}}
* base: core20 ← which base system to build on, core20 means Ubuntu 20.04 and is the current recommended.
* adopt-info: blinken ← Which Snap part to get the appstream info from.  This sets version, icon, description.


You are now inside an Ubuntu 18.04 container. We'll continue to setup the neon repositories and the latest  stable snapcraft.
You might also need to add <code>version</code> if it is not in the appstream file. This is just a version read by users it does not affect the revision number which is tracked by the store.


{{Input|<nowiki>
=== apps ===
apt-key adv --keyserver keyserver.ubuntu.com --recv E6D4736255751E5D
echo 'deb http://archive.neon.kde.org/unstable bionic main' > /etc/apt/sources.list.d/neon.list
apt update
snap install --stable --classic snapcraft
mkdir /workspace
</nowiki>}}
 
Now we can start writing our snapcraft.yaml. You can either install a command line editor and write it inside the container or write it on your system and "upload" it to the container with the command <code>sudo lxc file push --recursive snapcraft.yaml mycontainer/workspace/</code>. Let's start with the absolutely bare minimum.


{{Input|<syntaxhighlight lang="yaml" line>
{{Input|<syntaxhighlight lang="yaml" line>
---
apps:
name: kmplot
    blinken:
version: '0'
        extensions:
summary: Function Plotter
        - kde-neon
description: KmPlot is a program to plot graphs of functions.
        common-id: org.kde.blinken.desktop
confinement: strict
        command: usr/bin/blinken
grade: stable
        plugs:
base: core18
        - home
        - network
        - network-bind
        - audio-playback
        - removable-media
</syntaxhighlight>}}


parts:
<code>apps</code> are the programs which the snap includes for users to run.  Usually there is only one in a Snap but sometimes e.g. [https://invent.kde.org/neon/extras/calligra/-/blob/Neon/release/snapcraft.yaml Calligra] there are more than one.
    kmplot:
        plugin: cmake
        source: https://anongit.kde.org/kmplot.git
</syntaxhighlight>}}


We've defined (not very good) metadata for the snap and a single part to build. Attempting to build this using <code>snapcraft --destructive-mode</code> will however result in an error similar to
The [https://snapcraft.io/docs/kde-neon-extension KDE neon extension] adds some commonly used features to the KDE snaps including using the [https://snapcraft.io/kde-frameworks-5-qt-5-15-core20 KDE Frameworks 5 content Snap].


{{Output|<nowiki>
The <code>common-id</code> comes from the Appstream file.  You ''must'' check what it is in the appstream file. <code>org.kde.blinken.appdata.xml</code> contains <code><id>org.kde.blinken.desktop</id></code> so we use that.  Sometimes apps use the .desktop and sometimes they don't, this is at random.
CMake Error at CMakeLists.txt:1 (project):
  No CMAKE_CXX_COMPILER could be found.
</nowiki>}}


We haven't installed any of our build dependencies. An easy fix. We'll simply add build-packages to our part. In this case the compiler is missing and it's usually best pulled in via the package "build-essential". You'd continue building your build-packages list until the software starts building. Fortunately I already know all the stuff that is needed so we can move ahead.
The command to run is listed. The KDE neon extension will run a script first which sets many necessary environment variables.


{{Tip|1=For software which is packaged through KDE neon (which is just about everything KDE) you can get a good list to start with by looking at the debian/control file of the Neon/release branch of the packaging repository at https://packaging.neon.kde.org}}
The plugs give access to the outside system, see [https://snapcraft.io/docs/supported-interfaces Supported interfaces] for descriptions. When a Snap is installed from the Store it is up to the Store to say which plugs get used. Thost listed as auto connect in the docs are permitted.  Otherwise you must ask on the Snap forum for permission to have the Snap connected.  (Locally installed snaps with --devmode have access to everything, you can also manually connect snaps to interfaces on your local system.)


The part needs editing with build-packges specified before we can do another snapcraft run:
<code>slots</code> are the complement to plugs, they allow the outside system to access our Snap app.  In this case we are allowing a dbus interface into the Snap.  All KDE apps have a dbus interface and you can check what it is called by running the app and using <code>qdbus</code>.


{{Input|<syntaxhighlight lang="yaml" line>
<code>package-repositories</code> add the KDE neon apt repo to build against, this will give you the latest libraries to compile with.
parts:
    kmplot:
        plugin: cmake
        source: https://anongit.kde.org/kmplot.git
        source-branch: Applications/19.04 # not needed for master
        build-packages: [build-essential, extra-cmake-modules, libkf5widgetsaddons-dev, libqt5svg5-dev, libkf5parts-dev, libkf5doctools-dev, libkf5crash-dev, gettext]
</syntaxhighlight>}}


Snapcrafting our refined snapcraft.yaml should finally succeed in creating a snap. Unfortunately it will also print a long list of missing dependencies:
The source of a Snap is the <code>parts</code> and some snaps have several parts made of different sources e.g. [https://invent.kde.org/neon/extras/ktorrent/-/blob/Neon/release/snapcraft.yaml KTorrent] has both libktorrent and ktorrent parts. Blinken is not complex so it has only one part made of the compiles Blinken source.


{{Output|<nowiki>
=== Parts ===
[...]
Staging kmplot
Priming kmplot
The 'kmplot' part needs the following libraries that are not included in the snap or base:
usr/lib/x86_64-linux-gnu/libGL.so.1
usr/lib/x86_64-linux-gnu/libGLX.so.0
usr/lib/x86_64-linux-gnu/libGLdispatch.so.0
usr/lib/x86_64-linux-gnu/libKF5Archive.so.5
usr/lib/x86_64-linux-gnu/libKF5Attica.so.5
usr/lib/x86_64-linux-gnu/libKF5AuthCore.so.5
usr/lib/x86_64-linux-gnu/libKF5Codecs.so.5
usr/lib/x86_64-linux-gnu/libKF5Completion.so.5
usr/lib/x86_64-linux-gnu/libKF5ConfigCore.so.5
usr/lib/x86_64-linux-gnu/libKF5ConfigGui.so.5
usr/lib/x86_64-linux-gnu/libKF5ConfigWidgets.so.5
usr/lib/x86_64-linux-gnu/libKF5CoreAddons.so.5
usr/lib/x86_64-linux-gnu/libKF5Crash.so.5
usr/lib/x86_64-linux-gnu/libKF5DBusAddons.so.5
usr/lib/x86_64-linux-gnu/libKF5GlobalAccel.so.5
usr/lib/x86_64-linux-gnu/libKF5GuiAddons.so.5
usr/lib/x86_64-linux-gnu/libKF5I18n.so.5
usr/lib/x86_64-linux-gnu/libKF5IconThemes.so.5
usr/lib/x86_64-linux-gnu/libKF5ItemViews.so.5
usr/lib/x86_64-linux-gnu/libKF5JobWidgets.so.5
usr/lib/x86_64-linux-gnu/libKF5KIOCore.so.5
usr/lib/x86_64-linux-gnu/libKF5KIOWidgets.so.5
usr/lib/x86_64-linux-gnu/libKF5Parts.so.5
usr/lib/x86_64-linux-gnu/libKF5Service.so.5
usr/lib/x86_64-linux-gnu/libKF5SonnetCore.so.5
usr/lib/x86_64-linux-gnu/libKF5SonnetUi.so.5
usr/lib/x86_64-linux-gnu/libKF5TextWidgets.so.5
usr/lib/x86_64-linux-gnu/libKF5WidgetsAddons.so.5
usr/lib/x86_64-linux-gnu/libKF5WindowSystem.so.5
usr/lib/x86_64-linux-gnu/libKF5XmlGui.so.5
usr/lib/x86_64-linux-gnu/libQt5Core.so.5
usr/lib/x86_64-linux-gnu/libQt5DBus.so.5
usr/lib/x86_64-linux-gnu/libQt5Gui.so.5
usr/lib/x86_64-linux-gnu/libQt5Network.so.5
usr/lib/x86_64-linux-gnu/libQt5PrintSupport.so.5
usr/lib/x86_64-linux-gnu/libQt5Svg.so.5
usr/lib/x86_64-linux-gnu/libQt5TextToSpeech.so.5
usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
usr/lib/x86_64-linux-gnu/libQt5X11Extras.so.5
usr/lib/x86_64-linux-gnu/libQt5Xml.so.5
usr/lib/x86_64-linux-gnu/libX11.so.6
usr/lib/x86_64-linux-gnu/libXau.so.6
usr/lib/x86_64-linux-gnu/libXdmcp.so.6
usr/lib/x86_64-linux-gnu/libdouble-conversion.so.1
usr/lib/x86_64-linux-gnu/libfam.so.0
usr/lib/x86_64-linux-gnu/libfreetype.so.6
usr/lib/x86_64-linux-gnu/libgraphite2.so.3
usr/lib/x86_64-linux-gnu/libharfbuzz.so.0
usr/lib/x86_64-linux-gnu/libicudata.so.60
usr/lib/x86_64-linux-gnu/libicui18n.so.60
usr/lib/x86_64-linux-gnu/libicuuc.so.60
usr/lib/x86_64-linux-gnu/libpcre2-16.so.0
usr/lib/x86_64-linux-gnu/libpng16.so.16
usr/lib/x86_64-linux-gnu/libxcb-keysyms.so.1
usr/lib/x86_64-linux-gnu/libxcb.so.1
These dependencies can be satisfied via more stage-packages, more parts, or content sharing.
Snapping 'kmplot' |                                                                                                                                 
Snapped kmplot_18.12.1_amd64.snap
</nowiki>}}


We can build the software fine, but snapcraft is concerned that we haven't "snapped" all the necessary dependencies. The list it prints is by no means exhaustive, it's only things snapcraft can easily detect, such as missing shared libraries. Getting the necessary dependencies on board is a bit tricky. The easiest way would be to simply take the list of build-packages and use the exact same list as stage-packages. Stage packages (and their dependencies) get put into the final snap. So, by using the build-packages also as stage-packages we'd put a whole bunch of buildtime-only stuff into our final snap unless we explicitly exclude files from getting primed. Since that is somewhat unreliable and probably not particularly advisable unless you have a firm grasp of all concepts involved, we'll opt to do it the other way around and only stage packages we know are need. So from the list of build-packages packages we'll simply look at their dependencies and try to infer which of their dependencies we need (if any).
* plugin ← which [https://snapcraft.io/docs/supported-plugins Snap build plugin] to use
* build-packages ← most build packages are in the KDE Frameworks content snap but some need added explicitly and some are not in there.  They will be downloaded from the neon and ubuntu apt repos.  [https://invent.kde.org/neon/extras/ktorrent/-/blob/Neon/release/snapcraft.yaml#L40 KTorrent] uses non-KDE libraries and it needs to list the -dev packages in the <code>build-packages</code> then the library itself in the <code>stage-packages</code>.
* source ← link to the tar
* cmake-parameters ← copy and paste this, it sets the right paths.
* parse-info ← where the appstream file is to be installed
* filesets and prime ← snap parts get build then copied into a <code>stage</code> area, when all the parts are built they are copied into the <code>prime</code> area which is converted into the Snap package.  This lists a common set of excluded files we do not want copied.  You can add more here if you end up with unnecessary files in your snap.


* build-essential we'll leave out entirely, as the name suggests it only contains buildtime stuff such as make and gcc
== Building ==
* extra-cmake-modules similarly is only useful at build time as it contains cmake extensions
Install snapcraft with <code>snap install snapcraft --classic</code>.
* libkf5widgetsaddons-dev is a dev package of a library and thus actually needed at runtime.  


We'll inspect its package relationships with the command <code>apt show libkf5widgetsaddons-dev | grep -P "(Depends|Recommends)"</code>.
In the directory with the <code>snapcraft.yaml</code> run: <code>snapcraft --enable-experimental-package-repositories --enable-experimental-extensions --use-lxd</code>


{{Output|<nowiki>
This will start a virtual machine and build the package.  If all is well you will have <code>blinken_20.12.3_amd64.snap</code> or similar created.
Depends: libkf5widgetsaddons5 (= 5.56.0+p18.04+git20190322.0124-0), qtbase5-dev (>= 5.8.0~)
Recommends: libkf5widgetsaddons-doc (= 5.56.0+p18.04+git20190322.0124-0)
</nowiki>}}


We can ignore qtbase5, its another dev package. libkf5widgetsaddons-doc is library documentation which we also do not need. libkf5widgetsaddons5 is the actual library and we'll definitely want it staged.
Install with <code>snap install --devmode blinken_20.12.3_amd64.snap</code>
We'll continue this review for all build-packages:


* libqt5svg5-dev: libray is libqt5svg5
Run with <code>snap run blinken</code> or check it is in the app menu and run from there (remove any versions of blink you have installed from your normal distro packages just to be sure).
* libkf5parts-dev: library is libkf5parts5
* libkf5doctools-dev: buildtime only, builds documentation
* gettext: buildtime only, builds localization
* libkf5crash-dev: library is libkf5crash5


We now have a list of packages that need staging and can update our part accordingly:
== Quirks ==


{{Input|<syntaxhighlight lang="yaml" line>
=== alsa ===
parts:
    kmplot:
        plugin: cmake
        source: https://anongit.kde.org/kmplot.git
        source-branch: Applications/19.04 # not needed for master
        build-packages: [build-essential, extra-cmake-modules, libkf5widgetsaddons-dev, libqt5svg5-dev, libkf5parts-dev, libkf5doctools-dev, libkf5crash-dev, gettext]
        stage-packages: [libkf5widgetsaddons5, libqt5svg5, libkf5parts5, libkf5crash5]
</syntaxhighlight>}}


Snapcrafting our refined data should now create a snap without any additional warnings. This should be good enough to now. You can pull the snap out of the container with <code>sudo lxc file pull mycontainer/workspace/kmplot_0_amd64.snap .</code> and install it into with <code>snap install --force-dangerous kmplot_0_amd64.snap</code> and try to run it <code>snap run kmplot</code>. Unsuccessfully...
Some KDEGames use libkdegames's KgSound class which uses libopenal which uses libsnd which uses alsa. (Most other KDE software uses QtMultimedia or Phonon which uses Pulseaudio.)  See [https://invent.kde.org/neon/kde/kblocks/-/blob/Neon/release/snapcraft.yaml KBlocks] for one way to make this work. Use the alsa extension, use layers to move files around, exclude the pulse alsa file.


{{Output|<nowiki>
=== Qt Only ===
error: cannot find app "kmplot" in "kmplot"
</nowiki>}}


Before snapd can run an application the snap first needs to declare oneLet's add one:
You may want to simplify your Snap by using Qt directly instead of the KDE neon extension and KDE Frameworks content SnapGood luck :)


{{Input|<syntaxhighlight lang="yaml" line>
=== Patches ===
apps:
    kmplot:
        command: kmplot
        plugs: [x11, opengl, desktop, desktop-legacy]
</syntaxhighlight>}}


Plugs are a bit out of scope, for more information on them refer to the upstream snapcraft documentation. Snapcrafting, installing and running the new snap will unfortunately still result in problems:
If you need to update some code in the release you can patch it in the Snap package.  But please get the patch upstream into the Git archive first.


{{Output|<nowiki>
[https://invent.kde.org/neon/neon-packaging/falkon/-/blob/70fb51728cc9c9f6da4d417bca96e2c34c96c52c/snapcraft.yaml#L45 Falkon does this].
kmplot(4481)/(qt.qpa.plugin) unknown: Could not find the Qt platform plugin "xcb" in ""
kmplot(4481)/(default) unknown: This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this proble
</nowiki>}}


This is where the environment helpers come in. Qt attempts to find the xcb plugin in /usr/... but actually needs to look in $SNAP/usr/...
== Help ==


{{Construction}}
[https://snapcraft.io/docs Snapcraft docs] including tutorials on using and building
[https://snapcraft.io/docs/snap-format snapcraft.yaml format]
[https://forum.snapcraft.io/ Snap forum] for asking for help or asking to get the store to allow your Snaps to auto connect.
[https://webchat.kde.org/#/room/#kde-neon:kde.org KDE neon devs] talk to Riddell or Sitter for help getting your app into KDE neons builds and into the Store.


== Glossary ==
== Glossary ==
Line 376: Line 207:
Words you'll hear and not know what they mean:
Words you'll hear and not know what they mean:


* '''snap''': The actual bundle format.
* '''snap''': The actual package format.
* '''snapd''': The daemon that manages snap on a system.
* '''snapd''': The daemon that manages snap on a system.
* '''snapcraft''': The build tool for building a snap.
* '''snapcraft''': The build tool for building a snap.

Revision as of 14:01, 19 April 2021

Put Your App in the Snap Store

Snap Store KDE Page

It is a KDE goal to be All About the Apps to deliver our apps directly to our users. Snaps is one of the ways to do this. Snaps are Linux app packages that can run on pretty much any Linux distro. There is a single centralised Snap store that delivers them to users. Take a look at the KDE page on the Snap Store to see what's available.

Snap intro

A Snap package typically contains all the files, including libraries and data files, to run the app. There are also Content Snaps which contain reuseble libraries. In KDE land we have the KDE Frameworks Content Snap which includes recent Qt and KDE Frameworks and this is shared between all KDE apps so we do not have to waste disk space and build resources.

Give it a try by installing a package or two on your system

snap install kcalc

And run kcalc from your apps menu.

This will have downloaded the kcalc Snap package from the Snap store into e.g. /var/lib/snapd/snaps/kcalc_73.snap and mounted it into e.g. /snap/kcalc/current/. You can also just download it to a local directory with snap download kcalc, use lesspipe kcalc*snap to see what it inside it.

snap list will show your currently installed snaps and it will now show that you have kcalc and the content snap kde-frameworks-5-qt-5-15-core20 as well as the core20 content snap installed.

Snaps are containers, similar to Docker. From inside the Snap container access to the file system and system resources are limited. This is good for inter-app security but means the app sees your system quite differently from how it might expect. You can "log" into the container with snap run --shell kcalc to have a look at how the Snapped kcalc app sees your system.

To give the app controlled permissions to the system it plugs connections into resources such as the network or container snaps. Run snap connections kcalc to see what it gets given access to. The connections are controlled by the store and app maintainers need to ask the store to apply the auto-connections. They can also be overridden locally.

You can take a look at the snap package with snap download kcalc which will download files such as kcalc_73.assert and kcalc_73.snap. The .assert has the checksums and signatures for the package. The .snap has the (non-store) metadata and all the files of the package. lesspipe kcalc_73.snap to take a look.

Concepts

Snaps are usually one app per Snap package. The Snap package contains all the libraries and resources it needs to run except those in the shared content kde-frameworks Snap.

In practice this means all of Qt and KF5 including Breeze icons and themes are in the kde-frameworks content Snap and your app Snap only needs to compile its own sources. If you apps needs other libraries it can either install these as Apt packages from the Ubuntu or KDE neon or other repository, or it can compile them from source as well. You will need to manually list the build-packages (all the -dev packages) and the stage-packages used in the final package, it'll warn you if any final libraries it expects are missing.

Snapcraft is used to build snaps. It can be installed as a snap with snap install snapcraft --classic. Snap packages are defined with snapcraft.yaml files. Snapcraft will build them inside a virtual machine, we use LXD to build the KDE ones (the default is to use Multipass another virtual machine manager but that has problems on cloud machines). Using a virtual machine makes it reliable to build the Snaps on any system with identical results.

snapcraft.yaml files are kept in either KDE neon git repositories or in the apps repository. They can be built on the KDE neon Jenkins CI. All KDE Developers have access to all these git repos and use of the Jenkins CI. Stable versions are kept in the Neon/release branches or the stable KDE git branch of your app. Unstable versions in Neon/unstable or the unstable branch of your app (usually master).

Our Snaps read metadata from AppStream metadata files so it is important the metadata is up to date including current release versions.

The Snap Store is the centralised app store by Canonical. There is no practical way to use other stores or repositories with Snaps. It is what Snapcraft uploads built snaps to and what your local snapd will download and install snaps from. It also says what permissions those snaps should have. As an app developer if you want your app to have extra permissions (for example kdf uses mount-observe) then you need to ask for it on the snapcraft forum.

A Classic containment Snap has no restrictions on what files it can see on your system or what external executable can be run. This is useful for IDEs and similar apps such as Kate which runs external programs. Again this needs to be set in your snapcraft.yaml then you need to ask on the Snap forum for the store to set it to classic. The Store will then tell snapd for anyone installing the Snap to have it installed as a Classic confinement Snap.

There is a KDE account on the Snap store which is run by the KDE neon developers Jonathan Riddell and Harald Sitter. One Snap on the store can be shared between more than one account so app maintainers can also create a separate account if they want to have more control over when their app is released and what the store says about it.

The store has four channels for different levels of stability. Our stable branch builds get uploaded to the Candidate channel and can be moved to the Stable channel once tested.

Example

Blinken is an exciting memory game from KDE. It's available on the Snap store. The Snap package is defined by a snapcraft.yaml file which is in the Neon/release branch of KDE neon's Blinken packging. Any update to that branch triggest a build of the Blinken Snap job in KDE neon's Jenkins builder. If the build is successful it will be uploaded to the Candidate channel of the Snap store ready for review.

The snapcraft.yaml file looks like this:

---
name: blinken
confinement: strict
grade: stable
base: core20
adopt-info: blinken
apps:
    blinken:
        extensions:
        - kde-neon
        common-id: org.kde.blinken.desktop
        command: usr/bin/blinken
        plugs:
        - home
        - network
        - network-bind
        - audio-playback
        - removable-media
slots:
    session-dbus-interface:
        interface: dbus
        name: org.kde.blinken
        bus: session
package-repositories:
-   type: apt
    components:
    - main
    suites:
    - focal
    key-id: 444DABCF3667D0283F894EDDE6D4736255751E5D
    url: http://origin.archive.neon.kde.org/user
    key-server: keyserver.ubuntu.com
parts:
    blinken:
        plugin: cmake
        build-packages:
        - libkf5doctools-dev
        - libphonon4qt5-dev
        - libphonon4qt5experimental-dev
        source: http://download.kde.org/stable/release-service/20.12.3/src/blinken-20.12.3.tar.xz
        cmake-parameters:
        - "-DKDE_INSTALL_USE_QT_SYS_PATHS=ON"
        - "-DCMAKE_INSTALL_PREFIX=/usr"
        - "-DCMAKE_BUILD_TYPE=Release"
        - "-DENABLE_TESTING=OFF"
        - "-DBUILD_TESTING=OFF"
        - "-DKDE_SKIP_TEST_SETTINGS=ON"
        - "-DCMAKE_FIND_ROOT_PATH=/usr\\;/root/stage\\;/snap/kde-frameworks-5-qt-5-15-core20-sdk/current"
        parse-info:
        - usr/share/metainfo/org.kde.blinken.appdata.xml
        filesets:
            exclusion:
            - "-usr/lib/*/cmake/*"
            - "-usr/include/*"
            - "-usr/share/ECM/*"
            - "-usr/share/doc/*"
            - "-usr/share/man/*"
            - "-usr/share/icons/breeze-dark*"
            - "-usr/bin/X11"
            - "-usr/lib/gcc/x86_64-linux-gnu/6.0.0"
            - "-usr/lib/aspell/*"
        prime:
        - "$exclusion"

Check Snapcraft YAML reference if unsure.

Top Level

  • name: blinken ← the snap name registered on the snap store
  • confinement: strict ← Snaps are a containerised format and can't see the outside system from inside their container. Strict is the normal container method. Classic is also possible which allows it to see the outside system and is used by e.g. Kate because Kate needs to run external programs like Git. It can only be Classic on request. Can also be devmode for testing.
  • grade: stable ← It must be stable to be in a released channel, can also be devel.
  • base: core20 ← which base system to build on, core20 means Ubuntu 20.04 and is the current recommended.
  • adopt-info: blinken ← Which Snap part to get the appstream info from. This sets version, icon, description.

You might also need to add version if it is not in the appstream file. This is just a version read by users it does not affect the revision number which is tracked by the store.

apps

apps:
    blinken:
        extensions:
        - kde-neon
        common-id: org.kde.blinken.desktop
        command: usr/bin/blinken
        plugs:
        - home
        - network
        - network-bind
        - audio-playback
        - removable-media

apps are the programs which the snap includes for users to run. Usually there is only one in a Snap but sometimes e.g. Calligra there are more than one.

The KDE neon extension adds some commonly used features to the KDE snaps including using the KDE Frameworks 5 content Snap.

The common-id comes from the Appstream file. You must check what it is in the appstream file. org.kde.blinken.appdata.xml contains <id>org.kde.blinken.desktop</id> so we use that. Sometimes apps use the .desktop and sometimes they don't, this is at random.

The command to run is listed. The KDE neon extension will run a script first which sets many necessary environment variables.

The plugs give access to the outside system, see Supported interfaces for descriptions. When a Snap is installed from the Store it is up to the Store to say which plugs get used. Thost listed as auto connect in the docs are permitted. Otherwise you must ask on the Snap forum for permission to have the Snap connected. (Locally installed snaps with --devmode have access to everything, you can also manually connect snaps to interfaces on your local system.)

slots are the complement to plugs, they allow the outside system to access our Snap app. In this case we are allowing a dbus interface into the Snap. All KDE apps have a dbus interface and you can check what it is called by running the app and using qdbus.

package-repositories add the KDE neon apt repo to build against, this will give you the latest libraries to compile with.

The source of a Snap is the parts and some snaps have several parts made of different sources e.g. KTorrent has both libktorrent and ktorrent parts. Blinken is not complex so it has only one part made of the compiles Blinken source.

Parts

  • plugin ← which Snap build plugin to use
  • build-packages ← most build packages are in the KDE Frameworks content snap but some need added explicitly and some are not in there. They will be downloaded from the neon and ubuntu apt repos. KTorrent uses non-KDE libraries and it needs to list the -dev packages in the build-packages then the library itself in the stage-packages.
  • source ← link to the tar
  • cmake-parameters ← copy and paste this, it sets the right paths.
  • parse-info ← where the appstream file is to be installed
  • filesets and prime ← snap parts get build then copied into a stage area, when all the parts are built they are copied into the prime area which is converted into the Snap package. This lists a common set of excluded files we do not want copied. You can add more here if you end up with unnecessary files in your snap.

Building

Install snapcraft with snap install snapcraft --classic.

In the directory with the snapcraft.yaml run: snapcraft --enable-experimental-package-repositories --enable-experimental-extensions --use-lxd

This will start a virtual machine and build the package. If all is well you will have blinken_20.12.3_amd64.snap or similar created.

Install with snap install --devmode blinken_20.12.3_amd64.snap

Run with snap run blinken or check it is in the app menu and run from there (remove any versions of blink you have installed from your normal distro packages just to be sure).

Quirks

alsa

Some KDEGames use libkdegames's KgSound class which uses libopenal which uses libsnd which uses alsa. (Most other KDE software uses QtMultimedia or Phonon which uses Pulseaudio.) See KBlocks for one way to make this work. Use the alsa extension, use layers to move files around, exclude the pulse alsa file.

Qt Only

You may want to simplify your Snap by using Qt directly instead of the KDE neon extension and KDE Frameworks content Snap. Good luck :)

Patches

If you need to update some code in the release you can patch it in the Snap package. But please get the patch upstream into the Git archive first.

Falkon does this.

Help

Snapcraft docs including tutorials on using and building snapcraft.yaml format Snap forum for asking for help or asking to get the store to allow your Snaps to auto connect. KDE neon devs talk to Riddell or Sitter for help getting your app into KDE neons builds and into the Store.

Glossary

Words you'll hear and not know what they mean:

  • snap: The actual package format.
  • snapd: The daemon that manages snap on a system.
  • snapcraft: The build tool for building a snap.
  • 'app: In the context of snapcraft/snapd this is the (portable) description of an 'executable' exposed to the outside (i.e. something snapd knows how to run).
  • parts: In the context of snapcraft a part refers to one build entity. They describe where to get the source of the entity, how to build it, how to stage it into the final snap and which other parts are a dependency and need to be built first. A part is much like a "makefile" target.
  • interfaces: A way for a snap to talk to the outside world (or another snaps). Split into slots and plugs. Each of which has their own security permissions as a client may need to be able to do different things from a server. https://docs.snapcraft.io/interface-management
  • slot: The provider part of an interface. e.g. a kwin snap might have a wayland-client slot which exposes a way for clients to talk to kwin.
  • plug: The client part of an interface. e.g. an application may plug into the wayland-client slot of kwin to talk to it.
  • Core: A special snap containing the core aspects of any Linux OS (libc/libpthread/...). All snaps depend on exactly one core which provides the snap's understanding of what will be in "/" from the snap's POV. The core does not include a kernel! Kernels may be snaps.
  • Content Snap: Special kind of snap that implements the "content" interface. It's kind of like a shared dependency between snaps allowing one snap to be bound into the scope of another snap. For example the KF5 content snap may be used to share all of KF5 across multiple snaps.
  • Build Snap: Also a special kind of snap, it's the build-time variant of the Content Snap and contains header files etc. necessary to build against a Content Snap.
  • stage, staging: As part of snapcrafting parts get "staged". This kind of means the same as make install, but it's actually a separate step after make install. For the process of staging, snapcraft will copy all files created by make install into a stage directory. You may also exclude certain files or reorganize the files (e.g. rename, or move to different directory). The stage is available for parts ordered after the current one, meaning that they for example can link against a newly built library.
  • prime, priming: Is similar to staging but happens once all parts are built and staged. Priming is the process by which the snap tree is actually constructed. Priming, like staging, allows for excluding files (e.g. dev headers may be staged so other parts can build using them but later excluded from priming and thus left out of the final bundle).