Neon/Git: Difference between revisions

From KDE Community Wiki
No edit summary
 
(18 intermediate revisions by 3 users not shown)
Line 1: Line 1:
The Neon Git repositories are at http://packaging.neon.kde.org.uk
The Neon Git repositories are at https://invent.kde.org/neon


All KDE contributors can commit to neon* branches or repositories under neon/ directory, it will reject commits to other branches.
All KDE contributors can commit to Neon/* branches or any branch for repositories under neon/, forks/, neon-packages/ directories, it will reject commits to other branches.


They are clones of the repositories used by Debian pkg-kde team http://anonscm.debian.org/cgit/pkg-kde/.  Any commits made to a Debian pkg-kde repository will be pulled into Neon Git immediately.
They are clones of the repositories used by Debian pkg-kde team https://salsa.debian.org/qt-kde-team/.  Any commits made to a Debian pkg-kde repository will be pulled into Neon Git immediately.


Add this to <code>~/.gitconfig</code>
Add this to <code>~/.gitconfig</code>
{{Input|1=<nowiki>
{{Input|1=<nowiki>
[url "git://anongit.neon.kde.org/"]
[url "git@invent.kde.org:"]
     insteadOf = neon:
     insteadOf = invent:
[url "neon@git.neon.kde.org:"]
[url "git@invent.kde.org:"]
     pushInsteadOf = neon:
     pushInsteadOf = invent:
</nowiki>}}
</nowiki>}}
and clone with:
and clone with:
{{Input|1=<nowiki>git clone neon:plasma/khotkeys</nowiki>}}
{{Input|1=<nowiki>git clone invent:neon/kde/khotkeys</nowiki>}}


=== Anonymous Access ===
=== Anonymous Access ===


If you don't have a KDE account you can still clone the repo at
If you don't have a KDE account you can still clone the repo at
{{Input|1=<nowiki>git clone git://anongit.neon.kde.org/plasma/khotkeys</nowiki>}}
{{Input|1=<nowiki>git clone https://invent.kde.org/neon/kde/khotkeys</nowiki>}}


=== Admin ===
=== Settings up new repositories ===
 
The repository <code>neon:gitolite-admin</code> has gitolite setup, hooks and hook server for those with admin rights (kde sysadmin does).
 
neon-hooks/pre-receive is a hook to check the branch can be committed to
 
debian-hooks/debian-to-neon-post-receive is a hook for debian pkg-kde git repos to call the neon server on updates


server/neon-fetch-request-server.py is an XML-RPC server run on the neon server which is called by the debian hook and fetches updates
This is now done through invent.kde.org which can also be set to clone another git repo


sync-repos-from-debian.py checks for all the repositories on in debian pkg-kde and adds any missing to neon.  This gets run nightly on Jonathan's server ''embra''.
=== Pushing merges ===


It uses gitolite3 on weegie.edinburghlinux.co.uk, a server used by Neon and Kubuntu and administrated by Jonathan Riddell and KDE sysadmin.
KDE git has a hook to prevent 100s of commits in one push but sometimes that is what we need if we are merging in from e.g. Debian. Push with
{{Input|1=<nowiki>git push -o notification.summarise</nowiki>}}
This will also disable some check with backports-focal etc for when syncing with Debian and their commits have translations and funny e-mail addresses that pre-commit hooks would block.


Web interface is cgit.
=== Archiving Repos ===
 
=== Settings up new repositories ===


To set up a new repository it is nice to make it first on the Debian server so the syncing is already set up and Debian/Kubuntu packagers know to use it.
Archiving/removing of a repo can only be done by sysadmins you have to [https://phabricator.kde.org/maniphest/task/edit/form/2/ file a ticket]. Additionally to hide the projects immediately from the CI tooling you can add the topic 'neon-archived' in the general projects settings on gitlab (obviously only if you have neon admin access) e.g https://invent.kde.org/neon/neon-packaging/plasma-systemmonitor/edit
* Ask someone with access (Jonathan etc) to <code>ssh git.debian.org; cd /git/pkg-kde/place-to-put-it; ../setup-git-repository <new-repo></code>
* Then wait 15 minutes while the website for git.debian.org syncs the new repo
* Then run <code>./sync-repos-from-debian.py</code> (just tell Jonathan to run it on his embra server)
* Make a Neon/unstable branch on the neon git repo and get a Jenkins admin to run <code>pangea-tooling jenkins_jobs_update_nci.rb</code>


=== The Set Up===
=== The Set Up===
Line 51: Line 41:
* neon-hooks/pre-receive rejects pushes to non-neon branches and sets up remote to Debian if it exists
* neon-hooks/pre-receive rejects pushes to non-neon branches and sets up remote to Debian if it exists
* neon/website and neon/releases.neon.kde.org have a post-receive hook to update their websites (no longer used for websites at least)
* neon/website and neon/releases.neon.kde.org have a post-receive hook to update their websites (no longer used for websites at least)
* sync-repos-from-debian.py is run nightly on an external server to sync any new repos from Debian pkg-kde (probably no need to be external user, it does need to ssh into the server to remove repositories)
* cron on the gitolite server runs bits to restart the servers and run the check scripts:  
* cron on the gitolite server runs bits to restart the servers and run the check scripts:  
{{Input|1=<nowiki>
{{Input|1=<nowiki>
* * * * * /usr/bin/flock -n /tmp/server.lockfile /var/lib/gitolite3/neon-fetch-request-server.py
# Hourly sync of new repositories from salsa.debian.org
* * * * * /usr/bin/flock -n /tmp/gitdaemon.lockfile git daemon --reuseaddr --base-path=/var/lib/gitolite3/repositories /var/lib/gitolite3/repositories
@hourly python3 ~/config-sync/gitolite-admin/maintenance/update-gitolite-setup.py
20 3 * * * /var/lib/gitolite3/check-symlink-repos.py
# Daily full sync of all repositories from salsa.debian.org
40 3 * * * /var/lib/gitolite3/check-repo-remotes.py
@daily python3 ~/config-sync/gitolite-admin/maintenance/update-gitolite-setup.py --fetch-all-repositories
</nowiki>}}
</nowiki>}}

Latest revision as of 13:46, 18 October 2021

The Neon Git repositories are at https://invent.kde.org/neon

All KDE contributors can commit to Neon/* branches or any branch for repositories under neon/, forks/, neon-packages/ directories, it will reject commits to other branches.

They are clones of the repositories used by Debian pkg-kde team https://salsa.debian.org/qt-kde-team/. Any commits made to a Debian pkg-kde repository will be pulled into Neon Git immediately.

Add this to ~/.gitconfig

[url "[email protected]:"]
    insteadOf = invent:
[url "[email protected]:"]
    pushInsteadOf = invent:

and clone with:

git clone invent:neon/kde/khotkeys

Anonymous Access

If you don't have a KDE account you can still clone the repo at

git clone https://invent.kde.org/neon/kde/khotkeys

Settings up new repositories

This is now done through invent.kde.org which can also be set to clone another git repo

Pushing merges

KDE git has a hook to prevent 100s of commits in one push but sometimes that is what we need if we are merging in from e.g. Debian. Push with

git push -o notification.summarise

This will also disable some check with backports-focal etc for when syncing with Debian and their commits have translations and funny e-mail addresses that pre-commit hooks would block.

Archiving Repos

Archiving/removing of a repo can only be done by sysadmins you have to file a ticket. Additionally to hide the projects immediately from the CI tooling you can add the topic 'neon-archived' in the general projects settings on gitlab (obviously only if you have neon admin access) e.g https://invent.kde.org/neon/neon-packaging/plasma-systemmonitor/edit

The Set Up

  • neon:gitolite-admin has hooks in .gitolite/hooks/common which are symlinks to neon-hooks/ that get added to new repositories
  • .gitolite/hooks/update is the normal hook from gitolite that does whatever gitolite needs
  • neon-hooks/post-receive pings jenkins to start a new build
  • neon-hooks/pre-receive rejects pushes to non-neon branches and sets up remote to Debian if it exists
  • neon/website and neon/releases.neon.kde.org have a post-receive hook to update their websites (no longer used for websites at least)
  • cron on the gitolite server runs bits to restart the servers and run the check scripts:
# Hourly sync of new repositories from salsa.debian.org
@hourly python3 ~/config-sync/gitolite-admin/maintenance/update-gitolite-setup.py
# Daily full sync of all repositories from salsa.debian.org
@daily python3 ~/config-sync/gitolite-admin/maintenance/update-gitolite-setup.py --fetch-all-repositories