Neon/Git: Difference between revisions

From KDE Community Wiki
No edit summary
No edit summary
(11 intermediate revisions by 2 users not shown)
Line 3: Line 3:
All KDE contributors can commit to Neon/* branches or any branch for repositories under neon/, forks/, neon-packages/ directories, it will reject commits to other branches.
All KDE contributors can commit to Neon/* branches or any branch for repositories under neon/, forks/, neon-packages/ directories, it will reject commits to other branches.


They are clones of the repositories used by Debian pkg-kde team http://anonscm.debian.org/cgit/pkg-kde/.  Any commits made to a Debian pkg-kde repository will be pulled into Neon Git immediately.
They are clones of the repositories used by Debian pkg-kde team https://salsa.debian.org/qt-kde-team/.  Any commits made to a Debian pkg-kde repository will be pulled into Neon Git immediately.


Add this to <code>~/.gitconfig</code>
Add this to <code>~/.gitconfig</code>
Line 13: Line 13:
</nowiki>}}
</nowiki>}}
and clone with:
and clone with:
{{Input|1=<nowiki>git clone neon:plasma/khotkeys</nowiki>}}
{{Input|1=<nowiki>git clone neon:kde/khotkeys</nowiki>}}


=== Anonymous Access ===
=== Anonymous Access ===


If you don't have a KDE account you can still clone the repo at
If you don't have a KDE account you can still clone the repo at
{{Input|1=<nowiki>git clone git://anongit.neon.kde.org/plasma/khotkeys</nowiki>}}
{{Input|1=<nowiki>git clone git://anongit.neon.kde.org/kde/khotkeys</nowiki>}}


=== Admin ===
=== Admin ===


The repository <code>neon@git.neon.kde.org:gitolite-admin</code> has gitolite setup, hooks and hook server for those with admin rights (kde sysadmin does).
The repository <code>neon:gitolite-admin</code> has gitolite setup, hooks and hook server for those with admin rights (kde sysadmin does). Don't use anongit to check this out, do an explicit <code>git clone [email protected]:gitolite-admin</code>


neon-hooks/pre-receive is a hook to check the branch can be committed to
<code>neon-hooks/pre-receive</code> is a hook to check the branch can be committed to


debian-hooks/debian-to-neon-post-receive is a hook for debian pkg-kde git repos to call the neon server on updates
<code>debian-hooks/debian-to-neon-post-receive</code> is a hook for Debian pkg-kde Git repos to ping the neon server on updates


server/neon-fetch-request-server.py is an XML-RPC server run on the neon server which is called by the debian hook and fetches updates
<code>server/neon-fetch-request-server.py</code> is an XML-RPC server run on the neon server which is called by the Debian hook and fetches updates from the Debian repos into the equivalent branches in our repos


sync-repos-from-debian.py checks for all the repositories on in debian pkg-kde and adds any missing to neon.  This gets run nightly on Jonathan's server ''embra''.
<code>maintenance/</code> scripts sync that all the Debian pkg-kde repos also exist on Neon Git.
 
<code>local/hooks/common/post-receive</code> pings the neon and DCI Jenkins servers to start a new build.


It uses gitolite3 on code.kde.org, administrated by the KDE sysadmin team.
It uses gitolite3 on code.kde.org, administrated by the KDE sysadmin team.


Web interface is phpgit.
Web interface is cgit.
 
Should you get a DuplicatedRepos (from DuplicatedRepos.Check) error during job updater because Debian has added a repo which was already in neon-packaging/ you will have to create Neon branches in the neon (aka neon:extras/coolpackage) sync of the Debian repo, remove the old repo from neon's gitolite conf, this should remove the repo automatically.


=== Settings up new repositories ===
=== Settings up new repositories ===


To set up a new repository it is nice to make it first on the Debian server so the syncing is already set up and Debian/Kubuntu packagers know to use it.   
To set up a new repository it is nice to make it first on the Debian server so the syncing is already set up and Debian/Kubuntu packagers know to use it.   
* Ask someone with access (Jonathan etc) to <code>ssh git.debian.org; cd /git/pkg-kde/place-to-put-it; ../setup-git-repository <new-repo></code>
* Ask someone in #debian-qt-kde to create a repo in salsa - provide a link to upstream source when asking.
* The repository should get automatically get picked up by neon infrastructure at most 60 minutes after creation
* The repository should get automatically get picked up by neon infrastructure at most 60 minutes after creation
* Make a Neon/unstable branch on the neon git repo and get a Jenkins admin to run <code>pangea-tooling jenkins_jobs_update_nci.rb</code>
* Make a Neon/unstable branch on the neon git repo and get a Jenkins admin to run <code>pangea-tooling jenkins_jobs_update_nci.rb</code>
To add a new repo directly to Neon under forks/ neon/ or neon-packaging/ ask an admin (Jonathan, Harald etc) to add it to the gitolite-admin conf/gitolite.conf file.  Then check it out directly from the master server, it won't be mirrored to the anongit server, e.g. <code>git clone [email protected]:forks/mustache-d</code> and add a commit to master.
If the repo is a fork of debian packaging outside of pkg-kde, then create a mirror of said repo.
To set up a mirror:
e.g.<code>git clone https://salsa.debian.org/science-team/kbibtex</code>
<code>cd kbibtex</code>
<code>git push --mirror neon:forks/kbibtex</code>
Then create neon branches as usual.


=== The Set Up===
=== The Set Up===
Line 52: Line 64:
* cron on the gitolite server runs bits to restart the servers and run the check scripts:  
* cron on the gitolite server runs bits to restart the servers and run the check scripts:  
{{Input|1=<nowiki>
{{Input|1=<nowiki>
# Hourly sync of new repositories from git.debian.org
# Hourly sync of new repositories from salsa.debian.org
@hourly python3 ~/config-sync/gitolite-admin/maintenance/update-gitolite-setup.py
@hourly python3 ~/config-sync/gitolite-admin/maintenance/update-gitolite-setup.py
# Daily full sync of all repositories from git.debian.org
# Daily full sync of all repositories from salsa.debian.org
@daily python3 ~/config-sync/gitolite-admin/maintenance/update-gitolite-setup.py --fetch-all-repositories
@daily python3 ~/config-sync/gitolite-admin/maintenance/update-gitolite-setup.py --fetch-all-repositories
</nowiki>}}
</nowiki>}}

Revision as of 15:02, 3 March 2020

The Neon Git repositories are at http://packaging.neon.kde.org

All KDE contributors can commit to Neon/* branches or any branch for repositories under neon/, forks/, neon-packages/ directories, it will reject commits to other branches.

They are clones of the repositories used by Debian pkg-kde team https://salsa.debian.org/qt-kde-team/. Any commits made to a Debian pkg-kde repository will be pulled into Neon Git immediately.

Add this to ~/.gitconfig

[url "git://anongit.neon.kde.org/"]
    insteadOf = neon:
[url "[email protected]:"]
    pushInsteadOf = neon:

and clone with:

git clone neon:kde/khotkeys

Anonymous Access

If you don't have a KDE account you can still clone the repo at

git clone git://anongit.neon.kde.org/kde/khotkeys

Admin

The repository neon:gitolite-admin has gitolite setup, hooks and hook server for those with admin rights (kde sysadmin does). Don't use anongit to check this out, do an explicit git clone [email protected]:gitolite-admin

neon-hooks/pre-receive is a hook to check the branch can be committed to

debian-hooks/debian-to-neon-post-receive is a hook for Debian pkg-kde Git repos to ping the neon server on updates

server/neon-fetch-request-server.py is an XML-RPC server run on the neon server which is called by the Debian hook and fetches updates from the Debian repos into the equivalent branches in our repos

maintenance/ scripts sync that all the Debian pkg-kde repos also exist on Neon Git.

local/hooks/common/post-receive pings the neon and DCI Jenkins servers to start a new build.

It uses gitolite3 on code.kde.org, administrated by the KDE sysadmin team.

Web interface is cgit.

Should you get a DuplicatedRepos (from DuplicatedRepos.Check) error during job updater because Debian has added a repo which was already in neon-packaging/ you will have to create Neon branches in the neon (aka neon:extras/coolpackage) sync of the Debian repo, remove the old repo from neon's gitolite conf, this should remove the repo automatically.

Settings up new repositories

To set up a new repository it is nice to make it first on the Debian server so the syncing is already set up and Debian/Kubuntu packagers know to use it.

  • Ask someone in #debian-qt-kde to create a repo in salsa - provide a link to upstream source when asking.
  • The repository should get automatically get picked up by neon infrastructure at most 60 minutes after creation
  • Make a Neon/unstable branch on the neon git repo and get a Jenkins admin to run pangea-tooling jenkins_jobs_update_nci.rb

To add a new repo directly to Neon under forks/ neon/ or neon-packaging/ ask an admin (Jonathan, Harald etc) to add it to the gitolite-admin conf/gitolite.conf file. Then check it out directly from the master server, it won't be mirrored to the anongit server, e.g. git clone [email protected]:forks/mustache-d and add a commit to master. If the repo is a fork of debian packaging outside of pkg-kde, then create a mirror of said repo. To set up a mirror: e.g.git clone https://salsa.debian.org/science-team/kbibtex cd kbibtex git push --mirror neon:forks/kbibtex Then create neon branches as usual.

The Set Up

  • neon:gitolite-admin has hooks in .gitolite/hooks/common which are symlinks to neon-hooks/ that get added to new repositories
  • .gitolite/hooks/update is the normal hook from gitolite that does whatever gitolite needs
  • neon-hooks/post-receive pings jenkins to start a new build
  • neon-hooks/pre-receive rejects pushes to non-neon branches and sets up remote to Debian if it exists
  • neon/website and neon/releases.neon.kde.org have a post-receive hook to update their websites (no longer used for websites at least)
  • cron on the gitolite server runs bits to restart the servers and run the check scripts:
# Hourly sync of new repositories from salsa.debian.org
@hourly python3 ~/config-sync/gitolite-admin/maintenance/update-gitolite-setup.py
# Daily full sync of all repositories from salsa.debian.org
@daily python3 ~/config-sync/gitolite-admin/maintenance/update-gitolite-setup.py --fetch-all-repositories