Akademy/2023/Jenkins Retirement BoF

From KDE Community Wiki
< Akademy‎ | 2023
Revision as of 09:42, 18 July 2023 by Vkrause (talk | contribs) (Created page with "= Meeting Notes = ==== Craft Caches ==== * building a cache on Gitlab isn't the problem but rather how to harvest and publish the cache * Windows cache is signed, so we are also stuck on the signing problem here ** this isn't a security issue but a performance optimization for signing the final installers ** we could skip this for now * where should the cache builds live? sysadmin/ci-management ** somewhat similar to the seed job, similar regarding access rights, so let...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Meeting Notes

Craft Caches

  • building a cache on Gitlab isn't the problem but rather how to harvest and publish the cache
  • Windows cache is signed, so we are also stuck on the signing problem here
    • this isn't a security issue but a performance optimization for signing the final installers
    • we could skip this for now
  • where should the cache builds live? sysadmin/ci-management
    • somewhat similar to the seed job, similar regarding access rights, so let's use that
  • does this need extended build times?
    • average Android cache build job runtime is <= 20min, but those are incremental
    • even timeout is not blocking, we just run the job incrementally
    • by using sysadmin/ci-management we already get extended job runtimes (up to 6h)
  • Ben to set up an ssh key for craft cache updates
    • done, $CRAFT_UPLOAD_KEY env var contains the key for uploading, account is locked to sftp upload
  • Hannah to set up a craft cache job there for testing, starting with Android Qt6 as that wont break anything

Craft build logs

  • Gitlab 500kB log limit is hard to change
  • Craft should put extended logs into Gitlab artifact
    • Option 1: tee the output into a file, let's you see the full output below 500kB right away, but nothing beyond that until the build is done
    • Option 2: kdesrc-build like behavior: let's you see the progress, will require downloading the log to see any error

WebEngine

  • Linux WebEngine build kills other VMs
  • what's the maximum memory use for the WebEngine build?
  • special handling for WebEngine would needs awkward splits of the cache build jobs due to this being in the middle of the dependency chain
  • would extra hardware solve this? we can upgrade the worker to 128GB RAM for limited amount of money

Signing

  • Ingo working on our own solution as the existing solutions didn't work for us
  • using a job queue on the signer that the Gitlab workers upload to
  • results are uploaded to another queue that the workers then poll
  • currently in Ingo's Invent namespace, needs to move to the main repos
  • also going to be used for uploads and website publishing, which have similar security requirements
  • implements APK and APPX signing
  • remaining issue is how to limit who can feed things into this
  • Gitlab protected variables would solve this: those are set on repos/groups and only on protected branches (master, stable release branches),
    • they would contain the credentials for uploading to the signing service
    • this is already in use for CI artifact uploads
  • appimage signing currently done but yet to be implemented in the new system

Package downloads/Status dashboard

  • this is also (implicitly) provided by Jenkins
  • we link to random artifact for downloads on Jenkins, going to be less obvious with Gitlab artifacts
    • sometimes due to lacking proper releases
  • Jenkins also gives us a full overview status page
  • we also need proper solutions for all that, but that's a separate discussion