Kexi/Porting to Qt&KF 5: Difference between revisions

From KDE Community Wiki
Line 176: Line 176:
  git filter-branch --prune-empty -f --tree-filter \
  git filter-branch --prune-empty -f --tree-filter \
  '(mkdir -p src;mv Predicate src/predicate;mv Drivers src/drivers) 2> /dev/null || true' -- HEAD
  '(mkdir -p src;mv Predicate src/predicate;mv Drivers src/drivers) 2> /dev/null || true' -- HEAD
3. Before your push your branch to a new repo, you may need some postprocessing if the commits contain non-Unix line endings, git push refuses to work then:
3. Before your push your branch to a new repo, you may need some '''postprocessing''' if the commits contain non-Unix line endings, git push refuses to work then:
  remote: Audit failure - Commit 9460d32f6f2f51c2a182b8faa40ff7537b7ac5c6 - End of Line Style (non-Unix)
  remote: Audit failure - Commit 9460d32f6f2f51c2a182b8faa40ff7537b7ac5c6 - End of Line Style (non-Unix)


To remove the non-Unix line endings (see also [http://blog.gyoshev.net/2013/08/29/normalizing-line-endings-in-git-repositories/]):
To remove the non-Unix line endings (see also [http://blog.gyoshev.net/2013/08/29/normalizing-line-endings-in-git-repositories/]).


3.1. Create this script and call it ''fix-eol.sh'':
3.1. Create this script and call it ''fix-eol.sh'':
Line 187: Line 187:
(the ''find'' and ''grep'' commands here try very hard to find text files)
(the ''find'' and ''grep'' commands here try very hard to find text files)


3.2. Run the actual rewrite:
3.2. Run the actual rewrite. Note, we call this ''postprocessing'' because it's time-consuming so it's worth to run it after the step #2 only on files that we really need in the edited repo.
  cd /path/to/the/repo
  cd /path/to/the/repo
  git filter-branch --tree-filter fix-eol.sh  -- --all
  git filter-branch --tree-filter fix-eol.sh  -- --all

Revision as of 14:52, 16 December 2015

Note

Draft. Based on notes from the Calligra 2014 Sprint. Each step leads to compiling (running?) state.
Started by jstaniek (talk) 19:06, 7 July 2014 (CET).


The grand plan:

  1. [100%, builds, example and autotests work] Port PredicateKDb↓ lib to Qt5/KF5
    1. [100%] Port the SQLite driver
    2. [100%] Port the MySQL driver
    3. [10%] Port the PostgreSQL driver
    4. [10%] Port the xBase driver
    5. [10%] Port the Sybase driver
  2. [100%, builds, example and autotests work] Move koproperty lib from calligra to a separate KProperty↓ repo, port to Qt5/KF5
  3. [100%, builds, example and autotests work] Move koreport lib from calligra to a separate KReport↓ repo, port to Qt5/KF5
    1. Remove scripting or port scripting?
  4. [75%] Port Kexi (without KexiDB/calligradb, reporting, koproperty, koreport) to Qt 5 and use KDb/KProperty/KReport
    1. [100%] Remove unused calligradb lib and parts of KexiDB lib
    2. [92%] Port the Table plugin
    3. [92%] Port the Query plugin
    4. [90%] Port the Form plugin
    5. [90%] Port the Report plugin
    6. [50%] Port the Import/Export plugin
    7. [20%] Port the Migration module
    8. [0%] Reimplement report scripting using a simple solution to keep backward compatibility with Kexi 2 (see Scripting::*, KRScriptConstants, KRScriptDraw, KRScriptDebug)
  5. [35%] Improve Kexi stability after porting
  6. -- Kexi 3.0 beta release here. To indicate the nature of this version we never release 3.0 stable. --
  7. Add tests and/or examples to KReport, make it easier to understand as a framework
    1. Port some code from Kexi Reports (e.g. KexiReportView [1]) to KReport
  8. Make KReport lib dependent on KDb and remove code/API that becomes redundant (e.g. KoReportData)
  9. Add a new scripting module
  10. Use the module in Kexi Reports too, keeping backward compatibility with Kexi 2 scripting (add unit tests before porting to be very sure that compatibility is kept)

Rules

  • To build Kexi, first build & install KDb, KProperty, KReport (master branches), i.e. for each:
cd <<BUILDDIR>>
cmake -DPRODUCTSET=kexi -DCMAKE_INSTALL_PREFIX=<<PREFIX>> -DCMAKE_BUILD_TYPE=Debug <<SRCDIR>>
  • Then use calligra.git 'master' branch: type "git checkout master".
  • Use personal git scratch repos to avoid stress with buildbreaks and unnecessary BUG/review notifications
  • Boud's advice: My approach is more like, run each of montel's scripts, fix what's wrong, then go on. It's not something that's easy to paralellize.
  • Don't run astyle on the code, it pollutes history, and astyle-kdelibs is a bit broken
  • Based on "Porting notes" below, keep porting of one aspect in one commit to simplify reviews
  • Use the Phabricator for reviews
  • Use kde-dev-scripts.git porting scripts, in particular
  • Libs that are separate repos should use the framework template (see Links) and be ported to this layout (for now it's just kdb, kproperty, kreport).

TODO...

Notes and Status

  • TODO: publish detailed checklist (in a spreadsheet?)

Dependencies

Please read all this section carefully before taking action.

  • Qt 5. Get >=5.2 stable version from your distro, not forgetting about development packages (headers). If you feel you want it, build Qt 5, but it's usually not necessary extra work.
  • CMake: version >=2.8.12+ and 3.x are both supported. Get it from your distro (recommended) or build it.
  • KDE Frameworks 5 (KF5) are needed as build-time and run-time dependencies for Calligra, Kexi and our frameworks (KProperty, etc).
  • Use the kdesrc-build tool to conveniently build KF5. Building by hand in an option if you accept extra work usually not needed for our development (see Frameworks/Building/Details).
  • There is the Adjust the paths step. For Kexi development (and Calligra in general) it's good to re-use paths that we already use except the install prefix (kdedir), see the Recommended setup of the Calligra build instructions. This means that recommendation for the kdesrc-buildrc config file is:
 source-dir /home/YOU/kde/src
 build-dir /home/YOU/kde/build
 kdedir /home/YOU/kde/inst5

Note

If you used /home/YOU/kde/inst as a dir the Calligra 2.x and other kdelibs4-based software, do not mix it with the Calligra 3.x+ and other kf5-based software, otherwise you can experience crashes or other strange behaviour. Use /home/YOU/kde/inst5 for example.


  • Speed up: before using the KF5 with kdesrc-build, edit the kf5-qt5-build-include file and comment these lines if you don't want to build the workspace, applications and KDEPIM. This way only include kf5-frameworks-build-include lefts.

include kf5-workspace-build-include include kf5-applications-build-include include kf5-kdepim-build-include

  • In particular, the kdesrc-build will install extra CMake Modules: [2], which is needed to make our software able to find the KF5 components at build time.

KDb

  • Download/Build/Release info → KDb.
  • Use master branch, official for Qt 5/KF 5, for all platforms, always stable
  • Our target version is 3.0.0.
  • To avoid conflicts, Qt 4 version kept for reference, has version 0.0.0.
  • Depends only on system libraries and Qt, so it's a tier 1 framework
  • Windows and Mac-specific matters.
  • Porting to Qt 5 started by Wojtek Kosowicz (Jan 2015)
  • Porting to Qt 5/Mac OS X started by Wojtek Kosowicz too (Jan 2015)
  • Directory structure reorganized according to recommendations --Jstaniek (talk) 20:59, 23 February 2015 (UTC)

KProperty

  • Our target version is 3.0.0 and KProperty project in Playground/Libs category of KDE projects
  • Depends on system libraries and Qt, plus minor deps on KF5WidgetsAddons/KF5GuiAddons for some editors; later we may move this dependency to a plugin (so the core would stay Qt-only)
  • Official git repo: kproperty, master branch (use: git clone kde:kproperty)
  • Directory structure reorganized according to recommendations --Jstaniek (talk) 20:59, 23 February 2015 (UTC)

KReport

  • Our target version is 3.0.0 and KReport project in Playground/Libs category of KDE projects
  • To avoid conflicts, Qt 4 version kept for reference, has version 0.0.0.
  • Depends on system libraries, Qt, and KProperty (tier 1), so it's a tier 2 framework, some deps on KF5WidgetsAddons (may be moved to a plugin so the core would stay Qt-only)
  • Official git repo: kreport, master branch (use: git clone kde:kreport)
  • Directory structure reorganized according to recommendations --Jstaniek (talk) 20:59, 23 February 2015 (UTC)
  • Deps from calligra libs (copied, renamed): KoRuler, KoUnit

Build instructions

Kexi 3/Calligra 3 builing is explained in a (work-in-progress) doc: Calligra/Building/3. Our dependencies (KDb, KPropert, KReport) can be built in a usual way (cmake, make install). Don't forget to set environment variables in the same way as you do for Kexi/Calligra builds.

Porting scripts usage status

These porting scripts from the kf5 dir of the kde:kde-dev-scripts repo would be applied.
Legend: - = not run, + = run, N = not applicable, T = TODO(we know it's needed)

KDb | KProperty | KReport | Kexi | script-name
+ + + + adapt_cmakelists_file.pl
+ + + + adapt_knewstuff3_includes.pl
+ + + + add_missing_kpart_include.pl
+ + + + clean-forward-declaration.sh
+ + + + clean-includes.sh
N N N - cmakelists_install_vars.pl (N/A because KDb/KProperty/KReport don't use KDE_*DIR variables)
N N N N convert-kabc-to-kcontacts.pl (unused)
+ + + + convert-kaction.pl
+ + + - convert-kbuttongroup.pl
T + + + convert-kcmdlineargs.pl
+ + + N convert-kcolordialog.pl
+ + + + convert-kdebug.pl
N N N N convert-kdebug-with-argument.sh (N/A because KDb/KProperty/KReport don't use that)
N + N + convert-kdialog.pl (helps but KProperty is largery broken by this script)
N N N N convert-kdoublenuminput.pl
N N N N convert-kfiledialog.pl
N N N N convert-kfontcombobox.pl
N N N T convert-kicon.pl
N N N N convert-kimageio.pl
N N N N convert-kintnuminput.pl
N N N N convert-kintspinbox.pl
N N N N convert-kjob.pl
N N N + convert-klineedit.pl
N N N + convert-klistwidget.pl
N N N N convert-klocale-formatdate.pl
N N N + convert-kmd5.pl
N N N N convert-kmenubar.pl
N N N + convert-kmenu.pl
N N + + convert-kmimetype.pl
N N N + convert-kprogressdialog.pl
N N N + convert-kpushbutton.pl
N N N + convert-ksavefile.pl
N N N + convert-ksharedptr.pl
N N N + convert-kshortcut.pl
N N N N convert-ksplashscreen.pl
N N N + convert-kstandarddirs.pl
N N N + convert-ktabwidget.pl
N N N + convert-ktempdir.pl
+ N + + convert-ktemporaryfile.pl
N N N + convert-ktextbrowser.pl
N + + + convert-kurl.pl
N N N + convert-kvbox.pl
+ + + - convert-qkdebug-to-qcdebug.sh
+ + + + convert-qt5.pl
+ + + + convert-to-cmake-automoc.pl
T N N T convert-to-k4aboutdata.pl
- - - - convert-to-new-signal-slot-signal.pl
N N N N dnssd-to-kdnssd.sh
+ + + + fix-ecm-install-icons.pl
- - - - fix-kcm-module-desktop-file.sh
- - - - install_forwarding_headers.pl
N N N N port-kauthactions.pl
N N N + port_to_autogenerate_export_header.sh
+ + + + remove-kde4support.pl
N N N N remove_virtual_prefix.pl
N N N + resolve_kuit.py
T T T T search-kdelibs4support-header.sh
+ + + + frameworks/plasma-framework/src/tools/port-cmake-style.sh
+ + + - frameworks/plasma-framework/src/tools/port-qslots.sh
+ + + - frameworks/plasma-framework/src/tools/port-includes.sh

Git surgery

1. For editing large history you may need a fast filesystem. So a Linux RAM disk such as /dev/shm is your friend, deep-copy your repo temporarily there: [3].

  • To deep-copy use:
cd /path/to/ramdisk git clone /path/to/the/orig/repo --no-hardlinks --no-local
  • Consider switching off swap ('sudo swapoff -a') as swapping makes no sense, when we want to use RAM exclusively.
  • Don't forget to create a backup of the original repository and then replace the original repository with the copy edited, using git clone again:
cd /path/to/the/orig
rm -rf repo
git clone /path/to/ramdisk/repo --no-hardlinks --no-local

2. (Jstaniek (talk) 09:54, 23 February 2015 (UTC)) Repos are prepared by history rewrites such as:

git filter-branch --prune-empty -f --tree-filter \
'(mkdir -p src;mv Predicate src/predicate;mv Drivers src/drivers) 2> /dev/null || true' -- HEAD

3. Before your push your branch to a new repo, you may need some postprocessing if the commits contain non-Unix line endings, git push refuses to work then:

remote: Audit failure - Commit 9460d32f6f2f51c2a182b8faa40ff7537b7ac5c6 - End of Line Style (non-Unix)

To remove the non-Unix line endings (see also [4]).

3.1. Create this script and call it fix-eol.sh:

#!/bin/bash
find . -type f -exec grep -Iq . {} \; -and -print | xargs dos2unix -k -q

(the find and grep commands here try very hard to find text files)

3.2. Run the actual rewrite. Note, we call this postprocessing because it's time-consuming so it's worth to run it after the step #2 only on files that we really need in the edited repo.

cd /path/to/the/repo
git filter-branch --tree-filter fix-eol.sh  -- --all

Links

Docs

Background