Kexi/Porting to Qt&KF 5: Difference between revisions

From KDE Community Wiki
(Update the path to build-include file)
 
(11 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{Note|Draft. Based on [http://mail.kde.org/pipermail/calligra-devel/2014-July/011603.html notes] from the [[Calligra/Meetings/Spring_2014_Sprint|Calligra 2014 Sprint]]. Each step leads to compiling (running?) state.<br/>Started by [[User:Jstaniek|jstaniek]] ([[User talk:Jarosław|talk]]) 19:06, 7 July 2014 (CET).}}
{{Note|Based on [http://mail.kde.org/pipermail/calligra-devel/2014-July/011603.html notes] from the [[Calligra/Meetings/Spring_2014_Sprint|Calligra 2014 Sprint]]. Each step leads to compiling (running?) state.<br/>Started by [[User:Jstaniek|jstaniek]] ([[User talk:Jarosław|talk]]) 19:06, 7 July 2014 (CET).}}
 
__TOC__


The grand plan:
The grand plan:
Line 5: Line 7:
## [100%] Port the SQLite driver
## [100%] Port the SQLite driver
## [100%] Port the MySQL driver
## [100%] Port the MySQL driver
## [10%] Port the PostgreSQL driver
## [100%] Port the PostgreSQL driver
## [10%] Port the xBase driver
## [10%] Port the xBase driver
## [10%] Port the Sybase driver
## [10%] Port the Sybase driver
# [100%, builds, example and autotests work] Move koproperty lib from calligra to a separate [[#kproperty|KProperty]]&darr; repo, port to Qt5/KF5
# [100%, builds, example and autotests work] Move koproperty lib from calligra to a separate [[#KProperty|KProperty]]&darr; repo, port to Qt5/KF5
# [100%, builds, example and autotests work] Move koreport lib from calligra to a separate [[#kreport|KReport]]&darr; repo, port to Qt5/KF5
# [100%, builds, example and autotests work] Move koreport lib from calligra to a separate [[#KReport|KReport]]&darr; repo, port to Qt5/KF5
## Remove scripting or port scripting?
## Remove scripting or port scripting?
# [75%] Port Kexi (without KexiDB/calligradb, reporting, koproperty, koreport) to Qt 5 and use KDb/KProperty/KReport
# [100%] Port Kexi (without KexiDB/calligradb, reporting, koproperty, koreport) to Qt 5 and use KDb/KProperty/KReport
## [100%] Remove unused calligradb lib and parts of KexiDB lib
## [100%] Remove unused calligradb lib and parts of KexiDB lib
## [92%] Port the Table plugin
## [100%] Port the Table plugin
## [92%] Port the Query plugin
## [100%] Port the Query plugin
## [90%] Port the Form plugin
## [100%] Port the Form plugin
## [90%] Port the Report plugin
## [100%] Port the Report plugin
## [50%] Port the Import/Export plugin
## [100%] Port the Import/Export plugin
## [20%] Port the Migration module
## [20%] Port the Migration module
## [0%] Reimplement report scripting using a simple solution to keep backward compatibility with Kexi 2 (see [http://api.kde.org/bundled-apps-api/calligra-apidocs/libs/koreport/html/namespaceScripting.html Scripting::*], [http://api.kde.org/bundled-apps-api/calligra-apidocs/libs/koreport/html/classKRScriptConstants.html KRScriptConstants], [http://api.kde.org/bundled-apps-api/calligra-apidocs/libs/koreport/html/classKRScriptDraw.html KRScriptDraw], [http://api.kde.org/bundled-apps-api/calligra-apidocs/libs/koreport/html/classKRScriptDebug.html KRScriptDebug])
## [0%] Reimplement report scripting using a simple solution to keep backward compatibility with Kexi 2 (see [http://api.kde.org/bundled-apps-api/calligra-apidocs/libs/koreport/html/namespaceScripting.html Scripting::*], [http://api.kde.org/bundled-apps-api/calligra-apidocs/libs/koreport/html/classKRScriptConstants.html KRScriptConstants], [http://api.kde.org/bundled-apps-api/calligra-apidocs/libs/koreport/html/classKRScriptDraw.html KRScriptDraw], [http://api.kde.org/bundled-apps-api/calligra-apidocs/libs/koreport/html/classKRScriptDebug.html KRScriptDebug])
# [35%] Improve Kexi stability after porting
# [40%] Improve Kexi stability after porting
# '''-- Kexi 3.0 beta release here. To indicate the nature of this version we never release 3.0 stable. --'''
# '''-- Kexi 3.0 beta release here. To indicate the nature of this version we never release 3.0 stable. --'''
# Add tests and/or examples to KReport, make it easier to understand as a framework  
# Add tests and/or examples to KReport, make it easier to understand as a framework  
Line 29: Line 31:


==Rules==
==Rules==
* To build Kexi, first build & install KDb, KProperty, KReport (master branches), then use calligra's 'framework' branch:
* To build Kexi, first build & install KDb, KProperty, KReport (master branches), i.e. for each:
  cd <<BUILDDIR>>
  cd <<BUILDDIR>>
  cmake -DPRODUCTSET=kexi -DCMAKE_INSTALL_PREFIX=<<PREFIX>> -DCMAKE_BUILD_TYPE=Debug <<SRCDIR>>
  cmake -DPRODUCTSET=kexi -DCMAKE_INSTALL_PREFIX=<<PREFIX>> -DCMAKE_BUILD_TYPE=Debug <<SRCDIR>>
* Because Kexi 3 uses KReport/KDb/KProperty frameworks (a code pulled out of calligra/), its Qt5 port has larger patches than other apps. To avoid troubles for Boud who merges changes calligra/2.9->frameworks I am keeping the port for Kexi in a branch '''kexi-frameworks-staniek''' of calligra.git. ''staniek'' merges it with frameworks and calligra/2.9 occasionally. Type "git checkout kexi-frameworks<<N>>-staniek". Type "git cherry -v frameworks" to see what commits we have here. (N>=8, check the largest available number using git branch -a grep kexi-frameworks"). Post-commit audits on Phabricator is a good approach for reviews.
* Then use calligra.git 'master' branch: type "git checkout master".  
*Calligra apps (Kexi too) sit in the ''framework'' branch during porting. ''master'' branch is frozen to avoid merge conflicts. After a successful port of all planned code, we'll be back to development in ''master'' for unstable code and calligra-3.x for 3.x stable code.
*Use [[Sysadmin/GitKdeOrgManual#Personal_scratch_repositories|personal git scratch repos]] to avoid stress with buildbreaks and unnecessary BUG/review notifications
*Use [[Sysadmin/GitKdeOrgManual#Personal_scratch_repositories|personal git scratch repos]] to avoid stress with buildbreaks and unnecessary BUG/review notifications
*Synchronize with (boud's?) branch of the Qt5 port. Boud's advice: ''My approach is more like, run each of montel's scripts, fix what's wrong, then go on. It's not something that's easy to paralellize.
*Boud's advice: ''My approach is more like, run each of montel's scripts, fix what's wrong, then go on. It's not something that's easy to paralellize.
*Don't run astyle on the code, it pollutes history, and astyle-kdelibs is a bit broken  
*Don't run astyle on the code, it pollutes history, and astyle-kdelibs is a bit broken  
*Based on "Porting notes" below, keep porting of one aspect in one commit to simplify reviews
*Based on "Porting notes" below, keep porting of one aspect in one commit to simplify reviews
Line 42: Line 43:
**[http://kfunk.org/2015/01/11/new-porting-helper-convert-to-cmake-automoc-pl convert-to-cmake-automoc.pl]
**[http://kfunk.org/2015/01/11/new-porting-helper-convert-to-cmake-automoc-pl convert-to-cmake-automoc.pl]
*Libs that are separate repos should use the framework template (see Links) and be ported to this layout (for now it's just kdb, kproperty, kreport).
*Libs that are separate repos should use the framework template (see Links) and be ported to this layout (for now it's just kdb, kproperty, kreport).
TODO...


==Notes and Status==
==Notes and Status==
Line 58: Line 57:
*There is the [[Frameworks/Building#Adjust_the_paths|Adjust the paths]] step. For Kexi development (and Calligra in general) it's good to re-use paths that we already use except the install prefix (kdedir), see the [[Calligra/Building/3#Recommended_Setup|Recommended setup]] of the Calligra build instructions. This means that recommendation for the <tt>kdesrc-buildrc</tt> config file is:
*There is the [[Frameworks/Building#Adjust_the_paths|Adjust the paths]] step. For Kexi development (and Calligra in general) it's good to re-use paths that we already use except the install prefix (kdedir), see the [[Calligra/Building/3#Recommended_Setup|Recommended setup]] of the Calligra build instructions. This means that recommendation for the <tt>kdesrc-buildrc</tt> config file is:
   source-dir /home/YOU/kde/src
   source-dir /home/YOU/kde/src
   build-dir /home/YOU/kde/build
   build-dir /home/YOU/kde/buildi
   kdedir /home/YOU/kde/inst5
   kdedir /home/YOU/kde/inst5


{{Note|If you used ''/home/YOU/kde/inst'' as a dir the Calligra 2.x and other kdelibs4-based software, '''do not mix it with the Calligra 3.x+ and other kf5-based software''', otherwise you can experience crashes or other strange behaviour. '''Use ''/home/YOU/kde/inst5'' for example.}}
{{Note|If you used ''/home/YOU/kde/inst'' as a dir the Calligra 2.x and other kdelibs4-based software, '''do not mix it with the Calligra 3.x+ and other kf5-based software''', otherwise you can experience crashes or other strange behaviour. '''Use ''/home/YOU/kde/inst5'' for example.}}


*Speed up: before using the KF5 with kdesrc-build, edit the kf5-qt5-build-include file and comment these lines if you don't want to build the workspace, applications and KDEPIM. This way only include kf5-frameworks-build-include lefts.
*Speed up: before using the KF5 with kdesrc-build, edit the kf5-qt5.ksb file and comment these lines if you don't want to build the workspace, applications and KDEPIM. Thus only "include kf5-frameworks.ksb" is left.
include kf5-workspace-build-include
:include kf5-workspace.ksb
include kf5-applications-build-include
:include kf5-applications.ksb
include kf5-kdepim-build-include
:include kf5-kdepim.ksb
 
 
*In particular, the kdesrc-build will install extra CMake Modules: [https://community.kde.org/Frameworks/Building/Details#Extra_CMake_Modules], which is needed to make our software able to find the KF5 components at build time.
*In particular, the kdesrc-build will install extra CMake Modules: [https://community.kde.org/Frameworks/Building/Details#Extra_CMake_Modules], which is needed to make our software able to find the KF5 components at build time.


Line 166: Line 167:


==Git surgery==
==Git surgery==
([[User:Jstaniek|Jstaniek]] ([[User talk:Jstaniek|talk]]) 09:54, 23 February 2015 (UTC)) Repos are prepared by history rewrites such as:
1. For editing large history you may need a fast filesystem. So a Linux '''RAM disk''' such as /dev/shm is your friend, deep-copy your repo temporarily there: [http://kvz.io/blog/2007/07/18/create-turbocharged-storage-using-tmpfs/].
* To deep-copy use:
cd /path/to/ramdisk git clone /path/to/the/orig/repo --no-hardlinks --no-local
* Consider switching off swap ('sudo swapoff -a') as swapping makes no sense, when we want to use RAM exclusively.
* Don't forget to create a backup of the original repository and then replace the original repository with the copy edited, using git clone again:
cd /path/to/the/orig
rm -rf repo
git clone /path/to/ramdisk/repo --no-hardlinks --no-local
2. ([[User:Jstaniek|Jstaniek]] ([[User talk:Jstaniek|talk]]) 09:54, 23 February 2015 (UTC)) Repos are prepared by history rewrites such as:
  git filter-branch --prune-empty -f --tree-filter \
  git filter-branch --prune-empty -f --tree-filter \
  '(mkdir -p src;mv Predicate src/predicate;mv Drivers src/drivers) 2> /dev/null || true' -- HEAD
  '(mkdir -p src;mv Predicate src/predicate;mv Drivers src/drivers) 2> /dev/null || true' -- HEAD
3. Before your push your branch to a new repo, you may need some '''postprocessing''' if the commits contain non-Unix line endings, git push refuses to work then:
remote: Audit failure - Commit 9460d32f6f2f51c2a182b8faa40ff7537b7ac5c6 - End of Line Style (non-Unix)
To remove the non-Unix line endings (see also [http://blog.gyoshev.net/2013/08/29/normalizing-line-endings-in-git-repositories/]).
3.1. Create this script and call it ''fix-eol.sh'':
#!/bin/bash
find . -type f -exec grep -Iq . {} \; -and -print | xargs dos2unix -k -q
(the ''find'' and ''grep'' commands here try very hard to find text files)
3.2. Run the actual rewrite. Note, we call this ''postprocessing'' because it's time-consuming so it's worth to run it after the step #2 only on files that we really need in the edited repo.
cd /path/to/the/repo
git filter-branch --tree-filter fix-eol.sh  -- --all


==Links==
==Links==

Latest revision as of 17:51, 10 December 2023

Note

Based on notes from the Calligra 2014 Sprint. Each step leads to compiling (running?) state.
Started by jstaniek (talk) 19:06, 7 July 2014 (CET).


The grand plan:

  1. [100%, builds, example and autotests work] Port PredicateKDb↓ lib to Qt5/KF5
    1. [100%] Port the SQLite driver
    2. [100%] Port the MySQL driver
    3. [100%] Port the PostgreSQL driver
    4. [10%] Port the xBase driver
    5. [10%] Port the Sybase driver
  2. [100%, builds, example and autotests work] Move koproperty lib from calligra to a separate KProperty↓ repo, port to Qt5/KF5
  3. [100%, builds, example and autotests work] Move koreport lib from calligra to a separate KReport↓ repo, port to Qt5/KF5
    1. Remove scripting or port scripting?
  4. [100%] Port Kexi (without KexiDB/calligradb, reporting, koproperty, koreport) to Qt 5 and use KDb/KProperty/KReport
    1. [100%] Remove unused calligradb lib and parts of KexiDB lib
    2. [100%] Port the Table plugin
    3. [100%] Port the Query plugin
    4. [100%] Port the Form plugin
    5. [100%] Port the Report plugin
    6. [100%] Port the Import/Export plugin
    7. [20%] Port the Migration module
    8. [0%] Reimplement report scripting using a simple solution to keep backward compatibility with Kexi 2 (see Scripting::*, KRScriptConstants, KRScriptDraw, KRScriptDebug)
  5. [40%] Improve Kexi stability after porting
  6. -- Kexi 3.0 beta release here. To indicate the nature of this version we never release 3.0 stable. --
  7. Add tests and/or examples to KReport, make it easier to understand as a framework
    1. Port some code from Kexi Reports (e.g. KexiReportView [1]) to KReport
  8. Make KReport lib dependent on KDb and remove code/API that becomes redundant (e.g. KoReportData)
  9. Add a new scripting module
  10. Use the module in Kexi Reports too, keeping backward compatibility with Kexi 2 scripting (add unit tests before porting to be very sure that compatibility is kept)

Rules

  • To build Kexi, first build & install KDb, KProperty, KReport (master branches), i.e. for each:
cd <<BUILDDIR>>
cmake -DPRODUCTSET=kexi -DCMAKE_INSTALL_PREFIX=<<PREFIX>> -DCMAKE_BUILD_TYPE=Debug <<SRCDIR>>
  • Then use calligra.git 'master' branch: type "git checkout master".
  • Use personal git scratch repos to avoid stress with buildbreaks and unnecessary BUG/review notifications
  • Boud's advice: My approach is more like, run each of montel's scripts, fix what's wrong, then go on. It's not something that's easy to paralellize.
  • Don't run astyle on the code, it pollutes history, and astyle-kdelibs is a bit broken
  • Based on "Porting notes" below, keep porting of one aspect in one commit to simplify reviews
  • Use the Phabricator for reviews
  • Use kde-dev-scripts.git porting scripts, in particular
  • Libs that are separate repos should use the framework template (see Links) and be ported to this layout (for now it's just kdb, kproperty, kreport).

Notes and Status

  • TODO: publish detailed checklist (in a spreadsheet?)

Dependencies

Please read all this section carefully before taking action.

  • Qt 5. Get >=5.2 stable version from your distro, not forgetting about development packages (headers). If you feel you want it, build Qt 5, but it's usually not necessary extra work.
  • CMake: version >=2.8.12+ and 3.x are both supported. Get it from your distro (recommended) or build it.
  • KDE Frameworks 5 (KF5) are needed as build-time and run-time dependencies for Calligra, Kexi and our frameworks (KProperty, etc).
  • Use the kdesrc-build tool to conveniently build KF5. Building by hand in an option if you accept extra work usually not needed for our development (see Frameworks/Building/Details).
  • There is the Adjust the paths step. For Kexi development (and Calligra in general) it's good to re-use paths that we already use except the install prefix (kdedir), see the Recommended setup of the Calligra build instructions. This means that recommendation for the kdesrc-buildrc config file is:
 source-dir /home/YOU/kde/src
 build-dir /home/YOU/kde/buildi
 kdedir /home/YOU/kde/inst5

Note

If you used /home/YOU/kde/inst as a dir the Calligra 2.x and other kdelibs4-based software, do not mix it with the Calligra 3.x+ and other kf5-based software, otherwise you can experience crashes or other strange behaviour. Use /home/YOU/kde/inst5 for example.


  • Speed up: before using the KF5 with kdesrc-build, edit the kf5-qt5.ksb file and comment these lines if you don't want to build the workspace, applications and KDEPIM. Thus only "include kf5-frameworks.ksb" is left.
include kf5-workspace.ksb
include kf5-applications.ksb
include kf5-kdepim.ksb


  • In particular, the kdesrc-build will install extra CMake Modules: [2], which is needed to make our software able to find the KF5 components at build time.

KDb

  • Download/Build/Release info → KDb.
  • Use master branch, official for Qt 5/KF 5, for all platforms, always stable
  • Our target version is 3.0.0.
  • To avoid conflicts, Qt 4 version kept for reference, has version 0.0.0.
  • Depends only on system libraries and Qt, so it's a tier 1 framework
  • Windows and Mac-specific matters.
  • Porting to Qt 5 started by Wojtek Kosowicz (Jan 2015)
  • Porting to Qt 5/Mac OS X started by Wojtek Kosowicz too (Jan 2015)
  • Directory structure reorganized according to recommendations --Jstaniek (talk) 20:59, 23 February 2015 (UTC)

KProperty

  • Our target version is 3.0.0 and KProperty project in Playground/Libs category of KDE projects
  • Depends on system libraries and Qt, plus minor deps on KF5WidgetsAddons/KF5GuiAddons for some editors; later we may move this dependency to a plugin (so the core would stay Qt-only)
  • Official git repo: kproperty, master branch (use: git clone kde:kproperty)
  • Directory structure reorganized according to recommendations --Jstaniek (talk) 20:59, 23 February 2015 (UTC)

KReport

  • Our target version is 3.0.0 and KReport project in Playground/Libs category of KDE projects
  • To avoid conflicts, Qt 4 version kept for reference, has version 0.0.0.
  • Depends on system libraries, Qt, and KProperty (tier 1), so it's a tier 2 framework, some deps on KF5WidgetsAddons (may be moved to a plugin so the core would stay Qt-only)
  • Official git repo: kreport, master branch (use: git clone kde:kreport)
  • Directory structure reorganized according to recommendations --Jstaniek (talk) 20:59, 23 February 2015 (UTC)
  • Deps from calligra libs (copied, renamed): KoRuler, KoUnit

Build instructions

Kexi 3/Calligra 3 builing is explained in a (work-in-progress) doc: Calligra/Building/3. Our dependencies (KDb, KPropert, KReport) can be built in a usual way (cmake, make install). Don't forget to set environment variables in the same way as you do for Kexi/Calligra builds.

Porting scripts usage status

These porting scripts from the kf5 dir of the kde:kde-dev-scripts repo would be applied.
Legend: - = not run, + = run, N = not applicable, T = TODO(we know it's needed)

KDb | KProperty | KReport | Kexi | script-name
+ + + + adapt_cmakelists_file.pl
+ + + + adapt_knewstuff3_includes.pl
+ + + + add_missing_kpart_include.pl
+ + + + clean-forward-declaration.sh
+ + + + clean-includes.sh
N N N - cmakelists_install_vars.pl (N/A because KDb/KProperty/KReport don't use KDE_*DIR variables)
N N N N convert-kabc-to-kcontacts.pl (unused)
+ + + + convert-kaction.pl
+ + + - convert-kbuttongroup.pl
T + + + convert-kcmdlineargs.pl
+ + + N convert-kcolordialog.pl
+ + + + convert-kdebug.pl
N N N N convert-kdebug-with-argument.sh (N/A because KDb/KProperty/KReport don't use that)
N + N + convert-kdialog.pl (helps but KProperty is largery broken by this script)
N N N N convert-kdoublenuminput.pl
N N N N convert-kfiledialog.pl
N N N N convert-kfontcombobox.pl
N N N T convert-kicon.pl
N N N N convert-kimageio.pl
N N N N convert-kintnuminput.pl
N N N N convert-kintspinbox.pl
N N N N convert-kjob.pl
N N N + convert-klineedit.pl
N N N + convert-klistwidget.pl
N N N N convert-klocale-formatdate.pl
N N N + convert-kmd5.pl
N N N N convert-kmenubar.pl
N N N + convert-kmenu.pl
N N + + convert-kmimetype.pl
N N N + convert-kprogressdialog.pl
N N N + convert-kpushbutton.pl
N N N + convert-ksavefile.pl
N N N + convert-ksharedptr.pl
N N N + convert-kshortcut.pl
N N N N convert-ksplashscreen.pl
N N N + convert-kstandarddirs.pl
N N N + convert-ktabwidget.pl
N N N + convert-ktempdir.pl
+ N + + convert-ktemporaryfile.pl
N N N + convert-ktextbrowser.pl
N + + + convert-kurl.pl
N N N + convert-kvbox.pl
+ + + - convert-qkdebug-to-qcdebug.sh
+ + + + convert-qt5.pl
+ + + + convert-to-cmake-automoc.pl
T N N T convert-to-k4aboutdata.pl
- - - - convert-to-new-signal-slot-signal.pl
N N N N dnssd-to-kdnssd.sh
+ + + + fix-ecm-install-icons.pl
- - - - fix-kcm-module-desktop-file.sh
- - - - install_forwarding_headers.pl
N N N N port-kauthactions.pl
N N N + port_to_autogenerate_export_header.sh
+ + + + remove-kde4support.pl
N N N N remove_virtual_prefix.pl
N N N + resolve_kuit.py
T T T T search-kdelibs4support-header.sh
+ + + + frameworks/plasma-framework/src/tools/port-cmake-style.sh
+ + + - frameworks/plasma-framework/src/tools/port-qslots.sh
+ + + - frameworks/plasma-framework/src/tools/port-includes.sh

Git surgery

1. For editing large history you may need a fast filesystem. So a Linux RAM disk such as /dev/shm is your friend, deep-copy your repo temporarily there: [3].

  • To deep-copy use:
cd /path/to/ramdisk git clone /path/to/the/orig/repo --no-hardlinks --no-local
  • Consider switching off swap ('sudo swapoff -a') as swapping makes no sense, when we want to use RAM exclusively.
  • Don't forget to create a backup of the original repository and then replace the original repository with the copy edited, using git clone again:
cd /path/to/the/orig
rm -rf repo
git clone /path/to/ramdisk/repo --no-hardlinks --no-local

2. (Jstaniek (talk) 09:54, 23 February 2015 (UTC)) Repos are prepared by history rewrites such as:

git filter-branch --prune-empty -f --tree-filter \
'(mkdir -p src;mv Predicate src/predicate;mv Drivers src/drivers) 2> /dev/null || true' -- HEAD

3. Before your push your branch to a new repo, you may need some postprocessing if the commits contain non-Unix line endings, git push refuses to work then:

remote: Audit failure - Commit 9460d32f6f2f51c2a182b8faa40ff7537b7ac5c6 - End of Line Style (non-Unix)

To remove the non-Unix line endings (see also [4]).

3.1. Create this script and call it fix-eol.sh:

#!/bin/bash
find . -type f -exec grep -Iq . {} \; -and -print | xargs dos2unix -k -q

(the find and grep commands here try very hard to find text files)

3.2. Run the actual rewrite. Note, we call this postprocessing because it's time-consuming so it's worth to run it after the step #2 only on files that we really need in the edited repo.

cd /path/to/the/repo
git filter-branch --tree-filter fix-eol.sh  -- --all

Links

Docs

Background