Kexi/Porting to Qt&KF 5: Difference between revisions

From KDE Community Wiki
No edit summary
(61 intermediate revisions by one other user not shown)
Line 2: Line 2:


The grand plan:
The grand plan:
# [75%, builds, example runs] Port <strike>Predicate</strike>[[#KDb|KDb]]&darr; lib to Qt 5
# [100%, builds, example and autotests work] Port <strike>Predicate</strike>[[#KDb|KDb]]&darr; lib to Qt5/KF5
# [97%] Move koproperty lib from calligra to a separate [[#KProperty]]&darr; repo, make it Qt-only and port to Qt5
## [100%] Port the SQLite driver
# [30%] Move koreport lib from calligra to a separate [[#KReport]]&darr; repo, make it Qt-only and port to Qt 5
## [100%] Port the MySQL driver
## [100%] Port the PostgreSQL driver
## [10%] Port the xBase driver
## [10%] Port the Sybase driver
# [100%, builds, example and autotests work] Move koproperty lib from calligra to a separate [[#KProperty|KProperty]]&darr; repo, port to Qt5/KF5
# [100%, builds, example and autotests work] Move koreport lib from calligra to a separate [[#KReport|KReport]]&darr; repo, port to Qt5/KF5
## Remove scripting or port scripting?
## Remove scripting or port scripting?
# Port Kexi (without KexiDB/calligradb, reporting, koproperty, koreport) to Qt 5 and port Kexi to KDb and the new kproperty/kreport
# [100%] Port Kexi (without KexiDB/calligradb, reporting, koproperty, koreport) to Qt 5 and use KDb/KProperty/KReport
## Reimplement report scripting using a simple solution to keep backward compatibility with Kexi 2
## [100%] Remove unused calligradb lib and parts of KexiDB lib
## Remove then unused calligradb lib and parts of KexiDB lib
## [100%] Port the Table plugin
# Improve Kexi stability after porting
## [100%] Port the Query plugin
# '''-- Kexi 3.0 release here --'''
## [100%] Port the Form plugin
## [100%] Port the Report plugin
## [100%] Port the Import/Export plugin
## [20%] Port the Migration module
## [0%] Reimplement report scripting using a simple solution to keep backward compatibility with Kexi 2 (see [http://api.kde.org/bundled-apps-api/calligra-apidocs/libs/koreport/html/namespaceScripting.html Scripting::*], [http://api.kde.org/bundled-apps-api/calligra-apidocs/libs/koreport/html/classKRScriptConstants.html KRScriptConstants], [http://api.kde.org/bundled-apps-api/calligra-apidocs/libs/koreport/html/classKRScriptDraw.html KRScriptDraw], [http://api.kde.org/bundled-apps-api/calligra-apidocs/libs/koreport/html/classKRScriptDebug.html KRScriptDebug])
# [40%] Improve Kexi stability after porting
# '''-- Kexi 3.0 beta release here. To indicate the nature of this version we never release 3.0 stable. --'''
# Add tests and/or examples to KReport, make it easier to understand as a framework
## Port some code from Kexi Reports (e.g. KexiReportView [https://todo.kde.org/?controller=task&action=show&task_id=1206]) to KReport
# Make KReport lib dependent on KDb and remove code/API that becomes redundant (e.g. KoReportData)
# Make KReport lib dependent on KDb and remove code/API that becomes redundant (e.g. KoReportData)
# Add a new [[Kexi/Plugins/Scripts|scripting module]]
# Add a new [[Kexi/Plugins/Scripts|scripting module]]
Line 16: Line 29:


==Rules==
==Rules==
*Kexi's port itself is kept with other Calligra apps in the ''framework'' branch. During this time ''master'' branch is frozen to avoid merge conflicts. After a successful port of all planned code, we'll be back to development in ''master'' for unstable code and calligra-3.x for 3.x stable code.
* To build Kexi, first build & install KDb, KProperty, KReport (master branches), i.e. for each:
cd <<BUILDDIR>>
cmake -DPRODUCTSET=kexi -DCMAKE_INSTALL_PREFIX=<<PREFIX>> -DCMAKE_BUILD_TYPE=Debug <<SRCDIR>>
* Then use calligra.git 'master' branch: type "git checkout master".  
*Use [[Sysadmin/GitKdeOrgManual#Personal_scratch_repositories|personal git scratch repos]] to avoid stress with buildbreaks and unnecessary BUG/review notifications
*Use [[Sysadmin/GitKdeOrgManual#Personal_scratch_repositories|personal git scratch repos]] to avoid stress with buildbreaks and unnecessary BUG/review notifications
*Synchronize with (boud's?) branch of the Qt5 port. Boud's advice: ''My approach is more like, run each of montel's scripts, fix twhat's wrong, then go on. It's not something that's easy to paralellize.
*Boud's advice: ''My approach is more like, run each of montel's scripts, fix what's wrong, then go on. It's not something that's easy to paralellize.
*Don't run astyle on the code, it pollutes history, and astyle-kdelibs is a bit broken  
*Don't run astyle on the code, it pollutes history, and astyle-kdelibs is a bit broken  
*Based on "Porting notes" below, keep porting of one aspect in one commit to simplify reviews
*Based on "Porting notes" below, keep porting of one aspect in one commit to simplify reviews
*Use the reviewboard for reviews
*Use the Phabricator for reviews
*Use [http://quickgit.kde.org/?p=kde-dev-scripts.git kde-dev-scripts.git] porting scripts, in particular
*Use [http://quickgit.kde.org/?p=kde-dev-scripts.git kde-dev-scripts.git] porting scripts, in particular
**[http://kfunk.org/2015/01/11/new-porting-helper-convert-to-cmake-automoc-pl convert-to-cmake-automoc.pl]
**[http://kfunk.org/2015/01/11/new-porting-helper-convert-to-cmake-automoc-pl convert-to-cmake-automoc.pl]
*Libs that are separate repos should use the framework template (see Links) and be ported to this layout (for now it's just kdb, kproperty, kreport).
*Libs that are separate repos should use the framework template (see Links) and be ported to this layout (for now it's just kdb, kproperty, kreport).
TODO...


==Notes and Status==
==Notes and Status==
Line 39: Line 53:
*KDE Frameworks 5 (KF5) are needed as build-time and run-time dependencies for Calligra, Kexi and our frameworks (KProperty, etc).
*KDE Frameworks 5 (KF5) are needed as build-time and run-time dependencies for Calligra, Kexi and our frameworks (KProperty, etc).
*[https://community.kde.org/Frameworks/Building Use the kdesrc-build] tool to conveniently build KF5. Building by hand in an option if you accept extra work usually not needed for our development (see [[Frameworks/Building/Details]]).
*[https://community.kde.org/Frameworks/Building Use the kdesrc-build] tool to conveniently build KF5. Building by hand in an option if you accept extra work usually not needed for our development (see [[Frameworks/Building/Details]]).
*There is the [[Frameworks/Building#Adjust_the_paths|Adjust the paths]] step. For Kexi development (and Calligra in general) it's good to re-use paths that we already use, see the [[Calligra/Building#Recommended_Setup|Recommended setup of the Calligra build instructions. This means that recommendation for the <tt>kdesrc-buildrc</tt> config file is:
*There is the [[Frameworks/Building#Adjust_the_paths|Adjust the paths]] step. For Kexi development (and Calligra in general) it's good to re-use paths that we already use except the install prefix (kdedir), see the [[Calligra/Building/3#Recommended_Setup|Recommended setup]] of the Calligra build instructions. This means that recommendation for the <tt>kdesrc-buildrc</tt> config file is:
   source-dir /home/YOU/kde/src
   source-dir /home/YOU/kde/src
   build-dir /home/YOU/kde/build
   build-dir /home/YOU/kde/build
Line 46: Line 60:
{{Note|If you used ''/home/YOU/kde/inst'' as a dir the Calligra 2.x and other kdelibs4-based software, '''do not mix it with the Calligra 3.x+ and other kf5-based software''', otherwise you can experience crashes or other strange behaviour. '''Use ''/home/YOU/kde/inst5'' for example.}}
{{Note|If you used ''/home/YOU/kde/inst'' as a dir the Calligra 2.x and other kdelibs4-based software, '''do not mix it with the Calligra 3.x+ and other kf5-based software''', otherwise you can experience crashes or other strange behaviour. '''Use ''/home/YOU/kde/inst5'' for example.}}


*Speed up: before using the KF5 with kdesrc-build, edit the kf5-qt5-build-include file and comment these lines if you don't want to build the workspace, applications and KDEPIM. This way only include kf5-frameworks-build-include lefts.
*Speed up: before using the KF5 with kdesrc-build, edit the kf5-qt5-build-include file and comment these lines if you don't want to build the workspace, applications and KDEPIM. Thus only "include kf5-frameworks-build-include" is left.
include kf5-workspace-build-include
:include kf5-workspace-build-include
include kf5-applications-build-include
:include kf5-applications-build-include
include kf5-kdepim-build-include
:include kf5-kdepim-build-include
 
 
*In particular, the kdesrc-build will install extra CMake Modules: [https://community.kde.org/Frameworks/Building/Details#Extra_CMake_Modules], which is needed to make our software able to find the KF5 components at build time.
*In particular, the kdesrc-build will install extra CMake Modules: [https://community.kde.org/Frameworks/Building/Details#Extra_CMake_Modules], which is needed to make our software able to find the KF5 components at build time.


===KDb===
===KDb===
*Download/Build/Release info &rarr; [[Predicate]].
*Download/Build/Release info &rarr; [[KDb]].
*Our target [[Predicate/Releases|version]] is 1.0.0.
*Use '''master''' branch, official for Qt 5/KF 5, for all platforms, always stable
*Our target [[KDb/Releases|version]] is 3.0.0.
*To avoid conflicts, Qt 4 version kept for reference, has version 0.0.0.
*To avoid conflicts, Qt 4 version kept for reference, has version 0.0.0.
*Depends only on system libraries and Qt, so it's a tier 1 framework
*Depends only on system libraries and Qt, so it's a tier 1 framework
*[[Predicate/Windows|Windows]] and [[Predicate/Mac|Mac]]-specific matters.
*[[KDb/Windows|Windows]] and [[KDb/Mac|Mac]]-specific matters.
*Porting to Qt 5 started by Wojtek Kosowicz (Jan 2015)
*Porting to Qt 5 started by Wojtek Kosowicz (Jan 2015)
*Porting to Qt 5/Mac OS X started by Wojtek Kosowicz too (Jan 2015)
*Porting to Qt 5/Mac OS X started by Wojtek Kosowicz too (Jan 2015)
*Directory structure reorganized according to [[Frameworks/Policies#Framework_directory_structure|recommendations]] --[[User:Jstaniek|Jstaniek]] ([[User talk:Jstaniek|talk]]) 20:59, 23 February 2015 (UTC)
*Directory structure reorganized according to [[Frameworks/Policies#Framework_directory_structure|recommendations]] --[[User:Jstaniek|Jstaniek]] ([[User talk:Jstaniek|talk]]) 20:59, 23 February 2015 (UTC)
*A temporary repo: git clone kde:clones/predicate/staniek/work predicate
Branches:
'''qt4''' - old Qt4-based master moved there, always stable
'''kdb''' - official Qt 5 branch, all platforms, always stable
To have it build on that branch please export qt5 install dir
<syntaxhighlight lang="bash">
export QTDIR=<path to your qt5 install dir, or qtbase build dir if using uninstalled>
export PATH=$QTDIR:$PATH</syntaxhighlight>
'''qt5mac-kosowicz''' - Wojtek's Qt 5/Mac OS X


===kproperty===
===KProperty===
*Our target version is 1.0.0 and '''KProperty''' [https://projects.kde.org/projects/playground/libs/kreport project in Playground/Libs category of KDE projects]
*Our target version is 3.0.0 and '''KProperty''' [https://projects.kde.org/projects/playground/libs/kreport project in Playground/Libs category of KDE projects]
*Depends on system libraries and Qt, plus minor deps on KF5WidgetsAddons/KF5GuiAddons for some editors; later we may move this dependency to a plugin (so the core would stay Qt-only)
*Depends on system libraries and Qt, plus minor deps on KF5WidgetsAddons/KF5GuiAddons for some editors; later we may move this dependency to a plugin (so the core would stay Qt-only)
*Official git repo: kproperty, master branch (use: git clone kde:kproperty)
*Official git repo: kproperty, master branch (use: git clone kde:kproperty)
*Directory structure reorganized according to [[Frameworks/Policies#Framework_directory_structure|recommendations]] --[[User:Jstaniek|Jstaniek]] ([[User talk:Jstaniek|talk]]) 20:59, 23 February 2015 (UTC)
*Directory structure reorganized according to [[Frameworks/Policies#Framework_directory_structure|recommendations]] --[[User:Jstaniek|Jstaniek]] ([[User talk:Jstaniek|talk]]) 20:59, 23 February 2015 (UTC)


===kreport===
===KReport===
*Our target version is 1.0.0 and '''KReport''' [https://projects.kde.org/projects/playground/libs/kreport project in Playground/Libs category of KDE projects]
*Our target version is 3.0.0 and '''KReport''' [https://projects.kde.org/projects/playground/libs/kreport project in Playground/Libs category of KDE projects]
*To avoid conflicts, Qt 4 version kept for reference, has version 0.0.0.
*To avoid conflicts, Qt 4 version kept for reference, has version 0.0.0.
*Depends on system libraries, Qt, and KProperty (tier 1), so it's a tier 2 framework, some deps on KF5WidgetsAddons (may be moved to a plugin so the core would stay Qt-only)
*Depends on system libraries, Qt, and KProperty (tier 1), so it's a tier 2 framework, some deps on KF5WidgetsAddons (may be moved to a plugin so the core would stay Qt-only)
Line 90: Line 92:
*Directory structure reorganized according to [[Frameworks/Policies#Framework_directory_structure|recommendations]] --[[User:Jstaniek|Jstaniek]] ([[User talk:Jstaniek|talk]]) 20:59, 23 February 2015 (UTC)
*Directory structure reorganized according to [[Frameworks/Policies#Framework_directory_structure|recommendations]] --[[User:Jstaniek|Jstaniek]] ([[User talk:Jstaniek|talk]]) 20:59, 23 February 2015 (UTC)
*Deps from calligra libs (copied, renamed): KoRuler, KoUnit
*Deps from calligra libs (copied, renamed): KoRuler, KoUnit
==Build instructions==
Kexi 3/Calligra 3 builing is explained in a (work-in-progress) doc: [[Calligra/Building/3]].
Our dependencies (KDb, KPropert, KReport) can be built in a usual way (cmake, make install). Don't forget to set environment variables in the same way as you do for Kexi/Calligra builds.


==Porting scripts usage status==
==Porting scripts usage status==
These porting scripts from the kf5 dir of the kde:kde-dev-scripts repo would be applied. <br/>
These porting scripts from the kf5 dir of the kde:kde-dev-scripts repo would be applied. <br/>
Legend: - = not run, + run, N - not applicable
Legend: - = not run, + = run, N = not applicable, T = TODO(we know it's needed)
<pre>
<pre>
kdb kproperty kreport script-name
KDb | KProperty | KReport | Kexi | script-name
- - + adapt_cmakelists_file.pl
+ + + + adapt_cmakelists_file.pl
- - + adapt_knewstuff3_includes.pl
+ + + + adapt_knewstuff3_includes.pl
- - + add_missing_kpart_include.pl
+ + + + add_missing_kpart_include.pl
- - + clean-forward-declaration.sh
+ + + + clean-forward-declaration.sh
- - + clean-includes.sh
+ + + + clean-includes.sh
- - - cmakelists_install_vars.pl
N N N - cmakelists_install_vars.pl (N/A because KDb/KProperty/KReport don't use KDE_*DIR variables)
- - - convert-kabc-to-kcontacts.pl
N N N N convert-kabc-to-kcontacts.pl (unused)
- - - convert-kaction.pl
+ + + + convert-kaction.pl
- - - convert-kbuttongroup.pl
+ + + - convert-kbuttongroup.pl
- - - convert-kcmdlineargs.pl
T + + + convert-kcmdlineargs.pl
- - - convert-kcolordialog.pl
+ + + N convert-kcolordialog.pl
- - - convert-kdebug.pl
+ + + + convert-kdebug.pl
- - - convert-kdebug-with-argument.sh
N N N N convert-kdebug-with-argument.sh (N/A because KDb/KProperty/KReport don't use that)
- - - convert-kdialog.pl
N + N + convert-kdialog.pl (helps but KProperty is largery broken by this script)
- - - convert-kdoublenuminput.pl
N N N N convert-kdoublenuminput.pl
- - - convert-kfiledialog.pl
N N N N convert-kfiledialog.pl
- - - convert-kfontcombobox.pl
N N N N convert-kfontcombobox.pl
- - - convert-kicon.pl
N N N T convert-kicon.pl
- - - convert-kimageio.pl
N N N N convert-kimageio.pl
- - - convert-kintnuminput.pl
N N N N convert-kintnuminput.pl
- - - convert-kintspinbox.pl
N N N N convert-kintspinbox.pl
- - - convert-kjob.pl
N N N N convert-kjob.pl
- - - convert-klineedit.pl
N N N + convert-klineedit.pl
- - - convert-klistwidget.pl
N N N + convert-klistwidget.pl
- - - convert-klocale-formatdate.pl
N N N N convert-klocale-formatdate.pl
- - - convert-kmd5.pl
N N N + convert-kmd5.pl
- - - convert-kmenubar.pl
N N N N convert-kmenubar.pl
- - - convert-kmenu.pl
N N N + convert-kmenu.pl
- - - convert-kmimetype.pl
N N + + convert-kmimetype.pl
- - - convert-kprogressdialog.pl
N N N + convert-kprogressdialog.pl
- - - convert-kpushbutton.pl
N N N + convert-kpushbutton.pl
- - - convert-ksavefile.pl
N N N + convert-ksavefile.pl
- - - convert-ksharedptr.pl
N N N + convert-ksharedptr.pl
- - - convert-kshortcut.pl
N N N + convert-kshortcut.pl
- - - convert-ksplashscreen.pl
N N N N convert-ksplashscreen.pl
- - - convert-kstandarddirs.pl
N N N + convert-kstandarddirs.pl
- - - convert-ktabwidget.pl
N N N + convert-ktabwidget.pl
- - - convert-ktempdir.pl
N N N + convert-ktempdir.pl
- - - convert-ktemporaryfile.pl
+ N + + convert-ktemporaryfile.pl
- - - convert-ktextbrowser.pl
N N N + convert-ktextbrowser.pl
- - + convert-kurl.pl
N + + + convert-kurl.pl
- - - convert-kvbox.pl
N N N + convert-kvbox.pl
- - - convert-qkdebug-to-qcdebug.sh
+ + + - convert-qkdebug-to-qcdebug.sh
- - - convert-qt5.pl
+ + + + convert-qt5.pl
- - - convert-to-cmake-automoc.pl
+ + + + convert-to-cmake-automoc.pl
- - - convert-to-k4aboutdata.pl
T N N T convert-to-k4aboutdata.pl
- - - convert-to-new-signal-slot-signal.pl
- - - - convert-to-new-signal-slot-signal.pl
- - - dnssd-to-kdnssd.sh
N N N N dnssd-to-kdnssd.sh
- - - fix-ecm-install-icons.pl
+ + + + fix-ecm-install-icons.pl
- - - fix-kcm-module-desktop-file.sh
- - - - fix-kcm-module-desktop-file.sh
- - - install_forwarding_headers.pl
- - - - install_forwarding_headers.pl
- - - port-kauthactions.pl
N N N N port-kauthactions.pl
- - - port_to_autogenerate_export_header.sh
N N N + port_to_autogenerate_export_header.sh
- - - remove-kde4support.pl
+ + + + remove-kde4support.pl
- - - remove_virtual_prefix.pl
N N N N remove_virtual_prefix.pl
- - - resolve_kuit.py
N N N + resolve_kuit.py
- - - search-kdelibs4support-header.sh
T T T T search-kdelibs4support-header.sh
+ + + + frameworks/plasma-framework/src/tools/port-cmake-style.sh
+ + + - frameworks/plasma-framework/src/tools/port-qslots.sh
+ + + - frameworks/plasma-framework/src/tools/port-includes.sh
</pre>
</pre>


==Git surgery==
==Git surgery==
([[User:Jstaniek|Jstaniek]] ([[User talk:Jstaniek|talk]]) 09:54, 23 February 2015 (UTC)) Repos are prepared by history rewrites such as:
1. For editing large history you may need a fast filesystem. So a Linux '''RAM disk''' such as /dev/shm is your friend, deep-copy your repo temporarily there: [http://kvz.io/blog/2007/07/18/create-turbocharged-storage-using-tmpfs/].
* To deep-copy use:
cd /path/to/ramdisk git clone /path/to/the/orig/repo --no-hardlinks --no-local
* Consider switching off swap ('sudo swapoff -a') as swapping makes no sense, when we want to use RAM exclusively.
* Don't forget to create a backup of the original repository and then replace the original repository with the copy edited, using git clone again:
cd /path/to/the/orig
rm -rf repo
git clone /path/to/ramdisk/repo --no-hardlinks --no-local
2. ([[User:Jstaniek|Jstaniek]] ([[User talk:Jstaniek|talk]]) 09:54, 23 February 2015 (UTC)) Repos are prepared by history rewrites such as:
  git filter-branch --prune-empty -f --tree-filter \
  git filter-branch --prune-empty -f --tree-filter \
  '(mkdir -p src;mv Predicate src/predicate;mv Drivers src/drivers) 2> /dev/null || true' -- HEAD
  '(mkdir -p src;mv Predicate src/predicate;mv Drivers src/drivers) 2> /dev/null || true' -- HEAD
3. Before your push your branch to a new repo, you may need some '''postprocessing''' if the commits contain non-Unix line endings, git push refuses to work then:
remote: Audit failure - Commit 9460d32f6f2f51c2a182b8faa40ff7537b7ac5c6 - End of Line Style (non-Unix)
To remove the non-Unix line endings (see also [http://blog.gyoshev.net/2013/08/29/normalizing-line-endings-in-git-repositories/]).
3.1. Create this script and call it ''fix-eol.sh'':
#!/bin/bash
find . -type f -exec grep -Iq . {} \; -and -print | xargs dos2unix -k -q
(the ''find'' and ''grep'' commands here try very hard to find text files)
3.2. Run the actual rewrite. Note, we call this ''postprocessing'' because it's time-consuming so it's worth to run it after the step #2 only on files that we really need in the edited repo.
cd /path/to/the/repo
git filter-branch --tree-filter fix-eol.sh  -- --all


==Links==
==Links==
Line 164: Line 195:
*'''[[Calligra/Schedules/3.0/Porting_Notes|Calligra 3.0 Porting Notes]]'''
*'''[[Calligra/Schedules/3.0/Porting_Notes|Calligra 3.0 Porting Notes]]'''
*'''[https://todo.kde.org/?controller=board&action=show&project_id=7 Kexi tasks at todo.kde.org] (KDE Identity needed), [https://todo.kde.org/?controller=board&action=readonly&token=d39ee567c32511a5f32e69054902b64a4118930be455b4cf7a9c7b83fe87 public read-only view]'''
*'''[https://todo.kde.org/?controller=board&action=show&project_id=7 Kexi tasks at todo.kde.org] (KDE Identity needed), [https://todo.kde.org/?controller=board&action=readonly&token=d39ee567c32511a5f32e69054902b64a4118930be455b4cf7a9c7b83fe87 public read-only view]'''
*'''[https://todo.kde.org/?controller=board&action=show&project_id=7 Predicate tasks at todo.kde.org] (KDE Identity needed), [https://todo.kde.org/?controller=board&action=readonly&token=d39ee567c32511a5f32e69054902b64a4118930be455b4cf7a9c7b83fe87 public read-only view]'''
*OLD: '''[https://todo.kde.org/?controller=board&action=show&project_id=7 Predicate tasks at todo.kde.org] (KDE Identity needed), [https://todo.kde.org/?controller=board&action=readonly&token=d39ee567c32511a5f32e69054902b64a4118930be455b4cf7a9c7b83fe87 public read-only view]'''
*[http://quickgit.kde.org/?p=kdeexamples.git&a=tree&f=framework-template Framework repository template] - use for creating new frameworks
*[http://quickgit.kde.org/?p=kdeexamples.git&a=tree&f=framework-template Framework repository template] - use for creating new frameworks
**real world examples: [https://projects.kde.org/projects/extragear/graphics/kdiagram KDiagram] (recommended because it's a Calligra project -- next-gen kdchart, kdgantt), [https://projects.kde.org/projects/frameworks/karchive/repository KArchive] (tier 1), [https://projects.kde.org/projects/frameworks/kio/repository KIO] (tier 3), [https://projects.kde.org/projects/frameworks/kimageformats/repository KImageFormats plugins]
**real world examples: [https://projects.kde.org/projects/extragear/graphics/kdiagram KDiagram] (recommended because it's a Calligra project -- next-gen kdchart, kdgantt), [https://projects.kde.org/projects/frameworks/karchive/repository KArchive] (tier 1), [https://projects.kde.org/projects/frameworks/kio/repository KIO] (tier 3), [https://projects.kde.org/projects/frameworks/kimageformats/repository KImageFormats plugins]

Revision as of 10:24, 20 October 2017

Note

Draft. Based on notes from the Calligra 2014 Sprint. Each step leads to compiling (running?) state.
Started by jstaniek (talk) 19:06, 7 July 2014 (CET).


The grand plan:

  1. [100%, builds, example and autotests work] Port PredicateKDb↓ lib to Qt5/KF5
    1. [100%] Port the SQLite driver
    2. [100%] Port the MySQL driver
    3. [100%] Port the PostgreSQL driver
    4. [10%] Port the xBase driver
    5. [10%] Port the Sybase driver
  2. [100%, builds, example and autotests work] Move koproperty lib from calligra to a separate KProperty↓ repo, port to Qt5/KF5
  3. [100%, builds, example and autotests work] Move koreport lib from calligra to a separate KReport↓ repo, port to Qt5/KF5
    1. Remove scripting or port scripting?
  4. [100%] Port Kexi (without KexiDB/calligradb, reporting, koproperty, koreport) to Qt 5 and use KDb/KProperty/KReport
    1. [100%] Remove unused calligradb lib and parts of KexiDB lib
    2. [100%] Port the Table plugin
    3. [100%] Port the Query plugin
    4. [100%] Port the Form plugin
    5. [100%] Port the Report plugin
    6. [100%] Port the Import/Export plugin
    7. [20%] Port the Migration module
    8. [0%] Reimplement report scripting using a simple solution to keep backward compatibility with Kexi 2 (see Scripting::*, KRScriptConstants, KRScriptDraw, KRScriptDebug)
  5. [40%] Improve Kexi stability after porting
  6. -- Kexi 3.0 beta release here. To indicate the nature of this version we never release 3.0 stable. --
  7. Add tests and/or examples to KReport, make it easier to understand as a framework
    1. Port some code from Kexi Reports (e.g. KexiReportView [1]) to KReport
  8. Make KReport lib dependent on KDb and remove code/API that becomes redundant (e.g. KoReportData)
  9. Add a new scripting module
  10. Use the module in Kexi Reports too, keeping backward compatibility with Kexi 2 scripting (add unit tests before porting to be very sure that compatibility is kept)

Rules

  • To build Kexi, first build & install KDb, KProperty, KReport (master branches), i.e. for each:
cd <<BUILDDIR>>
cmake -DPRODUCTSET=kexi -DCMAKE_INSTALL_PREFIX=<<PREFIX>> -DCMAKE_BUILD_TYPE=Debug <<SRCDIR>>
  • Then use calligra.git 'master' branch: type "git checkout master".
  • Use personal git scratch repos to avoid stress with buildbreaks and unnecessary BUG/review notifications
  • Boud's advice: My approach is more like, run each of montel's scripts, fix what's wrong, then go on. It's not something that's easy to paralellize.
  • Don't run astyle on the code, it pollutes history, and astyle-kdelibs is a bit broken
  • Based on "Porting notes" below, keep porting of one aspect in one commit to simplify reviews
  • Use the Phabricator for reviews
  • Use kde-dev-scripts.git porting scripts, in particular
  • Libs that are separate repos should use the framework template (see Links) and be ported to this layout (for now it's just kdb, kproperty, kreport).

Notes and Status

  • TODO: publish detailed checklist (in a spreadsheet?)

Dependencies

Please read all this section carefully before taking action.

  • Qt 5. Get >=5.2 stable version from your distro, not forgetting about development packages (headers). If you feel you want it, build Qt 5, but it's usually not necessary extra work.
  • CMake: version >=2.8.12+ and 3.x are both supported. Get it from your distro (recommended) or build it.
  • KDE Frameworks 5 (KF5) are needed as build-time and run-time dependencies for Calligra, Kexi and our frameworks (KProperty, etc).
  • Use the kdesrc-build tool to conveniently build KF5. Building by hand in an option if you accept extra work usually not needed for our development (see Frameworks/Building/Details).
  • There is the Adjust the paths step. For Kexi development (and Calligra in general) it's good to re-use paths that we already use except the install prefix (kdedir), see the Recommended setup of the Calligra build instructions. This means that recommendation for the kdesrc-buildrc config file is:
 source-dir /home/YOU/kde/src
 build-dir /home/YOU/kde/build
 kdedir /home/YOU/kde/inst5

Note

If you used /home/YOU/kde/inst as a dir the Calligra 2.x and other kdelibs4-based software, do not mix it with the Calligra 3.x+ and other kf5-based software, otherwise you can experience crashes or other strange behaviour. Use /home/YOU/kde/inst5 for example.


  • Speed up: before using the KF5 with kdesrc-build, edit the kf5-qt5-build-include file and comment these lines if you don't want to build the workspace, applications and KDEPIM. Thus only "include kf5-frameworks-build-include" is left.
include kf5-workspace-build-include
include kf5-applications-build-include
include kf5-kdepim-build-include


  • In particular, the kdesrc-build will install extra CMake Modules: [2], which is needed to make our software able to find the KF5 components at build time.

KDb

  • Download/Build/Release info → KDb.
  • Use master branch, official for Qt 5/KF 5, for all platforms, always stable
  • Our target version is 3.0.0.
  • To avoid conflicts, Qt 4 version kept for reference, has version 0.0.0.
  • Depends only on system libraries and Qt, so it's a tier 1 framework
  • Windows and Mac-specific matters.
  • Porting to Qt 5 started by Wojtek Kosowicz (Jan 2015)
  • Porting to Qt 5/Mac OS X started by Wojtek Kosowicz too (Jan 2015)
  • Directory structure reorganized according to recommendations --Jstaniek (talk) 20:59, 23 February 2015 (UTC)

KProperty

  • Our target version is 3.0.0 and KProperty project in Playground/Libs category of KDE projects
  • Depends on system libraries and Qt, plus minor deps on KF5WidgetsAddons/KF5GuiAddons for some editors; later we may move this dependency to a plugin (so the core would stay Qt-only)
  • Official git repo: kproperty, master branch (use: git clone kde:kproperty)
  • Directory structure reorganized according to recommendations --Jstaniek (talk) 20:59, 23 February 2015 (UTC)

KReport

  • Our target version is 3.0.0 and KReport project in Playground/Libs category of KDE projects
  • To avoid conflicts, Qt 4 version kept for reference, has version 0.0.0.
  • Depends on system libraries, Qt, and KProperty (tier 1), so it's a tier 2 framework, some deps on KF5WidgetsAddons (may be moved to a plugin so the core would stay Qt-only)
  • Official git repo: kreport, master branch (use: git clone kde:kreport)
  • Directory structure reorganized according to recommendations --Jstaniek (talk) 20:59, 23 February 2015 (UTC)
  • Deps from calligra libs (copied, renamed): KoRuler, KoUnit

Build instructions

Kexi 3/Calligra 3 builing is explained in a (work-in-progress) doc: Calligra/Building/3. Our dependencies (KDb, KPropert, KReport) can be built in a usual way (cmake, make install). Don't forget to set environment variables in the same way as you do for Kexi/Calligra builds.

Porting scripts usage status

These porting scripts from the kf5 dir of the kde:kde-dev-scripts repo would be applied.
Legend: - = not run, + = run, N = not applicable, T = TODO(we know it's needed)

KDb | KProperty | KReport | Kexi | script-name
+ + + + adapt_cmakelists_file.pl
+ + + + adapt_knewstuff3_includes.pl
+ + + + add_missing_kpart_include.pl
+ + + + clean-forward-declaration.sh
+ + + + clean-includes.sh
N N N - cmakelists_install_vars.pl (N/A because KDb/KProperty/KReport don't use KDE_*DIR variables)
N N N N convert-kabc-to-kcontacts.pl (unused)
+ + + + convert-kaction.pl
+ + + - convert-kbuttongroup.pl
T + + + convert-kcmdlineargs.pl
+ + + N convert-kcolordialog.pl
+ + + + convert-kdebug.pl
N N N N convert-kdebug-with-argument.sh (N/A because KDb/KProperty/KReport don't use that)
N + N + convert-kdialog.pl (helps but KProperty is largery broken by this script)
N N N N convert-kdoublenuminput.pl
N N N N convert-kfiledialog.pl
N N N N convert-kfontcombobox.pl
N N N T convert-kicon.pl
N N N N convert-kimageio.pl
N N N N convert-kintnuminput.pl
N N N N convert-kintspinbox.pl
N N N N convert-kjob.pl
N N N + convert-klineedit.pl
N N N + convert-klistwidget.pl
N N N N convert-klocale-formatdate.pl
N N N + convert-kmd5.pl
N N N N convert-kmenubar.pl
N N N + convert-kmenu.pl
N N + + convert-kmimetype.pl
N N N + convert-kprogressdialog.pl
N N N + convert-kpushbutton.pl
N N N + convert-ksavefile.pl
N N N + convert-ksharedptr.pl
N N N + convert-kshortcut.pl
N N N N convert-ksplashscreen.pl
N N N + convert-kstandarddirs.pl
N N N + convert-ktabwidget.pl
N N N + convert-ktempdir.pl
+ N + + convert-ktemporaryfile.pl
N N N + convert-ktextbrowser.pl
N + + + convert-kurl.pl
N N N + convert-kvbox.pl
+ + + - convert-qkdebug-to-qcdebug.sh
+ + + + convert-qt5.pl
+ + + + convert-to-cmake-automoc.pl
T N N T convert-to-k4aboutdata.pl
- - - - convert-to-new-signal-slot-signal.pl
N N N N dnssd-to-kdnssd.sh
+ + + + fix-ecm-install-icons.pl
- - - - fix-kcm-module-desktop-file.sh
- - - - install_forwarding_headers.pl
N N N N port-kauthactions.pl
N N N + port_to_autogenerate_export_header.sh
+ + + + remove-kde4support.pl
N N N N remove_virtual_prefix.pl
N N N + resolve_kuit.py
T T T T search-kdelibs4support-header.sh
+ + + + frameworks/plasma-framework/src/tools/port-cmake-style.sh
+ + + - frameworks/plasma-framework/src/tools/port-qslots.sh
+ + + - frameworks/plasma-framework/src/tools/port-includes.sh

Git surgery

1. For editing large history you may need a fast filesystem. So a Linux RAM disk such as /dev/shm is your friend, deep-copy your repo temporarily there: [3].

  • To deep-copy use:
cd /path/to/ramdisk git clone /path/to/the/orig/repo --no-hardlinks --no-local
  • Consider switching off swap ('sudo swapoff -a') as swapping makes no sense, when we want to use RAM exclusively.
  • Don't forget to create a backup of the original repository and then replace the original repository with the copy edited, using git clone again:
cd /path/to/the/orig
rm -rf repo
git clone /path/to/ramdisk/repo --no-hardlinks --no-local

2. (Jstaniek (talk) 09:54, 23 February 2015 (UTC)) Repos are prepared by history rewrites such as:

git filter-branch --prune-empty -f --tree-filter \
'(mkdir -p src;mv Predicate src/predicate;mv Drivers src/drivers) 2> /dev/null || true' -- HEAD

3. Before your push your branch to a new repo, you may need some postprocessing if the commits contain non-Unix line endings, git push refuses to work then:

remote: Audit failure - Commit 9460d32f6f2f51c2a182b8faa40ff7537b7ac5c6 - End of Line Style (non-Unix)

To remove the non-Unix line endings (see also [4]).

3.1. Create this script and call it fix-eol.sh:

#!/bin/bash
find . -type f -exec grep -Iq . {} \; -and -print | xargs dos2unix -k -q

(the find and grep commands here try very hard to find text files)

3.2. Run the actual rewrite. Note, we call this postprocessing because it's time-consuming so it's worth to run it after the step #2 only on files that we really need in the edited repo.

cd /path/to/the/repo
git filter-branch --tree-filter fix-eol.sh  -- --all

Links

Docs

Background