Necessitas/Java/Redesign

Overview drawing

Base technique

At QCS we found out that it is very impractical to have that many Java classes in sourcecode form in each application. Simply because every class becomes part of the public interface. This will cause a huge maintenance burden. I'd simply say it would not work. We need to move large parts of the code into places that are independent from the application. This however strongly required the need to add classes from a foreign APK or DEX file to the application's classpath. We were not sure whether Android would allow this but it indeed does. As a proof of concept we wrote a demo[0] application which can load any (!) APK or DEX file and create an instance of any class and call its toString() method. The POC worked in the way we wanted on a rooted but also on a normal locked device.

With this technique at hand we can do the following:

• Move all code that is not really necessary for app startup into separate APK files
• Define an interface between App and Loader which can handle updates (that allows for incompatible changes in the App code over time)

App

This is the source code that is part of each Android-Qt application. The user *may* change little pieces of the code as long as it does not violate the interface to the Loader or Ministro. What the user can do is calling any methods of the public interfaces to the loader for example. We will mark the pieces of code that the user *must* not remove or modify otherwise. This whole thing will be as small as possible! Whenever there is the need to change the App code in an incompatible way we need to introduce a new Loader interface. E.g. we will have a LoaderVersion1 interface for now. When it changes the App code will ask for LoaderVersion2. The Loader's reponsibilities will be explained below.

Loader

The Loader connects methods from the App with the Android Activity class. It will implement all available methods in order to gain maximum control over the App code without letting the App decide by itself. The Loader is published by us and can as such be updated over time allowing *us* to fix bugs, introduce new feature to *existing* applications without changing them. There might be applications with different versions of the starter code in it. Each of those version requires a specific interface in the Loader. The most up to date version of the Loader will understand (=implement) and offer all interfaces. That way old and new application start code is always compatible with the Loader. If a Loader cannot satisfy the requested interface of an application then this means that the current version of the Loader is too old and it will update itself. (The Loader will technically be part of Ministro, when Ministro is installed. If the user is a developer and chose 'use local libs' then the loader will also be copied to a well known directory and directly included into the Apps classpath. As such at runtime of an App the Loader APK will be part of the App's process space!)

Ministro

Ministro does what it does right now: It downloads the libs (if necessary), provides the locations of the libraries and tells the app that it can start. I'm actually thinking of putting the 'satisfy a certain Loader interface version' into Ministro, too. Because that seems like the best place for it). Unlike now the interface between App and Ministro will become upgrade compatible. While changes will happen less frequently it is very important that we allow and can deal with incompatible changes to the interface between Ministro and the App.

Bridge

These are all the classes that interact in some way with the C++ code of the Android-Qt port. Unlike now they will be a private interface. Meaning that we can do *any* change we want in them. For each supported Qt version (4.8, 5.0 etc.) there will be an accompanying bridge. The bridge may even consist of multiple APKs if we want to save space (e.g. if QtMobility is not used, then the classes for it do not need to be available as well).

Qt

These are the native Qt libraries. No change needed. Except that any code can *assume* the existence of certain bridge classes and can resolve class names at will.

Caveats

Android-Qt changes

From a development perspective this is quite a task. The Java code changes are not so difficult (for me at least) but I have no clue how we can modify the Android-Qt build to also compile Java classes, dexify and make an APK out of them. Because this is what we need. When you compile Qt for Android you'll need a Java compiler, the DEX tool and something that creates an APK for you. IOW this is where I need you help.

This problem is conceptually solved. The build process will be enhanced to support compiling Java classes via QMake.

Ministro changes

The Ministro build also needs to be changed a bit. The Ministro APK could contain the Loader classes. That would not be a problem. However somehow QtCreator needs the Loader classes too when using 'use local libs' flag. I propose that the SDK installer also downloads the Loader classes (which might be the same as Ministro but as a library not an application) from a known location.

The decision where the Loader classes end up is something that is completely under our control. I propose that we start simple and bundle them in Ministro. When things become more difficult (multiple Loaders) we can transparently change to a repository based model. In fact the Proof-Of-Concept brought this insight.

Sharing code

I plan to use real Java interfaces in order to define the public interface. An alternative approach would be to rely on reflection only. However that will make the interface very opaque and the code will be difficult to read for anyone with only a small knowledge of Java. Having interfaces however means we will need to share bits of sourcecode between the projects (namely the interface file). I'm still contemplating how to do this properly.

Proof of concept - Java

This repository contains a proof of concept implementation in pure Java. There are three Ministro implementations: One is a first generation Ministro and the other two are a 2nd generation of which only one can satisfy two generations of the Loader. There are also three Applications: One is a 1st generation app, the other two are 2nd generation app with different requirements on the Loader generation.

The first generation app can work together with all Ministro implementations. This is because the newer ones supports the older interface.

To see it all working download and unpack the file, run './compile' from the top folder, then go into the 'app_v1', 'app_v2' and/or 'app_v2_2' folder and run the scripts 'run-against-v1', 'run-against-v2' and/or 'run-against-v2_2'.

All the source files contain extensive documentation at the beginning of the file which explains it purpose in the framework. For a good understanding I suggest the following order of reading: v1, app_v1, v2, app_v2, v2_2, app_v2_2. Many files are unchanged in the higher generation implementations. So don't worry it is not that much text.

Proof of concept - Android

With a better understand for the problem an Android PoC is being developed. The sourcecode is available from this repository. There is an App called appv1, a Ministro implementation and a Loader.