GSoC/2021/StatusReports/LucasWang
KDE Connect iOS <> Status Report & Reflections
Key Links, Commits, People, and Articles
Links:
- GSoC Project Link
- KDE Connect iOS 2021 Repository (currently separate from the KDE Connect iOS 2014 repository)
- Network Tester repository, for testing various frameworks to judge their feasibility for integration with KDE Connect iOS
Key Commits:
People:
Posts, Docs, and Articles:
How It Started:
As of April 2021, the iOS KDE Connect application possesses several issues:
Outdated UI design and features such as optimizing for newer aspect ratios, icons, fonts, and accessibility, etc.
The app is lacking many core functionalities of KDE Connect that other currently available versions have. Since Apple has implemented many of those features into iOS since 2014, the app would now be able to use them to offer the missing functionalities.
The backend portion of the app that governs LAN network communications is currently functioning inconsistently and only under a very specific series of actions.
The goal of this project is to address the problems listed above with an "analysis and rewrite" process of much of the app’s current code base with the newest Apple development language and UI framework, Swift and SwiftUI. Aside from solving the immediate problems with the current app, rewriting the foundation in Swift and SwiftUI would give the codebase a strong footing to expand and adapt to future contributions. SwiftUI’s cross-platform deployment capabilities also give the possibilities for forks of the codebase for other Apple platforms such as macOS and watchOS.
As such, the initial proposal, finalized in early April, outlined the following goals and deliverables:
- A completely revamped interface of the KDE Connect iOS application written in SwiftUI and adhering to the newest iOS design language.
- Additional functions were added to the new app to match the core functionalities present on other versions of KDE Connect such as the Android version.
- A revised network communications backend.
- Documentation of the unit testing done throughout the project and, using the Android KDE Connect as a reference, document the state and progress of the iOS version.
How It's Going: Key Periods & Moments
Note: Some of these events overlap with each other, and they are also not in strictly sequential order.
May - June 7th: I Discussed and re-checked project proposal with mentors (timeline, other factors such as project resources, etc.). I joined KDE's Apple Developer Group to assess the situation as KDE Connect iOS 2021 is likely KDE's first project to potentially reach the App Store Connect stage. I also raised the question and potential problem of accessibility to Apple's platforms and frameworks needed to develop and contribute to this project, and floated the idea of future consideration of the MacStadium FOSS program which provides FOSS projects with a free Cloud Mac Mini. Having dealt with issues like this in software clubs in school before, I really did not want the lack of accessibility to Apple's platforms to be an obstacle for potential contributors.
June 7th - 14th: GSoC's coding period started just as I wrapped up my university spring quarter and was drowning in final projects and exams. Per previous revision with mentors, the coding period for this project will be starting roughly a week later around June 14th.
June 14th - July 7th: The first prototype of the UI/UX built completely in SwiftUI was ready for presentation at the upcoming monthly KDE Connect Development meeting. Other components of the app were also re-written in Swift, such as UserDefaults for persistent storage, cross-app data flow, etc.
July 3rd: It was discovered that the huge, breaking change between iOS 12 (which my mentor Inoki was running) and iOS 14 (which I was running) was responsible for the confusing and seemingly unexplainable difficulties that I was having getting UDP broadcast to work. iOS 14 requires developer to apply for an entitlement in order to use broadcast. I began looking into the complex Apple Developer group structure as the application would requires coordination with the KDE Admins.
July 13th: The revamped UI and the Network testing app was presented to the KDE Connect developer group at the monthly meeting. The project was met with positive feedback from the group. KDE Admins were also present so I was able to inform them of the upcoming coordination required to apply for the broadcast entitlement from Apple.
July 1st - July 20th: The first prototype of the LAN communication backend built entirely using the native Swift Network framework is functional. The tester app has an intractable selector for changing modes (UDP/TCP) and is capable of sending and receiving both messages as well as sending files via the standard system file picker and receiving files by saving them into the app's document directory to and from any IP address (including broadcasting) and using any port. I found the tester at this point to be quite similar to the command line tools of netcat, and actually started using it with netcat on my Linux and Android devices to transfer files and links.
July 27th: Further testing with a backend revision in Swift with the native Network framework was halted by this thread on the Apple Dev Forums. The lack of ability to start TLS on an existing TCP connection without spawning a new one is a dealbreaker for the compatibility of KDE Connect iOS with the Qt and Android versions. Maybe the Network framework can still be used in the future for a new file transfer backend? But even so, we might have spent weeks exploring a framework that we might not end up using. Priorities now are to try to bridge the 2014 Objective-C backend and see what comes out of it.
July 28th - August 11th: After putting aside the work that we've already done regarding the Swift Network framework, we started focusing entirely on getting the existing Objective-C backend from 2014 to work and started integrating it with the new Swift codebase. We managed to get pairing working and optimized as well as to lay down a solid foundation for future work to be done for the Plugins. File transferring is working as well (but could be optimized further). There's still some additional work that needs to be done to the LAN backend, namely the storing and verification of certificates.
August 12th - August 20th: Despite having a solid foundation already in place, we're running out of time to implement the more complex Plugins such as remote mouse control and MPRIS, as well as to implement certificates to be stored in keychains. Realistically, (due to my schedule and other events) August 20th is the last day that I would have to close up GSoC. I've spent most of the weekdays discussion reflections with my mentors and getting them in words into blog posts and documentations. I'm going to pre-configure the 3 blog posts to be published over the next 2 days while I'm away.
Key Progress Point: Revamped UI/UX in SwiftUI
A new UI, written entirely in SwiftUI, is a major goal of the project as it would not only modernize the interface’s appearance and functionality but also place the project and new codebase on a solid footing for any future contributions and developments.
SwifUI (introduced in 2019) comes to mind as a strong candidate for building UI that adheres closely to the Cupertino language and also one that provides quite a few nice and (mostly) straightforward abstractions for factors such as implementing accessibility features (voice lines for voiceover, etc.) and its declarative approach (similar to that of Flutter’s) allows for relatively clean code with single sources-of-truth, automatic re-render of views when designated variables change, and involves very little interactions with the ViewControllers present in UIKit and Storyboard.
In addition to adhering to the Cupertino design language, KDE Connect iOS also aims to provide an experience and interactive process that is as close to the Android version as possible by implementing Material design elements into their “equivalents” in Cupertino.
Key Progress Point: Getting set-up on the Apple Developer Platform & Applying for the Multicast Entitlement
After getting added into the KDE e.V. Apple Developer Group in late May, I spent a fair amount of time exploring and getting used to the Apple Developer Group portal. KDE's portal was mostly empty, with only a certificate for signing the macOS version of Krita, a different KDE project built with Qt. It seems like KDE Connect iOS would likely be the first KDE project built with Apple's native frameworks that makes it to the App Store Connect stage.
Key Progress Point: Bridging of the 2014 Backend with the new Swift Components
The programming language(s) of the project is certainly interesting, containing perhaps an equal mix of Swift and Objective-C, two completely different languages in both appearance and styles. Objective-C, the language that KDE Connect iOS 2014 was written entirely in, dates back to the NeXT days before Steve Jobs returned to Apple in the 1990s. Swift, on the other hand, was introduced from scratch in 2014 to serve as the “flagship” programming language for Apple native development going forward. SwiftUI, the youngest of them all, was introduced in 2019 as the new UI/UX framework with a declarative syntax (similar to that of Flutter’s), a ViewController-less architecture, single sources of truth with binding variables, and more. Therefore it is important that we somehow get these seemingly very different frameworks working together to produce the final KDE Connect iOS 2021.
To achieve this, a considerable amount of bridging (more than I had anticipated when I first started the project) was done between the Objective-C and Swift codebase. The general gist of the structure is that: most Objective-C classes and methods can be directly called from Swift (though for some reason Xcode seems to randomly rename the names of classes and methods and provide them in a suggested fix), while Swift classes and methods need to conform to @objc and then be automatically pulled into an Xcode-generated header file for importing into Objective-C code.
The final product of the bridging retained the file structure of the original project, but with Swift and Objective-C replaced and inserted wherever needed. The file structure can be roughly categorized into 6 sections:
LAN backend section: consisting of BaseLink.h/m, LanLink.h/m, BaseLinkProvider.h/m, LanLinkProvider.h/m. Responsible for establishing and managing LAN connections, including sending and receiving of data, handling sockets and connections, loading and verifying TLS identities, etc.
Network package section: consisting of NetworkPackage.h/m. Responsible for defining, serializing, and deserializing Network packets, the fundamental communication blocks of KDE Connect.
Device section: consisting of Device.h/m. Responsible for defining device objects. These are the objects that represent any remote devices that the host has interacted with, including their information, status, and any functionalities that they might have.
App background function: consisting of BackgroundService.h/m. Responsible for defining and executing functionalities that are required somewhere else in the app, such as refreshing device lists, saving remembered devices, taking the appropriate actions when device status changes (pairing, rejected, etc.).
Plugins section: consisting of PluginService.swift and the various .swift plugin files/classes. These classes are responsible for handling individual plugin functions, much like how plugins are handled in the Qt and Android versions of KDE Connect.
Views/UI section: consisting of the various .swift files, written in the SwiftUI frontend framework, to define the visible components
