GSoC/2020/StatusReports/KitaeKim: Difference between revisions

From KDE Community Wiki
< GSoC‎ | 2020‎ | StatusReports
 
(91 intermediate revisions by the same user not shown)
Line 1: Line 1:
[[File:Kirogi Goose 800x655.png|frame|center]]
== Imporove MAVLink integration of Kirogi ==
== Imporove MAVLink integration of Kirogi ==


The Kirogi is Ground Control Station(GCS) for controlling Unmanned Vehicles like drones developed recently.
The Kirogi is a Ground Control Station(GCS) for managing Unmanned Aerial Vehicles like drones.


It's comparatively young project so it dose not support all standards of MAVLink protocol which is the famous protocol between GCS and UAVs yet.
It's comparatively young project so it dose not support all standards of MAVLink protocol which is used to communicate with drone's firmware like PX4 and Ardupilot.


My goal is to imporove MAVLink protocol integration of Kirogi so that it supports all functionalities of the MAVLink protocol.
The goal is to imporove MAVLink protocol integration of Kirogi so that it supports most functionalities of the protocol.


'''Mentors:''' Tomaz Canabrava, Eike Hein, Patrick José Pereira, Sung Jae Cho
'''Mentors:''' Tomaz Canabrava, Eike Hein, Patrick José Pereira, Sung Jae Cho


== Work reports ==
== Progress Reports ==
=== Community Bonding Period ===
 
# Study about MAVLink protocol. ''- Done''
# Read code of Kirogi and QGroundControl in comparison. ''- Done''
# Study about Qt and C++ development. ''- In progress''
 
The first thing i did during this period is studying more about MAVLink protocol.
 
The MAVLink protocol packet is structured in this way:
 
[[File:A mavlink packet structure.png|frame|center]]<ref>https://mavlink.io/en/</ref>
 
This protocol uses sub-protocols to extend it's functionality. Those are identified using 'MSG ID' section of the packet. Additional information for sub-protocol is transported through 'PAYLOAD' section of the packet.


=== Work report 01 ===
First feature that will be implemented during phase 1 is integration of heartbeat protocol.


==== Brief description ====
The heartbeat protocol is one of MAVLink's sub-protocols. It is used to identify existence of vehicle along with its properties. This allows us to discover vehicle connected to the network and notice when they have been disconnected. It also allows us to handle incoming messages appropriately based on type of the vehicle and route messages to system on different interface.


Period: 04.17 ~ 06.14
Second thing i did during this period is reading code of Kirogi and QGroundControl in comparison. The QGroundControl is a standard de facto control station for MAVLink drones.


Topics:
The QGroundControl has MultiVehicleManager and LinkManager which are used to manage vehicles and links as name suggests. The MultiVehicleManager creates vehicle object when incoming message has unknown system id. The vehicle object contains information of the actual vehicle like system id and connection status. The LinkManager creates link object when user wants to establish new connection. Both have pointer to ToolBox which contains pointer to main components like MultiVehicleManager and LinkManager so they can reference each other when it's needed.
# Command queue
# Bug fix
# Heartbeat
# Multi-vehicle support
# Plugin for GUI


The command protocol, one of MAVLink subprotocols, specifies that commands must be acknowledged by ack message.
The Kirogi aims to support various firmwares including PX4 and Ardupilot. For that, it uses KPlugin.
So i implemented command queue to store commands and send repeatedly until it gets acknowledged.


I found minor bug in Kirogi's plugin system. Kirogi used deprecated function to load plugins.
Kirogi's UI is general to all plugins. I felt that this generality prevent me to implement various features for plugins. It would be modified to load some plugin-specific elements at runtime in future.


I'm trying to support all functionalities of heartbeat protocol. this is still WIP.
Main components of MAVLink plugin are MAVLinkVehicle and MAVLinkConnection. The MAVLinkVehicle contains information of the actual vehicle. This class has method to processes incoming message and manage connection. The MAVLinkConnection contains QUdpSocket and method to receive and parse incoming messages. Other plugins of Kirogi have same structure.


And i'm also trying to make Kirogi support multiple vehicles at the same time. (the protocol supports up to 255 vehicles)
It seems implementing all functionalities of MAVLink not breaking general structure of Kirogi is quite challenging.


Most challenging thing in this period was GUI element plugin. I need to load GUI elements for Mission Planner, Vehicle Configuration in runtime. It seemed easy at first glance but i felt that most ways to approach makes codebase dirty.
=== Coding Period Phase 1 ===
Tomaz and Patrick implemented POC for that and waiting for review. So i will try to find best way to implement it and help them.


==== Accepted commits ====
# Improve integration of command protocol. ''- Done''
* d63efbf54e1726d0ea07d25dc5c76295f0c7a04a
# Implement vehicle manager for managing multiple vehicles. ''- Done''
* b19d034b204c73849793fd824270cd897911cea6
# Write unit tests of implementation. ''- Cancelled''
* 15b48b52ef3e80bf3d2e14031623bbd11c9b2c6c
# Write a documentation about implementation. ''- Done''
* 0d9d7fe32a6c0eaed32770b50de5e36496caad06
# Study about Qt and C++ development. ''- Keep doing''


==== Personal commits ====
Command protocol specifies that every commands should be followed by ack message from the vehicle that receives the command. If no ack message was received, GCS should resend message until it reaches max count. This feature was implemented before GSoC begins.
* 2f70d1a2b563ae70e9cba016fba1c15e9e0137eb
 
* 2dbaa3d96288b7768094fc84203d2d44e7b26689
New MAVLinkVehicleManager class creates vehicle object when incoming message has unknown system id. It stores vehicle object in MAVLinkVehicleModel class. This class inherits from QAbstractList so that in can provide list of vehicles to UI.
* 2307ac6929597c3d5b8902bf861e5cdc4bd712a3
 
* f9aabb2f86ef57468e02bf4eb555321674e3facb
The Kirogi use KPlugin XT but it can't configure plugin-specific options. The MAVLinkConnectionConfig class is for configuring MAVLink's UDP/TCP/Serial connection. This class is connected to UI so that users can configure options using methods.
* 44f77d7e259e00778b58c2f1f0cb48d89a8b3142
 
* ...
About the future works, Kirogi's UI is generic to every plugins but it seems some plugins need it's own UI. In example, Plan Manager for MAVLink plugin. My conclusion is that I can't implement UI that is really generic to all plugins so I need to load some of UI elements at runtime based on the plugin. 'Check List', 'Global Drawer', 'Navigation Map', 'Settings' pages would be appropriate places to load those elements.
 
Some miscellaneous works like renaming some redundant variables or modifying existing classes are done during this phase.
 
I was planning to adopt Test-Driven development at the first time, but it seems there are many other things to learn like how drone's firmware works so that I have not enough time to study and adopt it.
 
=== Coding Period Phase 2 ===
 
* Support TCP and Serial connection. ''- Done''
* Implement UI for selecting multiple vehicles. ''- Done''
* Divide PR into smaller pieces. ''- In progress''
* Write a documentation about implementation. ''- Done''
* Study about Qt and C++ development. ''- Keep doing''
 
Because there's no code for TCP and Serial connection in Kirogi's codebase, I created classes for
connection abstraction including 'ConnectionConfiguration', 'TcpConfiguraiton' and so on.
 
There's an 'AbstractConnection' class for connection abstraction but if we want to manage it's
properties at Kirogi's base level(libkirogi and QML), we have to provide a generic way to manage
connection's property.
 
We can't just use same connection class for all plugins because the way how connection-related
procedures defined depends on the type of drone.
 
At the first time, I was thinking to implement things like 'AbstractUdpConnection',
'AbstractTcpConnection' that defines common properties of those connections and implement plugin's
connection inheriting them.
 
But I found that some connections need more than one types of methods to operate. In example, some
connection may use TCP for handshaking and UDP for communicating.
 
So I decided to mix 'AbstractConnection' and some kind of connection configuration class to
implement plugin's connection. In this way, we can implement 'ArbitraryConnection' mixing
'AbstractConnection' with 'UdpConfiguration' and 'TcpConfiguration' which inherits
'ConnectionConfiguration'.
 
I implemented connection classes for mavlink like 'MAVLinkUdpConnection', 'MAVLinkTcpConnection' and
'MAVLinkSerialConnection' using them.
 
I also wrote code for managing multiple connections as well although it's not a thing that I planned
do at this phase. That's because I thought that I had to think about implementations that make use
of connection-related classes I wrote while actually writing them.
 
I also implemented basic drawler for selecting vehicle inside of 'Check List' element.
 
I was doing everything in one branch and created one PR for everything which results in that my
mentors couldn't review my work efficiently. So I started break things into smaller pieces.
 
=== Coding Period Phase 3 ===
 
* Divide PR into smaller pieces. ''- Done''
* Implement connection manager for managing multiple connections. ''- Done''
* Implement UI for managing multiple connections. ''- Not done''
* Write a documentation about implementation. ''- Done''
* Study about Qt and C++ development. ''- Keep doing''
 
I divided the branch that I mentioned previous progress report into four smaller branches.
 
One for basic chamges for all of other branches, others for connection abstraction, vehicle
management and connection management. Latter three branches depend on first one and one for
connection management depends on conneciton abstraction.
 
I also implemented connection manager for managing multiple connections. new
'AbstractConnectionModel' class, which is used for managing connections, creates new configuration
with default value when user wants to create new connection. Then user can edit it and create
connection based on the configuraiton.
 
When user wants to edit existing one, it duplicates configuration of selected connection and user
can edit it. When user choose to save it, the model copies value that user wrote into existing
connection's configuration.
 
The reason why I decided to copy values rather than replacing existing connection with new
connection is because there maybe a vehicle that uses connection that is being edited. In that case,
it's more easier copy values than replacing connection object and make vehicle to use it.
 
This class also provides list of available serial port. Some invalid ports will be filtered to
prevent users to select them.
 
I was thinking to implement basic UI for managing connections but I didn't. because my mentors
pointed out that it's better to manage vehicles and connections in more generic way rather than
implementing managers for each plugin. In this way, users can control different types of vehicles
like parrot bebop and pixhawk at the same time. It seems good idea for me so I'll start implementing
UI for manging vehicles and connections after backend architecture for that feature gets specifed.
 
=== Blog Writings ===
 
* [https://www.develoot.me/improve-mavlink-integration-of-kirogi-progress-report-1/ Improve MAVLink Integration of Kirogi – Progress Report 1]
* [https://www.develoot.me/improve-mavlink-protocol-integration-progress-report-2/ Improve MAVLink Integration of Kirogi – Progress Report 2]
* [https://www.develoot.me/improve-mavlink-integration-of-kirogi-progress-report-3/ Improve MAVLink Integration of Kirogi – Progress Report 3]
 
== Branches and PRs ==
 
* [https://invent.kde.org/utilities/kirogi/-/merge_requests/126 Improve basic integration of mavlink protocol]
* [https://invent.kde.org/develoot/kirogi/-/tree/01-mavlink-connection-types 01-mavlink-connection-types]
* [https://invent.kde.org/develoot/kirogi/-/tree/02-mavlink-vehicle-model 02-mavlink-vehicle-model]
* [https://invent.kde.org/develoot/kirogi/-/tree/03-mavlink-connection-model 03-mavlink-connection-model]


== Related Links ==
== Related Links ==


==== Project Proposal ====
* [https://docs.google.com/document/d/1J0uazpHGZaybt9lW7UqP8c4xnRVXMbqxOMJqTNBHsZg/edit?usp=sharing Proposal: Improve MAVLink integration of Kirogi]
* [https://invent.kde.org/develoot/kirogi Kitae Kim / Kirogi]
 
== Contacts ==


[https://docs.google.com/document/d/1J0uazpHGZaybt9lW7UqP8c4xnRVXMbqxOMJqTNBHsZg/edit?usp=sharing Improve MAVLink integration of Kirogi]
Email: develoot@gmail.com


==== GitLab Personal Repository ====
LinkedIn: https://www.linkedin.com/in/kitae-kim-511032182/


[https://invent.kde.org/develoot/kirogi Kitae Kim / Kirogi]
== Footnotes ==

Latest revision as of 01:40, 31 August 2020

Imporove MAVLink integration of Kirogi

The Kirogi is a Ground Control Station(GCS) for managing Unmanned Aerial Vehicles like drones.

It's comparatively young project so it dose not support all standards of MAVLink protocol which is used to communicate with drone's firmware like PX4 and Ardupilot.

The goal is to imporove MAVLink protocol integration of Kirogi so that it supports most functionalities of the protocol.

Mentors: Tomaz Canabrava, Eike Hein, Patrick José Pereira, Sung Jae Cho

Progress Reports

Community Bonding Period

  1. Study about MAVLink protocol. - Done
  2. Read code of Kirogi and QGroundControl in comparison. - Done
  3. Study about Qt and C++ development. - In progress

The first thing i did during this period is studying more about MAVLink protocol.

The MAVLink protocol packet is structured in this way:

[1]

This protocol uses sub-protocols to extend it's functionality. Those are identified using 'MSG ID' section of the packet. Additional information for sub-protocol is transported through 'PAYLOAD' section of the packet.

First feature that will be implemented during phase 1 is integration of heartbeat protocol.

The heartbeat protocol is one of MAVLink's sub-protocols. It is used to identify existence of vehicle along with its properties. This allows us to discover vehicle connected to the network and notice when they have been disconnected. It also allows us to handle incoming messages appropriately based on type of the vehicle and route messages to system on different interface.

Second thing i did during this period is reading code of Kirogi and QGroundControl in comparison. The QGroundControl is a standard de facto control station for MAVLink drones.

The QGroundControl has MultiVehicleManager and LinkManager which are used to manage vehicles and links as name suggests. The MultiVehicleManager creates vehicle object when incoming message has unknown system id. The vehicle object contains information of the actual vehicle like system id and connection status. The LinkManager creates link object when user wants to establish new connection. Both have pointer to ToolBox which contains pointer to main components like MultiVehicleManager and LinkManager so they can reference each other when it's needed.

The Kirogi aims to support various firmwares including PX4 and Ardupilot. For that, it uses KPlugin.

Kirogi's UI is general to all plugins. I felt that this generality prevent me to implement various features for plugins. It would be modified to load some plugin-specific elements at runtime in future.

Main components of MAVLink plugin are MAVLinkVehicle and MAVLinkConnection. The MAVLinkVehicle contains information of the actual vehicle. This class has method to processes incoming message and manage connection. The MAVLinkConnection contains QUdpSocket and method to receive and parse incoming messages. Other plugins of Kirogi have same structure.

It seems implementing all functionalities of MAVLink not breaking general structure of Kirogi is quite challenging.

Coding Period Phase 1

  1. Improve integration of command protocol. - Done
  2. Implement vehicle manager for managing multiple vehicles. - Done
  3. Write unit tests of implementation. - Cancelled
  4. Write a documentation about implementation. - Done
  5. Study about Qt and C++ development. - Keep doing

Command protocol specifies that every commands should be followed by ack message from the vehicle that receives the command. If no ack message was received, GCS should resend message until it reaches max count. This feature was implemented before GSoC begins.

New MAVLinkVehicleManager class creates vehicle object when incoming message has unknown system id. It stores vehicle object in MAVLinkVehicleModel class. This class inherits from QAbstractList so that in can provide list of vehicles to UI.

The Kirogi use KPlugin XT but it can't configure plugin-specific options. The MAVLinkConnectionConfig class is for configuring MAVLink's UDP/TCP/Serial connection. This class is connected to UI so that users can configure options using methods.

About the future works, Kirogi's UI is generic to every plugins but it seems some plugins need it's own UI. In example, Plan Manager for MAVLink plugin. My conclusion is that I can't implement UI that is really generic to all plugins so I need to load some of UI elements at runtime based on the plugin. 'Check List', 'Global Drawer', 'Navigation Map', 'Settings' pages would be appropriate places to load those elements.

Some miscellaneous works like renaming some redundant variables or modifying existing classes are done during this phase.

I was planning to adopt Test-Driven development at the first time, but it seems there are many other things to learn like how drone's firmware works so that I have not enough time to study and adopt it.

Coding Period Phase 2

  • Support TCP and Serial connection. - Done
  • Implement UI for selecting multiple vehicles. - Done
  • Divide PR into smaller pieces. - In progress
  • Write a documentation about implementation. - Done
  • Study about Qt and C++ development. - Keep doing

Because there's no code for TCP and Serial connection in Kirogi's codebase, I created classes for connection abstraction including 'ConnectionConfiguration', 'TcpConfiguraiton' and so on.

There's an 'AbstractConnection' class for connection abstraction but if we want to manage it's properties at Kirogi's base level(libkirogi and QML), we have to provide a generic way to manage connection's property.

We can't just use same connection class for all plugins because the way how connection-related procedures defined depends on the type of drone.

At the first time, I was thinking to implement things like 'AbstractUdpConnection', 'AbstractTcpConnection' that defines common properties of those connections and implement plugin's connection inheriting them.

But I found that some connections need more than one types of methods to operate. In example, some connection may use TCP for handshaking and UDP for communicating.

So I decided to mix 'AbstractConnection' and some kind of connection configuration class to implement plugin's connection. In this way, we can implement 'ArbitraryConnection' mixing 'AbstractConnection' with 'UdpConfiguration' and 'TcpConfiguration' which inherits 'ConnectionConfiguration'.

I implemented connection classes for mavlink like 'MAVLinkUdpConnection', 'MAVLinkTcpConnection' and 'MAVLinkSerialConnection' using them.

I also wrote code for managing multiple connections as well although it's not a thing that I planned do at this phase. That's because I thought that I had to think about implementations that make use of connection-related classes I wrote while actually writing them.

I also implemented basic drawler for selecting vehicle inside of 'Check List' element.

I was doing everything in one branch and created one PR for everything which results in that my mentors couldn't review my work efficiently. So I started break things into smaller pieces.

Coding Period Phase 3

  • Divide PR into smaller pieces. - Done
  • Implement connection manager for managing multiple connections. - Done
  • Implement UI for managing multiple connections. - Not done
  • Write a documentation about implementation. - Done
  • Study about Qt and C++ development. - Keep doing

I divided the branch that I mentioned previous progress report into four smaller branches.

One for basic chamges for all of other branches, others for connection abstraction, vehicle management and connection management. Latter three branches depend on first one and one for connection management depends on conneciton abstraction.

I also implemented connection manager for managing multiple connections. new 'AbstractConnectionModel' class, which is used for managing connections, creates new configuration with default value when user wants to create new connection. Then user can edit it and create connection based on the configuraiton.

When user wants to edit existing one, it duplicates configuration of selected connection and user can edit it. When user choose to save it, the model copies value that user wrote into existing connection's configuration.

The reason why I decided to copy values rather than replacing existing connection with new connection is because there maybe a vehicle that uses connection that is being edited. In that case, it's more easier copy values than replacing connection object and make vehicle to use it.

This class also provides list of available serial port. Some invalid ports will be filtered to prevent users to select them.

I was thinking to implement basic UI for managing connections but I didn't. because my mentors pointed out that it's better to manage vehicles and connections in more generic way rather than implementing managers for each plugin. In this way, users can control different types of vehicles like parrot bebop and pixhawk at the same time. It seems good idea for me so I'll start implementing UI for manging vehicles and connections after backend architecture for that feature gets specifed.

Blog Writings

Branches and PRs

Related Links

Contacts

Email: [email protected]

LinkedIn: https://www.linkedin.com/in/kitae-kim-511032182/

Footnotes