In 2006 a Usability Survey was done. This page contains the restored results.
The purpose of this study was to get insights into Kivio's major user base, their tasks and needs. Learning more about the usage of diagramming tools, the context of diagram creation and application, as well as the required preciseness were important questions in this scope.
Based on this information, the survey should allow us to illustrate typical usage patterns, and to learn about feature relevance and priorities. It should help us to determine if the features planned in the scope of Kivio's redesign are important to users.
Also, the survey aimed at determining the users' opinions about Kivio and evaluate the differences to other diagramming applications. Reasons for prominence and gaps should be identified and considered in the future work on Kivio.
Finally, the survey should give us widespread information about users of diagramming applications which may be used for persona creation. This includes where they live, their age and gender, education and how experienced they are with computers and with creating diagrams.
The survey was publically announced on planetkde, kdedevelopers and several Linux- and Windows forums. Furthermore, private invitations were sent to known Kivio users. The survey addressed not only actual Kivio users, but anybody who occasionally creates diagrams, no matter if using a software tool or paper and pen.
It was expected that the choice of these publication channels would influence the type of participants towards a rather technical user base. As it is assumed that the majority of Kivio's actual user base is technically oriented this fact was appreciated.
After the public announcement on June 16th the survey was online for two weeks, until June 30th.
For the survey, the UCCASS survey tool was utilised. All in all, approximately 35 questions were asked, partly limited choice, partly Likert-scales, partly free-form comments. They addressed information about user demographics, general computer experience, diagramming purpose, habits and experience, Kivio experience and evaluation, experience and evaluation regarding other diagramming tools, and finally an importance rating of several planned features for the Kivio rewrite.
The complete list of questions can be seen in the survey preparation section.
inexperienced - 13
14.94% - 36
41.38% - 22
25.29% - 14
16.09% experienced - 2
2.30% Total Answers - 87
unsatisfied - 5
5.75% - 27
31.03% - 32
36.78% - 22
25.29% satisfied - 1
1.15% Total Answers - 87
Of the 170 participants who made use of other applications, most claimed to make use of Dia, Visio, OpenOffice.org, Umbrello and Pen and Paper. Dia - 47
14.16% Visio - 44
13.25% SmartDraw - 3
0.90% Omnigraffle - 3
0.90% Umbrello - 39
11.75% Power Point or Open Office Impress - 40
12.05% Pen and Paper - 92
27.71% Other - 64
19.28% Total Answers - 332
All in all, the users of other diagramming applications were slightly more satisfied with that application than the Kivio users: On a five-point scale, the majority was between indifferent to satisfied, as compared to the rather equalised distribution of Kivio's satisfaction measure (see above). unsatisfied - 13
7.65% - 27
15.88% - 55
32.35% - 63
37.06% satisfied - 12
7.06% Total Answers - 170
60% of the users of other applications had tried Kivio before. They had stopped using it by various reasons.
On different rating scales, the participants were asked to rate the importance of various features for them.
As important to very important, the majority of participants rated the opportunity to edit points along connector lines. Almost equally important was an option to automatically arrange and align shapes on the page or relative to each other.
More equally distributed among the levels of importance, but with a clear tendency towards a high relevancy, were the following features: Creating your own custom stencils or shapes and group them into sets. Linking shapes to other objects. Using elements other than plain text in the labels for shapes.
A tendency towards a lower importance showed the features to assign gradients to stencils, as well as transparency.
The detailed results can be seen on the Appendix A page.
As expected, the majority of participants was technically oriented. Mostly, they reported to create diagrams for university or their IT-related job. Accordingly, the produced diagrams ranged from flowcharts, electrical circuits, UML, networking or database diagrams.
The most valuable insights with regard to user requirements were extracted from the free-form comments. Here, the participants were asked to list the advantages and disadvantages of Kivio or their other preferred diagramming application. All in all, the comments were very detailed and rich of information.
In a great part of these free-form comments, Kivio was valued for its KDE and KOffice-integration, and its ease of learning and using. However, this ease of learning was criticised by others who experienced a lack of certain features.
In this scope, a number of features was named repeatedly:
1. The quality and choice of shapes was too limited, especially for UML, several Database diagrams and electrical circuits. 2. The connectors showed to be problematic, especially with regard to alignment preciseness, curving, auto-routing and special shapes. 3. Transforming shapes, especially rotating and flipping was missed. 4. More flexible export options, especially SVG and EPS were asked for. Some groups asked for a closer integration with other KOffice tools. 5. The page-oriented work space of Kivio showed to be problematic for several groups of users. Others missed an option to format a page with headers and footers. 6. Windows compatibility was one of the major reasons, why certain user groups did not make use of Kivio any longer.
Another aspect that was named over an over again was the rendering quality of Kivio. Certain user groups were not satisfied with the look of the produced diagrams - regardless of the design of the shapes.
In accordance with the free-form comments, the importance rating of planned features showed that users set a higher value on functionality that affects the basic creation of the tool: Editing points along connector lines and automatically arranging and aligning shapes were rated as the most important features, followed by an option to create your own custom stencils, linking shapes to other objects, or using elements other than plain text in the labels for shapes. Of a lower importance were features to polish a diagram such as assigning gradients to stencils and transparency.
Based on these results, we suggest the following next steps:
1. Feature priority list: Create a feature priority list. (El 17:50, 10 August 2006 (CEST): ready for review) 2. Validate Use Cases: Validate the list of use cases along the survey results and feature priority list. (El 17:50, 10 August 2006 (CEST): ready for review) 3. Sketch Personas: Identify and sketch the three major user groups of Kivio. (El 00:46, 21 September 2006 (CEST): ready for review) 4. Reconsider the overall interaction paradigm: Devise a design to integrate stencil and connector management, to obtain a flexible more flexible work space, etc.