Accessibility/qt-atspi: Difference between revisions

From KDE Community Wiki
(Try to breathe new life into old information)
 
(10 intermediate revisions by 3 users not shown)
Line 1: Line 1:
{{Note|This page has old content. See page history.}}
= Qt AT-SPI 2 Bridge =
= Qt AT-SPI 2 Bridge =
The '''Qt AT-SPI 2 bridge''' ([[wikipedia:Assistive Technology Service Provider Interface|Assistive Technology Service Provider Interface]]) is a [https://wiki.qt.io/Qt_Accessibility Qt Accessibility] plugin that exposes the internals of a Qt application to accessibility clients such as screen readers.


== Info ==
== Source code ==
This is about the Qt AT-SPI 2 bridge. The bridge is a Qt Accessibility plugin that exposes the internals of a Qt application to interested accessibility clients such as screenreaders.
The source code for the bridge is [https://invent.kde.org/unmaintained/qtatspi archived on KDE Invent] following the closure of the original Gitorious repository<ref name="gitorious">Formerly accessible at <code>git:&sol;/gitorious.org/qt-at-spi.git</code>.</ref> by the Qt developers in 2016.


[http://www.linuxfoundation.org/collaborate/workgroups/accessibility/atk/at-spi/at-spi_on_d-bus AT-SPI on D-Bus at Linuxfoundation]
== Tools ==
Thanks to the efforts of the GNOME Project to implement their AT-SPI 2 implementation (named ATK, the [[wikipedia:Accessibility Toolkit|'''A'''ccessibility '''T'''ool'''K'''it]]), we have some really helpful tools for further development in this area:
* [https://help.gnome.org/users/accerciser/stable/index.html.en Accerciser], an interactive accessibility explorer written in Python.
* [https://orca.gnome.org Orca], an extensible screen reader that converts textual content into speech and/or braille.
* [https://webstats.gnome.org/Accessibility/PythonPoweredAccessibility#Code_example "Python-powered" Accessibility] code example.


The source code for the bridge can be found here:
[[File:Accessibility - a11y.png|thumb|right|x152px|link=Category:Accessibility|illustration of the numeronym ''a11y'', where eleven refers to the number of letters omitted]]
[http://gitorious.org/qt-at-spi Qt AT-SPI on gitorious]
=== Testing/Development ===
Development of these components requires a very recent environment, most notably to include:
* [https://gitlab.gnome.org/GNOME/at-spi2-core at-spi2-core],
* [https://gitlab.gnome.org/Archive/at-spi2-atk at-spi2-atk], and
* [https://gitlab.gnome.org/GNOME/pyatspi2 pyatspi2]
 
Inside a GNOME session, the [[wikipedia:Computer accessibility|accessibility software]] and other [[wikipedia:Assistive technology|assistive technology]] (often referred to by the numeronym '''''a11y''''') use a dedicated D-Bus, so if you want to monitor what is going on you should use a tool like [https://dbus.freedesktop.org/doc/dbus-monitor.1.html dbus-monitor]:
 
<syntaxhighlight lang="shell" highlight="1">
dbus-monitor --address $(xprop -root | grep AT_SPI_BUS | sed -e 's/.*= "//' -e 's/"$//')
</syntaxhighlight>


=== Tools ===
== To do ==
With the Gnome efforts to implement AT-SPI 2 we have some really helpful tools:
''Complex widgets with children? Tabs … clicking on a file in qgit … (crash fix committed, but still needs proper fix)''


Accerciser [http://library.gnome.org/devel/accerciser/stable/index.html.en accerciser manual]
[https://invent.kde.org/unmaintained/qtatspi/-/blob/15ab90659dbbbf81bedfd26a5b9656e08bdbe732/src/atspiadaptor.cpp#L206 <code>GetApplicationBusAddress</code>] seems to be [https://invent.kde.org/unmaintained/qtatspi/-/blob/535816e1176eda0a7e1e79f6b9d9734b66c7b149/xml/Application.xml#L17 undefined in the XML], but is actually called
mgorse&lsqb;''sic, ??''&rsqb;, that ought to be defined somewhere. It is used to try to set up a direct D-Bus connection between libatspi and the application. libatspi will revert to using the session bus if it returns an empty string or an error.


Orca [http://live.gnome.org/Orca/Debugging Orca Debugging]
* [[File:Checkbox blue unchecked.png|17px|link=]] Don't pretend to be [https://gitlab.gnome.org/Archive/gail gail], instead coordinate with the Orca developers
* [[File:Checkbox blue unchecked.png|17px|link=]] Test apps with multiple windows
* [[File:Checkbox blue unchecked.png|17px|link=]] Fix action for menus inside Qt
* [[File:Checkbox blue unchecked.png|17px|link=]] Scroll bars have a value interface, but don't update and cannot be updated, value interface in general…
* [[File:Checkbox blue unchecked.png|17px|link=]] Text attributes (bold, etc.)
* [[File:Checkbox blue unchecked.png|17px|link=]] Tabs


[http://techbase.kde.org/Development/Tutorials/Accessibility Tutorial on techbase]
=== Widgets ===
<div style="columns: 3; padding-left: 10em;">
* Combobox
* List
* Tree
* Table
* Spinner
* Scrollbar
</div>


[https://webstats.gnome.org/Accessibility/PythonPoweredAccessibility#Code_example Python]
===  ===


=== Testing/Development ===
{| class="wikitable" style="margin: 1em auto 1.5em; min-width: 28vw;"
You need a very recent environment, most notably:
|+ '''<big>KDE Widgets</big>'''
pyatspi2, at-spi2-core and at-spi2-atk
! style="min-width: 13em;" | Widget
! Status
|-
| style="font-weight: 500; text-align: center;" | ExampleWidget1
| style="padding-left: 2em;" | Ready.
|-
| style="font-weight: 500; text-align: center;" | ExampleWidget2
| style="padding-left: 2em;" | Mostly done, still needs fooBar() call.
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/frameworks/kwidgetsaddons/html/classKCharSelect.html KCharSelect]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/frameworks/kwidgetsaddons/html/classKColorButton.html KColorButton]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | KColorCombo
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | KComboBox
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/legacy/4.14-api/kdelibs-apidocs/kdeui/html/classKCModule.html KCModule]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | KDateWidget
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | KDatePicker
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://develop.kde.org/docs/administration/kdialog/ KDialog]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/legacy/2.1-api/classref/kdeui/KEditListBox.html KEditListBox]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/legacy/pykde-4.5-api/kdeui/KComboBox.html KFontComboBox]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/frameworks/kwidgetsaddons/html/classKFontChooser.html KFontChooser]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/frameworks/kwidgetsaddons/html/classKFontRequester.html KFontRequester]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/legacy/4.12-api/kdelibs-apidocs/kdeui/html/classKGradientSelector.html KGradientSelector]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/frameworks/kcompletion/html/classKHistoryComboBox.html KHistoryComboBox]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/legacy/4.12-api/kdelibs-apidocs/kdeui/html/classKHueSaturationSelector.html KHueSaturationSelector]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/legacy/pykde-4.7-api/khtml/index.html KHtmlView]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/frameworks/kwidgetsaddons/html/classKLed.html KLed]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/legacy/4.14-api/kdelibs-apidocs/kdeui/html/dir_000011_000013.html KListWidget]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/frameworks/kcompletion/html/classKLineEdit.html KLineEdit]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/legacy/4.14-api/kdelibs-apidocs/kdeui/html/kpushbutton_8cpp_source.html KPushButton]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/legacy/3.5-api/kdelibs-apidocs/kdeui/html/classKRestrictedLine.html KRestrictedLine]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/legacy/pykde-4.7-api/kdeui/KIntSpinBox.html KIntSpinBox]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/frameworks/kwidgetsaddons/html/classKRuler.html KRuler]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/frameworks/kwidgetsaddons/html/classKSqueezedTextLabel.html KSqueezedTextLabel]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/legacy/3.5-api/kdelibs-apidocs/kdeui/html/classKTextBrowser.html KTextBrowser]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/frameworks/ktextwidgets/html/classKTextEdit.html KTextEdit]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/frameworks/kwidgetsaddons/html/classKUrlLabel.html KURLLabel]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/legacy/pykde-4.5-api/kio/KUrlComboRequester.html KURLComboRequester]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/frameworks/kio/html/classKUrlRequester.html KURLRequester]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/legacy/pykde-4.5-api/kdeui/KIntNumInput.html KIntNumInput]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/legacy/3.5-api/kdelibs-apidocs/kdeui/html/classKDoubleNumInput.html KDoubleNumInput] || style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/legacy/4.12-api/kdelibs-apidocs/kdeui/html/classKDateTimeWidget.html KDateTimeWidget]
| style="padding-left: 2em;" | Unknown
|-
| style="font-weight: 500; text-align: center;" | [https://api.kde.org/legacy/3.5-api/kdelibs-apidocs/kdeui/html/classKDateTable.html KDateTable]
| style="padding-left: 2em;" | Unknown
|}


Inside a Gnome session the a11y uses a dedicated d-bus. So if you want to monitor what is going on, you should use:
=== Memory Management ===
<code>
Qt's current [https://doc.qt.io/qt-6/qaccessibleinterface.html QAccessibleInterface] is a pain to deal with.
dbus-monitor --address `xprop -root | grep AT_SPI_BUS | sed -e 's/.*= "//' | sed -e 's/"$//'`
# We can only (safely?) hold on to interfaces created by navigate or queryInterface.
</code>
# Interfaces that we get through notifyAccessibility are deleted by Qt behind our back, so we need to never keep those around (Frederik will try to fix that now).


== Todo ==
== References ==
* Menus are broken
<references />
* Complex widgets with children? Tabs... clicking on a file in qgit (crash fix committed, but still needs proper fix)


GetApplicationBusAddress seems not defined in the xml but is actually called
== External links ==
mgorse: That ought to be defined in the xml. It is used to try to set up a direct dbus connection between libatspi and the application. libatspi will revert to using the session bus if it returns an empty string or an error
* [[techbase:Development/Tutorials/Accessibility|'''KDE Accessibility''' tutorial]] on Techbase
* [https://www.freedesktop.org/wiki/Accessibility/AT-SPI2/ '''AT-SPI2'''] on Freedesktop.org Wiki
* [https://wiki.linuxfoundation.org/accessibility/atk/at-spi/start Linux Foundation's ATK/AT-SPI Special Interest Group]
** [https://wiki.linuxfoundation.org/accessibility/atk/at-spi/at-spi_on_d-bus '''AT-SPI on D-Bus'''] <small>(last updated on {{#formatdate:2011-06-01|mdy}})</small>




* GetNSelections in text is probably buggy in gnome: it gets an extra parameter
[[Category:Accessibility|Qt AT-SPI 2 Bridge]]
* speed is even worse then with gnome
* don't pretend to be gail, coordinate with orca devels
* Application::Id is read/write - why, implement properly
* Test apps with >1 window
* Make special keys for Orca work (flat review)

Latest revision as of 06:42, 3 August 2024

Note

This page has old content. See page history.

Qt AT-SPI 2 Bridge

The Qt AT-SPI 2 bridge (Assistive Technology Service Provider Interface) is a Qt Accessibility plugin that exposes the internals of a Qt application to accessibility clients such as screen readers.

Source code

The source code for the bridge is archived on KDE Invent following the closure of the original Gitorious repository[1] by the Qt developers in 2016.

Tools

Thanks to the efforts of the GNOME Project to implement their AT-SPI 2 implementation (named ATK, the Accessibility ToolKit), we have some really helpful tools for further development in this area:

illustration of the numeronym a11y, where eleven refers to the number of letters omitted

Testing/Development

Development of these components requires a very recent environment, most notably to include:

Inside a GNOME session, the accessibility software and other assistive technology (often referred to by the numeronym a11y) use a dedicated D-Bus, so if you want to monitor what is going on you should use a tool like dbus-monitor:

dbus-monitor --address $(xprop -root | grep AT_SPI_BUS | sed -e 's/.*= "//' -e 's/"$//')

To do

Complex widgets with children? Tabs … clicking on a file in qgit … (crash fix committed, but still needs proper fix)

GetApplicationBusAddress seems to be undefined in the XML, but is actually called mgorse[sic, ??], that ought to be defined somewhere. It is used to try to set up a direct D-Bus connection between libatspi and the application. libatspi will revert to using the session bus if it returns an empty string or an error.

  • Don't pretend to be gail, instead coordinate with the Orca developers
  • Test apps with multiple windows
  • Fix action for menus inside Qt
  • Scroll bars have a value interface, but don't update and cannot be updated, value interface in general…
  • Text attributes (bold, etc.)
  • Tabs

Widgets

  • Combobox
  • List
  • Tree
  • Table
  • Spinner
  • Scrollbar

KDE Widgets
Widget Status
ExampleWidget1 Ready.
ExampleWidget2 Mostly done, still needs fooBar() call.
KCharSelect Unknown
KColorButton Unknown
KColorCombo Unknown
KComboBox Unknown
KCModule Unknown
KDateWidget Unknown
KDatePicker Unknown
KDialog Unknown
KEditListBox Unknown
KFontComboBox Unknown
KFontChooser Unknown
KFontRequester Unknown
KGradientSelector Unknown
KHistoryComboBox Unknown
KHueSaturationSelector Unknown
KHtmlView Unknown
KLed Unknown
KListWidget Unknown
KLineEdit Unknown
KPushButton Unknown
KRestrictedLine Unknown
KIntSpinBox Unknown
KRuler Unknown
KSqueezedTextLabel Unknown
KTextBrowser Unknown
KTextEdit Unknown
KURLLabel Unknown
KURLComboRequester Unknown
KURLRequester Unknown
KIntNumInput Unknown
KDoubleNumInput Unknown
KDateTimeWidget Unknown
KDateTable Unknown

Memory Management

Qt's current QAccessibleInterface is a pain to deal with.

  1. We can only (safely?) hold on to interfaces created by navigate or queryInterface.
  2. Interfaces that we get through notifyAccessibility are deleted by Qt behind our back, so we need to never keep those around (Frederik will try to fix that now).

References

  1. Formerly accessible at git://gitorious.org/qt-at-spi.git.

External links