Skip to content

Feat: Korean input (XIM) and AT-SPI2 accessibility for the Kotlin/Native Linux renderer - #131

Merged
b-re-w merged 10 commits into
developfrom
feat/linux-ime-atspi
Oct 4, 2026
Merged

b-re-w merged 10 commits into
developfrom
feat/linux-ime-atspi

Conversation

@b-re-w

@b-re-w b-re-w commented Oct 4, 2026

Copy link
Copy Markdown
Member

Closes #127

Covers the Linux half of NFR-14 acceptance criteria 5 and 6 (section 6 IME checklist, section 7 accessibility with Orca), NFR-8 and FR-5. Both stay inside renderer/linux, .github and the consumer test fixture.

Input method: XIM, with no D-Bus input context

  • XimContext.kt opens the desktop's input method (XOpenIM, XSetLocaleModifiers("")), asks which input styles it offers and picks preedit callbacks plus no status (falling back to the styles where the input method draws its own window), and creates the context with the four preedit callbacks.
  • The preedit draw callback edits a run of characters (PreeditBuffer, counted in characters as the protocol counts them). Compose is given the whole run each time as SetComposingTextCommand through the text input session the focused field opened; committed text arrives through Xutf8LookupString as CommitTextCommand. Both are WindowEvents recorded into the window's existing event log, so they reach the scene in order with every other event. Nothing goes through Rust, and nothing bypasses startInputMethod.
  • Every event is offered to XFilterEvent first; a key it takes is not handled twice. A plain ASCII key still carries its character in the key event as before; anything else printable is committed as text.
  • The candidate window follows the caret: each turn, after the frame, the focused text rectangle from PlatformTextInputMethodRequest.focusedRectInRoot is sent as XNSpotLocation. The input context has focus only while the window has the keyboard and a field has a session.
  • A click while composing resets the input context (Xutf8ResetIC) and keeps what was typed.
  • The start callback is reinterpreted as returning an int (-1, no limit): the input method reads the return register, and a Unit function leaves garbage in it.

Why XIM and not the ibus or fcitx D-Bus protocols

  • Both daemons ship an XIM frontend (ibus-daemon --xim, which the GNOME session starts; fcitx5's xim addon), and this window is an X11 window on every desktop, XWayland included. One implementation therefore serves ibus-hangul, fcitx5-hangul and anything older.
  • The two D-Bus input context protocols differ from each other, ibus's is tied to its private bus, and neither is a stable interface. Writing them would mean two more implementations of the same preedit logic to keep working per daemon release, for the same Korean input.
  • XIM falls short in one known case: a session where the daemon's XIM server is not running (for example ibus started without --xim, or XMODIFIERS unset). There XOpenIM fails and the window falls back to what it did before (Latin-1 through XLookupString). That is the case a D-Bus route would cover, and it is the first thing to look at if the real-hardware checklist shows it. Not done here for the reasons above.

Accessibility: AT-SPI2

  • AtspiBridge asks the session bus for the accessibility bus (org.a11y.Bus.GetAddress), connects to it with the existing PosixBusConnection (now opening an address as well as the session bus), and registers with org.a11y.atspi.Socket.Embed. No hand-written glue, no new cinterop: the wire format is the existing DBusReader/DBusWriter, AtspiWire.kt adds the server side (parse a call, build a return, an error and a signal).
  • AtspiServer answers Accessible, Component, Action, Text, Application, Properties, Introspectable and Peer from the tree the window last published. AtspiSemantics.kt reads Compose's semantics (after the frame that placed it) into AtspiNodes: role, name, states (enabled, focusable, focused, checked, selected, editable, single/multi line, showing), actions (click, which runs the control's own OnClick) and bounds. GrabFocus runs RequestFocus. Layout with nothing to say is left out and its children are attached to its parent.
  • Signals: ChildrenChanged (add and remove), StateChanged (focused, checked, selected, enabled, indeterminate, active), PropertyChange (accessible-name), Window:Activate and Deactivate.
  • NO_AT_BRIDGE=1 turns it off. No accessibility bus means no bridge and the window carries on.

Tests

  • InputMethodTest (Kotlin/Native, renderer/linux/test): a Hangul syllable composed and committed, jamo-wise backspace, edits counted by character (including outside the BMP), commit forwarded by the input method with no key behind it, shortcuts never commit text, style choice, locale check, candidate spot.
  • AtspiTest: the role and state numbers, the role rule for each Compose role, joining the bus against a fake session bus and registry, and a test playing a screen reader (walk, names, states, actions, extents in each coordinate system, point lookup, text, unknown object and method errors) and each signal.
  • scripts/tests/linux-input-access.test.sh: the wiring (events offered to the input method first, composition goes through the window's log and Compose's text input session, never to the Host, the start callback returns an int, the bridge is pumped on the window's thread).
  • .github/scripts/check-linux-input-access.sh (new step in the linux-static job, with the Linux unit tests): under Xvfb on a private session bus it starts the at-spi2 registry, walks the window with libatspi (what Orca and Accerciser are built on), presses the Save button through the Action interface and checks the Host heard the click; then starts ibus with its XIM server and the hangul engine and types a Korean word with xdotool, checking the field received preedit and then committed text and ended up holding it. The consumer fixture gains --input-check (a field and a button) for this.

Not verifiable by CI (needs hands)

Typing with ibus-hangul and fcitx5-hangul on GNOME and KDE, candidate window placement, and Orca speaking the window. The checklist is in the final report on the issue.

b-re-w added 6 commits October 4, 2026 17:46
…FR-14)

XOpenIM and XCreateIC with preedit callbacks, so ibus and fcitx5 reach the
window through the XIM server each of them runs. The preedit draw callback
edits a run of characters that is handed to Compose as composing text, and
committed text is read with Xutf8LookupString. The caret rectangle Compose
reports places the candidate window through XNSpotLocation.

The pure half (preedit edits, which of a key's two meanings is sent, input
style choice) is unit tested with fakes. Covers the Linux row of the section 6
checklist up to the point a person has to type.
… (NFR-8)

Joins the accessibility bus the session bus names, registers with the
registry (Socket.Embed), and serves Accessible, Component, Action and Text for
Compose's semantics tree: roles, names, states, actions and focus, with
ChildrenChanged, StateChanged, PropertyChange and Window events. The bus
protocol reuses the existing D-Bus reader and writer, and the session bus
connection is opened by PosixBusConnection.

The role and state rule and the object server are unit tested against a fake
registry that a test plays a screen reader against.
…ux window (NFR-14, NFR-8)

The window offers every event to the input method first and handles what it
does not take, records preedit and committed text into the same event log as
everything else (so it reaches Compose's text input session in order, never the
Host), keeps the input method's focus and caret spot up to date each turn, and
ends a composition on a click. The accessibility bridge starts before the loop,
answers the registry's calls once a turn and publishes the tree after each frame
that changed it.

The consumer fixture gains --input-check (a text field and a button), and the
native renderer workflow runs the Linux unit tests and
.github/scripts/check-linux-input-access.sh: ibus with its XIM server and the
hangul engine, an at-spi2 registry, a libatspi client that walks and presses
the window, and Korean typed with xdotool.
The shared key numbering is AppKit's, where 0 is the letter A, so a Linux key
whose keysym had no entry reached the scene as A. The existing test for it had
never run on Linux; it ran for the first time here and failed.
@b-re-w
b-re-w merged commit cf96e5e into develop Oct 4, 2026
10 checks passed
@b-re-w
b-re-w deleted the feat/linux-ime-atspi branch October 4, 2026 10:10
b-re-w added a commit that referenced this pull request Oct 4, 2026
The Linux input-method and accessibility checks from #131 run where the
Linux static renderer is built (static-renderer.yml, linux-x64), since
that is the job that has the patched Compose and renderer/build/linux.
@b-re-w

b-re-w commented Oct 17, 2026

Copy link
Copy Markdown
Member Author

For the record: the Kotlin/Native XIM and AT-SPI code from this PR now lives in the Compose fork's extended/window/native/linux, and its platform-neutral AT-SPI protocol moved to extended/window/common (org.thisisthepy.compose.window.atspi) in thisisthepy/compose-multiplatform-core-extended#127, where the GraalVM X11 window serves it too. compose-rust #251 repins.

b-re-w added a commit that referenced this pull request Oct 17, 2026
…answers the frame time

The merge of thisisthepy/compose-multiplatform-core-extended #131.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant