Repository navigation
Fix halfwidth katakana voiced and semi-voiced sound marks in width calculations - #89
Merged
Merged
Conversation
Manishearth
approved these changes
May 26, 2026
Co-authored-by: Jules Bertholet <[email protected]>
Jules-Bertholet
approved these changes
May 28, 2026
Contributor
Author
|
@Manishearth Do you agree with the current change? Is there anything that prevents this PR from merged? |
Member
|
Just busy and haven't been able to get around to it |
Contributor
Author
|
That's too bad. Please do it when you have the chance. |
Manishearth
approved these changes
Jul 2, 2026
Contributor
Author
|
Thank you for the merge. |
Contributor
Author
|
Since the Unicode version has been updated, isn't it about time to release a new version? |
hakula139
added a commit
to hakula139/oxide-code
that referenced
this pull request
Oct 28, 2026
…rks one cell wide (#717) ## Summary Renovate's #711 bumps unicode-width to 0.2.3, and every Rust and Nix job then fails `testing::tests::buffer_rows::read_an_offset_area_and_skip_the_cells_wide_symbols_cover`, which reads `["界ｶﾞa", " "]` where it expects `["界ｶﾞa ", " "]`. The new width table is correct, but ratatui double counts with it. ratatui-core 0.1.2 adds a cell to each halfwidth katakana sound mark (U+FF9E, U+FF9F) in `CellWidth` (ratatui/ratatui#2499) because unicode-width used to count them as zero-width, and unicode-width 0.2.3 counts them as one cell itself (unicode-rs/unicode-width#89). Together they lay `ｶﾞ` out over three cells while terminals draw it in two, so ratatui reserves a cell after each mark that the terminal never draws. This PR holds unicode-width at 0.2.2, where ratatui paints each mark one cell wide. The hold has a cost: `ox`'s own measures, such as `wrap.rs` and the `ox --list` padding, still count each mark as zero cells, as on `main`, one cell less than ratatui and the terminal draw. unicode-width 0.2.3 would fix `ox --list`, which writes to the terminal directly, but not the TUI, since ratatui would then paint one cell more than `wrap.rs` measures. Both gaps close once a fixed ratatui lets the bound go. - **Bound:** `Cargo.toml` requires `unicode-width = ">=0.2, <0.2.3"`, so `cargo update` and lock file maintenance stay on 0.2.2. ratatui-core 0.1.2 is its latest release, ratatui's `main` still adds the cell, and its manifest accepts any unicode-width from 0.2.0, so no ratatui update fixes the combination yet. - **Renovate:** Renovate widens a range with a `<` bound by default, and a local dry run proposed `>=0.2, <0.2.4` in the Rust dependencies group. A `packageRules` entry with `allowedVersions: "<0.2.3"` stops that, so Renovate's rebase of #711 leaves only xattr 1.7.0. - **Upstream:** No ratatui issue or PR reports the double count yet. #718 tracks dropping the bound once a ratatui-core release counts each mark once. ## Test plan - [x] With #711's lock on `main` (unicode-width 0.2.3, xattr 1.7.0), `cargo test -p ox-tui --lib testing::tests` fails with the same `["界ｶﾞa", " "]` as CI - [x] With the bound, `cargo update -p unicode-width` locks nothing, and a full `cargo update` changes only xattr 1.6.1 → 1.7.0, on which `cargo test --workspace` passes - [x] `renovate-config-validator --strict --no-global .github/renovate.json` passes, and `renovate --platform=local --dry-run=lookup --enabled-managers=cargo` proposes no unicode-width update and keeps xattr 1.7.0 in `renovate/rust-dependencies` - [x] `cargo fmt --check`, `cargo build`, `cargo clippy --all-targets`, `cargo clippy`, and `cargo doc --no-deps` pass - [x] `cargo llvm-cov`: 3543 tests pass with 99.17% line coverage - [x] `cargo deny check` passes: advisories, bans, licenses, and sources ok - [x] `pnpm format`, `pnpm lint`, and `pnpm spellcheck` pass - [x] CI passes on every job at 64f1c61, including Rust and Nix on every platform, Lint, MSRV, and the release builds
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related: microsoft/terminal#18087
Halfwidth katakana (semi-)voiced sound marks U+FF9E & U+FF9F are the only Grapheme Extenders that belong to Letter (Lm). They are typical edge cases when you cosider grapheme clusters.
They should be counted as 1 (their EAW is H), not 0.
English transtion of initial Prompt (Copilot + GPT-5.4 high):
Add
test_halfwidth_katakana:ﾊﾟｸﾞ→4 characters
English translation of additional handling prompt after discovering that the added test fails (because GPT started flailing around trying to fix it):
The half-width Kana voiced mark test failed because they're Grapheme Extenders despite being Letters. The current spec sucks, so please fix it together
"pug dog" was hand-written. The rest part of that comment was completed by Copilot line completion.