Repository navigation
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
AI review status for this pull request.
|
lexical
@lexical/a11y
@lexical/clipboard
@lexical/code
@lexical/code-core
@lexical/code-prism
@lexical/code-shiki
@lexical/compiler
@lexical/devtools-core
@lexical/dragon
@lexical/eslint-plugin
@lexical/extension
@lexical/fast-check
@lexical/file
@lexical/hashtag
@lexical/headless
@lexical/history
@lexical/html
@lexical/internal
@lexical/link
@lexical/list
@lexical/mark
@lexical/markdown
@lexical/mdast
@lexical/offset
@lexical/overflow
@lexical/plain-text
@lexical/react
@lexical/rich-text
@lexical/selection
@lexical/table
@lexical/tailwind
@lexical/text
@lexical/utils
@lexical/y
@lexical/yjs
commit: |
5d5982c to
5a57e05
Compare
5a57e05 to
50a9dd4
Compare
There was a problem hiding this comment.
This looks good. The re-land covers what #9702 asked for: entities now keep their NodeState when they become text (the root cause, #9703), and registerLexicalTextEntityJoin.test.ts covers the #9704 join. The new tests fail on main and pass here, and the playground no longer drops the keystroke in the #9637 case. The loop breakers can keep an entity that a fresh parse would not make, which the description presents as the price of never throwing; that seems like the right call. The suggestions below are optional and non-blocking.
Optional suggestions
- Keep one copy of the NodeState helpers.
$carryNodeState(packages/lexical-text/src/registerLexicalTextEntity.ts:47) duplicates the new core$carryNodeState(packages/lexical/src/LexicalNodeState.ts:1319), andhaveEquivalentNodeState(registerLexicalTextEntity.ts:123) re-implementsnodeStatesAreEquivalent(LexicalNodeState.ts:1155). If the core merge rule changes,$wouldMergewill silently disagree with how the nodes then merge. Exporting the core helpers asINTERNAL_functions, asINTERNAL_$recordTextMovealready is, avoids the drift. - Changeset wording. The
Verify:line in.changeset/lexical-text-entity-settle.md:11mentions mentions, but the repository's mentions come from the typeahead and never go throughregisterLexicalTextEntity. "hashtags and keywords" would point testers at the right surfaces (format per.changeset/README.md).
The first landing was reverted after a downstream regression, so maintainer validation is needed before merging.
| * whether two text nodes merge: known values compared with their | ||
| * `StateConfig.isEqual`, values not read yet compared as they are | ||
| */ | ||
| function haveEquivalentNodeState(a: TextNode, b: TextNode): boolean { |
There was a problem hiding this comment.
Optional: this re-implements the core merge rule nodeStatesAreEquivalent (packages/lexical/src/LexicalNodeState.ts:1155), and $carryNodeState above duplicates the core one at LexicalNodeState.ts:1319. Exporting both as INTERNAL_ helpers, as INTERNAL_$recordTextMove is, would keep $wouldMerge from drifting away from how text nodes actually merge.
|
|
||
| A hashtag or other entity right after text in another format is now turned back into text when typing makes it touch that text (`b#tag`), as a fresh parse would. | ||
|
|
||
| Verify: text entities (hashtags, mentions, keywords) right after text in another format or style, before and after typing next to them. |
There was a problem hiding this comment.
Nit: the repository's mentions use the typeahead and do not go through registerLexicalTextEntity; "hashtags and keywords" is the accurate list.
50a9dd4 to
d9ef20e
Compare
There was a problem hiding this comment.
This looks good. The re-land covers what #9702 asked for: entities now keep their NodeState when they become text (the root cause, #9703), and registerLexicalTextEntityJoin.test.ts covers the #9704 join. The new tests fail on main and pass here, and in the playground the space typed between xcongratscongratsy's bold neighbours is no longer dropped. Keeping a token, segmented or touching entity after the third round, rather than throwing, is stated clearly in the description and seems like the right trade. Pinning the @lexical/y undo issue as a test.fails linked to #9716 is a reasonable way to track it; the same test fails the same way on main.
The suggestions below are optional and non-blocking.
Optional suggestions
- Keep one copy of the NodeState helpers.
$carryNodeState(packages/lexical-text/src/registerLexicalTextEntity.ts:47) duplicates the new core$carryNodeState(packages/lexical/src/LexicalNodeState.ts:1320), andhaveEquivalentNodeState(registerLexicalTextEntity.ts:123) re-implementsnodeStatesAreEquivalent(LexicalNodeState.ts:1155), the rule that decides whether text nodes merge. If that rule changes,$wouldMergewill silently disagree with how the nodes then merge. Exporting the core helpers asINTERNAL_functions, asINTERNAL_$recordTextMovealready is, would avoid the drift. - A two-peer check of a demotion that carries NodeState. Apart from the #9716 pin, no collaboration test asserts both peers' documents after a hashtag with NodeState becomes text. A small
@lexical/yjsor@lexical/ycase would guard the new behaviour in both bindings.
Since the first landing was reverted, maintainer validation is needed before merging.
| * holds keeps its value (when the text is made an entity: state that | ||
| * `createNode` gave it). | ||
| */ | ||
| function $carryNodeState(from: TextNode, to: TextNode): void { |
There was a problem hiding this comment.
Optional, non-blocking: this duplicates the new core $carryNodeState (packages/lexical/src/LexicalNodeState.ts:1320), and haveEquivalentNodeState below re-implements nodeStatesAreEquivalent, the rule that decides whether text nodes merge. Exporting the core helpers as INTERNAL_ functions (as INTERNAL_$recordTextMove is) would keep $wouldMerge from drifting if that rule changes.
d9ef20e to
bd29bae
Compare
There was a problem hiding this comment.
Thanks for the re-land and the thorough tests. The new Settle, Join and NodeState cases fail on main and pass on this branch. In the playground, the #9637 keyword case now keeps the typed space instead of throwing. One interaction needs a fix before this merges; the other points are optional.
Needs a fix: a hashtag pasted at the edge of a MarkStateExtension comment now joins the comment
packages/lexical-text/src/registerLexicalTextEntity.ts, lines 642-645: the promotion carry from the first commit (#9712).
To reproduce, build an editor with RichTextExtension, HashtagExtension, MarkExtension and MarkStateExtension. Mark see in see more as c1, put the caret right after it, and paste #tag. One CONTROLLED_TEXT_INSERTION_COMMAND or insertText from code does the same, and so does #tag pasted right before see:
main: "see"[c1] " " hashtag "#tag" " more"
this branch: "see"[c1] " " hashtag "#tag"[c1] " more"
What happens:
- The pasted characters land in the marked text node.
- The hashtag transform splits
#tagoff with that node's ids, and the carry hands them to the new hashtag. MarkStateExtensionthen treats those ids as the hashtag's own and does not split it.
The "Inserted text" rule in packages/lexical-website/docs/collaboration/comments.md says text inserted at a mark's edge does not join it, even when a transform then splits it off into a new TextNode. main and MarkNode both follow that rule. Typing one key at a time after the mark is unaffected.
There are two ways to fix it:
- skip the promotion carry for NodeState the characters only got by landing in an existing node during this update; or
- extend
MarkStateExtension's$isSplitOffEntityrule to an entity whose ids are the same array as the node it was split from.
A test next to text inserted after a marked hashtag is not marked, like MarkNode in packages/lexical-mark/src/__tests__/unit/MarkStateExtension.test.ts would pin it.
Optional suggestions
- Changeset (
.changeset/lexical-text-entity-node-state.md): editing inside a segmented node now also keeps the node's detail (unmergeable,directionless) on the plain text that replaces it. That changes the text's saved JSON and whether it merges with its neighbours. AGENTS.md asks a fix that changes observable output to say what to check underVerify:, so a line there would help. - One carry helper:
$carryNodeStateexists both inpackages/lexical/src/LexicalNodeState.tsand as a private copy inregisterLexicalTextEntity.ts, andhaveEquivalentNodeStatemirrors core'snodeStatesAreEquivalent. Exporting oneINTERNAL_helper fromlexicalwould keep the two from drifting. - #9716: as the description notes, entities that keep their comments reach the pinned
@lexical/yundo bug more often. It may be worth scheduling that fix close to this one.
| if (splitOff === null) { | ||
| // The NodeState of the text becomes the entity's | ||
| $carryNodeState(nodeToReplace, replacementNode); | ||
| } |
There was a problem hiding this comment.
Needs a fix. This carry gives a hashtag inserted at the edge of a MarkStateExtension comment that comment's ids. With RichTextExtension, HashtagExtension, MarkExtension and MarkStateExtension, mark see in see more as c1, put the caret after it and paste #tag:
main: "see"[c1] " " hashtag "#tag" " more"
this branch: "see"[c1] " " hashtag "#tag"[c1] " more"
The pasted characters land in the marked node, and the transform splits #tag off with a copy of that node's ids. This line then hands those ids to the hashtag, which MarkStateExtension takes for its own and does not split. The "Inserted text" rule in packages/lexical-website/docs/collaboration/comments.md says edge-inserted text does not join a mark, even when a transform splits it off into a new TextNode. Either skip the carry for NodeState the characters only got by landing in an existing node in this update, or extend $isSplitOffEntity in MarkStateExtension to cover this case. Then add a test next to text inserted after a marked hashtag is not marked, like MarkNode.
|
|
||
| [fix] `registerLexicalTextEntity` keeps the NodeState of a text entity (a hashtag, a keyword) that stops matching and becomes text, such as comment ids stored as NodeState, instead of dropping it | ||
|
|
||
| The NodeState also comes back when that text is made an entity again in the same update, and plain text keeps its NodeState when it becomes an entity (state that `createNode` gives the entity wins). A token or segmented entity that text follows keeps its NodeState as well: it is turned back into text with it, and after twice in one update (or in updates batched into one commit) it is kept (taking the text its match extends into only when that text would merge with it), instead of looping forever. Editing inside a segmented node (typing, or removing a range that starts or ends in it) keeps its NodeState too. |
There was a problem hiding this comment.
Optional: editing inside a segmented node now also keeps the node's detail (unmergeable, directionless) on the plain text that replaces it. That changes the text's saved JSON and whether it merges with its neighbours. AGENTS.md asks a fix that changes observable output to say what to check under Verify:, so a line here would help.
bd29bae to
c1bac01
Compare
…hen they become text ## Description `registerLexicalTextEntity` turns an entity back into plain text when it stops matching (typing right before a hashtag, deleting its `#`, joining blocks), with `$replaceWithSimpleText`. That built the new `TextNode` from the entity's text, format, style and detail, but not its NodeState, so anything an app keeps on the characters as NodeState (comment ids, suggestion marks, block ids) was dropped (#9703). The same happened in the other direction: plain text with NodeState that becomes an entity gave the entity none. What carries, and why: - **Demotion (entity to text): all of the entity's NodeState**, nested and flat, known and not yet parsed (read from JSON), the same value objects (ids compared by identity stay identical). NodeState is the ad-hoc data an app keeps on text; what is inherent to the entity type is its class (its fields) and its mode, which stay with the type, as before. This is what `splitText` already does for the pieces it cuts, and what the code highlighters do since #9503 when a code token becomes text. A key that the plain text type declares with a different `StateConfig` is parsed with the text type's config. Format, style and detail carry as before. - **Promotion (text to entity): the text's NodeState too**, for every key `createNode` did not set (its state wins), with two exceptions, both characters that were just inserted into a node and so only have a copy of its NodeState: text that the entity's own transform splits off its end in the same update (text typed or pasted right after a hashtag), and an entity made only of text inserted at the start or the end of a node that existed in this update, when no entity was turned into text in that block in the update (pasting ` #tag` right after commented text: the hashtag does not join the comment, as on `main` and as `MarkStateExtension`'s rule for text inserted at a comment's edge says). `splitText` gives that piece a copy of the entity's NodeState, but those characters were just inserted, so it is not theirs; an entity made from it keeps the old behaviour (MarkStateExtension relies on telling inserted text from marked text this way). An entity turned into text and back in one update gets its own NodeState back. - **Token and segmented entities** that text follows are turned into text so that they merge with it and are parsed again. With their NodeState kept they do not merge with text that has other NodeState, and the transforms would make the entity again and turn it back forever (on `main` this already looped whenever that text had NodeState, or another format). So, as in #9686, when the same entity (same parent, text, and place: the first node before it that is no text entity, and the size of the entities between) comes back to be turned into text a third time in one update (or updates batched into one commit), it is kept, with its NodeState. It takes the text its match extends into, as an entity in normal mode does, only when that text would merge with it (the same format, style and detail, and NodeState equal by the rule text nodes merge by: `StateConfig.isEqual`, so two distinct value objects are two values; the selection stays where it was in that text); text with other NodeState or format stays apart, so it neither takes the entity's NodeState nor makes the entity drop its own. Until the third time the transforms run as before. The count belongs to the pending editor state, so an update that fails does not count toward the next. - The text split off the end of an entity is tracked per editor, for every registration, so an entity of another registration made from it (a keyword typed after a hashtag) does not take the copy either. - **Core: editing inside a segmented node** replaces it with a plain `TextNode` built from its text, format and style, the same way `$replaceWithSimpleText` did, and dropped its NodeState the same way (and its detail, such as `unmergeable`, which `$replaceWithSimpleText` kept): typing inside it (outside composition, `LexicalSelection`) and removing a range that starts or ends inside it (`LexicalCaretUtils`). Both now keep them (detail, and NodeState with `$carryNodeState`, internal to `lexical`, with the same rules as above; keys the new node already holds, such as state a registered `TextNode` replacement sets, keep their value). Segmented entities that this change keeps or makes reach these paths more often, so they are fixed here. Two text nodes merge only when their NodeState is equal, so text that keeps an entity's NodeState stays its own node next to text with other NodeState, as text in another format does. That is the point (the characters keep their comment), and it is the only structural difference: text with no NodeState merges as before. Callers in this repo, all through `registerLexicalTextEntity`: `@lexical/hashtag` (`registerLexicalHashtag`, `HashtagExtension`, `LexicalHashtagPlugin`), `@lexical/react` `useLexicalTextEntity`, and the playground's `KeywordsExtension`. Mentions (typeahead) and `@lexical/link` autolink have their own transforms and do not share this path. Fixes #9703 ## Test plan `.changeset/lexical-text-entity-node-state.md` (`lexical` and `@lexical/text`, patch, `[fix]`). `MarkStateExtension.test.ts`: a hashtag pasted or inserted at the edge of a comment (after it, and before it) is not added to it, with `MarkStateExtension` and with `MarkNode` (as on `main`). `LexicalSelection.test.ts` (3 tests, each fails on `main`): typing inside a segmented node keeps its NodeState and detail; a registered `TextNode` replacement keeps the NodeState it sets and takes the rest; removing a range from inside a segmented node to the next block keeps the NodeState and detail of what is left. `registerLexicalTextEntityNodeState.test.ts` (new, 24 tests): #9703's repro; deleting a hashtag's `#`; merging into neighbours with the same NodeState and, without NodeState, merging as before; promotion with NodeState, `createNode`'s state winning, text inserted after a hashtag not taking the copy, and a hashtag made only of inserted text not taking it; a reference-typed value carried as the same object; NodeState read from JSON and not parsed yet; NodeState declared in the entity's `$config` (nested and flat) and exported nested on the text; a key the text type declares with another config; a hashtag inside a MarkNode; undo and redo; and every path that turns an entity into text: the text transform with no match, a token entity that text follows (same NodeState: it takes the text; other NodeState: kept, the text stays apart; the selection after pasted text; text with its own NodeState, which loops on `main`; a value object distinct from the text's, equal as JSON), the next entity that stops matching, adjacent entities, an entity followed by another, and a join across text in another format. `registerLexicalTextEntityNodeStateProperty.test.ts` (new): random layouts of entities (hashtags, and token mentions) and text in several formats, with NodeState on both and MarkNodes around some, and random edits (typing, backspace and delete, joining blocks, removing ranges across nodes). Each entity has a letter no other text has; after every edit that turns an entity into text, every character of it that is still there has the entity's NodeState and MarkNode (or sits inside another entity that grew over it, which takes characters with its own NodeState, as before). 200 seeds per matcher (`TEXT_ENTITY_NODE_STATE_SEEDS`). Before (main): ``` $ pnpm exec vitest run --project unit packages/lexical-text/src/__tests__/unit/registerLexicalTextEntityNodeState.test.ts packages/lexical-text/src/__tests__/unit/registerLexicalTextEntityNodeStateProperty.test.ts × #9703: a hashtag that stops matching keeps its NodeState when it becomes text (typing before it) × deleting the # of a hashtag keeps its NodeState (reverse transform, no match) × a hashtag that becomes text merges into neighbors with the same NodeState × plain text with NodeState that becomes a hashtag keeps it × state that createNode gives the entity wins over the text it replaces × a reference-typed state value is carried as the same object × NodeState read from JSON (not parsed yet) is carried and exported × NodeState declared in the entity $config (nested and flat) is carried to the text × a key the plain text type declares with another config is parsed with that config × a hashtag inside a MarkNode stays inside it when it becomes text × undo and redo of a demotion restore the entity and the text with their NodeState × the text transform: the entity before it stops matching with its text (prevMatch null) × with the same NodeState as the text: it keeps it and takes the text × with other NodeState than the text: kept with its NodeState, the text stays apart × typed text after it: the selection stays after the typed text × with other NodeState than text that its match extends into, which keeps it apart: kept, each keeps its own × with a value object distinct from the text after it, equal as JSON: each keeps its own × the text transform: the entity after it stops matching (nextMatch null) × the reverse transform: adjacent entities both become text × the reverse transform: an entity followed by an entity × hashtag: an edit that turns an entity into text keeps its NodeState and marks on its characters × mention: an edit that turns an entity into text keeps its NodeState and marks on its characters Tests 22 failed | 3 passed (25) ``` After: ``` $ pnpm exec vitest run --project unit packages/lexical-text packages/lexical-hashtag packages/lexical-mark packages/lexical-history Tests 1378 passed (1378) $ pnpm exec vitest run --project unit packages/lexical/src/__tests__/unit/LexicalSelection.test.ts Tests 103 passed (103) $ TEXT_ENTITY_NODE_STATE_SEEDS=20000 pnpm exec vitest run --project unit packages/lexical-text/src/__tests__/unit/registerLexicalTextEntityNodeStateProperty.test.ts ✓ hashtag: an edit that turns an entity into text keeps its NodeState and marks on its characters ✓ mention: an edit that turns an entity into text keeps its NodeState and marks on its characters ``` The existing `@lexical/text` tests pass unchanged, including `registerLexicalTextEntityEquivalence.test.ts` (JSON, selection, mutations and keys after random sessions, against the helper before #9483, which has no NodeState in its layouts), and `MarkStateExtension`'s hashtag tests (text inserted after a marked hashtag is not marked). A differential against `main` over random editing sessions (typing, backspace and delete, joins across paragraphs, headings, quotes, list items and table cells, ranges, formatting, paste with NodeState, undo and redo, IME composition) with hashtags, a keyword matcher, a token and a segmented matcher, autolink and MarkNodes: the NodeState of an entity's characters is kept in tens of thousands of steps where `main` drops it; no text is lost; token and segmented entities that loop on `main` settle. Where NodeState now keeps text apart, the entities can differ from `main`, as they do next to text in another format; a keyword-style matcher can then reach #9637's loop in a layout `main` merged (a few per 10,000 sessions; hashtags never do), which #9637's fix settles.
…ttle between text nodes of other formats (re-land of #9686) This re-lands #9686, reverted by #9702 because it turns more entities back into text (every edit that leaves a hashtag after text in another format), and turning an entity into text dropped its NodeState (#9703): joining a block that starts with a commented hashtag onto text in another format lost the comment (#9704). #9712 fixes #9703 (entities keep their NodeState when they become text) and brings the loop breaker for token and segmented entities; this PR is #9686 on top of it, unchanged otherwise, plus #9704's case and the long-session join as tests. Review the second commit; the first is #9712. `registerLexicalTextEntity` decides whether a match at the start of a text node is an entity in two places that disagreed: - the node's own transform made the first match in its own text an entity, without the text node before it; - the transform of a simple text node before it checks the entity that follows it with its own text in front, and turns it back into text when that is no match. Take plain text between two text nodes of another format (bold `x`, `dogcat`, bold `y`) and type a space after `dog`. The first transform made `dog` an entity, the second undid it (`xdog` has no match), the nodes merged again, and so on until Lexical threw "One or more transforms are endlessly triggering additional transforms" and the edit was lost (#9637). Now a match at the start of a node is left as text when the simple text node before it would turn it straight back into text, that is when the match's text after that node's text is no match (`getMatch(previousText + matchText) === null`, the check that node's transform makes). It is still made an entity when the first match of both nodes' text is this match, which a match that depends on the text after it can be (the entity `createNode` makes may then match on its own). Any match right after one left as text stays text too, as it does after a match next to another entity. The match is still found in the node's own text, so the entity's own transform agrees as before, and the extra check is one `getMatch` call (the first match is already computed), made only for a match at the start of a node after a simple text node. Where that transform returned early, before its loop (its text has no match after the text node before it, or the first starts in that node), it never checked the entity after it, as its loop does. So typing a letter right before a hashtag that follows text in another format left the hashtag an entity (`b#tag`), which a fresh parse of the same text does not make. In that early return the transform now turns such an entity back into text unless the transforms would keep it there (below). A match left as text at the start of the next node stays text until that node is edited, as before. Whether the transforms keep an entity right after a simple text node is one rule, used by both transforms and by the loop's check of the next node: the first match from that node's text on, passing the matches of its text (at most 8, as the loop passes over them; past that, nothing is changed), must start where the entity does, with the text after the entity in the node it was split from (the simple text node after it in the same format, style and detail), as the check that made it an entity found it; and that match must still hold with the text of any other text node after it, as the check that keeps a match at the end of a node from becoming an entity found it. So a match that needs the text after it (`/#[a-z]+(?= )/`) is kept where it was made, and text after it in another format, or a token node, does not keep an entity that a fresh parse would not make. An entity's own transform, when the transforms no longer keep it after the simple text node before it (the text after it changed, say), marks that node dirty, once per node for each pending editor state (other entities' transforms may leave it dirty again), so it is turned back into text in the same update rather than at the next edit of the text before it. A token or segmented entity with text after it in its node looped on `main`: the transform of that text turns it back into text, the text merges, and the transform makes it again. #9712 keeps such an entity the third time it comes back to be turned into text in one update, and this PR keeps that rule. Two more changes come with the re-land, for orders of transforms that #9686 alone reached before the other checks: - A normal-mode entity grows over the text after it (takes the characters its match extends into) only when it is an entity the transforms keep: not right after another text entity (the two are turned back into text by the rule for adjacent entities), and not after a simple text node that it is no entity after (`isEntityAfter`; when that cannot tell, after too many matches in the text before, it stays as it is without growing, and the text after it is parsed as usual). Otherwise it is turned back into text first, rather than grown and then turned into text with the next node's characters, which then lost their format and NodeState (two hashtags next to each other before italic text, a `createNode` that changes the case). - Two entities next to each other are turned back into text, as before, but when the same entity comes back to that a third time in one update it is kept (the same counter as the token breaker, counted apart): entities of two registrations that touch (a segmented `$B` next to a token `@x`) looped forever on `main`; now they settle, kept as they are. Each of these checks is a few `getMatch` calls, bounded by the 8 matches passed over, only on the paths above. A matcher whose match does not hold on the entity's own text (`/dog(?=cat)/`, with `createNode` keeping the text) still loops, on plain text too, as on `main`: the entity's own transform turns it back into text every time. `registerLexicalTextEntity` needs an entity's text to be a match on its own. Two registrations whose entities end up touching can still loop through the rule that turns two adjacent entities into text (with a hashtag and a segmented upper-case matcher, `#xabDOG` loops on `main` from a fresh parse); with one registration, or matches that cannot touch, this does not arise. The source text of an entity is told by format, style and detail; NodeState is not compared (text keeps its NodeState when it becomes an entity, but `createNode` may give the entity its own). No API is added or changed. Entities can differ from before only next to the boundary between a simple text node and the node after it, where one of the checks above was skipped or disagreed with the other. Closes #9637 Closes #9704 `.changeset/lexical-text-entity-settle.md` (`@lexical/text`, patch, `[fix]`). `registerLexicalTextEntitySettle.test.ts` (new, 41 tests): the issue's two cases and its controls; a later match in the node; a lookbehind-style matcher; a hashtag-like matcher after a formatted letter and a formatted space; two entities next to each other; `dogcat` between every pair of four formats and four neighbour texts, with four typed characters; IME composition; pasting formatted runs; bolding the text before an entity; typing a letter before an entity after text in another format; a matcher whose earlier match must end where the text ends; matchers that need the text after a match (with a match at the start of a node, after typing, and after removing the text after an entity), one with a `createNode` that upper-cases; text after an entity in another format; removing the text between an entity and a token after it; a token or segmented entity with text after it in its node (loops on `main`), and one after a match left as text, with the nodes each settles on; an update that fails not counting toward the next; two registrations, one making token or segmented entities, in both orders, with the nodes they settle on; text after an entity with another detail; an entity after a node with an earlier match; an entity checked again in a later update; one dirty node per keystroke in text before a formatted hashtag, with and without an earlier match in that text (so history merges typing, as on `main`); a match at the start of the next node that typing makes valid, made an entity when that node next changes; `createNode` factories that lower-case, upper-case or add text; and a bounded number of `getMatch` calls per keystroke in a long node before an entity, however long. Then, for three matchers, 2,000 random layouts each (override with `TEXT_ENTITY_SETTLE_SEEDS`): - a fresh parse settles, and transforming every node again leaves it as it is; - typing or deleting anywhere in text between two nodes of other formats (#9637's shape) settles on the entities a fresh parse of the result finds; - sessions of 4 edits (typing or deleting anywhere, bolding a node) settle, and so does transforming every node again after each (a guard against new loops: these pass on `main` too). `Hashtags.spec.mjs`: the playground's keywords, bold `x`, `congratscongrats`, bold `y`, and a space after the first `congrats` (the loop on `main`: it needs a later match that the next node turns back into text, which a hashtag cannot be, since text after a hashtag only extends it); and hashtags typed after a bold letter and between bold text, with the exact text. ``` $ pnpm exec vitest run --project unit packages/lexical-text/src/__tests__/unit/registerLexicalTextEntitySettle.test.ts × bold "x" | "dogcat" | bold "y", a space after "dog": settles, no entity × bold "x" | "dogcat" | bold "dog", an emoji after "dog": settles, no entity × entity at the edges of a node, every format combination around it × IME composition between two formatted nodes settles, and so does its end × typing a letter before an entity after text in another format turns it back into text × a match at the start of a node that needs the text after it settles after a formatted node × an entity after a node with an earlier match is checked too × removing the text between an entity and a token after it turns the entity back into text × an entity is checked again in each update × typing that makes a match at the start of the next node one leaves it text until that node changes × a token or segmented entity with text after it in its node settles × two registrations, one making token or segmented entities, settle × keyword: 2000 random layouts parse, and parse again the same × keyword: 2000 random edits between two formatted nodes settle on a fresh parse × tag: 2000 random edits between two formatted nodes settle on a fresh parse × lookbehind: 2000 random layouts parse, and parse again the same × lookbehind: 2000 random edits between two formatted nodes settle on a fresh parse + "seed 231 [\"_😀gdo\",\"*cat dog\",\"docatdog \",\"_gdo!g\"]: Error: One or more transforms are endlessly triggering additional transforms. Ma", + "seed 956 [\"_😀\",\"cat \",\"*_y\"] [[\"text\",\"😀\",2],[\"test-keyword\",\"cat\",0],[\"text\",\" \",0],[\"text\",\"y\",3]] type \"d\" at 1:0: [[\"text\",\"😀\",2],[\"text\",\"d\",0],[\"test-keyword\",\"cat\",0],[\"text\",\" \",0],[\"text\",\"y\",3]] != [[\ Tests 17 failed | 24 passed (41) $ TEXT_ENTITY_SEED=30449 pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/TextEntityEdits.test.ts # and 41459 Error: One or more transforms are endlessly triggering additional transforms. May have encountered infinite recursion caused by transforms that have their preconditions too lose and/or conflict with each other. ``` ``` $ pnpm exec vitest run --project unit packages/lexical-text/src/__tests__/unit/registerLexicalTextEntitySettle.test.ts Tests 41 passed (41) $ TEXT_ENTITY_SETTLE_SEEDS=60000 pnpm exec vitest run --project unit --testTimeout=0 packages/lexical-text/src/__tests__/unit/registerLexicalTextEntitySettle.test.ts Tests 41 passed (41) $ TEXT_ENTITY_SEED=30449 pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/TextEntityEdits.test.ts # and 41459, the seeds that found #9637 Tests 1 passed (1) $ pnpm run ci-check Found 0 errors ``` `TextEntityEdits.test.ts` also passes with `TEXT_ENTITY_SEEDS=25000`. The existing `@lexical/text` tests pass unchanged, including `registerLexicalTextEntityEquivalence.test.ts`, which compares the JSON, selection, mutations and keys after 300 random editing sessions with a copy of the helper before #9483. `packages/lexical-y/src/__tests__/unit/EntityUndoSpread.test.ts` (new, `test.fails`, #9716): two peers; a comment removed by one peer and both peers' undos leave the comment on every block after it. A `@lexical/y` bug (undo restores a mark opener without its closer), on `main` too with this test; this change reaches it more often because entities keep their comments. Pinned so it flips when #9716 is fixed. `registerLexicalTextEntityJoin.test.ts` (new, 11 tests): #9704's repro (a list item starting with a commented hashtag joined onto italic text), the same join by backspace and by forward delete, a block before that ends with a tab (all with NodeState), a hashtag with `MarkStateExtension` comment ids joined onto italic text, two hashtags turned into text before italic text with NodeState (the text keeps its format, NodeState and the selection), typing before a hashtag that a segmented and a token entity follow (settles, with a bounded number of `getMatch` calls), and a hashtag after text with 7 and 8 earlier matches (the most `isEntityAfter` checks) that does not grow over the commented italic text after it, while a later hashtag in that text is still made. With #9686 alone the first three lose the NodeState and the `MarkStateExtension` one loses its comment ids; with #9712 alone the hashtag stays a hashtag, as on `main`; here it becomes text and keeps them. ``` $ pnpm exec vitest run --project unit packages/lexical-text/src/__tests__/unit/registerLexicalTextEntityJoin.test.ts Tests 11 passed (11) ```
c1bac01 to
6621b27
Compare
There was a problem hiding this comment.
This looks good, thanks: the two transforms in registerLexicalTextEntity now agree about a match at the start of a node, so the #9637 layouts settle instead of throwing. Entities also keep their NodeState when they turn into text and back, which fixes the comment loss behind the revert in #9702. The new tests fail on main and pass here, and the playground keyword sequence keeps the typed space. Both suggestions below are optional and non-blocking.
- Changeset: say that state the entity type declares now reaches plain text (
.changeset/lexical-text-entity-node-state.md).$replaceWithSimpleTextcarries all of an entity's NodeState onto the plainTextNode. That includes keys the entity class declares in its own$configstateConfigs(flat ones too, which then serialize nested under$), asregisterLexicalTextEntityNodeState.test.tsasserts. An app that recognises a mention or tag by such a key will find it on plain text after demotion. CONTRIBUTING.md asks for the entry to be written for someone who uses Lexical, so one sentence saying this would tell those apps what to check. - One carry helper instead of two (
packages/lexical-text/src/registerLexicalTextEntity.ts:47,packages/lexical/src/LexicalNodeState.ts:1320). The two$carryNodeStatefunctions apply the same rule today. Exporting the core one as anINTERNAL_helper and using it from@lexical/textwould stop them drifting apart.
The @lexical/y undo issue pinned by EntityUndoSpread.test.ts (#9716) is already tracked, and the test.fails pin is a good way to keep it visible.
| '@lexical/text': patch | ||
| --- | ||
|
|
||
| [fix] `registerLexicalTextEntity` keeps the NodeState of a text entity (a hashtag, a keyword) that stops matching and becomes text, such as comment ids stored as NodeState, instead of dropping it |
There was a problem hiding this comment.
Optional: $replaceWithSimpleText carries all of an entity's NodeState onto the plain text, including keys the entity class declares in its own $config stateConfigs (flat ones then serialize nested under $, as registerLexicalTextEntityNodeState.test.ts asserts). An app that recognises a mention or tag by such a key will now find it on plain text after demotion. A sentence here would tell those apps what to check; CONTRIBUTING.md asks for the entry to be written for someone who uses Lexical.
| * holds keeps its value (when the text is made an entity: state that | ||
| * `createNode` gave it). | ||
| */ | ||
| function $carryNodeState(from: TextNode, to: TextNode): void { |
There was a problem hiding this comment.
Nit (optional): this is a second copy of $carryNodeState from packages/lexical/src/LexicalNodeState.ts:1320. The two apply the same rule today. Exporting the core one as an INTERNAL_ helper and using it here would keep them from drifting apart.
This re-lands #9686, reverted by #9702 because it turns more entities back into text (every edit that leaves a hashtag after text in another format), and turning an entity into text dropped its NodeState (#9703): joining a block that starts with a commented hashtag onto text in another format lost the comment (#9704). #9712 fixes #9703 (entities keep their NodeState when they become text) and brings the loop breaker for token and segmented entities; this PR is #9686 on top of it, unchanged otherwise, plus #9704's case and the long-session join as tests. Review the second commit; the first is #9712.
registerLexicalTextEntitydecides whether a match at the start of a text node is an entity in two places that disagreed:Take plain text between two text nodes of another format (bold
x,dogcat, boldy) and type a space afterdog. The first transform madedogan entity, the second undid it (xdoghas no match), the nodes merged again, and so on until Lexical threw "One or more transforms are endlessly triggering additional transforms" and the edit was lost (#9637).Now a match at the start of a node is left as text when the simple text node before it would turn it straight back into text, that is when the match's text after that node's text is no match (
getMatch(previousText + matchText) === null, the check that node's transform makes). It is still made an entity when the first match of both nodes' text is this match, which a match that depends on the text after it can be (the entitycreateNodemakes may then match on its own). Any match right after one left as text stays text too, as it does after a match next to another entity. The match is still found in the node's own text, so the entity's own transform agrees as before, and the extra check is onegetMatchcall (the first match is already computed), made only for a match at the start of a node after a simple text node.Where that transform returned early, before its loop (its text has no match after the text node before it, or the first starts in that node), it never checked the entity after it, as its loop does. So typing a letter right before a hashtag that follows text in another format left the hashtag an entity (
b#tag), which a fresh parse of the same text does not make. In that early return the transform now turns such an entity back into text unless the transforms would keep it there (below). A match left as text at the start of the next node stays text until that node is edited, as before.Whether the transforms keep an entity right after a simple text node is one rule, used by both transforms and by the loop's check of the next node: the first match from that node's text on, passing the matches of its text (at most 8, as the loop passes over them; past that, nothing is changed), must start where the entity does, with the text after the entity in the node it was split from (the simple text node after it in the same format, style and detail), as the check that made it an entity found it; and that match must still hold with the text of any other text node after it, as the check that keeps a match at the end of a node from becoming an entity found it. So a match that needs the text after it (
/#[a-z]+(?= )/) is kept where it was made, and text after it in another format, or a token node, does not keep an entity that a fresh parse would not make.An entity's own transform, when the transforms no longer keep it after the simple text node before it (the text after it changed, say), marks that node dirty, once per node for each pending editor state (other entities' transforms may leave it dirty again), so it is turned back into text in the same update rather than at the next edit of the text before it.
A token or segmented entity with text after it in its node looped on
main: the transform of that text turns it back into text, the text merges, and the transform makes it again. #9712 keeps such an entity the third time it comes back to be turned into text in one update, and this PR keeps that rule.Two more changes come with the re-land, for orders of transforms that #9686 alone reached before the other checks:
isEntityAfter; when that cannot tell, after too many matches in the text before, it stays as it is without growing, and the text after it is parsed as usual). Otherwise it is turned back into text first, rather than grown and then turned into text with the next node's characters, which then lost their format and NodeState (two hashtags next to each other before italic text, acreateNodethat changes the case).$Bnext to a token@x) looped forever onmain; now they settle, kept as they are.Each of these checks is a few
getMatchcalls, bounded by the 8 matches passed over, only on the paths above.A matcher whose match does not hold on the entity's own text (
/dog(?=cat)/, withcreateNodekeeping the text) still loops, on plain text too, as onmain: the entity's own transform turns it back into text every time.registerLexicalTextEntityneeds an entity's text to be a match on its own. Two registrations whose entities end up touching can still loop through the rule that turns two adjacent entities into text (with a hashtag and a segmented upper-case matcher,#xabDOGloops onmainfrom a fresh parse); with one registration, or matches that cannot touch, this does not arise. The source text of an entity is told by format, style and detail; NodeState is not compared (text keeps its NodeState when it becomes an entity, butcreateNodemay give the entity its own).No API is added or changed. Entities can differ from before only next to the boundary between a simple text node and the node after it, where one of the checks above was skipped or disagreed with the other.
Closes #9637
Closes #9704
.changeset/lexical-text-entity-settle.md(@lexical/text, patch,[fix]).registerLexicalTextEntitySettle.test.ts(new, 41 tests): the issue's two cases and its controls; a later match in the node; a lookbehind-style matcher; a hashtag-like matcher after a formatted letter and a formatted space; two entities next to each other;dogcatbetween every pair of four formats and four neighbour texts, with four typed characters; IME composition; pasting formatted runs; bolding the text before an entity; typing a letter before an entity after text in another format; a matcher whose earlier match must end where the text ends; matchers that need the text after a match (with a match at the start of a node, after typing, and after removing the text after an entity), one with acreateNodethat upper-cases; text after an entity in another format; removing the text between an entity and a token after it; a token or segmented entity with text after it in its node (loops onmain), and one after a match left as text, with the nodes each settles on; an update that fails not counting toward the next; two registrations, one making token or segmented entities, in both orders, with the nodes they settle on; text after an entity with another detail; an entity after a node with an earlier match; an entity checked again in a later update; one dirty node per keystroke in text before a formatted hashtag, with and without an earlier match in that text (so history merges typing, as onmain); a match at the start of the next node that typing makes valid, made an entity when that node next changes;createNodefactories that lower-case, upper-case or add text; and a bounded number ofgetMatchcalls per keystroke in a long node before an entity, however long. Then, for three matchers, 2,000 random layouts each (override withTEXT_ENTITY_SETTLE_SEEDS):maintoo).Hashtags.spec.mjs: the playground's keywords, boldx,congratscongrats, boldy, and a space after the firstcongrats(the loop onmain: it needs a later match that the next node turns back into text, which a hashtag cannot be, since text after a hashtag only extends it); and hashtags typed after a bold letter and between bold text, with the exact text.TextEntityEdits.test.tsalso passes withTEXT_ENTITY_SEEDS=25000. The existing@lexical/texttests pass unchanged, includingregisterLexicalTextEntityEquivalence.test.ts, which compares the JSON, selection, mutations and keys after 300 random editing sessions with a copy of the helper before #9483.packages/lexical-y/src/__tests__/unit/EntityUndoSpread.test.ts(new,test.fails, #9716): two peers; a comment removed by one peer and both peers' undos leave the comment on every block after it. A@lexical/ybug (undo restores a mark opener without its closer), onmaintoo with this test; this change reaches it more often because entities keep their comments. Pinned so it flips when #9716 is fixed.registerLexicalTextEntityJoin.test.ts(new, 11 tests): #9704's repro (a list item starting with a commented hashtag joined onto italic text), the same join by backspace and by forward delete, a block before that ends with a tab (all with NodeState), a hashtag withMarkStateExtensioncomment ids joined onto italic text, two hashtags turned into text before italic text with NodeState (the text keeps its format, NodeState and the selection), typing before a hashtag that a segmented and a token entity follow (settles, with a bounded number ofgetMatchcalls), and a hashtag after text with 7 and 8 earlier matches (the mostisEntityAfterchecks) that does not grow over the commented italic text after it, while a later hashtag in that text is still made. With #9686 alone the first three lose the NodeState and theMarkStateExtensionone loses its comment ids; with #9712 alone the hashtag stays a hashtag, as onmain; here it becomes text and keeps them.