Skip to content

Lexer: reserved Unicode syntax must beat generic infix fallback #4

Description

@isomorphisms

IR deliberately allows broad Unicode glyphs to act as user-definable infix operators. SPECIAL is therefore not supposed to be narrowed to only a small approved set.

The invariant is precedence: glyphs selected for built-in syntax must be recognized by their explicit lexer rules before the generic Unicode infix fallback can claim them.

The immediate case is equality. must tokenize as the existing equality token/semantics, not as a generic SPECIAL operator. The ASCII aliases ?=, =?, ?=?, and ¿=? should likewise resolve to the same equality operation.

Acceptance criteria:

  • tokenizes as equality, never generic SPECIAL.
  • ?=, =?, ?=?, and ¿=? are tested as aliases for .
  • Existing reserved Unicode syntax such as assignment arrows continues to beat the generic infix fallback.
  • Broad user-definable Unicode infix behavior remains available for unreserved glyphs.
  • Regression tests include at least one unreserved Unicode infix glyph proving that fallback still works.
  • Code-point literals used by these lexer/tests carry the rendered glyph beside them, and tests verify important code-point↔glyph pairs (for example U+03C0 really renders as π) so the comments cannot silently drift from what the code prints.

This is a lexer-priority invariant: explicit syntax first, generic Unicode infix fallback second.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions