StreamSafe: carry the non-starter count across separate inputs (chunked streams) #119

StreamSafe<I> implements UAX15-D4 over one iterator: it counts the non-starters since the last starter (in NFKD) and emits U+034F once the count passes 30. The count lives in a private field (nonstarter_count), and StreamSafe::new always starts it at zero.

Text that arrives in pieces — an SSE stream of an LLM's output, a file read in blocks — cannot use it as it is: each piece is a separate &str, so each piece.chars().stream_safe() starts a fresh count, and a run of non-starters that straddles a piece boundary is never capped. The guarantee D4 gives (no more than 30 non-starters after a starter, so an NFC/NFD pass needs bounded lookahead) is exactly what such a consumer wants, since it is holding text back until a safe cut point and needs a bound on how much it may have to hold.

The only state that has to survive a piece boundary is the count: the one-character buffer is drained by the next call after a CGJ is emitted, so it is always None once a piece's iterator is exhausted. Reimplementing the counter outside the crate means reimplementing classify_nonstarters (leading/trailing non-starters of a character's decomposition, the Hangul case), which is pub(crate) — so today the choice is a copy of that logic or a lossy approximation by raw canonical_combining_class.

Would you take a small API for resuming? Something like:

#[derive(Clone, Copy, Debug, Default)]
pub struct StreamSafeState { nonstarter_count: usize }

impl<I: Iterator<Item = char>> StreamSafe<I> {
    pub fn with_state(iter: I, state: StreamSafeState) -> Self;
    /// The state to resume from; meaningful once this iterator returned `None`.
    pub fn state(&self) -> StreamSafeState;
}

Usage per piece: let it = StreamSafe::with_state(piece.chars(), state); … consume to the end …; state = it.state();. No change to new, stream_safe() or the iterator itself; the state type stays opaque so the counting rule can change without an API break. An alternative that would also do is making classify_nonstarters public, but a resumable iterator keeps the D4 logic in one place.

Happy to send a PR for whichever you prefer.