Word

Word :: # (opaque)

Unicode 17 default UAX #29 word-boundary ranges.

These are all ranges between default word boundaries: whitespace and punctuation are ordinary ranges, not filtered dictionary words.

iter_ranges : Str -> Iter(ByteRange)

Lazily visit nonempty default word-boundary ranges in source order.

fold_ranges : Str, state, (state, ByteRange -> state) -> state

Fold default word-boundary ranges without materializing them.

ranges : Str -> List(ByteRange)

Collect default word-boundary ranges.

slices : Str -> List(Str)

Return seamless slices, retaining the source backing allocation.

owned : Str -> List(Str)

Return independently owned strings for each word-boundary range.

Cursor

Word.Cursor :: # (opaque)

init : {  } -> Cursor

push : Cursor, Str, state, (state, ByteRange -> state) -> [
    Pushed({ cursor : Cursor, state : state, consumed : U64 }),
    Failed({ cursor : Cursor, state : state, consumed : U64, error : Error }),
]

Push a scalar-aligned chunk. Pushed accepts the full chunk; Failed is terminal and returns a faulted cursor. Empty chunks are no-ops.

finish : Cursor, state, (state, ByteRange -> state) -> [
    End({ cursor : Cursor, state : state }),
    Failed({ cursor : Cursor, state : state, error : Error }),
]

Resolve the final pending decision and emit the final nonempty range.