ScriptItemization

ScriptItemization :: # (opaque)

Script runs under the package's explicitly named ConservativeScxV1 policy.

This is shaping-oriented package policy, not a normative Unicode itemization algorithm. It uses Unicode 17 Script/Script_Extensions facts and the Unicode 17 extended grapheme algorithm. Grapheme clusters are atomic. Restricted scx candidates use nearest explicit neighbors, then explicit preference order, then their explicit primary Script, otherwise Common. Broadly Common spans inherit one equal/single explicit side; conflicting or absent sides remain Common. Inherited-only content becomes Common. Unknown remains visible and blocks propagation. Paired punctuation and language are deliberately not inferred by ConservativeScxV1.

with_preferred : List(Value) -> ConservativeScxV1

Construct explicit application preference order. The policy retains the caller's immutable list; construction does not copy it. Resolving one restricted cluster scans preferences in order, so its lookup cost is O(P). Applications should keep this explicit tailoring list short. Common, Inherited, Unknown, duplicates, and scripts absent from a candidate set have no effect; no language is inferred.

iter_runs : Str, ConservativeScxV1 -> Iter(Run)

Lazily traverse ordered, coalesced, half-open runs.

This complete-string exact policy performs interval-local replay: definite scripts/Unknown delimit ambiguous intervals in the first scan, and each such interval is classified once more from its grapheme-aligned start. No unresolved text, substring, or descriptor tape is copied. Every scalar is classified at most twice; auxiliary state is constant. The iterator retains the source and explicit preference list for its lifetime; yielded Run values retain neither.

fold_runs : Str, ConservativeScxV1, state, (state, Run -> state) -> state

Fold without collecting. The user callback is a shallow consumer of the non-generic exact replay iterator; it is not threaded through Unicode decoding, grapheme transition, or replay-machine specializations.

runs : Str, ConservativeScxV1 -> List(Run)

Collect runs. Allocation is proportional to returned run count only.

cursor_init : ConservativeScxV1, U64 -> Cursor

Start an exact non-replayable chunk cursor with an explicit bound on retained unresolved grapheme-cluster descriptors.

cursor_push : Cursor, Str, state, (state, Run -> state) -> [Pushed({ cursor : Cursor, state : state }), Failed({ cursor : Cursor, state : state, error : Error })]

Consume one scalar-aligned chunk. A chunk end is not end of text. Limit/coordinate failure is atomic: returned caller state is unchanged, the crossing descriptor is not retained, and the returned cursor is terminally failed.

cursor_finish : Cursor, state, (state, Run -> state) -> [FinishedCursor({ cursor : Cursor, state : state }), Failed({ cursor : Cursor, state : state, error : Error })]

Resolve end-of-text once, emit the final coalesced run, and complete.

Run : {
    range : TextRange,
    script : Value,
}

One coalesced script run. Both axes of range identify the same grapheme-aligned span of the logical source; the value retains no text.

Cursor

:= {
    machine : Machine,
    cluster : ClusterState,
    byte_offset : U64,
    scalar_offset : U64,
    pending : List(Unit),
    left : ExplicitNeighbor,
    last : [NoRun, LastRun(Run)],
    policy : ConservativeScxV1,
    max_pending_units : U64,
    status : CursorStatus,
}

Exact ConservativeScxV1 state for non-replayable scalar-aligned chunks. Pending entries are compact cluster descriptors; no chunk or slice is retained. Storage is proportional to unresolved units and bounded by max_pending_units.