Luke Boswell (Dec 19 2024 at 23:58): Luke Boswell (Dec 19 2024 at 23:58):Sam Mohr said:
Which means creating an
Arg := [Unix (List U8), Windows (List U16)]type and just crashing onWindowsfor now. Once these are implemented, it should be a simple change to properly support 16-bit encoded strings inbasic-cliand Weaver
Just wanting to clarify here @Richard Feldman @Sam Mohr -- I think we want an alias, not an opaque type right?
Luke Boswell (Dec 19 2024 at 23:58):Is another way to say that "structural"? like non-opaque
Sam Mohr (Dec 19 2024 at 23:58):I was thinking opaque because I want it to eventually be compatible with static dispatch
Sam Mohr (Dec 19 2024 at 23:59):Which only works for custom types
Luke Boswell (Dec 20 2024 at 00:00):That was the whole Thing.to_raw_os_str() thing right?
Sam Mohr (Dec 20 2024 at 00:00):Yep
Sam Mohr (Dec 20 2024 at 00:00):Or the arg.display() : Str thing
Notification Bot (Dec 20 2024 at 00:01):7 messages were moved here from #API design > Lossy unicode conversion builtins by Luke Boswell.
Richard Feldman (Dec 20 2024 at 00:02):yeah I think this should be opaque
Luke Boswell (Dec 20 2024 at 00:16):Link to beginning of original discussion in basic-cli
Luke Boswell (Dec 20 2024 at 00:17):Did we decide on replacement for display : Arg -> Str or crash?
Richard Feldman (Dec 20 2024 at 00:18):replacement
Luke Boswell (Dec 20 2024 at 00:36):Ofc... hence the whole Str.from_utf8_lossy : List U8 -> Str functions :smiley:
Luke Boswell (Dec 20 2024 at 00:37):I think the plan is for Weaver just to crash in the meantime until we got those
Sam Mohr (Dec 20 2024 at 00:37):Yep
Luke Boswell (Dec 20 2024 at 00:37):And basic-cli with Arg.display too
Luke Boswell (Dec 20 2024 at 00:44):Ok, I think we've got a draft
module [
Arg,
to_os_raw,
display,
]
# os_str's are not necessarily valid utf-8 or utf-16
# so we store as raw bytes internally to avoid
# common mistakes
Arg := [
Unix (List U8),
Windows (List U16),
]
## Unwrap the raw bytes for decoding, typically this will be
## consumed by a package and not an end user
to_os_raw : Arg -> [Unix (List U8), Windows (List U16)]
to_os_raw = \@Arg inner -> inner
## Convert an Arg to a Utf8 string for display purposes
##
display : Arg -> Str
display = \@Arg arg ->
when arg is
Unix maybe_utf8 ->
# TODO replace with Str.from_utf8_lossy : List U8 -> Str
# see https://github.com/roc-lang/roc/issues/7390
when Str.fromUtf8 maybe_utf8 is
Ok str -> str
Err _ -> crash "tried to display invalid utf-8"
Windows _maybe_utf16 ->
# TODO replace with Str.from_utf16_lossy : List U16 -> Str
# see https://github.com/roc-lang/roc/issues/7390
crash "display for utf-16 not yet supported"
Looks good to me! I think the suggestion for naming on to_os_raw was just to_raw, but either works for me
Luke Boswell (Dec 20 2024 at 00:46):It's a nod to OsStr
Sam Mohr (Dec 20 2024 at 00:46):Yep
Luke Boswell (Dec 20 2024 at 00:49):Call me crazy... but I'm going to try and make this a package and see if that works. I'm assuming we can have the platform re-export the type so app authors don't need to import the package also just to work with it. But it could mean that Weaver could also use the same opaque type.
Luke Boswell (Dec 20 2024 at 00:50):I'll experiment for a little to see if it works
Sam Mohr (Dec 20 2024 at 00:55):That would probably be a better way to go about this, we would just need to get used to managing an upstream library from basic-cli
Sam Mohr (Dec 20 2024 at 00:55):Aka we don't have semver yet, so I don't know how this interaction will work
Luke Boswell (Dec 20 2024 at 00:56):Yeah, it doesn't work I think. Looks like something hangs with roc check and I'm not keen to dig too much rn.. I'd rather get this into basic-cli
Luke Boswell (Dec 20 2024 at 02:15):This ArgOs / OsStr / OsStrExt thing is a deep rabbit hole. Just digging through the rust API's for this in std::env and std::os::windows etc and there's a lot here to get it right
Luke Boswell (Dec 20 2024 at 02:15):Really validates for me this is the right direction we're heading in... but slowing down an intial implementation
Luke Boswell (Dec 20 2024 at 02:38):Found the answers I'm looking for in https://docs.rs/widestring/latest/widestring/
Anthony Bullard (Dec 20 2024 at 03:13):Luke Boswell said:
This ArgOs / OsStr / OsStrExt thing is a deep rabbit hole. Just digging through the rust API's for this in
std::envandstd::os::windowsetc and there's a lot here to get it right
Don't forget CowStr
Anthony Bullard (Dec 20 2024 at 03:14):Which I was just reminded of, looking at our html rendering code
Luke Boswell (Dec 20 2024 at 03:31):I've got something drafted... https://github.com/roc-lang/basic-cli/pull/293
It's working on macos, but not linux for some reason. Just investigating
Last updated: Jul 06 2025 at 12:14 UTC