Where can I find the functions available in the String (Str ?) library?
this is probably the most up-to-date list https://github.com/rtfeldman/roc/blob/trunk/compiler/module/src/symbol.rs#L905
not very discoverable or friendly, but that's what's actually implemented
I'll work on improving that tonight!
Excellent! I've decoded just a tad, see section 6 of https://l1-lab.lamdera.app/g/jxxcarlson-the-roc-language-by-example-2021-08-05 -- will belay and look at what you do.
Here is the beginning of some notes on parser combinators: https://l1-lab.lamdera.app/g/jxxcarlson-roc-parser-combinator-notes-2021-08-07
Some questions there (all very brief)
One question. Is it possible to split a string (x:xs)
into Pair x xs
? By (x:xs)
, I mean x = first char
, xs = the rest
.
Something like String.uncons
in Elm
One question. Is it possible to split a string
(x:xs)
intoPair x xs
? By(x:xs)
, I meanx = first char
,xs = the rest
.
I don't think so. I find elm has a small number of builtins in general so if it's in elm it might be worth including in roc as well, @Richard Feldman @Folkert de Vries.
About the questions in the notes:
Can we import stuff into the repl?
No, I dont't think so. Although I think we planned on adding that in the future.
Need to learn how put these in a module (or interface???) and how to make use of what is in that file in a test app or in the repl.
examples/benchmarks/QuicksortApp.roc uses things from the Quicksort interface. That should be a good example for you.
Thanks @Anton , very helpful!
ok, I wrote up a bunch of notes on string parsing API thoughts!
https://gist.github.com/rtfeldman/ae69d41e0f73d3059d14b7d54186edc1
some of those things that aren't implemented should be very quick, and good beginner projects for contributing to the compiler - if anyone wants to pair on implementing any of them this weekend, DM me!
also, love to hear any feedback anyone has about that gist :point_up:
I like that the design of Str
has the specific use case of "writing arbitrary string parsers" and pattern matching APIs geared towards it. In my experience with scripting languages, there's a perpetual temptation to hack together code that sort of works using clever hash-table and string manipulation, so if the ergonomics of Roc can guide users towards modeling their problem with data structures instead, that's a win.
To that end, I'd propose keeping the API of strings reasonably minimal. What exists now seems reasonable, and a few more helpers might be nice, but I'd get nervous about adding too much regular expression sugar or Rubyims like squeeze
and friends. I also think the temptation to write Str.length
and other similar functions gives us the chance to add lots of lovely compiler messages about why strings can be so confusing and why you might want to choose something else.
Could we build mechanisms into the compiler/editor to help application and platform writers decide where their parsing functions belong? I could envision a scenario where a platform author adds a built-in after discovering application developers writing lots of similar parsing code. Would adding explicit tooling for that use-case be worthwhile? Probably not yet, but maybe in the future...
Philosophically, I think users tend to be happier when they understand the why of an API, not merely the what. I thought the gist did a good job conveying that :smile:
Richard Feldman said:
also, love to hear any feedback anyone has about that gist :point_up:
Makes a lot of sense to me. I like it.
gist nice
Elixir has the best string story I’ve seen so far.
https://elixir-lang.org/getting-started/binaries-strings-and-char-lists.html
they take into account graphemes and have some gnarly pattern matching for them too
Woaaah, the bitstring matching looks freaking slick
yea aside from that matching roc has a lot of what elixir does with strings
‘(‘ vs. 40
In elixir a list of numbers will print as a Charlist if the numbers are a readable sequence of characters
well in roc you could match on bytes but elixir has that wild thing where you can say what sizes each chunk should be
Wild is the right word, I wanna go read a parser in Elixir now
https://github.com/elixir-ecto/ecto/blob/v3.5.4/lib/ecto/uuid.ex
this is how one would generate a UUID with the same syntax
I don't know if I'd want to read code that looks like this every day, but in isolation... wow, that's gorgeous
Thanks so much! The link above was really helpful, and as I mentioned in my reply to your Gist, I think that the foundation is almost there for writing fault-tolerant pasrsers.
Richard, I put a very brief comment in your gist. I'm satisfied that this lays a good foundation for what I want to do — implement something along the lines of Hughes and Meijer: Parse Combinators. I'm going to have a go at it and can probably have more specific feedback after I've gotten into the code.
Is there an analogue of Elm's Debug.toString
?
there is not...the plan is to do that through the editor, but that's a ways off :sweat_smile:
OK!
FYI, @James Carlson you can now find up-to-date (as in, what's actually currently implemented) documentation for the Str
module at https://www.roc-lang.org/builtins/Str - so far I've only added Str
and Bool
there, but I'm hoping to add the other builtin modules soon!
there's a bug where Bool
is always the expanded module on the sidebar; I'll fix that tomorrow!
Fantastic!
Extremely cool that List.len
returns a Nat
! That's the way it's supposed to be.
Why this error?
I expected to reach the end of the file, but got stuck here:
1│ interface MiniParsec exposes [ result, test1 ] imports []
2│
3│ result = \v -> (\inp -> [Pair v inp])
4│
5│ test1 n = (result "a") n |> List.len
^
no hints%
There is a blank line 6
Oops, the caret should be at the beginning of line 5
can you file an issue for it? that's a confusing error message!
the problem is that functions are always defined using lambdas in Roc - should be test1 = \n -> ...
Ah ok, i forgot that. Will file
ok, I fixed the sidebar bug on https://www.roc-lang.org/builtins/Str - you have to refresh the page to get it instead of the cached version, because I haven't set up hashed assets on Netlify yet :laughing:
Very nice! Here is another one. First, we have
# MiniParsec.roc
interface MiniParsec exposes [ result, test1, idNat] imports []
result : a -> (b -> List [ Pair a b ]*)
result = \v -> (\inp -> [Pair v inp])
test1 : Nat -> Nat
test1 = \n -> (result "a") n |> List.len
idNat : Nat -> Nat
idNat = \n -> n
Then this:
# ParseApp.roc
app "parseap"
packages { base: "platform" }
imports [base.Task, MiniParsec]
provides [ main ] to base
main : Task.Task {} []
main =
Task.after Task.getInt \n ->
MiniParsec.idNat n
|> Str.fromInt
|> Task.putLine
But the below hangs after entering an integer (5):
➜ roc git:(trunk) ✗ cargo run run jim/ParseApp.roc
Finished dev [unoptimized + debuginfo] target(s) in 0.77s
Running `target/debug/roc run jim/ParseApp.roc`
5
I'll take a look
the compiler hanging is odd. Usually it just panics, I think it hanging could only be an infinite loop ?!
well, really your program here is running forever
which, if it's a compiler problem would usually segfault
got it. getInt
returns a I64
, but then you try to use it as an Int
normally, that should give a good error message, but in this case I guess you built the compiler with --release
(turning off debug messages) and the application hits a panic at a later stage because the input is illtyped and therefore malformed
I just got it too :)
using idNat (Num.intCast n)
allows you to cast any integer type to another
OK, on my end, I will clean up my bad types
Hmm .. I am getting the same behavior with
idI64 : I64 -> I64
idI64 = \n -> n
Hmmm, do you have something different than I?
interface MiniParsec exposes [ result, test1, idI64] imports []
result : a -> (b -> List [ Pair a b ]*)
result = \v -> (\inp -> [Pair v inp])
test1 : Nat -> Nat
test1 = \n -> (result "a") n |> List.len
idI64 : I64 -> I64
idI64 = \n -> n
app "parseap"
packages { base: "platform" }
imports [base.Task, MiniParsec]
provides [ main ] to base
main : Task.Task {} []
main =
Task.after Task.getInt \n ->
MiniParsec.idI64 n
|> Str.fromInt
|> Task.putLine
This is working for me
We seem to have identical files (after insertion of whitespace). I verified with diff
. However, it may be that I had not run git pull
. Now I'm not sure which command to run (after the git pull):
➜ roc git:(trunk) ✗ cargo run run jim/ParseApp.roc
Finished dev [unoptimized + debuginfo] target(s) in 0.28s
Running `target/debug/roc run jim/ParseApp.roc`
`roc run` is deprecated! (You no longer need the `run` - just do `roc [FILE]` instead of `roc run [FILE]` like before.
➜ roc git:(trunk) ✗ roc jim/ParseApp.roc
zsh: command not found: roc
➜ roc git:(trunk) ✗ cargo roc jim/ParseApp.roc
error: no such subcommand: `roc`
Did you mean `doc`?
➜ roc git:(trunk) ✗ cargo run roc jim/ParseApp.roc
Finished dev [unoptimized + debuginfo] target(s) in 0.25s
Running `target/debug/roc roc jim/ParseApp.roc`
File not found: roc
Hmmmm ....
With cargo it's cargo run jim/ParseApp.roc
now
I'll improve the suggestion in the deprecation message
The suggestion refers to roc as in target/debug/roc
It's really helpful to have you trying out all this stuff @James Carlson :)
I am well-known for my ability to commit errors , blunders, oversights, and mess-ups of all kinds :laughing:
Hmm .. really strange. I copied Anton's code verbatim, have done git pull, and cargo run jim/ParseApp.roc
still "hangs" when I enter "5<RETURN>"
Can you tell me what platform you are using? I put the two files in examples/benchmarks
so I'm using examples/benchmarks/platform
maybe he isn’t using a platform and that’s why stuff is acting up?
I think that would result in some kind of file error tho so maybe not
Yep I just tried it. That's what happens when there is no platform. I'll make an issue for a better error.
#1552
I moved jim/
into ./examples/benchmarks
. However, I get the same error:
➜ benchmarks git:(trunk) ✗ cargo run jim/ParseApp.roc
Finished dev [unoptimized + debuginfo] target(s) in 0.68s
Running `/Users/jxxcarlson/dev/roc/roc/target/debug/roc jim/ParseApp.roc`
9
^C
I've pushed a new branch jim
with my code.
hmm, I don't see a jim
folder when I check out that branch, or a ParseApp.roc
:thinking: - maybe something didn't get pushed?
but here are the critical ingredients:
cargo run
command from the root roc/
directory (with the main rtfeldman/roc
README file in it)cargo run
command is being passed the path to your .roc
application file, relative to the current directory - so for example if you're running cargo
from ~/roc/
and the .roc
file you want to run is in ~/roc/examples/ParseApp.roc
then you'd run that with cargo run examples/ParseApp.roc
ParseApp.roc
has a platform/
folder in the same directory as it (for example, in examples/hello-world/
there's examples/hello-world/Hello.roc
and also examples/hello-world/platform/
)so for the command cargo run jim/ParseApp.roc
to work, you'd need to be able to run ls jim/ParseApp.roc
and have that succeed, and also ls jim/platform/
and have that succeed too
if either of those ls
commands fails, then something is amiss!
@Richard Feldman — All is good! I moved the contents of examples/benchmarks/jim
into examples/benchmarks/
and ran this:
➜ roc git:(jim) ✗ cargo run examples/benchmarks/ParseApp.roc
Finished dev [unoptimized + debuginfo] target(s) in 0.25s
Running `target/debug/roc examples/benchmarks/ParseApp.roc`
8
8
runtime: 466.063ms
➜ roc git:(jim) ✗
This was of course a dry run for doing real work. :laughter_tears:
Yes, I forgot to commit some files. I had misgivings about putting stuff in examples/benchmarks
, but then I realized that no one can see them in the working branch. ((Hurrah!))
FWIW, I pushed jim
awesome, glad we got it resolved! :smiley:
also, this experience report helped identify several improvements to make, so thank you so much for your patience! :pray:
I love the bleeding edge, LOL!
Question: is there some kind of Task.getLine
to pair with Task.putline
?
depends on the platform. I think you're using the one in benchmarks
https://github.com/rtfeldman/roc/blob/trunk/examples/benchmarks/platform/Task.roc
and I don't see a getLine in that one. There seems to be a getInt tho
Never mind, all good and getting better!
Nice
Hmm ... What does this mean?
➜ roc git:(dev) ✗ vr parser
Finished dev [unoptimized + debuginfo] target(s) in 0.68s
Running `target/debug/roc examples/benchmarks/ParseApp.roc`
thread 'main' panicked at 'There were still outstanding Arc references to module_ids', /Users/jxxcarlson/dev/roc/roc/compiler/load/src/file.rs:1576:33
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error: Failed at the parser script
Here vr parser == cargo run examples/benchmarks/ParseApp.roc
where vr == velociraptor
What's velociraptor?
Velociraptor is a script runner very lean and clean: https://dev.to/umbo/velociraptor-an-npm-style-script-runner-for-deno-26a
It behaves the same without velociraptor right?
Can you push it to the Jim branch?
Yes, the same . Just pushed the script. I really like velociraptor. No bloat.
Is there an analogue of Elm's
List.concat : List (List a) -> List a
??
Ah: List.join
!! Figures!
I've made progress with MiniParsec.roc
in the jim
branch: the primitive parsers zero
(always fails), result
(always succeeds, consumes no input), and item
: consumes one character and otherwise fails. Also, possibly correct Parser.map
and Parser.andThen
where Parser = MiniParsec
. Pretty much trying to follow the paper by Hughes & Meijer.
What would speed things up enormously is the ability to import interfaces into the repl to experiment with them. Not sure if this is easy, medium, or hard, but it will be great when it gets there.
I'll probably give the primitive parsers better names and rename the parser interface to something less pretentious.
As part of this, I wrote the beginnings of an interface Pair.roc
with functions first
and second
, like Elm's Tuple
stuff, and also some mapping functions:
first : [Pair a b] -> a
first = \(Pair a _) -> a
second : [Pair a b] -> b
second = \(Pair _ b) -> b
mapFirst : [Pair a x], (a -> b) -> [Pair b x]
mapFirst =
\(Pair a b), f -> (Pair (f a) b)
mapSecond : [Pair a x], (x -> y) -> [Pair a y]
mapSecond =
\(Pair a b), f -> (Pair a (f b))
Overall, quite a pleasant coding experience after the first painful day. Thanks all!!!
Perhaps a Pair
library already exists?
I can work on adding imports to the repl this weekend!
I'd say it's a medium-sized project :big_smile:
Perhaps a
Pair
library already exists?
No, I don't think so.
cargo run examples/benchmarks/ParseApp.roc
Is working for me on the latest Jim branch.
I'm wondering if it could be the rust version, can you send over the output of rustup show
@James Carlson ?
Can you also check if cargo test
(from the root of the repo) runs without failures?
Everything good with cargo test
, with the excption of. test cli_run::rbtree_insert ... FAILED
Everything is working for me now. One thing that had stumped for a while is a mysterious error message that occurs when there is a missing comma in the import list. I will file an issue.
Is there a roc testing library?
Is there a block comment in roc?
no and no. That rbtree issue should go away on a second try. We have some race condition in the tests when a file is not present (i.e. on first run)
@Anton or @Folkert de Vries , if you have a moment, could you look at commit 8997bf8be
in the jim
branch regarding the definition of andThen
:
andThen : Parser a, (a -> Parser b) -> Parser b
andThen = \p, q ->
\input -> p input |> List.map (\(Pair a input2) -> (q a) input2) |> List.join
You can run a test on it with cargo run examples/benchmarks/ParseApp.roc
. Here is the test:
app "parseapp"
packages { base: "platform" }
imports [base.Task, Parser]
provides [ main ] to base
main : Task.Task {} []
main =
# TEST Parser.andThen
Parser.runToString Parser.showU8 "abcd" ( Parser.andThen (Parser.satisfy (\u -> u == 97)) (\u2 -> Parser.satisfy (\u3 -> u3 == u2)) )
|> Task.putLine
I am not feeding the second parser q a
properly. If I replace (\u3 -> u3 == u2)
with (\u3 -> u3 == 98)
, I get the expected output, but that of course is not the point.
Thanks!
what is the problem you observe?
I get an error message Parse error (runToString)
from the function runToString
.
Because I get the expected output with the subsitution (\u3 -> u3 == 98)
for (\u3 -> u3 == u2)
, I know that the problem is in how the second parser in andThen
is being fed its info (or handling that info)
Here is the run
code:
run : Str, Parser a -> Result a [ListWasEmpty]
run =
\str, parser -> parser (Str.toUtf8 str) |> List.map Pair.first |> List.first
runToString : (a -> Str), Str, Parser a -> Str
runToString =
\toString, str, parser ->
when run str parser is
Ok a -> toString a
_ -> "Parse error (runToString)"
Pretty sure the problem is not there.
but u3 == u2
should fail right? b != c
Well, I can't disagree! I think I see what the problem is. Thanks!
@Folkert de Vries My test was incorrect. The below works:
# TEST Parser.andThen
# Recognize strings beginning with "aa"
Parser.runToString Parser.showU8 "aaxyz" ( Parser.andThen (Parser.satisfy (\u -> u == 97)) (\u2 -> Parser.satisfy (\u3 -> u3 == u2)) )
nice
Hmmm ... Error on git push
:
➜ roc git:(jim) git push
Enumerating objects: 15, done.
Counting objects: 100% (15/15), done.
Delta compression using up to 16 threads
Compressing objects: 100% (11/11), done.
Writing objects: 100% (11/11), 1.38 KiB | 1.38 MiB/s, done.
Total 11 (delta 7), reused 0 (delta 0)
remote: Resolving deltas: 100% (7/7), completed with 4 local objects.
remote: fatal error in commit_refs
To github.com:rtfeldman/roc.git
! [remote rejected] jim -> jim (failure)
error: failed to push some refs to 'git@github.com:rtfeldman/roc.git'
This might help
Hmm ... never used git gc
or git fsck
before. Unfortunately, the error persists. Just checked: the compiler repo is online
Kind of a hack, but maybe create a new branch based on trunk and copy over your files.
I've never encountered this error either.
I'm encountering it too; maybe GitHub is down?
Uhu, looks like it.
I'll wait a while and see what happens. When I dogit fsck
, I get this:
➜ roc git:(jim) git fsck
Checking object directories: 100% (256/256), done.
Checking objects: 100% (71877/71877), done.
dangling commit 19b2d41cd70df2f24b1b91aeb387a359130100ba
dangling tree bce4c448a8cea397f51573f8951e2096a47d3a9d
dangling commit b617a55998ea6c81c52b3f1e5ac3a96ab4c3c985
dangling commit 32c9f99ee8cc7090481fd4edf45a63cf2f31a555
dangling tree b5d7b19d0dc3e36e51a7786e004ab52582ab7bfb
dangling tree 4577eab778e36ff32f767c843a4c0354bb1977b3
dangling tree 379fa66772dc46659b618374a98e3c2ee05ad2f7
dangling commit 68ceda8f3ab08c8e751c283ebbdcb87bde931403
dangling commit b5d84e7ca957d950322508b184ca020760fced15
dangling blob e62c9f7e5dc3baa8a285536ddb10d302b612b7e8
I have the type definition Step state a : [ Loop state, Done a ]
and Counter : { counter : I64, value : I64 }
The functionloop : (state -> Step state a), state -> a
is used with updateCounter : Counter -> Step Counter I64
as the first argument and with a Counter
value as the second.
Later I say this:
test3 : Str
test3 =
out = loop updateCounter { counter : 4, value : 0}
if out == Done 10 then "Ok" else "Fail"
But calling test3
gives a type error:
41│ if out == Done 10 then "Ok" else "Fail"
^^^^^^^
This Done global tag application has the type:
[ Done (Num a) ]b
But isEq needs the 2nd argument to be:
I64
Shouldn't the various I64
declarations above make this not happen?
Or is there some way to ensure that Done 10
is Done I64
?
Can you push it to the jim branch? github should be good again
Done!
My first thought is that out
is a I64
, not a [ Done I64 ]b
which is why isEq
would not work.
Ah yes indeed!! let me verify that.
Works -- thanks!
Last updated: Jul 05 2025 at 12:14 UTC