Stream: show and tell

Topic: Weaver: record builder CLI parsing library


view this post on Zulip Sam Mohr (Apr 08 2024 at 12:17):

I've been pretty excited by the record builder changes that got added last year, and I've been seeing some discussion around us having a good example of using them. I thought of all of my experience using Clap in Rust, since their #[derive(Parser)] user experience is great because you can "Parse, not Validate". I spent the last week cracking out a library to do type-safe CLI parser generation using the record builder pattern and I'm really happy with the results!

Some of the nice features we get:

Below is a small example of using the lib, and here's the repo if you're interested: https://github.com/smores56/weaver

expect
    subcommandParser =
        cliBuilder {
            d: <- numOption { name: Short "d", help: "A required number." },
            f: <- maybeNumOption { name: Short "f", help: "An optional number." },
        }
        |> finishSubcommand { name: "sub", description: "A specific action to take.", mapper: Sub }

    { parser, config: _ } =
        cliBuilder {
            alpha: <- numOption { name: Short "a", help: "Set the alpha level." },
            beta: <- flagOption { name: Both "b" "beta" },
            xyz: <- strOption { name: Long "xyz" },
            verbosity: <- occurrenceOption { name: Both "v" "verbose" },
            sc: <- subcommandField [subcommandParser],
        }
        |> finishCli { name: "app", version: "v0.0.1", authors: ["Some One <some.one@mail.com>"] }
        |> assertCliIsValid # crash immediately on unrecoverable issues, e.g. empty flag names

    out = parser ["app", "-a", "123", "-b", "--xyz", "some_text", "-vvvv", "sub", "-d", "456"]

    out
    == SuccessfullyParsed {
        alpha: 123,
        beta: Bool.true,
        xyz: "some_text",
        verbosity: 4,
        sc: Ok (Sub { d: 456, f: Err NoValue }),
    }

As mentioned in the README, I've got the basics mostly sorted, but I've got to go through and do a good bit more testing than is currently being done. Then, the next step is documentation, and once this feels stable, I'll start doing GitHub releases so you can use this library without having to download it first :sweat_smile:

I have to go to bed now, but I'd love to discuss any ideas you have for "must-have" features or ergonomic fixes you'd make to the API or internals. Feel free to ping me!

view this post on Zulip Isaac Van Doren (Apr 08 2024 at 12:25):

Awesome!

view this post on Zulip Anton (Apr 08 2024 at 14:02):

Clever use of the record builder syntax!

view this post on Zulip Richard Feldman (Apr 08 2024 at 15:58):

wow, fantastic!!! :heart_eyes:

view this post on Zulip Richard Feldman (Apr 08 2024 at 15:59):

I love that you got the automatic help text generation in there...being able to support that was one of the original motivating reasons behind having the record builder syntax, and it's super exciting to see the first-ever example of it in action! :grinning_face_with_smiling_eyes:

view this post on Zulip Richard Feldman (Apr 08 2024 at 16:01):

what do you think of putting the different argument styles into a separate module? e.g. d: <- Arg.num { ... } instead of d: <- numOption { ... }

view this post on Zulip Sam Mohr (Apr 08 2024 at 17:08):

Yeah, that seems like a good way to compactify the method calls. I was also trying to do something terse for the argument names with default records, but I realized that default values in type definitions universalize, meaning two function calls to the same function can't omit different fields from the same record definition. Is that intended behavior?

view this post on Zulip Hristo (Apr 08 2024 at 17:15):

I'm still a beginner, so my apologies. I could be misunderstanding, but if I'm not, this isn't intended behaviour. It was recently discussed here.

view this post on Zulip Sam Mohr (Apr 08 2024 at 17:16):

I figured, thanks for the link!

view this post on Zulip Agus Zubiaga (Apr 08 2024 at 19:03):

Nice! Looking forward to using this :smiley:

view this post on Zulip Sam Mohr (Apr 10 2024 at 10:55):

Weaver has now been released on GitHub, so you can use it in your applications now! The example in the README shows a working example of using Weaver from the net. I've documented all of the public features for now, and I've released said docs on GitHub pages here using the GitHub workflows I stole borrowed from @Luke Boswell 's roc-json library.

Since you last saw it, I made the change that @Richard Feldman suggested to name functions assuming they'll be called as module members, so numOption is now Opt.num, and strListParam is now Param.strList, and so on. (I didn't call it Arg.num because that overlaps with basic-cli's Arg module and we can't currently rename imports, though I know that's in progress).

I probably won't be going nearly as fast in developing it now that's it's in an MVP state, but I'll still be working on it after my vacation comes and goes this weekend. Happy Weaving!

Repo: https://github.com/smores56/weaver
Docs: https://smores56.github.io/weaver/Cli/
Download URL: https://github.com/smores56/weaver/releases/download/0.1.0/MnJi0GTNzOI77qDnH99iuBNsM5ZKnc-gZTLFj7sIdqo.tar.br

view this post on Zulip Sam Mohr (May 05 2024 at 09:21):

Weaver has now reached v0.2.0! The main changes are:

At the suggestion of Richard Feldman, I wrote an article that explains why I wrote Weaver, which should be ready to go. However, before it's properly disseminated, I'd love to get a pair of eyes that aren't mine on it just to double check it. Then we can get more Roc propaganda on HackerNews!

view this post on Zulip Luke Boswell (May 05 2024 at 09:29):

Nice.

Noticed some issues with the article on my phone using chrome.

Screenshot_20240505-112812_Chrome.jpg

view this post on Zulip Luke Boswell (May 05 2024 at 09:29):

I cant read or scroll to see on the right.

view this post on Zulip Sam Mohr (May 05 2024 at 09:31):

Yeah, seems that making it narrower on my computer wasn't narrow enough

view this post on Zulip Luke Boswell (May 05 2024 at 09:35):

Can you put the Arg.List! inline in the when statement? If that doesnt work I might need to look into the unrwapping for that part.

    when Cli.parseOrDisplayMessage cli (Arg.list!) is

view this post on Zulip Luke Boswell (May 05 2024 at 09:36):

Only if you want to do it like that

view this post on Zulip Sam Mohr (May 05 2024 at 09:50):

Can you put the Arg.list! inline?

Done! And your mobile should work better, please let me know if you've still got issues

view this post on Zulip Sam Mohr (May 05 2024 at 09:52):

I tested that on the basic.roc example from the Weaver repo and it worked

view this post on Zulip Luke Boswell (May 05 2024 at 10:35):

Looks good on mobile now :+1:

view this post on Zulip Luke Boswell (May 05 2024 at 10:37):

Btw, I've been using Weaver on something I've been working on and it's super awesome. :ok: It just works nicely and haven't had any issues. Looking forward to trying out v2.

view this post on Zulip Hannes (May 05 2024 at 10:52):

Nice article, I'm looking forward to use Weaver for my next Roc project!

I'm confused about the Python example though, that syntax is very unfamiliar to me, is it real code or just hypothetical?

Using the Python stdlib, I would write something like:

import argparse

parser = argparse.ArgumentParser(description='my-app')
parser.add_argument('-f', '--force', action='store_true')
parser.add_argument('file', type=str)
parser.add_argument('amount', type=int)

args = parser.parse_args()

print("File:", args.file)
print("Amount:", args.amount)
print("Force:", args.force)

The type arguments are functions that are run on the corresponding argument to parse the string to a specified type, which makes it seem like there could be some type inference, but the args variable has the Namespace type, and all the properties have the Any type.

view this post on Zulip Chris (May 05 2024 at 15:04):

great article! I don't really understand the builder syntax (and applicative in general lol), so I'm even more impressed by weaver

view this post on Zulip Chris (May 05 2024 at 15:05):

i believe the type for allColors here would be a List [Red, Green, Custom U64]?
image.png

view this post on Zulip Richard Feldman (May 05 2024 at 16:14):

ziutech said:

I don't really understand the builder syntax (and applicative in general lol)

@Agus Zubiaga and I have talked about a way to make it simpler from both a types and syntax perspective, so this is encouraging feedback to hear! :smiley:

I'll write that design up, maybe in the next week or two

view this post on Zulip Sam Mohr (May 05 2024 at 17:54):

Hannes said:

I'm confused about the Python example though, that syntax is very unfamiliar to me, is it real code or just hypothetical?

It's hypothetical. I tried to clarify that a bit in the article by saying it's a "simplified example" and "if I used the stdlib, it'd be nicer, but only because dynamic type system"

view this post on Zulip Sam Mohr (May 05 2024 at 17:54):

Okay @Richard Feldman it's ready to send out (link), at your leisure. Unless you'd rather I send it out?

view this post on Zulip Richard Feldman (May 05 2024 at 17:54):

go for it! :smiley:

view this post on Zulip Richard Feldman (May 05 2024 at 17:54):

thanks for writing it up!

view this post on Zulip Sam Mohr (May 05 2024 at 17:55):

Not a social media person, so uh... where should I send it? I don't have a twitter or an HN account.

view this post on Zulip Sam Mohr (May 05 2024 at 17:56):

I'll put it on Reddit for starters

view this post on Zulip Sam Mohr (May 05 2024 at 18:26):

https://www.reddit.com/r/programming/comments/1ckxqyi/announcing_weaver_an_ergonomic_cli_parsing/
https://www.reddit.com/r/functionalprogramming/comments/1ckxnub/announcing_weaver_an_ergonomic_cli_parsing/

I posted it to these two subreddits to get a large audience, and a small but more-appropriate audience. Feel free to upvote if you want to!

view this post on Zulip Richard Feldman (May 05 2024 at 18:29):

very nice, will do!

view this post on Zulip Richard Feldman (May 05 2024 at 18:30):

thanks for writing it up, and on a personal note, thank you for the kind words in it :hug:

view this post on Zulip Hannes (May 06 2024 at 00:56):

I've submitted it to Lobsters, I'm not sure if Lobsters penalises upvotes from a direct link like HN does, so if you want to upvote there, go to https://lobste.rs/recent and search for "Weaver" :)

view this post on Zulip Sam Mohr (Dec 02 2024 at 11:06):

Weaver has been updated to work with current Roc: https://github.com/smores56/weaver/releases/tag/0.4.0

You should be able to use it for all your purity inference needs once that rolls out.

view this post on Zulip Sam Mohr (Dec 22 2024 at 01:56):

v0.5.1 - OS-aware arg parsing!

download here: https://github.com/smores56/weaver/releases/download/0.5.1/nqyqbOkpECWgDUMbY-rG9ug883TVbOimHZFHek-bQeI.tar.br

This release brings OS-aware arg parsing to Weaver!

Most operating systems don't guarantee UTF-8 encoding of arguments, and Windows doesn't even use bytes, but U16 codepoints. To handle this for you, Weaver and basic-cli have provided OS-aware arguments that are U8 lists for Unix systems and U16 lists for Windows systems. You'll get these from main! in basic-cli, and you can call Weaver almost the same way you used to, but you'll need to pass one extra argument to Cli.parse_or_display_message:

main! = \args ->
    data =
        Cli.parse_or_display_message cli_parser args Arg.to_os_raw
        |> try Result.onErr! \message ->
            try Stdout.line! message
            Err (Exit 1 "")

    try Stdout.line! "Successfully parsed! Here's what I got:"
    try Stdout.line! ""
    try Stdout.line! (Inspect.toStr data)

    Ok {}

You'll see that there's an extra argument, Arg.to_os_raw. Cli.parse_or_display_message now takes a generic List arg for the args, and that third function is something that converts such a generic arg to a [Unix (List U8), Windows (List U16)]. Arg.to_os_raw is provided by basic-cli in the Arg module, but you could use any such function if you have a different arg type.

This new argument should be temporary, as with the accepted static dispatch proposal we should be able to infer a method to do this conversion.

NOTE: basic-cli only just merged the PR for OS-aware Args, so you'll need to wait a few days for you to upgrade to using this Weaver version until basic-cli at least makes a pre-release.

CAUTION: Str and Num * arguments can now fail decoding if the provided command line args are not valid UTF-8.

Two new Opt and Param types: Arg and List U8 (bytes)

Now that Args are represented in a variable encoding, there are two new groups of methods available for both Opt and Param:

The same APIs are available for Param. These arg types are a great way to support reading file names might not be UTF-8 encoded.

view this post on Zulip Anthony Bullard (Dec 22 2024 at 01:59):

Great job Luke!

view this post on Zulip Sam Mohr (Dec 22 2024 at 02:00):

Yes, thanks to @Luke Boswell for implementing the basic-cli side of things, and to the team for helping design this!

view this post on Zulip Isaac Van Doren (Dec 22 2024 at 13:59):

Thanks for your work on this Sam!


Last updated: Jul 06 2025 at 12:14 UTC