module and package changes · ideas

Stream: ideas

Topic: module and package changes

Richard Feldman (Jul 03 2023 at 20:11):

here's a proposal for some changes to how modules and packages work: https://docs.google.com/document/d/1E_77fO-44BtoBtXoVeWyGh1xN2KRTWTu8q6i25RNNx0/edit?usp=sharing

any thoughts and feedback welcome!

Ajai Nelson (Jul 04 2023 at 02:40):

I like the idea of submodules being private. Can a submodule access siblings of its parent? For example, say we have Foo.roc, Bar.roc, and Bar/InsideBar.roc. Then Foo can't import InsideBar. But can InsideBar import Foo?

Richard Feldman (Jul 04 2023 at 10:25):

:thinking: I hadn't considered that scenario...what are your thoughts?

Anton (Jul 04 2023 at 10:40):

I would prefer all the headers a little different, I'll make new topics for each one.

Sky Rose (Jul 04 2023 at 13:10):

In the app header, removing the explicit reference to the platform makes it harder to know what the platform is. That's a really important part of an app, and I expect to often look at a random app on github and want to know what the platform is. This proposal makes it so the only way to do that is by reading through the (potentially long) list of packages and knowing which one is a platform.

Making it impossible to import multiple platforms may be limiting. Maybe there's a feature-rich-cli platform with lots of useful libraries included. My app uses custom-barebones-cli as its platform, which is based on feature-rich-cli and very similar. So I want to reference feature-rich-cli to use it as a library but not as a platform. Enforcing one platform in packages makes that impossible.

What if app used the platform keyword. Like (very roughly, this won't work as is)

app [main]
  platform Cli
  packages [Stdout from "https://…" as Cli]

app [main] packages [
  platform Stdout from "https://…"
]

Sky Rose (Jul 04 2023 at 13:17):

Does importing from a private module require 2 imports? One to import the module, and one to import the value from the module?

Sky Rose (Jul 04 2023 at 13:31):

If you're making a large app (say 100,000 lines), what's the recommended way to organize it now? Many private packages? A bunch of modules?

Brendan Hansknecht (Jul 04 2023 at 15:49):

Ajai Nelson said:

I like the idea of submodules being private. Can a submodule access siblings of its parent? For example, say we have Foo.roc, Bar.roc, and Bar/InsideBar.roc. Then Foo can't import InsideBar. But can InsideBar import Foo?

I think this should match the package proposal where for InsideBar to import Foo, it must import it from Bar, which means Bar must import Foo. I think that would help avoid accidental circular dependencies and make the import allowance/chain clear.

Ajai Nelson (Jul 04 2023 at 16:19):

Sky Rose said:

If you're making a large app (say 100,000 lines), what's the recommended way to organize it now? Many private packages? A bunch of modules?

Yeah, that's sort of what I was starting to get at. Is the primary difference now between a package and a module just that a package allows you to export submodules but a module doesn't? Because I'm wondering if we could simplify things by combining the concepts of a package and a module.

Brendan Hansknecht (Jul 04 2023 at 16:32):

Yeah, my personal preference would be that the only difference is how they are imported. A module would be imported via the hierarchy/tree structure that modules create. A package would be explicitly imported via url or path. It would then contain modules.

In my head this would mean that all modules not in the same folder are private by default and must be explicitly exposed.

Richard Feldman (Jul 04 2023 at 17:38):

:thinking: what would the tradeoffs be between packages and "modules can selectively expose their submodules"?

Richard Feldman (Jul 04 2023 at 17:42):

Sky Rose said:

What if app used the platform keyword?

interesting! Given that in this design we import modules from platforms directly, one idea would be to have the same syntax as packages (or whatever we end up calling it) except there's only one entry, e.g.

app [main]
    platform [Module1, Module2] from "https://..."
    packages [ ... ]

Brendan Hansknecht (Jul 04 2023 at 19:22):

:thinking: what would the tradeoffs be between packages and "modules can selectively expose their submodules"?

Packages are for distribution, sharing between multiple projects, or very clear unit separation.

Modules are just your generic unit of encapsulation and grouping.

Richard Feldman (Jul 04 2023 at 19:29):

hm, what's the difference between "very clear unit separation" and "unit of encapsulation and grouping"?

Richard Feldman (Jul 04 2023 at 19:39):

some considerations regarding allowing modules to selectively expose their submodules:

modules have uppercase names (e.g. Http) which means we'd need to syntactically disambiguate how to expose modules from how to expose types (e.g. if a module says it exposes [Foo], does that mean a submodule or a type named Foo? One possible answer, supposing the module is named Blah it could be [Foo from Blah] which means "I am module Blah and I am exposing my submodule named Foo" - and then [Foo] would mean "I am exposing the type or ability named Foo)
in the current proposal, packages can be imported from URLs and filesystem paths, whereas individual modules cannot; if we replaced packages with nested modules, then any import could be used to bring in something from a separate URL, including imports used inside expressions. Languages with static imports typically don't allow that.
in the current proposal, when importing multiple modules from a package, they can be listed in a comma-delimited list because packages don't have an outermost module name. In contrast, a nested module (like let's say Http is a nested module for example) does have an outermost name, so if I want to import Http from a URL and then import some of its exposed submodules, to maintain that style there would need to be some separate syntax to say "I want to import these submodules from here without the wrapper module"

Brendan Hansknecht (Jul 05 2023 at 15:18):

what's the difference between "very clear unit separation" and "unit of encapsulation and grouping"?

That is just my wording not being clear. Here is a better attempt at a definition:

Packages are modules that are meant to be shared and can have url and/or path dependencies. They can only be imported via url or path dependencies. So really, they are about packaging, depending on, and sharing chunks of code.

which means we'd need to syntactically disambiguate how to expose modules from how to expose types

I'm not sure they do. I think it just needs to be a naming collision bug. If you have a type named Foo, you can't have an imported module name Foo as well. So if you import Foo and then expose Foo, you are exposing a sub-module. If you define Foo and then expose Foo, you are exposing a type. I do agree it could be nice to visually distinguish them, but I don't think it would much of an issue to have the overlap. I think the overlap is common.

then any import could be used to bring in something from a separate URL, including imports used inside expressions.

I guess I wasn't fully clear. packages are still special in that they are also the only location that you can import from a url or path. It is just that a package is a module with a bit of extra information and functionality.

"I want to import these submodules from here without the wrapper module"

I think that is an orthogonal problem. I think if we have nested submodules that can be exposed, we want that functionality anyways.

import Foo.Bar.Baz.{ Something, someFunc } likely should not bring Foo and Bar into scope. (maybe not Baz as well).

Fábio Beirão (Jul 10 2023 at 14:55):

I'm still catching up with everything that has happened in Zulip :). From glancing at this proposal, I am a bit confused at the different syntaxes for stating "a list of things".
Specifically, for the packages proposal, there seem to be two syntaxes for "a list of things". Perhaps this is just a syntactic nitpick but I do remember that this has tripped me up as a beginner in other languages (like F#). Here is what I mean:
image.png
image.png

Idea: imports should always be lists, even if it is just one single thing that is being imported. So for the example above, it would become

package [ParserCore, ParserCsv, ParserStr]
    packages [
        [ JsonDecode ] from "https://.../json/...",
        [ CodePt, Segment ] from "https://.../unicode/...",
    ]

This would make it clear to the user how to go from "I imported one thing, but now I need to import 2".
We could leverage roc format to add the missing brackets if the user didn't specify them.

Richard Feldman (Jul 10 2023 at 16:40):

yeah I thought that looked a bit weird when there's just 1 package because of the [[ but I guess in practice it'll almost always be multiline like this anyway.

I'm open to this especially if we change app to have a platform option, because then hello world wouldn't be affected

Richard Feldman (Jul 10 2023 at 16:40):

what do others think?

Richard Feldman (Aug 05 2023 at 03:04):

here's the thread for the follow-up proposal (the one I referred to as WIP earlier; it is no longer WIP!)

https://roc.zulipchat.com/#narrow/stream/304641-ideas/topic/module.20params

Richard Feldman (Aug 15 2023 at 12:23):

I just realized a problem with getting rid of the package shorthand: if importing a package makes their modules get imported unqualified, then the following situation can come up

I'm using import Foo to import Foo.roc file from my filesystem
later, someone else on my team adds a package dependency and brings Foo into scope
now my import Foo silently does something else, and they probably don't realize it because it's distant in the code base

a partial fix for this would be to check every import that resolves to a module from a package to see if there happens to be a .roc file on the filesystem with the same name, but that's very expensive for the compiler to do - could result in a huge amount of syscalls that 99.99% of the time will just say "yup, no file there" - and we'd have to do them even when reading from the cache, because the cache can't know the state of the local filesystem

Richard Feldman (Aug 15 2023 at 12:23):

but that fix is still a bad UX because it means you import a package and cause an error that has to be fixed somehow

Richard Feldman (Aug 15 2023 at 12:23):

so I'm inclined to drop that part of the proposal and have it be import pf.Foo like today

Richard Feldman (Aug 15 2023 at 12:23):

that way it's always unambiguous which package (if any) the module is being imported from

Richard Feldman (Aug 15 2023 at 12:27):

so then it would be:

app [main]
    imports {
        pf: platform "https://…/basic-cli/…",
        json: "https://…/json/…",
        uc: "https://…/unicode/…",
    }

and hello world would be:

app [main] imports { pf: platform "https://…/basic-cli/…" }

import pf.Stdout

main =
    Stdout.line "Hello, World!"

Richard Feldman (Aug 15 2023 at 12:28):

I guess imports doesn't make sense as a module header keyword in that design, since it's not actually bringing modules into scope :thinking:

Richard Feldman (Aug 15 2023 at 12:29):

I think someone suggested depends at some point?

app [main] depends { pf: platform "https://…/basic-cli/…" }

import pf.Stdout

main =
    Stdout.line "Hello, World!"

although you also depend on local modules you import :sweat_smile:

Richard Feldman (Aug 15 2023 at 12:29):

I don't think we've considered "loads" before

app [main] loads { pf: platform "https://…/basic-cli/…" }

import pf.Stdout

main =
    Stdout.line "Hello, World!"

Richard Feldman (Aug 15 2023 at 12:29):

or "includes"

app [main] includes { pf: platform "https://…/basic-cli/…" }

import pf.Stdout

main =
    Stdout.line "Hello, World!"

Richard Feldman (Aug 15 2023 at 12:30):

actually maybe "packages" as a verb is appropriate in this design? :thinking:

app [main] packages { pf: platform "https://…/basic-cli/…" }

import pf.Stdout

main =
    Stdout.line "Hello, World!"

Brian Carroll (Aug 15 2023 at 19:30):

requires?

app [main] requires { pf: platform "https://…/basic-cli/…" }

import pf.Stdout

main =
    Stdout.line "Hello, World!"

Fábio Beirão (Aug 16 2023 at 13:04):

uses ?

Richard Feldman (Aug 17 2023 at 18:25):

I like requires!

Agus Zubiaga (Dec 06 2023 at 04:20):

@Richard Feldman I'll to start working on the new app headers soon. Is this want we wanted to go with for specifying packages? I want to make sure I'm not missing a newer decision

Agus Zubiaga (Dec 06 2023 at 04:35):

Also, I believe if we go with requires for packages, we might have to change the platform's header which also uses requires for something else: Proposed platform changes.

Richard Feldman (Dec 06 2023 at 11:48):

I think it's ok if they both use requires for different things

Richard Feldman (Dec 06 2023 at 11:48):

we can change it if it's confusing in practice

Agus Zubiaga (Dec 06 2023 at 11:58):

Oh ok. So platform package would use packages?

Kevin Gillette (Jan 01 2024 at 16:41):

From the Module and package privacy changes section of the doc

The rule would be that from Foo could only be used to import modules like this from within the Foo module itself.

You can tell syntactically that it's importing a module rather than types/values/etc because there are no braces after import

It's not clear to me what this means, and what some of the context of this scenario is. Can you give examples with and without braces? I had inferred, earlier in the doc, that [curly] braces in imports were being eliminated.

Is the module author decomposing a single Roc file into multiple within the same directory, yet importing and re-exposing the new files' functionality from the original file for compatibility?

If so, can anyone importing the original also just import those newer modules directly? At least in other languages, that would sometimes be desirable and sometimes not (depending on whether the split provides the better public API, or whether the split is only for internal organization). Is there a way to have an enforced internal/private module?

Agus Zubiaga (Apr 03 2024 at 23:09):

About the private submodules described here.

What if instead of doing this?

# Somewhere in Foo.roc

import Foo exposing Bar
import Bar exposing [Type1, Type2, val1]

We had something like this:

import .Bar exposing [Type1, Type2, val1]

Agus Zubiaga (Apr 03 2024 at 23:11):

The original feels a little weird to me. It uses the import keyword but as described it's more like it allows it to be imported, and it's unclear whether the first line is enough to use the qualified Bar name in expressions.

Agus Zubiaga (Apr 03 2024 at 23:12):

Having a specific syntax for private submodules names like .Bar seems more intentional, and would allow the same name to be used at different levels, which might be desirable.

Agus Zubiaga (Apr 03 2024 at 23:14):

We could also go with:

import Foo.Bar exposing [Type1, Type2, val1]

but I think beginners would be surprised to find you can only do this from Foo

Last updated: Jun 16 2026 at 16:19 UTC