Ethan Resnick (May 29 2024 at 00:12): Ethan Resnick (May 29 2024 at 00:12):Hey all, this isn't so much an idea, but a question: what's the planned story for private packages in Roc (i.e., packages that an organization wants to publish only for use by people within their organization)? I can imagine a few directions here, but was curious if this is something that's already been thought about
Notification Bot (May 29 2024 at 00:12):A message was moved here from #ideas > basic-webserver configuration by Ethan Resnick.
Luke Boswell (May 29 2024 at 00:21):There is a design for a centralised package index that Richard has been working on. That would be for all public packages at least.
On a slightly related tangent... I would love to see a world where library authors can publish a package on the roc index and somehow get paid for their work. Maybe a private package that requires a subscription or licence maybe, or even sponsorships or donations.
I think it could be a great way to ensure the roc foundation is funded long term, by taking a small percentage of sales. By having a non-profit provide the service, it can be driven by the values of the foundation and community and not purely profits.
... but this is just me spouting ideas without having really thought about the topic. It kind of feels like what Steam is for the gaming community, where Gabe doesn't seem interested in squeezing all the profits out of the community.
On a technical level... you can use a URL release today behind a firewall or on a VPN if you want to prevent others from accessing it.
Luke Boswell (May 29 2024 at 00:23):It would be really healthy for the ecosystem if it was possible for people to make a living providing high quality packages.
Luke Boswell (May 29 2024 at 00:29):I think the roc cli only supports downloading from URLs that are https and without any kind of authentication. I could imagine we may want to support being able to use tokens or basic authentication or maybe SFTP options.
@Ethan Resnick do you have much experience with enterprise requirements for private packages? I'm interested to know what directions you have in mind.
Ethan Resnick (May 29 2024 at 03:58):@Luke Boswell Having paid packages, or offering a paid service to host private packages, both sound like interesting/useful possibilities, and like potential ways to help fund Roc. But that's a much, much larger social conversation.
From a purely technical POV, I assumed that, like you said, using a firewall/proxy or VPN is how it would have to work today, but I think that's not great DX. In the large enterprise context, a proxy or VPN will already be set up and will be the desired way to work. But lots of smaller orgs I've worked at don't have all their devs working through a VPN all the time (and these VPNs are surprisingly finicky to set up in my experience, with lots of subtle incompatibilities between VPN clients and OpenVPN server versions, and weirdness with DNS resolution etc, especially in various split DNS scenarios). If paid, private packages might even be used in simple hobby projects, I think requiring a VPN to access them would make for a pretty bad experience.
So, to me, that all suggests the need for some sort of auth system like you suggested (a token or basic auth or whatever). The token presumably can't live hardcoded in the URL (as committing tokens to source control isn't a good idea). So, I'm imagining the credentials living in an env var or command line flag or something (with one credential per origin). The applicable token would be sent when the package is first downloaded. Presumably, this download will always happen during a build step (except in local dev) and then the package will be cached or compiled into the final executable, so that the token doesn't have to be present in production.
Richard Feldman (May 29 2024 at 06:25):I personally don't like the idea of charging for private packages
Richard Feldman (May 29 2024 at 06:25):I'm also personally skeptical that private package repos are a good idea though :laughing:
Ethan Resnick (May 31 2024 at 02:57):@Richard Feldman What’s the issue with private packages? The vast majority of orgs I’ve worked at have used them for some code sharing/reuse use case or another
Hristo (May 31 2024 at 07:58):@Ethan Resnick Roc does support private packages in a sense - those are local packages. They can be imported via relative paths (and one could imagine scenarios where the limits aren't the local machine).
When a package manager gets introduced, I suppose it'd still make it possible for local packages to be installable and manageable.
Richard Feldman (May 31 2024 at 13:04):Ethan Resnick said:
Richard Feldman What’s the issue with private packages? The vast majority of orgs I’ve worked at have used them for some code sharing/reuse use case or another
it feels like a solution to the self-inflicted problem of using multiple code repositories across the organization instead of one
Richard Feldman (May 31 2024 at 13:05):in other words, if you have a monorepo then I don't know why you'd need a private package repo instead of reading the package sources from normal folders on the disk
Richard Feldman (May 31 2024 at 13:06):and if you don't have a monorepo, then it feels like private packages help delay fixing that mistake :stuck_out_tongue:
Richard Feldman (May 31 2024 at 13:08):all that said, I appreciate that others disagree with me about whether monorepos are the best way to organize code across an organization (or they agree but can't realistically change how things are organized), so I think having some way to do private package repos should be a part of the design for how Roc does package repos
Richard Feldman (May 31 2024 at 13:08):but I don't want to encourage it, and I definitely don't want to charge for it
Richard Feldman (May 31 2024 at 13:10):that would feel to me like a conflict of interest: on the one hand it's not something I genuinely think people should do, but on the other hand if I want people to pay for it, then I'm incentivized to recommend it when otherwise I'd recommend against it :big_smile:
Hristo (May 31 2024 at 13:30):Yes, indeed, paid packages sounds like a distinct matter. In my opinion it somehow also feels like a huge red flag, when it's considered within the context of a growing ecosystem. Something paid and/or developed behind closed doors encourages all kinds of the wrong incentives, including but not limited to monopolism.
In open source development, there exist well established funding streams and sources which do enable proper alignment of incentives, whilst ensuring that development remains in the open.
In a paid ecosystem, you get Windows (prior to versions 10 and 11), i.e., lots of figurative blue screens.
Ethan Resnick (May 31 2024 at 20:44):@Richard Feldman I think it's totally fair to not want to encourage private packages (or paid packages), but I'm glad you're open to supporting them, because I do think they're necessary sometimes — even though I'm generally a huge monorepo fan.
Here's the use case I'm thinking of:
I've worked at companies where some code contains very sensitive intellectual property (e.g., proprietary trading models at a financial services company), and it's critical that only a handful of employees can see this code. I don't know of a way to have a monorepo that limits certain developers' ability to see individual directories. So, I've been at companies that have a monorepo for much of their code, but nevertheless break off these sensitive parts into separate repos. Those repos still need to reuse a lot of the organization's common libraries, though, and private packages make that possible.
Moreover, even if the version control system has a way to limit developers ability to see certain directories in a repo, I think you'd still need private packages for the situation above. The reason is that, if a developer made a breaking change to one of the org's internal libraries, they wouldn't be able to see whether that change breaks the usage of the library in the directories that they don't have access to. Therefore, these sensitive directories need the ability to update to a new version of the org's internal libraries at their own pace (with the upgrades being done by someone on the team that can see the sensitive code). Private packages, with versions, again enable that.
If you know a way to address this use case without private packages, though, I'm definitely interested!
Agus Zubiaga (Jun 04 2024 at 13:56):Would private packages really protect IP? You’re still shipping the code to everyone who needs to build the app
Ethan Resnick (Jun 04 2024 at 15:50):Sorry if I wasn’t clear: the private packages hold code that’s not especially sensitive (ie, all employees at the company can see it, even if it shouldn’t be visible publicly). However, that code needs to be in a package so that it can be imported by code that has to live in a different repo because it is sensitive and limited to certain employees
Hristo (Jun 04 2024 at 17:31):@Ethan Resnick, my apologies I seem to be missing something - could you please clarify why this is impossible to be achieved via relative package imports? Thanks!
Ethan Resnick (Jun 06 2024 at 02:31):@Hristo I thought the reasoning was pretty clear, so now I'm wondering if maybe I'm missing something about the capabilities of relative package imports work. Can you explain how you would handle this with relative package imports (when some consumers of the package are in a separate repo and need to be able to update their version of the package on a separate cadence, but the package code still shouldn't be public to the world)?
Brendan Hansknecht (Jun 06 2024 at 02:40):I mean you could vendor the rest of the code into the secret repo
Brendan Hansknecht (Jun 06 2024 at 02:40):Then relative import
Brendan Hansknecht (Jun 06 2024 at 02:41):That would keep all code private to the company and the super secret code extra protected
Hristo (Jun 06 2024 at 10:13):Yes, exactly. I was referring to something along the lines of what @Brendan Hansknecht said.
@Ethan Resnick a good analogy (albeit not perfect, but a perfect analogy isn't necessary in this case) here is how one is able to install a Python package from a local directory/repository via pip install . or how one is able to build a C or C++ shared library from a local directory/repository and then utilise it as necessary. Roc allows importing packages by their relative paths, with respect to the local machine (and this could be arbitrarily expanded, I imagine, via aliases, virtual file systems etc), in addition to specifying remote packages via their full URL. The former effectively fulfils the requirements, as discussed in this thread - that's been my understanding at least.
If I understand correctly, in your most recent message, there's a requirement that the dependent repos need to have access to arbitrary versions of a local package. If the requirement is it use a sub-repository, then this could perhaps be alternatively achieved via a manually triggered script or some sort of a git hook (depending on the specifics of the requirements) that'd check out a specific commit or a tag from the package repo (of the package which is the dependency) into a local directory (eg, .cache) which could then be added to the local .gitignore etc. I think this is more of a general version-management consideration that's not specific to Roc per se, but I might've wildly misunderstood the use-case.
Last updated: Jun 16 2026 at 16:19 UTC