Stream: ideas

Topic: Package info

Martin Stewart (Jul 14 2022 at 13:07):

Georges Boris said:

kinda drifting the topic but in response to you Martin Stewart - it would be awesome if Roc's package system contained more information than Elm's... having multiple published packages is a much smaller problem when you have more info available "what is the most used? what is the one with most recent adoption?" etc.

Richard Feldman said:

this is probably worth a separate thread, but I think "what is the most used?" and "what is the one with the most recent adoption?" are problems that haven't been reliably solved in any language :sweat_smile:

Indeed, new topic time :smile:

I think the best way to help the user with picking the best package when there are multiple to choose from is just have a isGood flag that some trusted people can set for hosted packages.

This isn't feasible if the ecosystem is really big, but at least in the beginning I think this is a good approach.

Richard Feldman (Jul 14 2022 at 13:18):

an example of why "number of times package has been downloaded" doesn't actually tell you how many people are using it: https://mobile.twitter.com/hillelogram/status/1546894728800735237

Whenever software packages brag about being downloaded millions of times a week, keep in mind that's mostly automated builds. If you deploy an app to a hundred docker containers then every one of your dependencies gets 100 more downloads that week.

- Hillel (@hillelogram)

Richard Feldman (Jul 14 2022 at 13:23):

(Hacker News/Reddit comment voice) "Roc is run by an unelected cabal of Blessed Ones who literally decide whether or not someone's open source contribution that they made to the Roc ecosystem isGood or not. It's a walled garden and if you aren't friends with the inner circle, you have no chance of making something that gets popular. This happened to me personally with a package I spent thousands of hours making..."

Anton (Jul 14 2022 at 13:24):

Github can do some decent usage tracking, don't they already do this for some languages? This project is used by all these projects...

Martin Stewart (Jul 14 2022 at 13:27):

Richard Feldman said:

(Hacker News/Reddit comment voice) "Roc is run by an unelected cabal of Blessed Ones who literally decide whether or not someone's open source contribution that they made to the Roc ecosystem isGood or not. It's a walled garden and if you aren't friends with the inner circle, you have no chance of making something that gets popular. This happened to me personally with a package I spent thousands of hours making..."

I wouldn't let cynical hacker news posters affect decision making if I were you :stuck_out_tongue:

Martin Stewart (Jul 14 2022 at 13:29):

Also if it really does cause people to distrust the people assigning isGood then you could remove it. But I still think it's worth trying and seeing if it's an issue.

Martin Stewart (Jul 14 2022 at 13:30):

Or you could go the Elm route and push packages to the top of the search results based on how many talks the owner has given at conferences about the language :smiley:

Richard Feldman (Jul 14 2022 at 13:31):

yeah Elm's approach uses an objective metric, which is an important distinction

Anton (Jul 14 2022 at 13:32):

I can definitely see how users would like to know if a package is good but it doesn't seem easy to decide when that is. You would also need to be transparent about it and make sure the transparent rules do not get gamed.

Martin Stewart (Jul 14 2022 at 13:35):

Richard Feldman said:

yeah Elm's approach uses an objective metric, which is an important distinction

It's on a per-user basis though. So you can't mark a particular package as isGood that should be the standard for managing color or something without also promoting a bunch of less interesting packages made by the same person.

Richard Feldman (Jul 14 2022 at 13:36):

Github can do some decent usage tracking, don't they already do this for some languages? This project is used by all these projects...

so whatever criterion you use to make packages appear higher in the search rankings, it creates an incentive for people to game that metric so their package will appear higher. It's essentially SEO but for packages.

if the metric is "total downloads," that creates an incentive for people to induce more downloads of the package to happen (not necessarily by a script, maybe just by encouraging other package authors to take it on as a dependency, even if it's unnecessarily, because that will boost its download numbers).

if the metric is "is used by all these projects" then it creates an incentive to pester other package authors to add it as a dependency, or just to make a ring of projects whose main purpose is to depend on each other and make each other rise higher in search results

Richard Feldman (Jul 14 2022 at 13:37):

as an aside, is there any precedent for any package ecosystem having the characteristic that there's a set of userspace packages that came out early and everyone uses them? (like the email parsing example)

Richard Feldman (Jul 14 2022 at 13:38):

the closest I can think of is Elm having packages that aren't purely userspace (e.g. elm/http can run kernel code and other HTTP alternatives can't; the closest they can do is to build on elm/http)

jan kili (Jul 14 2022 at 13:40):

We don't want to stagnate userland with blessings that lag behind innovation!

Martin Stewart (Jul 14 2022 at 13:42):

Richard Feldman said:

as an aside, is there any precedent for any package ecosystem having the characteristic that there's a set of userspace packages that came out early and everyone uses them? (like the email parsing example)

Elm has elm-ui and elm-geometry. JS has JQuery.

Richard Feldman (Jul 14 2022 at 13:44):

hm, I thought the original premise was that the Elm package ecosystem isn't an example of the goal :sweat_smile:

Richard Feldman (Jul 14 2022 at 13:44):

because of the multiple email parsing packages etc

Martin Stewart (Jul 14 2022 at 13:46):

I used the email example from the Elm ecosystem because that's what I'm familiar with but overall I'd say the Elm ecosystem is pretty good about having a single good package for solving a particular issue. Just email, time, and a few other use cases stand out as having many packages that aren't great.

Richard Feldman (Jul 14 2022 at 13:46):

for reference:
Martin Stewart said:

I think it's really valuable for an ecosystem when there are single, well written, packages to address particular needs. For example, a single package for managing colors that all other packages build upon, rather than several color packages with inconsistent feature sets and have opaque types you need to convert between because PackageA uses ColorPackageA and PackageB uses ColorPackageB.

Edit: Here's another example from the Elm community image.png. And to make it worse, I have a package for sending emails that uses its own implementation because none of these 3 suit my needs. This is the sort of situation want to want avoid in Roc :sweat_smile:

Richard Feldman (Jul 14 2022 at 13:47):

ahh gotcha

Richard Feldman (Jul 14 2022 at 13:47):

so the point is wanting to minimize occurrences of that happening

Richard Feldman (Jul 14 2022 at 13:50):

btw regarding:

I wouldn't let cynical hacker news posters affect decision making if I were you :stuck_out_tongue:
Also if it really does cause people to distrust the people assigning isGood then you could remove it. But I still think it's worth trying and seeing if it's an issue.

unfortunately, HN/Reddit posters (regardless of whether their posts are accurate, fair, etc.) are a very real part of how people perceive the language and decide whether or not it's worth their time to get involved in

Martin Stewart (Jul 14 2022 at 13:51):

JanCVanB said:

We don't want to stagnate userland with blessings that lag behind innovation!

I agree! But I don't think promoting some packages has to conflict with that. For one thing I think we should promote the idea of people contributing to existing packages when possible. And promoting packages should be done for stuff that is unlikely to see lots of innovation (maybe isGood is a bad name, how about isStandard?). Things like nonempty-list, nonempty-dict, and nonempty-set are all packages that I'd like to see exactly one of that everyone uses.

Richard Feldman (Jul 14 2022 at 13:53):

for example, I was at a conference once talking with a bunch of people who aren't Web developers, and their impression of Elm was that people are constantly getting banned from public forums for expressing dissenting views. In reality, the average number of bans was between 0 and 1 per year, and the bans were for breaking forum rules, but the meme took off on HN and that became other peoples' impressions of the language.

Ed Kelly (Jul 14 2022 at 13:56):

if the metric is "is used by all these projects" then it creates an incentive to pester other package authors to add it as a dependency, or just to make a ring of projects whose main purpose is to depend on each other and make each other rise higher in search results

I feel like malevolent/self-aggrandizing package authors certainly _could_ do these things, and if Roc ever gets as big as JavaScript or Python, then the possibility of this happening approaches 100%. But up until that point, "how many other packages in the registry depend on this package" seems like quite a useful metric to me, at least for questions like "which of these colour/date data structures should I use if I want something that is likely to be compatible with most other packages in the ecosystem?" I wonder if there could be a way to combine this type of metric with some other safeguards (isGood or whatever) to mitigate the risk of bad behaviour and still take advantage of the wisdom of the crowd.

Richard Feldman (Jul 14 2022 at 13:56):

so while I have absolutely no intention of letting potential HN comments drive design decisions, I'm also not going to engage in the wishful thinking that there would be no significant consequences for Roc if I were to discount likely HN comments altogether :big_smile:

jan kili (Jul 14 2022 at 13:56):

I see most libraries as following an arc of usefulness/popularity/support: early innovation, then hopefully stable growth, then hopefully standardization, then hopefully maintenance. Deviations from that path usually feel like disappointments. Perhaps instead of isGood, we can mark milestones at which packages satisfy a certain level of maturity?

jan kili (Jul 14 2022 at 13:59):

@Ed Kelly I wonder if an algorithm like PageRank would be useful for that.

Richard Feldman (Jul 14 2022 at 14:00):

I think there might be some precedent for people gaming PageRank and getting lower-quality content to rise up the search results rankings... :wink:

jan kili (Jul 14 2022 at 14:03):

Oh, totally, but as a naive implementation

jan kili (Jul 14 2022 at 14:04):

There's also feedback between the metrics. If someone games their dependedOnBy score and other metrics for attention, their user submitted ratings should drop.

Brendan Hansknecht (Jul 14 2022 at 14:26):

I think that roc has an extra complexity here compared to elm for example. Due to the varied use cases and platforms, I think roc is much much more likely to have many different sub communities that would not want the same packages. Some times there would be overlap, but many packages would be better if you get the tailored version for a specific sub community.

Brendan Hansknecht (Jul 14 2022 at 14:27):

I think trying to promote the 1 good x type of package would in general be a mistake

Brendan Hansknecht (Jul 14 2022 at 14:28):

I also think that will make ranking more complex because there is probably a good chance that you will want the ranking to be able to take the platform or tags on the platform into consideration.

Brendan Hansknecht (Jul 14 2022 at 14:28):

So that is a whole extra set of relations to record

Brendan Hansknecht (Jul 14 2022 at 14:34):

@JanCVanB if we tried to do something like page rank, what would be our "link"?

jan kili (Jul 14 2022 at 19:32):

An import? I haven't thought it through...

jan kili (Jul 14 2022 at 19:34):

But I did just realize that this idea of "approved" standards can be crowdsourced a bit by encouraging some kind of curation - for example, Parsing Metalibrary X that simply imports high-quality parsers from other libraries into a "good" set of tools. Then, new developers can choose a metalibrary based on popularity or style or subcommunity, allowing them to get curation without the language devs themselves blessing winners.

Brendan Hansknecht (Jul 14 2022 at 20:56):

Thats an interesting idea.

Also, the reason I ask for page rank is that it is likely easily game-able. For example. If you go by imports. Some left pad library that transitively imported everywhere would have a giganitic page rank score. As such, it can choose to import any random library and cause it to instantly have a huge score.

Brendan Hansknecht (Jul 14 2022 at 20:56):

But maybe that wouldn't be a major issue in practice.

Brendan Hansknecht (Jul 14 2022 at 20:57):

Though also with imports you have to scan random user land projects to see what they import. So might be hard to record.

jan kili (Jul 14 2022 at 21:16):

In defense of that "gaming", if half of the internet depends on the unpaid unseen unappreciated work of one person, I care what kind of coffee they drink - even if their library is only one line long. If they're a vertebra in the spine of the Roc ecosystem, they deserve to be somewhat of a tastemaker, too.

jan kili (Jul 14 2022 at 21:17):

That said, I hope that Roclibland doesn't have many one-line hyperused libraries.

Brendan Hansknecht (Jul 14 2022 at 21:21):

Yeah, thats a really fair defense for this type of potential gaming. And yeah, I don't expect a left pad equivalent, but was trying to make a point.

jan kili (Jul 14 2022 at 21:23):

That "that said" said, I often undersynpathize with npm overcomplication complaints and JavaScript framework fatigue, because they seem like good problems to have. What a healthy ecosystem, to have too many good options to choose from! There's something to be said for allowing room for gray area, so that developers either need to do their own research before choosing or just admit that they're happy to pick one at random and see how it goes.

jan kili (Jul 14 2022 at 21:30):

I'm reminded somehow of ESLint configurations, where it's fine to just pick a popular config and move on - your code will be more readable than formatting manually. The choice becomes "do you want your code to look like Google's or Sindre Sorhus's?" which is an easy choice for most, whichever direction they lean. Therefore, I hope to someday see Google and Sindre Sorhus make metalibrary repos named like roc-parsers-google and unicorn-parsing where they maintain their preferred blend of parsers for email addresses, HTML, etc.

jan kili (Jul 14 2022 at 21:32):

(I don't think parsers are a good example for crowdsourced library curation, but I feel like there are good use cases for that approach)

Qqwy / Marten (Jul 15 2022 at 10:17):

This is very interesting and important to think about.

In the Elixir community, people consider from time to time time why Elixir's library ecosystem has matured in the way it has. Namely: There is virtually always 'one' big package for a particular task and it is able to fulfill this role without being 'blessed' by the language core team or anything. From time to time, some of them will be challenged by a new approach (c.f. JSON parsers, low-level HTTP handlers, background job processing) but it is usually very clear what the differences and similarities between them are, and how to swap one of them out for the other. (A replacement package is often made with API compatibility in mind).

Maybe part of it is that the package manager itself is searchable and that virtually all packages have stellar documentation (as this is valued highly in the community) which itself is also searchable. This decreases the chance of someone re-inventing an existing package. (Elm's current package management is not searchable and my educated guess is that this might be a cause of more semi-duplicate packages existing.)

Another reason which I've heard many people consider as probable factor, is that the community grew at a steady pace (rather than a single sudden hype-driven explosion), and that many programmers come to the language after already knowing something else (so the average 'maturity' --however you want to define it-- might be somewhat bigger than in, say, JavaScript or Python land).

Last updated: Jun 16 2026 at 16:19 UTC