Richard Feldman (May 11 2023 at 03:03): Richard Feldman (May 11 2023 at 03:03):I just realized the "AWS temporary credentials that automatically expire and must be renewed periodically, and the way you usually find that out is that your request to e.g. S3 fails and you have to go make a separate request for new temporary credentials" has similarities to the random number seed
Richard Feldman (May 11 2023 at 03:04):in that when you make a call to S3, and it's a Task which automatically takes care of the credential-refreshing if necessary (that is, if the request you actually care about fails because of expired temporary credentials), then it will give you back (Credentials, ThingYouActuallyWanted)
Richard Feldman (May 11 2023 at 03:04):much like how stepping a random number generator will give you back a seed in addition to the thing you want
Richard Feldman (May 11 2023 at 03:05):and also similarly, you always want to use the most recent Credentials you received from AWS
Agus Zubiaga (May 11 2023 at 03:05):Yeah, you can also use something like the generator pattern for that
Richard Feldman (May 11 2023 at 03:05):yeah
Agus Zubiaga (May 11 2023 at 03:13):Although, I would expect Task composition to be even more relevant there
Agus Zubiaga (May 11 2023 at 03:25):When people are writing actual apps though, I don’t think this is gonna be a problem.
For CLI apps, you’re unlikely to have to refresh the token.
For long running programs, like GUI apps, games or servers, you’re probably going to have something like a Model (in TEA) that you can update every time you deal with an event.
Agus Zubiaga (May 11 2023 at 03:27):I don’t think shadowing would help with this because no scope would live for that long, you know?
Agus Zubiaga (May 11 2023 at 03:42):Frameworks have the option to provide a Task wrapper type (that you’d use in e.g. each request handler) which can carry this kind of secondary state seamlessly
Agus Zubiaga (May 11 2023 at 03:50):I did something like this in F# for an old gig and it turned out quite nice
Brendan Hansknecht (May 11 2023 at 04:13):You would need shadowing that is allowed to affect sub scope only for function arguments (specifically for the backpassed variables)
Brendan Hansknecht (May 11 2023 at 04:13):Otherwise, yeah, shadowing wouldn't help due to the new name coming up in a new sub function due to calling a task and making a continuation
Richard Feldman (May 11 2023 at 11:02):Agus Zubiaga said:
Frameworks have the option to provide a
Taskwrapper type (that you’d use in e.g. each request handler) which can carry this kind of secondary state seamlessly
or alternately the platform could offer a Task that can read and write state from an in-memory store (so something like "store the token in Redis" but with less overhead)
Agus Zubiaga (May 11 2023 at 12:04):Yeah! That’d be cool
Brendan Hansknecht (May 11 2023 at 13:52):Why is that better thoigh
Brendan Hansknecht (May 11 2023 at 13:52):Just sounds like an easy way to hide any arbitrary mutation
Brendan Hansknecht (May 11 2023 at 13:54):Like feels like it is just more directly attacking the languages purity
Brendan Hansknecht (May 11 2023 at 13:54):This feels like something that should belong in the apps model
Agus Zubiaga (May 11 2023 at 14:33):The Redis example just sounded interesting to me. However, you don't really need platform help to do what I describe. It's still pure. This just couples an already effectful thing (calling AWS) with some state.
Brendan Hansknecht (May 11 2023 at 14:36):Yep. Though also, I wonder if there is ever a case for 2 AWS tokens (fill in arbitrary web access token for some API) and this type of design is just asking to hit walls/limitations.
Brendan Hansknecht (May 11 2023 at 14:38):Also, if you are storing tokens in the platform by string how do you make sure two different libraries dont overwrite the same token?
Brendan Hansknecht (May 11 2023 at 14:38):Just sounds like a bad thing to hide away in an effect called by a library
Brendan Hansknecht (May 11 2023 at 14:39):Not to mention it would require more platforms to support the arbitrary state storage API, which just doesn't sound great to me.
Agus Zubiaga (May 11 2023 at 14:40):Agreed, that's not great, but that isn't how I was thinking it would work.
I think an AWS package should just expose stateless functions that either return Task or raw data structures.
Then at the app level, you can have a few helpers that allow you to compose Task-like structures which can thread some state.
Agus Zubiaga (May 11 2023 at 14:42):You can do all that without platform support. It's just like the Random.Generator pattern, which I think is what Richard was going for.
Agus Zubiaga (May 11 2023 at 14:47):I definitely wouldn't like packages setting state willy-nilly
Agus Zubiaga (May 11 2023 at 14:48):It's hard to talk about this without a practical example though
Brendan Hansknecht (May 11 2023 at 14:50):Yeah, I was more commenting on Richard's proposals around state.
I do understand your generator idea, though it would make for weird composablity if you have to call multiple apis with different tokens, right?
Agus Zubiaga (May 11 2023 at 14:51):I imagine the state would be a type variable, so I don't think so
Brendan Hansknecht (May 11 2023 at 14:53):But wouldn't each api expect a different state? Cause they want a different type of taken? If it is user specified, the aws library won't know how to extract and use it. Or am I missing something?
Agus Zubiaga (May 11 2023 at 14:55):Yeah, that's why this type would live at the app level. The state would be a record or something that I control as the app author. Most of the time as I'm writing my app's code I don't directly interact with that record, but I have a helper per API that does.
Brendan Hansknecht (May 11 2023 at 14:57):Ah, so you have a helper that goes from (token, data) to updated state and a continuation just on data.
Agus Zubiaga (May 11 2023 at 14:58):Yeah, something like that.
Agus Zubiaga (May 11 2023 at 15:15):The way I understood what Richard said, was this but backed by Redis. I might have misunderstood, though.
Agus Zubiaga (May 11 2023 at 15:17):That can also be done at the app level, though. As long as you have the primitives necessary to talk to Redis and you can serialize your state.
Richard Feldman (May 11 2023 at 15:24):I agree that it's not great; unfortunately, the AWS API design is - as far as I can tell - basically like "this will be nice for our servers, and it will be fine for the client because they can just use the global mutable variables their language of choice provides"
Richard Feldman (May 11 2023 at 15:25):I think their design forces the client to choose between an ergonomic experience and an experience that avoids global mutable state
Richard Feldman (May 11 2023 at 15:27):so the choices are:
Task (this totally works, but unless there's global state somewhere, it means your entire code base now has to translate all tasks into this type, even ones that have nothing to do with AWS or even network requests, just to accommodate this AWS API) Richard Feldman (May 11 2023 at 15:28):I dislike both options, but if I have to use AWS APIs, I think I dislike the second one less :stuck_out_tongue:
Richard Feldman (May 11 2023 at 15:29):because at least then I can use normal Task everywhere instead of having a bunch of MyTask.fromTask calls all over the place just because this is how AWS designed their APIs
Richard Feldman (May 11 2023 at 15:30):which also couldn't benefit from future planned Task-specific compiler error message hints like "hint: should there be a |> Task.await at the end here?"
Richard Feldman (May 11 2023 at 15:34):also when writing tests you'd have to do a bunch of MyTask.toTask and then handling the credential-threading through the simulation tests API (I'm not sure how I would do that, but my assumption is that it's at least possible somehow)
Richard Feldman (May 11 2023 at 15:35):I find it extremely frustrating that this API is a real thing :angry:
Georges Boris (May 11 2023 at 15:36):could you give an example of what AWS expects from clients? why does it necessarily requires mutability?
Georges Boris (May 11 2023 at 15:38):oh - the refresh token thing? like - Task User would need to actually be Task (State -> State, User) because AWS might want to provide an auth token that we refresh as part of another request?
Richard Feldman (May 11 2023 at 15:39):the way it works is:
Richard Feldman (May 11 2023 at 15:41):so if I want to make a request to S3, I can't just authenticate once and get a session token that's valid until it gets explicitly invalidated (or expires after several days or something, at which point erroring out and resetting the entire application state becomes a reasonable choice)
Richard Feldman (May 11 2023 at 15:41):rather, any request I make to S3 must both include a piece of state (the temporary credentials) and also must be able to change that state (updating the temporary credentials if they had to be refreshed, which could happen on absolutely any request to AWS)
Richard Feldman (May 11 2023 at 15:42):because all subsequent requests must use the new credentials; the old ones will always fail
Georges Boris (May 11 2023 at 15:43):wouldn't handling that be an architecture choice of the given app though? like - at work we have something similar but we monitor the expiration date and refresh it before expiration (in Elm this happens through subscriptions)
I could do the same thing by having an API+State wrapper that could handle both retries and caching etc.
Richard Feldman (May 11 2023 at 15:44):that becomes risky if you have a bunch of tasks chained together, which I think will be way more common in Roc than in Elm
Richard Feldman (May 11 2023 at 15:45):unless you have some task-based way to read a fresh state, the entire chain is using whatever state it had at the beginning, so there's no opportunity for a subscription equivalent to modify the state in between
Georges Boris (May 11 2023 at 15:45):uhnnn yeah, they would probably need to mutate a state that is used in the task-chain and gets merged to the global state at the end of the pipeline
Georges Boris (May 11 2023 at 15:48):I think this can be platform specific in that a webserver would probably want a Roc app to represent a single request and the HTTP tasks could receive some shared global state as you proposed that lives outside of each request
Richard Feldman (May 11 2023 at 15:48):yeah it's a very user-hostile API design if your users want to avoid global mutable state, which I guess is understandable considering avoiding global mutable state has only been an industry-wide best practice for like 40 years :stuck_out_tongue:
Richard Feldman (May 11 2023 at 15:50):yeah there are definitely platform-specific options to sort of "contain the blast radius" but at the same time a package for accessing AWS APIs that works across platforms is a pretty obviously desirable thing
Richard Feldman (May 11 2023 at 15:50):also there are other services besides AWS which do this
Richard Feldman (May 11 2023 at 15:51):I guess one idea for how to "contain the blast radius" might be to have a platform-agnostic effect along the lines of "HTTP with refreshable credentials"
Agus Zubiaga (May 11 2023 at 15:51):Richard Feldman said:
because at least then I can use normal
Taskeverywhere instead of having a bunch ofMyTask.fromTaskcalls all over the place just because this is how AWS designed their APIs
I get your point, but in practice I don't think it's that big of a deal. Most Task-producing functions are followed by Task.await anyway, so you can have a helper that both converts it to your own task and await.
Richard Feldman (May 11 2023 at 15:52):true!
Agus Zubiaga (May 11 2023 at 15:52):It could literally be called await and be imported unqualified, since that'd be much more common than the stateful one.
Richard Feldman (May 11 2023 at 15:53):Richard Feldman said:
I guess one idea for how to "contain the blast radius" might be to have a platform-agnostic effect along the lines of "HTTP with refreshable credentials"
given that any HTTP request can already set and get external state, which isn't materially different from global state in the program, this doesn't seem like it would be so bad
Georges Boris (May 11 2023 at 15:53):how would we deal with other stateful things like database pools? "I need to acquire or wait fo a connection before continuing my task and also release the connection later" and that could also happen multiple times in a task chain
Notification Bot (May 11 2023 at 15:54):64 messages were moved here from #ideas > Shadowing & Redeclaration by Richard Feldman.
Richard Feldman (May 11 2023 at 15:54):I haven't thought through a specific design for that yet, but my intuition is that platforms can handle that behind the scenes
Georges Boris (May 11 2023 at 15:55):but we also want a shared postgres driver right
Richard Feldman (May 11 2023 at 15:55):because packages in the ecosystem don't usually need to interact with connection pools directly, it's just something that happens in the implementation of the effects themselves
Richard Feldman (May 11 2023 at 15:56):sure, but I think that can expose ways to help platforms do that
Agus Zubiaga (May 11 2023 at 15:56):Georges Boris said:
but we also want a shared postgres driver right
I think you can still have a driver written in Roc, and have the platform handle general resource pooling by providing hooks
Richard Feldman (May 11 2023 at 15:56):right, exactly
Georges Boris (May 11 2023 at 15:56):the same as we could with http, right?
Georges Boris (May 11 2023 at 15:57):like "http with http-state"
Georges Boris (May 11 2023 at 15:57):the platform could give each individual task it's own specific global state for these use cases
Georges Boris (May 11 2023 at 15:58):(http can also be something that benefits from a pool, it can also benefit from shared credentials, etc.)
Georges Boris (May 11 2023 at 15:59):so I'm +1 for handling that in the platform side of things
Agus Zubiaga (May 11 2023 at 15:59):The problem is where you draw the line, because you also probably need global mutable state for caching
Agus Zubiaga (May 11 2023 at 16:00):Maybe platforms are specific enough that they can handle everything you need to cache
Agus Zubiaga (May 11 2023 at 16:02):Like a GraphQL server platform can parse operation strings with a cache, but you don't even have to think about that when it calls your handler
Georges Boris (May 11 2023 at 16:02):sure... it would be great to have something that can cache views based on some value for instance. how would we do that in a way that is not effectful?
Georges Boris (May 11 2023 at 16:03):but maybe the platform just received some value from the view API for every node and caches things itself without any application state
Georges Boris (May 11 2023 at 16:04):these would be a clean way of having multiple state "buckets" without a messy TEA (messy in... I think doing all of that for things that are not a single web client can be quite tricky...)
Agus Zubiaga (May 11 2023 at 16:08):A healthy gradual pattern can be:
Task type wrapper as we described before Brendan Hansknecht (May 11 2023 at 21:46):Isn't 3 orthogonal to 1 and 2?
Bryce Miller (May 12 2023 at 09:32):Richard Feldman said:
which also couldn't benefit from future planned
Task-specific compiler error message hints like "hint: should there be a|> Task.awaitat the end here?"
Is there already a plan for how to implement error messages like this?
What if the information the compiler needed in order to generate these nice error messages was provided by the platform author in the form of an Ability or something? If the information came from an Ability, then presumably I could also define a similar ability for my custom Task wrapper.
What if there were a way for platform and library authors to offer improved compiler error messages for their types? That seems cool. No idea whether it would be feasible, or how exactly it would be done.
Sky Rose (May 12 2023 at 11:10):What if Roc had a language-based (not platform or app-based) global mutable state, but it could only be accessed during calls to/from the platform, where you're interacting with non-roc stuff that has global mutable state anyway?
Roc expressions get to keep their immutability, and AWS, databases, and everything else gets to be platform-agnostic.
The downsides:
Georges Boris (May 12 2023 at 12:03):I have the feeling we're almost talking about a Roc library that sits on top of one or more platforms and can handle a lot of your state for you and expose it's own API. With Elm this "framework" is built into the language but maybe with Roc there is room for something Roc-based that acts like a framework and it is not necessarily one specific platform.
Georges Boris (May 12 2023 at 12:04):Maybe this can be something like TEA or something completely different and libraries will have docs on how you can plug them into one or more of these standard "frameworks"
Brendan Hansknecht (May 12 2023 at 13:14):It is really easy for a platform to setup a stateful tea model. I don't think this needs any roc level support.
Brendan Hansknecht (May 12 2023 at 13:15):Personally i think that is the nicer option because it means each platform decides how state should be handled or if it is required to be stateless pure functions.
Brendan Hansknecht (May 12 2023 at 13:15):I think that cleanly fits into platform goals and designs
Last updated: Jun 16 2026 at 16:19 UTC