Luke Boswell (Dec 11 2024 at 07:18): Luke Boswell (Dec 11 2024 at 07:18):Ok, so we've landed basic-cli ... next step is our webserver.
I made a bit of a start yesterday, but haven't pushed anything yet. I'm WFH for the rest of the week so should have time to smash through it. I'd love to have something ready for a new release on Friday, but there's a lot to do.
Sam Mohr (Dec 11 2024 at 07:26):What platforms otherwise should get upgraded for Purity Inference?
Sam Mohr (Dec 11 2024 at 07:26):any of the templates?
Brendan Hansknecht (Dec 11 2024 at 07:30):All the platforms in the roc repo still need to updated along with docs.
Brendan Hansknecht (Dec 11 2024 at 07:30):I think we could update templates, but may be best to do them at the very end after all the docs and such are updated
Brendan Hansknecht (Dec 14 2024 at 05:58):Question, can the host import packages? If so, can we package all the roc files used to build the host API somehow?
Luke Boswell (Dec 14 2024 at 05:59):I don't quite follow
Luke Boswell (Dec 14 2024 at 05:59):The host is rust, so we can refactor out into crates and re-use those
Luke Boswell (Dec 14 2024 at 05:59):For the roc files, we can make a cross platform package like roc-lang/path
Luke Boswell (Dec 14 2024 at 05:59):But path hasn't matured enough to use it yet... though this is starting to lay the foundation for that, so it's easier to upgrade in future
Brendan Hansknecht (Dec 14 2024 at 06:03):So, I mean for all of the platform .roc files outside of Host.roc that have the exact same api (ex. File.roc, Stdin.roc, Stdout.roc, etc)
Brendan Hansknecht (Dec 14 2024 at 06:03):Can we share those files between basic webserver and basic cli
Luke Boswell (Dec 14 2024 at 06:04):Aside from a cross-platform package... not really
Luke Boswell (Dec 14 2024 at 06:04):I'm hoping to get the API's identical so that we can have basic-webserver just copy from basic-cli
Luke Boswell (Dec 14 2024 at 06:05):There's only a few things that wouldn't be shared like, MultiPartFormData, or SqLite... though we could potentially get them in both too... :shrug:
Luke Boswell (Dec 14 2024 at 06:06):In this pass I'm just trying to get the current features like Cmd, File, Dir, Env etc common
Luke Boswell (Dec 14 2024 at 06:07):It's probably a lot to chew, but the type safety gives me some confidence I can untangle it all
Brendan Hansknecht (Dec 14 2024 at 06:09):Long term, we definitely should look into allow platforms to directly share things like Stdout.roc if they have the same primitives (or though module parana)
Notification Bot (Dec 14 2024 at 06:10):13 messages were moved here from #contributing > Pull Request for Review by Luke Boswell.
Luke Boswell (Dec 14 2024 at 06:10):Is there any reason the Tcp API should be different between cli and webserver?
Luke Boswell (Dec 14 2024 at 06:11):For sending requests
Sam Mohr (Dec 14 2024 at 06:11):I think the best would be to try to develop some common set of only a couple generic functions that each platform would need to support, and then a module params-based package would handle giving a useful, platform agnostic interface
Sam Mohr (Dec 14 2024 at 06:11):Luke Boswell said:
Is there any reason the Tcp API should be different between cli and webserver?
I'd say no
Sam Mohr (Dec 14 2024 at 06:12):Unless one of them prepackages SSL certs or something?
Sam Mohr (Dec 14 2024 at 06:12):Which even then I wouldn't expect should be handled by the platform automatically
Luke Boswell (Dec 14 2024 at 06:14):Alg, I tend to agree. :thumbs_up:
Brendan Hansknecht (Dec 14 2024 at 06:31):Sam Mohr said:
I think the best would be to try to develop some common set of only a couple generic functions that each platform would need to support, and then a module params-based package would handle giving a useful, platform agnostic interface
For sure, would be nice if the platform could re expose and import
Basically import from a package passing in effects from the platform. Then re-expose that packages api
Sam Mohr (Dec 14 2024 at 06:40):Brendan Hansknecht said:
Sam Mohr said:
I think the best would be to try to develop some common set of only a couple generic functions that each platform would need to support, and then a module params-based package would handle giving a useful, platform agnostic interface
For sure, would be nice if the platform could re expose and import
Basically import from a package passing in effects from the platform. Then re-expose that packages api
Thinking about it, I think static dispatch should make this easier? If the platform just exports structural aliases to custom types, the "methods" of the custom types should get exposed as if they were written inline. Not sure if that would work
Sam Mohr (Dec 14 2024 at 06:41):I think it's not ideal if we have to manually copy over the docs and all functions (even codegen would be suboptimal)
Luke Boswell (Dec 14 2024 at 06:51):Ok, finally made a draft PR for this https://github.com/roc-lang/basic-webserver/pull/84
Luke Boswell (Dec 14 2024 at 06:54):Anyone will welcome to help continue with the upgrade :smile:
Next up will be going through all the Host.roc exposed functions one by one and implementing them in the host. Should be mostly just updating the function signature and then calling the impl in the crate from basic-cli.
Then it's on to upgrading the examples and verifying everything is wired up correctly and working.
I've disabled HTTP.send! for now... we need to dig into that a little an find an API that works for both cli and webserver. They're currently different.
Brendan Hansknecht (Dec 14 2024 at 07:11):update the false interpreter to purity inference and get it running as a test again: https://github.com/roc-lang/roc/pull/7369
Brendan Hansknecht (Dec 14 2024 at 07:14):limit morphic to trivial solving only to avoid the inplace mutation correctness bugs: https://github.com/roc-lang/roc/pull/7370
Surprisingly, this seems to increase performance in a number of cases by a few percent. My only guess is that the accidental mutation is leading to extra looping that shouldn't be happening. All changes to perf are within 5% and most are slightly positive.
Luke Boswell (Dec 18 2024 at 05:54):Made significant progress with https://github.com/roc-lang/basic-webserver/pull/84
I've hit another segfault... this time I think it's on the other side, roc passing the response back to the host.
I'm pretty tired from all the changes I've made today, so I'll maybe come back to this tomorrow. If anyone has time to look at it, the easiest repro is just building the host roc build.roc, then running the echo example roc examples/echo.roc.
Anton (Dec 18 2024 at 13:26):Valgrind output:
❯ valgrind ./examples/echo
==37674== Memcheck, a memory error detector
==37674== Copyright (C) 2002-2024, and GNU GPL'd, by Julian Seward et al.
==37674== Using Valgrind-3.23.0 and LibVEX; rerun with -h for copyright info
==37674== Command: ./examples/echo
==37674==
Listening on <http://127.0.0.1:8000>
==37674== Thread 34 tokio-runtime-w:
==37674== Conditional jump or move depends on uninitialised value(s)
==37674== at 0x12BECA: ??? (roc_app:0)
==37674== by 0x124F65: Box_unbox_3d7aff37b23cd9f9e6beb177d8bf818babb9d186ea278cc981a34be43b8cf34 (roc_app:0)
==37674== by 0x1299D4: _respond_for_host!_bad96aa871ccf5b068b2a1da7544fd3d07a932588efb92244e692b8beda99ce (roc_app:0)
==37674== by 0x129D1C: roc__respond_for_host_1_exposed_generic (roc_app:0)
==37674== by 0x196D2B: tokio::runtime::task::raw::poll (in /home/username/gitrepos/basic-webserver/examples/echo)
==37674== by 0x1E8C6D: std::sys_common::backtrace::__rust_begin_short_backtrace (in /home/username/gitrepos/basic-webserver/examples/echo)
==37674== by 0x1E8910: core::ops::function::FnOnce::call_once{{vtable.shim}} (in /home/username/gitrepos/basic-webserver/examples/echo)
==37674== by 0x1E4A5A: call_once<(), dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global> (boxed.rs:2022)
==37674== by 0x1E4A5A: call_once<(), alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global>, alloc::alloc::Global> (boxed.rs:2022)
==37674== by 0x1E4A5A: std::sys::pal::unix::thread::Thread::new::thread_start (thread.rs:108)
==37674== by 0x48ECA41: start_thread (in /nix/store/0wydilnf1c9vznywsvxqnaing4wraaxp-glibc-2.39-52/lib/libc.so.6)
==37674== by 0x496BE43: clone (in /nix/store/0wydilnf1c9vznywsvxqnaing4wraaxp-glibc-2.39-52/lib/libc.so.6)
==37674==
==37674== Conditional jump or move depends on uninitialised value(s)
==37674== at 0x12B3B9: decrement_refcounted_ptr_8 (roc_app:0)
==37674== by 0x12BED7: ??? (roc_app:0)
==37674== by 0x124F65: Box_unbox_3d7aff37b23cd9f9e6beb177d8bf818babb9d186ea278cc981a34be43b8cf34 (roc_app:0)
==37674== by 0x1299D4: _respond_for_host!_bad96aa871ccf5b068b2a1da7544fd3d07a932588efb92244e692b8beda99ce (roc_app:0)
==37674== by 0x129D1C: roc__respond_for_host_1_exposed_generic (roc_app:0)
==37674== by 0x196D2B: tokio::runtime::task::raw::poll (in /home/username/gitrepos/basic-webserver/examples/echo)
==37674== by 0x1E8C6D: std::sys_common::backtrace::__rust_begin_short_backtrace (in /home/username/gitrepos/basic-webserver/examples/echo)
==37674== by 0x1E8910: core::ops::function::FnOnce::call_once{{vtable.shim}} (in /home/username/gitrepos/basic-webserver/examples/echo)
==37674== by 0x1E4A5A: call_once<(), dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global> (boxed.rs:2022)
==37674== by 0x1E4A5A: call_once<(), alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global>, alloc::alloc::Global> (boxed.rs:2022)
==37674== by 0x1E4A5A: std::sys::pal::unix::thread::Thread::new::thread_start (thread.rs:108)
==37674== by 0x48ECA41: start_thread (in /nix/store/0wydilnf1c9vznywsvxqnaing4wraaxp-glibc-2.39-52/lib/libc.so.6)
==37674== by 0x496BE43: clone (in /nix/store/0wydilnf1c9vznywsvxqnaing4wraaxp-glibc-2.39-52/lib/libc.so.6)
==37674==
2024-12-18T13:26:11Z Get /
==37674== Invalid read of size 8
==37674== at 0x12C0B3: ??? (roc_app:0)
==37674== by 0x128322: #UserApp_respond!_1e4d2f1e6b4984301a1489b71481ade3a818d1fae80b8f87ea525c7bff923 (roc_app:0)
==37674== by 0x1299E6: _respond_for_host!_bad96aa871ccf5b068b2a1da7544fd3d07a932588efb92244e692b8beda99ce (roc_app:0)
==37674== by 0x129D1C: roc__respond_for_host_1_exposed_generic (roc_app:0)
==37674== by 0x196D2B: tokio::runtime::task::raw::poll (in /home/username/gitrepos/basic-webserver/examples/echo)
==37674== by 0x1E8C6D: std::sys_common::backtrace::__rust_begin_short_backtrace (in /home/username/gitrepos/basic-webserver/examples/echo)
==37674== by 0x1E8910: core::ops::function::FnOnce::call_once{{vtable.shim}} (in /home/username/gitrepos/basic-webserver/examples/echo)
==37674== by 0x1E4A5A: call_once<(), dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global> (boxed.rs:2022)
==37674== by 0x1E4A5A: call_once<(), alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global>, alloc::alloc::Global> (boxed.rs:2022)
==37674== by 0x1E4A5A: std::sys::pal::unix::thread::Thread::new::thread_start (thread.rs:108)
==37674== by 0x48ECA41: start_thread (in /nix/store/0wydilnf1c9vznywsvxqnaing4wraaxp-glibc-2.39-52/lib/libc.so.6)
==37674== by 0x496BE43: clone (in /nix/store/0wydilnf1c9vznywsvxqnaing4wraaxp-glibc-2.39-52/lib/libc.so.6)
==37674== Address 0xfffffffffffffff8 is not stack'd, malloc'd or (recently) free'd
==37674==
==37674==
==37674== Process terminating with default action of signal 11 (SIGSEGV)
==37674== Access not within mapped region at address 0xFFFFFFFFFFFFFFF8
==37674== at 0x12C0B3: ??? (roc_app:0)
==37674== by 0x128322: #UserApp_respond!_1e4d2f1e6b4984301a1489b71481ade3a818d1fae80b8f87ea525c7bff923 (roc_app:0)
==37674== by 0x1299E6: _respond_for_host!_bad96aa871ccf5b068b2a1da7544fd3d07a932588efb92244e692b8beda99ce (roc_app:0)
==37674== by 0x129D1C: roc__respond_for_host_1_exposed_generic (roc_app:0)
==37674== by 0x196D2B: tokio::runtime::task::raw::poll (in /home/username/gitrepos/basic-webserver/examples/echo)
==37674== by 0x1E8C6D: std::sys_common::backtrace::__rust_begin_short_backtrace (in /home/username/gitrepos/basic-webserver/examples/echo)
==37674== by 0x1E8910: core::ops::function::FnOnce::call_once{{vtable.shim}} (in /home/username/gitrepos/basic-webserver/examples/echo)
==37674== by 0x1E4A5A: call_once<(), dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global> (boxed.rs:2022)
==37674== by 0x1E4A5A: call_once<(), alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global>, alloc::alloc::Global> (boxed.rs:2022)
==37674== by 0x1E4A5A: std::sys::pal::unix::thread::Thread::new::thread_start (thread.rs:108)
==37674== by 0x48ECA41: start_thread (in /nix/store/0wydilnf1c9vznywsvxqnaing4wraaxp-glibc-2.39-52/lib/libc.so.6)
==37674== by 0x496BE43: clone (in /nix/store/0wydilnf1c9vznywsvxqnaing4wraaxp-glibc-2.39-52/lib/libc.so.6)
==37674== If you believe this happened as a result of a stack
==37674== overflow in your program's main thread (unlikely but
==37674== possible), you can try to increase the size of the
==37674== main thread stack using the --main-stacksize= flag.
==37674== The main thread stack size used in this run was 8388608.
==37674==
==37674== HEAP SUMMARY:
==37674== in use at exit: 149,092 bytes in 497 blocks
==37674== total heap usage: 2,438 allocs, 1,941 frees, 384,367 bytes allocated
==37674==
==37674== LEAK SUMMARY:
==37674== definitely lost: 8 bytes in 1 blocks
==37674== indirectly lost: 0 bytes in 0 blocks
==37674== possibly lost: 12,414 bytes in 38 blocks
==37674== still reachable: 136,670 bytes in 458 blocks
==37674== suppressed: 0 bytes in 0 blocks
==37674== Rerun with --leak-check=full to see details of leaked memory
==37674==
==37674== Use --track-origins=yes to see where uninitialised values come from
==37674== For lists of detected and suppressed errors, rerun with: -s
==37674== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 0 from 0)
Segmentation fault (core dumped)
Anton is truly our resident Valgrinder, I salute you for helping those of us with Macs :salute:
Anton (Dec 18 2024 at 16:32):#5772 has very similar valgrind output, it was fixed by PR#5892
Brendan Hansknecht (Dec 18 2024 at 17:30):Probably an issue with the model in this case
Brendan Hansknecht (Dec 18 2024 at 17:30):Probably can take a look in a bit
Brendan Hansknecht (Dec 18 2024 at 18:24):Taking a look now, on mac, looks to still be input side.
Brendan Hansknecht (Dec 18 2024 at 18:24):Crashes the first time it tries to load from the request to print it
Brendan Hansknecht (Dec 18 2024 at 18:29):segfaults when loading the body in roc on mac
Brendan Hansknecht (Dec 18 2024 at 18:29):oh... RocList::from_raw_parts would do that....
Brendan Hansknecht (Dec 18 2024 at 18:30):we can make a seemless slice here, but this is not what is being done
Luke Boswell (Dec 18 2024 at 18:42):ooops, forgot I did that. Was trying to be efficient and forgot about it. Does making an actual owned RocStr help?
Brendan Hansknecht (Dec 18 2024 at 18:44):I just made it a proper seamless slice and pushed that
Brendan Hansknecht (Dec 18 2024 at 18:45):could also do that with the headers if wanted
Brendan Hansknecht (Dec 18 2024 at 18:46):let me push that as well, should be easy at this point and avoid copies
Luke Boswell (Dec 18 2024 at 18:54):Oh nice. I managed to fix it with let body = RocList::from_slice(body.to_vec().as_slice()); but I imagine your impl is more efficient
Brendan Hansknecht (Dec 18 2024 at 18:55):yeah, will push shortly
Luke Boswell (Dec 18 2024 at 18:56):some voodoo magic in here... :stuck_out_tongue:
let const_refcount_allocation = (&REFCOUNT_CONSTANT as *const u64) as usize + size_of_val(&REFCOUNT_CONSTANT);
let const_seamless_slice = (const_refcount_allocation >> 1) | SEAMLESS_SLICE_BIT;
Just pushed another change
Brendan Hansknecht (Dec 18 2024 at 18:58):with more magic
Brendan Hansknecht (Dec 18 2024 at 18:58):but it is hidden behind a function now
Luke Boswell (Dec 18 2024 at 19:02):We've still got an issue somewhere, see roc examples/init-basic.roc
Luke Boswell (Dec 18 2024 at 19:03):hmm, maybe just a standalone bug somewhere.
Luke Boswell (Dec 18 2024 at 19:04):I haven't got any of the expect scripts passing. I'll need to dig into each now and figure out what I've changed
Luke Boswell (Dec 18 2024 at 19:05):ooh, GET -> Get... simple enough change, I think I switched to using Inspect.toStr because we didn't need method_to_str, turns out we do if we want these tags to print all uppercase
Luke Boswell (Dec 18 2024 at 19:06):Or do people think this looks fine for a log?
2024-12-18T19:05:04Z Get /
I'd prefer all caps
Anthony Bullard (Dec 18 2024 at 19:07):I prefer caps for methods
Brendan Hansknecht (Dec 18 2024 at 19:08):Can the tag simply be all uppercase?
Luke Boswell (Dec 18 2024 at 19:11):It's not an acronym, so it might be a little strange.
Luke Boswell (Dec 18 2024 at 19:11):We could do that...
Brendan Hansknecht (Dec 18 2024 at 19:12):Yeah, might not be worth, I just like being able to print things in the expect form via the default print and debug mechanism.
Richard Feldman (Dec 18 2024 at 19:13):it's not an acronym but they're always spelled in all caps. I think it's fine if it doesn't fit the usual identifier convention but does fit the way it's usually spelled
Luke Boswell (Dec 18 2024 at 19:16):Ok, changing to all caps for Http Method
Luke Boswell (Dec 18 2024 at 19:19):Ok, so we've restored most of our CI tests now
Luke Boswell (Dec 18 2024 at 19:20):Only a few are failing
Luke Boswell (Dec 18 2024 at 19:22):Something is really broken in examples/init-basic.roc, it's a super simple app but the server just hangs
Brendan Hansknecht (Dec 18 2024 at 19:44):I'll take a look now
Anthony Bullard (Dec 18 2024 at 19:49):It gets stuck during init or respond?
Brendan Hansknecht (Dec 18 2024 at 19:50):fails to load the model during respond I think
Brendan Hansknecht (Dec 18 2024 at 19:51):init gave me ðŸŽ
Brendan Hansknecht (Dec 18 2024 at 19:51):so beautiful.....
Brendan Hansknecht (Dec 18 2024 at 19:54):looks like roc does expect the box as a boxed_model: RocBox<()>, and not by pointer
Brendan Hansknecht (Dec 18 2024 at 19:55):Not sure why the data is wrong though
Brendan Hansknecht (Dec 18 2024 at 19:57):yeah, I think changing the model works, then rendering unicode seems to be failing
Brendan Hansknecht (Dec 18 2024 at 20:01):Also, refcounted things in a model for basic webserver technically have a race condition. Cause the outer box is set to a constant refcount, but none of the inner data is. As such when loading inner data, each thread will locally be incrementing and decrementing refcounts.
Brendan Hansknecht (Dec 18 2024 at 20:03):And I think that 🎠is the present, just rendering as ascii and not unicode.
Brendan Hansknecht (Dec 18 2024 at 20:04):So everything "working" with the box change
Brendan Hansknecht (Dec 18 2024 at 20:05):just pushed it
Brendan Hansknecht (Dec 18 2024 at 20:09):and "working" means that I think it should be doing the same as the old basic-webserver including bugs
Luke Boswell (Dec 18 2024 at 21:26):We're on the home stretch.... :smiley:
Screenshot 2024-12-19 at 08.25.09.png
I've ignored two failing tests, because I wanted to see how far we got through CI.
So just a couple things now to dig into, and we should be good to go.
Anthony Bullard (Dec 18 2024 at 22:02):Famous last words Luke…. :stuck_out_tongue_closed_eyes:
Luke Boswell (Dec 18 2024 at 22:03):1 down... 1 to go :chef's kiss:
Anthony Bullard (Dec 18 2024 at 22:11)::folded_hands:
Luke Boswell (Dec 19 2024 at 05:32):And we did it! :tada: :space_invader:
Screenshot 2024-12-19 at 16.30.45.png
A big thank you to @Brendan Hansknecht (for saving us 10x over) and @Sam Mohr for helping with this upgrade. It was much smoother this time round after landing basic-cli, and re-using crates from there.
I'm looking forward to testing it out and upgrading my demo web apps etc, and see what issues we can flush out.
Anthony Bullard (Dec 19 2024 at 10:18):Congrats Luke on landing this! Having the two big platforms moved over will help a lot
Anthony Bullard (Dec 19 2024 at 10:19):Congrats Luke on landing this! Having the two big platforms moved over will help a lot
Last updated: Jul 06 2025 at 12:14 UTC