Alternative platform languages · roctoberfest

Hey all :wave: . Are there any non-rust (zig or c) alternatives to the cli-platform? And if not, is there anywhere I could read about creating one? I would like to do some advent of code problems but don't have a rust compiler setup on this machine.

Brendan Hansknecht (Oct 08 2022 at 23:13):

Some platforms are zig based, but no equivalent of the cli platform. The benchmarks platform uses zig for example

Travis (Oct 08 2022 at 23:13):

Or maybe the easiest thing is to just embed the inputs into my .roc files? Does roc havemultiline strings?

Brendan Hansknecht (Oct 08 2022 at 23:15):

I'm not sure about multiline strings (we should, just not sure on syntax/if it has been added). If you choose to embed, you should be able to just based a simple zig platform (like the platform switching one) and return and string of what to print.

Travis (Oct 08 2022 at 23:15):

Oh interesting. I'll take a look at the benchmarks platform, maybe learn from that. I saw the rocLovesZig hello world platform. Maybe the benchmarks platform has more to look at.

Brendan Hansknecht (Oct 08 2022 at 23:16):

Otherwise, platforms don't really have good docs, though the API is significantly more stable than it use to be.

Travis (Oct 08 2022 at 23:18):

Are there any foreseeable roadblocks with using zig? Like anything rust-specific in the cli platform which I'll run into?

Richard Feldman (Oct 08 2022 at 23:18):

Roc does indeed have multiline strings! @Joshua Warner got them working recently

Richard Feldman (Oct 08 2022 at 23:18):

the syntax is triple quotes, and you can use them either on one line or across multiple lines

Brendan Hansknecht (Oct 08 2022 at 23:18):

If you have questions, feel free to ask. Also, i am gonna move this chain of messages out of the day 9 thread, into it's own thread.

Richard Feldman (Oct 08 2022 at 23:19):

if you do it across multiple lines, the """s each have to be on their own lines (with nothing else on the line) and indented the same amount

Brendan Hansknecht (Oct 08 2022 at 23:20):

Nope. Zig should be just fine. The main thing is that you would be redefining the cli-platform/creating a new platform.

Travis (Oct 08 2022 at 23:21):

Brendan Hansknecht (Oct 08 2022 at 23:21):

If you wanted the exact same platform, you would have to conform to the same api. Otherwise, zig is powerful enough to make any platform.

Travis (Oct 08 2022 at 23:24):

Ok thanks for the input. Maybe I'll look into recreating the cli-platform in zig. Would be nice to have...

Travis (Oct 08 2022 at 23:26):

Brendan Hansknecht (Oct 08 2022 at 23:28):

Go for it if you want. Would be interesting to see. Though if you just install rust, you shouldn't actually need to code in rust at all to use the cli platform to solve the advent of code problems.

Travis (Oct 08 2022 at 23:41):

oh and thanks for creating the new thread. i've now identified the 'new topic' menu item :eyes: and will consider using it.

jan kili (Oct 09 2022 at 00:27):

Sharing API logic across differently-hosted platforms sounds like a fun future project...

jan kili (Oct 09 2022 at 00:28):

Or maybe that's just a use case for extracting the common logic into a pure-Roc library for multiple implementation platforms to import

Travis (Oct 09 2022 at 00:35):

Brendan Hansknecht (Oct 09 2022 at 01:56):

I think this is highly unlikely. Long term, most Roc developers are not likely to be platform devs.

Travis (Oct 09 2022 at 02:47):

// in examples/cli/cli-platform/src/lib.rs
pub extern "C" fn roc_fx_fileReadBytes(roc_path: &RocList<u8>) -> RocResult<RocList<u8>, ReadErr>

const str = @import("str");
const RocStr = str.RocStr;

fn RocResult(comptime T: type) type {
    return extern struct {
        value: T,
        is_error: bool,
    };
}

pub export fn roc_fx_readFileBytes(roc_path: *RocStr) RocResult(RocStr)

$ roc 2021/1.roc --linker=legacy
🔨 Rebuilding platform...
ld: /tmp/roc_appYwTVxq.o: in function `roc_fx_fileReadBytes_fastcc_wrapper':
builtins-host:(.text+0xf2b6): undefined reference to `roc_fx_fileReadBytes'

Brendan Hansknecht (Oct 09 2022 at 02:50):

Travis (Oct 09 2022 at 02:51):

Brendan Hansknecht (Oct 09 2022 at 02:53):

I don't have time to debug tonight, but I can probably take a look tomorrow if you push it somewhere. Otherwise, I can ally attempt to answer questions.

Brendan Hansknecht (Oct 09 2022 at 02:53):

You could try manual export with name: @export(roc_fx_getInt, .{ .name = "roc_fx_getInt" });

Brendan Hansknecht (Oct 09 2022 at 02:53):

Brendan Hansknecht (Oct 09 2022 at 02:54):

Travis (Oct 09 2022 at 02:59):

manual export produces the same error. thanks. will let you know if i push code somewhere.

Brendan Hansknecht (Oct 09 2022 at 03:02):

Travis (Oct 09 2022 at 03:03):

Brendan Hansknecht (Oct 09 2022 at 03:20):

Brendan Hansknecht (Oct 09 2022 at 03:21):

Travis (Oct 09 2022 at 03:29):

$ roc ../roc/examples/cli/tui.roc
🔨 Rebuilding platform...

Hello World!

Travis (Oct 09 2022 at 03:30):

Travis (Oct 09 2022 at 03:45):

strange it looks like host.o contains the roc_fx_readFileBytes symbol. not sure what this means

$ nm platform/host.o | grep roc_fx
0000000000003ba0 T roc_fx_getInt
0000000000003d80 t roc_fx_getInt_help
0000000000003650 T roc_fx_putInt
00000000000038a0 T roc_fx_putLine
0000000000004030 T roc_fx_readFileBytes

Travis (Oct 09 2022 at 03:47):

i deleted all the artifacts in platform/ right before i ran this to verify i wasn't looking at an old object file

Travis (Oct 09 2022 at 04:01):

was getting a similar undefined reference to 'roc_fx_stdoutLine' and fixed it with

// host.zig
comptime {
    @export(roc_fx_putLine, .{ .name = "roc_fx_stdoutLine" });
}

Travis (Oct 09 2022 at 04:02):

Travis (Oct 09 2022 at 04:38):

pub export fn roc_fx_fileReadBytes(roc_path: *RocStr, output: *RocResult) callconv(.C) void

Travis (Oct 09 2022 at 04:52):

oops i guess the args are switched. no wonder i couldn't print the string :smirk:

Brendan Hansknecht (Oct 09 2022 at 06:05):

Travis (Oct 09 2022 at 06:14):

Brendan Hansknecht (Oct 09 2022 at 06:27):

Brian Carroll (Oct 09 2022 at 08:38):

@Travis I didn't see it mentioned above so FYI we use Zig 0.9.1 in the Roc standard library so that might be the best version to use in your platform too.

Travis (Oct 09 2022 at 08:39):

pub export fn roc_fx_fileReadBytes(result: *RocResult, path: *RocStr) callconv(.C) void {
    const file = std.fs.cwd().openFile(path.asSlice(), .{}) catch unreachable;
    defer file.close();
    const stat = file.stat() catch unreachable;

    var buf = malloc(stat.size) orelse unreachable;
    var slice = @ptrCast([*]u8, buf)[0..stat.size];
    const amt = file.readAll(slice) catch unreachable;
    std.debug.print("slice {s}\n", .{slice[0..10]});
    std.debug.assert(amt == stat.size);

    const usizes: [*]usize = @ptrCast([*]usize, @alignCast(@alignOf(usize), &result.bytes));
    usizes[0] = 1; // Ok tag == 1
    const outstr = @ptrCast(*RocStr, usizes + 1);
    // const outstr = std.mem.bytesAsValue(RocStr, @ptrCast([*]u8, @alignCast(1, (usizes + 1)))[0..@sizeOf(RocStr)]);
    outstr.* = .{ .str_bytes = slice.ptr, .str_len = slice.len, .str_capacity = slice.len };
}

it compiles and runs but doesn't seem to assign its result param correctly. i have verified that the incoming path is correct and that it can read the file's contents. however, nothing is printed from the roc program and subsequent prints in roc are broken after calling this function.
So i have a couple of questions:

pub const RocResult = extern struct {
    bytes: ?[*]u8,

    pub fn isOk(self: RocResult) bool {
        // assumptions
        //
        // - the tag is the first field
        // - the tag is usize bytes wide
        // - Ok has tag_id 1, because Err < Ok
        const usizes: [*]usize = @ptrCast([*]usize, @alignCast(@alignOf(usize), self.bytes));

        return usizes[0] == 1;
    }

    pub fn isErr(self: RocResult) bool {
        return !self.isOk();
    }
};

Travis (Oct 09 2022 at 08:40):

$ zig version
0.9.1

Travis (Oct 09 2022 at 08:51):

i'm looking through crates/compiler/builtins/bitcode/src/str.zig and think i might be answering some of these questions

Travis (Oct 09 2022 at 08:51):

    // allocate space for a (big or small) RocStr, but put nothing in it yet
    pub fn allocate(length: usize, capacity: usize) RocStr {

Brian Carroll (Oct 09 2022 at 09:05):

Roc code itself uses roc_alloc for heap allocations. Usually best to use that for anything that Roc code can touch.
It's often implemented as a wrapper around malloc anyway.
But Roc heap allocations also need reference counts. The standard library functions you found will do that for you.

Travis (Oct 09 2022 at 09:08):

another question: does this signature look correct? just noticed the rust version uses RocList<u8>

pub extern "C" fn roc_fx_fileReadBytes(roc_path: &RocList<u8>) -> RocResult<RocList<u8>, ReadErr> {

and that there is a zig version of RocList in crates/compiler/builtins/bitcode/src/list.zig

Travis (Oct 09 2022 at 09:14):

thanks @Brian Carroll i hadn't seen roc_alloc yet. i'm thinking that RocStr.allocate is the way to do this but still seeing no output in roc with it. is it correct to just read bytes from the file directly into the RocStr? or is there an encoding i'm missing?

Travis (Oct 09 2022 at 09:15):

Brendan Hansknecht (Oct 09 2022 at 14:53):

As long as zig uses utf8, reading into a RocStr should be correct. I am actually surprised the rust version using a RocList<U8>

Travis (Oct 09 2022 at 15:10):

ok thats good to know. zig strings are not encoded in any way, just byte slices.

Travis (Oct 09 2022 at 15:12):

Travis (Oct 09 2022 at 15:27):

pub export fn roc_fx_fileReadBytes(result: *RocResult, path: *RocStr) callconv(.C) void {
    const file = std.fs.cwd().openFile(path.asSlice(), .{}) catch unreachable;
    defer file.close();
    const stat = file.stat() catch unreachable;

    var rocstr = RocStr.allocate(stat.size, stat.size);
    var bytes = rocstr.str_bytes orelse unreachable;
    const slice = bytes[0..stat.size];
    const amt = file.readAll(slice) catch unreachable;
    // std.debug.print("slice {s}\n", .{slice[0..10]});
    std.debug.assert(amt == stat.size);

    const usizes: [*]usize = @ptrCast([*]usize, @alignCast(@alignOf(usize), &result.bytes));
    usizes[0] = 1; // Ok tag == 1
    const outstr = @ptrCast(*RocStr, usizes + 1);
    outstr.* = rocstr;
}

Brendan Hansknecht (Oct 09 2022 at 16:48):

Brendan Hansknecht (Oct 09 2022 at 16:49):

I am pretty sure we always put tags at the end to save space by making them able to pack smaller

Brendan Hansknecht (Oct 09 2022 at 16:51):

Also, I think you should be returning the RocResult, not taking it as an out param.

Travis (Oct 09 2022 at 17:10):

if this is the case, any idea what the first param is? path is being correctly passed as the second param and i'm able to print it out.

Brendan Hansknecht (Oct 09 2022 at 17:13):

Ah, that's just c-abi fun. I guess technically under c-abi the two function signatures are the same.

Brendan Hansknecht (Oct 09 2022 at 17:14):

Anything returned that is larger than 128bit is converted to return via pointer. But zigs callconv c should handle that for you. Also, zig handling it means you don't have to think about the differences as you change platforms.

Travis (Oct 09 2022 at 17:17):

ah that makes sense. so would it be proper to ignore the first param and return a RocResult by value or go back to the out param?

Brendan Hansknecht (Oct 09 2022 at 17:18):

Brendan Hansknecht (Oct 09 2022 at 17:19):

Also, as a base test, maybe make it just return a RocStr and not a result. That will ensure the pipelining before you figure out the RocResult correctly.

Travis (Oct 09 2022 at 17:19):

Brendan Hansknecht (Oct 09 2022 at 17:23):

Hmm. Also just double checked some code. How you set the roc result originally actually looks correct. We do put the tag at the beginning. I stand corrected.

Brendan Hansknecht (Oct 09 2022 at 17:27):

Hmmm....i am confused. In our rust code for roc result, the tag is at the end. In a util function in zig, the tag is at the beginning.....definitely a bug somewhere.

Brendan Hansknecht (Oct 09 2022 at 17:28):

Travis (Oct 09 2022 at 17:31):

yeah i was a little confused by that too. are you looking at const RocResult = in crates/compiler/builtins/bitcode/src/utils.zig ?

Brendan Hansknecht (Oct 09 2022 at 17:31):

Ok, yeah sorry for the confusion. Ignore the code in utils.zig it isn't used and is outdated and wrong. So I am pretty sure the tag is at the beginning and you should define RocResult like this (rust definition):

#[repr(C)]
pub struct RocResult<T, E> {
    payload: RocResultPayload<T, E>,
    tag: RocResultTag,
}

#[repr(u8)]
#[derive(Clone, Copy)]
enum RocResultTag {
    RocErr = 0,
    RocOk = 1,
}

#[repr(C)]
union RocResultPayload<T, E> {
    ok: ManuallyDrop<T>,
    err: ManuallyDrop<E>,
}

Travis (Oct 09 2022 at 17:34):

ok sounds good. one last question, would the result payload be a *RocStr or RocStr?

Travis (Oct 09 2022 at 17:35):

Brendan Hansknecht (Oct 09 2022 at 17:35):

Travis (Oct 10 2022 at 04:52):

Travis (Oct 10 2022 at 04:56):

so far it can only really do Stdout.line. the following works with the commented out lines currently broken.

main : Program
main =
    pathstr = "platform-test/1.txt"
    path = Path.fromStr pathstr
    task =
        _ <- Task.await (Stdout.line (Path.display path))
        _ <- Task.await (Stdout.line "this gets printed")
        # contents <- Task.await (File.readBytes path)
        # _ <- Task.await (Stdout.line ((Str.fromUtf8 contents) # silently fails, breaks subsequent prints
        #    |> (Result.withDefault "oops")))
        Stdout.line "this doesn't get printed"
    Program.quick task

Travis (Oct 10 2022 at 04:57):

Travis (Oct 10 2022 at 05:04):

$ ./build.sh
🔨 Rebuilding platform...
platform-test/1.txt
this gets printed
this doesn't get printed

Travis (Oct 10 2022 at 05:08):

Brendan Hansknecht (Oct 10 2022 at 06:24):

Just a general note for you, there is a tool roc glue that will make this significantly easier in the future. Currently it only supports rust, but it is made to take roc data structures and convert them to another language. Would eventually also do function signatures. That would more or less directly solve the main problems you are fighting here.

Travis (Oct 10 2022 at 07:06):

neat! will keep an eye on it. i'm guessing it depends on a rust/cargo? that might explain the panic when i try to use it

Travis (Oct 10 2022 at 13:41):

I forgot to mention that I did attempt this, and was glad for the suggestion. but I couldn't manage to create correct roc Effect code to call it.

Travis (Oct 10 2022 at 13:43):

Travis (Oct 10 2022 at 13:46):

Travis (Oct 10 2022 at 13:49):

Travis (Oct 10 2022 at 13:52):

Brendan Hansknecht (Oct 10 2022 at 14:03):

Brendan Hansknecht (Oct 10 2022 at 14:04):

Also, I'll try and take a look today to see if I missed something with the zig signature

Travis (Oct 10 2022 at 14:06):

are you thinking of @cImport or does roc automatically add host.c files to the compilation?

Travis (Oct 10 2022 at 14:08):

Brendan Hansknecht (Oct 10 2022 at 14:12):

I just mean that I know zig can be used to build projects that are a mix of c and zig. So this should be possible to set up

Travis (Oct 10 2022 at 14:18):

Brendan Hansknecht (Oct 10 2022 at 14:24):

We would probably have to change roc to support build.zig files, but should be doable.

Travis (Oct 10 2022 at 14:24):

do you happen to know much about how platforms are built? i want to know why its possible to @import("str") package in a host.zig

Travis (Oct 10 2022 at 14:25):

the only build.zig i could find in the whole project is ./crates/compiler/builtins/bitcode/build.zig and it doesn't seem to call addPackage() anywhere

Brendan Hansknecht (Oct 10 2022 at 14:26):

Just a simple hack were we call zig build-exe with an extra age for our internal str.zig file. Look at crstes/compiler/build/src/link.rs (full disclosure: very hacked together and naturally grown)

Travis (Oct 10 2022 at 14:26):

maybe there is a command line for it ie: $ zig build-exe ... --pkg-begin str .. --pkg-end

Travis (Oct 10 2022 at 14:26):

Brendan Hansknecht (Oct 10 2022 at 14:27):

Realistically, we shouldn't do that. Our str.zig file should be internal only. Platforms longer term should depend on glue for a proper definition. Just doesn't support anything but rust yet.

Travis (Oct 10 2022 at 14:28):

Brendan Hansknecht (Oct 10 2022 at 15:06):

pub export fn roc_fx_fileReadBytes(path: *RocList) callconv(.C) ResultRocList {
    std.debug.print("Called fileReadBytes\n", .{});
    if (path.bytes) |path_ptr| {
        const path_slice = path_ptr[0..path.len()];
        var realpathbuf: [256]u8 = undefined;
        const realpath = std.fs.cwd().realpath(".", &realpathbuf);
        std.debug.print("path '{s}' realpath {s}\n", .{ path_slice, realpath });
        const file = std.fs.cwd().openFile(path_slice, .{}) catch |e|
            roc_panic_print("{s} file path '{s}'\n", .{ @errorName(e), path_slice });
        defer file.close();
        const stat = file.stat() catch |e|
            roc_panic_print("{s} file path '{s}'\n", .{ @errorName(e), path_slice });
        // std.debug.print("stat.size {}\n", .{stat.size});
        var roclist = RocList.allocate(@alignOf(usize), stat.size, stat.size);
        if (roclist.bytes) |bytes| {
            const slice = bytes[0..stat.size];
            const amt = file.readAll(slice) catch unreachable;
            std.debug.assert(amt == stat.size);

            return ResultRocList{
                .payload = roclist,
                .tag = 1,
            };
        }
    }
    return ResultRocList{
        .payload = RocList.empty(),
        .tag = 0,
    };
}

Brendan Hansknecht (Oct 10 2022 at 15:06):

Note, using RocList to match the Effect that is defined: fileReadBytes : List U8 -> Effect (Result (List U8) InternalFile.ReadErr)

Brendan Hansknecht (Oct 10 2022 at 15:08):

        _ <- Task.await (Stdout.line (Path.display path))
        _ <- Task.await (Stdout.line "this gets printed")
        contents <- Task.await (File.readBytes path)
        _ <- Task.await (Stdout.line ((Str.fromUtf8 contents) # silently fails, breaks subsequent prints
            |> (Result.withDefault "oops")))
        Stdout.line "this doesn't get printed"

The last line doesn't print because the Task.await is returning a failure. That kills the rest of the task chain.

Brendan Hansknecht (Oct 10 2022 at 15:09):

Try commenting out Unrecognized I32 Str in InternalFile.roc. Now everything should run fine.

Travis (Oct 10 2022 at 15:10):

Brendan Hansknecht (Oct 10 2022 at 15:11):

The root issues is that the Error case is larger than the Ok case. As such, your result is not the right size. The payload is not the size of a RocList. It is the size of a tag union that contains a RocStr and an I32.

Brendan Hansknecht (Oct 10 2022 at 15:11):

Commenting out that line makes the payload smaller than a RocList and everything works.

Travis (Oct 10 2022 at 15:12):

that makes a lot of sense about the await chain being broken. i wondered about that.

Travis (Oct 10 2022 at 15:14):

i am surprised this works. i was never able to use path as the first param. maybe a c-ism like you talked about yesterday (returning a value vs out param)

Travis (Oct 10 2022 at 15:16):

Brendan Hansknecht (Oct 10 2022 at 15:18):

Brendan Hansknecht (Oct 10 2022 at 15:19):

Brendan Hansknecht (Oct 10 2022 at 15:31):

Oh, just noticed.
This is wrong: var roclist = RocList.allocate(@alignOf(usize), stat.size, stat.size);

Travis (Oct 10 2022 at 15:33):

i think i've made all the changes you indicated. but i'm not seeing any file output. did you see the file printed out?

Travis (Oct 10 2022 at 15:33):

$ ./build.sh
🔨 Rebuilding platform...
platform-test/1.txt
this gets printed
Called fileReadBytes
path 'platform-test/1.txt' realpath ~/roc/advent-of-code
roclist.bytes != null # added by me
runtime: 0.081ms

Travis (Oct 10 2022 at 15:37):

Brendan Hansknecht (Oct 10 2022 at 15:41):

Brendan Hansknecht (Oct 10 2022 at 15:42):

Travis (Oct 10 2022 at 15:42):

Travis (Oct 10 2022 at 15:46):

aw snap nevermind. i edited the wrong InternalFile.roc (i have a sub folder of the originals)

Travis (Oct 10 2022 at 15:46):

Brendan Hansknecht (Oct 10 2022 at 15:47):

I generally use an M1 mac (light weight and portable form factor), but I also use debian x64 pretty often.

Travis (Oct 10 2022 at 15:48):

ok only reason i asked is i thought it might account for the differences. but no just edited the wrong file. :laughing:

Travis (Oct 10 2022 at 15:50):

but thanks a lot! i'll have to think about this a bit maybe have some questions later on.

Travis (Oct 10 2022 at 15:50):

Travis (Oct 10 2022 at 15:56):

ok i think i'm following what you said about the payload sizes differing and how commenting that line out make the sizes comptible now.

Travis (Oct 10 2022 at 15:57):

Travis (Oct 10 2022 at 15:58):

#[repr(C)]
union RocResultPayload<T, E> {
    ok: ManuallyDrop<T>,
    err: ManuallyDrop<E>,
}

Travis (Oct 10 2022 at 15:59):

Brendan Hansknecht (Oct 10 2022 at 17:28):

Brendan Hansknecht (Oct 10 2022 at 17:29):

Brendan Hansknecht (Oct 10 2022 at 17:32):

Yeah....this is where glue would be quit useful. Instead you have to do it manually.

code: i32,
message: RocStr,
tag: u8

Where tag picks the specific ReadError in alphabetical order and code/message are only set when using the Unrecognized variant.

Brendan Hansknecht (Oct 10 2022 at 17:33):

Travis (Oct 10 2022 at 17:35):

oic so i would need to sync the sizes myself. makes sense. idk why i was thinking it could be provided dynamically somehow at compile time.

Travis (Oct 10 2022 at 17:35):

Brendan Hansknecht (Oct 10 2022 at 17:40):

Travis (Oct 11 2022 at 14:30):

hey @Brendan Hansknecht last night i tried running other's aoc roc solutions including yours. as a result i added Stdout.line support locally and then ran into foc_fx_args missing. do you have any recommendations on how implement that one?

Brendan Hansknecht (Oct 11 2022 at 14:31):

Travis (Oct 11 2022 at 14:31):

i tried to wing it by just making that function and building a RocList<RocString> from std.process.args but there were 0 args when i tried to run it which was strange.

Brendan Hansknecht (Oct 11 2022 at 14:31):

Brendan Hansknecht (Oct 11 2022 at 14:32):

Travis (Oct 11 2022 at 14:32):

i looked at your #3990 pr a bit but didn't get too far. i see there are roc_main and roc__mainForHost_1_exposed_generic functions but i'm not sure if those are important here.

Brendan Hansknecht (Oct 11 2022 at 14:33):

#[no_mangle]
pub extern "C" fn roc_fx_args() -> RocList<RocStr> {
    // TODO: can we be more efficient about reusing the String's memory for RocStr?
    std::env::args_os()
        .map(|os_str| RocStr::from(os_str.to_string_lossy().borrow()))
        .collect()
}

Travis (Oct 11 2022 at 14:33):

zero args. i tried using both process.args() and process.argsAlloc() and both had 0 args

Travis (Oct 11 2022 at 14:34):

yeah i looked at the rust version too and was following it. thats why i assumed i could just do what i did. is there something else that has to happen first?

Brendan Hansknecht (Oct 11 2022 at 14:37):

Also, 0 args when printing them from zig? If so, that is probably some sort of zig issue.

Brendan Hansknecht (Oct 11 2022 at 14:38):

If you can print out the args from zig, probably just some sort of issue generating the list of strings.

Brendan Hansknecht (Oct 11 2022 at 14:38):

Travis (Oct 11 2022 at 14:40):

ooh thats one thing i didn't consider much. i'll review the .roc files in my platform folder. might have commented something out or forgot to copy over.

Travis (Oct 11 2022 at 14:45):

i don't see anything amiss with the .roc files. i see Effect.roc/args and Process.roc/withArgs were copied over from the cli-platform.

Travis (Oct 11 2022 at 14:45):

Travis (Oct 11 2022 at 14:51):

$ day1/exec-part1 day1/data.txt
argv.len 0 # printed out from zig
Expected a file name passed on the command line

Travis (Oct 11 2022 at 14:52):

i'll search for that error message 'Expected a file name passed on the command line' see what that turns up

Travis (Oct 11 2022 at 14:58):

ah that message was from your advent day1 solution. :laughing: of course. so something is not happening on zig side. i wonder why there are no args there :thinking:

Brendan Hansknecht (Oct 11 2022 at 15:12):

I guess make a zig app without any roc and make sure it works correctly on your computer?

Travis (Oct 11 2022 at 15:26):

$ cat test.zig
const std = @import("std");

pub fn main() !void {
    const allocator = std.heap.page_allocator;
    const args = try std.process.argsAlloc(allocator);
    std.debug.print("args.len {}\n", .{args.len});
}
travis:/tmp/roc-aoc-2021
$ zig run test.zig -- asdf asdf
args.len 3

Brendan Hansknecht (Oct 11 2022 at 15:38):

Interesting. I wonder if it has to be run from main/roc is messing that up? What happens if you put the function in main instead of in roc_fx_args?

Travis (Oct 11 2022 at 15:44):

you mean what happens if i print out args from host.zig/main()? i tried that too last night and saw 0 args

Brendan Hansknecht (Oct 11 2022 at 15:52):

Travis (Oct 11 2022 at 15:56):

$ roc day1/part1.roc --linker=legacy -- day1/data.txt

Brendan Hansknecht (Oct 11 2022 at 16:12):

Travis (Oct 11 2022 at 16:25):

yeah me too. i'm looking at some of the rust platform examples like the false interpreter have an args param

    #[link_name = "roc__mainForHost_1_exposed_generic"]
    fn roc_main(output: *mut u8, args: &RocStr);

    #[link_name = "roc__mainForHost_1_exposed_generic"]
    fn roc_main(output: *mut u8);

Brendan Hansknecht (Oct 11 2022 at 16:40):

Travis (Oct 11 2022 at 17:30):

if you have time to try running it, i just pushed a wip commit with debug statements in host.zig/main() and roc_fx_args()

Travis (Oct 11 2022 at 17:30):

Travis (Oct 11 2022 at 17:31):

$ roc day1/part1.roc -- day1/data.txt
🔨 Rebuilding platform...
main() args.len 0
roc_fx_args() args.len 0 osargs.len 0
Expected a file name passed on the command line
runtime: 0.030ms

Brendan Hansknecht (Oct 12 2022 at 01:01):

So my only guess is the fact that we tell zig to build an object file. Then call ld directly to link it is somehow breaking things.

Brendan Hansknecht (Oct 12 2022 at 01:04):

Brendan Hansknecht (Oct 12 2022 at 01:15):

Even letting zig control the building and linking doesn't work: zig build-exe platform/host.zig ~/Projects/roc-aoc-2021/day1/exec-part1.o

Brendan Hansknecht (Oct 12 2022 at 01:15):

Brendan Hansknecht (Oct 12 2022 at 01:23):

Causes the program the have args and then crash for what look like proper reasons.

Brendan Hansknecht (Oct 12 2022 at 01:24):

Travis (Oct 12 2022 at 02:05):

thanks so much! :heart: i never would have thought of this. i'm seeing a similar crash here now but the length of the args looks correct. :+1:

Travis (Oct 12 2022 at 02:16):

got it working. fix was passing @ptrCast([*]u8, &rocstr) instead of rocstr.str_bytes

Travis (Oct 12 2022 at 02:23):

Travis (Oct 12 2022 at 02:24):

Travis (Oct 12 2022 at 02:25):

Travis (Oct 12 2022 at 02:28):

$ foo=bar roc platform-test/main.roc -- asdf asfd
🔨 Rebuilding platform...
platform-test/1.txt
...
cli arg 0: /proc/self/fd/3
cli arg 1: asdf
cwd ~/roc/advent-of-code
cwd2 should be /tmp ~/roc/advent-of-code
env:foo=bar
runtime: 0.109ms

Travis (Oct 12 2022 at 02:33):

arg 1 looks fine. exe path is wierd. i'll try making roc_fx_exePath next and see if that is different.

Brendan Hansknecht (Oct 12 2022 at 03:00):

Arg 0 is weird because we compile in memory and then launch from the roc run executable as opposed to launching the binary from disk

Travis (Oct 12 2022 at 03:10):

Travis (Oct 12 2022 at 04:05):

Travis (Oct 12 2022 at 04:44):

oops. Env.setCwd works fine. i was printing out the same path twice :face_palm: :laughter_tears:

Brendan Hansknecht (Oct 12 2022 at 04:54):

Travis (Oct 12 2022 at 05:01):

thanks :big_smile: . you deserve a lot of the credit. been very helpful and patient with my flails.

Travis (Oct 12 2022 at 05:01):

Travis (Oct 12 2022 at 16:25):

Travis (Oct 12 2022 at 16:26):

but i reworked as a union <T,E> like we discussed. not 100% sure about the layout.

Travis (Oct 12 2022 at 16:28):

but it atleast works after adding back the ReadError Unrecognized I32 Str, line

Travis (Oct 12 2022 at 16:29):

and doesn't panic anymore. the error handling was tricky. only real changes are to roc_fx_fileReadBytes

Travis (Oct 12 2022 at 16:32):

its strange that zig's Dir.openFile() wasn't returning an error for file not found. and then there was a panic in the File.stat() call which meant i had to check for fd == -1

Travis (Oct 12 2022 at 16:33):

Brendan Hansknecht (Oct 12 2022 at 16:47):

I think Payload needs to be sorted alphabetically, otherwise, I think that is correct. Would need to do some testing to confirm though.

Brendan Hansknecht (Oct 12 2022 at 16:47):

Brendan Hansknecht (Oct 12 2022 at 16:48):

Travis (Oct 12 2022 at 17:24):

Travis (Oct 12 2022 at 17:26):

if you have any ideas for testing i'm all ears. i was considering trying to pare things down into separate files.

Travis (Oct 12 2022 at 17:27):

i wonder if its possible to run a Task in an expect? i tried the other day but couldn't figure it out.

Richard Feldman (Oct 12 2022 at 17:37):

not yet! we're going to have a separate expect-fx keyword for that, but it doesn't exist yet :big_smile:

Richard Feldman (Oct 12 2022 at 17:39):

well, that's for testing in the sense of actually running the task - there's a separate concept of "simulating" tasks where you never actually run them, but rather sort of mock out their inputs and outputs; that's a separate topic

Travis (Oct 12 2022 at 17:47):

ooh expect-fx sounds neat. cool to hear thats in the works. and there was good reason it wasn't working for me.

Brendan Hansknecht (Oct 12 2022 at 17:52):

The manual way to test is to trigger each error from roc and then have roc print a message for each error. Should tell you if it gets every value correctly.

Travis (Oct 12 2022 at 20:18):

yeah i think i'll try to break up into smaller tests. now that cli args are working, will be a bit easier to make the errors happen. and just letting the program fail sounds good.

Travis (Oct 12 2022 at 20:22):

btw, i tried to repro the wierd zig 0.9.1 behavor i was describing earlier using on latest download build and wasn't able to do so.

Travis (Oct 12 2022 at 20:24):

$ zig version
0.10.0-dev.4280+c3d67c5c4

$ cat /tmp/test.zig
const std = @import("std");

export fn func() callconv(.C) isize {
    const fname = "/tmp/test.zigg";
    std.debug.print("1\n", .{});
    const f = std.fs.cwd().openFile(fname, .{}) catch {
        return -1;
    };
    std.debug.print("2\n", .{});
    defer f.close();
    std.debug.print("3\n", .{});
    const s = f.stat() catch {
        return -1;
    };
    std.debug.print("stat.size {}\n", .{s.size});
    return 0;
}

pub export fn main() callconv(.C) void {
    const x = func();
    std.debug.print("x {}\n", .{x});
}

$ zig run /tmp/test.zig
1
x -1

Travis (Oct 12 2022 at 20:38):

Brendan Hansknecht (Oct 12 2022 at 21:23):

I thought the issue was only with std.process.args/argsAlloc which you aren't calling in that example.

Brendan Hansknecht (Oct 12 2022 at 21:23):

Travis (Oct 12 2022 at 22:05):

a different issue. i couldn't figure out why zig's Dir.openFile and File.stat weren't returning errors and early returning in roc_fx_fileReadBytes's catches. thats where the funky error handling is required (checking if fd == 1).

Travis (Oct 12 2022 at 22:06):

what was actually happening is the underlying call to stat was panicking when that should have been caught

Travis (Oct 12 2022 at 22:09):

and finallly after i figured out what was going on (with wierd error traces), the defer file.close() was also panicking. :cry:

Travis (Oct 12 2022 at 22:11):

so that had to be move down a line. i don't remember 0.9.1 having these issues. something seems off.

Travis (Oct 12 2022 at 22:12):

Brendan Hansknecht (Oct 12 2022 at 23:11):

Travis (Oct 14 2022 at 15:51):

Brendan Hansknecht (Oct 14 2022 at 15:53):

Can probably do so sometime later in the day. Anything specific you want me to look at?

Travis (Oct 14 2022 at 15:54):

Travis (Oct 14 2022 at 15:56):

i've simplified host.zig too. seems closer to what what you indicated before. ReadErr is now just a struct

Travis (Oct 14 2022 at 15:56):

    code: i32,
    message: RocStr,
    tag: u8,

Travis (Oct 14 2022 at 15:59):

Travis (Oct 14 2022 at 16:01):

Travis (Oct 14 2022 at 16:11):

because when i remove when's else branch, there is a big error message suggesting to me that the ReadErr types are wrong.

Travis (Oct 14 2022 at 16:18):

Travis (Oct 14 2022 at 16:31):

Travis (Oct 14 2022 at 16:32):

$ roc test/file-read-errors.roc
🔨 Rebuilding platform...
not found

Travis (Oct 14 2022 at 16:32):

Travis (Oct 14 2022 at 16:34):

Travis (Oct 14 2022 at 16:41):

other errors look good now too. except for Unrecognized. when i try to return one of those i get a segfault. :thinking:

Travis (Oct 14 2022 at 17:01):

i pushed another commit. for now it returns an Unrecoginzed and segfaults when file path is 'foo'

Travis (Oct 14 2022 at 17:02):

Travis (Oct 14 2022 at 17:26):

this layout works with message and code now swapped from the way i had them. and added the _padding field

pub const ReadErr = extern struct {
    message: RocStr,
    code: i32,
    _padding: [7]u8 = undefined,
    tag: u8,

Travis (Oct 14 2022 at 17:27):

$ roc test/file-read-errors.roc
🔨 Rebuilding platform...
Unrecognized code 69 message fo ur twenty

Travis (Oct 14 2022 at 17:28):

Brendan Hansknecht (Oct 14 2022 at 19:55):

So I was looking at the rust datastructures generated by glue and trying to figure out what the correct generation is when I realized that glue definitely has a bug in generation. I am going to take a look at that and them hopefully get back to you on what the correct representation actually is.

Brendan Hansknecht (Oct 14 2022 at 20:29):

Ok. So I looked into this a bit more and realized what I though was a bug was actually me looking at the 32bit definition instead of the 64bit definition.

Brendan Hansknecht (Oct 14 2022 at 20:32):

pub const ReadErr = extern struct {
    code: i32,
    message: RocStr,
    tag: u8,
    _padding: [7]u8 = undefined,

This leads to 4 bytes for the i32, 4 bytes of padding, 24 bytes for the RocStr, then the tag as a u8. So 40 bytes total with the tag at index 32.

Travis (Oct 14 2022 at 20:41):

ah. hmm ok. well that seems to be correct. not sure how i was getting something that looked correct earlier. but this is good.

Travis (Oct 14 2022 at 20:42):

Travis (Oct 14 2022 at 20:44):

in the last little big ive been working on a strictly testing/debugging version of roc_fx_fileReadBytes which just returns errors.

Travis (Oct 14 2022 at 20:44):

Brendan Hansknecht (Oct 14 2022 at 20:46):

Travis (Oct 14 2022 at 20:48):

ooh wait. think i spoke too soon again. Unrecognized doesn't seem to work but the others look good.

Travis (Oct 14 2022 at 20:49):

Travis (Oct 14 2022 at 20:50):

Travis (Oct 14 2022 at 20:51):

Travis (Oct 14 2022 at 20:52):

Travis (Oct 14 2022 at 20:55):

and it segfaulted. i had to flip code/message back to the say i had it. this is what seems to work.

pub const ReadErr = extern struct {
    message: RocStr,
    code: i32,
    tag: u8,
    _padding: [7]u8 = undefined,

Travis (Oct 14 2022 at 20:56):

Travis (Oct 14 2022 at 20:58):

the reason i thought this was correct in the first place is this: line 182 in file_glue.rs

#[cfg(any(
    target_arch = "aarch64",
    target_arch = "x86_64"
))]
#[derive(Clone, Debug, Default, Eq, Ord, Hash, PartialEq, PartialOrd)]
#[repr(C)]
struct ReadErr_Unrecognized {
    pub f1: roc_std::RocStr,
    pub f0: i32,
}

Travis (Oct 14 2022 at 20:59):

Travis (Oct 14 2022 at 21:02):

Travis (Oct 14 2022 at 21:04):

#[cfg(any(
    target_arch = "arm",
    target_arch = "wasm32",
    target_arch = "x86"
))]
#[derive(Clone, Debug, Default, Eq, Ord, Hash, PartialEq, PartialOrd)]
#[repr(C)]
struct ReadErr_Unrecognized {
    pub f0: i32,
    pub f1: roc_std::RocStr,
}

Brendan Hansknecht (Oct 14 2022 at 21:04):

Brendan Hansknecht (Oct 14 2022 at 21:05):

Brendan Hansknecht (Oct 14 2022 at 21:06):

I didn't realize that this optimization also applied to tag unions, I thought it was just for records.

Travis (Oct 14 2022 at 21:06):

Travis (Oct 14 2022 at 21:07):

Travis (Oct 14 2022 at 21:08):

Brendan Hansknecht (Oct 14 2022 at 21:13):

Travis (Oct 14 2022 at 21:50):

i've got a good test script now passing in error names from the cli. i was experimenting trying to get them all corrected and i this is what i've come up with.

pub const ReadErr = extern struct {
    message: RocStr,
    code: i32,
    tag: usize,

Travis (Oct 14 2022 at 21:51):

with this all the error tags are good and the Unrecognized payload is too. :big_smile:

Brendan Hansknecht (Oct 14 2022 at 21:53):

Travis (Oct 14 2022 at 21:54):

Brendan Hansknecht (Oct 14 2022 at 21:55):

currently the only supported platforms are little endian and either 32 or 64 bit

Stream: roctoberfest

Topic: Alternative platform languages

Travis (Oct 08 2022 at 23:12):

Brendan Hansknecht (Oct 08 2022 at 23:13):

Travis (Oct 08 2022 at 23:13):

Brendan Hansknecht (Oct 08 2022 at 23:15):

Travis (Oct 08 2022 at 23:15):

Brendan Hansknecht (Oct 08 2022 at 23:16):

Travis (Oct 08 2022 at 23:18):

Richard Feldman (Oct 08 2022 at 23:18):

Richard Feldman (Oct 08 2022 at 23:18):

Brendan Hansknecht (Oct 08 2022 at 23:18):

Richard Feldman (Oct 08 2022 at 23:19):

Brendan Hansknecht (Oct 08 2022 at 23:20):

Travis (Oct 08 2022 at 23:21):

Brendan Hansknecht (Oct 08 2022 at 23:21):

Travis (Oct 08 2022 at 23:24):

Travis (Oct 08 2022 at 23:26):

Brendan Hansknecht (Oct 08 2022 at 23:28):

Travis (Oct 08 2022 at 23:41):

jan kili (Oct 09 2022 at 00:27):

jan kili (Oct 09 2022 at 00:28):

jan kili (Oct 09 2022 at 00:28):

Travis (Oct 09 2022 at 00:35):

Brendan Hansknecht (Oct 09 2022 at 01:56):

Travis (Oct 09 2022 at 02:47):

Brendan Hansknecht (Oct 09 2022 at 02:50):

Travis (Oct 09 2022 at 02:51):

Brendan Hansknecht (Oct 09 2022 at 02:53):

Brendan Hansknecht (Oct 09 2022 at 02:53):

Brendan Hansknecht (Oct 09 2022 at 02:53):

Brendan Hansknecht (Oct 09 2022 at 02:53):

Brendan Hansknecht (Oct 09 2022 at 02:54):

Travis (Oct 09 2022 at 02:59):

Brendan Hansknecht (Oct 09 2022 at 03:02):

Travis (Oct 09 2022 at 03:03):

Brendan Hansknecht (Oct 09 2022 at 03:20):

Brendan Hansknecht (Oct 09 2022 at 03:21):

Travis (Oct 09 2022 at 03:29):

Travis (Oct 09 2022 at 03:30):

Travis (Oct 09 2022 at 03:45):

Travis (Oct 09 2022 at 03:47):

Travis (Oct 09 2022 at 04:01):

Travis (Oct 09 2022 at 04:02):

Travis (Oct 09 2022 at 04:38):

Travis (Oct 09 2022 at 04:52):

Brendan Hansknecht (Oct 09 2022 at 06:05):

Travis (Oct 09 2022 at 06:14):

Brendan Hansknecht (Oct 09 2022 at 06:27):

Brian Carroll (Oct 09 2022 at 08:38):

Travis (Oct 09 2022 at 08:39):

Travis (Oct 09 2022 at 08:40):

Travis (Oct 09 2022 at 08:51):

Travis (Oct 09 2022 at 08:51):

Brian Carroll (Oct 09 2022 at 09:05):

Travis (Oct 09 2022 at 09:08):

Travis (Oct 09 2022 at 09:14):

Travis (Oct 09 2022 at 09:15):

Brendan Hansknecht (Oct 09 2022 at 14:53):

Travis (Oct 09 2022 at 15:10):

Travis (Oct 09 2022 at 15:12):

Travis (Oct 09 2022 at 15:27):

Travis (Oct 09 2022 at 15:27):

Brendan Hansknecht (Oct 09 2022 at 16:48):

Brendan Hansknecht (Oct 09 2022 at 16:49):

Brendan Hansknecht (Oct 09 2022 at 16:51):

Travis (Oct 09 2022 at 17:10):

Brendan Hansknecht (Oct 09 2022 at 17:13):

Brendan Hansknecht (Oct 09 2022 at 17:14):

Travis (Oct 09 2022 at 17:17):

Brendan Hansknecht (Oct 09 2022 at 17:18):

Brendan Hansknecht (Oct 09 2022 at 17:19):

Travis (Oct 09 2022 at 17:19):

Brendan Hansknecht (Oct 09 2022 at 17:23):

Brendan Hansknecht (Oct 09 2022 at 17:27):

Brendan Hansknecht (Oct 09 2022 at 17:28):

Travis (Oct 09 2022 at 17:31):

Brendan Hansknecht (Oct 09 2022 at 17:31):

Travis (Oct 09 2022 at 17:34):

Travis (Oct 09 2022 at 17:35):