I'm sure people are doing this but I just opened these two bug reports with reproducible examples and automated the entire process, from discovery to reproducibility, with AI (I still typed up the github issues and ran them myself to verify), but I wanted to share in case someone found this useful:
In kai, I added this line to my agent rule .md file:
- As you work, keep track of Roc-specific quirks, errors, and design/implementation recommendations you find in `docs/roc-isms/`. We are using the Roc compiler and as such expect many breaking changes. Keep track bugs by prefixing files with `BUG-*.md` e.g. `BUG-001.md`. Give a clear description of the issue and steps/exact commands to reproduce. For design or implementation bad practices, quirks, or suggestions, put those in `IMPROVEMENT-*.md`. We ignore these files via a `.gitignore`.
I then periodically have the agent copy whatever bugs are in there over to this issue tracking reproduction repo. Using a combination of roc-overlay and dogfooding kai, I can create exact reproducible environments + bash scripts for that bug! So then all you need to do would be to run kai shell <issue> or whatever and you'll be in a shell with the exact compiler version with code that reproduces the exact bug I received on my machine.
When I'm ready to open an issue, I ask it to double check that this error hasn't already been fixed or is a duplicate.
Anyway, just thought I'd share!
Last updated: Sep 03 2026 at 15:16 UTC