Aurélien Geron (Oct 02 2024 at 21:04): Aurélien Geron (Oct 02 2024 at 21:04):Has anyone tried to feed an AI (copilot or Claude.ai or ChatGPT or another) all the rust & zig code to look for potential bugs, especially memory related errors? I bet it could find a few that might otherwise take a while to hunt down.
Luke Boswell (Oct 02 2024 at 21:49):Sounds expensive. I've fed a lot in, just exploring the code base. But haven't looked for bugs or anything. It's pretty sketchy sometimes just asking what it's even doing, so I'm suspect it would find any bugs.
Aurélien Geron (Oct 02 2024 at 21:52):You mean it would not find any bugs?
Aurélien Geron (Oct 02 2024 at 21:53):I've found Claude.ai to be reasonably good at coding, better than ChatGPT in general. Last time I tried CoPilot it wasn't very useful, but it's probably improved a lot since then.
Aurélien Geron (Oct 02 2024 at 21:54):My thought is not to let it code for us (it's way too unreliable for that), but it can sometimes make interesting suggestions or catch bugs.
Luke Boswell (Oct 02 2024 at 22:04):Aurélien Geron said:
You mean it would not find any bugs?
I'm not sure. My gut feeling is that it probably wont.
Luke Boswell (Oct 02 2024 at 22:05):I have only really been using Chat GPT-4o though
Luke Boswell (Oct 02 2024 at 22:05):I've got Claude set up in Zed with the API but it's expensive, so I prefer to use the free browser version first before I resort to that
Aurélien Geron (Oct 02 2024 at 22:05):I've tried feeding it 3 zig files so far, and it hasn't found any bug. Perhaps there are none, or perhaps this AI sucks, oh well, I tried.
Luke Boswell (Oct 02 2024 at 22:06):I think we just have a very large code base (also in a niche domain), and so there is a lot of context that is need to understand.
Aurélien Geron (Oct 02 2024 at 22:07):Example output for list.zig:
listDropAt, there's a comment about duplicating the utils.decref call. This might be an area to review to ensure there's no double-freeing or memory leaks.listConcat function has complex logic for handling different cases (empty lists, unique lists, etc.). While it appears correct, this complexity increases the risk of subtle bugs. Aurélien Geron (Oct 02 2024 at 22:07):Yeah, it seems out of its depth
Luke Boswell (Oct 02 2024 at 22:08):It's usually pretty good at mechanical things, "like this API is using a builder pattern", or "this is using recursion", but it doesn't understand what its actually talking about I feel
Anton (Oct 04 2024 at 08:35):I've got Claude set up in Zed with the API but it's expensive, so I prefer to use the free browser version first before I resort to that
Might want to check your settings, zed offers their own custom version of claude sonnet 3.5 that's free to use for now.
Anton (Oct 04 2024 at 08:37):I've found AI can be helpful if you've already found the specific problematic section of code. Roc segmentation fault debugging is a great AGI milestone in my opinion :big_smile:
Last updated: Jun 16 2026 at 16:19 UTC