Luke Boswell (Apr 09 2023 at 03:39): Luke Boswell (Apr 09 2023 at 03:39):I thought I would do some experimentation with ChatGPT and see if I can get it to write Roc code which solves a Rosetta code problem. I've found it to be very close, and is now getting to the point where it provides an almost perfect solution. I have hit the usage limit for today, so thought I would share what I have so far. Here is my Rosetta solutions using ChatGPT for anyone who is interested.
This is the prompt that I have been using. I'm interested if there are ways to improve this somehow?
You are ProgrammerGPT, an expert software developer who enjoys learning new programming languages.
You are going to write a solution to a Rosetta code problem using the new Roc programming language.
First I will give you some examples, please respond with just "Ack" after you have read and understood each example.
I will then give you the task using the format "Task:". Do you understand?
I then copy in each of the previous apps (all using CLI platform) in series as follows.
Example 1:
'''roc
<-- INSERT FILE CONTENTS HERE -->
'''
I then asked ChatGPT to solve the task using the following.
Task: Write a program that prints the integers from `1` to `100` (inclusive). But:
- for multiples of three, print Fizz (instead of the number),
- for multiples of five, print Buzz (instead of the number), and
- for multiples of both three and five, print FizzBuzz (instead of the number).
You may find the following Roc functions useful;
- `Num.powInt : Int a, Int a -> Int a`
- `Num.mul : Num a, Num a -> Num a`
- `Num.sub : Num a, Num a -> Num a`
- `Num.isGt : Num a, Num a -> Bool`
- `Num.divCeil : Int a, Int a -> Int a`
- `Num.divTrunc : Int a, Int a -> Int a` Divide two integers, truncating the result towards zero
- `Num.rem : Int a, Int a -> Int a` Obtain the remainder (truncating modulo) from the division of two integers
- `Str.joinWith : List Str, Str -> Str` to concatenate `Str` values
Note that Roc uses `Bool.true` and `Bool.false` instead of `True` and `False`, and Roc uses `when ... is` instead of `case .. of`.
Sometimes I have had to prompt for tests and documentation comments for functions using something like the following.
In Roc you can write documentation comments using `##`.
Also you can write a unit test using the `expect` function like `expect 2 + 3 == 5`.
Write a documentation comment and some relevant units test for the following function.
The first and second apps needed a few rounds of feedback, and some major changes to get them to compile with roc check. The third and fourth were both pretty close to perfect on the first try. Both had a some minor issues to resolve manually. These were mostly due to having not being trained on Roc syntax and so it was using Elm or Haskell instead I think.
It feels likely that with more examples to feed it, or some training, ChatGPT could probably write simple Roc apps without too much trouble. I look forward to trying this again when my usage limit is lifted, and see how close I can get to a working app first try.
Luke Boswell (Apr 09 2023 at 04:15):I should probably mention this was using GPT-4.
Luke Boswell (Apr 09 2023 at 04:21):@Anton would love your thoughts on how I could adapt these examples so they may be suitable for training LLMs in the future. It would be useful to slowly build towards a dataset that can be used in that way. I guess one of the challenges is writing code for a particular platform? Maybe all the examples in a data set should be purely Roc language syntax only, and then have some way to separate out the platform specific code? Or if it is easy enough just try and keep all the examples on the latest cli-platform and re-train if there are any breaking changes.
Anton (Apr 10 2023 at 07:49):Interesting stuff, it would be nice to set this up with Auto-GPT but the GPT-4 API is still with a waitlist.
I think the following prompting strategy could work well; take 3 relatively short python examples for well-known algorithms along with their translation in roc. Make sure to choose the examples so that a lot of roc syntax is demonstrated. Finally provide the algorithm that you want to create in roc in python and ask to translate it to roc. Experimenting with alternatives to python like elm/haskell/scala may also yield improvements.
Anton (Apr 10 2023 at 07:53):@Anton would love your thoughts on how I could adapt these examples so they may be suitable for training LLMs in the future
I wouldn't overthink it here, we already include the version (link) of the platform in every app file so it will be able to pick up on that for breaking changes. We mostly need a lot of examples of good quality with some tests that ensure correctness of the implementation.
Brendan Hansknecht (Apr 10 2023 at 17:32):Bard from google is now allowed to code. I might test it with some of these examples to see how it performs.
Brendan Hansknecht (Apr 10 2023 at 17:32):It obviously doesn't know Roc, but hopefully it would also try.
Last updated: Jul 06 2025 at 12:14 UTC