Stream: beginners

Topic: Roc and realtime audio signal processing

Joe Salmon (Apr 12 2024 at 07:40):

I have been toying with the idea of using Roc for developing realtime audio applications - has anyone experimented in this area? If so how successful was it? In its most basic form this could look like using a lib like Portaudio in a platform and using Roc to generate the callback function that gets called with each sample tick. I’m a pretty inexperienced C developer though and I don’t yet fully understand the shape of the function that Roc outputs and the overhead it incudes that could impact performance in a tight audio loop. I’m keen to hear others experiences and share my own too, when the time comes!

Luke Boswell (Apr 12 2024 at 07:43):

Sounds very doable. I would reach for a zig host, and call into portaudio to write the platform.

I've written a little about developing a platform roc-ray-experiment article. There may be something there you could use.

Luke Boswell (Apr 12 2024 at 07:43):

I've got a few zig platforms we could probably hobble together to get something basic working.

Joe Salmon (Apr 12 2024 at 07:45):

Fantastic - thanks Luke! I’ll take a look.

Luke Boswell (Apr 12 2024 at 07:45):

It is's most basic form you could just have the main returned by roc be that callback function. Do you know what types you would like to work with?

Joe Salmon (Apr 12 2024 at 07:51):

A basic mono signal function (like a distortion lookup table) might be float -> float. I things might get more complicated when they need to use internal buffers to store samples in memory, such as a delay.

Joe Salmon (Apr 12 2024 at 07:54):

I would probably start with a simple main function that takes a float, does some simple transformation on it and returns the resulting float

Luke Boswell (Apr 12 2024 at 07:55):

That sounds like a really simple place to start.

Luke Boswell (Apr 12 2024 at 07:56):

If you need any assistance let us know on zulip. Also @Brendan Hansknecht will be interested I'm sure.

Joe Salmon (Apr 15 2024 at 09:54):

So I've implemented this in C and I have successfully managed to pass audio buffers through Roc in realtime. However the roc__mainForHost_1_exposed_generic(&rocOut, &rocIn); function is leaking memory. Does anyone have any clues as to why? Any suggestions appreciated!

It's also worth noting that the roc_alloc() and roc_dealloc() functions us malloc() and free(), which isn't recommended for sample accuracy. I don't think this is what is causing memory to leak however.

Here's my host.c:

#include <errno.h>
#include <signal.h>
#include <stdbool.h>
#include <stdio.h>
#include <portaudio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include <unistd.h>
#include <math.h>
#include <execinfo.h>

#ifdef _WIN32
#else
#include <sys/shm.h>  // shm_open
#include <sys/mman.h> // for mmap
#include <signal.h>   // for kill
#endif

#define SAMPLE_RATE 44100
#define BLOCK_SIZE 256
#define NUM_CHANNELS 1 // Adjust for mono (1) or stereo (2)

// Roc memory management

void *roc_alloc(size_t size, unsigned int alignment)
{
  return malloc(size);
}

void *roc_realloc(void *ptr, size_t new_size, size_t old_size, unsigned int alignment)
{
  return realloc(ptr, new_size);
}

void roc_dealloc(void *ptr, unsigned int alignment)
{
  free(ptr);
}

void roc_panic(void *ptr, unsigned int alignment)
{
  char *msg = (char *)ptr;
  fprintf(stderr,
          "Application crashed with message\n\n    %s\n\nShutting down\n", msg);
  exit(1);
}

// Roc debugging
void roc_dbg(char *loc, char *msg, char *src)
{
  fprintf(stderr, "[%s] %s = %s\n", loc, src, msg);
}

void *roc_memset(void *str, int c, size_t n)
{
  return memset(str, c, n);
}

int roc_shm_open(char *name, int oflag, int mode)
{
#ifdef _WIN32
  return 0;
#else
  return shm_open(name, oflag, mode);
#endif
}

void *roc_mmap(void *addr, int length, int prot, int flags, int fd, int offset)
{
#ifdef _WIN32
  return addr;
#else
  return mmap(addr, length, prot, flags, fd, offset);
#endif
}

int roc_getppid()
{
#ifdef _WIN32
  return 0;
#else
  return getppid();
#endif
}

// Define the structure of the audio I/O buffer the Roc will work
struct RocList
{
  float *data;
  size_t len;
  size_t capacity;
};

// Define the Roc function
extern void roc__mainForHost_1_exposed_generic(struct RocList *outBuffer, struct RocList *inBuffer);

// Audio loop callback function
static int callback(const void *in,
                    void *out,
                    unsigned long framesPerBuffer,
                    const PaStreamCallbackTimeInfo *timeInfo,
                    PaStreamCallbackFlags statusFlags,
                    void *userData)
{

  // Declare Roc's input and output buffers
  struct RocList rocIn = {(float *)in, framesPerBuffer, framesPerBuffer};
  struct RocList rocOut;

  // Call the main Roc function
  // Possibly leaking memory
  roc__mainForHost_1_exposed_generic(&rocOut, &rocIn);

  // Copy the output from the Roc buffer to the audio codec output buffer
  memcpy(out, rocOut.data, framesPerBuffer * sizeof(float));

  return paContinue;
}

int main()
{
  PaError err;
  PaStream *stream;

  // Initialize PortAudio
  err = Pa_Initialize();
  if (err != paNoError)
  {
    fprintf(stderr, "PortAudio error: %s\n", Pa_GetErrorText(err));
    return 1;
  }

  // Open a stream for playback and recording
  err = Pa_OpenDefaultStream(&stream, NUM_CHANNELS, NUM_CHANNELS, paFloat32, SAMPLE_RATE, BLOCK_SIZE, callback, NULL);
  if (err != paNoError)
  {
    fprintf(stderr, "PortAudio error: %s\n", Pa_GetErrorText(err));
    Pa_Terminate();
    return 1;
  }

  // Start, wait for user input, stop, close and terminate PortAudio
  err = Pa_StartStream(stream);
  if (err != paNoError)
  {
    fprintf(stderr, "PortAudio error: %s\n", Pa_GetErrorText(err));
    Pa_CloseStream(stream);
    Pa_Terminate();
    return 1;
  }
  printf("Press any key to stop...\n");
  getchar();
  err = Pa_StopStream(stream);
  if (err != paNoError)
  {
    fprintf(stderr, "PortAudio error: %s\n", Pa_GetErrorText(err));
  }
  Pa_CloseStream(stream);
  Pa_Terminate();

  return 0;
}

platform main.roc:

platform "audioPlatform"
    requires {} { main : List F32 -> List F32 }
    exposes []
    packages {}
    imports []
    provides [mainForHost]

mainForHost : List F32 -> List F32
mainForHost = \inputBuffer -> main inputBuffer

and my Roc app:

app "audioPlatformTest"
    packages { pf: "platform/main.roc" }
    imports []
    provides [main] to pf

main : List F32 -> List F32
main = \inputBuffer ->

    # Map an identity function
    List.map
        inputBuffer
        (\a ->

            a * 0.5
        )

Luke Boswell (Apr 15 2024 at 10:50):

Could it be something like Roc is expecting you to free the output RocList?

Luke Boswell (Apr 15 2024 at 10:51):

I'm guessing Roc will free the input RocList.

Luke Boswell (Apr 15 2024 at 10:51):

This is definitely something I have very little confidence in.

Brendan Hansknecht (Apr 15 2024 at 14:38):

Yeah

Brendan Hansknecht (Apr 15 2024 at 14:38):

Need to free the output list

Joe Salmon (Apr 15 2024 at 19:02):

Thanks both - I did try calling ‘free(rocOut.data)’ after the memcpy line in the callback but got a ‘pointer being freed was not allocated’ error

Brendan Hansknecht (Apr 15 2024 at 19:05):

Yeah, roc lists are a tad complicated. Currently only rust and zig have nice libraries for handling them.

Brendan Hansknecht (Apr 15 2024 at 19:06):

As a quick hack, try free(rocOut.data - size_of(size_t))

Joe Salmon (Apr 15 2024 at 23:50):

Thanks Brendan - no luck with this either. I'll see if I can address it using a different memory management strategy.

Brendan Hansknecht (Apr 16 2024 at 01:19):

Oh, the input array isn't being initialized in a valid way. That likely also causes issues here.

Joe Salmon (Apr 16 2024 at 01:23):

OK - how so?

Brendan Hansknecht (Apr 16 2024 at 01:24):

I'm on a plane only typing with my phone. Can't to dive into details rn. Sorry.

Brendan Hansknecht (Apr 16 2024 at 01:26):

The issue fundamentally is that it has no refcount so roc will grab random data before the beginning of the array and assume it is the refcount.

Brendan Hansknecht (Apr 16 2024 at 01:26):

Probably what you want to do is pass it in as a seamless slice so that you don't need to copy around any data. I'll share code when I have the chance.

Joe Salmon (Apr 16 2024 at 01:37):

Thank you!

Brendan Hansknecht (Apr 16 2024 at 16:39):

This is an example of passing a seamless slice to roc. By doing so, you retain control of the allocation and must free it. Roc essentially takes it in as read only: https://github.com/bhansconnect/roc-fuzz/blob/20388c307d7328168ebae4bfda2ff99bb10f2bc7/platform/host.cpp#L293-L296

Brendan Hansknecht (Apr 16 2024 at 16:40):

Avoids needing to create a new allocation, add a refcount, clone the data, and pass that into roc.

Brendan Hansknecht (Apr 16 2024 at 16:48):

As an extra note, if you truly need to avoid all allocations for performance, there are ways to enforce that, but it is more complex. You would need to control that buffer that pa is using and then correctly pass it in uniquely to roc. After that, you would block all allocations reporting an error to make sure that the list is edited in place and never duplicated. Possible, definitely faster, but more wiring to do right. I can help with that if you hit perf issues and find it needed.

Joe Salmon (Apr 16 2024 at 21:35):

Thanks for sharing this - really useful! I've tried applying the seamless slice approach. However the memory leak still persists. Please let me know if I'm applying the seamless slice incorrectly.

Here's the updated audio callback (including the definitions for RocList etc.) :

// Define the structure of the audio I/O buffer the Roc will work
struct RocList
{
  float *data;
  size_t len;
  size_t capacity;
};

const size_t SLICE_BIT = ((size_t)1) << (8 * sizeof(size_t) - 1);
const size_t REFCOUNT_MAX = 0;

// Define the Roc function
extern void roc__mainForHost_1_exposed_generic(struct RocList *outBuffer, struct RocList *inBuffer);

// Audio loop callback function
static int callback(const void *in,
                    void *out,
                    unsigned long framesPerBuffer,
                    const PaStreamCallbackTimeInfo *timeInfo,
                    PaStreamCallbackFlags statusFlags,
                    void *userData)
{
  size_t rc = REFCOUNT_MAX;
  size_t slice_bits = (((size_t)&rc) >> 1) | SLICE_BIT;

  // Declare Roc's input and output buffers
  struct RocList rocIn = {(float *)in, framesPerBuffer, slice_bits};
  struct RocList rocOut;

  // Call the main Roc function
  // Currently leaking memory
  roc__mainForHost_1_exposed_generic(&rocOut, &rocIn);

  // Copy the output from the Roc buffer to the audio codec output buffer
  memcpy(out, rocOut.data, framesPerBuffer * sizeof(float));

  return paContinue;
}

Joe Salmon (Apr 16 2024 at 21:58):

Logging from roc_alloc()and. roc_dealloc()shows that new memory is being allocated with each call to roc__mainForHost_1_exposed_generic() but not deallocated, hence the leak. I don't want to be dynamically allocating and deallocating memory inside an audio callback anyway so I need to explore another approach I think.

Brendan Hansknecht (Apr 16 2024 at 22:04):

You still have to free rocOut as well.

Joe Salmon (Apr 17 2024 at 03:25):

I have tried that (including your free(rocOut.data - size_of(size_t)) suggestion) but it was never allocated via malloc() so I get an error. There may be some other way though that I'm not aware of. Sorry - I'm really hitting the limits of my knowledge of systems-level programming here!

Luke Boswell (Apr 17 2024 at 04:29):

I imagine rocOut.data is allocated by roc using roc_alloc() and so it needs to be freed after the memcpy? I'm not sure, I haven't done any C in many years.

Luke Boswell (Apr 17 2024 at 04:31):

So can you call roc_dealloc(&rocOut, 0); maybe?

Brendan Hansknecht (Apr 17 2024 at 04:31):

I'm currently away on a business trip. When I get back, I'll definitely take a look. If you can put up a repo or gift with the full code, that would be helpful

Brendan Hansknecht (Apr 17 2024 at 04:32):

Should be able to take a look on Thursday or Friday

Joe Salmon (Apr 17 2024 at 04:55):

Thanks Brendan - here's the repo if you would like to take a closer look. I really really appreciate your help on this!

Joe Salmon (Apr 17 2024 at 04:57):

Luke Boswell said:

roc_dealloc(&rocOut, 0);

Thanks Luke - yeah I did try that and received one of these:
main(82727,0x16d78f000) malloc: *** error for object 0x16d78daa0: pointer being freed was not allocated

Luke Boswell (Apr 17 2024 at 05:19):

I wanted to test this out, also made a test repository. :smiley:

Luke Boswell (Apr 17 2024 at 05:20):

I have only copied your code above, so haven't investigated the leak or anything yet

Brendan Hansknecht (Apr 17 2024 at 05:21):

Would be good to double check that skipping the roc call is free from memory leaks.

Luke Boswell (Apr 17 2024 at 05:55):

Can confirm there is no leak if I comment out roc__mainForHost_1_exposed_generic(&rocOut, &rocIn);

Luke Boswell (Apr 17 2024 at 05:56):

Just running Instruments to check things. Took me a little while to figure out that I need to sign the binary before Instruments can target it.

Luke Boswell (Apr 17 2024 at 05:58):

Not calling roc

Screenshot-2024-04-17-at-15.57.05.png

With the call to roc

Screenshot-2024-04-17-at-15.57.59.png

Joe Salmon (Apr 17 2024 at 07:55):

Yup! This is pretty much what I'm seeing too. Roc doesn't seem to be deallocating anything?

Luke Boswell (Apr 17 2024 at 08:23):

Solved it

Luke Boswell (Apr 17 2024 at 08:23):

:smiley:

Luke Boswell (Apr 17 2024 at 08:24):

void *subtract_from_pointer(void *ptr) {
    char *char_ptr = (char *)ptr;  // Cast to char* to perform byte-wise arithmetic
    char_ptr -= 0x8;               // Subtract 8 bytes
    return (void *)char_ptr;       // Cast back to void* if needed
}

// then we can free the pointer
void* ptr = subtract_from_pointer(rocOut.data);
free(ptr);

Luke Boswell (Apr 17 2024 at 08:25):

rocOut.data actually points 1 byte ahead of where the pointer that is allocated

Luke Boswell (Apr 17 2024 at 08:26):

I'm sure there is a better explanation, but at least now I have a program without the memory leak.

Joe Salmon (Apr 17 2024 at 09:36):

That's fixed it - amazing! Thanks so much!

Joe Salmon (Apr 17 2024 at 09:43):

Otherwise, I'm thrilled to see how little overhead Roc adds to the audio loop (so far!). It will interesting to see how it performs once I start throwing some more complex computations in there. But for now I have a pure functional app that is processing realtime audio, which feels pretty exciting!

Brendan Hansknecht (Apr 17 2024 at 12:52):

Ah, that's what I missed. It is a void*. So shifting it didn't work.

Last updated: Jul 06 2025 at 12:14 UTC