Multi-Sample Granular Synth Concept

Motivation & Concept

Most granular synthesizers and processors use a single stereo sample or input signal. Using grains from multiple audio sources would open up numerous creative possibilities, more lushous textures, bigger sounding pads, or complex rhythmic patters.

To bring this concept to life in hardware, an RP2350 controller will be used. It may not be as fast as other well known embedded audio-development boards, such as the Teensy 4.1, or the Daisy Seed3, but it has the advantage of a dual core architectur and PIO co-processors. This allows it to multi-task more effectively without suffering from audiodropouts. Loading audio samples from an SD-card on one core, while doing audio processing on the other core mitigates audio dropouts. Core 0 loads audio from the SD-card into several ping-pong window buffers. This allows audio sample windows to be played from one buffer A, while loading is done into the other buffer B. The playback can be swapped to buffer B after loading is done, so buffer A is available for loading.
The audio data is streamed from the playback-ready window-buffers, into core 1 through a multi-voice granular engine. Additional audio-fx and processing (Pure Data/HVCC) is also done on core 1, and finally sent over I2S to a DAC for audio playback.

Software Architecture

flowchart TB
    %% Styling and Theme definitions (Matching Minimal Black)
    classDef default fill:#111,stroke:#444,stroke-width:1px,color:#eee;
    classDef coreBox fill:#1a1a1a,stroke:#555,stroke-width:2px,color:#fff,stroke-dasharray: 5 5;
    classDef buffer fill:#241432,stroke:#8a2be2,stroke-width:2px,color:#fff;
    classDef hardware fill:#0d0d0d,stroke:#333,stroke-width:1px,color:#888;
    classDef hvccBox fill:#1b2a47,stroke:#4169e1,stroke-width:2px,color:#fff;
    classDef midiBox fill:#2e1a1a,stroke:#b22222,stroke-width:1px,color:#fff;

    %% Target internal Mermaid arrow paths and labels to darken them
    classDef link stroke:#444!important,stroke-width:1px!important;
    classDef edgePath stroke:#444!important,stroke-width:1px!important;
    classDef edgeLabel background-color:#111!important,color:#888!important;

    %% Input Stage (MIDI Input Only)
    subgraph Inputs [MIDI Input Stage]
        USB_MIDI[USB MIDI Device / Host]:::hardware
        HW_MIDI[Hardware MIDI Port 
 Optocoupler UART]:::hardware
        MIDI_Mux{ }:::hardware
    end

    %% Hardware / Storage Layer
    subgraph Storage [Storage]
        SD[SD Card 
 Audio Samples]:::hardware
    end

    %% Core 0 Layout
    subgraph Core0 [Core 0: C++ Input & Disk I/O]
        FileIO[SD Card Reader / File Streamer]
        
        subgraph MIDIEngine [MIDI Processing Engine]
            MPE_Parser[MPE Parser 
 Per-Note Pitch Bend / Aftertouch]:::midiBox
            ClockSync[MIDI Clock Synchronizer 
 Grain Length / Tempo Sync]:::midiBox
        end
        
        subgraph Engine [Double-Buffered Multi-Sample Streaming Engine]
            direction LR
            subgraph S1 [Sample Stream 1]
                S1_A[Ping-Pong A]:::buffer <--> S1_B[Ping-Pong B]:::buffer
            end
            subgraph S2 [Sample Stream 2]
                S2_A[Ping-Pong A]:::buffer <--> S2_B[Ping-Pong B]:::buffer
            end
            subgraph S3 [Sample Stream 3]
                S3_A[Ping-Pong A]:::buffer <--> S3_B[Ping-Pong B]:::buffer
            end
            subgraph S4 [Sample Stream 4]
                S4_A[Ping-Pong A]:::buffer <--> S4_B[Ping-Pong B]:::buffer
            end
        end
    end

    %% Core 1 Layout
    subgraph Core1 [Core 1: Arduino-Pico Real-time Audio DSP]
        Grain_Engine[Grain Engine]
        
        subgraph HVCC_Core [HVCC Audio Effects Pipeline]
            PD_FX[Pure Data / Plug Data Patch 
 Heavy Compiler Generated C++]:::hvccBox
        end
    end

    %% External Output Layer
    subgraph Output [Output Stage]
        I2S[I2S Protocol Bus]:::hardware
        DAC[External DAC]:::hardware
        Speakers([Audio Out]):::hardware
    end

    %% Unified Input Connection
    USB_MIDI --> MIDI_Mux
    HW_MIDI --> MIDI_Mux
    MIDI_Mux -->|Unified MIDI In Stream| MIDIEngine
    
    %% Storage & Audio Generation
    SD -->|SPI / SDIO High Latency Fetch| FileIO
    FileIO -->|Fills inactive buffers| Engine
    
    %% Framework Concurrency Flow
    Engine -->|Shared Memory Audio Buffers| Grain_Engine
    MIDIEngine -->|Thread-Safe Control Parameters| Grain_Engine & PD_FX
    
    %% Processing Pipeline
    Grain_Engine --> PD_FX
    PD_FX -->|DMA Target / Low Latency| I2S
    I2S --> DAC
    DAC --> Speakers

    %% Apply Classes
    class Core0,Core1 coreBox;