Patterns don't make sound
Muse, and what happened when I tried to turn Ableton's idea of time into a language.
There's a clone of Tidal sitting in my projects folder, and the short version of why Muse exists is that I like Tidal, I like Ableton, and the two of them disagree about almost everything.
In Tidal you type d1 $ sound "bd*4" and a kick drum starts. The pattern is the sound. That's wonderful for about twenty minutes, and then you want a second section, or a bass that only comes in at bar 17, or to try a clip over the arrangement without committing to it, and you end up rebuilding Ableton out of d2 through d9 and a lot of personal discipline. Ableton has the opposite problem. The model is right: tracks own instruments, clips live on tracks, there's one timeline, and a session grid you can improvise on top of it. You just build all of it with a mouse.
So the first thing I committed was a scratch file of what I actually wanted. It's still in the repo, and it opens with a note to myself:
random notes, this is just a scratch pad like really random notes. needs a separate copy that is more structured.
Most of it is lists of nouns in curly braces, which is how I think when I don't know the shape of something yet: a surface language, a temporal engine, a sound engine, an orchestration engine, a concurrency model with five threads in it. Near the top, though, I wrote seven rules, and those ended up being the whole project:
1. All semantic time is rational.
2. Tracks are the unit of audible context.
3. Patterns are not audible by themselves.
4. Regions and clip activations map local time to global time.
5. Session and Arrangement share tracks but produce activations differently.
6. The audio thread renders only already-validated runtime state.
7. Failed edits never replace the last valid running state.
The first one reads like pedantry. Everything else stands on it.
Music is full of thirds. A triplet is a third of a beat, a quintuplet is a fifth, and once patterns can be stretched, reversed and nested inside other patterns, those fractions multiply into each other. A float gets a third slightly wrong every single time, and a loop that runs for an hour adds the error up for an hour. So every musical position in Muse is an exact rational number of beats: bar positions, durations, clip offsets, loop lengths, quantization boundaries. I was blunt in my notes about where floats are allowed:
only places where floating point time appears is when musical time is converted to:
{
seconds
sample frames
audio params
DSP math
}
That was the plan, and the same file already had a hole in it. Under "patterns" I'd listed what a pattern could be: a reusable description, a state machine, and then
QueryPattern = interval-query interpretation, maybe later
"Maybe later" lasted about as long as it took to write down the rest of what I wanted, which is to say it didn't survive the first commit. A state machine is the easy model. A pattern is a process, you ask it for the next event, it hands you one. SuperCollider works roughly this way and it's lovely to write. But I also wanted one global timeline that every track and clip shares, patterns with their own local timelines nested inside it, an arrangement you can zoom into, and edits that land while the music is still playing. Every one of those asks the same question: what happens between bar 3 and bar 4? A process can only answer by running from the beginning until it gets there.
So I made the meaning of a pattern a function from a window of time to the events inside it. That idea is Alex McLean's, and it sits underneath both Tidal (McLean, 2014) and Strudel (Roos and McLean, 2023):
Pattern a = QueryInterval -> List<Occurrence a>
In the Rust, an occurrence looks like this:
pub struct Occurrence<T = ValueRef> {
pub whole: PatternSpan,
pub part: PatternSpan,
pub value: T,
pub id: StableOccurrenceId,
pub provenance: Provenance,
pub source_order: SourceOrder,
}
whole and part are the clever bit, and they're Tidal's. If a note lasts from beat 1 to beat 3 and you ask for beats 2 to 4, you get the note back with whole covering 1–3 and part covering 2–3, so whoever asked knows they're looking at the back half of something that started earlier and shouldn't trigger it again. The id matters later, in the browser. The provenance is a source span, so every note that sounds can point back at the characters in the editor that made it. The runtime is still free to compile a pattern into a state machine or a lookup table when that's faster. The query is the meaning, and the runtime can compute it however it likes.
Once patterns are functions of time, they're also just values, and that's where the title comes from. Nothing in Muse makes sound unless it has been put on a track that has an instrument, at some point on the global timeline. Here's a cut-down version of the demo I ended up with:
project {
tempo 128 bpm
meter 4/4
track kick {
instrument synth("kick")
effects [gain(-6 dB), saturation(drive: 22 percent)]
}
}
let kick_four = sound "kick*4"
session {
clip kick.main on kick {
quant 1 bar
loop 1 bar
pattern kick_four
}
}
arrangement {
at bar 0 place kick.main for 4 bars on kick
}
kick_four on its own is silent. It's four kicks per cycle in the abstract, and you could hang it on three tracks and get three different sounds. It becomes audible through a clip that lives on a track, and the clip plays when the session launches it or the arrangement places it. That much is Ableton's model. The part that needed real care is the one Ableton never makes you think about: what happens when both of them want the same track at the same time.
Session and arrangement are separate producers of activations, and each track has exactly one owner at any moment: the arrangement, one specific session clip, muted, or an explicit hybrid policy that I reserved and haven't designed. Nothing mixes implicitly. Launch a clip over a track the arrangement is playing, and the clip takes the track. "Back to arrangement" is a state transition back. In Ableton that's the little orange button. In a language it has to be a rule, because otherwise two producers write into one instrument and the result depends on whichever order the scheduler happened to run them in.
The seventh rule is the one that matters on stage. Live coding means typing into a running program in front of people, and you will make typos. So a new version of the program has to get through parsing, name resolution, type and shape checks, asset loading and device validation before it's allowed anywhere near the running one. If it fails at any of those, the old one keeps playing and you get a diagnostic.
Even a valid edit leaves a question about the music that's already playing. I diff the old program against the new one, sort the changes into kinds (a parameter moved, an automation stream changed, a device got swapped, routing changed, a pattern changed shape), and make one decision:
let decision = if classes
.iter()
.any(|class| matches!(class, DiffClass::PatternShapeChange(_)))
{
StatePreservationDecision::Reject
} else if classes.iter().all(|class| {
matches!(
class,
DiffClass::NoChange
| DiffClass::ParamUpdate(_)
| DiffClass::AutomationStreamUpdate(_)
| DiffClass::LiveControlUpdate(_)
| DiffClass::DeviceParamUpdate(_)
| DiffClass::PatternTransformUpdate(_)
)
}) {
StatePreservationDecision::Preserve
} else {
StatePreservationDecision::RestartAtBoundary
};
Turn a knob and nothing restarts. Swap an instrument or reroute a track, and the change waits for the next boundary. Change the shape of a clip's pattern, and I block the edit outright. The old program keeps playing and a diagnostic says which clip changed shape. That's the bluntest answer available, and I'd like a finer one eventually, since plenty of shape changes are probably safe to take live.
All of that ran against a Rust reference backend first. Then I wanted to hear it in a browser, and that's where the interval query paid for itself. WebAudio wants sounds scheduled slightly ahead of time, so the browser runner keeps asking the pattern what happens in the next window. The stable occurrence ids make asking twice harmless: if two overlapping windows both return the same note, it has the same id and gets scheduled once.
Scheduling ahead also bit me, three times in one evening. First I found the voice limit counting notes that were scheduled and hadn't started yet, so a bass line queued ahead would steal voices from itself, and the limit was shared across every instrument instead of applying per device. Once that was fixed, stop turned out to be the next problem, so I gave every voice its own output gate that stop can close at an exact time, including on voices whose notes haven't started yet. Then a voice that had been stolen could fall out of the list stop walks, and I had to keep it reachable.
Somewhere in the middle of that last evening I also rewrote the browser smoke test in G minor. I don't have a technical justification for that one.