Skip to content

A function, played and read back

Turn a function into sound, then analyse the recording and compare what you recover with the original. We'll play the same curve as loudness and as pitch, and put the recording, measurements and a certified range (bounds on the function's values that interval arithmetic proves, not samples) on one timeline.

afield.math.functions.Functions builds the function and FnInterval certifies its range. A Dough graph (Field's audio signal graph) plays it, rendered offline to a 16-bit .wav; PatOnsetDetection and PatPitchDetection read the file back. The time-axis viewer brings the results together.

Most of what changes on the way is delay: one from how the function reaches the signal graph and one from how the pitch detector reports time. Both can be measured to the sample.

The function, and two ways to hear it

FnSoundDocs.contour is three seconds of a slow curve: a resting level, a broad swell, a step up to a plateau that wobbles at 6 Hz, and one bump with a 3 ms sigma, narrower than either analysis window. It is built only from ops that have a formula, so every interval of it has a certified range.

val f = with(Functions) {
    val d = Functions.Domain(0.0, 3.0)
    c(0.2, d) +
        gaussian(0.8, 0.2, 0.5, d) +
        smoothStep(1.4, 2.0, d) * (c(0.3, d) + sin(6.0, amplitude = 0.08)) +
        gaussian(2.6, 0.003, 0.25, d)
}

It is played two ways. As loudness, it is the amplitude of a sine at 375 Hz, a frequency chosen so one period is exactly 128 samples at 48 kHz. As pitch, it is the frequency of a sine at constant level, \( 220 \cdot 2^f \) Hz, so the curve spans about half an octave. Neither render is normalised, so the amplitude in the file is the amplitude the function asked for, and both analyses read the file back from disk.

Here is the loudness render:

And the pitch render, the same curve as a glide:

Two bridges from an Fn to a signal

The first way to bring an Fn into a Dough graph is a Param whose supplier reads it, Param { f(time.toDouble()).toFloat() }. A supplier is evaluated once per audio block, at the block's start, and Param ramps from the previous block's value to this one across the block. So at time t the audio hears, to first order, the function at t minus one block.

FnSignal, reached as f.toSignal(), is the per-sample alternative: sample i of a block is f(time + i / sampleRate). Both routes are measured below, and the clips above use toSignal(). In code, the two look like this:

val perSample = SineOsc(Param(375f)) * f.scaleY(0.8).toSignal()
val perBlock = SineOsc(Param(375f)) * Param { 0.8f * f(time.toDouble()).toFloat() }

What came back

The loudness is recovered by patEnergyEnvelope, an RMS per hop, with a window two periods of the 375 Hz sine long. Over a whole number of periods a steady sine's RMS is exactly its amplitude over √2, so the window adds no ripple of its own. The pitch is recovered by patF0Frames, the YIN estimator, told the pitch range to expect. The box below draws the function, both recovered curves and their errors on one timeline, with the loudness render underneath.

:::static
import afield.audio.synth.FnSoundDocs.Encoding

/*
Each roundTrip renders through FnSignal, writes a 16-bit .wav with no normalisation, reads that
file back, and analyses what it read. `inside` is checked against the proven bounds of
certifiedRange over the samples each frame read, not against a sampled curve.
*/
val loud = FnSoundDocs.roundTrip(Encoding.LOUDNESS)
val pitch = FnSoundDocs.roundTrip(Encoding.PITCH)
val f = loud.fn

:::fold the viewer, the audio, and the numbers
val sync = "chain-fn-roundtrip"
val viewer = SignalViewer(
    [
        certifiedSignalChannel("f, and its loudness read back", f, 0.1, 0.95).copy(comparison = loud::recoveredAt),
        certifiedSignalChannel("f, and its pitch read back", f, 0.1, 0.95).copy(comparison = pitch::recoveredAt),
        SignalChannel("loudness − f", loud::errorAt, -0.1, 0.1),
        SignalChannel("pitch − f", pitch::errorAt, -0.1, 0.1),
    ],
    0.0,
    FnSoundDocs.SECONDS,
    sampleStep = 1.0 / 4000,
    gaps = true,
)
displaySignal(
    viewer,
    "viewer",
    SignalViewerOptions(
        width = 1100,
        laneHeight = 120,
        label = "one function, played two ways and read back",
        comparisonLabel = "read back from the .wav",
        syncGroup = sync,
        viewKey = "chain-fn-roundtrip:viewer",
    ),
)
displayAudioBytes(
    loud.wav.readBytes(),
    channel = "audio",
    durationSeconds = FnSoundDocs.SECONDS,
    label = "the loudness render, as analysed",
    options = AudioVisualizerOptions(width = 1100, syncGroup = sync),
)

fun row(label: String, t: FnSoundDocs.RoundTrip) =
    "| $label | ${t.measured.size} of ${t.frames.size} | ${t.rmsError.f5} | ${t.maxAbsError.f4} at ${t.maxErrorAtSeconds.f3} s | " +
        "${t.insideCount} of ${t.measured.size}, worst ${"%.1e".format(t.worstOutside)} outside | ${t.blip.survival.f2} |"
markdown(
    "readout",
    """
| read back from | frames | RMS error | worst error | inside the certified range | of the bump's height |
|---|---|---|---|---|---|
${row("loudness (RMS)", loud)}
${row("pitch (YIN)", pitch)}
""",
    width = 1080,
)
:::endfold

A four-lane time-axis viewer over three seconds. The top two lanes draw the function in white, a slow swell, a step up to a wobbling plateau and one narrow spike, with its certified band behind it, too tight to separate from the line at this zoom, and the curves recovered from the loudness and from the pitch of the rendered sound drawn dashed on top of it and hidden by it. The two lower lanes are the recovery errors: flat near zero except for a spike at the narrow bump, with a faint ripple along the pitch lane's plateau. Below, an audio player showing the loudness render's waveform, and a table of frame counts, errors, lags and certified-band containment for both encodings

Over the slow parts both recovered curves sit on the function, close enough to hide under it at this zoom, and the error lanes are nearly flat. The worst error of each lies at the narrow bump, and the last column says how much of that bump's height each encoding kept. Neither window is much narrower than the bump, and a window can only report an average over what it read.

The certified band handles this. A windowed measurement cannot be expected to land on the function's value at one instant. It can be expected to land inside the range the function covers over the samples the window read, and certifiedRange bounds that range with a proof rather than a sample grid. Nearly every loudness frame lands inside. The few that do not miss by less than one 16-bit step. Every pitch frame lands inside, but a YIN frame reads nearly three times as many samples as an RMS frame here, and a wider window has a wider range to land in.

Delays, measured to the sample

The harness also asks what delay best aligns each recovered curve with the function, searching two blocks either way at one-sample resolution. The box below plays both encodings through both bridges and zooms in on the narrow bump.

:::static
import afield.audio.synth.FnSoundDocs.Encoding
import afield.audio.synth.FnSoundDocs.Route

// Two encodings through two bridges. bestLagSamples reads f at each frame's reported time minus
// the lag that best aligns the two.
val trips = Encoding.values().flatMap { e -> Route.values().map { r -> FnSoundDocs.roundTrip(e, r) } }
fun trip(e: Encoding, r: Route) = trips.single { it.encoding == e && it.route == r }

:::fold the bump up close, and the four delays
val f = trips.first().fn
val viewer = SignalViewer(
    [
        SignalChannel(
            "loudness through toSignal()",
            f::eval,
            0.35,
            0.8,
            comparison = trip(Encoding.LOUDNESS, Route.PER_SAMPLE)::recoveredAt,
        ),
        SignalChannel(
            "loudness through Param { }",
            f::eval,
            0.35,
            0.8,
            comparison = trip(Encoding.LOUDNESS, Route.PER_BLOCK)::recoveredAt,
        ),
    ],
    0.0,
    FnSoundDocs.SECONDS,
    sampleStep = 1.0 / 48000,
    gaps = true,
)
displaySignal(
    viewer,
    "viewer",
    SignalViewerOptions(
        width = 1100,
        laneHeight = 150,
        label = "the bump at 2.6 s, 30 ms wide",
        comparisonLabel = "read back from the .wav",
        initialStart = FnSoundDocs.BLIP_AT - 0.015,
        initialDuration = 0.03,
        viewKey = "chain-fn-roundtrip:delays",
    ),
)

val rows =
    trips.joinToString("\n") { t ->
        "| ${t.encoding.name.lowercase()} | ${if (t.route == Route.PER_SAMPLE) "`toSignal()`" else "`Param { }`"} | " +
            "${t.rmsError.f5} | ${t.bestLagSamples} | ${t.rmsErrorAtBestLag.f5} |"
    }
markdown(
    "delays",
    """
One audio block is ${FnSoundDocs.BLOCK} samples, ${(1000.0 * FnSoundDocs.BLOCK / FnSoundDocs.SR).f2} ms.

| encoding | bridge | RMS error | best delay, samples | RMS error at that delay |
|---|---|---|---|---|
$rows
""",
    width = 1080,
)
:::endfold

A two-lane time-axis viewer zoomed to thirty milliseconds around the narrow bump. In the top lane the curve read back from the per-sample render sits on the function; in the lower lane the curve read back from the per-block Param render is visibly shifted to the right of it. Below, a table of the four trips giving each one's RMS error, the delay that best aligns it in samples, and the RMS error at that delay

The first delay is the bridge. Through toSignal() the loudness comes back with no delay at all. Through Param { } it comes back exactly one block late, and removing that block removes most of the error. That is the ramp described above, and it is why a sharp feature driven through a supplier arrives a block after it was asked for. Param is built for control values set once a block, and it does that job. It is the wrong bridge for a function whose shape you mean to hear.

The analysis adds no delay of its own. Each pitch frame is reported at PatF0Frame.measuredCenterSample, the centre of the samples YIN compared at the lag it chose, so through toSignal() the pitch delay is within a sample or two of zero. Through Param { } it is one block again.

Where each piece is documented