HarmonicPhaseTracker.kt
src/jvmMain/kotlin/afield/audio/synth/HarmonicPhaseTracker.kt · 715 lines
@file:afield.runtime.completion.CompletionCategory("Synthesis")
package afield.audio.synth
import kotlin.math.PI
import kotlin.math.abs
import kotlin.math.atan2
import kotlin.math.cos
import kotlin.math.exp
import kotlin.math.sin
import kotlin.math.sqrt
Pitch and phase tracking for harmonically rich monophonic input, through HarmonicPhaseTracker.
A bank of complex harmonic demodulators votes on the fundamental's phase velocity, and a decimated autocorrelation search reacquires the period when the bank falls into an octave trap. The tracker is a PhaseTracker, so its state can drive a TrackedPhaseBank.
private const val TWO_PI = 2.0 * PI
class HarmonicPhaseTracker
Track ONE latent periodicity from MANY harmonics of a spectrally complex monophonic input.
This is the analysis front-end to use when a raw waveform is too rich for a single sinusoidal PLL: saws, bowed strings, voice, reeds, distorted basses, signals with a weak or entirely absent fundamental, etc.
The tracker has two cooperating parts.
FINE TRACKER — adaptive complex harmonic demodulators
For harmonic k we mix the real input against exp(-j k φ), where φ is the current master phase, and low-pass the result:
z_k ~= A_k/2 * exp(j (θ_k - kφ))
The ABSOLUTE phase of z_k contains the source's arbitrary timbral phase ψ_k, so this code does NOT average atan2(z_k)/k. That would incorrectly assume all harmonics have a common zero phase.
Instead we look at the phase ADVANCE of each baseband channel:
δ_k = arg(z_k[n] conj(z_k[n-1]))
and every partial votes for a residual fundamental phase velocity
r_k = δ_k / k.
A weighted median followed by a robust weighted mean fuses those votes. Constant harmonic phase offsets disappear under differentiation, while true common pitch movement survives division by k.
Each channel uses a two-pole low-pass whose cutoff grows with harmonic number. Thus each partial has roughly the same capture range when expressed back in fundamental-Hz units. Channel weights come from baseband energy with a mild preference for lower partials; large residual outliers are rejected before fusion.
COARSE TRACKER — periodicity reacquisition
A harmonic bank alone has octave traps: if the current f0 estimate is 2x the truth, channels at 1f0, 2f0 ... may still line up beautifully with even harmonics and report high confidence.
A decimated ring buffer supplies a snapshot for normalized autocorrelation. One lag is evaluated
per audio sample, bounding each step to coarseWindow products rather than a whole search.
Among peaks close to the best one it chooses the shortest lag (highest fundamental). For an ordinary harmonic waveform the first high-quality
repeated period is the fundamental period; later peaks at 2T, 3T ... are deliberately ignored.
This also gives useful missing-fundamental acquisition: a signal containing only 2f0, 3f0 and 5f0 still repeats every 1/f0, so it can seed the fine tracker at f0 without energy at f0.
No algorithm can infer an intentionally "missing" fundamental from an exactly even-only spectrum such as {2f,4f,6f}: that signal is literally periodic at 2f. In that case octave choice requires history, a prior, or a higher-level musical model.
PHASE OUTPUT
state.omega is the stable fused pitch estimate.
state.phaseAdvance is the more agile phase trajectory:
phaseAdvance = omega + phaseFollow * fusedResidual.
That distinction is useful downstream. A TrackedPhaseBank voice with lock=0 follows stable pitch; lock=1 inherits this cycle-level timing; intermediate values continuously interpolate.
state.harmonicity estimates how much recent broadband power is explained by the harmonic
demodulators. state.confidence combines harmonicity with agreement among their fundamental-
velocity votes. state.activePartials says how many channels actually voted.
This is still a MONOPHONIC model: it tries to explain a spectrum with one latent phase φ. Material with two independent pitches wants multiple HarmonicPhaseTrackers (or a future MultiPhaseTracker), not a wider value of harmonics.
declaration — 74 lines, from line 96
class HarmonicPhaseTracker(
override val sampleRate: Double = 48_000.0,
val minHz: Double = 35.0,
val maxHz: Double = 350.0,
val initialHz: Double = 100.0,
/** Number of harmonic demodulators. Channels above Nyquist automatically stop voting. */
val harmonics: Int = 12,
/**
* Fine-track capture bandwidth measured in FUNDAMENTAL Hz.
* Harmonic k receives a baseband LP cutoff of approximately k * this value.
*/
val captureHz: Double = 4.0,
/** Time constant with which the stable omega estimate absorbs the fused residual. */
val frequencyTrackingTimeSeconds: Double = 0.030,
/**
* How much of the instantaneous fused residual appears in phaseAdvance.
* 0 = only stable omega; 1 = strongly cycle-following. Values >1 are allowed experimentally.
*/
var phaseFollow: Double = 1.0,
/** Fundamental-Hz distance from the robust median after which a partial is rejected. */
val residualOutlierHz: Double = 7.0,
/** Mildly prefer low partials: weight is energy / k^harmonicWeightExponent. */
val harmonicWeightExponent: Double = 0.5,
/** Input RMS-ish level below which the tracker is considered gated. */
val gateLevel: Double = 0.01,
/** Baseband partial amplitude needed to cast a vote, relative to [gateLevel]. */
val partialGateScale: Double = 0.35,
// Broadband power follower for harmonicity / gate measurement.
broadbandPowerTimeSeconds: Double = 0.020,
// Confidence smoothing, mostly to prevent lock chatter.
confidenceTimeSeconds: Double = 0.025,
/** Confidence above which [TrackedPhaseState.locked] is true. */
val lockThreshold: Double = 0.65,
// ---- coarse periodicity reacquisition ----
/** Raw samples averaged into one coarse sample. 8 -> 6 kHz at a 48 kHz audio rate. */
val coarseDecimation: Int = 8,
/** Number of downsampled samples in each normalized-correlation comparison. */
val coarseWindow: Int = 384,
/** Re-run coarse correlation this many downsampled samples. */
val coarseHop: Int = 96,
/** Absolute normalized-correlation floor for accepting a periodicity candidate. */
val coarseMinCorrelation: Double = 0.60,
/**
* Accept peaks this close to the best correlation and choose the shortest-lag one.
* This avoids selecting the equally-good 2T / 3T autocorrelation peaks.
*/
val coarseRelativePeak: Double = 0.93,
/** Ignore coarse/fine disagreement smaller than this. Fine tracking has better precision. */
val coarseDeadbandHz: Double = 1.5,
/** Coarse pull while fine confidence is weak. */
val coarseUnlockedPull: Double = 0.65,
/** Coarse pull when the fine bank claims to be confident (principally octave-trap correction). */
val coarseLockedPull: Double = 0.12,
) : PhaseTracker {
override val state = TrackedPhaseState(sampleRate)
init
init {
require(minHz.isFinite() && maxHz.isFinite() && minHz > 0 && maxHz > minHz && maxHz < sampleRate / 2)
require(initialHz.isFinite() && initialHz in minHz..maxHz)
require(harmonics in 1..64)
require(captureHz.isFinite() && captureHz > 0 && residualOutlierHz.isFinite() && residualOutlierHz > 0)
require(coarseDecimation > 0 && sampleRate / coarseDecimation > 2 * maxHz)
require(coarseWindow in 16..8192 && coarseHop > 0)
require(sampleRate / coarseDecimation / minHz < 16384)
require(gateLevel.isFinite() && gateLevel > 0 && partialGateScale.isFinite() && partialGateScale > 0)
require(harmonicWeightExponent.isFinite() && phaseFollow.isFinite())
require(lockThreshold in 0.0..1.0 && coarseRelativePeak in 0.0..1.0 && coarseMinCorrelation in 0.0..1.0)
require(coarseUnlockedPull in 0.0..1.0 && coarseLockedPull in 0.0..1.0)
require(coarseDeadbandHz.isFinite() && coarseDeadbandHz >= 0)
for (seconds in doubleArrayOf(frequencyTrackingTimeSeconds, broadbandPowerTimeSeconds, confidenceTimeSeconds))
require(seconds.isFinite() && seconds >= 0)
}
private val harmonicPreferences = DoubleArray(harmonics) { 1.0 / Math.pow(it + 1.0, harmonicWeightExponent) }
private val minOmega = TWO_PI * minHz / sampleRate
private val maxOmega = TWO_PI * maxHz / sampleRate
private var omega = 0.0
private var phase = 0.0
private val frequencyAlpha =
if (frequencyTrackingTimeSeconds <= 0.0) 1.0
else 1.0 - exp(-1.0 / (frequencyTrackingTimeSeconds * sampleRate))
private val broadbandPowerKeep =
if (broadbandPowerTimeSeconds <= 0.0) 0.0
else exp(-1.0 / (broadbandPowerTimeSeconds * sampleRate))
private val confidenceKeep =
if (confidenceTimeSeconds <= 0.0) 0.0
else exp(-1.0 / (confidenceTimeSeconds * sampleRate))
Two cascaded complex one-poles per harmonic.
private val lp1Re = DoubleArray(harmonics)
private val lp1Im = DoubleArray(harmonics)
private val lp2Re = DoubleArray(harmonics)
private val lp2Im = DoubleArray(harmonics)
private val previousRe = DoubleArray(harmonics)
private val previousIm = DoubleArray(harmonics)
private val previousValid = BooleanArray(harmonics)
1-a coefficients for each harmonic's low-pass: fc_k ~= k*captureHz, bounded so the one-pole remains sensible near Nyquist.
private val lpAdd = DoubleArray(harmonics) { index ->
val k = index + 1
val fc = (captureHz * k).coerceAtMost(sampleRate * 0.20)
1.0 - exp(-TWO_PI * fc / sampleRate)
}
Preallocated robust-fusion scratch. Filled densely from [0, voteCount).
private val residualScratch = DoubleArray(harmonics)
private val weightScratch = DoubleArray(harmonics)
private var broadbandPower = 0.0
private var smoothedConfidence = 0.0
coarse ring / downsampler
private val coarseRate = sampleRate / coarseDecimation
private val coarseMinLag = (coarseRate / maxHz).toInt().coerceAtLeast(2)
private val coarseMaxLag = kotlin.math.ceil(coarseRate / minHz).toInt() + 1
private val coarseRingSize =
nextPowerOfTwo(coarseWindow + coarseMaxLag + 8)
private val coarseMask = coarseRingSize - 1
private val coarseRing = DoubleArray(coarseRingSize)
private val coarseScores = DoubleArray(coarseMaxLag + 2)
Copy one decimated window, then compute one lag per audio sample. No full autocorrelation burst sits on a single sample or requires a worker thread.
private val coarseSnapshot = DoubleArray(coarseWindow + coarseMaxLag)
private var analysisLag = 0
private var coarseWrite = 0
private var coarseCount = 0
private var coarseAccumulator = 0.0
private var coarseAccumulatorCount = 0
private var coarseSinceAnalysis = 0
var coarseHz: Double
Last accepted coarse-periodicity estimate, useful for diagnostics/UI.
var coarseHz: Double = 0.0
private set
var coarseCorrelation: Double
Normalized autocorrelation score of coarseHz.
var coarseCorrelation: Double = 0.0
private set
init { reset() }
override fun process(input: Float): TrackedPhaseState
body — 192 lines, from line 265
override fun process(input: Float): TrackedPhaseState {
val x = if (input.isFinite()) input.toDouble() else 0.0
// Broadband reference power. For a sum of sinusoids this lets us estimate what fraction
// the harmonic channels explain.
broadbandPower =
broadbandPowerKeep * broadbandPower +
(1.0 - broadbandPowerKeep) * (x * x)
// The coarse path runs independently of the fine demodulators and may pull omega if it
// discovers a convincing different periodicity.
pushCoarse(x)
advanceCoarse()
// Generate sin(k phi), cos(k phi) recursively: only one sin/cos pair per audio sample
// regardless of harmonic count.
val c1 = cos(phase)
val s1 = sin(phase)
var ck = c1
var sk = s1
var voteCount = 0
var explainedPower = 0.0
var activePartials = 0
// amplitude of a real sinus is ~2*|complex baseband| after demodulation.
val partialAmplitudeFloor = gateLevel * partialGateScale
val partialMagnitudeSquaredFloor =
0.25 * partialAmplitudeFloor * partialAmplitudeFloor
val nyquist = sampleRate * 0.5
for (index in 0 until harmonics) {
val k = index + 1
val harmonicHz = omega * sampleRate / TWO_PI * k
// Recurrence values ck/sk are still advanced even for channels above Nyquist so the
// loop remains branch-free and simple.
if (harmonicHz < nyquist * 0.98) {
// x * exp(-j k phi)
val mixRe = x * ck
val mixIm = -x * sk
val add = lpAdd[index]
lp1Re[index] += add * (mixRe - lp1Re[index])
lp1Im[index] += add * (mixIm - lp1Im[index])
lp2Re[index] += add * (lp1Re[index] - lp2Re[index])
lp2Im[index] += add * (lp1Im[index] - lp2Im[index])
val re = lp2Re[index]
val im = lp2Im[index]
val magnitudeSquared = re * re + im * im
// For x=A cos(...), |baseband|~=A/2 and sinusoidal average power=A²/2,
// therefore explained power is approximately 2|z|².
explainedPower += 2.0 * magnitudeSquared
if (magnitudeSquared >= partialMagnitudeSquaredFloor) {
activePartials++
if (previousValid[index]) {
val pr = previousRe[index]
val pi = previousIm[index]
val cross = im * pr - re * pi
val dot = re * pr + im * pi
val harmonicResidual = atan2(cross, dot)
// Convert harmonic residual to FUNDAMENTAL radians/sample.
val residual = harmonicResidual / k
// Reject absurd jumps before they can affect the robust estimator.
val hardLimit =
TWO_PI * (captureHz * 2.5) / sampleRate
if (abs(residual) <= hardLimit) {
residualScratch[voteCount] = residual
val harmonicPreference = harmonicPreferences[index]
weightScratch[voteCount] =
magnitudeSquared * harmonicPreference
voteCount++
}
}
previousRe[index] = re
previousIm[index] = im
previousValid[index] = true
} else {
// Do not measure phase advance across a long interval where this partial was
// below the usable floor.
previousValid[index] = false
}
} else {
previousValid[index] = false
}
// harmonic-angle recurrence:
// exp(j(k+1)phi) = exp(jkphi) exp(jphi)
val nextC = ck * c1 - sk * s1
val nextS = sk * c1 + ck * s1
ck = nextC
sk = nextS
}
val gateOpen =
sqrt(broadbandPower.coerceAtLeast(0.0)) >= gateLevel
val broadband = broadbandPower.coerceAtLeast(1e-20)
val harmonicity =
if (gateOpen)
(explainedPower / broadband).coerceIn(0.0, 1.0)
else
0.0
var fusedResidual = 0.0
var agreement = 0.0
if (gateOpen && voteCount > 0) {
// Sort only the dense scratch prefix. N is normally 8..16, so insertion sort is both
// allocation-free and cheaper than bringing a general collection sort into audio code.
insertionSortPairs(residualScratch, weightScratch, voteCount)
val median = weightedMedian(
residualScratch,
weightScratch,
voteCount
)
val outlierRadius =
TWO_PI * residualOutlierHz / sampleRate
var weightedResidual = 0.0
var acceptedWeight = 0.0
var totalWeight = 0.0
for (n in 0 until voteCount) {
val r = residualScratch[n]
val w = weightScratch[n]
totalWeight += w
if (abs(r - median) <= outlierRadius) {
weightedResidual += w * r
acceptedWeight += w
}
}
if (acceptedWeight > 0.0) {
fusedResidual = weightedResidual / acceptedWeight
// Agreement is the fraction of spectral voting weight surviving robust rejection.
agreement =
(acceptedWeight / totalWeight.coerceAtLeast(1e-30))
.coerceIn(0.0, 1.0)
omega =
(omega + frequencyAlpha * fusedResidual)
.coerceIn(minOmega, maxOmega)
}
}
val instantaneousConfidence =
if (gateOpen && voteCount > 0)
(harmonicity * agreement).coerceIn(0.0, 1.0)
else
0.0
smoothedConfidence =
confidenceKeep * smoothedConfidence +
(1.0 - confidenceKeep) * instantaneousConfidence
// The agile coordinate used by lock=1 bank voices.
val actualAdvance =
omega + (if (phaseFollow.isFinite()) phaseFollow else 0.0) * fusedResidual
phase = wrapTrackedPhase(phase + actualAdvance)
state.phase = phase
state.phaseAdvance = actualAdvance
state.omega = omega
state.level = sqrt(broadbandPower.coerceAtLeast(0.0))
state.confidence = if (gateOpen) smoothedConfidence else 0.0
state.gateOpen = gateOpen
state.locked = gateOpen && smoothedConfidence >= lockThreshold
state.harmonicity = harmonicity
state.activePartials = activePartials
return state
}
override fun reset()
override fun reset() {
phase = 0.0
omega =
(TWO_PI * initialHz / sampleRate)
.coerceIn(minOmega, maxOmega)
broadbandPower = 0.0
smoothedConfidence = 0.0
lp1Re.fill(0.0)
lp1Im.fill(0.0)
lp2Re.fill(0.0)
lp2Im.fill(0.0)
previousRe.fill(0.0)
previousIm.fill(0.0)
previousValid.fill(false)
coarseRing.fill(0.0)
coarseScores.fill(0.0)
analysisLag = 0
coarseWrite = 0
coarseCount = 0
coarseAccumulator = 0.0
coarseAccumulatorCount = 0
coarseSinceAnalysis = 0
coarseHz = 0.0
coarseCorrelation = 0.0
state.phase = 0.0
state.phaseAdvance = omega
state.omega = omega
state.level = 0.0
state.confidence = 0.0
state.gateOpen = false
state.locked = false
state.harmonicity = 0.0
state.activePartials = 0
}
Feed the cheap coarse periodicity path.
A box average precedes decimation. It is not intended to be a beautiful resampler; its only job is to make normalized autocorrelation cheap enough to run as an acquisition safety net.
source — 79 lines, from line 503
private fun pushCoarse(x: Double) {
coarseAccumulator += x
coarseAccumulatorCount++
if (coarseAccumulatorCount < coarseDecimation) return
val coarseSample = coarseAccumulator / coarseDecimation
coarseAccumulator = 0.0
coarseAccumulatorCount = 0
coarseRing[coarseWrite] = coarseSample
coarseWrite = (coarseWrite + 1) and coarseMask
coarseCount = (coarseCount + 1).coerceAtMost(coarseRingSize)
coarseSinceAnalysis++
if (
coarseCount < coarseWindow + coarseMaxLag + 2 ||
coarseSinceAnalysis < coarseHop
) {
return
}
if (analysisLag != 0) return
coarseSinceAnalysis = 0
for (j in coarseSnapshot.indices)
coarseSnapshot[j] = coarseRing[(coarseWrite - 1 - j) and coarseMask]
analysisLag = coarseMinLag
}
private fun advanceCoarse() {
val lag = analysisLag
if (lag == 0) return
var cross = 0.0
var e0 = 0.0
var e1 = 0.0
var sum0 = 0.0
var sum1 = 0.0
for (j in 0 until coarseWindow) {
val a = coarseSnapshot[j]
val b = coarseSnapshot[j + lag]
cross += a * b; e0 += a * a; e1 += b * b
sum0 += a; sum1 += b
}
// Center both windows so DC cannot masquerade as perfect periodicity.
cross -= sum0 * sum1 / coarseWindow
val denominator = e0 + e1 - (sum0 * sum0 + sum1 * sum1) / coarseWindow
coarseScores[lag] = if (denominator > 1e-20) (2.0 * cross / denominator).coerceIn(-1.0, 1.0) else 0.0
if (lag < coarseMaxLag) { analysisLag++; return }
analysisLag = 0
val estimate = estimateCoarsePeriodicity()
if (estimate <= 0.0 || broadbandPower < gateLevel * gateLevel) return
val candidateHz = estimate
val differenceHz =
abs(candidateHz - omega * sampleRate / TWO_PI)
if (differenceHz <= coarseDeadbandHz) return
val targetOmega =
(TWO_PI * candidateHz / sampleRate)
.coerceIn(minOmega, maxOmega)
val pull =
if (differenceHz > maxOf(2.0 * captureHz, candidateHz * 0.20)) 0.85
else if (smoothedConfidence < lockThreshold)
coarseUnlockedPull
else
coarseLockedPull
val before = omega
omega += pull * (targetOmega - omega)
omega = omega.coerceIn(minOmega, maxOmega)
// A large retune invalidates the fine filters' old baseband phase history.
if (abs(omega - before) > before * 0.10) {
clearFineHistory()
}
}
Returns Hz or <=0 when no convincing periodicity exists.
source — 129 lines, from line 586
private fun estimateCoarsePeriodicity(): Double {
var best = -1.0
for (lag in coarseMinLag..coarseMaxLag) best = maxOf(best, coarseScores[lag])
if (best < coarseMinCorrelation) {
coarseHz = 0.0
coarseCorrelation = best
return 0.0
}
val threshold =
maxOf(
coarseMinCorrelation,
best * coarseRelativePeak
)
var chosenLag = -1
// Smallest lag == highest frequency. The first peak almost as good as the global peak is
// preferred over equally strong 2T / 3T repetitions.
for (lag in (coarseMinLag + 1) until coarseMaxLag) {
val y = coarseScores[lag]
if (
y >= threshold &&
y >= coarseScores[lag - 1] &&
y > coarseScores[lag + 1]
) {
chosenLag = lag
break
}
}
if (chosenLag < 0) {
// Boundary / pathological fallback: choose the global maximum.
var maxScore = Double.NEGATIVE_INFINITY
for (lag in coarseMinLag..coarseMaxLag) {
if (coarseScores[lag] > maxScore) {
maxScore = coarseScores[lag]
chosenLag = lag
}
}
}
var refinedLag = chosenLag.toDouble()
if (chosenLag > coarseMinLag && chosenLag < coarseMaxLag) {
val ym = coarseScores[chosenLag - 1]
val y0 = coarseScores[chosenLag]
val yp = coarseScores[chosenLag + 1]
val curvature = ym - 2.0 * y0 + yp
if (abs(curvature) > 1e-12) {
refinedLag +=
(0.5 * (ym - yp) / curvature)
.coerceIn(-0.5, 0.5)
}
}
val hz =
(coarseRate / refinedLag)
.coerceIn(minHz, maxHz)
coarseHz = hz
coarseCorrelation = coarseScores[chosenLag]
return hz
}
private fun clearFineHistory() {
lp1Re.fill(0.0)
lp1Im.fill(0.0)
lp2Re.fill(0.0)
lp2Im.fill(0.0)
previousRe.fill(0.0)
previousIm.fill(0.0)
previousValid.fill(false)
smoothedConfidence *= 0.5
}
}
private fun insertionSortPairs(
values: DoubleArray,
weights: DoubleArray,
count: Int,
) {
for (i in 1 until count) {
val v = values[i]
val w = weights[i]
var j = i - 1
while (j >= 0 && values[j] > v) {
values[j + 1] = values[j]
weights[j + 1] = weights[j]
j--
}
values[j + 1] = v
weights[j + 1] = w
}
}
private fun weightedMedian(
values: DoubleArray,
weights: DoubleArray,
count: Int,
): Double {
var total = 0.0
for (n in 0 until count) total += weights[n]
val halfway = total * 0.5
var accumulated = 0.0
for (n in 0 until count) {
accumulated += weights[n]
if (accumulated >= halfway) return values[n]
}
return values[count - 1]
}
private fun nextPowerOfTwo(value: Int): Int {
var v = 1
while (v < value) v = v shl 1
return v
}