A fitting epiphany!

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Andrew Davie

I had a bit of a OMG epiphany - an idea regarding finding the correct settings for the fitting.

**SpeakJet emulation: finding what the scores couldn't see**

For a long time the fitting worked like this: every sound has a set of numbers (formant frequencies, levels, noise band, lengths), a search nudges them, and three scores decide whether the result is closer to the real chip. One for the overall spectrum, one for spectrum plus time-frequency detail plus level over time (E), and a perceptual one (P). Refit after refit, those scores crept down and then stalled.

The trouble is that all three scores are averages over the whole sound. If a 15 ms burst at the start of a sound is missing, or a stop's release comes 20 ms late, the averages move by a few tenths, which is about their noise level. And the search can only move numbers that already exist. If the engine has no way to make a burst, no amount of searching produces one.

So I started looking at it differently.

**The difference picture**

Every sound in the gallery now has a fourth panel: our sound minus the chip's, band by band and moment by moment. Red where ours is too heavy, blue where it's too light, white where we're within the chip's own take-to-take variation.


A sound that's structurally right is a white panel with a few specks. And the shape of the red and blue tells you what's wrong:

1_gallery_row_DO.png

- Scattered specks: noise, the real chip disagreeing with itself. Ignore.
- A thin strip at the start or end: timing. We start or stop a few ms early or late.
- A horizontal band right through the sound: a level. A formant or noise band too strong or weak throughout, which ordinary tuning can fix.
- A block confined in time: structure. Something is missing, extra, or in the wrong place: a burst, a bar, a tail.
- Alternating thin red/blue stripes: voice pulses drifting slightly out of step. Not structure.

2_se_onset_column.png

**Turning the picture into a number**

The same comparison gives one number per sound, the structure error: how far the difference stands beyond the real chip's own spread, averaged over the whole band × time grid. 0 means indistinguishable from the variation between two real AtariVoxes. The two units, the AtariVox+ and the original AtariVox, both recorded in the same session, define that spread, so chip-to-chip differences don't count as errors.

**Knobs, and a loop that uses them**

Each time a block showed up that nothing could fix, it became a new mechanism in the engine, with a "knob" per sound. Along the way that gave:

- a burst of low noise at the start of the sibilants (SE, SO, SH): the dark column the chip has and we didn't
- the voiced stops' bar and release: the chip's bar is long and weak, the release short; ours was the other way round
- a per-sound closure length for stops
- a white noise floor under WH's noise band (it had nothing above 3.5 kHz)
- a high-pass on the fricatives' noise, and per-sound corners for it
- a 3 ms switch-on/off for the unvoiced fricatives (we were fading them in over 35 ms; the chip switches them on)
- per-sound fade-in and fade-out lengths, and per-effect versions of them
- a separate fade-in for nasals

3_do_bar_release.png

5_wh_floor_and_onset.png

A structure loop then tries every knob on every sound, up and down, and keeps only what lowers that sound's structure error. It runs automatically before every refit, followed by a guard that throws away any change that didn't really help. A classifier then names what's left on each panel ("starts ~10 ms early", "1.3–2.3 kHz heavy throughout"), and that's what's printed in the corner of each difference panel. Whatever survives the loop is, by definition, something the knobs can't reach, so it's the to-do list for a new mechanism.

The refit's gate also changed: a candidate that makes the structure worse is now refused, whatever its other scores say. Before this, one voiced stop passed with its bar and release in completely the wrong places, because its averages happened to improve.

**What went wrong on the way (so you don't trust it blindly)**

- **Alignment is everything.** Several times the difference panels showed confident red and blue that was really the two signals misaligned by 8–15 ms, or a quiet sound's window landing on only part of it. The alignment now places the chip's sound as the loudest stretch of our sound's length, then fine-aligns on the band spectrograms. That's the version that agrees with your eyes.
- **Quiet cells.** A 15 dB difference between two levels 50 dB down is invisible on the spectrograms but showed as deep red. Cells quiet on both sides are now ignored.
- **One unit isn't enough.** Fitting to a single take made the edges noisy. Both units' newest takes it is.
- **Lengths are measured, not fitted.** The optimiser pushed some lengths badly wrong (SE to 165 ms when the chip's is 90). Lengths are now set directly from the chip's measured length, and the loop doesn't touch them.

**Where it stands**

Against the 0.3.0 release, scored the same way, before the refit has done its tuning:

              structure error  E
allophones    -33 %            -6 %
effects      -14 %            -4 %
all 127      -23 %            -5 %

That's pretty impressive, given that the improvements had flatlined for the past week or two. P and the overall spectrum have barely moved yet. They're mostly levels and formants, which the refit now running re-tunes around the corrected structure. The tables are at 4.9 KB of a 5 KB budget.

The real change is the method. Instead of hoping an average notices something, every sound now has a picture of exactly where it differs from the real chip, the tools act on it, and what's left is visible at a glance.

4_wh_clean.png

And that "what's left" is exactly the structural deficiencies in the current model of the SpeakJet. It's the things that tweaking the knobs could not remove.  A post-analysis of all the sounds to find commonality in these remnants may lead to an understanding of underlying structure not yet emulated.

It's really cool.

Andrew Davie

**Accuracy against the very first version.**

Note: original version was "1.0.0" but I'm retro-naming that 0.0.1

"Gap closed" means how far from 0.0.1 toward "as close as the two real chips are to each other" (the chip-to-chip score), where 100% would be a second real AtariVox:

                                                                      gap closed
            v0.0.1   v0.3.0   now     chip-to-chip   now vs v0.0.1  v0.3.0   now
E           13.54    5.55     5.21    3.39           -61.5%         78.7%   82.1%
P            3.67    2.16     2.09    1.60           -43.1%         72.8%   76.5%
spectrum    13.80    2.88     2.84    1.17           -79.4%         86.5%   86.8%
structure    5.63    0.52     0.37    0.00           -93.4%         90.8%   93.4%

That's about **77–82% of the way to a second real AtariVox on P and E**, up from 73–79% at v0.3.0. v0.0.1 made 8192 Hz audio with nothing above 4 kHz, which is much of its large spectrum and E figures. The full comparison over every recording is still running, and I'll add it when it lands.

Here's a full complement of graphs...