Yet another cleanroom implementation.

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Andrew Davie

Well, I decided to try yet another method for scoring the sounds.
Reason being, some of the sounds (215 for example) used 29 parts, each with many numbers. I thought that was getting pretty ridiculous. Essentially adding new parameters to "fine tune" the tables and get closer spectral matches. That seemed extremely wrong.

So, new method ditches all the scoring previously used and instead does it visually over just the spectrogram, and the line graph. I have constrained the number of parts to < 16 -- which is probably still way overkill.

                         committed (main)        branch visual
215 A5                   29 parts   349 numbers   4 parts   37 numbers

The diff graphs are shown side-by-side in the attached image. Huge improvement.
The stats confirm..

215 A5        0.4.3    now      improvement
diff V        251.4    37.9     85 % lower
lines L       1.84 dB  1.03 dB  44 % lower
numbers       349      37       89 % fewer
parts         29       4        86 % fewer

That's very promising. The new version uses (roughly) 15% of the memory. Mind you this is just for the "most expensive" sound.  We shall see how it holds up as processing continues.  But, promising -- the spectra look very similar!

Andrew Davie

#1
It's a slow process. Here's the second sound...

200 R0        0.4.3    now      improvement
diff V        66.4     29.3     56 % lower
lines L       2.52 dB  1.64 dB  35 % lower
numbers       21       51       2.4 x more
parts         2        4        2 x more

So, not all sunshine and roses;  way better accuracy, way worse memory cost.
I'm going with the accuracy, though!

Andrew Davie

I took a diversion into using "heard V" -- which is basically getting the fitting algorithm to ignore sounds that human ears cannot hear. It adjusted perceptual volume as well. Didn't go too well, and I eventually decided the best way forward was to continue with the current experiment of matching visuals of the line graph and the spectrogram.

It's only done a few sounds, but the results are... remarkable.  Each image represents one of the 120 or so sounds the SpeakJet makes. Of the four plots, the first is a line graph (blue = hardware, green = library reconstruction). We want those lines to be on top of each other.  The second graph is the original hardware spectrogram, and the third is the library reconstruction spectrogram. We want those to look the same. The last one is the difference between the two. We want that one to be as empty as possible.

These results so far are good.

sound      V                        L
215 A5     -81 %  (197.0 -> 37.9)   -23 %  (1.33 -> 1.03 dB)
200 R0     -74 %  (111.0 -> 29.3)   -43 %  (2.88 -> 1.64 dB)
153 OWRR   -58 %   (28.8 -> 12.1)   -21 %  (1.10 -> 0.87 dB)
158 IYEH   -43 %   (52.5 -> 29.9)   -40 %  (0.87 -> 0.52 dB)
128 IY     -29 %    (6.7 ->  4.8)    -2 %  (1.80 -> 1.78 dB)
185 WH     -22 %   (48.6 -> 37.9)   -21 %  (1.78 -> 1.41 dB)
136 AW     -20 %   (42.2 -> 33.7)   -29 %  (1.17 -> 0.84 dB)
190 TH     -18 %   (63.0 -> 51.6)    -7 %  (2.05 -> 1.91 dB)
149 IYRR    -2 %   (13.5 -> 13.2)   -29 %  (0.95 -> 0.68 dB)
134 UX       0 %   (49.4 -> 49.4)     0 %  (1.02 -> 1.02 dB)

mean       -35 %                    -21 %

I'm going to let the search keep running - slow as it is - because these results are on top of the previous best, which is impressive. Compared to the very first implementation, it's astounding...

sound      V                          L
215 A5     -99 %  (6628.1 -> 37.9)    -92 %  (12.80 -> 1.03 dB)
200 R0     -99 %  (2105.9 -> 29.3)    -87 %  (12.79 -> 1.64 dB)
153 OWRR   -99 %  (1251.7 -> 12.1)    -89 %   (7.77 -> 0.87 dB)
158 IYEH   -98 %  (1238.1 -> 29.9)    -93 %   (7.82 -> 0.52 dB)
128 IY    -100 %  (4214.6 ->  4.8)    -78 %   (8.03 -> 1.78 dB)
185 WH     -88 %   (308.1 -> 37.9)    -71 %   (4.90 -> 1.41 dB)
136 AW     -99 %  (4351.1 -> 33.7)    -87 %   (6.61 -> 0.84 dB)
190 TH     -74 %   (197.8 -> 51.6)    -64 %   (5.26 -> 1.91 dB)
149 IYRR   -99 %  (1102.8 -> 13.2)    -91 %   (7.69 -> 0.68 dB)
134 UX     -99 %  (5912.5 -> 49.4)    -85 %   (6.98 -> 1.02 dB)

mean       -95 %                      -84 %


Andrew Davie

Update...


Andrew Davie

#4
Update on stats... -50% means 50% closer to hardware than v0.4.3 was
V is the "visual" comparison of the spectrograms, as "eyeballed"
L is the "visual" comparison of the blue/green line convergence.

Here are all 24 refitted sounds, recomputed now against 0.4.3, both scored the same way. Minus means closer to the chip:

sound      V                        L
193 TS     -92 %  (523.7 -> 40.1)   -39 %  (3.04 -> 1.86 dB)
230 C0     -86 %  (146.0 -> 19.8)   -61 %  (1.27 -> 0.50 dB)
215 A5     -82 %  (197.0 -> 35.2)   -25 %  (1.33 -> 0.99 dB)
200 R0     -74 %  (111.0 -> 29.3)   -43 %  (2.88 -> 1.64 dB)
153 OWRR   -73 %   (28.8 ->  7.9)   -20 %  (1.10 -> 0.88 dB)   top-up gained
162 IHWW   -64 %   (52.7 -> 18.9)   -29 %  (1.17 -> 0.83 dB)   new
191 TT     -63 %  (121.2 -> 44.8)    -4 %  (2.97 -> 2.85 dB)
177 OD     -61 %  (136.8 -> 53.6)    -9 %  (2.40 -> 2.19 dB)
232 C2     -56 %  (330.8 -> 145.9)  -24 %  (1.29 -> 0.98 dB)   still searching from your drawing
170 BE     -44 %  (120.9 -> 67.1)   -10 %  (1.72 -> 1.54 dB)
159 EHLL   -42 %   (36.8 -> 21.3)    -8 %  (0.81 -> 0.74 dB)   new
158 IYEH   -41 %   (47.1 -> 27.8)   -45 %  (0.87 -> 0.48 dB)   top-up gained
128 IY     -29 %    (6.7 ->  4.8)    -2 %  (1.80 -> 1.78 dB)
199 PO     -26 %  (146.4 -> 108.3)  -15 %  (1.76 -> 1.49 dB)
135 OH     -24 %   (35.9 -> 27.3)    -6 %  (1.01 -> 0.96 dB)   top-up gained
185 WH     -22 %   (48.6 -> 37.9)   -21 %  (1.78 -> 1.41 dB)
173 OB     -22 %  (129.7 -> 100.9)   +5 %  (2.19 -> 2.30 dB)
136 AW     -20 %   (42.2 -> 33.7)   -29 %  (1.17 -> 0.84 dB)
190 TH     -18 %   (63.0 -> 51.6)    -7 %  (2.05 -> 1.91 dB)
181 OG     -17 %   (80.3 -> 66.5)   -12 %  (1.79 -> 1.57 dB)
149 IYRR    -2 %   (13.5 -> 13.2)   -29 %  (0.95 -> 0.68 dB)
157 OHIH     0 %   (54.0 -> 54.2)   -45 %  (1.04 -> 0.57 dB)
134 UX       0 %   (49.4 -> 49.4)     0 %  (1.02 -> 1.02 dB)
148 RR       0 %   (19.5 -> 19.5)     0 %  (0.91 -> 0.91 dB)   new

mean       -40 %                    -20 %


- 21 of 24 are better on V. The other three are unchanged: 134 UX, 148 RR and 157 OHIH, whose L still improved by 45 %.
- One figure got worse: 173 OB's L, +5 % (0.11 dB, within the 0.3 dB the decision allows).