wavetable

[roebel]

Hello,

Thanks for sharing this code!

I was quite interested in using your wavetable synthesis but I don't quite know how you make this work.
I tried to make it use with only a single wavetable but multiple frequency contours. This could be a wavetable with a
single sinusoid.

Anyway. In my examples using a singe wavetable (default parameters)
and a phase vector with phase.shape = (2,8000) the phase difference calculation produces
an intermediate internal phase difference matrix of about 1GB which explodes my memory (I have only a small GPU in my laptop) I made it work with phase.shape = (2,200) but this requires me to chunk my phase vectors
and create loops that seem rather annoying (and unnecessary).

So I finally decided to drop DDSP and implemented my own wavetable class.

In my trials using a Geforce 1050TX (laptop) this new implementation uses less than 1MB
as temporary memory (for phase.shape =(2,8000)). I compared the processing speed with
phase.shape = (2,200) ) and got these results

In [90]: timeit pt(f0[:, :200], use_DDSP_interpolation=True)
14.7 ms ± 52.6 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [91]: timeit pt(f0[:, :200], use_DDSP_interpolation=False)
997 µs ± 19.5 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

so this seems, at least on my GPU and using eager mode, to become about 14 times faster.

As I profited from your ideas I would like to share my modifications for integration into DDSP.
That would become a pull request that would integrate my changes into your WaveTable
synthesis class. I did not test and wont have time to see whether and how my implementation
translates also to multi wavetable setups. As far as I see and understand what you would do with multple
wavetables there should not be a problem.

Question is whether you are interested in these changes.

Best
Axel

[roebel]

Hello again,

I have found your test script core_test.py and in there I saw that I miss understood the number 64000 in your code. I saw that it is not the wavetable size but the number of samples! I saw that you use wavetables that are much smaller (max 2048). I also now understood what the multiple frames wavetable version is about!

Anyway, in fact I was curious how all this compares and integrated the proposed new_linear_lookup function into the test to compare speed and accuracy. I extended it slightly so that it covers all your test cases. For this I added a somewhat ugly axis mapping into the beginning of the new function. The new function has two modes, nearest (taking only the nearest neighbor from the wavetable, so no linear interpolation!) and linear interpolation (which I suppose is what your original interpolation is doing).

The implementation is here

ddsp_core_test.py.zip

In the following three sections I comment the results produced with the script that is attached to the post and you find the numbers in the results table at the end.

Accuracy

both modes - the simple one doing nearest neighbor lookup called Near as well as the new linear interpolation named NewL are both more accurate than the original one - in all cases ! While the new linear interpolation reduces error by more than 3 magnitudes, also just taking the closest sample reduces the the error by a factor of two!

This is quite astonishing - it may show that your implementation is buggy.

Computational efficiency

I only use a very simple wall clock timing here, but as I run this over 100 cycles I think the results are nevertheless quite reliable, which is supported by the observation that the simpler nearest mode is consistently 10 % faster than the new interpolation mode.

Compared to the original implementation both new modes are 20-30 times faster for all single frame cases. Only for the multiple frame case the computational efficiency is about the same. Given that the gather function I use here for all cases doe not really do much more in the multi frame case I would wonder if this speed problem is not a problem in the gather function.

Memory

I did not measure this, but as explained in the initial post I am pretty sure that the new version has much lower memory needs!

Please feel free to do with that whatever you like, and thanks again for the DDSP package.

760> ./ddsp_core_test.py 
================================= 
runing test high_frequency_wave, wavetable size 2048
Eval |NewL - numpy| : L1 err=5.1179e-07 runtime: 0.0010667s/it
Eval |Near - numpy| : L1 err=4.8858e-04 runtime: 0.00088891s/it
Eval |DDSP - numpy| : L1 err=9.7643e-04 runtime: 0.02265s/it
================================= 
runing test low_frequency_wave, wavetable size 1024
Eval |NewL - numpy| : L1 err=1.9903e-06 runtime: 0.0015629s/it
Eval |Near - numpy| : L1 err=9.7747e-04 runtime: 0.00094396s/it
Eval |DDSP - numpy| : L1 err=1.9530e-03 runtime: 0.030368s/it
================================= 
runing test one_frame, wavetable size 2048
Eval |NewL - numpy| : L1 err=5.1179e-07 runtime: 0.0010399s/it
Eval |Near - numpy| : L1 err=4.8858e-04 runtime: 0.00091199s/it
Eval |DDSP - numpy| : L1 err=9.7643e-04 runtime: 0.018281s/it
================================= 
runing test many_frames, wavetable size 2048
Eval |NewL - numpy| : L1 err=5.1179e-07 runtime: 0.045412s/it
Eval |Near - numpy| : L1 err=4.8858e-04 runtime: 0.04478s/it
Eval |DDSP - numpy| : L1 err=9.7643e-04 runtime: 0.046915s/it

[voodoohop]

Isn't the wavetable learnable? If it works you wouldn't need to provide different fixed wavetables but would just learn it. I never tested it though.