Memory and Specific effect issue

[WindowsNT]

Thanks for the code, I'm trying to run this in Windows to include in my audio sequencer.

After installing with pip and download the pytorch checkpoints, I can run it:
..\python scripts\remfx_detect.py +exp=remfx_detect +audio_input=r:\reverb.wav

1. Can I have a specific effect instead of "remfx_detect" ? If i put "reverb" there I got

Error executing job with overrides: ['+exp=reverb', '+audio_input=r:\\reverb.wav']
Traceback (most recent call last):
  File "F:\python\RemFx\scripts\remfx_detect.py", line 16, in main
    for effect in cfg.ckpts:
                  ^^^^^^^^^
omegaconf.errors.ConfigAttributeError: Key 'ckpts' is not in struct
    full_key: ckpts
    object_type=dict

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

2. Eventually it gets out of memory with my 32 GB and it's too slow if it doesn't. A way to make it a bit faster would help.

Best and thanks for all that.

[mhrice]

Hey, thanks for trying our project!

1. In general, we were aiming to tackle the problem where the effects present were unknown beforehand. However, if you want to run the code for a specific effect on an audio file, probably the simplest way is to hack the remfx_detect.py script. I haven't tested the following code, but you'll need to do something like below:

• In the RemFXChainInference() call, set classifier = None
• Modify batch = [audio, audio, None, [[0, 0, 0, 0, 0]]], set one of those zeros to 1 based on which effect you want. The effects are in the order of [[Reverb, Chorus, Delay, Distortion, Compressor]]

2. If you are using CPU, it will run very slowly, but GPU will speed it up significantly. If you follow the steps in 1., it should speed up the inference a decent amount (and use less memory overall) as it is not performing the classification, and only running 1 effect-specific model

[WindowsNT]

Thanks for your response, do I use the GPU by installing the GPU-enabled torch?

[WindowsNT]

I tried installing torch from site with GPU support and yet when I install remfx it still wants to install torch again and it uses CPU... any hints?

python scripts\remfx_detect.py +exp=remfx_detect +audio_input=r:\reverb.wav accelerator=gpu

Btw the hint for the specific effect with Batch [1...] worked, thanks :)

[WindowsNT]

I saw that you are checking for cuda in your code so the issue is at my end, will fix and notify you. Thanks for everything.

[WindowsNT]

I found a probable bug
scripts\pip install -e ./remfx ./remfx/umx
scripts\pip uninstall -y torch torchvision torchaudio functorch
scripts\pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
scripts\pip install --no-deps hearbaseline
cd remfx
scripts\download_ckpts.sh

but you use functorch which is not compatible with the cuda version of torch.

I have to install it again with
pip install git+https://github.com/pytorch/functorch.git

Then the cuda version works fine. I will build all that in my sequencer and let you know.
Congrats for the job.

[WindowsNT]

I still have the memory issue. I got 32 gb or ram, 8 GB of GPU rtx 4060 and yet I get memory errors even if only the reverb is being removed.

Thanks a lot.

[insinfo]

@WindowsNT
Did you manage to make this work without requiring more than 32GB of memory?

I'm looking for a solution to mitigate echo and distortion, something like this https://crowdunmix.org/the-best-way-to-recover-audio-distortion-using-ai/

[WindowsNT]

No, not yet, still trying.