Python - end to end instruction on using this library easily (also in notebook)

[snakers4]

Will post this instruction shortly

[snakers4]

Step by step instructions

Python libraries

Use your package manager to install

pydub
numpy
scipy

Versions do not matter

Library installation

git clone [email protected]:xiph/rnnoise.git
cd rnnoise
./autogen.sh
./configure
make
make install

Code

import wave
import os,sys
import ctypes
import contextlib
import numpy as np
from ctypes import util
from scipy.io import wavfile
from pydub import AudioSegment

lib_path = util.find_library("rnnoise")
if (not("/" in lib_path)):
    lib_path = (os.popen('ldconfig -p | grep '+lib_path).read().split('\n')[0].strip().split(" ")[-1] or ("/usr/local/lib/"+lib_path))

lib = ctypes.cdll.LoadLibrary(lib_path)
lib.rnnoise_process_frame.argtypes = [ctypes.c_void_p,ctypes.POINTER(ctypes.c_float),ctypes.POINTER(ctypes.c_float)]
lib.rnnoise_process_frame.restype = ctypes.c_float
lib.rnnoise_create.restype = ctypes.c_void_p
lib.rnnoise_destroy.argtypes = [ctypes.c_void_p]

# borrowed from here 
# https://github.com/Shb742/rnnoise_python
class RNNoise(object):
    def __init__(self):
        self.obj = lib.rnnoise_create()
    def process_frame(self,inbuf):
        outbuf = np.ndarray((480,), 'h', inbuf).astype(ctypes.c_float)
        outbuf_ptr = outbuf.ctypes.data_as(ctypes.POINTER(ctypes.c_float))
        VodProb =  lib.rnnoise_process_frame(self.obj,outbuf_ptr,outbuf_ptr)
        return (VodProb,outbuf.astype(ctypes.c_short).tobytes())

    def destroy(self):
        lib.rnnoise_destroy(self.obj)


def read_wave(path):
    """Reads a .wav file.
    Takes the path, and returns (PCM audio data, sample rate).
    """
    with contextlib.closing(wave.open(path, 'rb')) as wf:
        num_channels = wf.getnchannels()
        assert num_channels == 1
        sample_width = wf.getsampwidth()
        assert sample_width == 2
        sample_rate = wf.getframerate()
        assert sample_rate in (8000, 16000, 32000, 48000)
        pcm_data = wf.readframes(wf.getnframes())
        return pcm_data, sample_rate        
        
      
def frame_generator(frame_duration_ms,
                    audio,
                    sample_rate):
    """Generates audio frames from PCM audio data.
    Takes the desired frame duration in milliseconds, the PCM data, and
    the sample rate.
    Yields Frames of the requested duration.
    """
    n = int(sample_rate * (frame_duration_ms / 1000.0) * 2)
    offset = 0
    timestamp = 0.0
    duration = (float(n) / sample_rate) / 2.0
    while offset + n < len(audio):
        yield audio[offset:offset + n]
        offset += n        
        
denoiser = RNNoise()      

Let's try it

wav_path = 'path_to_your_file.wav'


TARGET_SR = 48000
TEMP_FILE = 'test.wav'

sound = AudioSegment.from_wav(wav_path)
sound = sound.set_frame_rate(TARGET_SR)
sound = sound.set_channels(1)

sound.export(TEMP_FILE,
             format="wav")

audio, sample_rate = read_wave(TEMP_FILE)
assert sample_rate == TARGET_SR

frames = frame_generator(10, audio, TARGET_SR)
frames = list(frames)
tups = [denoiser.process_frame(frame) for frame in frames]
denoised_frames = [tup[1] for tup in tups]

denoised_wav = np.concatenate([np.frombuffer(frame,
                                             dtype=np.int16)
                               for frame in denoised_frames])

wavfile.write('test_denoised.wav',
              TARGET_SR,
              denoised_wav)

[landonclark97]

I was able to get this working after passing None into init of the RNNoise class as such:

class RNNoise(object): def __init__(self): self.obj = lib.rnnoise_create(None)

[ilor2]

Is there a way with python (or matlab) to return the features?

Thanks!

[slavaGanzin]

@ilor2

Is there a way with python (or matlab) to return the features?

You need to import compute_frame_features function just like @snakers4 had with rnnoise_process_frame:

rnnoise/src/denoise.c

Line 306 in 9acc1e5

static int compute_frame_features(DenoiseState *st, kiss_fft_cpx *X, kiss_fft_cpx *P,

I don't know how to do it. But if you find a way post it here please.

p.s. @snakers4 btw thanks for open_stt!

[ilor2]

Thanks, I have a windows though and I don't think I can run autogen.sh on it. I'll post if I find a way around.

[ebadawy]

hey @snakers4 , the python script keeps crashing every time I try to run it! similar to what is mentioned in #50 , any idea how to fix that!?

[kmonachopoulos]

The example works as it should! I managed to inference an exported tflite model with a little bit of debugging. There is a way to separate the functionality into three parts (feature_extraction -> tflite_inference -> meta_process). The only difficult part is to properly allocate the numpys in Python, create c_type pointers and pass them by reference to the C functions.

The C functions looks like this :

Pre-process (feature extraction)

int rnnoise_extract_features(
  kiss_fft_cpx* X,
  kiss_fft_cpx* P,
  float *Ex,
  float *Ep,
  float *Exp,
  DenoiseState *st,
  float *features,
  const float *in) {

  float x[FRAME_SIZE];
  int silence;
  static const float a_hp[2] = {-1.99599, 0.99600};
  static const float b_hp[2] = {-2, 1};
  biquad(x, st->mem_hp_x, in, b_hp, a_hp, FRAME_SIZE);
  silence = compute_frame_features(st, X, P, Ex, Ep, Exp, features, x);

  return silence;
}

Meta-process

void rnnoise_meta_process(
  kiss_fft_cpx* X,
  kiss_fft_cpx* P,
  float *Ex,
  float *Ep,
  float *Exp,
  DenoiseState *st,
  float* g,
  float *out) {
  float gf[FREQ_SIZE]={1};

  pitch_filter(X, P, Ex, Ep, Exp, g);

  for (int i=0;i<NB_BANDS;i++) {
        float alpha = .6f;
        g[i] = MAX16(g[i], alpha*st->lastg[i]);
        st->lastg[i] = g[i];
    }
  interp_band_gain(gf, g);

  for (int i=0;i<FREQ_SIZE;i++) {
      X[i].r *= gf[i];
      X[i].i *= gf[i];
    }
  frame_synthesis(st, out, X);
}

[pranshurastogi29]

None

@landonclark97
I was able to get this working after passing None into init of the RNNoise class as such:

class RNNoise(object): def __init__(self): self.obj = lib.rnnoise_create(None)

hey can you help why none worked
thxs

[landonclark97]

@pranshurastogi29 In the rnnoise_demo.c file provided you'll see that the rnnoise_create() function is called with NULL passed in as its only parameter. I'm guessing that something needs to be passed in when we call rnnoise_create() in our Python script, so in a way None can act as the Python equivalent of NULL - but that's just a guess.

[katyamineeva]

hey @snakers4 , the python script keeps crashing every time I try to run it! similar to what is mentioned in #50 , any idea how to fix that!?

Hi! I had Segmentation fault when running this example, but during debugging I found out that adding print() before the denoiser initialization (denoiser = RNNoise()) fixes the problem. I've got no idea why it might happen, but maybe you'll manage to work around your problem with such trick

[vaishalibhardwaj]

You can do this on Linux.
try running sudo apt-get install autoconf
and you are good to go

[LG-SS]

hey @snakers4 , the python script keeps crashing every time I try to run it! similar to what is mentioned in #50 , any idea how to fix that!?

Hi! I had Segmentation fault when running this example, but during debugging I found out that adding print() before the denoiser initialization (denoiser = RNNoise()) fixes the problem. I've got no idea why it might happen, but maybe you'll manage to work around your problem with such trick

However, just as you mentioned, adding print() before the denoiser initialization (denoiser = RNNoise()) doesn't work for me. Do you know what cause such segmentation fault, Thx.