i-vector training, extraction and scoring routines

This is a small package that does basic i-vector training and extraction, a framework that is used in Automatic Speaker Recognition. The package relies on GaussianMixtures.jl for training a Universal Background Model (UBM) and computing statistics.

Pkg.clone("https://github.com/davidavdav/IVectors.jl")

Data is represented as a matrix with data "running down", i.e., the data matrix is formed by a stack of features as "row vectors". Typically, a data matrix represents a single audio file, and is variable in the size of the first dimension, but fixed in the second, which is the number of features for each data point.

Suppose you have a vector of such data matrices, x::Vector{Matrix}, then you can first train a diagonal covariance UBM with GaussianMixtures, using

using GaussianMixtures
ngauss = 1024
ubm = GMM(ngauss, Data(x), kind=:diag)

This may take a while, check out parallelization options in GaussianMixtures.jl.

Now suppose that you want to use the same data to train an iVector extractor. The first thing to do, is to extract centralized and scaled statistics for the train data:

css = map(data->CSstats(ubm, data), x)

Again, this may take a while, you might consider pmap() instead. GaussianMixtures::CSstats stores the zeroth and first order statistics w.r.t. the UBM. The type is parameterized, and it apprears that map() is better at registering the type than a comprehension would do, so we advice to use map() here.

An iVector extractor can be trained now for a given number of voices (target dimension):

using IVectors
nvoices = 100
ie = IExtractor(css, nvoices; nIter=7)

Saving and loading the IVector extractor relies on the FileIO and JLD packages.

using FileIO
save("file.iex", ie)
save("iex-noext" ie) ## another copy with a funny name

FileIO can recognize the format"IExtractor" format by the .iex extension. The registration of the format happens in IVectors, not in FileIO.

using IVectors
ie = load("file.iex") ## by extension
ie = load(File(format"IExtractor", "iex-noext")) ## by format
ie = load("iex-noext", IExtractor) ## by type

Although in literature the IVector extractor is always portrayed as a single "tall" matrix of CF x nvoices, where C is the number of Gaussian components and F feature dimension, we internally represent T as a length-C vector of F x nvoices matrices. This is also how it is serialized to disc. Further, upon loading or calculation of the IExtractor, a length-C vector of nvoices x nvoices matrices is kept in memory for computational efficiency.

iVectors are extracted using the same CSstats structure. For a data matrix data:

ivec = ivector(ie, CSstats(ubm, data))

A slightly more efficient way to extract ivectors is to use a bunch of files simultaneously:

blas_set_num_threads(2)
css = map(data->CSstats(ubm, data), x) ## x is vector of data matrices
ivecs = ivector(ie, css) ## a Vector of Vectors

This implementation uses Base.BLAS.gemm!() to compute ivectors simultaneously for multiple CSstats objects, we have found that we're not gaining much speed by setting the number of openblas threads too high.

Current status of the package is that it

• needs more validation
• needs support for full covariance UBMs. This should happen in CSstats, though.
• scoring (cosine distance is trivial)
• PLDA scoring