From e37ee8738b414a9940d066ca1b8a21b566b82340 Mon Sep 17 00:00:00 2001 From: Karel Vesely Date: Mon, 11 Feb 2019 10:15:41 +0100 Subject: [PATCH] update the docs: - update the history (inputs from Honza Cernocky), - fix URLs in nnet1 docs, - add 'usage' to cuda-gpu-available. --- src/doc/dnn1.dox | 16 ++++++++-------- src/doc/history.dox | 10 +++++++--- src/nnetbin/cuda-gpu-available.cc | 11 +++++++++++ 3 files changed, 26 insertions(+), 11 deletions(-) diff --git a/src/doc/dnn1.dox b/src/doc/dnn1.dox index 223b7665274..e8dcfd90d3f 100644 --- a/src/doc/dnn1.dox +++ b/src/doc/dnn1.dox @@ -35,13 +35,13 @@ show some \ref dnn1_advanced_features, and do a light introduction to the \ref d
\section dnn1_toplevel_scripts Top-level script -Let's have a look at the script egs/wsj/s5/local/nnet/run_dnn.sh. +Let's have a look at the script egs/wsj/s5/local/nnet/run_dnn.sh. This script assumes to use a single CUDA GPU, and that kaldi was compiled with CUDA (check for 'CUDA = true' in src/kaldi.mk). Also we assume that 'cuda_cmd' is set properly in egs/wsj/s5/cmd.sh either to a GPU cluster node using 'queue.pl' or to a local machine using 'run.pl'. And finally the script assumes we already have a SAT GMM system exp/tri4b and corresponding fMLLR transforms, as generated by egs/wsj/s5/run.sh. Note that for other databases the run_dnn.sh is typically in the same location s5/local/nnet/run_dnn.sh. -The script egs/wsj/s5/local/nnet/run_dnn.sh is split into several stages: +The script egs/wsj/s5/local/nnet/run_dnn.sh is split into several stages: 0. storing 40-dimensional fMLLR features to disk, steps/nnet/make_fmllr_feats.sh, this simplifies the training scripts, the 40-dimensional features are MFCC-LDA-MLLT-fMLLR with CMN @@ -100,7 +100,7 @@ Besides the DNN recipe, there are also other example scripts which can be handy:
\section dnn1_training_script_internals Training script internals -The main neural network training script steps/nnet/train.sh is invoked as: +The main neural network training script steps/nnet/train.sh is invoked as: \verbatim steps/nnet/train.sh @@ -111,11 +111,11 @@ The is used only in the special case when using LDA feature-transform The output (i.e. the trained networks and logfiles) goes into . Internally the script prepares the feature+target pipelines, generates a neural-network prototype and initialization, creates feature_transform and calls the scheduler script -steps/nnet/train_scheduler.sh, +steps/nnet/train_scheduler.sh, which runs the training epochs and controls the learning rate. -While looking inside steps/nnet/train.sh we see: +While looking inside steps/nnet/train.sh we see: 1. CUDA is required, the scripts exit if no GPU was detected or was CUDA not compiled in (one can still use '--skip-cuda-check true' to run on CPU, but it is 10-20x slower) @@ -165,12 +165,12 @@ $ cat exp/dnn5b_pretrain-dbn_dnn/nnet.proto 7. the network is initialized by : \ref nnet-initialize.cc , the DBN gets prepended in the next step using \ref nnet-concat.cc -8. finally the training gets called by running scheduler script steps/nnet/train_scheduler.sh +8. finally the training gets called by running scheduler script steps/nnet/train_scheduler.sh Note : both neural networks and feature transforms can be viewed by \ref nnet-info.cc, or shown in ascii by \ref nnet-copy.cc -While looking inside steps/nnet/train_scheduler.sh we see: +While looking inside steps/nnet/train_scheduler.sh we see: the initial cross-validation run and the main for-loop over $iter which runs the epochs and controls the learning rate. Typically, the train_scheduler.sh is called from train.sh. - the default learning-rate scheduling is based on the relative improvement of the objective function: @@ -310,7 +310,7 @@ AddMat 174.307s AddMatMat 1922.11s \endverbatim - Running steps/nnet/train_scheduler.sh directly: + Running steps/nnet/train_scheduler.sh directly: - The script train_scheduler.sh can be called outside train.sh, it allows to override the default NN-input and NN-target streams, which can be handy. - However the script assumes everything is set-up correctly, and there are almost no sanity checks, which makes it suitable for more advanced users only. - It is highly recommended to have a look at how train_scheduler.sh is usually called before trying to call it directly. diff --git a/src/doc/history.dox b/src/doc/history.dox index 40d46c7e32f..0813f2331cc 100644 --- a/src/doc/history.dox +++ b/src/doc/history.dox @@ -54,7 +54,8 @@ Sandeep Boda, Sandeep Reddy and Haihua Xu (who helped with coding, code cleanup and documentation); we were visited by Michael Riley (who helped us to understand OpenFst and gave some lectures on FSTs), and would like to acknowledge the help of - Honza Cernocky (for allowing us to have the workshop and helping to organize it), + Honza Cernocky (for negotiating the venue and some support for the workshop from + the Faculty of Information Technology of BUT and helping to organize it), Renata Kohlova (administration), and Tomas Kasparek (system administration). It is possible that this list of contributors contains oversights; any important omissions are unlikely to be intentional. @@ -62,13 +63,16 @@ A lot of code was written during the summer of 2010 but we still did not have a complete working system. Some of the participants of the 2010 workshop continued working to complete the toolkit and get a working set of training scripts. - The code was released on May 14th, 2011. + The code was released on May 14th, 2011, and presented to public at ICASSP 2011 + in Prague, + + see the recordings. Since the initial release, Kaldi has been maintained and developed to a large extent by Daniel Povey, working at Microsoft Research until early 2012 and since then at Johns Hopkins University; but also with major contributions by others: notably Karel Vesely, who developed the neural-net training framework, - and Arnab Ghoshal, who co-ordinated the acoustic modeling work early on; but + and Arnab Ghoshal, who coordinated the acoustic modeling work early on; but also other major contributors whom we do not name here because it is too hard to determine where to cut off the list; and a long tail of minor contributors; the total number of people who have contributed code or scripts or patches is diff --git a/src/nnetbin/cuda-gpu-available.cc b/src/nnetbin/cuda-gpu-available.cc index 390468d3046..4e6c1a025c1 100644 --- a/src/nnetbin/cuda-gpu-available.cc +++ b/src/nnetbin/cuda-gpu-available.cc @@ -40,6 +40,17 @@ void TestGpuComputation() { #endif int main(int argc, char *argv[]) try { + + /* only for Doxygen documentation, never shown in command line */ + const char *usage = + "Test if there is a GPU available, and if the GPU setup is correct.\n" + "A GPU is acquired and a small computation is done\n" + "(generating a random matrix and computing softmax for its rows).\n" + "\n" + "exit-code: 0 = success, 1 = compiled without GPU support, -1 = error\n" + "\n" + "Usage: cuda-gpu-available\n"; + char hostname[100] = "UNKNOWN-HOSTNAME"; #if !defined(_MSC_VER) && !defined(__CYGWIN__) if (gethostname(hostname, 100)) {