Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update the docs: #3031

Merged
merged 1 commit into from
Mar 14, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
update the docs:
- update the history (inputs from Honza Cernocky),
- fix URLs in nnet1 docs,
- add 'usage' to cuda-gpu-available.
  • Loading branch information
KarelVesely84 committed Mar 14, 2019
commit e37ee8738b414a9940d066ca1b8a21b566b82340
16 changes: 8 additions & 8 deletions src/doc/dnn1.dox
Original file line number Diff line number Diff line change
Expand Up @@ -35,13 +35,13 @@ show some \ref dnn1_advanced_features, and do a light introduction to the \ref d
<hr><!-- #################################################################################################################### -->

\section dnn1_toplevel_scripts Top-level script
Let's have a look at the script <b><a href="http:https://sourceforge.net/p/kaldi/code/HEAD/tree/trunk/egs/wsj/s5/local/nnet/run_dnn.sh">egs/wsj/s5/local/nnet/run_dnn.sh</a></b>.
Let's have a look at the script <b><a href="https:https://github.com/kaldi-asr/kaldi/blob/master/egs/wsj/s5/local/nnet/run_dnn.sh">egs/wsj/s5/local/nnet/run_dnn.sh</a></b>.
This script assumes to use a single CUDA GPU, and that kaldi was compiled with CUDA (check for 'CUDA = true' in src/kaldi.mk).
Also we assume that 'cuda_cmd' is set properly in egs/wsj/s5/cmd.sh either to a GPU cluster node using 'queue.pl' or to a local machine using 'run.pl'.
And finally the script assumes we already have a SAT GMM system exp/tri4b and corresponding fMLLR transforms, as generated by egs/wsj/s5/run.sh.
Note that for other databases the run_dnn.sh is typically in the same location s5/local/nnet/run_dnn.sh.

The script <a href="http:https://sourceforge.net/p/kaldi/code/HEAD/tree/trunk/egs/wsj/s5/local/nnet/run_dnn.sh">egs/wsj/s5/local/nnet/run_dnn.sh</a> is split into several stages:
The script <a href="https:https://github.com/kaldi-asr/kaldi/blob/master/egs/wsj/s5/local/nnet/run_dnn.sh">egs/wsj/s5/local/nnet/run_dnn.sh</a> is split into several stages:

0. <b>storing 40-dimensional fMLLR features to disk, steps/nnet/make_fmllr_feats.sh,</b>
this simplifies the training scripts, the 40-dimensional features are MFCC-LDA-MLLT-fMLLR with CMN
Expand Down Expand Up @@ -100,7 +100,7 @@ Besides the DNN recipe, there are also other example scripts which can be handy:
<hr><!-- #################################################################################################################### -->

\section dnn1_training_script_internals Training script internals
The main neural network training script <a href="http:https://sourceforge.net/p/kaldi/code/HEAD/tree/trunk/egs/wsj/s5/steps/nnet/train.sh">steps/nnet/train.sh</a> is invoked as:
The main neural network training script <a href="https:https://github.com/kaldi-asr/kaldi/blob/master/egs/wsj/s5/steps/nnet/train.sh">steps/nnet/train.sh</a> is invoked as:

\verbatim
steps/nnet/train.sh <data-train> <data-dev> <lang-dir> <ali-train> <ali-dev> <exp-dir>
Expand All @@ -111,11 +111,11 @@ The <lang-dir> is used only in the special case when using LDA feature-transform
The output (i.e. the trained networks and logfiles) goes into <exp-dir>.

Internally the script prepares the feature+target pipelines, generates a neural-network prototype and initialization, creates feature_transform and calls the scheduler script
<a href="http:https://sourceforge.net/p/kaldi/code/HEAD/tree/trunk/egs/wsj/s5/steps/nnet/train_scheduler.sh">steps/nnet/train_scheduler.sh</a>,
<a href="https:https://github.com/kaldi-asr/kaldi/blob/master/egs/wsj/s5/steps/nnet/train_scheduler.sh">steps/nnet/train_scheduler.sh</a>,
which runs the training epochs and controls the learning rate.


<b>While looking inside <a href="http:https://sourceforge.net/p/kaldi/code/HEAD/tree/trunk/egs/wsj/s5/steps/nnet/train.sh">steps/nnet/train.sh</a> we see:</b>
<b>While looking inside <a href="https:https://github.com/kaldi-asr/kaldi/blob/master/egs/wsj/s5/steps/nnet/train.sh">steps/nnet/train.sh</a> we see:</b>

1. CUDA is required, the scripts exit if no GPU was detected or was CUDA not compiled in (one can still use '--skip-cuda-check true' to run on CPU, but it is 10-20x slower)

Expand Down Expand Up @@ -165,12 +165,12 @@ $ cat exp/dnn5b_pretrain-dbn_dnn/nnet.proto

7. the network is initialized by : \ref nnet-initialize.cc , the DBN gets prepended in the next step using \ref nnet-concat.cc

8. finally the training gets called by running scheduler script <a href="http:https://sourceforge.net/p/kaldi/code/HEAD/tree/trunk/egs/wsj/s5/steps/nnet/train_scheduler.sh">steps/nnet/train_scheduler.sh</a>
8. finally the training gets called by running scheduler script <a href="https:https://github.com/kaldi-asr/kaldi/blob/master/egs/wsj/s5/steps/nnet/train_scheduler.sh">steps/nnet/train_scheduler.sh</a>

Note : both neural networks and feature transforms can be viewed by \ref nnet-info.cc, or shown in ascii by \ref nnet-copy.cc


<b>While looking inside <a href="http:https://sourceforge.net/p/kaldi/code/HEAD/tree/trunk/egs/wsj/s5/steps/nnet/train_scheduler.sh">steps/nnet/train_scheduler.sh</a> we see:</b>
<b>While looking inside <a href="https:https://github.com/kaldi-asr/kaldi/blob/master/egs/wsj/s5/steps/nnet/train_scheduler.sh">steps/nnet/train_scheduler.sh</a> we see:</b>

the initial cross-validation run and the main for-loop over $iter which runs the epochs and controls the learning rate. Typically, the train_scheduler.sh is called from train.sh.
- the default learning-rate scheduling is based on the relative improvement of the objective function:
Expand Down Expand Up @@ -310,7 +310,7 @@ AddMat 174.307s
AddMatMat 1922.11s
\endverbatim

<b> Running <a href="http:https://sourceforge.net/p/kaldi/code/HEAD/tree/trunk/egs/wsj/s5/steps/nnet/train_scheduler.sh">steps/nnet/train_scheduler.sh</a> directly:</b>
<b> Running <a href="https:https://github.com/kaldi-asr/kaldi/blob/master/egs/wsj/s5/steps/nnet/train_scheduler.sh">steps/nnet/train_scheduler.sh</a> directly:</b>
- The script train_scheduler.sh can be called outside train.sh, it allows to override the default NN-input and NN-target streams, which can be handy.
- However the script assumes everything is set-up correctly, and there are almost no sanity checks, which makes it suitable for more advanced users only.
- It is highly recommended to have a look at how train_scheduler.sh is usually called before trying to call it directly.
Expand Down
10 changes: 7 additions & 3 deletions src/doc/history.dox
Original file line number Diff line number Diff line change
Expand Up @@ -54,21 +54,25 @@
Sandeep Boda, Sandeep Reddy and Haihua Xu (who helped with coding, code cleanup
and documentation); we were visited by Michael Riley (who helped us to understand
OpenFst and gave some lectures on FSTs), and would like to acknowledge the help of
Honza Cernocky (for allowing us to have the workshop and helping to organize it),
Honza Cernocky (for negotiating the venue and some support for the workshop from
the Faculty of Information Technology of BUT and helping to organize it),
Renata Kohlova (administration), and Tomas Kasparek (system administration).
It is possible that this list of contributors contains
oversights; any important omissions are unlikely to be intentional.

A lot of code was written during the summer of 2010 but we still did not have a
complete working system. Some of the participants of the 2010 workshop
continued working to complete the toolkit and get a working set of training scripts.
The code was released on May 14th, 2011.
The code was released on May 14th, 2011, and presented to public at ICASSP 2011
in Prague,
<a href="https://www.superlectures.com/icassp2011/category.php?lang=en&id=131">
see the recordings</a>.

Since the initial release, Kaldi has been maintained and developed to a large
extent by Daniel Povey, working at Microsoft Research until early 2012 and
since then at Johns Hopkins University; but also with major contributions by
others: notably Karel Vesely, who developed the neural-net training framework,
and Arnab Ghoshal, who co-ordinated the acoustic modeling work early on; but
and Arnab Ghoshal, who coordinated the acoustic modeling work early on; but
also other major contributors whom we do not name here because it is too hard
to determine where to cut off the list; and a long tail of minor contributors;
the total number of people who have contributed code or scripts or patches is
Expand Down
11 changes: 11 additions & 0 deletions src/nnetbin/cuda-gpu-available.cc
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,17 @@ void TestGpuComputation() {
#endif

int main(int argc, char *argv[]) try {

/* only for Doxygen documentation, never shown in command line */
const char *usage =
"Test if there is a GPU available, and if the GPU setup is correct.\n"
"A GPU is acquired and a small computation is done\n"
"(generating a random matrix and computing softmax for its rows).\n"
"\n"
"exit-code: 0 = success, 1 = compiled without GPU support, -1 = error\n"
"\n"
"Usage: cuda-gpu-available\n";

char hostname[100] = "UNKNOWN-HOSTNAME";
#if !defined(_MSC_VER) && !defined(__CYGWIN__)
if (gethostname(hostname, 100)) {
Expand Down