docs: Add comment about the generation of no-answer samples in FARMRe…

…ader training (#3404) * Add comment about no-answer generation * Add comment about no-answer generation * Fix typo Co-authored-by: Sebastian <[email protected]> * Incorporate reviewer feedback * Incorporate reviewer feedback Co-authored-by: Sebastian <[email protected]>
deepset-ai · Oct 18, 2022 · 3bf5d43 · 3bf5d43
1 parent 15a59fd
commit 3bf5d43
Showing 1 changed file with 3 additions and 0 deletions.
diff --git a/haystack/nodes/reader/farm.py b/haystack/nodes/reader/farm.py
@@ -377,6 +377,9 @@ def train(
  Checkpoints can be stored via setting `checkpoint_every` to a custom number of steps.
  If any checkpoints are stored, a subsequent run of train() will resume training from the latest available checkpoint.
 
+ Note that when performing training with this function, long documents are split into chunks.
+ If a chunk doesn't contain the answer to the question, it is treated as a no-answer sample.
+
  :param data_dir: Path to directory containing your training data in SQuAD style
  :param train_filename: Filename of training data
  :param dev_filename: Filename of dev / eval data