DOC: Clarify that CrossEntropyLoss mean is weighted (pytorch#40991)

Summary:
Closes pytorch#40560

This adds the equation for the weighted mean to `CrossEntropyLoss`'s docs and the `reduction` argument for `CrossEntropyLoss` and `NLLLoss` no longer describes a non-weighted mean of the outputs.

Pull Request resolved: pytorch#40991

Differential Revision: D22395805

Pulled By: ezyang

fbshipit-source-id: a623b6dd2aab17220fe0bf706bd9b62d6ba531fd

torch/nn/modules/loss.py

@@ -157,11 +157,12 @@ class NLLLoss(_WeightedLoss):
  on :attr:`size_average`. When :attr:`reduce` is ``False``, returns a loss per
  batch element instead and ignores :attr:`size_average`. Default: ``True``
  reduction (string, optional): Specifies the reduction to apply to the output:
- ``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will be applied,
- ``'mean'``: the sum of the output will be divided by the number of
- elements in the output, ``'sum'``: the output will be summed. Note: :attr:`size_average`
- and :attr:`reduce` are in the process of being deprecated, and in the meantime,
- specifying either of those two args will override :attr:`reduction`. Default: ``'mean'``
+ ``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will
+ be applied, ``'mean'``: the weighted mean of the output is taken,
+ ``'sum'``: the output will be summed. Note: :attr:`size_average`
+ and :attr:`reduce` are in the process of being deprecated, and in
+ the meantime, specifying either of those two args will override
+ :attr:`reduction`. Default: ``'mean'``
 
  Shape:
  - Input: :math:`(N, C)` where `C = number of classes`, or
@@ -884,7 +885,11 @@ class CrossEntropyLoss(_WeightedLoss):
  .. math::
  \text{loss}(x, class) = weight[class] \left(-x[class] + \log\left(\sum_j \exp(x[j])\right)\right)
 
- The losses are averaged across observations for each minibatch.
+ The losses are averaged across observations for each minibatch. If the
+ :attr:`weight` argument is specified then this is a weighted average:
+
+ .. math::
+ \text{loss} = \frac{\sum^{N}_{i=1} loss(i, class[i])}{\sum^{N}_{i=1} weight[class[i]]}}
 
  Can also be used for higher dimension inputs, such as 2D images, by providing
  an input of size :math:`(minibatch, C, d_1, d_2, ..., d_K)` with :math:`K \geq 1`,
@@ -908,11 +913,12 @@ class CrossEntropyLoss(_WeightedLoss):
  on :attr:`size_average`. When :attr:`reduce` is ``False``, returns a loss per
  batch element instead and ignores :attr:`size_average`. Default: ``True``
  reduction (string, optional): Specifies the reduction to apply to the output:
- ``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will be applied,
- ``'mean'``: the sum of the output will be divided by the number of
- elements in the output, ``'sum'``: the output will be summed. Note: :attr:`size_average`
- and :attr:`reduce` are in the process of being deprecated, and in the meantime,
- specifying either of those two args will override :attr:`reduction`. Default: ``'mean'``
+ ``'none'`` | ``'mean'`` | ``'sum'``. ``'none'``: no reduction will
+ be applied, ``'mean'``: the weighted mean of the output is taken,
+ ``'sum'``: the output will be summed. Note: :attr:`size_average`
+ and :attr:`reduce` are in the process of being deprecated, and in
+ the meantime, specifying either of those two args will override
+ :attr:`reduction`. Default: ``'mean'``
 
  Shape:
  - Input: :math:`(N, C)` where `C = number of classes`, or