Updates

README.md

@@ -34,7 +34,7 @@ Phrases similarity with word embeddings model + word based machine translation w
 </td>
 </table>
 
-### Week 2: *Word Embeddings: Part 2* 
+### Week 3: *Word Embeddings: Part 2* 
 Introduction to PyTorch. Implementation of pet linear regression on pure numpy and pytorch. Implementations of CBoW, skip-gram, negative sampling and structured Word2vec models. 
 
 <table class="tfo-notebook-buttons" align="left">
@@ -46,4 +46,32 @@ Introduction to PyTorch. Implementation of pet linear regression on pure numpy a
  <a target="_blank" href="https://github.com/DanAnastasyev/DeepNLP-Course/blob/master/Week%2003/Week_03_Word_Embeddings_(Part_2).ipynb">
  <img width=32px src="https://www.tensorflow.org/images/GitHub-Mark-32px.png" />View source on GitHub</a>
 </td>
+</table>
+
+### Week 4: *Convolutional Neural Networks* 
+Introduction to convolutional networks. Relations between convolutions and n-grams. Simple surname detector on character-level convolutions + fun visualizations.
+
+<table class="tfo-notebook-buttons" align="left">
+<td>
+ <a target="_blank" href="https://colab.research.google.com/drive/1Vo_yuiA7xLjavUA_7ayLeosGJyMsyDAt">
+ <img src="https://www.tensorflow.org/images/colab_logo_32px.png" />Run in Google Colab</a> 
+</td>
+<td>
+ <a target="_blank" href="https://github.com/DanAnastasyev/DeepNLP-Course/blob/master/Week%2004/Week_04_Convolutional_Neural_Networks.ipynb">
+ <img width=32px src="https://www.tensorflow.org/images/GitHub-Mark-32px.png" />View source on GitHub</a>
+</td>
+</table>
+
+### Week 5: *RNNs: Part 1* 
+RNNs for text classification. Simple RNN implementation + memorization test. Surname detector in multilingual setup: character-level LSTM classifier.
+
+<table class="tfo-notebook-buttons" align="left">
+<td>
+ <a target="_blank" href="https://colab.research.google.com/drive/1-FoMnf7s-BYNM7jT9UF3u9m63h7dSq3_">
+ <img src="https://www.tensorflow.org/images/colab_logo_32px.png" />Run in Google Colab</a> 
+</td>
+<td>
+ <a target="_blank" href="https://github.com/DanAnastasyev/DeepNLP-Course/blob/master/Week%2005/Week_05_RNNs_Intro.ipynb">
+ <img width=32px src="https://www.tensorflow.org/images/GitHub-Mark-32px.png" />View source on GitHub</a>
+</td>
 </table>

Week 03/README.md

@@ -2,5 +2,5 @@
 
 - Задания, сделанные на семинаре: 5 баллов
 - Доделать CBoW модель: +1 балл
-- Реализовать Negative Sampling: +3 балл
+- Реализовать Negative Sampling: +4 балл
 - Structured Word2Vec: +2 балла

Week 03/Week_03_Word_Embeddings_(Part_2).ipynb

@@ -156,7 +156,14 @@
  },
  "cell_type": "markdown",
  "source": [
- "А затем говорим ей: \"Посчитай градиенты, пожалуйста\""
+ "А затем говорим ей: \"Посчитай градиенты, пожалуйста\". И происходит магия - какая-то такая:\n",
+ "\n",
+ "![graph](https://raw.githubusercontent.com/pytorch/pytorch/master/docs/source/_static/img/dynamic_graph.gif) \n",
+ "*From [github.com/pytorch/pytorch](https://github.com/pytorch/pytorch)*\n",
+ "\n",
+ "По описанной последовательности операций *на лету* строится граф вычислений, и обратный проход выполняется по нему.\n",
+ "\n",
+ "В этом ключевое отличие от tensoflow: граф не нужно компилировать до исполнения кода - это позволяет более гибко управлять его структурой."
  ]
  },
  {
@@ -799,8 +806,8 @@
  " central_word, context = central_words[data_ind], contexts[data_ind]\n",
  " \n",
  " words_to_use = random.sample(context, num_skips)\n",
- " batch_data.extend(words_to_use)\n",
- " batch_labels.extend([central_word] * num_skips)\n",
+ " batch_data.extend([central_word] * num_skips)\n",
+ " batch_labels.extend(words_to_use)\n",
  " \n",
  " yield batch_data, batch_labels"
  ],
@@ -1582,7 +1589,8 @@
  "[Skip-Thought Vectors (2015), Kiros, et al.](https://arxiv.org/pdf/1506.06726) \n",
  "\n",
  "### Backpropagation\n",
- "[Backpropagation, Intuitions, cs231n + next parts in the Module 1](https://cs231n.github.io/optimization-2/) \n",
+ "[Backpropagation, Intuitions, cs231n + next parts in the Module 1](https://cs231n.github.io/optimization-2/) \n",
+ "[Calculus on Computational Graphs: Backpropagation, Christopher Olah](https://colah.github.io/posts/2015-08-Backprop/)\n",
  "\n",
  "## Посмотреть\n",
  "[cs224n \"Lecture 2 - Word Vector Representations: word2vec\"](https://www.youtube.com/watch?v=ERibwqs9p38&index=2&list=PLqdrfNEc5QnuV9RwUAhoJcoQvu4Q46Lja&t=0s) \n",

Week 04/README.md

@@ -0,0 +1,6 @@
+# Разбалловка
+
+- Задания, сделанные на семинаре: 5 баллов
+- Доделать модель и оценить её качество - +1 балл
+- Визуализировать модель - +1 балл
+- Поэкспериментировать с улучшением модели - +1 балл