Course Schedule

Esfand 99

Session	Date	Topic
1	3 Esfand	Introduction
2	5 Esfand	Word representations (Distributional semantics, co-occurrence matrix, dimensionality reduction and SVD, language models) Readings: [cs224n-1][cs224n-1-notes]
3	10 Esfand	Word embeddings (Word2vec, GloVe) Readings: [cs224n-1][cs224n-1-notes]
4	12 Esfand	Word embeddings (Evaluation, cross-lingual space, ambiguity and sense embeddings) Readings: [cs224n-2] [cs224n-2-notes]
5	17 Esfand	Word embeddings (Sub-word embeddings, retrofitting, debiasing) Readings: [nn4nlp2021]
6	19 Esfand	Text classification and regression Readings: [info256-5][info256-6]
7	24 Esfand	Language modeling (n-gram, probability computation, back-off interpolation, sparsity and smoothing, feedforward NN for LM) Readings: [cs224n-5][Voita-LM]
8	26 Esfand	Language modeling with RNNs (backprop through time, text generation, perplexity, text generation, sampling with temprature) Readings: [cs224n-5][Voita-LM]

Farvardin 00

Session	Date	Topic
9	15 Farvardin	Vanishing/exploding gradients and fancy RNNs (LSTMs, bidirectional and stacked RNNs) Readings: [cs224n-6] [cs224n-6-notes]
10	17 Farvardin	Machine Translation (SMT, NMT, seq2seq models, beam-search decoding, evaluation) Readings: [cs224n-7] [cs224n-7-notes]
11	22 Farvardin	Paper discussion on RNNs
12	24 Farvardin	Attention mechanism (seq2seq attention, attention variants, hierarchical attention networks) Readings: [cs224n-7] [cs224n-7-notes]
13	29 Farvardin	Progress Report I
14	31 Farvardin	Word senses and contextualization (skipped)

Ordibehesht 00

Session	Date	Topic
15	5 Ordibehesht	Transformers (BERT model, self-attention, multi-head, positional encoding, contextualised embeddings, derivatives of BERT) Readings: [slides] [cs224n-9]
16	7 Ordibehesht	More about Transformers and Pretraining (subwords, byte-pair encoding, pretrain/finetune, architecture types: decoders, encoders, and encoder-decoders) Readings: [cs224n-10]
17	12 Ordibehesht	Paper discussion on Transformers
18	19 Ordibehesht	*Isotropicity of Semantic Spaces (Rajaee) Readings: [slides]
19	21 Ordibehesht	Question Answering (reading comprehension, SQuAD, LSTM-based and BERT models, BiDAF, open-domain QA) Readings: [cs224n-11]
20	26 Ordibehesht	Progress Report II
21	28 Ordibehesht	*LM-based Word Sense Disambiguation (Rezaee) Readings: [slides]

Khordad 00

Session	Date	Topic
22	2 Khordad	*Interpretability (Modaressi & Mohebbi) Readings: [slides]
23	4 Khordad	*Dialogue (Pourdabiri) Readings: [slides]
24	9 Khordad	Integrating knowledge in language models (knowledge-aware LMs, entity embedding, ERNIE, memory-based models, KGLM, kNN-LM, modified training, WKLM, evaluation, prompting) Readings: [cs224n-15]
25	11 Khordad	Neural Language Generation (applications, maximum likelihood training, teacher forcing, greedy and random sampling, top-k and nucleus sampling, unlikelihood training, exposure bias, evaluating NLG, bias and ethical concerns) Readings: [cs224n-12]
26	18 Khordad	*Zero-shot applictions of Cloze test (Tabasi) Readings: [slides]
27	23 Khordad	Paper discussion on knowledge-enhanced models
28	25 Khordad	Progress Report III