COMPSCI 682 Neural Networks: A Modern Introduction

Note

  • This is a tentative class outline and is subject to change throughout the semester.
  • Some slides listed here are from previous semsesters. If there are changes, slides will be updated after each lecture.
Event TypeDateDescriptionCourse Materials
Lecture Tuesday, Sept. 3 Intro to Deep Learning, historical context. [slides]
[python/numpy tutorial]
[jupyter tutorial]
Lecture Thursday, Sept. 5 Image classification and the data-driven approach
k-nearest neighbor
Linear classification		 [slides]
[image classification notes]
[linear classification notes]
Lecture Tuesday, Sept. 10 Loss functions [slides]
Lecture Thursday, Sept. 12 Optimization: Stochastic Gradient Descent and Backpropagation [slides]
[optimization notes]
Optional Discussion Friday, Sept. 13 (9:00-10:00am CS140) Slicing and broadcasting in Python [slicing and broadcasting ipynb]
Lecture Tuesday, Sept. 17 Backpropagation & Neural Networks I [slides]
[backprop notes]
[Efficient BackProp] (optional)
related: [1], [2], [3] (optional)
Lecture Thursday, Sept. 19 Neural Networks II
Higher-level representations, image features
Vector, Matrix, and Tensor Derivatives 		[slides]
handout 1: Vector, Matrix, and Tensor Derivatives
handout 2: Derivatives, Backpropagation, and Vectorization
Deep Learning [Nature] (optional)
Optional Discussion Friday, Sept. 20 (09:00-10:00 CS140) Vector, Matrix, Tensor Derivatives and Backpropagation [notes]
Lecture Tuesday, Sept. 24 Neural Networks III
Training Neural Networks I
Activation Functions 		[slides]
[Neural Nets notes 1] tips/tricks: [1], [2] (optional)
Lecture Thursday, Sept. 26 Training Neural Networks II:
weight initialization, batch normalization 		[slides]
[Neural Nets notes 2]
[Batch Norm]
Copula Normalization (optional)
Lecture Tuesday, Oct. 1 Training Neural Networks III:
babysitting the learning process, hyperparameter optimization 		[slides]
[Bengio 2012] (optional)
Lecture Thursday, Oct. 3 Training Neural Network IV:
model ensembles, dropout 		[slides]
[Neural Nets notes 3]
LeNet (optional)
Optional Discussion Friday, Oct. 4 (09:00-10:00 CS142) A closer look at the maths inside batch normalization [notes]
Lecture Tuesday, Oct. 8 Training Neural Network V:
parameter updates
 [slides]
Lecture Thursday, Oct. 10 Convolutional Neural Networks:
convolution layer, pooling layer, fully connected layer
[slides]
Optional Discussion Friday, Oct. 11 (09:00-10:00 CS140) Convolutional neural networks [slides]
No Class (Monday Schedule) Tuesday, Oct. 15 Monday class schedule will be followed
Lecture Thursday, Oct. 17 Convolutional Neural Networks: (cont.)
convolution layer, pooling layer, fully connected layer
Lecture Tuesday, Oct. 22 ConvNets for spatial localization, Object detection
 [slides]
FCN (optional)
mAP (optional)
Lecture Thursday, Oct. 24 ConvNets for spatial localization, Object detection (cont.)
TA Lecture Tuesday, Oct. 29 Special topic (lecture given by Hang Su): Content-adaptive Convolutions [slides]
PACNet (optional)
TA Lecture Thursday, Oct. 31 Special topic (lecture given by Pia Bideau): Optical flow prediction with PWCNet PWCNet (optional)
Lecture Tuesday, Nov. 5 Recurrent Neural Networks (RNN) [slides]
DL book RNN chapter (optional)
min-char-rnn, char-rnn, neuraltalk2 (optional)
Lecture Thursday, Nov. 7 Long Short Term Memory (LSTM)
The Unreasonable Effectiveness of RNN (optional)
Understanding LSTM Networks (optional)
Lecture Tuesday, Nov. 12 RNN and LSTM (cont.)
Lecture Thursday, Nov. 14 Understanding and visualizing Convolutional Neural Networks
Backprop into image: Visualizations, deep dream 		[slides]
[visualization notes]
Lecture Tuesday, Nov. 19 Visualization (cont.)
Lecture Thursday, Nov. 21 Generative Models and GANs [slides]
No Class Tuesday, Nov. 26 Thanksgiving recess
No Class Thursday, Nov. 28 Thanksgiving recess
Lecture Tuesday, Dec. 3 Training ConvNets in practice [slides]
Lecture Thursday, Dec. 5 Guest Lecture by Evan Shelhamer
Lecture Tuesday, Dec. 10 Additional topics (TBD)
Final Monday, Dec. 16 Final exam (8am-10am in Totman Gym)