Penalized and generalized linear models: Logistic regression, L1 and L2
penalization, elastic net, SCAD penalty, application to high dimensional
low sample size problems.
Intro to neural networks: Artificial neuron, single hidden layer, multiple
hidden layer, back propagation, momentum, loss functions, relation with
support vector machines and penalized logistic regression.
Convolutional neural networks: Convolutional layers, pooling layers, drop
out, VGGnet, inception modules, residual networks, deconv nets,
applications to object recognition.
Why deep learning works: Role of depth, closeness of local minima to
global minimal, predominance of saddle points and ridges vs. local minima.
Recurrent neural networks and LSTMs: lateral connections, LSTM units,
gated recurrent networks, applications to NLP.
Probabilistic Graphical Models: Factor graphs and belief networks, Deep
belief networks and Boltzmann machines, sampling methods including Gibbs
sampling, contrastive divergence, generative adversarial networks.
Deep Learning by Goodfellow, Bengio and Courville.
Probabilistic Graphical Models: Principles and Techniques by Koller and