Deep Learning Guest Lecture
A motivational talk about deep artificial neural networks, given to the students in FFR135 (Artificial neural networks). I gave motivations for using deep architechtures, and to learn hierarchical representations for data.
Reading (web):
- My blog post about neural machine translation
- Deep Learning Book, Goodfellow, Bengio, Courville
- playground.tensorflow.org
- cs231n.stanford.edu - Stanford's course on convnets, with nice browser-based demo!
Reading (publications):
- A Fast Learning Algorithm for Deep Belief Nets; Hinton, Osindero, Tehi; Neural Computation; 2006 PDF, cs.toronto.edu
- Exploring Strategies for Training Deep Neural Networks; Larochelle, Bengio, Louradour, Lamblin; JMLR 2009 PDF, jmlr.org
- Imagenet classification with deep convolutional neural networks (“AlexNet”); Krizhevsky, Sutskever, Hinton, NIPS 2012; PDF, cs.toronto.edu
- Very deep convolutional networks for large-scale image recognition (“VGGNet”); Simonyan, Zisserman; 2014; arXiv 1409.1556; PDF, arXiv
- Going Deeper with Convolutions (“GoogLeNet”); Szegedy, Liu, Jia, Sermanet, Reed, Anguelov, Erhan, Vanhoucke, Rabinovich; 2014; arXiv 1409.4842; PDF, arXiv
- Improving neural networks by preventing co-adaptation of feature detectors; Hinton, Srivastava, Krizhevsky, Sutskever, Salakhutdinov; 2012; arXiv:1207.0580 PDF, arXiv
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift; Ioffe, Szegedy; arXiv:1502.03167 PDF, arXiv
- Deep Residual Learning for Image Recognition (“ResNet”); He, Zhang, Ren, Sun; arXiv:1512.03385 PDF, arXiv
- The loss surfaces of multilayer networks; Choromanska, Henaff, Mathieu, Arous, LeCun; AISTATS 2015 PDF, arXiv
- Identifying and attacking the saddle point problem in high-dimensional non-convex optimization; Dauphin, Pascanu, Gulcehre, Cho, Ganguli, Bengio; NIPS 2014 PDF, papers.nips.cc
- Sequence to Sequence Learning with Neural Networks, Ilya Sutskever, Oriol Vinyals, Quoc V. Le. NIPS 2014 PDF, arXiv
- Neural Machine Translation of Rare Words with Subword Units, Rico Sennrich and Barry Haddow and Alexandra Birch, ACL 2016: PDF, aclweb.org
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation, Yonghui Wu et.al. (Google): PDF, arXiv
FFR135, Artificial Neural Networks, 2016-10-06
Olof Mogren