Fork me on GitHub

Deep Learning Guest Lecture

GoogLeNet (2014), a convolutional network with 22 layers.

A motivational talk about deep artificial neural networks, given to the students in FFR135 (Artificial neural networks). I gave motivations for using deep architechtures, and to learn hierarchical representations for data.

Reading (web):

Reading (publications):

  • A Fast Learning Algorithm for Deep Belief Nets; Hinton, Osindero, Tehi; Neural Computation; 2006 PDF,
  • Exploring Strategies for Training Deep Neural Networks; Larochelle, Bengio, Louradour, Lamblin; JMLR 2009 PDF,
  • Imagenet classification with deep convolutional neural networks (“AlexNet”); Krizhevsky, Sutskever, Hinton, NIPS 2012; PDF,
  • Very deep convolutional networks for large-scale image recognition (“VGGNet”); Simonyan, Zisserman; 2014; arXiv 1409.1556; PDF, arXiv
  • Going Deeper with Convolutions (“GoogLeNet”); Szegedy, Liu, Jia, Sermanet, Reed, Anguelov, Erhan, Vanhoucke, Rabinovich; 2014; arXiv 1409.4842; PDF, arXiv
  • Improving neural networks by preventing co-adaptation of feature detectors; Hinton, Srivastava, Krizhevsky, Sutskever, Salakhutdinov; 2012; arXiv:1207.0580 PDF, arXiv
  • Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift; Ioffe, Szegedy; arXiv:1502.03167 PDF, arXiv
  • Deep Residual Learning for Image Recognition (“ResNet”); He, Zhang, Ren, Sun; arXiv:1512.03385 PDF, arXiv
  • The loss surfaces of multilayer networks; Choromanska, Henaff, Mathieu, Arous, LeCun; AISTATS 2015 PDF, arXiv
  • Identifying and attacking the saddle point problem in high-dimensional non-convex optimization; Dauphin, Pascanu, Gulcehre, Cho, Ganguli, Bengio; NIPS 2014 PDF,
  • Sequence to Sequence Learning with Neural Networks, Ilya Sutskever, Oriol Vinyals, Quoc V. Le. NIPS 2014 PDF, arXiv
  • Neural Machine Translation of Rare Words with Subword Units, Rico Sennrich and Barry Haddow and Alexandra Birch, ACL 2016: PDF,
  • Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation, Yonghui Wu (Google): PDF, arXiv

Slides (PDF)

FFR135, Artificial Neural Networks, 2016-10-06
Olof Mogren

Olof Mogren, PhD, RISE Research institutes of Sweden. Follow me on Bluesky.