A deep learning primer

Institute of Space Technology AI Lecture Series, September 2022


A three-day mini-course that introduces students to techniques, methods, and practices needed to start working on deep learning. The course has two objectives: 1) introduce students to theoretical concepts in deep learning, including autoencoders, tSNE for data visualization, visual object detection, and recurrent neural networks; and 2) provide hands on training on how to develop a deep learning system using Python+PyTorch ecosystem. The selected topics provide an opportunity to discuss common concepts, such as unsupervised learning and generative models, deep features, techniques for understanding the inner working of deep networks, and sequence modeling.


Faisal Qureshi

Email: faisal.qureshi@ontariotechu.ca
Web: http://vclab.science.ontariotechu.ca

Forward

This mini-course covers a set of topics in deep learning. The course is organized as follows. Each day begins with a 90 minutes lecture that discusses the topic of the day. It is followed by a 120 minute hand-on session where students are encouraged to implement a deep learning system closely related to the discussed topic. We plan to use Python+PyTorch eco-system for implementation. It is expected that students have prior experience in Python programming.

Notes


Convolutional networks for object detection
  • A series of lectures on convolutional networks for object detection.
    • A brief history of neural networks
    • Machine learning
    • Linear regression
    • Neural networks
    • Visual object detection

Gradient Descent
  • Minimizing loss and the need for numerical techniques
  • Gredient desent
    • Recipe
    • Update rule
  • Batch update
  • Mini-batch update
  • Stochastic (or online) gradient descent
  • Learning rate
    • Changing learning rate to achieve faster convergence
  • Newton’s method
    • How to choose a step size?
  • Momentum

tSNE
  • Stochastic Neighbour Embedding (SNE)
  • t-Distributed Stochastic Neighbour Embeding (t-SNE)

Autoencoders
  • Autoencoders
  • Variational autoencoders
  • Class-conditional autoencoders

Recurrent Neural Networks
  • Sequential processing of fixed inputs
  • Recurrent neural networks
  • LSTM

Python development
  • A series of sketches on how to setup a Python development environment.
    • Python development hello world
    • Setting up Python development environments
    • Docker setup for Python development

Copyright and License

© Faisal Qureshi

Creative Commons Licence
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.


Last update: 2022-09-08 19:49
Webify version: 4.23
© Faisal Qureshi