News

Course Info

Syllabus

Lectures

• Wednesday, 11:10-2:00 in UA 3230

Communication

https://piazza.com/class/lbhxjy4q6c83do

Office hours

• Monday, 11:15-12:15
• in-person in UA4032
• online at https://meet.google.com/ssi-osuu-pwg
• Or by appointment

Canvas (requires login)

Student work will be submitted via the course canvas site.

Description

This is an introductory graduate course in machine learning. This course will focus on both supervised and un-supervised learning methods, covering both theory and practice. The course is geared towards students who wish to develop a working knowledge of the recent advances in machine learning, and how these are applied in various domains.

Machine learning deals with how to design computer programs that learn from “experience.” Residing at the intersection of computer science and statistics, machine learning aims to extract useful information from data (often referred to as the training data) and leverages this information to create computer models capable of carrying out useful, non-trivial tasks, such as designing cars that can drive on their own, filters for blocking junk email, diagnostics tools for disease discovery, etc. By many accounts machine learning is the “greatest export” of computer science (and statistics) to other disciplines.

The course will cover the following topics:

• regression;
• classification;
• dimensionality reduction; and
• neural networks and deep learning.

Pre-requisites

The course assumes that students are comfortable with statistics, basic linear algebra, and programming.

I recommend reading Part 1 of “Deep Learning” by I. Goodfellow, Y. Bengio and A. Courville to brush up on linear algebra and statistics. The book is available at here

We will be using Python for the programming part of this course. For Python, I recommend the Anaconda distribution, which comes pre-loaded for nearly all the packages that we will be using in this course. Of course you are welcome to use any variant/distribution of Python that suits you.

Here you’ll find a number of tutorials showcasing Python use in machine learning. I strongly recommend that you become comfortable with the following four Python packages/environment:

• numpy;
• scipy;
• matplotlib; and
• jupyter notebook.

Grading

• Exam, 40%, in class on March 22
• Participation, 10%
• Project, 50%

A student needs to get 60% marks in the project to successfully complete the course.

Important dates

• Exam, March 22
• Project proposals due, Feb 16
• Project reports due, Apr 11

Ontario Tech University’s academic calendar that lists important dates (and deadlines) is available at here.

Course Calendar

Week 1

• ML Introduction
• Principal Component Analysis

Notebooks

• PCA

Week 2

• Linear regression

Notebooks

• Linear regression
• Linear regression in 2D
• Fitting polynomials
• Linear regression with Gaussian basis functions

Week 3

• A probabilistic view of linear regression
• Logistic regression
• Softmax classifier

Notebooks

• Logistic regression

Week 4

• Layered architecture
• Neural networks
• A brief history of neural networks

Notebooks

• Autodiff example

Week 5

• Gradient descent

Week 6

• Notes on linear layer
• Notes on convolutional and transposed-convolutional layers
• Notes on maxpool layer
• Notes on temporal convolutional networks

Week 7

• Convolutional networks for object detection
• Autoencoders

Week 8

• Paper presentations (check course piazza for details)
  • DenseCap: Fully Convolutional Localization Networks for Dense Captioning
  • Distilling the knowledge in a neural network
  • Generative Adversarial Networks
  • Dropout: A simple way to prevent neural networks from overfitting
  • Learning to discover cross-domain relations with generative adversarial networks
  • VQA: Visual Question Answering
  • Learning to learn by gradient descent by gradient descent

Week 9

• Paper presentations (check course piazza for details)
  • Improved Techniques for Training GANs
  • Hyperbolic Image Segmentation
  • Unsupervised Representation learning with deep convolutuional generative adversial networks
  • DeepFace: Closing the Gap to Human-Level Performance in Face Verification
  • Attention-based LSTM for Aspect-level Sentiment Classification

Week 10

• Midterm exam
• Paper presentations (check course piazza for details)
  • Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
  • Building High-level Features Using Large Scale Unsupervised Learning
  • Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations

Week 11

• Recurrent Neural Networks

Week 12

Project

The course project is an independent exploration of a specific problem within the context of this course. It is worth 50% of the marks. A project can be implementation oriented—where a student implements a machine learning algorithm—or application oriented—where a student attempts to solve a problem (of suitable difficulty) by applying machine learning techniques. Project topic will be decided in consultation with the instructor. Project grade will depend on the ideas, how well you present them in the report, how well you position your work in the related literature, how thorough are your experiments and how thoughtful are your conclusions.

Project proposal

One page project proposals are due by Feb. 16. The project report, in the form of a paper, is due by Apr. 11. Time permitting there will be an opportunity to give a short presentation about your project.

Final Report

For your final project write-up you must use ACM SIG Proceedings Template (available at the ACM website). Project report is at most 12 pages long, plus extra pages for references.