CS 559: Deep Learning

Fall 2024

Instructor: Hamdi Dibeklioglu

Office hours: by appointment / online

Class hours:

Tuesday 10:30-12:20, Thursday 15:30-17:20
While every other week we will have 2+2 hours of lectures, other weeks there will be one 2-hours lecture. Please see the schedule below for the dates of the lectures.

Class room:

EB-101

UPDATE:

All important dates (including homework, project/project progress, and survey submission/presentation deadlines, and midterm date) have been announced.
Homework submission deadline has been updated as 24.11.2024.
Homework details have been announced, please see the homework section below (24.10.2024).
Lectures start as of 17 September 2024.

Catalog Description: Overview of machine learning and its applications. Loss functions, numerical optimization and back-propagation. Fundamentals of feedforward neural networks. Modern architectures and techniques for training deep networks. Convolutional neural networks: basics, visualization, and techniques for efficient spatial localization in images. Recurrent neural networks and their variants. Applications of recurrent neural networks in language and image understanding, and image captioning. Recent advances in generative models learning, generative adversarial networks and variational auto encoders. Unsupervised and self-supervised representation learning. Deep reinforcement learning.

Recommended Books:

I. Goodfellow, Y. Bengio, A. Courville. Deep Learning, MIT Press, 2016. [Available Online]
K. P. Murphy. Machine Learning: A Probabilistic Perspective, MIT Press, 2012.
C. M. Bishop. Pattern Recognition and Machine Learning, Springer, 2006.

Assessment Methods:

Homework	20%
Literature Survey and Presentation	10% + %5
Midterm:Essay/written	30%
Project	35%

Any of the following will directly result in an F grade:

not submitting a project or homework (including report),
not preparing/presenting a survey on the pre-scheduled date,
being absent in the midterm,
being absent in a project presentation.

Passing Grade: No predetermined grade to pass the course.

Makeup Policy: Medical report holders will be entitled for the midterm make up. Makeup exam will be comprehensive.

Homework:
Please follow this link to access the homework details. Homework details/materials are accessible only within Bilkent network. Use VPN to access from home.

Literature Survey and Presentation:

Groups of three (if required four) students will choose a topic related to deep learning, and prepare a short survey on it.
Surveys should be based on about 12 papers (report: 5 pages max.).
You will make a presentation on your survey in class. The presentation should be in parallel with your report.
Survey topics should be confirmed first. Very similar topics to others’ will not be allowed (priority: first come, first served).
Your chosen survey topic and a few lines explanation (indicating group members) should be sent to dibeklioglu@cs.bilkent.edu.tr by 30 September 2024, 23.59 (Turkey time).

Project:

By groups of three or four students.
Explore novel applications of contemporary deep learning techniques or develop novel deep learning techniques.
Projects related to your research topics are encouraged.
Three stages:

Proposal: one-page description of the project topic and the planning for the project (indicating group members) should be sent to dibeklioglu@cs.bilkent.edu.tr by 2 October 2024 (23:59 Turkey time)
Progress report & presentation (report: 2 pages max.)
Final report & presentation (report: 5 pages max.)

You are allowed and encouraged to use mainstream deep learning libraries like TensorFlow, PyTorch, Torch, etc.

Template for the Reports:
All reports (including homework report) must be prepared using the IEEE double-column transactions article template (i.e. "bare_jrnl.tex").

Important Dates:

Event	Date / Deadline
Midterm Exam	05 December 2024
Literature Survey Proposal submission [subject line: cs559_2024f_survey]	30 September 2024, 23:59
Literature Survey Report submission (including the presentation) [via Moodle]	10 December 2024, 23:59
Literature Survey Presentation	10/12 December 2024
Homework submission (including the report) [via Moodle]	22 November 2024,23:59 24 November 2024,23:59
Project Proposal submission [subject line: cs559_2024f_project]	2 October 2024, 23:59
Project Progress Presentation	21 November 2024
Project Progress Report submission (including the report and presentation) [via Moodle]	21 November 2024, 23:59
Project Final Presentation	24 December 2024
Project submission (including the report and presentation) [via Moodle]	25 December 2024, 23:59

Tentative Schedule & Lecture Notes:
Lecture notes below are downloadable only within Bilkent network. Use VPN to access from home.

Week	Topic	Dates	Lecture Notes
1	Introduction, Basics	17 September 2024 (10:30-12:20) 19 September 2024 (15:30-17:20)
2	Loss Functions	24 September 2024 (10:30-12:20)
3	Optimization Feedforward networks and training (1)	01 October 2024 (10:30-12:20) 03 October 2024 (15:30-17:20)
4	Feedforward networks and training (2)	08 October 2024 (10:30-12:20)
5	Convolutional neural networks	15 October 2024 (10:30-12:20) 17 October 2024 (15:30-17:20)	>>
6	Spatial localization and detection	22 October 2024 (10:30-12:20)
7	Segmentation	31 October 2024 (15:30-17:20)
8	Understanding and Visualizing CNNs	05 November 2024 (10:30-12:20)
9	Recurrent Neural networks Word Embeddings and Language Models	12 November 2024 (10:30-12:20) 14 November 2024 (15:30-17:20)
10	Unsupervised Learning and Generative Models, Project Progress Presentations	19 November 2024 (10:30-12:20) 21 November 2024 (15:30-18:00)
11	No Lecture	N/A
12	Unsupervised Learning and Generative Models, Midterm	03 December 2024 (10:30-12:20) 05 December 2024 (15:30-17:20)
13	Presentations
14	Deep reinforcement learning
15	Presentations