CSCI 315 : Artificial Intelligence through Deep Learning


Professor: Simon D. Levy
Lecture: M/W/F 1:25 - 2:20 Science Addition 114
Office: Parmly 407B
E-mail: simon.d.levy@gmail.com
Office Hours: MWF 10:00-11:00, 2:30-3:30 and by appointment


Textbook: Nikhil Budima, Fundamentals of Deep Learning (O'Reilly 2017), available to W&L students online

Note that we are not covering the textbook chapters in strict order. Until we begin learning TensorFlow, you can skip over the textbook material on that technology. Once you start working with the TensorFlow examples in the book, you will likely encounter typos. Fixing these typos will be part of the assignments! Despite these typos, Buduma's book is currently the most accessible introduction to DeepLearning.


Objectives

The goal of this course is to give you the skills and knowledge to participate in the exciting new field of Deep Learning. For the first half of the course you will learn to design, train, and test neural networks using the NumPy package in Python. In the second half of the course you will learn how to use the popular TensorFlow Python package to train and test much more powerful deep-learning networks that exploit the Graphical Processing Unit (GPU) available on our computers. In addition to being able to design, train, and test Deep Learning networks, you will gain an understanding of the history and philolsophy of AI, the current challenges it faces, and the prospects for the future.


Attendance and Preparation

I look at this course as preparation for professional work in a research or industry setting, and I expect you to act professionally: show up for every class, participate fully, and submit your work on time without excuses. Consistent with our university's mission statement, I expect everyone to conduct themselves with honor, integrity, and civility: if you are talking, texting, or otherwise causing a distraction in class, I will ask you to leave.


Grading

All work should be submitted through Sakai as Python .py files. The fast pace of the course means that no late work can be accepted.The only three exceptions to this rule are: As a way of helping with unanticipated emergencies and bad days, I will drop your lowest exam or project grade.Given the size of the class and the amount of work involved, there will be no opportunity for extra credit if you are not happy with your grade as the end of the course approaches. Serious problems (health / family / personal emergencies) should be handled through the Office of the Dean.

The grading scale will be 93-100 A; 90-92 A-; 87-89 B+; 83-86 B; 80-82 B-; 77-79 C+; 73-76 C; 70-72 C-; 67-69 D+; 63-66 D; 60-62 D-; below 60 F.


Accommodations

Washington and Lee University makes reasonable academic accommodations for qualified students with disabilities. All undergraduate accommodations must be approved through the Office of the Dean of the College. Students requesting accommodations for this course should present an official accommodation letter within the first two weeks of the (fall or winter) term and schedule a meeting outside of class time to discuss accommodations. It is the student's responsibility to present this paperwork in a timely fashion and to follow up about accommodation arrangements. Accommodations for test-taking should be arranged with the professor at least a week before the date of the test or exam.


Final Project

In lieu of a big final exam, I would like to see what kinds of cool projects you can devise. Given our resource constraints and large class size, I encourage you to work in teams of up to four people.

You may elect to do a short (around 10 pages double-spaced) paper instead of a final project. This could be a discussion (with references of course) about a current issue of interest or controversy in AI. Here is a possible topic to look into.

A third option for your project would be to look at the other popular deep-learning textbook. You could either (a) do a project from that book or (b) write a short review of the comparative merits of these two books.


Schedule, Including Due Dates and On-line Class Notes

For each exam, you are responsible for all linked material posted before that exam.

Monday

Wednesday

Friday

04 Sep
Week 0

Course Outline

What is (A)I?

The Myth of a Superhuman AI

11 Sep
Week 1

Neural Net Intro Video
(sign up for Coursera)

Deep Learning Intro Article

Bengio's Deep Learning Intro (Section I)

Why Neural Nets?

Linear Least Squares

Lecture: Perceptron Learning

Reading: Buduma Chapter 1

 Due: Assignment #1

18 Sep
Week 2

Continue: Perceptron Learning Limits of Perceptrons Review Weeks 1-3

25 Sep
Week 3

Exam #1

Back-propagation with hidden units

Reading: Buduma Chapter 2

Backprop: One Weird Trick

Cheat Sheet

Guest lecture by Upol Ehsan '13

Due: Assignment #2

02 Oct
Week 4

Back-prop, continued Back-prop II: Improvements

Reading: Buduma Chapter 4

Back-prop II, continued

09 Oct
Week 5

Back-prop II, continued Logistic Regression & Soft-Max Reading Day; no class

16 Oct
Week 6

Logistic Regression & Soft-Max

Reading: Buduma Chapter 3

Due: Assignment #3

Logistic Regression & Soft-Max  Exam #2

23 Oct
Week 7

Exam #2 follow-up Intro to TensorFlow

Reading: Buduma Chapter 3

TensorFlow, continued

30 Oct
Week 8

TensorFlow Part II

Due: Assignment #4

TensorFlow II, continued TensorFlow II, concluded

06 Nov
Week 9

Convolutional Networks

Reading: Buduma Chapter 5

Convolutional Networks, continued

Due: Assignment #5

Convolutional Networks, concluded

13 Nov
Week 10

Recurrent Networks

Optional Reading:
The Unreasonable Effectiveness of Recurrent Networks (alluding to this classic)

Recurrent Networks Recurrent Networks

Due: Assignment #6

Due: Final Project Proposal (email or conversation team with team)

27 Nov
Week 11

Recurrent Networks

Reading: Buduma Chapter 7

Reading: Understanding LSTM Networks

Recurrent Networks, continued Recurrent Networks, continued

04 Dec
Week 12

Recurrent Networks, concluded Review for Exam #3 Exam #3

11 Dec
Finals Week

 Project Presentations