Here are my solutions for the assignments of DLSYS.

Followings are the descriptions of the course.

Over the past few years, deep learning has become an important technique to successfully solve problems in many different fields, such as vision, NLP, robotics. An important ingredient that is driving this success is the development of deep learning systems that efficiently support the task of learning and inference of complicated models using many devices and possibly using distributed resources. The study of how to build and optimize these deep learning systems is now an active area of research and commercialization, and yet there isn’t a course that covers this topic.

This course is designed to fill this gap. We will be covering various aspects of deep learning systems, including: basics of deep learning, programming models for expressing machine learning models, automatic differentiation, memory optimization, scheduling, distributed learning, hardware acceleration, domain specific languages, and model serving. Many of these topics intersect with existing research directions in databases, systems and networking, architecture and programming languages. The goal is to offer a comprehensive picture on how deep learning systems works, discuss and execute on possible research opportunities, and build open-source software that will have broad appeal.

We will have two classes per week. Each week will have one lecture. Another class will either be lab/discuss session or guest lectures. Each lecture will study a specific aspect of deep learning systems. The lab/discussion session will contain tutorials to implement that specific aspect and will include case studies of existing systems, such as Tensorflow, Caffe, Mxnet, PyTorch, and others.

Tianqi Chen

Haichen Shen

Arvind Krishnamurthy

Qiao Zhang

Time: Tuesday/Thursday 1:00pm to 2:20pm.

There are two joint lecture with CSE548 happens on Wed/Friday, checkout the schedule for details

Place: Normal lectures and labs happens at MGH 254

We will be using CSE 305 for some of the guest lectures, checkout the schedule for details

Slack: Join https://uw-cse.slack.com dlsys channel for course discussions and announcements

Proficiency in Python, familar in C/C++

We will mainly be using python for case study the existing systems, and C/C++ for some of the background hacking.

Basic knowledge of machine learning (e.g CSE 546)

Prior knowledge in system (operating system/database) is useful but not required.

We will have assignments and a final project.

Course project: 60%

Homeworks: 30%

Discussion participation: 10%

This course is created with help from DLsys seminar group at University of Washington.