Designing Efficient Deep Learning Systems - Spring Session

Designing Efficient Deep Learning Systems Graphic Header

This course will also be offered on MIT's campus July 23-24

Deep learning is widely used for many artificial intelligence (AI) applications including computer vision, speech recognition, robotics, etc. While deep learning delivers state-of-the-art accuracy on many AI tasks, it comes at the cost of high computational complexity. Accordingly, designing efficient hardware systems to support deep learning is an important step towards enabling its wide deployment, particularly for embedded applications such as mobile, Internet of Things (IOT), and drones.

This course aims to provide a comprehensive tutorial and survey about the recent advances towards enabling the efficient processing of deep learning. Specifically, it will provide an overview of deep learning, discuss various hardware platforms and architectures that support deep learning, and highlight key trends in recent efficient processing techniques that reduce the cost of computation for deep learning either solely via hardware design changes or via joint hardware design and network algorithm changes. It will also summarize various development resources that can enable researchers and practitioners to quickly get started on deep learning design, and highlight important benchmarking metrics and design considerations that should be used for evaluating the rapidly growing number of deep learning hardware designs, optionally including algorithmic co-design, being proposed in academia and industry.

EARN A PROFESSIONAL CERTIFICATE IN MACHINE LEARNING AND ARTIFICIAL INTELLIGENCE

This course may be taken individually or as an elective for the Professional Certificate Program in Machine Learning and Artificial Intelligence.

Read More

Lead Instructor(s): 

Vivienne Sze

Dates: 

Mar 28, 2018 - Mar 29, 2018

Course Length: 

2 Days

Course Fee: 

$2,500

CEUs: 

1.2

Status: 

  • Closed
This course will be held at the Samsung Research America Mountain View Campus, California (additional information in the Location section below)

Participant Takeaways: 

  • Understand the basics of deep learning, how it is applied to various applications, and how it is processed on various platforms
  • Outline the key design considerations for deep learning systems
  • Be able to evaluate different deep learning implementations with benchmarks and comparison metrics
  • Understand the strengths and weakness of various hardware architectures and platforms
  • Be able to assess the utility of various design techniques for efficient processing for deep learning
  • Understand and evaluate recent implementation trends and opportunities in deep learning systems

Who Should Attend: 

This course is designed for research scientists, engineers, developers, project managers, startups and investors/venture capitalists who work with or develop artificial intelligence for hardware and systems, as well as mobile or embedded applications:

  • For project managers and investors/venture capitalists whose work involves assessing the viability or potential impact of a deep learning system and selecting a research direction or acquisition, this course aims to provide an overview of the recent trends as well as methods to assess the technical benefits and drawbacks of each approach or solution based on a comprehensive set of metrics.
  • For research scientists and engineers whose work involves designing and building deep learning systems, this course aims to provide an overview of the various state-of-the-art techniques that are being used to address the challenges of building efficient deep learning systems.
  • For startups and developers whose work involves developing deep learning algorithms and solutions for embedded applications and systems, this course aims to provide the insights necessary to select the best platform for your goals and needs. It will also highlight techniques that can be applied at the algorithm level to improve the energy-efficiency and speed of your proposed solution.

Course Schedule: 

Day Schedule
Day 1:  9:30am – 5:30pm

AM:

  • Introduction to Deep Learning
  • Deep Learning Applications
  • Development Resources for Deep Learning

PM:

  • Deep Learning on Programmable Platforms
  • Deep Learning on Specialized Hardware
Day 2: 9:30am – 5:30pm

AM:

  • Co-optimization of Algorithms and Hardware for Deep Learning
  • Application of Advanced Technologies to Deep Learning Systems

PM:

  • Training with Deep Learning
  • Metrics for evaluating Deep Learning Systems
  • Discussion on trends in Deep Learning 
  • Reception (4:00pm – 5:30pm)

Instructors: 

Vivienne Sze

Vivienne Sze, Associate Professor of Electrical Engineering and Computer Science, Electrical Engineering and Computer Science (EECS)
Vivienne Sze

Vivienne Sze is an Associate Professor at MIT in the Electrical Engineering and Computer Science Department. Her research interests include energy-aware signal processing algorithms, and low-power circuit and system design for deep learning, computer vision, autonomous navigation and image/video processing. Prior to joining MIT, she was a Member of Technical Staff in the R&D Center at TI, where she developed algorithms and hardware for the latest video coding standard H.265/HEVC.  She is a co-editor of the book entitled High Efficiency Video Coding (HEVC): Algorithms and Architectures (Springer, 2014). 

Prof. Sze received the B.A.Sc.degree from the University of Toronto in 2004, and the S.M. and Ph.D. degree from MIT in 2006 and 2010, respectively. In 2011, she was awarded the Jin-Au Kong Outstanding Doctoral Thesis Prize in electrical engineering at MIT for her thesis on “Parallel Algorithms and Architectures for Low Power Video Decoding." She is a recipient of the 2017 Qualcomm Faculty Award, the 2016 Google Faculty Research Award, the 2016 AFOSR Young Investigator Research Award, the 2016 3M Non-tenured Faculty Award, the 2014 DARPA Young Faculty Award, the 2007 DAC/ISSCC Student Design Contest Award and a co-recipient of the 2016 MICRO Top Picks and the 2008 A-SSCC Outstanding Design Award.

For more information about research in the Energy-Efficient Multimedia Systems Group at MIT visit: http://www.rle.mit.edu/eems/

Links & Resources: 

News/Articles:

Location: 

This course is held at:

Samsung Research America
665 Clyde Avenue
Mountain View, CA 94043

Hotel and campus information (pdf)

Content: 

Fundamentals: Core concepts, understandings, and tools (40%) 40
Latest Developments: Recent advances and future trends (30%) 30
Industry Applications: Linking theory and real-world (30%) 30

Delivery Methods: 

Lecture: Delivery of material in a lecture format (70%) 70
Discussions or Group Work: Participatory learning (20%) 20
Labs: Demonstrations, experiments, simulations (10%) 10

Levels: 

Introductory: Appropriate for a general audience (50%) 50
Specialized: Assumes experience in practice area or field (30%) 30
Advanced: In-depth exploration at the graduate level (20%) 20