Master the data and tools you need—from numerical linear algebra to convex programming—to make smarter decisions and drive enhanced results. With the guidance of MIT experts, you’ll gain a greater understanding of how to apply cutting-edge digital strategies to practical vision, learning, and graphics challenges. You’ll then apply your newfound skills by taking part in hands-on coding and mathematical exercises designed to strengthen your knowledge.
THIS COURSE MAY BE TAKEN INDIVIDUALLY OR AS part of THE PROFESSIONAL CERTIFICATE PROGRAM IN MACHINE LEARNING & ARTIFICIAL INTELLIGENCE.
Optimization algorithms lie at the heart of machine learning (ML) and artificial intelligence (AI). The distinctive feature of optimization within ML is the strong departure from textbook approaches: the focus is now on a different set of goals driven by big data, non-convex deep learning, and high-dimensions. This departure and the different focus make it challenging for newcomers and even experienced users to obtain a solid grasp of the fundamental ideas without getting lost in myriad tutorials, blogs, and papers.
This course provides an accessible entry point to Modeling and Optimization for Machine Learning, key skills needed to use state-of-the-art software and algorithms from machine learning. It covers underlying theoretical motivations behind widely-used optimization algorithms (the “science”), while diving deep into aspects of mathematical modeling (the “art”) to provide students with an intuitive, foundational introduction to this modern and fast-moving research area.
Modeling reduces messy engineering or computational problems to mathematical forms that can be solved by using standard software and techniques. By recognizing mathematical patterns “in the wild,” participants will develop an intuition for which problems are solvable using standard numerical modeling techniques and gain the knowledge and skills to then solve them in practice.
After we develop an appropriate model for a machine learning problem, the next step is to choose an optimization technique. Participants in the course will learn to pair mathematical models with efficient optimization algorithms, from stochastic gradient descent to cone programming. Participants will delve into the details of how popular optimization methods work and will receive practical experience interfacing with optimization software through case studies and exercises.
By the end of the course, participants will learn how to boil real-world challenges down to their computational essence to make a reasonable estimate of how difficult it would be to design a numerical method to solve them. We will cover a breadth of tools, from numerical linear algebra to convex programming and stochastic/deterministic gradient descent, in the context of practical problems drawn from emerging applications in learning, computer vision, time series analysis, and imaging. Coding and mathematical exercises will reinforce these ideas and expose participants to standard software packages for optimization.
Participants in the course will learn how to:
- Recognize classes of optimization problems in machine learning and related disciplines.
- Learn concepts that demystify the “why” and “how” of ubiquitous topics such as regression, deep learning, and large-scale optimization, with a focus on convex and non-convex models.
- Interface with software for computing optimal solutions to a given machine learning problem.
- Understand the mathematical underpinnings of optimization methods via examples drawn from machine learning, computer vision, engineering, and data analysis.
- Understand foundational optimization ideas including gradient descent, stochastic gradient methods, higher-order methods, and more advanced optimization algorithms.
- Classify optimization problems by their tractability, difficulty, and compatibility with existing software.
- Learn to cut through the hype to make more informed choices for their own applications.
Who Should Attend
This course is designed for people working in data science, finance, marketing, computer-aided design, operations, strategy, engineering, research, or computer vision. Typical roles include engineer, programmer, developer, data scientist, researcher, consultant, or marketing analyst.
Participants are required to have a background in linear algebra and multivariable calculus, as well as at least basic programming in Python.
Laptops (or tablets) with Python are required for this course. Participants should have administrative privileges for their computers in case Python packages need to be installed during the course.
The course begins with the fundamentals of modeling and optimization, including case studies converting regression and classification problems to mathematical models as well as the basics of deterministic and stochastic gradient descent. We then broaden the capabilities of our modeling language by showing how to incorporate constraints and accelerate optimization with second-order information. After establishing the basics, we consider a variety of more advanced models in machine learning, including neural network training, sparsity and low-rank regularization, metric learning, time-series analysis, and adversarial training of robust models. We conclude with practical discussion drawn from research projects at MIT as well as from participants’ domain areas.
Our course will include daily practicum exercises in which students will experiment with optimization tools applied to modeling problems drawn from machine learning applications. We will complete the exercises using Google Colab in Python.
Links & Resources
- Improving global health equity by helping clinics do more with less, MIT News, June 25, 2020
- Smoothing out sketches’ rough edges: MIT-developed tool improves automated image vectorization, saving digital artists time and effort, MIT News, September 11, 2018
The type of content you will learn in this course, whether it's a foundational understanding of the subject, the hottest trends and developments in the field, or suggested practical applications for industry.
How the course is taught, from traditional classroom lectures and riveting discussions to group projects to engaging and interactive simulations and exercises with your peers.
What level of expertise and familiarity the material in this course assumes you have. The greater the amount of introductory material taught in the course, the less you will need to be familiar with when you attend.