Modeling and Optimization for Machine Learning

Numerical modeling is the skill of reducing a messy engineering or computational problem to a mathematical form that can be solved by using standard algorithms and techniques. By recognizing mathematical patterns “in the wild,” participants will develop an intuition for which problems are solvable using standard numerical modeling techniques and gain the knowledge and skills to then solve them.

Computer science is experiencing a fundamental shift in its approach to modeling and problem solving. Early computer scientists primarily studied discrete mathematics, focusing on structures like databases and arrays composed of distinct pieces. With the introduction of modern applications in “big data,” three-dimensional scanning, machine learning, and noisy sensor communications, practitioners now must design robust methods for processing real-valued data. The latest generation of programmers, computer scientists, and engineers must be able to reason about not just bits-and-bytes, but also calculus, linear algebra, and optimization.

By the end of the course, participants will learn how to boil real-world challenges down to their computational essence to make a reasonable estimate of how difficult it would be to design a numerical method to solve them. We will cover a breadth of possible tools, from numerical linear algebra to convex programming and stochastic/deterministic gradient descent, in the context of practical problems drawn from emerging applications in vision, learning, and graphics. Coding and mathematical exercises will reinforce these ideas and expose participants to standard software packages for optimization.


This course may be taken individually or as an elective for the Professional Certificate Program in Machine Learning and Artificial Intelligence.

Lead Instructor(s): 

Justin Solomon
Suvrit Sra


Jun 1, 2020 - Jun 5, 2020

Course Length: 

5 Days

Course Fee: 





  • Open

It is highly recommended that you apply for a course at least 6-8 weeks before the start date to guarantee there will be space available. After that date you may be placed on a waitlist. Courses with low enrollment may be cancelled up to 4 weeks before start date if sufficient enrollments are not met. If you are able to access the online application form, then registration for that particular course is still open.

Registration for this program will close by May 8

Participant Takeaways: 

Participants of this program will learn to:

  • Recognize linear, eigenvalue, convex optimization, and nonconvex optimization problems underlying engineering challenges.
  • Interface with software for automatically identifying the optimal solution to a numerical problem.
  • Understand the mathematical underpinnings of case studies in computer vision, machine learning, and graphics
  • Understand the basics of optimization techniques including gradient descent, line search, and the barrier method
  • Develop an intuition for classifying numerical problems by their tractability or difficulty

Who Should Attend: 

This course is designed for people working in the areas of data, finance, marketing, computer-aided design, operations, strategy, engineering, research, computer graphics, or computer vision. Typical roles include engineer, programmer, developer, data scientist, researcher, consultant, or marketing analyst.


Participants are required to have a background in linear algebra and multivariable calculus, as well as at least basic programming in Python.

Computer Requirements

Laptops (or tablets) with Python are required for this course. Participants should have administrative privileges for their computers in case Python packages need to be installed during the course.

Program Outline: 

The course will be organized around case studies drawn from application areas, organized by level of computational difficulty. We will begin with recognizing linear algebra problems “in the wild,” including case studies in statistical regression and camera transformations from computer vision. Then we will broaden to convex optimization (e.g. transportation logistics), nonconvex optimization (e.g. 3D surface deformation in computer graphics), and stochastic optimization (e.g. machine learning on large, messy datasets). Our focus is on reducing problems to their standard mathematical forms and identifying software tools or simple algorithms to solve them.

Course Schedule: 

View 2020 schedule (pdf, subject to change)

This course meets 10:00 am - 5:00 pm on Monday, 9:30 am - 5:00 pm Tuesday through Thursday, and 9:30 am - 4:30 pm on Friday.



This course takes place on the MIT campus in Cambridge, Massachusetts. We can also offer this course for groups of employees at your location. Please complete the Custom Programs request form for further details.


Fundamentals: Core concepts, understandings, and tools (30%) 30
Latest Developments: Recent advances and future trends (42%) 42
Industry Applications: Linking theory and real-world (28%) 28

Delivery Methods: 

Lecture: Delivery of material in a lecture format (50%) 50
Discussions or Group Work: Participatory learning (30%) 30
Labs: Demonstrations, experiments, simulations (20%) 20


Introductory: Appropriate for a general audience (75%) 75
Advanced: In-depth exploration at the graduate level (25%) 25