Machine Learning for Large Scale Optimization

Traditional approaches to solve practical optimization problems often rely on general purpose optimization algorithms combined with specialized heuristic components. While these solvers have been highly successful for many types of optimization problems, defining the heuristic components is a complicated process that requires expert knowledge. Moreover, the heuristics used by these solvers need to apply to a general class of problems and cannot consider any patterns that occur in a specific set of problem instances. In many practical settings, however, one needs to repeatedly solve a set of closely related instances, e.g., in job scheduling the demand might change, but production infrastructure will not. Traditional solvers fail to exploit such patterns.  

Recent research has proposed replacing heuristics with data-driven learning-based approaches. By using machine learning models to create learning solvers, the solvers can exploit problem specific distributions from historical data. This allows the solvers to effectively learn tailored heuristics for a common subset of problem instances. Several successful machine learning approaches have been proposed for combinatorial optimization problems such as the traveling salesperson problem, mixed integer linear programming, computer chip layout design, etc... These approaches confirm that machine learning models can be trained to generate solutions to hard optimization problems. 

In this project you will investigate the combination of traditional optimization and search methods with data driven approaches. The goal is to design novel hybrid optimization methods that can exploit historical patterns and scale to exceptionally large optimization problems, with a special focus on methods that are applicable to optimization problems in engineering and chip design. 

Team 

This project is an initiative of the Compute Systems Architecture Unit (CSA). CSA researches emerging workloads and their performance on large-scale supercomputer architectures for next-generation Artificial Intelligence (AI) and high-performance computing (HPC) applications. The team is responsible for algorithm research, runtime management innovations, performance modeling, architecture simulation and prototyping for these future applications and the future systems to execute them, to reach multiple orders of magnitude better performance, energy-efficiency, and total-cost-of-ownership.