Model-Based Reinforcement Learning for Scheduling Optimization in Semiconductor manufacturing (FAB) Environment

Semiconductor fabrication (FAB) facilities are among the most complex and capital-intensive production environments, characterized by complex process flows, stochastic processing times, strict quality constraints and costly equipment utilisation. Efficient scheduling of jobs is key for FAB performance, directly impacting throughput, queue times and cycle time. Traditional approaches, such as dispatching rules and linear solvers, struggle to adapt to the dynamic behaviour such as machine breakdowns or sudden product mix changes.  

In the context of the EU Chips Act, which aims to strengthen Europe’s semiconductor ecosystem and ensure technological sovereignty through resilient, efficient, and cutting-edge manufacturing, optimizing FAB operations has become even more critical. Enhancing scheduling efficiency directly contributes to the competitiveness, productivity, and sustainability of European semiconductor facilities, aligning with the strategic goals of securing supply chains and boosting innovation capacity.

Reinforcement Learning (RL) has emerged as a powerful tool for learning adaptive scheduling policies by interacting with an environment. Preliminary results highlight the promising potential of RL-based scheduling in the high-mix, complex FAB environment of imec with a Discreet Event Simulator (DES). The RL algorithm determines the schedule for each tool in the FAB. However, model-free RL methods require vast amounts of data and long training horizons, making them impractical in real FAB systems. Model-based RL offers a more sample-efficient and generalisable alternative by integrating predictive models of environment dynamics into the learning process. Despite its promise, the application of model-based RL in large-scale, stochastic manufacturing such as FABS remains underexplored.

The research objectives of this PhD topic are focused on designing and applying model-based RL methodologies for generating robust, adaptive and sample-efficient scheduling policies for a complex high-mix FAB environment. You will work together with an academic team of RL researcher to investigate state-of-the-art work on RL in process control, and the imec FAB operator team and developers to assist with the development of the DES simulator and apply your research on the actual imec FAB to optimise its operations. Potential research objectives include, but are not limited to:

• Identifying relevant KPI metrics to characterise the imec FAB environment at multiple levels (i.e., global performance down to single tool behaviour);
• Develop validated prediction models of FAB dynamics to serve as a foundation for the model-based RL;
• Investigating model-based RL methodologies (e.g., Dreamer, MuZero, etc) and designing advanced reward functions for them;
• Encoding FAB states using graph networks or transformers;
• Employ self-competition strategies to manage constrained rewards (e.g., maximizing throughput without deadline violations);
• Validation and benchmarking of the proposed methodologies against current FAB operations and propose improvements;
• Investigate the robustness of learned policies under uncertainty (e.g., machine failures, product mix variations, risk management on supply chain or disasters etc.);
• Explore transferability of policies onto different FAB setups.