A novel double Q-learning with invalid action masking for semiconductor ion implantation scheduling problem

As semiconductor fabrication technologies advance and critical dimensions (CD) shrink, process stages become increasingly complex, placing more stringent demands on scheduling throughput optimization and resource allocation. Simultaneously, environmental sustainability has emerged as an industry imperative: manufacturers must boost productivity, conserve energy, and control carbon emissions. This study focuses on the practical the semiconductor ion implantation stage scheduling problem (IISP), accounting for machine setup times, availability, and job priorities under multiple constraints, and proposes an intelligent scheduling framework based on reinforcement learning.

This study integrates double Q-learning with invalid action masking (DQIAM) method, which employs two Q-value estimators alongside an action-masking mechanism to filter out infeasible choices. The composite reward function minimizes the makespan, setup count, and scrap quantity, improving scheduling performance and production flexibility. To validate the performance of the proposed DQIAM approach, this study compares DQIAM with the shortest processing time (SPT) heuristic, genetic algorithm (GA), particle swarm optimization (PSO), standard Q-learning (QRL), and invalid action masking QRL (IQRL). The results show that DQIAM outperforms all benchmark methods on key performance indicators, achieving reduced completion times, fewer setup operations, and lower scrap quantities while providing stable and adaptive scheduling decisions.