Abstract
In the production scheduling for chemical industries, such as copper foil manufacturing, the inventory level directly affects the production cost. The Stochastic Cutting Stock Problem (SCSP) can be considered as the complex inventory level in scheduling problem due to the presence of random event factors. With traditional deterministic mathematical methods such as integer programming, the speed of the solution grows exponentially with the complexity of the process. Reinforcement Learning (RL) is a novel method to immediately provide the best scheduling solution. However, RL is doubtful if it deals with process constraints. Pitombeira-Neto and Murta (2022) provided a model-free off-policy approximate policy iteration algorithm to show the RL scheduling performance and ensure that action does not violate the constraints. Yet, the excessive mathematizations and exhaustive random iterations led to huge training time, making industrial applicability low. Hence, the purpose of the study was to adopt Advantage Actor-Critic (A2C) for continuous action space to solve a classic SCSP for more reproducible and easier implementations in industries. Also, we proposed a two-stage discount factor (?) algorithm for training a model that could make a well trade-off between two goals: avoiding violate the constraints and providing the lowest cost. The results showed that the training of the A2C method was much faster than the literature method with a sufficiently nice solution. A surprising result is that the A2C agent can continuously provide action that met the constraints for over ten thousands interactions that showed a wide range of potential applications in similar scenarios, i.e., problems with some constraints.