Abstract
Stochastic optimization has wide applications in many research areas. Several solution methods have been proposed, but there is no one of them can solve all stochastic optimization problems. Furthermore, few large-scale comparative studies for solution methods have been published. Basing on this end, this dissertation is conducted in two directions: One is to propose a quasi-Newton method for stochastic optimization problems; the other is to compare the proposed quasi-Newton method and other existing methods on a personal computer.The proposed quasi-Newton method is based on the conventional BFGS quasi-Newton method used in deterministic optimization. The basic idea is to use the estimated gradient calculated from different replications and a metric matrix updated from the BFGS formula to yield a quasi-Newton search direction. To avoid misjudging the minimal point in both the line search and the quasi-Newton iterations due to the stochastic nature, a t-test instead of a simple comparison of the mean responses is performed. Most important, the method is proved to be able to converge to the optimal solution under some mild conditions.Empirical work is conducted in two parts: simulation response optimization and stochastic programming. For simulation response optimization, firstly, a stochastic function is adopted for testing the power of five methods, including BFGS quasi-Newton method (BFGS), conjugate direction method of Powell (PW), conjugate gradient method of Fletcher-Reeves (CG), alternating variable method of Hooke-Jeeves (HJ), and RS+S9 method of Barton and Ivey. All of the first four methods outperform RS+S9, especially BFGS and PW are the best methods. For solving stochastic systems such as queueing problems and (s,S) inventory problems, BFGS, PW, CG, and HJ are better than the stochastic approximation method of Andradottir and the hybrid method of Fu and Healy, respectively. The BFGS, especially, is the most effective and efficient method, and it is very robust. The performance of PW for inventory problems is as good as that of BFGS. For stochastic programming, tolerance allocation problems are adopted for comparing the proposed sequential quadratic programming (SQP) developed from the quasi-Newton method, genetic algorithm of Lee and Johnson, and TOL-M method of Lee and Woo. The SQP turns out to be the best. The results obtained from SQP are not only with fewer costs and with higher yield rates, but also more robust.For stochastic optimization problems, there are no overwhelming solution method derived for the time being. According to the results of solving commonly used test problems in related studies, the proposed quasi-Newton method in this study outperforms the existing methods, and it possesses convergence property theoretically. This indicates that the quasi-Newton method is an effective and robust method for solving stochastic optimization problems.