Comparing Random Forest and Gaussian Process Modeling in the Gp-demo Algorithm
Miha Mlakar, Tea Tušar, Bogdan Filipic Department of Intelligent Systems, Jozef Stefan Institute and Jozef Stefan International Postgraduate School & Jamova cesta 39, SI-1000 Ljubljana, Slovenia
Abstract: In surrogate-model-based optimization, the selection of an appropriate surrogate model is very important. If solution approximations returned by a surrogate model are accurate and with narrow confidence intervals, an algorithm using this surrogate model needs less exact solution evaluations to obtain results comparable to an algorithm without
surrogate models. In this paper we compare two well known modeling techniques, random forest (RF) and Gaussian process (GP) modeling. The comparison includes the approximation accuracy and confidence in the approximations (expressed as the confidence interval width). The results show that GP outperforms RF and that it is more suitable for use in a surrogatemodel- based multiobjective evolutionary algorithm.
Keywords: Random Forest Modeling, Gaussian Process Modeling Comparing Random Forest and Gaussian Process Modeling in the Gp-demo Algorithm
References: [1] Breiman, L. (2001). Random forests. Machine Learning, 45 (1) 5–32.
[2] Deb, K. (2001). Multi-Objective Optimization Using Evolutionary Algorithms. Wiley, New York.
[3] Deb, K., Pratap, A., Agarwal, S., Meyarivan, T. (2002). A fast and elitist multiobjective genetic algorithm: NSGAII. IEEE
Transactions on Evolutionary Computation, 6 (2) 182–197.
[4] Mlakar, M., Petelin, D., Tušar, T., Filipic, B. (2014). GPDEMO: Differential evolution for multiobjective optimization based on
Gaussian process models. European Journal of Operational Research, 2014, doi: 10.1016/j.ejor.2014.04.011.
[5] Mlakar, M., Tušar, T., Filipic, B. (2014). Comparing solutions under uncertainty in multiobjective optimization. Mathematical
Problems in Engineering, 2014, doi: 10.1155/2014/817964.
[6] Poloni, C., Giurgevich, A., Onesti, L., Pediroda, V. (2000). Hybridization of a multi-objective genetic algorithm, a neural
network and a classical optimizer for a complex design problem in fluid dynamics. Computer Methods in Applied Mechanics and
Engineering, 186 (2) 403–420.
[7] Rasmussen, C. E., Williams, C. (2006). Gaussian Processes for Machine Learning. MIT Press, Cambridge, MA.
