Efficient Sequential Metamodel-based Approach for Stochastic Simulation Level-set Estimation
A Sequential Metamodel-based Approach for Stochastic Simulation Level-set Estimation
We investigate sequential metamodel-based level-set estimation under heteroscedasticity. Two methods are proposed: predictive variance reduction (PVR) and expected classification improvement (ECI). The former focuses on predictive uncertainty reduction, whereas the latter exploits classification improvement. The connection between these two methods is also revealed. Besides, a budget allocation feature is incorporated into the proposed methods to tackle heteroscedasticity’s impact better. Numerical studies demonstrate the proposed methods’ superior performance compared to state-of-the-art benchmarking approaches.
Efficient Expected Classification Improvement for Sequential Level-Set Estimation
The numerical study conducted for sequential metamodel-based LSE reveals two notable issues regarding the scalability of the metamodel-based LSE approaches: 1) the absence of an integrated budget allocation scheme and 2) the increasingly low computational efficiency as the number of design points grows. The expected classification improvement (ECI) method is employed to demonstrate potential solutions. Regarding the first issue, a principled budget allocation scheme that effectively balances the speed of budget allocation and the LSE accuracy is proposed. Besides, this new budget allocation scheme can estimate the total number of design points accumulated during the sequential sampling process. The Nystrom approximation technique is incorporated to reduce the time complexity. Three numerical examples are used to demonstrate the efficacy of the proposed techniques.
Zhang, Yutong and Chen, Xi. "A Sequential Metamodel-based Approach for Stochastic Simulation Level-set Estimation." Submitted.
Zhang, Yutong and Chen, Xi. "Efficient Expected Classification Improvement for Sequential Level-set Estimation." Manuscript in preparation.
Empirical Uniform Bounds for Heteroscedastic Metamodeling
We propose pointwise variance estimation-based and metamodel-based empirical uniform bounds for heteroscedastic metamodeling based on the state-of-the-art nominal uniform bound available from the literature by considering the impact of noise variance estimation. Numerical results show that the existing nominal uniform bound requires a relatively large number of design points and a high number of replications to achieve a prescribed target coverage level. On the other hand, the metamodel-based empirical bound outperforms the nominal bound and other competing bounds in terms of empirical simultaneous coverage probability and bound width, especially when the simulation budget is small. However, the pointwise variance estimation-based empirical bound is relatively conservative due to its larger width. When the budget is sufficiently large so that the impact of heteroscedasticity is low, both empirical bounds’ performance approaches that of the nominal bound.
Zhang, Yutong and Chen, Xi. "Empirical Uniform Bounds For Heteroscedastic Metamodeling." Proceedings of the 2022 Winter Simulation Conference, 1 - 12.