Efficient Sequential Metamodel-based Approach for Stochastic Simulation Level-set Estimation

A Sequential Metamodel-based Approach for Stochastic Simulation Level-set Estimation

We investigate sequential metamodel-based level-set estimation under heteroscedasticity. Two methods are proposed: predictive variance reduction (PVR) and expected classification improvement (ECI). The former focuses on predictive uncertainty reduction, whereas the latter exploits classification improvement. The connection between these two methods is also revealed. Besides, a budget allocation feature is incorporated into the proposed methods to tackle heteroscedasticity’s impact better. Numerical studies demonstrate the proposed methods’ superior performance compared to state-of-the-art benchmarking approaches.

Efficient Expected Classification Improvement for Sequential Level-Set Estimation

The numerical study conducted for sequential metamodel-based LSE reveals two notable issues regarding the scalability of the metamodel-based LSE approaches: 1) the absence of an integrated budget allocation scheme and 2) the increasingly low computational efficiency as the number of design points grows. The expected classification improvement (ECI) method is employed to demonstrate potential solutions. Regarding the first issue, a principled budget allocation scheme that effectively balances the speed of budget allocation and the LSE accuracy is proposed. Besides, this new budget allocation scheme can estimate the total number of design points accumulated during the sequential sampling process. The Nystrom approximation technique is incorporated to reduce the time complexity. Three numerical examples are used to demonstrate the efficacy of the proposed techniques.

Empirical Uniform Bounds for Heteroscedastic Metamodeling

We propose pointwise variance estimation-based and metamodel-based empirical uniform bounds for heteroscedastic metamodeling based on the state-of-the-art nominal uniform bound available from the literature by considering the impact of noise variance estimation. Numerical results show that the existing nominal uniform bound requires a relatively large number of design points and a high number of replications to achieve a prescribed target coverage level. On the other hand, the metamodel-based empirical bound outperforms the nominal bound and other competing bounds in terms of empirical simultaneous coverage probability and bound width, especially when the simulation budget is small. However, the pointwise variance estimation-based empirical bound is relatively conservative due to its larger width. When the budget is sufficiently large so that the impact of heteroscedasticity is low, both empirical bounds’ performance approaches that of the nominal bound.