Model-free Factor Screening and Sensitivity Analysis

An Efficient Morris Method-based Framework for Simulation Factor Screening

In this work, we study the methodological underpinnings of the Morris elementary effects method, a model-free factor-screening technique originally proposed for deterministic simulation experiments, and develop an efficient Morris method–based framework (EMM) for simulation factor screening. Equipped with an efficient cluster-sampling procedure, EMM can simultaneously screen the main and interaction (or nonlinear) effects of all factors and control the overall false discovery rate at a prescribed level. Despite focusing on deterministic simulation experiments, we reveal the connections between EMM (also the Morris method) and other factor-screening methods, such as sequential bifurcation, and examine the resulting implications in the stochastic simulation setting under some commonly stipulated assumptions in design of experiments. Numerical experiments are presented to demonstrate the efficiency and efficacy of EMM.

Efficient Budget Allocation Strategies for Elementary Effects Method in Stochastic Simulation

This work focuses on extending the Morris' elementary effects method (MM) for sensitivity analysis/factor screening originated in the context of deterministic computer experiments to the stochastic simulation setting. Given a fixed simulation budget to expend, the main objective is to provide efficient and accurate estimates of main and interaction (or nonlinear) effects coined by the standard MM for characterizing the importance of each factor, despite the impact of simulation errors. Taking into account both the factor/input sampling uncertainty rooted in MM and the random errors inherent in a stochastic simulation, we develop efficient budget allocation strategies for implementing MM in this new context. Under each strategy proposed, we derive its corresponding optimal budget partition and optimal budget allocation rules. Numerical results corroborate the practical effectiveness of the proposed budget allocation strategies.

Controlled Morris Method: A New Factor Screening Approach Empowered by a Distribution-free Sequential Multiple Testing Procedure

The Morris method (MM) is a well known model-free factor screening approach that is considered particularly effective when the number of factors is large or when the computer model is computationally expensive to run. In this paper, we propose the controlled Morris method (CMM) for simulation-based factor screening, which integrates a distribution-free sequential multiple testing procedure with MM to control the Type I and Type II familywise error rates at the prescribed levels while achieving a high computational efficiency. Numerical experiments are provided to demonstrate the efficacy and efficiency of CMM.

Cluster Sampling for Morris Method Made Easy

In this paper we provide a thorough investigation of the cluster sampling scheme for Morris' elementary effects method (MM), a popular model-free factor screening method originated in the setting of design and analysis of computational experiments. We first study the sampling mechanism underpinning the two sampling schemes of MM (i.e., cluster sampling and noncluster sampling) and unveil its nature as a two-level nested sampling process. This in-depth understanding sets up a foundation for tackling two important aspects of cluster sampling: budget allocation and sampling plan. On the one hand, we study the budget allocation problem for cluster sampling under the analysis of variance framework and derive optimal budget allocations for efficient estimation of the importance measures. On the other hand, we devise an efficient cluster sampling algorithm with two variants to achieve enhanced statistical properties. The numerical evaluations demonstrate the superiority of the proposed cluster sampling algorithm and the budget allocations derived (when used both separately and in conjunction) to existing cluster and noncluster sampling schemes.


Input Modeling and Uncertainty Quantification for Improving Volatile Residential Load Forecasting

Residential load forecasting has been playing an increasingly important role in operation and planning of power systems. Over the recent years, accurate forecasts of individual loads have become ever more challenging due to the proliferation of distributed energy resources. This paper identifies and verifies the opportunity of improving load forecasting performance by incorporating suitable input modeling and uncertainty quantification, and proposes a two-stage approach that enjoys the following features. (1) It provides input modeling and quantifies the impact of input errors, rather than neglect or mitigate the impact—a prevalent practice of existing methods. (2) It propagates the impact of input errors into the ultimate point and interval predictions for the target customer’s load for improved predictive performance. (3) A variance-based global sensitivity analysis method is further proposed for input-space dimensionality reduction in both stages to enhance the computational efficiency. Numerical experiments show that the proposed two-stage approach outperforms competing load forecasting methods with respect to both point predictive accuracy and coverage ability of the predictive intervals achieved.

Assessing and Mitigating the Impacts of Defect Complaints on Vehicle Recall Delays: An Integrated Agent-based Simulation and Factor Screening Approach

Quality and safety are the core concerns of manufacturers of any product, especially automobile manufacturers. Defect complaints have emerged as the primary reason for vehicle recalls. However, with the emergence of platform economics and social media, complaints have sharply increased, presenting significant challenges for government agencies and associated companies in managing them efficiently. This work fills the gap by developing a powerful analytics approach to examine and configure an efficient and effective complaint-recall process. The proposed method comprises two essential components: agent-based modeling and multi-response simulation factor screening. The agent-based model provides an experimental testbed for studying complex behaviors and dynamics of a complaint-recall process, built on an in-depth understanding of micro-level processes embedded and a comprehensive collection of real-world data. A multi-response Morris' elementary effects method is devised to unravel which operation factors affect the complaint-recall process performance most. The numerical study reveals that ever-growing defect complaints can significantly deteriorate the system performance in terms of delays if the increasing trend persists over time. The proposed approach can help stakeholders better comprehend system dynamics, optimize operating conditions, and achieve satisfactory recall system performance without incurring exorbitant costs.