Model-free Factor Screening and Sensitivity Analysis

In this work, we study the methodological underpinnings of the Morris elementary effects method, a model-free factor-screening technique originally proposed for deterministic simulation experiments, and develop an efficient Morris method–based framework (EMM) for simulation factor screening. Equipped with an efficient cluster-sampling procedure, EMM can simultaneously screen the main and interaction (or nonlinear) effects of all factors and control the overall false discovery rate at a prescribed level. Despite focusing on deterministic simulation experiments, we reveal the connections between EMM (also the Morris method) and other factor-screening methods, such as sequential bifurcation, and examine the resulting implications in the stochastic simulation setting under some commonly stipulated assumptions in design of experiments. Numerical experiments are presented to demonstrate the efficiency and efficacy of EMM.

The Morris elementary effects method (MM) is a widely recognized model-free approach for factor screening and sensitivity analysis, extensively used across various industrial domains. However, traditional MMs demand significant computational effort due to their one-factor-at-a-time design. In this paper, we introduce a novel Sequential-Group Morris Method (SGMM) that operates in a multi-stage framework, applicable to both deterministic and stochastic simulation settings. The core innovation of SGMM lies in leveraging the directional information of factors to form a newly introduced group elementary effect. This allows for the elimination of groups exhibiting no significant main or interaction effects in subsequent stages. At each stage, SGMM employs a tailored, distribution-free sequential probability ratio test to assess the importance of grouped elementary effects, ensuring desired statistical performance with prescribed Type-I and Type-II familywise error rates. Numerical experiments demonstrate the superior computational efficiency and statistical efficacy of SGMM compared to existing factor screening methods.

The Morris method (MM) is a well known model-free factor screening approach that is considered particularly effective when the number of factors is large or when the computer model is computationally expensive to run. In this paper, we propose the controlled Morris method (CMM) for simulation-based factor screening, which integrates a distribution-free sequential multiple testing procedure with MM to control the Type I and Type II familywise error rates at the prescribed levels while achieving a high computational efficiency. Numerical experiments are provided to demonstrate the efficacy and efficiency of CMM.

In this paper we provide a thorough investigation of the cluster sampling scheme for Morris' elementary effects method (MM), a popular model-free factor screening method originated in the setting of design and analysis of computational experiments. We first study the sampling mechanism underpinning the two sampling schemes of MM (i.e., cluster sampling and noncluster sampling) and unveil its nature as a two-level nested sampling process. This in-depth understanding sets up a foundation for tackling two important aspects of cluster sampling: budget allocation and sampling plan. On the one hand, we study the budget allocation problem for cluster sampling under the analysis of variance framework and derive optimal budget allocations for efficient estimation of the importance measures. On the other hand, we devise an efficient cluster sampling algorithm with two variants to achieve enhanced statistical properties. The numerical evaluations demonstrate the superiority of the proposed cluster sampling algorithm and the budget allocations derived (when used both separately and in conjunction) to existing cluster and noncluster sampling schemes.

Residential load forecasting has been playing an increasingly important role in operation and planning of power systems. Over the recent years, accurate forecasts of individual loads have become ever more challenging due to the proliferation of distributed energy resources. This paper identifies and verifies the opportunity of improving load forecasting performance by incorporating suitable input modeling and uncertainty quantification, and proposes a two-stage approach that enjoys the following features. (1) It provides input modeling and quantifies the impact of input errors, rather than neglect or mitigate the impact—a prevalent practice of existing methods. (2) It propagates the impact of input errors into the ultimate point and interval predictions for the target customer’s load for improved predictive performance. (3) A variance-based global sensitivity analysis method is further proposed for input-space dimensionality reduction in both stages to enhance the computational efficiency. Numerical experiments show that the proposed two-stage approach outperforms competing load forecasting methods with respect to both point predictive accuracy and coverage ability of the predictive intervals achieved.

Quality and safety are the core concerns of manufacturers of any product, especially automobile manufacturers. Defect complaints have emerged as the primary reason for vehicle recalls. However, with the emergence of platform economics and social media, complaints have sharply increased, presenting significant challenges for government agencies and associated companies in managing them efficiently. This work fills the gap by developing a powerful analytics approach to examine and configure an efficient and effective complaint-recall process. The proposed method comprises two essential components: agent-based modeling and multi-response simulation factor screening. The agent-based model provides an experimental testbed for studying complex behaviors and dynamics of a complaint-recall process, built on an in-depth understanding of micro-level processes embedded and a comprehensive collection of real-world data. A multi-response Morris' elementary effects method is devised to unravel which operation factors affect the complaint-recall process performance most. The numerical study reveals that ever-growing defect complaints can significantly deteriorate the system performance in terms of delays if the increasing trend persists over time. The proposed approach can help stakeholders better comprehend system dynamics, optimize operating conditions, and achieve satisfactory recall system performance without incurring exorbitant costs.