RQ2

RQ2: How do different modeling techniques and corresponding configurations impact the quality of the abstract model?

Building on RQ1’s findings, RQ2 delves into assessing the influence of various hyperparameters on the abstract models' efficacy, focusing on state abstraction and trace construction. We analyze factors such as PCA dimensions (Low, Medium, High), history steps, partition techniques (GMM, K-means, Grid), and modeling methods (DTMC, HMM), aiming to understand their impact on key metrics like Succinctness (SUC), Stationary Distribution Entropy (SDE), and others.

 To enable a relative comparison across settings, we normalize metric values based on rank, where a higher normalized value indicates a superior rank and, consequently, better performance in that metric. This approach ensures that our assessment emphasizes the metrics' relative performance, providing clearer insights into the factors that enhance the abstract models' quality.

PCA Dimension Statistics:

Presented in the table is a detailed breakdown of model-wise metrics in relation to PCA Dimension, spanning across three distinct datasets. The table meticulously outlines the maximum, minimum, and mean values for each metric, offering a comprehensive view of how different PCA dimensions influence the model’s behavior and characteristics, as seen through various performance measures.

Partition Method Statistics:

This table provides a thorough analysis of the impact of various partition methods on model-wise metrics across different datasets. It systematically enumerates the maximum, minimum, and mean values for each performance metric, facilitating a clear understanding of how distinct partitioning approaches influence the model’s characteristics. The comprehensive data presented in the table is instrumental for evaluating the effectiveness of each partition method, guiding users in making informed decisions for optimal model configuration.

In this table, we turn our attention to examining the relationship between different abstract model types and their corresponding model-wise metrics. By listing out the maximum, minimum, and mean values for a variety of metrics, the table provides a panoramic view of how each abstract model type performs across various datasets. This meticulous compilation of data aids in discerning the nuances of each model type, laying a solid foundation for users to identify the most suitable abstract model for their specific analysis needs.