Measurement

Reliability of Experienced Sampling Method (RESM)

Who: ESM Researchers

What: 

• Evaluating and assessing Reliability Methods in ESM

• Review of reliability reporting in ESM + Monte Carlo simulation to evaluate different methods of computing reliability 

• Recommendations for choosing appropriate reliability methods for ESM studies and implications

• R Script tutorial for computing computing fit indices and estimates of within-person reliability

Where/When: Monte Carlo simulation testing three popular reliability methods (level-specific alpha, multilevel CFA, HLM) across varying conditions (number of items, measurement points, autocorrelation)

Why: Experience sampling methods have been identified as an integral piece to examination of dynamic relationships and situations in the workplace. ESM enables researchers to better understand temporal process-related questions based on sampling experiences as they naturally occur. We address some of the challenges researchers face when adapting ESM methodology in regards to the appropriate computation and reporting of within-person reliability, as this is critical to do correctly as it has implications on the validity of ESM research and for development of temporal-based theories.