Validation Criteria of User Study

We deemed hastily completed or incomplete questionnaires as low-quality or invalid responses. Quality validation was based on answers, duration, and completion of questionnaires. 

38.9% of the invalid responses were identified via the selection of irrelevant choices in topic questions, indicating hasty completion. We adopted a quality control method [1] by including an apparent irrelevant choice for each multi-choice topic question, treating any selection of this choice as invalid.

Responses completed within two minutes (27.8%) or unfinished (33.3%) were also considered invalid.

[1] 4 Types of Inattentive Survey Participant & How to Handle Them. https://www.cloudresearch.com/resources/guides/ultimate-guide-to-survey-data-quality/how-to-identify-handle-invalid-survey-responses/, 2023.