Our work aims to address three main challenges in the study of dark patterns: inconsistencies and incompleteness in classification, limitations of detection tools, and inadequacies in data comprehensiveness.
In this paper, we introduce a comprehensive framework, called the Dark Pattern Analysis Framework (DPAF). Utilizing this framework, we construct a comprehensive and standardized taxonomy of dark patterns, each type labeled with its impact on users and the likely scenarios in which it appears, validated through an industry survey.
Upon assessing the capabilities of detection tools and data completeness, we find that of all the dark patterns, the five mainstream detection tools can only identify 32, yielding a coverage rate of merely 50%. Even though the existing four datasets collectively contain 5,566 instances, they cover only 32 of all types of dark patterns, also resulting in a total coverage rate of 50%. The results discussed above suggest that there is still significant room for advancement in the field of dark mode recognition.
Through this research, we not only deepen our understanding of dark pattern classification and detection tools, but also offer valuable insights for future research and practice in this field.
The framwork for Dark Patterns Taxonomy Analysis
Answer to RQ1: Through a review of 76 scientific papers, we identified and generalized 64 unique types of dark patterns. We further organized and annotated these patterns according to their categories, descriptions, impacts on users, and common scenarios. This taxonomy was validated by industry participants, with more than 87% of responses deeming it rationality, completeness, understandability, and usefulness.
Answer to RQ2: From the five tools selected, they collectively identify 32 out of the 64 dark patterns from our taxonomy, achieving a 50% coverage. However, 32 dark patterns (50% of the total) remain undetected by all tools.
Answer to RQ3: The four datasets collectively contain 5,566 instances, and only cover 32 distinct dark patterns, yielding a total coverage rate of 50% (i.e., 32 out of 64 types). Furthermore, the non-inclusion of 32 dark patterns, constituting 50% of the total, limits the capability in recognizing them.