Embedded Files
Third-party agent skills extend LLM-based agents with instruction files and executable code that run on users' machines. Skills execute with user privileges and are distributed through community registries with minimal vetting, but no ground-truth dataset exists to characterize the resulting threats. Weconstruct the first labeled dataset of malicious agent skills by behaviorally verifying 98,380 skills from two community registries, confirming 157 malicious skills with 632 vulnerabilities. These attacks are not incidental. Malicious skills average 4.03 vulnerabilities across a median of 3 kill chain phases, and the ecosystem has split into two archetypes: Data Thieves that exfiltrate credentials through supply chain techniques, and Agent Hijackers that subvert agent decision-making through instruction manipulation. A single actor accounts for 54.1% of confirmed cases through templated brand impersonation. Shadow features, capabilities absent from public documentation, appear in 0% of basic attacks but 100% of advanced ones; several skills go further by exploiting the AI platform's own hook system and permission flags. Responsible disclosure led to 93.6% removal within 30 days. We release the dataset and analysis pipeline to support future work on agent skill security.
Overview of our study
Overview of our study
Labeled benchmark dataset
Labeled benchmark dataset
We construct and release the first three-tiered labeled dataset of agent skill security: 98,380 skills (ecosystem snapshot), 4,287 suspicious candidates, and 157 behaviorally-confirmed malicious skills with 632 labeled vulnerabilities and shadow feature annotations. Behavioral verification achieves 99.6\% precision, a 150$\times$ improvement over static-only approaches, providing reliable ground truth for evaluating future detection systems.
Multi-dimensional threat characterization
Multi-dimensional threat characterization
We conduct the first systematic measurement of the malicious agent skill ecosystem, organized around three research questions that map the threat landscape (RQ1), identify two attack archetypes with divergent coordination strategies (RQ2), and characterize evasion mechanisms including platform-native attack vectors (RQ3).
Ecosystem impact validation
Ecosystem impact validation
Responsible disclosure led to 93.6% removal within 30 days, independently validating the dataset's accuracy and its utility as actionable threat intelligence.
A real-world malicious agent skill
A real-world malicious agent skill
Agent skill ecosystem growth
Agent skill ecosystem growth
Page updated
Google Sites
Report abuse