MoMMMMr
Morris II Worm
Embedded Files
Background
Morris Worm, also known as the Great Worm / Internet Worm.
A hacker launched the Morris worm in 1988, and many people consider it one of the very first public attacks on computer systems.
Morris Worm was designed to spread across Unix-based systems by exploiting vulnerabilities, primarily a flaw in the sendmail program and weak passwords. Morris did not intend for the worm to cause damage, but due to a programming error, it ended up infecting computers multiple times and causing system slowdowns.
After his legal troubles, He co-founded the Y Combinator startup accelerator,
How was the Morris worm stopped?
Morris's exploits became generally obsolete due to decommissioning rsh- Remote Shell Protocol (normally disabled on untrusted networks), fixes to sendmail and finger, widespread network filtering, and improved awareness of weak passwords.
Morris II worm
Morris II, a prototype AI worm developed by a group of researchers.
Unlike the original Morris worm from 1988, this one doesn't infect computers directly. It is a zero-click malware.
According to the research papers [1], this first-generation AI worm can steal data, spread malware and spam users through AI-powered email assistants. However, it's important to note that this research was conducted in a controlled environment and the worm has not been deployed in the real world [2].
How does it affect?
Steals data: Morris II can steal sensitive information you provide to AI-powered email assistants, including things like credit card details.
Spreads through AI: This worm can spread itself through interactions between different AI models.
Spams emails: It can use infected AI models to send spam emails.
Researchers haven't released Morris II to the real world. They created it to highlight the security risks of AI and reported it to companies like Google and OpenAI.This helps these companies improve their security measures to prevent similar attacks in the future.
The purpose of creating Morris II wasn't to cause damage but to demonstrate how vulnerable AI systems can be. It's like a controlled experiment to show the potential dangers.
Morris II was a simulated attack, a test case, not an actual weapon unleashed. It served as a wake-up call for the AI industry to be more aware of potential security weaknesses.
Research Process explained
Input: Worm designed using “adversarial self-replicating prompts”
Targets: Gen AI ecosystems
Demo explains:
Demonstrated it against GenAI-powered email assistants in two use cases (spamming and exfiltrating personal data), under two settings (black-box and white-box accesses), using two types of input data (text and images) and against three different GenAI models (Gemini Pro, ChatGPT 4.0, and LLaVA)
Results:
Input prompt triggers the Gen AI model to output the input prompt so it will be replicated in the next inference as well. (replication)
Trigger the Gen AI to perform malicious activity - exfiltrate sensitive user data provided in the query. (payload)
These inputs compel the agent to deliver them to new agents by exploiting the connectivity within the GenAI ecosystem (propagate)
While a regular prompt is essentially code that triggers the GenAI model to output data, an adversarial self-replicating prompt is a code that triggers the GenAI model to output code [5].
Resembles classic cyber-attacks that exploited the idea of changing data into code in order to carry their attack (Ex: an SQL injection attack that embeds code inside a query, or a buffer overflow attack that is intended to write data into areas known to hold executable code).
https://whimsical.com/9Dibq7Fj1m11Ti8b7igCNv@FNpptVQ16BqeSRfDXgu5pubKLBqKw
This innovative development highlights the risks associated with connected, autonomous AI ecosystems as these systems become more advanced and are given more freedom. The Morris II worm represents a new kind of cyberattack that leverages the capabilities of generative AI to spread across systems, potentially compromising data security and integrity [4].
The worm exploits bad architecture design for the GenAI ecosystem and is not a vulnerability in the GenAI service. [5]
Countermeasures
For Replication:
Rephrasing the entire output- Reword the GenAI's response completely to avoid including any parts that directly match the user's input. This reduces the risk of the AI accidentally revealing sensitive information that was included in the question.
Deployment in two ways -
Inside the AI agent itself: The AI model would be modified to automatically rephrase its responses before delivering them.
On the GenAI server: The server that runs the AI model could handle the rephrasing before sending the response back to the user.
For Propagation:
by monitoring the interactions in the GenAI ecosystem
by monitoring the interactions of the agents in the GenAI ecosystem (3rd party services, such as SMTP servers, and messaging application services)
For the RAG-based worm, the easiest method to prevent the propagation of the worm is to use a non-active RAG, this security measure protects sensitive information but might make the AI a little outdated [ Info: Karthikeyan]
References:
[1] https://www.wired.com/story/here-come-the-ai-worms/
[3] https://www.youtube.com/watch?v=FL3qHH02Yd4&t=205s
[5] https://sites.google.com/view/compromptmized
Report abuse