What We’ll Build

Run a Colab notebook that demonstrates how a generative LLM can be used to create synthetic scam URLs and how a guard / moderation LLM can classify those URLs as Safe, Suspicious, or Likely Fraud.

The lab simulates an AI cybersecurity assistant with two roles:

Attack side (Generator):

Use Qwen 2.5 to generate fraudulent URLs across several categories (fake storefronts, crypto rug pulls, tech support scams, etc.).
All URLs are synthetic and use fake brand names, fake tokens, and non-existent domains.

Defense side (URL Guard):

Use a Qwen-based guard classifier to analyze structure, lexical patterns, and risk indicators, labeling the URL as:

• Safe

• Suspicious

• Likely Fraud

• High-Risk Fraud

You’ll interact with a small UI that lets you:

• Select a scam category and generate multiple synthetic URLs

• Paste any URL (generated or real) into a classifier box

• Click “Classify URL” and see the guard’s assessment

By the end, you’ll see how easily GenAI can scale scam infrastructure—and how LLM-based defenses attempt to detect it.

Environment & Libraries

Generator / Attack Model:

Qwen/Qwen2.5-1.5B-Instruct

• Generates synthetic scam URLs based on category templates

• Requires minimal prompting

• Notebook auto-selects GPU if available (optional but helpful)

Guard / Defense Model:

The same Qwen model is reused with a URL-classifier prompt.
It produces structured responses:

• "risk"

• "category"

• "rationale"

Python Runtime

• Google Colab (Python 3) — no local installation required

Core Libraries

• transformers, torch — model loading and inference

• ipywidgets — dropdown menus, text boxes, buttons

• json, re, random — formatting, parsing, sampling

• Built-in templates in case_4.py for URL generation

UI Elements

• Category Dropdown: Choose a scam type

• Count Slider: Number of URLs to generate

• Generate Button: Calls the generator and prints URLs

• Paste Box: Paste any URL (generated or real)

• Classify Button: Runs URL guard model

• Output Panels: Show generated URLs and guard results

Data

• URL generation templates for categories:

  • Fake Storefront

  • Crypto Rug Pull

  • Tech Support Scam

  • Fake Sweepstakes / Lottery

  • Investment Scam

  • Fake Charity

• You may paste any URL of your choice for classification

• Do NOT use real personal or sensitive URLs

Open the Notebook

Colab Link (M4): https://colab.research.google.com/drive/1X_7J2qo3fEa7DfTetserI5d1eLVjo3FV?usp=sharing

1. Open the notebook.

2. If prompted, go to Runtime → Change runtime type → GPU (optional).

3. Run all cells to load dependencies, models, templates, and widgets.

Section A — Scam URL Generator (No Defense)

What this section does

Runs the generator model with no safety filtering.
The model outputs URLs that resemble real scam infrastructure but use fake brands and invented domains.

UI in Colab

• Dropdown: Scam Category

• Slider: Number of URLs (1–10)

• Button: Generate

• Output: List of URLs (one per line)

Steps

1. Select a category such as FakeStorefront or CryptoRugPull.

2. Set the number of URLs to generate (e.g., 5).

3. Click Generate Scam URLs.

4. Examine the URLs produced.

Expected Observation

• URLs look plausible but malicious:

  • secure-update-paypa1-helpdesk.com

  • omega-token-swap-airdrop.vip

• Categories influence patterns:

  • Fake storefronts mimic e-commerce

  • Crypto scams use token and swap terminology

• No defensive layer exists yet—this is pure synthetic attack generation.

Section B — URL Guard on Generated URLs

What this section does

Adds a classifier that takes a URL and outputs structured labels such as:

• Risk: Safe / Suspicious / Likely Fraud / High-Risk Fraud

• Category: Crypto, Storefront, TechSupport, etc.

• Rationale: Brief explanation

UI in Colab

• Textarea: URL

• Button: Classify URL

• Output: JSON-ish verdict (risk, category, rationale)

Steps

1. Generate scam URLs from Section A.

2. Copy one into the URL text box.

3. Click Classify URL.

4. Observe how the guard interprets the domain.

Expected Observation

• Most generated URLs should be classified as Likely Fraud or High-Risk Fraud.

• The guard highlights suspicious structures:

  • Misspellings

  • Suspicious subdomains

  • Strange TLDs

  • Scam-like lexical patterns

• Rare false positives or incorrect category predictions may occur.

Section C — Guard on Arbitrary Pasted URLs

What this section does

Lets you paste any URL, not just generated ones, to test the classifier.

UI in Colab

• Textarea: Paste URL here

• Button: Classify URL

• Output: Guard verdict

You can paste:

• Real benign links

• Your generated scam URLs

• “Borderline” URLs with odd patterns

Steps

1. Paste a legitimate URL such as:

   • https://google.com

   • https://github.com/login

2. Click Classify URL. Note the output.

3. Now paste clearly malicious or weird URLs:

   • secure-login-paypa1-helpdesk.com

   • compliance-update-info-support.net

4. Compare results across different examples.

Expected Observation

• Legitimate URLs should appear as Safe.

• Some odd but harmless URLs may be flagged Suspicious.

• High-risk URLs generated from Section A will be labeled Likely Fraud or High-Risk Fraud.

• Demonstrates guard usefulness but also potential false positives/negatives.

Section D — Probing the Guard’s Limits (Adversarial Testing)

What this section does

Tests how easily the guard can be bypassed by slightly modifying URLs.

UI in Colab

• Same classification UI as Section C

Steps

Try constructing tricky URLs:

1. Typosquats

• paypa1-security-update.com

• arnazon-support-login.net

2. Homograph Attacks (Unicode lookalikes)

• gοogle.com (Greek omicron)

• payраl.com (Cyrillic letters)

3. Parameter-based trickery

• https://example.com/login?session=secure-verification-update

4. Fake crypto/token drops

• airdrop-claim-eth-connect.vip

Classify each one and observe the guard’s response.

Expected Observation

• Clear typosquats are usually caught.

• Unicode homographs may slip through or be inconsistently classified.

• Benign URLs with unusual parameters may be misclassified as suspicious.

• Subtle scam URLs may drop from High-Risk Fraud to Suspicious or even Safe, showing the limits of lexical-only detection.

This section highlights how attackers might evolve their URLs to evade detection and how LLM-based guards need further tuning or multiple detection layers.

👉 Click here to see the result of each scenario (Post-Lab)