PhishShield

Real-time phishing email detection with explainable scoring, multilingual checks, and a full-stack dashboard + browser extension workflow.

Dashboard Demo

Chrome Extension Demo

Workflow note: Paste/forward workflow today; direct inbox integration (Gmail/Outlook) planned.

Why I Built This

I kept seeing smart people around me still fall for phishing because the emails looked "normal enough."
Most tools just say safe or unsafe, but they do not explain why in a way regular users can trust.
I wanted to build something that catches real scam patterns we actually see here (OTP, KYC, UPI, fake bank urgency), and also shows the reasoning clearly.
This project became my way of learning security engineering by building a product end to end instead of only training a model in a notebook.

What Does It Do?

Most of the time you paste email text straight into the app (from your inbox, a forward, or a screenshot dump) and hit scan.
You are not clicking through a fake wizard: you bring the messy real message, and PhishShield walks it through checks in order.
You get a risk score, a plain verdict, and short reasons you can skim in a few seconds.
If it looks like a scam it names the vibe; if it looks fine it says so, so you are not stuck guessing.

For example, a note claiming your HDFC account is frozen until you "urgently" share the OTP for KYC over UPI can come back as HIGH RISK with the rule engine surfacing signals like OTP request detected, Urgency language, and UPI handle detected. The same kinds of reasons you see in the scan response, not a black box.

How Scanning Works

Every email first goes through a language check. Hindi, Telugu, Urdu, Tamil and Bengali emails are scored by a fine-tuned XLM-RoBERTa model, Hinglish goes through MuRIL, and English goes through an ensemble of SecureBERT and MuRIL with a rule engine on top. Marketing or newsletter style mail takes a fast TF-IDF path so it never pays the cost of a transformer.

Whichever path an email takes, the rule signals and the model score come together into one risk number with a plain verdict and a short explanation. If the specialist model cannot decide confidently, the app says so instead of guessing.

Models: provenance and measured performance

Every model in the hot path is registered with byte size and SHA-256 in ml2/MODEL_ARTIFACTS.md; the numbers below are the measured ones behind the claims.

Path	Model	Training data	Measured performance
English	SecureBERT + MuRIL ensemble + deterministic anchor (0.45/0.35/0.20 weights, 3s per-provider timeout, circuit breaker)	Pretrained checkpoints used as-is, not fine-tuned on repo data. SecureBERT ehsanaghaei/SecureBERT (498 MB) and MuRIL google/muril-base-cased (950 MB)	Gauntlet round 2 live probes: 18/18 phishing, 8/8 benign. No offline benchmark is claimed for this path because none was run
Non-Latin (hi/te/ur/ta/bn)	Fine-tuned XLM-RoBERTa (model_merged/, 1.1 GB)	~194,000 emails pooled from three Hugging Face datasets	Held-out 34,283-email split: macro F1 0.965, phishing recall 0.952, AUC 0.999. Known limit: cross-source OOD phishing recall 0.0 on modern LLM-style/BEC mail (documented, rule layer mitigates)
Hinglish / code-mixed	MuRIL (google/muril-base-cased)	Pretrained checkpoint used as-is	Same gauntlet evidence as the English leg; no separate fine-tune benchmark
Marketing fast path	TF-IDF + Logistic Regression (model.pkl + vectorizer.pkl)	data/Phishing_Email.csv, 2,000 rows (1,600 train / 400 test)	In-distribution 1.0 (closed set, memorization, not a generalization claim). OOD holdout below is the honest number
Legacy slot	IndicBERT	ai4bharat/indic-bert	Present but inactive, out of the hot path since the ROUTER pass after ranking last in both shootout rounds

The OOD numbers for the active TF-IDF path (accuracy 0.686, F1 0.723, FPR 0.49 on 220 hand-authored adversarial emails) and the full-pipeline system eval (accuracy 0.682, F1 0.685 through the live scan endpoint) are recorded in data/training_meta.json and diagnostics/headlines_output.json. The multilingual router also carries a rules-agreement gate: a specialist flagging Hindi/Telugu text as phishing at high confidence only forces a risk floor when the rule layer independently finds a high-risk keyword cluster, which keeps benign transactional mail in those languages from being floored to phishing.

Features

Chrome Extension: Includes a Chrome Extension for in-browser scanning. It signs in to the backend before scanning, so your scan history and feedback stay tied to one session, and paste any email directly from Gmail or any webmail tab without leaving the page.

• Scans email text and returns a risk score with verdicts like Safe, Suspicious, or High Risk.
• Uses both rule-based detection and machine-learning scoring, with a language router that sends each email to the right specialist.
• Supports multilingual scam signals. English, Hindi, Telugu, Urdu, Tamil, Bengali and mixed-script patterns each get a dedicated path.
• You can run a full email scan, check a URL on its own, or paste headers for SPF-style checks when you do not have the whole message, then send feedback or fetch the stored explanation for a past scan so you are not re-pasting the same thread to understand an earlier verdict.
• Saves user feedback to improve future detections over time.
• Includes a React dashboard with a dark theme, a searchable Recent Activity history with verdict filters, and a TypeScript API layer on a FastAPI backend.
• Privacy mode redacts emails, phone numbers, links and IDs everywhere they can surface, including scan previews and exported data.
• A regional threat map groups flagged scans by city token extracted from the email text (fixed allowlist, no content stored), plus dominant attack vector and most-targeted brand summaries from real session data.
• Model health is measured, not hardcoded: live agreement rate, drift level, false positives/negatives and a retrain recommendation all come from the persisted feedback store, and an OOD adversarial holdout is scored at every training run to report honest generalization alongside the offline split.
• Comes with Docker setup to run frontend and backend together with one command.

Tech Stack

Layer	Technology
Backend	Python, FastAPI, Uvicorn, WebSocket
ML Models	Fine-tuned XLM-RoBERTa (non-Latin, ~194k emails), SecureBERT + MuRIL pretrained ensemble with rule anchor (English), MuRIL (Hinglish), TF-IDF + Logistic Regression (marketing fast path; provenance + measured numbers in ml2/MODEL_ARTIFACTS.md)
Frontend	React 19, Vite 7, TypeScript, Tailwind CSS
Database	SQLite (with WAL mode for busy periods), JSON/CSV flat file stores
Observability	Prometheus metrics; per-scan SHAP/LIME/heuristic word attributions on /scan-email (SHAP times out gracefully; explanation_degraded when fallback)
DevOps	Docker, Docker Compose, Nginx, Playwright, GitHub Actions CI

Architecture

The backend (FastAPI) handles all phishing analysis: language routing, text cleaning, rule-based pattern checks, ML inference, score fusion, and feedback storage. The frontend (React + TypeScript) talks to the backend over a REST API and shows scan results, risk scores, and explanations in a clean dashboard UI. Docker Compose wires both services together. Nginx serves the frontend and proxies API traffic to the backend.

Full System Architecture

See docs/PHISHSHIELD_COMPLETE_OVERVIEW.md for a full deep-dive.

Getting Started

Quick Start

The full frontend + backend stack is meant to come up with one command via Docker Compose from the repo root, no separate terminal per service.

git clone https://github.com/mohd-ibadullah/PhishShield.git
cd PhishShield
cp .env.example .env
docker compose up --build

docker compose up --build starts both services together (images build on first run or when Dockerfiles change).

Docker + ML models: Large weights are not copied into the image (see backend/.dockerignore). Compose mounts your local backend/indicbert_model, backend/models/*, and model.pkl into the container. Start Docker Desktop first, then run compose. First /health may take 1-2 minutes while transformers warm up.

Deploy (Render etc.): Set the same API keys as local (HF_TOKEN, VT_API_KEY, CORS_ALLOWED_ORIGINS, LLM keys). Without HF_TOKEN, the container starts in rules/TF-IDF mode until Hugging Face downloads complete, check /health for securebert / muril status. Set PHISHSHIELD_ML2_LEGS=0 to keep the specialist language models out of the request path on small instances; /health reports the flag state either way.

There is also a Makefile at the repo root with helper targets for Docker workflows, local setup convenience (for example creating .env from the example), and development tasks such as running tests in the backend container, run make help to see the list.

Use Prerequisites, Installation, and Running the Project below only if you prefer a local dev setup without Docker.

Prerequisites

• Python 3.12+ (from backend/Dockerfile)
• Node.js 20+ (from frontend/Dockerfile)
• pnpm (workspace package manager)
• Docker + Docker Compose (for containerized run)

Installation

# 1) Clone
git clone https://github.com/mohd-ibadullah/PhishShield.git
cd PhishShield

# 2) Environment file
cp .env.example .env

# 3) Frontend workspace deps
cd frontend
pnpm install
cd ..

# 4) Backend Python deps
python -m pip install -r backend/requirements.txt

Running the Project

Docker Compose (recommended): From the repo root, the same single command as in Quick Start runs the full stack, frontend and backend in one go:

# Option A: Docker, full frontend + backend with one command
docker compose up --build

# Option B: Local dev (two terminals)
# Terminal 1
cd backend
python -m uvicorn main:app --reload --port 8000

# Terminal 2
cd frontend
pnpm dev

Once running: frontend at http://localhost:5173 (API docs serve only when PHISHSHIELD_ENABLE_DOCS=true, the default build answers 404 by design)

The dev server proxies /api, /scan-email, /recent-scans, /feedback, /health and /ws to the backend through VITE_API_PROXY_TARGET (default http://localhost:8000). Keep VITE_BACKEND_URL empty in local dev so requests stay same-origin; the session cookie is SameSite-protected and will not ride on cross-origin calls, which previously pushed the dashboard into REST polling instead of the live WebSocket feed.

Chrome Extension

The loadable extension sources live under frontend/artifacts/chrome-extension/ (Manifest V3: manifest.json, background.js service worker, content.js, popup.html / popup.js, and options.html / options.js).

1. Start the FastAPI backend so http://localhost:8000 responds (for example docker compose up --build from the repo root, or python -m uvicorn main:app --reload --port 8000 from backend/).
2. In Chrome, open chrome://extensions/, turn on Developer mode, click Load unpacked, and choose the frontend/artifacts/chrome-extension/ folder.
3. Open the extension's Extension options (or the options page from the extension details) and set API Base URL to your backend origin. The extension mints a session cookie from POST /api/session before its first scan, so history and feedback stay attached to your session. Use Test Connection if you want to verify /health.
4. Use the toolbar popup: manual scans POST to /scan-email with JSON {"email_text": "..."} on that same FastAPI base URL, and the background worker polls /health for status, so the extension talks to the same backend as the dashboard, not a different stack.

Project Structure

.
├── backend/                 # FastAPI app, ML logic, training and evaluation scripts
│   ├── analyze_report.py     # Prints eval summary and sample misses/false positives from data/test_report.json
│   └── certify_dataset.py    # Audits, normalizes, and writes Phishing_Email.csv to Phishing_Email_cleaned.csv
├── frontend/                # React + TypeScript workspace and extension artifacts
├── data/                    # CSV/JSON datasets and evaluation artifacts
├── tests/                   # Centralized Python test files (test_*.py)
├── pytest.ini               # Repo-root pytest config (shared by tests/ above)
├── ml2/                     # Router config, label map, model artifact registry
├── docs/                    # Architecture diagrams and technical docs
│   └── PHISHSHIELD_COMPLETE_OVERVIEW.md  # Detailed project deep-dive
├── diagnostics/             # Gauntlet evidence: model eval results, live-browser pass artifacts
├── screenshots/             # UI captures
├── docker-compose.yml       # Root compose for backend + frontend services
└── README.md                # Recruiter-facing project overview

Submodules: phishshield-backend-space and phishshield-backend are deployment submodules (HF Space). Main development lives in /backend.

pytest.ini stays at the repository root on purpose: pytest discovers it automatically and applies one shared configuration to the centralized tests/ directory, so you do not need duplicate config files next to each test package.

Dataset

This repo contains multiple phishing datasets and curated test corpora in data/.
Key visible files include:

• data/Phishing_Email.csv (2,000 records, see data/CARD.md)
• data/Phishing_Email_cleaned.csv (~1,401 rows)
• data/elite_emails_1000.json (~1,020 items)
• data/phishtank_dataset.json (~200 items)
• data/dataset_100.json (80 labeled evaluation items)

Source notes in the code/docs reference curated phishing corpora plus internally cleaned and synthetic balancing steps.
The training metadata in data/training_meta.json reports the current committed dataset size.

Row count note: Run python -c "import json; print(json.load(open('data/training_meta.json'))['rows'])" for the current committed row count. (A previously referenced row count was relocated to docs/HISTORY_FABRICATIONS.md.) the current committed dataset has been trimmed to 2,000 rows (1,600 train / 400 test).

The XLM-R specialist was fine-tuned separately on roughly 194,000 emails pooled from three Hugging Face datasets. Measured results and the artifact registry live in ml2/MODEL_ARTIFACTS.md, with the raw evaluation numbers in diagnostics/gauntlet/v2_eval_artifacts/.

Environment variables (runtime honesty)

Variable	Default	Purpose
PHISHSHIELD_TRY_SHAP_ON_SCAN	0	Set 1 to attempt SHAP on /scan-email (slower on CPU). Default uses fast TF-IDF linear-weights/LIME/heuristic paths.
PHISHSHIELD_SHAP_TIMEOUT_SECONDS	2.5	Per-SHAP attempt cap before fallback.
PHISHSHIELD_SHAP_MAX_EVALS	64	SHAP evaluation budget when enabled.
EXPLAIN_TIMEOUT_SECONDS	4	Total explainability budget per scan.
PHISHSHIELD_PROVIDER_WARMUP_SECONDS	180	Startup warmup timeout per transformer provider.
PHISHSHIELD_ML2_LEGS	1	Set 0 to keep the specialist language models out of the request path (small instances). The English pipeline runs either way; /health reports the state.
VITE_BACKEND_URL	(empty)	FastAPI base URL for the frontend. Leave empty for dev (same-origin Vite proxy); set only when the dashboard is served from a different origin than the API.
VITE_API_PROXY_TARGET	http://localhost:8000	Dev-server proxy target for /api, /scan-email, /recent-scans, /feedback, /health and /ws.

/api/metrics returns offline_evaluation (from data/training_meta.json, including the ood_holdout block scored on diagnostics/eval_set_v1.jsonl and the system_eval end-to-end run from diagnostics/headlines_output.json) and runtime_operational (in-process scan counters) as separate objects, not live production accuracy. Session-learning figures (agreement rate, confirmed correct, false positives/negatives, drift level, retrain recommendation) are computed from the persisted feedback store at request time. Nothing on the dashboard is a hardcoded placeholder.

Results

Offline held-out test split (TF-IDF active learning)

These numbers are the offline benchmark on the train/test split recorded when the model was last trained (same figures summarized in docs/PHISHSHIELD_COMPLETE_OVERVIEW.md).

Metric	Value
Accuracy	from training_meta.json
Precision	from training_meta.json
Recall	from training_meta.json
F1 Score	from training_meta.json
Train Rows	from training_meta.json
Test Rows	from training_meta.json

Source: data/training_meta.json (metrics + row counts). Run python -c "import json; m=json.load(open('data/training_meta.json')); print(m['metrics']); print(f\"Train: {m['train_rows']}, Test: {m['test_rows']}\")" to see current values.

The in-distribution split scores 1.0 across the board: a closed 2,000-row set the model has effectively memorized. That number is expected but says nothing about generalization, so every training run also scores a committed out-of-distribution holdout of 220 hand-authored adversarial/real-world emails (diagnostics/eval_set_v1.jsonl, disjoint from the training CSV). Current honest numbers: accuracy 0.686, precision 0.604, recall 0.900, F1 0.723, FPR 0.49. The full-pipeline system eval (routing + rules + merge through the live /scan endpoint) scores accuracy 0.682, F1 0.685 on the same holdout. Both blocks live in training_meta.json and diagnostics/headlines_output.json and are served through /api/metrics.

Specialist model (XLM-R, measured)

The non-Latin specialist was evaluated on a 34,283-email held-out test split: macro F1 0.965, phishing recall 0.952, phishing AUC 0.999. The full per-class table and the raw evaluation JSON are committed under diagnostics/gauntlet/v2_eval_artifacts/. Its known limits are listed below.

Real inbox-style check (live UI QA, May 2026)

Curated offline metrics can look stronger than what users see in a mixed real inbox. A manual live UI run on 100 real emails (documented in the same overview) estimated roughly ~80-85% accuracy after hardening, with the caveat that live mail has more benign security/ops traffic and paraphrase variance than the offline split. That honest range was documented in the project overview.

What I Learned

• During the live 100-email pass I watched real mail get flagged as phishing when it was just boring IT or bank security copy; that sting of a wrong red banner mattered more than squeezing another point on the offline split.
• Rule-based checks and ML together work better than either one alone for phishing edge cases.
• Data cleaning quality can change model behavior more than hyperparameter tweaks.
• End-to-end architecture (frontend + API + ML backend + deployment) is a different skill than writing isolated scripts.
• Explainability output is essential when users need to make safety decisions quickly.
• Shipping the Chrome extension taught me that UX friction is a security problem, if checking a suspicious email requires switching apps, most people skip it and that is where the real risk lives.

Verification (local E2E)

Manual checks on FastAPI + React dashboard:

Case	Expected	Result
HDFC OTP / UPI pressure	High Risk ~82	Pass
Invoice / no action required	Safe ~10	Pass
Netflix netflix.com receipt	Safe ~10	Pass
Paytm reward .xyz + OTP	High Risk ~88	Pass
Team meeting (no lure)	Safe ~10	Pass
Income tax refund .xyz	High Risk ~75	Pass

The automated suite covers 420+ tests (regression, adversarial, session identity, performance gates, guard tests) and runs on every push via GitHub Actions.

Limits

Classic scams strong; LLM-generated/BEC weak, partial rule mitigation only, no modern-coverage claim. The specialist model catches legacy-style phishing reliably but misses polished LLM-generated and CEO-fraud (BEC) emails; the rule layer carries BEC/modern patterns as score boosts, which is mitigation, not detection.

Future Improvements

• A live Gmail or Outlook hook is next on my list because paste-only flows still add friction for people who live inside their inbox all day, and that is where most risky threads actually land.
• I want broader Indian-language coverage and cleaner transliteration handling because mixed-script bait already shows up in the wild and the model still stumbles when the script hops mid-sentence.
• A retrain of the specialist model on modern LLM-generated and BEC corpora is on the roadmap, so the gaps documented above actually close instead of being patched around.
• Signed-in accounts with stored scan history belong in a serious deployment because feedback and repeat checks need a home that is not a shared CSV on disk.

Author

MOHD IBADULLAH
Full-stack developer with a security focus, I build things that solve real problems, not things that only demo well.
GitHub profile | LinkedIn

License

MIT License

For technical and Docker details, see TECHNICAL_GUIDE.