feature-steering

A complete end-to-end pipeline for LLM interpretability with sparse autoencoders (SAEs) using Llama 3.2, written in pure PyTorch and fully reproducible.

Feature steering for open LLMs: find an interpretable SAE feature, steer the model, measure the causal effect

Reproducible case study of pitfalls in contrastive SAE discovery and steering for "consciousness" features (GemmaScope SAEs, Gemma 3 4B/12B): reconstruction confound, delta-steering fix, matched controls, and false-positive scaling law vs dataset size.

Agnostic PNG scanner that extracts hidden tEXt chunks, decodes base64 payloads and reads PyTorch steering vectors.

Explore limitations of contrastive SAE steering in identifying causal consciousness features and introduce delta-steering to improve experiment validity.

Use GemmaScope and Neuronpedia features to steer Gemma 3 4B IT locally

Built with DINOv2, sparse autoencoders, FAISS and automatic feature naming with a local vision-language model.

Reproducing Anthropic's Golden Gate Claude on an open model: extracting a concept direction from Qwen3-1.7B's residual stream with TransformerLens, then steering generation with a single vector addition — no fine-tuning, no gradients.

Minimal replication of Anthropic's Golden Gate Claude on consumer hardware. Trains a Sparse Autoencoder on Qwen2.5-1.5B, discovers interpretable features, and steers model behavior, all on an RTX 3070 Ti.