Machine Learning and Deep Learning

Interactive demo · GitHub repository · CI checks

A reproducible laboratory for probabilistic inference, modern prediction systems, conditional image generation and distribution shift. Implementations connect equations to trained models, held-out comparisons and usable saved-model inference.

Developed by Navid Bagheri-Shouraki, from earlier machine learning and deep learning coursework into a reproducible research and applied data science portfolio. The theoretical foundation follows Murphy's Probabilistic Machine Learning and Goodfellow, Bengio, and Courville's Deep Learning. The extensions focus on calibration, conformal inference, distribution shift, representation learning, statistical comparisons, practical decision costs, and image inverse problems with explicit observation models.

The modern extension compares conditional GANs, conditional VAEs and diffusion with frozen DDColor, CT2, BigColor, SwinIR and Real-ESRGAN on original 256px photographs. It also adds native categorical/quantile boosting, three-seed ConvNeXt transfer and natural-image denoising under noise mismatch. Measured comparisons, model equations and protocols, and the implementation audit separate practical gains from remaining failures. The upgrade work log records current completion.

Open the interactive explorer locally by double-clicking it, or serve the repository with python -m http.server 8000 and visit http://localhost:8000/docs/explorer.html. It runs entirely in your browser. The demo guide explains all eight concepts, controls, evidence and optional GitHub Pages setup.

The completed comparison extension adds 243 classical prediction models across five dataset/split cases and three seeds, 63 clustering fits, and 20 controlled restoration fits spanning hinge, logistic/R1, least-squares and WGAN-GP objectives. Validation-selected comparisons and 20 fixed final-epoch diagnostics expose unsuccessful updates as well as useful changes. These are separate deployment and diagnostic policies.

The modern report includes every stochastic draw, paired image intervals and comparisons against a trained direct predictor. It also explains why a grayscale baseline's mean PSNR can misrepresent typical color recovery.

The user interaction audit documents exhaustive enabled dropdown checks, representative joint slider settings and source-bound asset checks.

What is inside

Study	Substantial question	Implementations and evidence
Classical prediction	Which modeling assumptions and ensembles work on each split?	LDA/QDA, logistic/linear models, KNN, SVM/SVR, bagging, forests, boosting, voting and leakage-aware stacking; separate selection/calibration; saved-model prediction sets and intervals
Clustering geometry	When do density, connectivity and Gaussian assumptions recover structure?	KMeans, full/variational GMM, Ward, spectral, DBSCAN and HDBSCAN; Wine, Digits and unequal-density mixtures; noise coverage and external diagnostics
Adversarial objectives	What do objective and regularization changes actually change?	Matched warm starts, five controlled families, both tasks and seeds; validation includes epoch zero; fixed final endpoints reveal effects hidden by selection
Conditional colorization	Does a generative model improve appearance, exact color, or diversity?	Direct predictor, conditional GAN, CVAE and velocity diffusion; frozen DDColor, CT2 and BigColor; 256px source photographs, prior-only sampling, LPIPS, DeltaE and dispersion
Modern 4× super-resolution	Which details are reconstructed, and which depend on a learned prior?	Matched local direct/GAN/CVAE/diffusion models, frozen SwinIR and Real-ESRGAN; actual LR observations, fidelity/perception comparisons and blur/noise/JPEG stress tests
Native boosting	Do categorical and quantile models improve practical decisions?	CatBoost with isolated selection and calibration; subgroup reliability, permutation diagnostics, deployable prediction sets and intervals; random/geographic housing protocols
Modern image transfer	What does fine-tuning add with 5,000 fitting labels?	Three-seed ImageNet ConvNeXt fine-tuning; 5,000 additional selection/calibration labels; identical CIFAR partitions, paired NLL comparisons, confusion matrices, precision-bound deployment prediction sets and corruption failures
Natural-image noise shift	Does a strong denoiser remain useful at a different noise level?	Wavelet BayesShrink versus frozen SwinIR on 400 photographs at three noise levels; retained sigma50 failure
Probabilistic foundations	What does a posterior represent, and when does clustering recover geometry?	Gaussian posterior from scratch; log-domain full-covariance EM; held-out density, BIC, clustering stability
Tabular decisions	Does a neural network improve a well-tuned classical model?	Adult official splits; logistic regression, histogram boosting, residual tabular network; calibration, utility, subgroup and permutation diagnostics
Regression and uncertainty	Does a nominal 90% interval remain reliable under geographic shift?	California Housing; ridge, Bayesian ridge, boosting and numerical feature transformer; posterior intervals, split conformal and CQR
Generalization	Why can interpolation coexist with good prediction, and what does GD prefer?	Repeated double-descent study with exact population risk; ridge controls; logistic implicit-bias experiment and hard-margin reference
Kernels	When do nonlinear features pay off with limited data?	Exact RBF SVM, Nyström approximation, logistic regression; validation-selected regularization; repeated label budgets
Image representations	What changes when labels, unlabeled images, or external pretraining are available?	Full CIFAR-10; residual CNN, contrastive pretraining, frozen random features and ImageNet transfer; identical low-label splits, calibrated prediction sets and corruption diagnostics
Signal equalization	Can a learned receiver remove nonlinear distortion and channel memory?	Gray-coded 16-QAM simulator; linear window equalizer, MLP, temporal CNN, known-channel inversion; BER across SNR and channel shift; saved-checkpoint decoding
Course mixture geometry	Can a density model recover overlapping labels or impute missing observations?	Supplied coordinates; custom/library/variational GMMs; categorical EM with missing data
Bayesian reconstruction	Does the actual corruption match the stated likelihood?	Stable mixture conditioning, MMSE versus hard estimates, six patch sizes, archive audit, validation-only forward calibration
Denoising	When does residual learning beat PCA and an autoencoder?	10,000 MNIST test images; three seeds; noise-conditioned CNN; PCA shrinkage; noise shift
Super-resolution	What survives a 4× information bottleneck?	CC BY natural images; bicubic/autoencoder/residual pixel shuffle; two seeds; blur mismatch; paired comparisons
Colorization	How do exact-color fidelity and plausible chroma differ?	Lab autoencoder, ImageNet transfer and balanced palette classification; DeltaE, chroma and entropy
Modern pretrained reference	What changes with a large externally trained color decoder?	Pinned official DDColor-ModelScope; lightness-only inputs, identical test images, checked weights, throughput and explicit exposure limits

All tables and figures are built from saved experiment records. Compact integration runs are separate from reference studies. The foundations report, course-project report, and modern report document their respective budgets, variability, comparisons and limitations.

The course-project extension report adds reconstruction galleries, complete training curves, paired-image evidence, and an audit of the newly provided briefs and corrupted images.

Completed modern measurements: ConvNeXt fine-tuning reaches 96.65% CIFAR-10 accuracy with 5,000 fitting labels, 5,000 selection/calibration labels and ImageNet initialization (three seeds); native CatBoost reaches Adult AUROC 0.9271 (three seeds). SwinIR achieves SR PSNR 27.13 dB; local diffusion achieves LPIPS 0.1486 with PSNR 23.88 dB, showing a fidelity/perception tradeoff. Geographic interval failure, noise mismatch, ambiguous colors and weak CVAE diversity remain visible in the modern report. The laboratory contains 68 completed reference protocols, 20 additional fixed-endpoint evaluations and 22 English walkthroughs.

Fixed visual comparisons across all restoration pipelines

Four attributed held-out photographs, with seed0/draw0 fixed for every pipeline. Full galleries retain all fitted seeds and stochastic draws.

Completed reference results: Adult boosting AUROC 0.9257 (three seeds); CIFAR-10 full supervised accuracy 92.79% (one run); temporal receiver BER 0.000510 at 18 dB (three seeds). The housing study documents pronounced coverage loss under geographic shift. Read the full report for protocols and limitations.

Added project results: residual denoising 29.85 dB versus PCA 23.40 dB; 4× residual SR 23.87 dB versus bicubic 22.14 dB; supplied m=4 image reconstruction 18.86 dB after validation-only calibration, versus 9.39 dB for the corrupted input. The extension report includes seed variation, fixed galleries, failure analysis and the frozen DDColor comparison.

Run it

Python 3.10+; Linux/macOS CPU works for probabilistic, kernel and generalization studies. CUDA is useful for neural studies. Install the appropriate PyTorch build for your machine using the official installation instructions.

python -m venv .venv
source .venv/bin/activate
python -m pip install -e '.[deep,notebooks,dev]'
python -m pytest -q
python -m learning_lab.cli probabilistic --quick

For native boosting and modern restoration, add the corresponding optional dependencies:

python -m pip install -e '.[deep,boosting,restoration,pretrained,notebooks,dev]'
python scripts/make_portable_configs.py --help
python scripts/restore_generative.py --help
python scripts/restore_external.py --help
python scripts/predict_modern.py --help
python scripts/predict_classical.py --help

Saved-model inference does not require training data. Model checkpoints and downloaded datasets stay outside the published repository; the public release contains their provenance, metrics and compact numerical evidence. Explicit preparation and fitting commands are documented in reproduction.

The reference configs use the author's shared dataset cache. For another machine, prepare a local cache and pass its path explicitly:

python scripts/prepare_data.py --data-root .local/data
python -m learning_lab.cli classification --data-root .local/data --device cpu --quick
python -m learning_lab.cli regression --data-root .local/data --device cpu
python -m learning_lab.cli generalization
python -m learning_lab.cli kernels
python scripts/run_classical.py --dataset breast_cancer --seed 0 --quick
python scripts/run_clustering.py --help

Image training requires the official CIFAR-10 Python archive and ResNet18 ImageNet weights. See reproduction for explicit preparation, full runs, GPU selection, CPU overrides, artifact regeneration, and environment details. Training never silently downloads data or model weights.

Follow an idea into code

Start with the learning path, or open the notebooks. Each chapter includes assumptions, equations, implementation links, an experiment to inspect, and an exercise with a worked answer.

src/learning_lab/       reusable numerical methods, networks, experiments and CLI
configs/               complete reference budgets and explicit data/model paths
notebooks/             executed English walkthroughs
docs/                  derivations, protocols, model cards, results and references
assets/                generated plots (PNG and PDF)
results/reference/     metrics, histories, exact splits and environment records
tests/                 mathematical, statistical and interface checks
scripts/               explicit preparation, reporting, inference and release checks
legacy/                source inventory; original files remain local and excluded from Git

The project demonstrates experimental design, statistical reasoning, model implementation, applied error analysis, and reproducible engineering. Classical methods remain meaningful baselines; neural and pretrained models receive explicit data and compute budgets. Reported findings describe these experiments rather than claiming new algorithms or state-of-the-art performance. Model cards explain intended use and practical limits.

Attribution

The public code and English explanations are a new implementation. Original assignment files and third-party paper PDFs are preserved locally; they are excluded from the release. The source audit maps the original topics to the completed studies. Methods and datasets are credited in references.

MIT license for repository code and original documentation. Dataset and pretrained-model terms remain with their respective sources. To cite the laboratory, use CITATION.cff. Natural-image figures and their compact source arrays retain CC BY photo attribution.