Machine Learning Algorithms

A curated collection of classification, regression, tree-based learning, and ensemble experiments developed during my MSc in Artificial Intelligence.

The repository separates from-scratch implementations from scikit-learn baselines. It retains both successful and unsuccessful experiments where the comparison provides a useful technical lesson.

Notebook guide

Notebook	Focus	Implementation
01 — KNN classifier	Euclidean distance, majority vote, k-fold evaluation	From scratch
02 — KNN regressor	Manhattan distance, regression metrics, runtime	From scratch
03 — Linear SVM classification	ROC/AUC, threshold selection, confusion matrices	scikit-learn
04 — Linear SVM regression	Scaled SVR baseline and cross-validation	scikit-learn
05 — Decision-tree classification	Pre-pruning vs cost-complexity pruning	scikit-learn
06 — Decision-tree regression	Regression, cross-validation, rule extraction	scikit-learn
07 — Decision tree and random forest	Mixed-type splits, entropy, pruning, bagging	From scratch
08 — Ensemble-learning experiments	AdaBoost–MLP and perceptron forest	From scratch / hybrid

What is implemented from scratch

• Euclidean and Manhattan distance functions
• KNN classification and regression prediction
• Decision-tree nodes and recursive tree construction
• Entropy and information-gain split selection
• Numeric and categorical split handling
• Pre-pruning and reduced-error post-pruning
• Bootstrap aggregation and random feature selection
• Majority-vote forest prediction
• AdaBoost-style sample weighting with MLP base estimators
• A perceptron-based random forest

Selected observations

• The KNN classifier produced consistently strong fold-level performance on the Wisconsin Diagnostic Breast Cancer dataset.
• Pre-pruning gave the best balance between tree complexity and predictive performance in the Abalone age-group experiment.
• The perceptron ensemble reached 87.32% accuracy on the Ionosphere experiment.
• The AdaBoost–MLP configuration collapsed toward one class. It is deliberately retained with a critical diagnosis rather than presented as a successful model.

Datasets

The notebooks fetch public datasets from the UCI Machine Learning Repository at runtime:

• Breast Cancer Wisconsin (Diagnostic)
• Bike Sharing
• Abalone, DOI: 10.24432/C55C7W
• Ionosphere, DOI: 10.24432/C5W01B

Dataset rights remain with their respective creators and repositories.

Run locally

python -m venv .venv
source .venv/bin/activate       # Windows: .venv\Scripts\activate
pip install -r requirements.txt
jupyter lab

An internet connection is required the first time a notebook downloads its dataset.

Notes on scope

These notebooks are learning and portfolio artefacts, not production ML services. Reproducibility controls, additional baselines, nested model selection, and automated tests would be natural next steps for production use.

Author

Kübra Yurduseven