Offroad Segmentation - IBA Datathon ML

A semantic segmentation solution for offroad environment recognition using DINOv2 backbone with a ConvNeXt-style segmentation head, optimized for Kaggle training environment.

Project Overview

This project implements a deep learning model for multi-class semantic segmentation of offroad scenes. The model identifies 10 different classes of offroad elements including trees, bushes, grass, rocks, logs, and landscape features.

Competition Platform: Kaggle
Target Metric: Mean IoU (Intersection over Union)

Quick Start

For Kaggle Users

Upload Notebook: kaggle_train.ipynb to Kaggle
Add Dataset: "IBA Train" via Add Data
Enable GPU: Select P100 or T4 accelerator
Run All: Execute all cells sequentially

That's it! The notebook auto-detects paths and generates predictions.

Expected Output

Training Time: ~1-2 hours (10 epochs on P100)
Final Val IoU: 0.48-0.56
Test Predictions: 80 PNG files in /kaggle/working/test_predictions/
Visualizations: Training curves + prediction samples

Model Architecture

Backbone: DINOv2 (Vision Transformer, small variant - vits14)
- Pre-trained on large-scale image data
- Frozen during training (eval mode)
- Embedding dimension: 384
Segmentation Head: ConvNeXt-style architecture
- Input: Patch tokens from DINOv2 (384 channels)
- Architecture: 7×7 depthwise convolutions + GELU activations
- Output: 10-class segmentation masks

Dataset

Classes (10 total):

Class ID	Raw Value	Class Name	Color (Visualization)
0	0	Background	Black
1	100	Trees	Forest Green
2	200	Lush Bushes	Lime Green
3	300	Dry Grass	Yellow
4	500	Dry Bushes	Saddle Brown
5	550	Ground Clutter	Olive
6	700	Logs	Sienna
7	800	Rocks	Dark Gray
8	7100	Landscape	Tan
9	10000	Sky	Sky Blue

Note: Raw mask values are automatically converted to class IDs (0-9) during training.

Structure:

Offroad_Segmentation_Training_Dataset/
├── train/
│   ├── Color_Images/
│   └── Segmentation/
└── val/
    ├── Color_Images/
    └── Segmentation/

Training Configuration

Input Resolution: 476×266 (downscaled from 960×540, divisible by 14 for ViT patches)
Batch Size: 2
Optimizer: SGD with momentum (0.9)
Learning Rate: 1e-4
Loss Function: Cross-Entropy Loss
Epochs: 10

Evaluation Metrics

The model is evaluated using three metrics:

Mean IoU (Intersection over Union): Measures segmentation accuracy per class
Dice Score (F1 Score): Harmonic mean of precision and recall
Pixel Accuracy: Overall percentage of correctly classified pixels

Implementation Details

Data Processing

Images are resized to 476×266 pixels
Normalization: ImageNet statistics (mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
Mask values are converted from raw pixel values to class IDs (0-9)

Training Process

Load DINOv2 backbone from torch.hub (frozen)
Extract patch token embeddings (no gradient computation)
Train only the segmentation head
Evaluate on validation set after each epoch
Save training curves and metrics

Outputs

The training script generates:

Model & Metrics (/kaggle/working/):

segmentation_head.pth: Trained model weights
train_stats/evaluation_metrics.txt: Detailed numerical results with best epoch tracking
train_stats/all_metrics_curves.png: Combined visualization (Loss, IoU, Dice, Pixel Accuracy)

Visualizations (/kaggle/working/train_stats/):

predictions_visualization.png: 6 validation samples showing Input, Ground Truth, Prediction, and Overlay
class_legend.png: Color-coded class reference

Test Predictions (/kaggle/working/test_predictions/):

80 grayscale PNG files with class IDs for submission
Predictions at original resolution (540×960)

Key Features

✅ Kaggle-Optimized

Auto-Detection System:

Automatically detects dataset location in /kaggle/input/
Handles multiple naming conventions: iba-train, IBA Train, IBA-Train
Auto-discovers train/val directory structure
Validates dataset integrity with file count verification

GPU Acceleration:

Runs on Kaggle GPU (Tesla P100/T4)
Mixed precision training support (optional)
Efficient memory usage with batch size 2

Complete Pipeline:

Training with real-time metrics (IoU, Dice, Pixel Accuracy)
Validation evaluation after each epoch
Test set inference for 80 public test images
Comprehensive visualization generation

Performance Evaluation

Expected Performance

Based on the training configuration with DINOv2-Small backbone:

Metric	Expected Range	Interpretation
Mean IoU	0.48-0.56	Primary submission metric
Dice Score	0.55-0.70	Typically 5-10% higher than IoU
Pixel Accuracy	0.75-0.85	Can be misleading due to class imbalance

Per-Class Performance (Predicted)

Class	IoU Range	Difficulty	Main Challenge
Sky	0.70-0.80	Easy	Largest, most distinct class
Landscape	0.60-0.70	Easy	Large regions
Trees	0.55-0.65	Medium	Boundary blur with sky
Lush Bushes	0.50-0.60	Medium	Mixed with trees
Background	0.50-0.60	Medium	Confused with clutter
Dry Grass	0.40-0.50	Hard	Similar to dry bushes
Rocks	0.35-0.45	Hard	Small objects, texture confusion
Dry Bushes	0.35-0.45	Hard	Texture similarity
Ground Clutter	0.30-0.40	Very Hard	Most ambiguous class
Logs	0.25-0.35	Very Hard	Smallest objects, rare

Known Failure Modes

Boundary Inaccuracies: Blurry edges due to patch-based ViT processing (14×14 patches)
Small Object Confusion: Logs and rocks often misclassified due to limited patches
Texture-Based Errors: Dry vegetation classes confused (grass/bushes/clutter)
Sky Oversegmentation: Sky bleeds into trees and horizon objects

Improvement Recommendations

Quick Wins (Expected +10-15% IoU):

Add horizontal flip augmentation
Switch to AdamW optimizer with OneCycleLR scheduler
Increase training to 15-20 epochs
Add color jitter augmentation

Medium Impact (+8-12% IoU):

Upgrade to DINOv2-Base (768-dim embeddings)
Implement class-weighted loss for rare classes
Multi-scale testing ensemble

Advanced (+10-15% IoU):

Replace segmentation head with UperNet decoder
Fine-tune last 2 DINOv2 layers
Test-time augmentation with flips and scales

Target Performance: With recommended improvements, mIoU > 0.65 is achievable.

System Requirements

Minimum:

Python 3.8+
PyTorch 2.0+
CUDA-capable GPU (recommended for training)
8GB RAM (16GB recommended)

Dependencies:

# Automatically installed in notebook first cell
pip install opencv-python-headless tqdm matplotlib pillow --quiet

# Pre-installed on Kaggle
torch torchvision numpy

Submission Guide

Preparing Submission Files

After training completes, the notebook automatically generates test predictions:

Output Location: /kaggle/working/test_predictions/
File Format: Grayscale PNG images
Pixel Values: Class IDs (0-9)
Resolution: 540×960 (original image size)
Count: 80 files matching test set

Verification Checklist

Before submitting, verify predictions:

import numpy as np
from PIL import Image

# Check a sample prediction
pred = Image.open('/kaggle/working/test_predictions/test_0001.png')
arr = np.array(pred)

print(f"✓ Shape: {arr.shape}")           # Should be (540, 960)
print(f"✓ Unique values: {np.unique(arr)}")  # Should be [0,1,2,...,9]
print(f"✓ Dtype: {arr.dtype}")           # Should be uint8
print(f"✓ File count: 80")               # Should match test set size

Download from Kaggle

Navigate to Kaggle notebook output panel (right sidebar)
Download test_predictions.zip or entire /kaggle/working/ folder
Extract predictions and verify file count
Submit according to competition guidelines

Usage

Running on Kaggle

Upload Notebook: Upload kaggle_train.ipynb or iba-ipynb.ipynb to Kaggle
Add Dataset: Add "IBA Train" dataset in Kaggle's "Add Data" section
Enable GPU: Select GPU accelerator (P100 or T4 recommended)
Run All Cells: Execute cells sequentially

No Manual Configuration Required! The notebook automatically:

Detects dataset location
Finds train/val directories
Configures paths for outputs

Dataset Paths (Auto-Detected)

The notebook searches for datasets in this order:

# 1. Check wrapped datasets folder
if os.path.exists('/kaggle/input/datasets'):
    dataset_root = '/kaggle/input/datasets/<user>/<dataset-name>'

# 2. Fallback to common names
for name in ['iba-train', 'IBA Train', 'IBA-Train']:
    if os.path.exists(f'/kaggle/input/{name}'):
        dataset_root = f'/kaggle/input/{name}'
        break

Local Development

For local testing with adjusted paths:

# Modify dataset paths in the notebook
train_dir = './Offroad_Segmentation_Training_Dataset/train'
val_dir = './Offroad_Segmentation_Training_Dataset/val'
test_dir = './test_public_80'

Project Structure

IBA_Datathon_ML/
├── kaggle_train.ipynb              # Main training notebook (Kaggle-optimized)
├── iba-ipynb.ipynb                 # Alternative training notebook
├── test.ipynb                      # Testing/inference notebook
├── README.md                       # This file
├── Offroad_Segmentation_Training_Dataset/
│   ├── train/
│   │   ├── Color_Images/           # Training RGB images
│   │   └── Segmentation/           # Training masks
│   └── val/
│       ├── Color_Images/           # Validation RGB images
│       └── Segmentation/           # Validation masks
└── test_public_80/
    ├── Color_Images/               # 80 test images
    └── Segmentation/               # Test masks (optional)

# Kaggle Output Structure (auto-generated)
/kaggle/working/
├── segmentation_head.pth           # Trained model
├── test_predictions/               # 80 prediction PNGs
└── train_stats/
    ├── evaluation_metrics.txt      # Numerical results
    ├── all_metrics_curves.png      # Training curves
    ├── predictions_visualization.png
    └── class_legend.png

Acknowledgments

DINOv2: Meta AI Research (facebookresearch/dinov2)
Dataset: IBA Train - Offroad Segmentation Dataset
Platform: Kaggle Notebooks (GPU: Tesla P100/T4)
Framework: PyTorch 2.0+

Technical Notes

Model Design Choices

Why DINOv2?

Self-supervised pre-training on diverse images
Strong feature extraction without task-specific training
Efficient inference with frozen backbone

Why ConvNeXt-Style Head?

Depthwise convolutions reduce parameters
GELU activations improve gradient flow
Simple architecture for faster prototyping

Why Small Variant?

Fits in Kaggle GPU memory with batch size 2
384-dim embeddings provide sufficient capacity
Training completes in ~1-2 hours for 10 epochs

Training Strategy

The notebook uses a frozen backbone + trainable head approach:

Backbone: DINOv2 in eval mode (no gradients)
Head: Full gradient updates with SGD
Rationale: Leverages pre-trained features while avoiding overfitting

Future Work

Implement multi-scale training
Add auxiliary loss heads for better feature learning
Experiment with transformer-based decoders (Mask2Former)
Fine-tune backbone last layers for domain adaptation
Ensemble multiple checkpoints for test-time inference

Project: IBA Datathon ML - Offroad Segmentation
Last Updated: February 15, 2026
Version: 1.0

Name		Name	Last commit message	Last commit date
.gitignore		.gitignore
README.md		README.md
class_legend.png		class_legend.png
iba-ipynb.ipynb		iba-ipynb.ipynb
predictions_visualization.png		predictions_visualization.png

Latest commit

History

Folders and files

Repository files navigation