Siamese neural network are widely used to compare pairs of objects, such as faces, vehicles, and handwritten signatures. They learn an embedding space in which sample from the same class are mapped to nearby points, while sample from different classes are mapped farther apart. Consequently, the final decision relies on a distance threshold: two samples are considered to belong to the same class if the distance between their embedings is smaller than this threshold. However, defining an appropriate threshold is a challenging task and typically requires labeled data.
This repository accompanies a method for estimating the decision threshold under the assumption that the distribution of pairwise distances produced by a Siamese network follows a bimodal distribution. The proposed approach identifies the minimum point between the two modes as the decision threshold, eliminating the need for labeled data during threshold estimation. Experimental results show that the proposed method achieves accuracy comparable to the Equal Error Rate (EER) criterion. The method was evaluated on four benchmark datasets: MNIST, CIFAR-10, LFW, and PKLot.
.
├── CITATION.cff
├── LICENSE
├── models
│ ├── balanced_with_l2
│ └── balanced_without_l2
├── README.md
├── requirements.txt
├── results
│ └── balanced
│ ├── with_l2
│ │ ├── cifar10
│ │ ├── lfw
│ │ ├── mnist
│ │ ├── pklot1
│ │ ├── pklot2
│ │ └── pklot3
│ └── without_l2
│ ├── cifar10
│ ├── lfw
│ ├── mnist
│ ├── pklot1
│ ├── pklot2
│ └── pklot3
├── scripts
└── src
├── datasets
├── evaluation
├── metrics
├── models
├── training
├── utils
└── visualization
The src/ directory contains the complete implementation of the proposed framework, while scripts/ provides the experiment launchers used to reproduce the results reported in the paper.
This project was developed using Python 3.12 and PyTorch 2.6.0. The required dependencies are listed in the requirements.txt file.
The following software is required:
- Python >= 3.10
- pip >= 23.0
- NVIDIA GPU with CUDA support (recommended for training)
We recommend using a virtual enviroment to install the project dependencies.
Using venv:
python3 -m venv siamese-env
source siamese-env/bin/activateOr using Conda:
conda create -n siamese python=3.10
conda activate siameseInstall the required Python packages:
pip install --upgrade pip
pip install -r requirements.txtFor GPU acceleration, install the CUDA-enabled version of PyTorch compatible with your NVIDIA driver.
This project uses publicy available datasets for training, validation, and evaluation of Siamese neural networks.
Each dataset was independently used in the experiments and was divided into training, validation, and test subsets according to the experimental protocol.
| Dataset | Description | Source |
|---|---|---|
| MNIST | Dataset of 70,000 grayscale images of handwritten digits (0-9), widely used as a benchmark for image classification and representation learning. | Torchvision |
| CIFAR-10 | Dataset containing 60,000 color images from 10 object classes, commonly used for evaluating image classification and deep learning models. | CIFAR-10 |
| LFW | Face recognition dataset containing more than 13,000 face images from 5,749 identities, designed for face verification and identity recognition tasks. | LFW |
| PKLot | Parking lot occupancy detection dataset containing images of parking spaces captured under different weather and illumination conditions. | Not publicly available in the version used in this work. |
The datasets are not included in this repository due to size. They must be downloaded separately from their original sources.
After downloading, organize the datasets to the following structure:
.
└── datasets
├── cifar10
│ ├── test
│ └── train
├── lfw
│ ├── test
│ └── train
├── mnist
│ ├── test
│ └── train
├── pklot1
│ ├── test
│ └── train
├── pklot2
│ ├── test
│ └── train
└── pklot3
├── test
└── train
Each dataset must follow the ImageFolder directory structure, where each class is represented by a separate directory containing its corresponding images.
The PKLot experiments use three predefined cross-parking evaluation protocols. Since the version of the dataset used in this work is not publicly distributed in the required format, users must manually organize the data before running the experiments.
Create the following directory structure:
.
└── datasets
├── pklot1
│ ├── test
│ └── train
├── pklot2
│ ├── test
│ └── train
└── pklot3
├── test
└── train
Each protocol corresponds to a different combination of parking lots:
| Dataset | Training Set | Test Set |
|---|---|---|
PKLot1 |
UFPR04 + UFPR05 | PUCPR |
PKLot2 |
UFPR04 + PUCPR | UFPR05 |
PKLot3 |
UFPR05 + PUCPR | UFPR04 |
The training directories must contain the images from the corresponding training parking lots merged into a single train/ directory. Likewise, the test directories must contain the imaes from the designated test parking lot organizer under test/.
The repository does not automatically generate these splits. Users are responsible for manually creating the directory structure before executing the experiments.
All images are preprocessed using a standardized transformation pipeline before being used by the Siamese network.
The preprocessing steps are:
- Image resizing: Images are resized to a fixed resolution of
220 x 220pixels. - Color conversion: Images are converted to RGB format to ensure a three-channel input representation.
- Tensor conversion: Images are transformed into PyTorch tensors.
- Normalization: When ImageNet pretrained weights are used, images are normalized using the ImageNet mean and standard deviation:
- Mean:
[0.485, 0.456, 0.406] - Standard deviation:
[0.229, 0.224, 0.225]
- Mean:
The preprocessing pipeline is implemented in: src/datasets/transform.py.
The datasets are divided into training and test subsets. The validation set is generated during the experiment execution by splitting the training subset.
The validation split strategy depends on the pair_generation argument:
- default: the split is performed ate the sample level.
- pklot or lfw: the split is performed at the class level to avoid samples from the same class appearing in both training and validation sets.
The pair generation strategy depend on the dataset and its controlled by the pair_generation argument.
Avaiable strategies:
- default: Random pair generation according to the
pair_positive_rateargument. - pklot: FOr non-imbalanced experiments, each reference image generates one positive and one negative pair in alternating iterations.
- lfw: Positive and negative pairs are generated in alternating iterations.
Triplets are generated during Siamese network training. For each reference image (anchor), one image from the same class (positive sample) and one image from a different class (negative sample) are randomly selected.
Each dataset follows its original license and usage restrictions.
Users must consult the original dataset repositories before downloading and using the data.
All experiments are configured through the run_experiment.sh script, which defines the model, dataset, training, evaluation, runtime, and output parameters required to reproduce the experiments.
The experiments use a Siamese network with a MobiliNet backbone.
| Parameter | Value |
|---|---|
| Backbone | MobileNetV3 Large |
| Pretrained | ImageNet |
| Model ID | 6 |
| Distance Metric | Euclidean Distance |
The default training configuration is summarized below.
| Parameter | Value |
|---|---|
| Epochs | 30 |
| Batch size | 64 |
| Learning rate | 0.001 |
| Margin | 1.0 |
| Train ratio | 0.7 |
| Anchor swap | Enabled |
| Number of runs | 5 |
The evaluation is performed using sliding window with the following settings.
| Parameter | Value |
|---|---|
| Test mode | window |
| Step size | 1024 |
Whenever available, experiments are automatically executed using an NVIDIA GPU (CUDA). Otherwise, execution fall back to CPU.
The number of data loading workers is automatically determined according to the number of available CPU cores.
The following directories are created during execution:
| Directory | Description |
|---|---|
results/ |
Evaluation metrics and generated plots. |
checkpoints/ |
Saved model checkpoints. |
Outputs are automatically separated into balanced and unbalanced directories.
The experiments reported in the paper can be reproduced with the generic launcher:
./scripts/run_experiment.sh \
--dataset <DATASET>
--protocol <PROTOCOL>The PKLot Dataset contains images collected from three different parking lots: UFPR04, UFPR05, and PUCPR. Three cross-parking configurations are evaluated, where two parking lots are used for training and the remaining one for testing.
| Dataset | Description | Pair Generation |
|---|---|---|
MNIST |
Handwritten digit recognition dataset. | default |
CIFAR10 |
Natural object image classification dataset. | default |
LFW |
Face verification dataset. | lfw |
PKLot1 |
PKLot cross-parking protocol: Train: UFPR04 + UFPR05 Test: PUCPR | pklot |
PKLot2 |
PKLot cross-parking protocol: Train: UFPR04 + PUCPR Test: UFPR05 | pklot |
PKLot3 |
PKLot cross-parking protocol: Train: UFPR05 + PUCPR Test: UFPR04 | pklot |
| Protocol | Description |
|---|---|
balanced_with_l2 |
Balanced experiments with L2-normalized embeddings. |
balanced_without_l2 |
Balanced experiments without L2 normalization. |
imbalanced_with_l2 |
Imbalanced PKLot experiments with L2-normalized embeddings. |
| Dataset | Protocol |
|---|---|
| MNIST | balanced_with_l2 |
| MNIST | balanced_without_l2 |
| MNIST | imbalanced_with_l2 |
| CIFAR10 | balanced_with_l2 |
| CIFAR10 | balanced_without_l2 |
| CIFAR10 | imbalanced_with_l2 |
| LFW | balanced_with_l2 |
| LFW | balanced_without_l2 |
| LFW | imbalanced_with_l2 |
| PKLot1 | balanced_with_l2 |
| PKLot1 | balanced_without_l2 |
| PKLot1 | imbalanced_with_l2 |
| PKLot2 | balanced_with_l2 |
| PKLot2 | balanced_without_l2 |
| PKLot2 | imbalanced_with_l2 |
| PKLot3 | balanced_with_l2 |
| PKLot3 | balanced_without_l2 |
| PKLot3 | imbalanced_with_l2 |
All experiments are executed using the generic launcher:
./scripts/run_experiment.sh \
--dataset <DATASET> \
--protocol <PROTOCOL>For example, to reproduce the balanced MNIST experiment with L2 normalization:
./scripts/run_experiment.sh \
--dataset MNIST \
--protocol balanced_with_l2The launcher automatically configures the selected dataset, experimental protocol, model, training, evaluation, and output settings before invoking the main pipeline:
python3 -m src.main ...The available datasets and experimental protocols are described in the Experimental Configuration section.
Users who wish to customie the experimental protocol can modify the configuration parameters in scripts/run_experiment.sh or extend the launcher by adding new datasets or experimental protocols.
| Experiment | Command |
|---|---|
| MNIST (balanced, L2) | ./scripts/run_experiment.sh --dataset MNIST --protocol balanced_with_l2 |
| CIFAR-10 (balanced, no L2) | ./scripts/run_experiment.sh --dataset CIFAR10 --protocol balanced_without_l2 |
| LFW (balanced, L2) | ./scripts/run_experiment.sh --dataset LFW --protocol balanced_with_l2 |
| PKLot1 (imbalanced, L2) | ./scripts/run_experiment.sh --dataset PKLot1 --protocol imbalanced_with_l2 |
Pretrained Siamese network models for selected experiments reported in the paper are available in this repository.
The available checkpoints correspond to the balanced experiments.
models/
├── balanced_with_l2/
├── balanced_without_l2/
Each model checkpoint contains the trained network weights required for embedding extraction and threshold evaluation.
The provided checkpoints correspond to the final trained models used for embedding extraction. Threshold estimation is performed during evaluation and does not require additional model training.
The checkpoints from the imbalanced PKLot experiments are not included in this repository. These experiments can be reproduced by executing the corresponding training scripts described in the Execution section.
The results reported in the paper can be reproduced by executing the experiment configurations described in the Execution section.
During execution, the framework generates the following file:
.
├── checkpoints
│ ├── balanced
│ │ ├── with_l2
│ │ │ └── mnist
│ │ │ ├── Run_1
│ │ │ │ └── ImageNet_V3_Large
│ │ │ │ ├── checkpoint_e1.pth
│ │ │ │ └── checkpoint_e2.pth
│ │ │ └── Run_2
│ │ │ └── ImageNet_V3_Large
│ │ │ ├── checkpoint_e1.pth
│ │ │ └── checkpoint_e2.pth
│ │ └── without_l2
│ │ └── mnist
│ │ ├── Run_1
│ │ │ └── ImageNet_V3_Large
│ │ │ ├── checkpoint_e1.pth
│ │ │ └── checkpoint_e2.pth
│ │ └── Run_2
│ │ └── ImageNet_V3_Large
│ │ ├── checkpoint_e1.pth
│ │ └── checkpoint_e2.pth
│ └── imbalanced
│ └── with_l2
│ └── mnist
│ ├── Run_1
│ │ └── ImageNet_V3_Large
│ │ ├── checkpoint_e1.pth
│ │ └── checkpoint_e2.pth
│ └── Run_2
│ └── ImageNet_V3_Large
│ ├── checkpoint_e1.pth
│ └── checkpoint_e2.pth
└── results
├── balanced
│ ├── with_l2
│ │ └── mnist
│ │ ├── Bimodal_test.tikz
│ │ ├── Bimodal_validation.tikz
│ │ ├── Run_1
│ │ │ ├── test_distances.csv
│ │ │ ├── validation_e1.csv
│ │ │ └── validation_e2.csv
│ │ └── Run_2
│ │ ├── test_distances.csv
│ │ ├── validation_e1.csv
│ │ └── validation_e2.csv
│ └── without_l2
│ └── mnist
│ ├── Bimodal_test.tikz
│ ├── Bimodal_validation.tikz
│ ├── Run_1
│ │ ├── test_distances.csv
│ │ ├── validation_e1.csv
│ │ └── validation_e2.csv
│ └── Run_2
│ ├── test_distances.csv
│ ├── validation_e1.csv
│ └── validation_e2.csv
└── imbalanced
└── with_l2
└── mnist
├── Bimodal_test.tikz
├── Bimodal_validation.tikz
├── Run_1
│ ├── test_distances.csv
│ ├── validation_e1.csv
│ └── validation_e2.csv
└── Run_2
├── test_distances.csv
├── validation_e1.csv
└── validation_e2.csv
The output directory contains the evaluation metrics, threshold estimatation results, and auxiliary files generated during experiments.
Each experiment generates checkpoints, intermediate results, evaluation metrics, and visualization files.
| File | Description |
|---|---|
checkpoint_e*.pth |
Model checkpoint saved after each training epoch. |
validation_e*.csv |
Validation metrics computed at the end of each epoch. |
test_distances.csv |
Embedding distances computed for the test set. |
Bimodal_validation.tikz |
Bimodal distribution fitted to the validation distances. |
Bimodal_test.tikz |
Bimodal distribution fitted to the test distances. |
The following threshold estimation strategies are evaluated during the experiments.
| Method | Description |
|---|---|
| Bimodal (Updated) | The proposed bimodal threshold is periodically re-estimated during evaluation using the embedding distances observed so far. The update frequency depends on the selected evaluation mode (batch or sliding window). |
| Bimodal (Fixed) | The proposed bimodal threshold is estimated once and remains constant throughout the entire test set. |
| EER (Updated) | The Equal Error Rate (EER) threshold is periodically recomputed during evaluation using the embedding distances observed so far. |
| EER (Fixed) | The Equal Error Rate (EER) threshold is estimated once and kept fixed during the entire evaluation. |
| Margin | A fixed threshold equal to the margin used during training, serving as a baseline for comparison. |
FOr the Updated methods, the decision is periodically recomputed during evaluation using newly observed embedding distances.
The update strategy depends on the selected evaluation mode:
- Batch: the threshold is updated after processing a fixed number of image pairs (
step_size). - Sliding window: the threshold is updated using a moving window containing the most recent embedding distances.
The Fixed methods estimate the threshold only once and use the same value for all test samples.
The evaliation pipeline reports the following metrics for each experiment.
| Metric | Description |
|---|---|
AUC |
Area Under the Receiver Operating Characteristic (ROC) Curve. |
BIM |
Threshold estimated by the proposed bimodal method during evaluation. |
EET |
Threshold estimated using the Equal Error Rate (EER) criterion during evaluation. |
ACCbim |
Classification accuracy obtained using the dynamically updated bimodal threshold. |
ACCeet |
Classification accuracy obtained using the dynamically updated EER threshold. |
ACCmar |
Classification accuracy obtained using the fixed margin value adopted during training. |
ACCfixBim |
Classification accuracy obtained using the bimodal threshold estimated once. |
ACCfixEet |
Classification accuracy obtained using the EER threshold estimated once. |
minBIM |
Minimum bimodal threshold estimated during evaluation. |
maxBIM |
Maximum bimodal threshold estimated during evaluation. |
fixBIM |
Fixed bimodal threshold estimated. |
minEET |
Minimum EER threshold estimated during evaluation. |
maxEET |
Maximum EER threshold estimated during evaluation. |
fixEET |
Fixed EER threshold estimated. |
Three types of threshold values are reported:
- Updated thresholds (
BIMandEET): recomputed during evauation using either the batch or sliding-window strategy. - Fixed thresholds (
fixBIMandfixEET): estimated once and kept constant throughout the test. - Minimum and maximum thresholds (
minandmax): smallest and largest threshold values observed during the dynamic evaluation.
Each experiment is executed multiple times using independent training runs.
The output of each run is stored in its corresponding Run_<N>directory, allowing the variability of the training process to be analyzed separately.
The generated .tikz files contain publication-quality figures that can be directly included in LaTeX documents. They illustrate:
- the distribution of embedding distances;
- the fitted bimodal Gaussian model;
- the estimated decision threshold.
This repository provides all necessary components to reproduce the experiments reported in the paper, including the source code, experiment cofigurations, and evaluation pipeline.
The reproduction workflow consists of the following steps:
-
Eviroment setup
Install the required dependencies:
pip install -r requirements.txt
-
Dataset preparation
Download the datasets from their original sources and organize them according to the structure described in the Dataset section.
The datasets are not included in this repository due to size and licensing restrictions.
-
Experimental execution
Run the desired experiment using the launcher:
./scripts/run_experiment.sh \ --dataset <DATASET> \ --protocol <PROTOCOL>The available datasets and protocols are described in the Experimental Configuration section.
-
Model evaluation
The trained models can be evaluated using the generated checkpoints or the provided pretrained models.
-
Results analysis
The evaluation pipeline generates the metrics and files required to compare the results with those reported in the paper.
All experiments reported in the paper can be reproducec using the configuration provided in this repository.
The experiments are repeated multiple times using the same configuration. The reported results are obtained from multiple independent runs.
The following parameters are fixed across experiments:
- Model architecture
- Training hyperparameters
- Dataset slit protocol
- PAir/Triplet generation strategy
- Evaluation procedure
If you use this repository in your reserarch, please cite:
@inproceedings{viotto2026siamese,
title = {How Far is Too Far? Defining the Distance Threshold for Verification Siamese Networks},
author = {Viotto, Heloísa and Samonek, Cauê and Garcia Pedroso, Lucas and Sunye, Marcos and Abed Grégio, André and Lisboa de Almeida, Paulo},
booktitle = {2026 IEEE International Conference on Systems, Man, and Cybernetics (SMC)},
year = {2026},
note = {To appear}
}The source code of this repository is released under the MIT License.
Datasets and pretrained models may be subject to additional terms specified by their respective owners. User must consult the original dataset providers before downloading and using the data.