InfaTrackVision — Non-contact vibration measurement for SHM

InfaTrackVision is a non-contact vibration measurement system that extracts structural displacement time histories from camera images and uses them as input to time-series deep-learning models for structural-state diagnosis in Structural Health Monitoring (SHM).

The system tracks QR-code markers attached to a structure through an industrial camera, using an image-processing pipeline of template matching, sub-pixel localization, and pixel-to-millimetre calibration. This repository contains the measurement software, the deep-learning training code, and the experimental dataset used to benchmark the approach against a contact-based accelerometer.

This code and data accompany the research paper "InfaTrackVision: A non-contact vibration measurement system using computer vision combined with deep learning for structural health monitoring" (under review). Please cite the paper when using these materials.

Repository layout

.
├── infatrackvision_app/   # Desktop app: camera capture, ROI tracking, displacement extraction
├── deep_learning/          # Training scripts, trained-run artefacts, and result summaries
└── datasets/               # Synchronized displacement (camera) and acceleration (PASCO) data

Experiment at a glance

A three-storey laboratory frame was excited continuously (band-limited white noise) while two sensing modalities recorded its response simultaneously:

• InfaTrackVision — horizontal displacement of three QR markers (one per storey), from a Basler acA2040-90um camera with a 75 mm lens at ~2.5 m.
• PASCO PS-3216 — contact accelerometers on the three storeys (100 Hz).

Eight structural states are created by redistributing added mass across the storeys (a mass-redistribution surrogate for dynamic-property change, not mechanical damage):

Label	0 damage	a	b	c	ab	ac	bc	abc
Added mass at storey	—	1	2	3	1,2	1,3	2,3	1,2,3

Each state has 5 records (~5 min each) per source.

Results

Both sources are windowed (W = 1024, non-overlapping), z-score normalized per window, and classified by three architectures. Test-set performance:

Data source	1D-CNN	CNN-LSTM	WaveNet
Camera displacement	97.26%	99.32%	100.00%
PASCO acceleration	100.00%	99.57%	100.00%

The small gap between the two sources indicates that the non-contact displacement data preserves the dynamic features needed for state classification in this experimental setting. See deep_learning/model_performance_summary.txt.

datasets/

datasets/
├── basler_camera_displacement/<state>/T{1..5}.xlsx   # 3-channel displacement (mm), columns c/b/a DX,DY
└── pasco_acceleration/<state>/T{1..5}.csv            # 3-channel acceleration (m/s^2), x/y/z per sensor

• <state> ∈ {0 damage, a, b, c, ab, ac, bc, abc}.
• Camera sampling rate is the effective acquisition rate (~63.6 fps); PASCO is 100 Hz. The training code windows each source independently.

deep_learning/

Grouped by data source, one folder per model:

deep_learning/<source>/{1d_cnn,cnn_lstm,wavenet}/
├── train_*.py                     # training entry point
├── run_info_*.json                # config, split, best epoch
├── scores_*.json                  # accuracy/precision/recall/F1 + confusion matrix
├── history_*.json                 # per-epoch metrics
├── learning_curve_*.png
└── confusion_matrix_*_test.png

Run a training job (PyTorch):

pip install torch numpy pandas scikit-learn matplotlib openpyxl tqdm
python deep_learning/basler_camera_displacement/wavenet/train_basler_wavenet.py

Each script auto-discovers the dataset under datasets/, converts the raw files to .npy, builds the windowed splits, trains, and writes the artefacts listed above. The split is per-record (3 train / 1 validation / 1 test per state) with a fixed seed, so runs are reproducible.

infatrackvision_app/

Desktop application (PyQt) for non-contact displacement measurement from a live camera, an RTSP stream, or a video file.

cd infatrackvision_app
pip install -r requirements.txt
python main.py

Typical workflow: open a source → auto-detect QR markers (ROI) → track across frames with template matching + quadratic sub-pixel refinement → calibrate pixel→mm from the known marker size → export displacement time histories.

RTSP note: config/rtsp_config.txt holds a placeholder URL. Replace <username>, <password>, and <camera-ip> with your own device settings. Never commit real credentials.

Requirements

Python 3.10+. The app dependencies are pinned in infatrackvision_app/requirements.txt; the training scripts additionally need torch.

License

Released for academic and research use. If you build on this work, please cite the accompanying paper.