Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
5fd4f23
feat: run Just-Bird once per recording in a browser worker
El1as42 Oct 23, 2026
487ba57
feat: zoomable spectrogram viewer for the browser demo
El1as42 Oct 23, 2026
7344146
feat: audio input for the browser demo
El1as42 Oct 23, 2026
0ede6f5
feat: Demo tab with the browser demo across the full page
El1as42 Oct 23, 2026
1d1f006
feat: PCEN spectrogram in the Demo tab
El1as42 Oct 23, 2026
1e06b29
docs: point the ONNX demo page to the Demo tab
El1as42 Oct 23, 2026
bbe4eee
test: parity of the Demo tab's fusion with the ONNX graph
El1as42 Oct 23, 2026
716d6ca
chore: drop an unused import in the demo
El1as42 Oct 23, 2026
49c414c
feat: Demo tab starts muted, with a volume fader
El1as42 Oct 26, 2026
df33f03
fix: clicking a box in the Demo tab no longer scrolls the page
El1as42 Oct 26, 2026
a5b70ac
feat: the demos use the BirdNET website's sample soundscape
El1as42 Oct 26, 2026
d01ab30
feat: the Demo tab's help panel starts closed
El1as42 Oct 26, 2026
4fbdb15
fix: sound in the Demo tab was too quiet to hear after unmuting
El1as42 Oct 26, 2026
3b9ad52
fix: play quiet recordings louder in the Demo tab
El1as42 Oct 26, 2026
8e23b1d
docs: replace the ONNX demo page with the Demo tab
El1as42 Oct 26, 2026
47afd7d
fix: N and P keep the selected box's row on screen
El1as42 Oct 26, 2026
a674c00
chore: remove the Streamlit app
El1as42 Oct 26, 2026
da3aef4
docs: move Installation into the Home tab
El1as42 Oct 26, 2026
57f8104
docs: README shows the Demo screenshot in the reader's color scheme
El1as42 Oct 26, 2026
82f9362
docs: citation names both licenses
El1as42 Oct 26, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ parity_results*/
*.engine
!docs/models/*.onnx

# currently online streamlit app
# local deployment files, not part of the repository
online/

# mkdocs
Expand Down
4 changes: 0 additions & 4 deletions .streamlit/config.toml

This file was deleted.

25 changes: 13 additions & 12 deletions NOTICE.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ authoritative.
treats models trained or fine-tuned with it as AGPL-3.0, so the BirdBox model
weights are AGPL-3.0 in every format (`.pt`, `.onnx`, `.tflite`, `.engine`).
Running BirdBox together with Ultralytics or the BirdBox weights, distributing
it in that form, or offering it to users over a network (for example the
Streamlit app) must therefore comply with AGPL-3.0. In particular, if you
it in that form, or offering it to users over a network must therefore
comply with AGPL-3.0. In particular, if you
modify that combined work and let users interact with it over a network, you
must offer them its corresponding source code (AGPL-3.0, section 13). The
weights stay AGPL-3.0 even when they run without the `ultralytics` package,
Expand All @@ -40,10 +40,11 @@ them.
|---|---|---|
| Everything authored by BirdNET-Team and contributors not listed below (`src/`, `tests/`, `taxonomies/*.py`, `install.py`, `run_*.sh`/`.ps1`/`.bat`, `docs/`, configuration) | MIT | Source files carry `SPDX-License-Identifier: MIT` headers. |
| `mel_filterbank()` in `src/inference/onnx_export.py` | ISC | Adapted from `librosa.filters.mel` (librosa 0.11.0), marked with `SPDX-Snippet` tags. Rest of the file is MIT. Notice [below](#librosa). |
| Inferno colormap table in `docs/javascripts/onnx_spectrogram.js` | CC0-1.0 | 8-bit samples of the inferno colormap by Nathaniel J. Smith and Stéfan van der Walt, released under CC0 and shipped with matplotlib. |
| Inferno, magma and viridis colormap tables in `docs/javascripts/demo/colormaps.js` | CC0-1.0 | 8-bit samples of the matplotlib colormaps by Nathaniel J. Smith and Stéfan van der Walt, released under CC0. |
| `docs/models/Just-Bird_fp16.onnx` | AGPL-3.0 | YOLO11 weights trained with Ultralytics, exported to ONNX. |
| Model weights distributed outside this repository ([TUC-Cloud](https://tuc.cloud/index.php/s/ET4KE4LdSaysSSL), default model of the Streamlit app) | AGPL-3.0 | YOLO11 weights trained with Ultralytics. |
| `tests/test.wav`, `docs/audio/sample_15s.wav`, and the spectrogram images `docs/img/example_viz.png`, `docs/img/getting-started/streamlit_ui_screenshot.png`, `docs/img/getting-started/detect_and_merge_transparent.png` | CC-BY-4.0 | Soundscape `UHH_001_S01_20161121_150000` from the BirdNET Hawaii soundscape collection, and spectrograms of recordings from that collection. Attribution [below](#audio-samples). |
| Model weights distributed outside this repository ([TUC-Cloud](https://tuc.cloud/index.php/s/ET4KE4LdSaysSSL)) | AGPL-3.0 | YOLO11 weights trained with Ultralytics. |
| `tests/test.wav` and the spectrogram images `docs/img/example_viz.png`, `docs/img/how-it-works/detect_and_merge_transparent.png` | CC-BY-4.0 | Soundscape `UHH_001_S01_20161121_150000` from the BirdNET Hawaii soundscape collection, and spectrograms of recordings from that collection. Attribution [below](#audio-samples). |
| `docs/audio/BirdNET_soundscape_32kHz.flac` and the screenshots `docs/img/demo_screenshot_light.png`, `docs/img/demo_screenshot_dark.png` | MIT | Example soundscape `birdnet_analyzer/example/soundscape.wav` of [BirdNET-Analyzer](https://github.com/birdnet-team/BirdNET-Analyzer) (Copyright (c) 2024 birdnet-team, MIT License), resampled from 48 kHz to 32 kHz. The same file is the sample of the BirdNET website demo. The screenshots show it in the Demo tab. |
| `taxonomies/Cornell-to-AviList-mapping.json` | CC-BY-4.0 | Data derived from the AviList v2025 checklist. Attribution [below](#avilist). |
| Soundscapes fetched by `tests/parity_audio/download.py` | Not MIT | Not part of the repository. xeno-canto recordings under the Creative Commons licenses listed in `tests/parity_audio/recordings.csv`. |

Expand All @@ -53,12 +54,13 @@ BirdBox installs or loads third-party packages that are not part of this
repository and keep their own licenses: the Python packages in
`requirements.txt`, `model_format_requirements/` and `install.py` (notably
`ultralytics`, AGPL-3.0), and the scripts the documentation site loads from
CDNs (see `extra_javascript` in `mkdocs.yml`).
CDNs (see `extra_javascript` in `mkdocs.yml`, and ONNX Runtime Web in
`docs/javascripts/demo/ort-worker.js` for the Demo tab).

## Audio samples

`tests/test.wav`, `docs/audio/sample_15s.wav` and the spectrogram images listed
above are taken from or made from:
`tests/test.wav` and the spectrogram images listed above are taken from or made
from:

> Amanda Navine, Stefan Kahl, Ann Tanimoto-Johnson, Holger Klinck, Patrick Hart
> (2022). *A collection of fully-annotated soundscape recordings from the Island
Expand All @@ -69,10 +71,9 @@ Licensed under [Creative Commons Attribution 4.0 International (CC BY 4.0)](http

Changes: `tests/test.wav` is recording `UHH_001_S01_20161121_150000`, converted
from FLAC to 16-bit mono WAV at 32 kHz. The samples are identical to the original.
`docs/audio/sample_15s.wav` is the first 15 seconds of `tests/test.wav`.
`docs/img/example_viz.png` and `docs/img/getting-started/streamlit_ui_screenshot.png`
show spectrograms of `tests/test.wav` with BirdBox detections drawn on top.
`docs/img/getting-started/detect_and_merge_transparent.png` shows spectrogram
`docs/img/example_viz.png` shows a spectrogram of `tests/test.wav` with BirdBox
detections drawn on top.
`docs/img/how-it-works/detect_and_merge_transparent.png` shows spectrogram
excerpts of a recording from the collection, annotated to illustrate the
detect-and-merge policy.

Expand Down
16 changes: 9 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,18 +21,20 @@ Everything you need, including installation instructions, an interactive demo, m

## Quick Links

- [Installation](https://birdnet-team.github.io/BirdBox/getting-started/installation/) - set up the environment
- [Installation](https://birdnet-team.github.io/BirdBox/installation/) - set up the environment
- [Data Flow](https://birdnet-team.github.io/BirdBox/data/fileflows/) - pipeline description
- [Models and Metrics](https://birdnet-team.github.io/BirdBox/models-and-metrics/overview/) - list of models with corresponding metrics
- [CLI Reference](https://birdnet-team.github.io/BirdBox/cli/workflows/) - command line interface
- [API Reference](https://birdnet-team.github.io/BirdBox/api/api-type/) - application programming interface

## Interactive Demo
## Demo

Try out the [Interactive Demo](https://birdnet.cornell.edu/birdbox/) or browse the [Demo Documentation](https://birdnet-team.github.io/BirdBox/getting-started/demo_streamlit/).
If everything is working as expected, the web interface will look like this:
Try the [Demo](https://birdnet-team.github.io/BirdBox/demo/) in your browser: it runs the Just-Bird model on your device, with no installation and no upload. Load a recording, tune the confidence and phrase gap, review and edit the boxes, and export them in the BirdBox output formats.

![Streamlit app screenshot](docs/img/getting-started/streamlit_ui_screenshot.png)
<picture>
<source media="(prefers-color-scheme: dark)" srcset="docs/img/demo_screenshot_dark.png">
<img src="docs/img/demo_screenshot_light.png" alt="The BirdBox Demo with the sample soundscape">
</picture>

## Citation

Expand All @@ -41,7 +43,7 @@ Feel free to use BirdBox for your acoustic analyses and research. If you do, ple
```bibtex
@software{Schlosser_BirdBox,
author = {Schlosser, Elias and Kahl, Stefan and Eibl, Maximilian},
license = {MIT},
license = {MIT (code), AGPL-3.0 (project as a whole)},
title = {{BirdBox}},
url = {https://github.com/birdnet-team/BirdBox}
}
Expand All @@ -53,7 +55,7 @@ BirdBox's own source code is licensed under the [MIT License](LICENSE).

BirdBox is built on [Ultralytics YOLO11](https://github.com/ultralytics/ultralytics), which is licensed under the [AGPL-3.0](LICENSE-AGPL-3.0). Ultralytics also treats models trained or fine-tuned with it as AGPL-3.0, so the BirdBox model weights (`.pt` and every exported format) are AGPL-3.0 as well. As a result, the project as a whole is subject to the AGPL-3.0:

- **Using BirdBox with Ultralytics or the BirdBox models** (running, redistributing, or offering it over a network, such as the Streamlit app) has to comply with the AGPL-3.0. If you modify it and let users interact with it over a network, you must offer them its source code.
- **Using BirdBox with Ultralytics or the BirdBox models** (running, redistributing, or offering it over a network) has to comply with the AGPL-3.0. If you modify it and let users interact with it over a network, you must offer them its source code.
- **Reusing BirdBox's own code** on its own, in a project that uses neither Ultralytics nor the BirdBox models, is possible under the MIT terms.

A few files are under other licenses (a function adapted from librosa, xeno-canto audio samples, AviList-derived taxonomy data). See [NOTICE.md](NOTICE.md) for the full breakdown.
Expand Down
4 changes: 2 additions & 2 deletions docs/api/api-type.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ BirdBox is a **Python library**. There is no server, no HTTP endpoint, and no RE
from inference.detect_birds import BirdCallDetector # ← local Python import
```

If you are looking for a point-and-click interface or want to run detection without writing code, use the [CLI Reference](../cli/workflows.md) or the [Streamlit demo](../getting-started/demo_streamlit.md) instead.
If you are looking for a point-and-click interface or want to run detection without writing code, use the [CLI Reference](../cli/workflows.md) or the [Demo](../demo.md) instead.

---

Expand All @@ -31,4 +31,4 @@ sys.path.insert(0, "src") # only needed when running outside the installed pac
from inference.detect_birds import BirdCallDetector
```

See [Installation](../getting-started/installation.md) for the full setup instructions.
See [Installation](../installation.md) for the full setup instructions.
Binary file added docs/audio/BirdNET_soundscape_32kHz.flac
Binary file not shown.
Binary file removed docs/audio/sample_15s.wav
Binary file not shown.
2 changes: 1 addition & 1 deletion docs/cli/detect-birds.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ Detect bird calls in arbitrary-length audio files using a trained YOLO model. Pr
| `--verbose` | flag / off | No | Print per-file processing details, clip-level progress bars, and a detection summary. Default is a single file-level progress bar. |

!!! danger "Environment must match the model format"
Each model format (`.pt`, `.onnx`, `.tflite`, `.engine`) requires a different Python environment. Using the wrong environment raises an import error or silently degrades results. Run `python install.py --model-format <FORMAT>` once for each format you want to use. See [Install Parameters](../getting-started/installation.md#install-parameters) for the full table.
Each model format (`.pt`, `.onnx`, `.tflite`, `.engine`) requires a different Python environment. Using the wrong environment raises an import error or silently degrades results. Run `python install.py --model-format <FORMAT>` once for each format you want to use. See [Install Parameters](../installation.md#install-parameters) for the full table.

## Parameter Deep-Dives

Expand Down
4 changes: 2 additions & 2 deletions docs/data/audio-and-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ Pretrained releases on [TUC-Cloud](https://tuc.cloud/index.php/s/ET4KE4LdSaysSSL
Each format requires its own Python environment. Install the correct runtime before switching formats.

!!! danger "Environment must match the model format"
Loading a `.tflite`, `.onnx`, or `.engine` model from a `.pt` environment will fail with an import error or silently produce incorrect results. Run `python install.py --model-format <FORMAT>` to set up the right environment. See [Install Parameters](../getting-started/installation.md#install-parameters) for the full format-to-runtime table.
Loading a `.tflite`, `.onnx`, or `.engine` model from a `.pt` environment will fail with an import error or silently produce incorrect results. Run `python install.py --model-format <FORMAT>` to set up the right environment. See [Install Parameters](../installation.md#install-parameters) for the full format-to-runtime table.

!!! info "Format parity"
Detection quality is validated across all formats by running inference on the same audio file and comparing results against the `.pt` baseline. See [Just-Bird format parity](../models-and-metrics/just-bird-model-types.md) and [All-In-One format parity](../models-and-metrics/all-in-one-model-types.md) for the current scores.
Expand All @@ -59,7 +59,7 @@ Each format requires its own Python environment. Install the correct runtime bef
## Species mapping (`--species-mapping`)

The mapping name must match the label space the model was trained on. It is **not** inferred from the weights filename. You pass it explicitly.
For specific mappings see [Species Mapping](https://github.com/birdnet-team/BirdBox/blob/main/src/config.py#L19){ target="_blank" rel="noopener noreferrer" }.
For specific mappings see [Species Mapping](https://github.com/birdnet-team/BirdBox/blob/main/src/config.py#L17){ target="_blank" rel="noopener noreferrer" }.

Allowed values for the species-mapping parameter:

Expand Down
96 changes: 96 additions & 0 deletions docs/demo.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
---
title: Demo
template: demo.html
hide:
- navigation
- toc
- footer
---

## About this demo

The demo runs the self-contained Just-Bird graph `docs/models/Just-Bird_fp16.onnx` with [ONNX Runtime Web](https://onnxruntime.ai/docs/tutorials/web/){ target="_blank" rel="noopener noreferrer" } in your browser. The graph computes the PCEN spectrogram, runs the YOLO detector on 3 s windows every 1.5 s and applies NMS, just like [`detect_birds.py`](cli/detect-birds.md) with the Just-Bird model. There are no species labels: every box is a bird vocalization (`bird`).

**Instant sliders.** The model runs once per recording, at the lowest confidence of the slider (0.05). The demo asks the graph for its per-window boxes (`raw_all`) and repeats the graph's window fusion in JavaScript: window ownership and weighted box fusion, joining calls into phrases, removing nested boxes. NMS only lets higher-scoring boxes suppress lower ones, so the boxes for any confidence and phrase gap are the same as a graph run with those values.

**Spectrogram.** The viewer shows an STFT spectrogram by default. In its display settings, *PCEN* switches to the PCEN mel spectrogram BirdBox detects on.

**Review.** Boxes can be verified, moved, resized, removed and drawn. The exports use the BirdBox [output formats](data/outputs.md): the JSON of `--output-format json-with-algorithm-metadata`, the simplified CSV and the Raven selection table, plus a CSV with every column of the table.

!!! warning "Browser limits"
- The model is downloaded once (about 35 MB) and then comes from the browser cache.
- Audio is decoded locally, resampled to 32 kHz mono and truncated to **10 minutes**.
- The browser's resampler is not librosa's, so files that are not recorded at 32 kHz can give slightly different boxes than the CLI.
- ONNX Runtime runs single-threaded here: about 10 s for 2 minutes of audio on a laptop. Safari can be slower than Chromium or Firefox.
- Use the [CLI](cli/detect-birds.md) for long files and batch jobs.

The sample soundscape is the example recording of [BirdNET-Analyzer](https://github.com/birdnet-team/BirdNET-Analyzer){ target="_blank" rel="noopener noreferrer" } (`birdnet_analyzer/example/soundscape.wav`, MIT License), resampled to 32 kHz. The [BirdNET website demo](https://birdnet.cornell.edu/demo/){ target="_blank" rel="noopener noreferrer" } uses the same file. Recordings loaded by number come from [Xeno-canto](https://xeno-canto.org){ target="_blank" rel="noopener noreferrer" } under the license of each recording.

## Adding models

The demo runs the first model in `docs/models/manifest.json` with `"merge_method": "fusion"`. Only window-fusion exports work: the instant sliders need the graph's per-window boxes (`raw_all`) and the JavaScript port of its window fusion (`docs/javascripts/demo/fusion.js`). The export uses window fusion for Just-Bird, Hawaii and Southern-Sierra-Nevada (`--merge auto`); `--merge fusion` forces it for the other models.

1. Export the model.

=== "Linux / macOS"
```bash
python src/inference/onnx_export.py \
--output-path docs/models/Just-Bird_fp16.onnx \
--precision fp16
```

=== "Windows (PowerShell)"
```powershell
python src/inference/onnx_export.py `
--output-path docs/models/Just-Bird_fp16.onnx `
--precision fp16
```

=== "Windows (CMD)"
```cmd
python src/inference/onnx_export.py ^
--output-path docs/models/Just-Bird_fp16.onnx ^
--precision fp16
```

`--precision fp16` stores the network weights in fp16, which makes the file about a third smaller (35 MB instead of 53 MB for Just-Bird). Feature extraction and merging stay in fp32. On 13 test recordings, 99.4% of the fp32 graph's boxes were reproduced (mean IoU 0.996). The export prints the merge it baked in (`Merge (baked in)`) and the suggested defaults (`Default conf`, `Default song_gap`), which it also stores in the ONNX metadata.

2. Place the `.onnx` file under `docs/models/` if the export path was elsewhere.

3. Add an entry to `docs/models/manifest.json`, first in the list if the demo should run it.

```json
{
"models": [
{
"file": "Just-Bird_fp16.onnx",
"label": "Just-Bird_fp16.onnx",
"merge_method": "fusion",
"conf": 0.15,
"phrase_gap": 0.05,
"species_mapping": "Just-Bird",
"names": { "0": "bird" }
}
]
}
```

| Field | Meaning |
| :--- | :--- |
| `file` | The ONNX file, relative to `docs/models/` |
| `label` | Name shown at the top of the demo |
| `merge_method` | `fusion`, as printed by the export. Other entries are skipped. |
| `conf` | Where the confidence slider starts: the printed `Default conf` (0.15 for Just-Bird, 0.2 for the other models) |
| `phrase_gap` | Where the phrase gap slider starts: the printed `Default song_gap` (0.05 for Just-Bird, 0 for Hawaii and Southern-Sierra-Nevada) |
| `species_mapping` | Written to `model_config` of the exported JSON |
| `names` | Class labels for the exports, by class index (ONNX Runtime Web does not expose ONNX metadata) |

4. Check that the demo's fusion still matches the graph:

```bash
cd tests/js && npm install
node demo_fusion_parity.mjs ../../docs/models/Just-Bird_fp16.onnx
```

The script compares `fusion.js` with the graph's own detections for 16 slider settings on `tests/test.wav` and fails on any difference. Run it after exporting a new model or changing the fusion in `src/inference/onnx_export.py`.

Loading
Loading