Skip to content

feat: Demo tab can switch to All-In-One, smaller ONNX graphs - #33

Merged
El1as42 merged 3 commits into
mainfrom
feat/demo
Oct 27, 2026
Merged

El1as42 merged 3 commits into
mainfrom
feat/demo

Conversation

@El1as42

@El1as42 El1as42 commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Model menu in the Demo tab: Just-Bird stays the default; All-In-One (282 classes) can be picked from the menu at the top. With All-In-One the table names the species of each box.
  • Smaller ONNX graphs: onnx_export.py no longer stores the 15.7 MB STFT kernel; the graph rebuilds it when the model loads, bit-identical. Just-Bird_fp16.onnx shrinks from 35 MB to 20 MB, the new All-In-One_fp16.onnx is 51 MB instead of 66 MB.
  • No species badge: the species-282 badge is gone from the README and the docs landing page. The recommended workflow combines BirdBox boxes with BirdNET for labels, so BirdBox's species count should not be up front. The landing page's Python badge now links to 3.12.13 like the README.

Model menu

  • The menu lists docs/models/manifest.json; the first entry is the default. Switching runs detection again with the model's suggested slider values. Drawn boxes stay; verified, moved or removed detected boxes are reset, with a notice.
  • All-In-One merges by song reconstruction, not window fusion. songs.js ports reconstruct_songs_tensor, so the sliders stay instant; the second slider becomes Song gap (default 0.1 s).
  • With more than one class, a Species column (common name and eBird code) and the box tooltip name the species. The manifest gains song_gap and common_names (from src/config.py).
  • The JSON export records the model's merge method and gap (phrase_gap for fusion, song_gap_threshold for reconstruct, as detect_birds.py writes them).
  • The page title reads "BirdBox in your browser" now that the model can change.

Rebuilding the STFT kernel at load

  • The kernel's phase at bin k and sample n only depends on (k * n) % 2048. The file stores one period of cosine and sine values plus the window (about 50 KB); dft_kernel_builder adds nodes that do the lookup and products in float64 and cast to float32, as dft_convolution_kernel does. The loop body reads the kernel as an outer-scope value.
  • The export checks in numpy that the lookup reproduces the kernel exactly and stops otherwise.
  • Every export with onnx_export.py uses it, for every model and precision. fp32 graphs shrink too (Just-Bird 53 → 38 MB, All-In-One 114 → 99 MB).

Results

Graph (13 recordings, tests/onnx_graph_parity.py) Boxes of detect_birds.py with the .pt model matched Mean IoU
Just-Bird_fp16.onnx 99.8% 0.995
All-In-One_fp16.onnx 99.7%, same species (1066 of 1069) 0.997
All-In-One_fp32.onnx (reference) 100% 1.000

Test plan

  • Old and new graphs give bit-identical detections and raw_all on tests/test.wav, in ONNX Runtime (Python) and ONNX Runtime Web 1.20.1 (WASM); same load time
  • tests/onnx_graph_parity.py for both demo models (table above)
  • tests/js/demo_merge_parity.mjs (renamed from demo_fusion_parity.mjs, checks fusion.js or songs.js by the manifest entry): 16/16 settings identical for Just-Bird and for All-In-One
  • Headless Chromium, built site: Just-Bird → All-In-One → Just-Bird on the sample (55, 29, 55 boxes), species column, song gap slider, JSON export, phone width, dark mode; no console errors
  • check_license_headers.py passes
  • Real browsers on macOS / iOS with All-In-One (about 3 times slower than Just-Bird: 31 s for the 2-minute sample in headless Chromium)

docs/models/All-In-One_fp16.onnx adds 51 MB to the repository.

🤖 Generated with Claude Code

El1as42 and others added 3 commits October 27, 2026 09:33
The species-282 badge put BirdBox's species labels up front, although the
recommended workflow combines BirdBox boxes with BirdNET for the labels. It
is gone from the README and the docs landing page. The landing page's Python
badge now links to the 3.12.13 release like the README and the installation
page.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The STFT kernel (1920 x 1 x 2048 float32, 15.7 MB) was the largest constant
of every exported graph, almost half of Just-Bird_fp16.onnx. Its phases only
depend on (bin * sample) % n_fft, so the file now stores one period of cosine
and sine values plus the window, and nodes in the parent graph rebuild the
kernel in float64 and cast it to float32, as dft_convolution_kernel does. The
loop body reads it as an outer-scope value. The export checks in numpy that
the lookup reproduces the kernel exactly.

The re-exported Just-Bird_fp16.onnx shrinks from 35 MB to 20 MB. Its
detections and raw_all are bit-identical to the previous file in ONNX Runtime
(Python) and ONNX Runtime Web 1.20.1 (WASM), with the same load time.
tests/onnx_graph_parity.py: 99.8% of the detect_birds.py boxes, mean IoU
0.995.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
A model menu at the top of the Demo lists docs/models/manifest.json.
Just-Bird_fp16.onnx stays the default; All-In-One_fp16.onnx (51 MB, exported
from models/All-In-One.pt) is the second entry. Switching runs the detection
again with the new model and its suggested slider values; the person's drawn
boxes stay, changes to detected boxes are reset.

All-In-One merges by song reconstruction, not window fusion. songs.js ports
reconstruct_songs_tensor so its sliders stay instant, and the second slider
becomes the song gap (default 0.1 s). With more than one class, the table and
the tooltip name the species (common names from src/config.py in the
manifest), and the JSON export records the model's merge and gap.

tests/js/demo_fusion_parity.mjs becomes demo_merge_parity.mjs and checks
fusion.js or songs.js, depending on the model's manifest entry: 16 of 16
settings identical to the graph for both models. tests/onnx_graph_parity.py:
the fp16 graph matches 99.7% of the All-In-One.pt boxes with the same
species, mean IoU 0.997.

The title reads "BirdBox in your browser" now that the model can change.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@El1as42
El1as42 merged commit 60469c3 into main Oct 27, 2026
1 check passed
@El1as42
El1as42 deleted the feat/demo branch October 27, 2026 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant