diff --git a/.gitignore b/.gitignore index 7577abc..b6aa379 100644 --- a/.gitignore +++ b/.gitignore @@ -47,7 +47,6 @@ __pycache__/ .venv/ node_modules/ ml/dataset/ -ml/.venv/ runs/ backend/data/* !backend/data/seed_components.json diff --git a/1st.jpg b/1st.jpg deleted file mode 100644 index ecd00ba..0000000 Binary files a/1st.jpg and /dev/null differ diff --git a/2nd.jpg b/2nd.jpg deleted file mode 100644 index e1fb908..0000000 Binary files a/2nd.jpg and /dev/null differ diff --git a/3rd.jpg b/3rd.jpg deleted file mode 100644 index 2976906..0000000 Binary files a/3rd.jpg and /dev/null differ diff --git a/4th.jpg b/4th.jpg deleted file mode 100644 index bda1cf2..0000000 Binary files a/4th.jpg and /dev/null differ diff --git a/README.md b/README.md index c64ce1b..3a86704 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ [![TypeScript](https://img.shields.io/badge/TypeScript-5.9-blue.svg)](https://www.typescriptlang.org/) [![React](https://img.shields.io/badge/React-19-cyan.svg)](https://react.dev/) [![ReactFlow](https://img.shields.io/badge/ReactFlow-11-purple.svg)](https://reactflow.dev/) -[![Python](https://img.shields.io/badge/Python-3.14-yellow.svg)](https://www.python.org/) +[![Python](https://img.shields.io/badge/Python-3.11+-yellow.svg)](https://www.python.org/) [![YOLOv8](https://img.shields.io/badge/YOLOv8-Ultralytics-000000.svg)](https://docs.ultralytics.com/) **DigiSim** is an interactive, browser-based digital logic circuit simulator that combines **manual drag-and-drop circuit creation** with **automated ML-based image-to-circuit detection**. @@ -43,7 +43,7 @@ should use the seeded cells or package their own. - **🔍 ⌘K Component Search**: Instant search and filtering across the entire component palette. - **⚙️ Real-time Simulation Engine**: Relaxation-based digital gate evaluation, a switch-level CMOS solver (NMOS/PMOS + supplies resolve to correct truth tables), and Modified Nodal Analysis (MNA) solver for analog nodes with live signal propagation (HIGH/LOW/Z/X). - **📸 ML Image-to-Circuit Detection**: - - **YOLOv8 Schematic Detection**: Detects drawn logic gates and uses OpenCV line transforms (Hough) to extract wire paths into a NetworkX graph. + - **YOLOv8 Schematic Detection**: Detects schematic symbols and uses OpenCV line transforms (Hough) to extract wire paths into a NetworkX graph. See [Machine learning status](#-machine-learning-status) for what the detector currently recognises. - **DINOv2 Physical Board Analysis**: Physical board photo component recognition using DINOv2 embeddings and OCR. - **📄 Human & LLM-Parsable Netlist JSON Schema**: Export and import circuits using an intuitive, semantic JSON format (`input_A.out -> and_1.a`) with 100% loss-free round-trip reconstruction. @@ -57,7 +57,7 @@ should use the seeded cells or package their own. - **Styling**: Tailwind CSS v4, Glassmorphism, DaisyUI v5 ### Backend (Image Recognition & Pipeline) -- **Engine**: Python 3.14, Flask, `uv` +- **Engine**: Python 3.11+ (the Docker image uses 3.12), Flask, `uv` - **Machine Learning**: Ultralytics YOLOv8, PyTorch, DINOv2, OpenCV, NetworkX --- @@ -130,7 +130,69 @@ deploy steps and required secrets: [`docs/deploy-vercel-render.md`](docs/deploy- --- +## 🧠 Machine Learning Status + +The image-to-circuit pipeline is four strictly sequential stages: YOLOv8 detection +(`backend/pipeline/detector.py`), classical OpenCV wire extraction +(`wire_extractor.py`, no neural network), NetworkX graph building +(`graph_builder.py`) and DigiSim JSON export (`circuit_exporter.py`). Details: +[`agent_docs/ml-pipeline.md`](agent_docs/ml-pipeline.md). + +What is true today: + +- **The detector has not caught up with the transistor-first app.** The only + trained detector was built for the earlier gate-era vocabulary (12 classes: + AND, OR, NOT, NAND, NOR, XOR, XNOR, Switch, Input, Output, LED, Junction). + It does not recognise NMOS/PMOS transistors or supply symbols, and it detects + nothing useful on transistor-level schematics. +- **Retraining on transistor-level classes is in progress.** It needs a new + class schema, a synthetic transistor-level generator, real hand-drawn data + (see [Data and licences](#-data-and-licences)), a GPU training run and + net tracing that no longer assumes left-to-right gates. Until that lands, + treat detection results on transistor schematics as unsupported. +- **Training is transfer learning only** (pretrained COCO weights, never random + init), with the gate: F1 >= 0.95 and accuracy >= 0.95 per class before a phase + advances. Training and evaluation entry points are `ml/train.py` and + `ml/evaluate.py`. +- **Trained detector weights are not in git.** They are loaded from `MODEL_WEIGHTS_PATH` + (see `backend/.env.example`); without them the detection endpoints cannot + produce results. + +--- + +## 📜 Data and Licences + +| Item | Source | Licence | Notes | +|------|--------|---------|-------| +| `My First Project.yolov8/` (46 images, 6 gate classes) | Roboflow Universe, workspace `cirzer`, project `my-first-project-yz9wf`, exported 2026-06-15 | CC BY 4.0 (annotations) | Attribution required: "cirzer / my-first-project-yz9wf" via https://universe.roboflow.com/cirzer/my-first-project-yz9wf. The licence covers the annotations; some source images appear to be figures from the web and may carry their own licences, so do not redistribute the images as a public dataset. | +| Synthetic training images | Generated by `backend/data_gen/` | Project's own (intended MIT; no `LICENSE` file yet) | Produced by code in this repository; no third-party imagery. | +| CGHD (hand-drawn circuit diagrams, DFKI) | Zenodo record 15333233 (planned, not vendored) | CC BY 4.0 per the Zenodo record | Planned source of real hand-drawn transistor-level data. Not in this repository; keep attribution when used. The dataset's own repo and mirrors state conflicting licences, so follow the Zenodo terms. | + +Tracked in git today: the Roboflow export above, both as the extracted folder +`My First Project.yolov8/` and as `My First Project.yolov8.zip`, and +`Sampleimages/` (51 files) whose provenance and licence are not documented +yet. Other datasets and generated data are not committed; weights, large +datasets and training runs stay out of git (see `.gitignore`). + +### Third-party model licences + +- **Ultralytics YOLOv8 / YOLO11 is licensed under AGPL-3.0.** The detector + depends on the `ultralytics` package, and weights trained or fine-tuned with + it are generally treated as derivative of it. If you run a modified + DigiSim as a network service or redistribute it together with such weights, + AGPL-3.0 obligations (offering the corresponding source to users) can apply, + and they differ from this repository's intended MIT licence. Ultralytics sells an + Enterprise licence as the alternative. Review this before any commercial or + closed deployment; this note is not legal advice. +- **DINOv2** (physical board analysis) is distributed by Meta under Apache-2.0 + and is loaded through `torch.hub` at runtime. + +--- + ## 📄 License -Distributed under the MIT License. See `LICENSE` for details. +DigiSim's own code is intended to be MIT-licensed, but no `LICENSE` file is in +the repository yet, so no licence grant is in place until one is added. +Third-party models, datasets and their different terms are listed in +[Data and Licences](#-data-and-licences).