Add canonical component manifests - #665
Conversation
Performance Comparison
|
There was a problem hiding this comment.
Pull request overview
This PR introduces a canonical, immutable “component manifest” layer that centralizes per-component metadata (name, module path, optimization role, and HF source-path ownership). It integrates this manifest into component inspection and build-time optimization role selection while preserving existing graph/weight behavior.
Changes:
- Added
ComponentDescriptor/ComponentManifestplusresolve_component_manifest()andget_hf_component_sources()as the canonical component-metadata API. - Added
ModelTask.component_manifest()and updatedinspect_components()to source roles + HF paths through the manifest. - Switched build-time optimization role selection in
build_from_module()to consult the manifest.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/mobius/tasks/_base.py | Adds ModelTask.component_manifest() as a task-level entrypoint to resolve canonical component metadata. |
| src/mobius/_inspect.py | Routes HF component source resolution + inspect_components() output through the new manifest API. |
| src/mobius/_component_manifest.py | Introduces immutable component descriptor/manifest types and the canonical resolution helpers. |
| src/mobius/_component_manifest_test.py | Adds unit tests for manifest resolution, dynamic HF source resolvers, and duplicate-name validation. |
| src/mobius/_builder.py | Uses the manifest for selecting optimization roles per package entry during build-time optimization. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
🏗️ Architecture Diff
No architecture changes detected. ✅ Legend: ⚪ No change · 🔵 Minor (attrs/inits) · 🟡 Moderate (nodes added/removed) · 🔴 Major (interface changed) |
b549476 to
501c342
Compare
Resolve task roles, module paths, and HuggingFace source ownership into one immutable manifest. Switch inspection and optimization-role lookup to the shared metadata without changing graph or weight behavior. Signed-off-by: Xiaoyu Zhang <[email protected]>
Derive candidate HuggingFace module names from component-local paths so later loader stages can apply exact and regex quantization exclusions consistently. Signed-off-by: Xiaoyu Zhang <[email protected]>
Only derive HuggingFace module candidates when a declared source root shares an anchor with the component-local path, preventing unrelated roots such as lm_head from matching every decoder module. Signed-off-by: Xiaoyu Zhang <[email protected]>
Allow model declarations to map component-local module prefixes to HuggingFace source prefixes when structural anchor inference is insufficient. Signed-off-by: Xiaoyu Zhang <[email protected]>
Rename module_path to module_attribute_path and document task-defined optimization roles so callers cannot confuse Python module ownership with package or checkpoint prefixes. Signed-off-by: Xiaoyu Zhang <[email protected]>
501c342 to
1f47daa
Compare
justinchuby
left a comment
There was a problem hiding this comment.
Is there a test to ensure all new models will have required information/manifest?
|
@copilot resolve the merge conflicts in this pull request |
Resolve the builder conflict by preserving canonical manifest role lookup while integrating per-component quantization setup from main. Signed-off-by: GitHub <[email protected]> Co-authored-by: xiaoyu-work <[email protected]>
Resolved and pushed in |
|
|
Summary
ComponentDescriptorandComponentManifesttypesHF_COMPONENT_SOURCESthrough one canonical APIinspect_componentsand build-time optimization role selection to the manifestComponentSpecand graph/weight behaviorStack
Validation