Skip to content

docs: add Halo fine-tuning guide for LFM MoE - #131

Merged
iamleonie merged 3 commits into
Liquid4All:mainfrom
advpropsys:docs/halo-lfm-fine-tuning
Nov 10, 2026
Merged

iamleonie merged 3 commits into
Liquid4All:mainfrom
advpropsys:docs/halo-lfm-fine-tuning

Conversation

@advpropsys

@advpropsys advpropsys commented Sep 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds lfm/fine-tuning/halo.mdx, a guide to training LFM MoE models with Halo, White Circle's open-source distributed training framework. Halo adds multi-GPU and multi-node parallelism to the native Hugging Face workflow while models remain standard transformers modules and checkpoints remain loadable with from_pretrained.

The guide:

  • explains how one LFM integration and YAML structure carry across SFT, LoRA, GRPO, and environmental GRPO
  • documents FSDP, EP, TP, ETP, and EP+TP support for LFM MoE models
  • includes the published single-B300 LFM2.5 comparison: 31,262 vs 28,330 tok/s/GPU at 8K and 35,517 vs 29,520 at 16K, against Axolotl
  • provides a runnable EP2 SFT path for LFM2.5-8B-A1B
  • links to the complete Halo LFM2 cookbook for larger models, LoRA, inference, and RL
  • adds Halo to the fine-tuning overview and navigation

Validation

  • npm run snapshot:check
  • verified the external Halo, White Circle, and model links

@advpropsys
advpropsys force-pushed the docs/halo-lfm-fine-tuning branch from a791c97 to afc85c0 Compare September 20, 2026 14:03
@advpropsys
advpropsys force-pushed the docs/halo-lfm-fine-tuning branch from afc85c0 to 546b2e1 Compare September 20, 2026 14:13
@iamleonie
iamleonie requested a review from mlabonne as a code owner September 21, 2026 09:39
…ning

# Conflicts:
#	docs.json
#	guides/migration-guide.mdx
#	lfm/help/faqs.mdx
#	lfm/models/complete-library.mdx
@mintlify

mintlify Bot commented Nov 10, 2026

Copy link
Copy Markdown

Preview approval required

This pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit 8fe165a.

Approve preview

@iamleonie
iamleonie merged commit 435db85 into Liquid4All:main Nov 10, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants