Skip to content
View Victarry's full-sized avatar
168 followers · 134 following

Achievements

Achievement: Pair Extraordinairex4Achievement: QuickdrawAchievement: YOLOAchievement: Starstruckx2Achievement: Arctic Code Vault ContributorAchievement: Pull Sharkx2

Achievements

Achievement: Pair Extraordinairex4Achievement: QuickdrawAchievement: YOLOAchievement: Starstruckx2Achievement: Arctic Code Vault ContributorAchievement: Pull Sharkx2

Block or report Victarry

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
Victarry/README.md

Victarry

Dennis (Zhenhuan) Liu — Megatron-Core, NVIDIA.
MoE at multi-thousand-GPU scale: expert parallelism, communication/computation overlap, pipeline scheduling.

github.com/Victarry

Projects

Repo Focus
PP-Schedule-Visualization Interactive visualizer for pipeline-parallel schedules viz
PyTorch-Memory-Profiler Tensor-level GPU memory allocation profiler profiling
Megatron-LM Large-scale transformer training and Mixture-of-Experts moe
Image-Generation-models From-scratch studies of generative image models research

Lab log

Victarry's recent merged pull requests, reviews, releases, and commits

Recent merged PRs, reviews, releases, and commits — sourced from GitHub and cached at the edge.

Pinned Loading

  1. NVIDIA/Megatron-LM NVIDIA/Megatron-LM Public

    Ongoing research training transformer models at scale

    Python 17.1k 4.2k

  2. NVIDIA/TransformerEngine NVIDIA/TransformerEngine Public

    A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide better performance…

    Python 3.4k 775

  3. PyTorch-Memory-Profiler PyTorch-Memory-Profiler Public

    Python 47 1

  4. Image-Generation-models Image-Generation-models Public

    Generative models (GAN, VAE, Diffusion Models, Autoregressive Models) implemented with Pytorch, Pytorch_lightning and hydra.

    Python 61 9

  5. PP-Schedule-Visualization PP-Schedule-Visualization Public

    Pipeline Parallelism Emulation and Visualization

    Python 85 9

  6. Megatron-MoE-Echo Megatron-MoE-Echo Public

    Python 11 3