This repository was archived by the owner on Jan 21, 2025. It is now read-only.

This repository was archived by the owner on Jan 21, 2025. It is now read-only.

mask_1_flat and mask_2_flat applied to gates twice? #378

Open

opened

on Mar 31, 2022

mesh/mesh_tensorflow/transformer/moe.py

Line 1694 in 6b31c0f

gate_1 * mask_1_flat

It should not cause any issues I guess. Just unnecessary computation?

to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

Labels

No labels

No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests