about training and sampling cvfm

[KinamSalad]

Hello. First of all, thank you for your great work!
First of all. I have a question about CVFM.
Your tutorial showed how to move from one moon distribution x0 to another moon distribution x1 with a 270-degree turn.
However, in reality, we usually train flow matching from Gaussian distribution x0 to Moon distribution x1. When x0 is Gaussian, does CVFM still work properly considering its principle? I wonder this method still work in real condition like text to image or something. I experienced that the optimal transport(Tong et al.) does not work at real world CFM problem...

Thanks.

[agenerale]

Thanks for taking the time to look through our work. The development of CVFM is primarily targeted towards learning dynamical systems from measurements at discrete marginal distributions across time, where it would not be possible to choose a simple base density (e.g., Gaussian).

Theoretically, OT-FM should still work with conditioning from a base N(0,1) distribution as shown in their Conditional MNIST Notebook example. In contrast to CVFM, this is just because we can choose any arbitrary value for $y_0$ to coincide with $y_1$ and just call it $y$ into the network. That being said, I've encountered more challenging convergence issues when applying conditioning to OT-FM (Tong et al. (2023)) in comparison to the original FM paper (Lipman et al. (2022)), which I suspect is due to the noise scheduling on the paths, although this would be something interesting to look into further.

To further help, I've pushed this CVFM/COT-FM Notebook, which discusses limitations of COT-FM (something distinct to OT-FM), and later on has 2 examples of mapping from $\mathcal{N}(0,1)$ to $\hat{p}_1$.

Hope this helps!