Skip to content

The scale of timesteps in the text modeling examples #17

@realfolkcode

Description

@realfolkcode

Hi! Really enjoying the code and the paper. These are the most comprehensive resources on flow matching.

I noticed that the text modeling example might contain a subpar encoding of timesteps. In short, it uses the default max_period parameter from the GLIDE repository, which is a diffusion model with discrete timesteps in the range {0, 1, 2, ...}. However, as I understand correctly, in your example, all timesteps are sampled from the [0, 1] time horizon. See the similar issue that I've opened in the SiT repository. I think the most elegant solution is to rescale the timesteps similar to how Flux handles them.

Let me know if I am misunderstanding something, as I am just on my way of learning the flow matching framework.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions