Which is the "pixel normalization"？ you mean the "norm_out" before the conv_out?

[CuddleSabe]

[hyc9]

It refers to the LayerNorm operation commonly used during the ViT patch embedding stage.

Specifically, it refers to the standard patchification process where:

1. The input is divided into patches.
2. The flattened pixels within each patch are normalized using LayerNorm (this is the "pixel normalization").
3. The normalized patches are then passed through a linear projection layer to obtain the patch embeddings.