-
Notifications
You must be signed in to change notification settings - Fork 2.1k
I-JEPA (Generative vision Example) #2136
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
/gemini review |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
The code introduces an I-JEPA implementation for generative vision examples, including data loading, model definition, pretraining, and downstream task evaluation. The code includes a DataLoader class, IJEPA model components, and downstream task models for ImageNet classification and Clevr object counting. The code also includes pretraining and downstream task training loops.
train_loader = DataLoader(ds_train, batch_size=128, patch_size=patch_size, input_shape=input_shape) | ||
test_loader = DataLoader(ds_test, batch_size=128, patch_size=patch_size, input_shape=input_shape) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
x, y = data | ||
with tf.GradientTape() as tape: | ||
y_pred = self(x, training=True) | ||
loss = self.compute_loss(y=y, y_pred=y_pred) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
x, y = data | ||
with tf.GradientTape() as tape: | ||
y_pred = self(x, training=True) | ||
loss = self.compute_loss(y=y, y_pred=y_pred) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
1 similar comment
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
JEPA-class of models are interesting for a lot of people and having an example of JEPA for vision can be valuable. The goal of the code example would be to show that once JEPA's encoder is trained, it can be used for different downstream tasks. The .py file only has the boilerplate code and some minor descriptions; if approved I will expand on the content. The tutorial will align closely with the I-JEPA paper.