Implements SSL-EY - #378
Conversation
|
hey @jameschapman19 thanks for contributing. I'll try to start checking your code soon. |
| @@ -0,0 +1,81 @@ | |||
| defaults: | |||
| - _self_ | |||
| - augmentations: ssley.yaml | |||
There was a problem hiding this comment.
You already have the augmentations below, either remove this or move the augmentations to a separate file (both are fine)
There was a problem hiding this comment.
Good spot - a heads up I think this is also the case for scripts/pretrain/imagenet-100/vicreg.yaml in that case.
This method should plug in wherever VICReg is used. In many ways the benefit of the loss function is it's got a lot of the nice properties of VICReg but without needing to tune the three loss function parameters.
There was a problem hiding this comment.
No idea why VICReg is like that haha. I'm fine with having a separate file for the augmentations or having it merged with the main file. Should also prob fix VICReg after that.
|
@vturrisi - just working through ironing things out perhaps a premature PR I just wanted to get a hang of the CI. Will give you a shout when I'm passing more of these. |
|
@jameschapman19 no worries, take your time :) and ping me when you are done |
Codecov Report
Additional details and impacted files
*This pull request uses carry forward flags. Click here to find out more.
|
2 participants
Adding SSL-EY from (https://arxiv.org/abs/2310.01012, https://neurips.cc/virtual/2023/80864)
For the moment I've implemented the simple version but it should in principle plug into multicrop like SWAV.
Will have a stab at running the shorter benchmarks shortly.