Skip to content

[WIP][Benchmark] SSL Baselines (MAE, SimCLR, LeJEPA, BarlowTwins, NNCLR, Dino 1)#41

Open
sami-bg wants to merge 54 commits intogalilai-group:mainfrom
sami-bg:ssl-baselines
Open

[WIP][Benchmark] SSL Baselines (MAE, SimCLR, LeJEPA, BarlowTwins, NNCLR, Dino 1)#41
sami-bg wants to merge 54 commits intogalilai-group:mainfrom
sami-bg:ssl-baselines

Conversation

@sami-bg
Copy link
Copy Markdown
Contributor

@sami-bg sami-bg commented Jan 30, 2026

What does this PR do?

Here are the instructions for submitting SSL benchmark runs to SLURM:

  SSL Benchmark SLURM Submission

  # Activate the virtual environment
  source /oscar/home/sboughan/stable-datasets/.venv/bin/activate

  # Single dataset, single model
  SLURM_PARTITION=3090-gcondo python -m stable_datasets.benchmarks.self_supervised.main \
      --multirun --config-name slurm 'hydra.launcher.qos=null' \
      dataset=cifar10 model=simclr backbone=vit_small

  # Multiple datasets, single model
  SLURM_PARTITION=3090-gcondo python -m stable_datasets.benchmarks.self_supervised.main \
      --multirun --config-name slurm 'hydra.launcher.qos=null' \
      dataset=cifar10,stl10,svhn model=dino backbone=vit_small

  # Multiple models on multiple datasets (creates one job per combo)
  SLURM_PARTITION=3090-gcondo python -m stable_datasets.benchmarks.self_supervised.main \
      --multirun --config-name slurm 'hydra.launcher.qos=null' \
      dataset=cifar10,cifar100 model=simclr,dino,mae,lejepa,nnclr,barlow_twins backbone=vit_small

  # All datasets (auto-expands to every available dataset)
  SLURM_PARTITION=3090-gcondo python -m stable_datasets.benchmarks.self_supervised.main \
      --multirun --config-name slurm 'hydra.launcher.qos=null' \
      dataset=all model=simclr backbone=vit_small

  # Collect results into a LaTeX table (with supervised baselines)
  python -m stable_datasets.benchmarks.self_supervised.collect_results \
      --entity stable-ssl --project stable-datasets-benchmarks \
      --supervised-entity samibg --supervised-project stable-datasets \
      --latex results.tex

  Notes:
  - Set SLURM_PARTITION to your partition (e.g., gpu, 3090-gcondo)
  - Make sure to put the right paths on config.yaml (it's hardcoded for my path)
  - Add 'hydra.launcher.qos=null' if your partition doesn't require a QOS
  - Results are logged to W&B under stable-ssl/stable-datasets-benchmarks
  - The collect_results.py script caches finished W&B runs locally, so re-runs are fast

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a Github issue? Please add a link to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines
  • Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@Leon-Leyang
Copy link
Copy Markdown
Collaborator

Could you run pre-commit run --all-files so that the precommit test passes.

@sami-bg sami-bg changed the title [WIP][BENCHMARK] SSL Baselines (MAE, SimCLR, LeJEPA) [WIP][Benchmark] SSL Baselines (MAE, SimCLR, LeJEPA, VICReg, Dino 1) Feb 2, 2026
@sami-bg sami-bg changed the title [WIP][Benchmark] SSL Baselines (MAE, SimCLR, LeJEPA, VICReg, Dino 1) [WIP][Benchmark] SSL Baselines (MAE, SimCLR, LeJEPA, BarlowTwins, NNCLR, Dino 1) Feb 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants