Address embedding update with updated models

[egrace479]

BioCLIP 2.5 will be the new family of models (BioCLIP 2.5-H, BioCLIP 2.5-L, etc., where the letter references the CLIP model size).

• All these models are/will be trained on the revised TreeOfLife-200M, which includes an additional 19M images and more cleaning, the taxonomy remains un-changed.
• Text embeddings will all be under model name folders here.
  • This is a breaking change for the pybioclip embed feature (as explained below).
• pybioclip default model will remain BioCLIP 2 because 2.5 is quite a bit larger, but it can still be used with pybioclip (we may update to 2.5-L, once that is ready).

The question for pybioclip is how we address a breaking change to the embed feature, since embeddings are a hard-coded pull from embeddings/ txt_emb_species.npy and embeddings/ txt_emb_species.json at the repo root (see code).

Related to #160

[hlapp]

I would argue we'll need to keep the embeddings used by currently released versions of pybioclip where they are. Otherwise we allow released versions of pybioclip to break from one day to another without a user doing anything differently, which would not be good.

[egrace479]

I would argue we'll need to keep the embeddings used by currently released versions of pybioclip where they are. Otherwise we allow released versions of pybioclip to break from one day to another without a user doing anything differently, which would not be good.

Yes, @thompsonmj was also looking at options for tombstoning the files to tell users to update.

[hlapp]

I'm not sure how tombstoning will change anything for a pybioclip user?

[egrace479]

Yes, it wouldn't keep the existing functionality, just suggest an update. I suppose it wouldn't make much difference if we leave the BioCLIP and BioCLIP 2 embeddings in place in TOL-10M & 200M, respectively, but update the code in a new version to call the embeddings from embeddings/<model-name>/txt_emb_species.<npy or json>. The only place it would matter would be for the original BioCLIP, since we'd want to ensure the model and embeddings called do match (re: discussion in #160).

[egrace479]

The BioCLIP taxonomic update is essentially a "bug fix", but we don't want to mess up existing versions. As a result, both BioCLIP and BioCLIP 2 implementations will keep the two text embeddings files in the embeddings/ directories for their respective training datasets, with the addition of <model-name>/ directories containing the text embeddings for the particular models. The BioCLIP and TreeOfLife-10M updates will be pushed at the same time to ensure the embeddings and model version are always matching.

The <model-name>/ directory will be the string used to call the model from Hugging Face to ensure the proper one is called, avoiding a need to update with each new BioCLIP 2.5 model (unless it is to be the new default). We can add a check that "bioclip" is in the name, since the non-embedding part of this package also works for other CLIP-based models.

[egrace479]

Set up a branch to start this (feat/tol-models). It won't work until we have the model name folders in the embeddings/ directories on the dataset repos, but was checking the updates required to ensure I set the datasets up correctly. The BioCLIP and BioCLIP 2 text embeddings will be duplicated: leaving them directly under the embeddings/ directories in TOL-10M and TOL-200M, respectively, as they currently are and setting up folders for the models there.

It also still needs tests.

[egrace479]

Further discussion and evaluation, we don't have the same JSON across all models within each dataset, so we'll stick with naming by model (as in #171) for better data organization/clarity.

Example for TreeOfLife-10M:

embeddings/
    txt_emb_biocap.npy
    txt_emb_biocap.json
    txt_emb_bioclip.npy
    txt_emb_bioclip.json
    # generic 'species' emb files are for pybioclip backward-compatibility
    txt_emb_species.json
    txt_emb_species.npy

I've adjusted my solution for this in the code and we will match it in the dataset repos.