Finetune example and concepts docs #943
Conversation
PicoCentauri
left a comment
PicoCentauri
left a comment
There was a problem hiding this comment.
Yes, it is already much better as before. I gave it a quick read. I will do a more in depth review on the text when the docs are building.
docs/src/concepts/fine-tuning.rst
Outdated
| learning_rate: 1e-5 | ||
| Please note, that in most use cases you should invoke a new energy head by specifying | ||
| a new energy variant. The variant naming follows the simple pattern |
There was a problem hiding this comment.
Should we link the variants page on metatomic?
https://docs.metatensor.org/metatomic/latest/outputs/variants.html
| .. _label_fine_tuning_concept: | ||
|
|
||
| Fine-tune a pre-trained model | ||
| ============================= |
There was a problem hiding this comment.
Here also goes the flow chart, right? 🤓
and maybe we add also a table of content after the flow chart?
There was a problem hiding this comment.
I thought flowchart is rendered a bit useless, since we mostly recommend full finetuning with new head and usually inheriting weights?
There was a problem hiding this comment.
okay fine for me. But a TOC should be useful.
| .. warning:: | ||
| This section of the documentation is only relevant for PET model so far. | ||
| Finetuning is currently only available for the PET architecture. |
There was a problem hiding this comment.
same comment as in the concept.
| @@ -1,4 +1,7 @@ | |||
| """ | |||
| .. _parity-plot-example: | |||
There was a problem hiding this comment.
You don't need this as sphinx gallery automagically creates labels. They are a bit cumbersome but you can find them in the generated rst. For this example the label is
sphx_glr_generated_examples_0-beginner_04-parity_plot.py
Co-authored-by: Philip Loche <[email protected]>
Co-authored-by: Philip Loche <[email protected]>
Co-authored-by: Philip Loche <[email protected]>
Co-authored-by: Philip Loche <[email protected]>
Co-authored-by: Philip Loche <[email protected]>
PicoCentauri
left a comment
PicoCentauri
left a comment
There was a problem hiding this comment.
This is vert useful @MarkusFas. I left some minor comments for better styling. After the round we could merge from my side.
| .. hint:: | ||
|
|
||
| If you want to fine-tune an existing model | ||
| check out :ref:`label_fine_tuning_concept`. |
There was a problem hiding this comment.
I like it. But should we better kink directly to a tutorial?
| .. code-block:: bash | ||
| The basic fine-tuning strategy is a good choice in the case when the level of theory | ||
| which is used for the original training is the same, or at least similar to the one used | ||
| for the new dataset. However, since this is not always the case, we also provide more | ||
| advanced fine-tuning strategies described below. | ||
| wget https://huggingface.co/lab-cosmo/pet-mad/resolve/v1.1.0/models/pet-mad-v1.1.0.ckpt |
There was a problem hiding this comment.
I think you don't need this here. Below you do the same. You can maybe below the python code paste the bash command.
| different from the reference of the pretrained model, it is recommended to create a | ||
| new energy head. Using this so-called energy variant can be simply invoked by | ||
| requesting a new target in the options file. Follow the nomenclature | ||
| energy/{yourname}. |
There was a problem hiding this comment.
| energy/{yourname}. | |
| ``energy/{yourname}``. |
| options in the ``options.yaml`` file. The following options need to be set: | ||
| A simple ``options-ft.yaml`` file for this task could look like this: | ||
| .. code-block:: yaml |
There was a problem hiding this comment.
you could directly import the yaml file that you wrote using literalinclude. This avoids duplication.
| subprocess.run( | ||
| [ | ||
| "wget", | ||
| "https://huggingface.co/lab-cosmo/pet-mad/resolve/v1.1.0/models/pet-mad-v1.1.0.ckpt", | ||
| ] | ||
| ) | ||
| subprocess.run(["rm", "-rf", "outputs"]) |
There was a problem hiding this comment.
all of them should have check=true.
| # %% | ||
| # |
There was a problem hiding this comment.
Maybe add some text what the next step after the training is.
Basically saying the we plot the training curve now.
| validation_energy_RMSE = structured["validation energy/finetune RMSE (per atom)"] | ||
| validation_forces_MAE = structured["validation forces[energy/finetune] MAE"] | ||
|
|
||
| fig, axs = plt.subplots(1, 2, figsize=(12, 5)) |
There was a problem hiding this comment.
The labels of the figure are a bit small with figsize=(12, 5). Maybe you stack the plot vertically?
| # %% | ||
| # |
There was a problem hiding this comment.
Maybe give some text here between the blocks or make it one big code block
-beginner example finetuning
-added variants to concepts
Contributor (creator of pull-request) checklist
Maintainer/Reviewer checklist
📚 Documentation preview 📚: https://metatrain--943.org.readthedocs.build/en/943/