Skip to content

Adds gradient cap for teacher student distillation #91

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
May 9, 2025
Merged

Adds gradient cap for teacher student distillation #91

merged 3 commits into from
May 9, 2025

Conversation

alessandroassirelli98
Copy link
Contributor

This PR adds a gradient cap to the teacher-student distillation setup.
The goal is to prevent excessively large gradients from destabilizing training.

📌 Changes
Introduced a clipping mechanism to cap the gradients during backpropagation in the distillation process.

Helps improve training stability, especially in early iterations.

Mayankm96
Mayankm96 reviewed May 9, 2025
View reviewed changes
Copy link
Member

@Mayankm96 Mayankm96 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the MR!

Hm not sure why there are formatting issues on the runner file. Could we undo those please?

Also is there a value of max grad norm that works decently? Might make sense to keep them as default.

@alessandroassirelli98
Copy link
Contributor Author

Yeah I was wondering the same.
I added this because I am trying to bring HOVER up to date with the latest rsl_rl. In that paper they use 0.2, should I set that as default? Also we should update the RslRlDistillationAlgorithmCfg in isaaclab to keep consistency

@Mayankm96
Copy link
Member

Mayankm96 commented May 9, 2025 •
edited
Loading

You are right. Making the default 0.2 might or might not work for all usecases. It's alright to keep it None.

Yes, we need to modify IsaacLab configure to support this parameter as well.

Merging this MR then. Not sure why the formatting happened differently but CI didn't complain so it might be just pre-commit version differences.

@Mayankm96 Mayankm96 changed the title Add gradient cap for teacher student distillation Adds gradient cap for teacher student distillation May 9, 2025
@Mayankm96 Mayankm96 merged commit c6834de into leggedrobotics:main May 9, 2025
@Mayankm96
Copy link
Member

@ClemensSchwarke after merging #84, #85, #87 -- might make sense to do a patch release of rsl-rl as well.

@alessandroassirelli98 alessandroassirelli98 mentioned this pull request May 9, 2025
Merged
6 tasks
bikcrum pushed a commit to bikcrum/rsl_rl that referenced this pull request May 9, 2025
@alessandroassirelli98 @bikcrum
Adds gradient cap for teacher student distillation (leggedrobotics#91)
88e5f2c 
This PR adds a gradient cap to the teacher-student distillation setup.
The goal is to prevent excessively large gradients from destabilizing training.

📌 Changes
Introduced a clipping mechanism to cap the gradients during backpropagation in the distillation process.

Helps improve training stability, especially in early iterations.

---------

Co-authored-by: alessandro.assirelli <[email protected]>
Mayankm96 added a commit to isaac-sim/IsaacLab that referenced this pull request Jun 2, 2025
@alessandroassirelli98 @Mayankm96
Adds gradient clipping parameter for distillation using RSL-RL (#2454)
dbaf0d2 
# Description

Added `max_grad_norm` field to `RslRlDistillationAlgorithmCfg` in order
to be compatible with leggedrobotics/rsl_rl#91

## Type of change

- New feature (non-breaking change which adds functionality)

## Checklist

- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [x] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

---------

Signed-off-by: Mayank Mittal <[email protected]>
Co-authored-by: alessandro.assirelli <[email protected]>
Co-authored-by: Mayank Mittal <[email protected]>
fan-ziqi pushed a commit to fan-ziqi/IsaacLab that referenced this pull request Jun 5, 2025
@alessandroassirelli98 @Mayankm96 @fan-ziqi
Adds gradient clipping parameter for distillation using RSL-RL (isaac…
e2c3a3d 
…-sim#2454)

# Description

Added `max_grad_norm` field to `RslRlDistillationAlgorithmCfg` in order
to be compatible with leggedrobotics/rsl_rl#91

## Type of change

- New feature (non-breaking change which adds functionality)

## Checklist

- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [x] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

---------

Signed-off-by: Mayank Mittal <[email protected]>
Co-authored-by: alessandro.assirelli <[email protected]>
Co-authored-by: Mayank Mittal <[email protected]>
yrh012 pushed a commit to aica-technology/isaac-lab that referenced this pull request Jun 16, 2025
@alessandroassirelli98 @Mayankm96 @yrh012
Adds gradient clipping parameter for distillation using RSL-RL (isaac…
4d96d6e 
…-sim#2454)

# Description

Added `max_grad_norm` field to `RslRlDistillationAlgorithmCfg` in order
to be compatible with leggedrobotics/rsl_rl#91

## Type of change

- New feature (non-breaking change which adds functionality)

## Checklist

- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [x] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

---------

Signed-off-by: Mayank Mittal <[email protected]>
Co-authored-by: alessandro.assirelli <[email protected]>
Co-authored-by: Mayank Mittal <[email protected]>
enisycn pushed a commit to enisycn/IsaacLab_Humanoid that referenced this pull request Jul 8, 2025
@alessandroassirelli98 @Mayankm96
Adds gradient clipping parameter for distillation using RSL-RL (#2454)
3458421 
# Description

Added `max_grad_norm` field to `RslRlDistillationAlgorithmCfg` in order
to be compatible with leggedrobotics/rsl_rl#91

## Type of change

- New feature (non-breaking change which adds functionality)

## Checklist

- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [x] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

---------

Signed-off-by: Mayank Mittal <[email protected]>
Co-authored-by: alessandro.assirelli <[email protected]>
Co-authored-by: Mayank Mittal <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Mayankm96 Mayankm96 Mayankm96 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@alessandroassirelli98 @Mayankm96