Skip to content

Add roformer to NormalizedConfigManager - #2465

Open
Pateljainam069 wants to merge 2 commits into
huggingface:mainfrom
Pateljainam069:add-roformer-normalized-config
Open

Add roformer to NormalizedConfigManager#2465
Pateljainam069 wants to merge 2 commits into
huggingface:mainfrom
Pateljainam069:add-roformer-normalized-config

Conversation

@Pateljainam069

Copy link
Copy Markdown

What does this PR do?

NormalizedConfigManager raises a KeyError for roformer because it's
missing from the _conf mapping in optimum/utils/normalized_config.py,
blocking ONNX Runtime optimization for RoFormer models.

RoFormerConfig uses the standard hidden_size/num_attention_heads/
num_hidden_layers naming (verified against transformers.RoFormerConfig),
so it maps directly to NormalizedTextConfig, same as bert/roberta
no new normalized config class needed.

Part of #351.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

Who can review?

@echarlaix @JingyaHuang @michaelbenayoun @IlyasMoutawwakil

Two notes on what I changed:

  • Removed the HTML comment blocks (they were invisible anyway, just cleaning the raw text) and the leftover Fixes # (issue) placeholder line, replaced with Part of Add all available ONNX models to ORTConfigManager #351 in the body — that's what actually creates the GitHub link to the issue.
  • Left the first checkbox unchecked-turned-checked is debatable — actually leave all three unchecked (- [ ]), since this isn't a docs/typo fix and doesn't need new docs or tests (matches the precedent PRs for this same file). Let me know if you'd rather I explain why not to check that box — happy to adjust.

RoFormer uses the standard hidden_size/num_attention_heads/num_hidden_layers
naming (verified against transformers.RoFormerConfig), so it maps directly
to NormalizedTextConfig like bert/roberta. Part of huggingface#351.
…ion, ibert, levit, mobilebert, mobilevit to NormalizedConfigManager

All ten map onto the existing NormalizedTextConfig / NormalizedVisionConfig
patterns with no new subclasses needed:

- NormalizedTextConfig (hidden_size/num_attention_heads/num_hidden_layers):
  convbert, data2vec-text, ibert, mobilebert
- NormalizedVisionConfig (image_size/num_channels):
  beit, convnext, convnextv2, data2vec-vision, levit, mobilevit

Each was verified against the corresponding transformers config class
(model_type string, attribute presence, and an end-to-end
NormalizedConfigManager.get_normalized_config_class(...) round trip).

Part of huggingface#351.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant