Skip to content

Larger model fine-tuning and evaluation #3

@purvapruthi

Description

@purvapruthi
  1. Train larger pre-trained models such as LLaMa-8B and evaluate their performance on both diverse and uniform benchmarks using the following train-test split strategies.
  • Combination_K, max_length_K: For the uniform benchmark, the expectation is that these models would have zero performance for direct models and strong generalization performance for the SBS models for closer K's. Figures 1 and 2 in the paper.
    For the diverse benchmark, non-zero performance for the direct models, as predicted by the % of compositions equivalent across the train and test splits.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions