Skip to content

changelog_en.md

Latest commit

 

History

History
93 lines (78 loc) · 5.13 KB

changelog_en.md

File metadata and controls

93 lines (78 loc) · 5.13 KB
Raw

Changelog (English)

v4.0

  • Increased num_iterations from 1 to 4: Making clip_high (DeepCoder method) more effective:
    • More iterations allow better exploration of the policy space
    • Combined with asymmetric clipping (epsilon=0.2, epsilon_high=0.5), significantly improves Pass@1
  • Switched to DeepCoder-Preview-Dataset (lcbv5, 28 samples):
    • Uses high-quality lcbv5 subset from agentica-org/DeepCoder-Preview-Dataset
    • Dramatically reduces training data while improving results
  • Performance improvements:
    • Pass@1 improved by 100% (1.82% → 3.64%)
    • Compile Rate improved by 4.13% (73.33% → 76.36%)
    • Average Score improved by 5.03% (0.4274 → 0.4489)
  • Efficiency improvements:
    • Training data reduced by 99.8% (13328 → 28)
    • Training time reduced by 73% (4h24m → 1h12m)
  • Updated README with v4.0 results and detailed comparison links

v3.5

  • Remove test case examples from prompt: Stop concatenating input_output/public_tests to the chat template:

    • Some samples carry extremely large test cases (200K+ chars each), causing prompt bloat and OOM during training
    • Now prompt contains only problem description without examples
    • This change significantly reduces memory usage during training

  • Limit test cases during training:

    • In code_reward_func, limit to first 5 test cases to avoid excessive subprocess fork pressure (each fork creates a large model process; too many forks → OOM killer)

  • DeepCoder data loading optimization:

    • Trim input_output to first 5 test cases at load time
    • Some samples carry huge test cases causing multi-GB memory bloat → OOM kill

  • Clip High (from DeepCoder/DAPO paper): Implemented asymmetric clipping in GRPO loss:

    • Added epsilon=0.2 (lower bound: 1 - 0.2 = 0.8)
    • Added epsilon_high=0.5 (upper bound: 1 + 0.5 = 1.5)
    • Unlike symmetric clipping [1-ε, 1+ε], Clip High increases only the upper bound
    • This encourages more exploration and prevents premature convergence when correct solutions are found
    • Refer to DeepCoder paper for details

  • DeepCoder Dataset Support: Added support for DeepCoder-Preview-Dataset:

    • Added download_DeepCoder-Preview-Dataset.py script (parquet format) to download and preprocess DeepCoder dataset
    • Supports 4 configs: codeforces, lcbv5, primeintellect, taco
    • Added --dataset CLI argument to choose between 'code_contests' (default) and 'deepcoder'
    • Modified reward function to support both dataset formats
    • DeepCoder dataset requires at least 5 test cases per problem (filtered during download)

  • Renamed download_dataset.py to download_code_contests.py for clarity

  • Updated GRPOConfig to use new epsilon and epsilon_high parameters (replacing deprecated clip_range)

  • Updated README to include DeepCoder dataset badge and usage instructions

  • Added DeepCoder dataset citation to README

v3.1.5

  • Inspired by DeepCoder: adjusted two key training hyperparameters:
    • Set KL divergence coefficient beta=0 in train_rl.py, allowing the policy to explore freely without being anchored to the base model during training;
    • Increased clip_range from the default 0.2 to 0.3, allowing larger policy update steps when correct solutions are found.
  • Made TinyLoRA SVD rank rank a configurable CLI argument (--rank N, default: 2), enabling fine-grained control over the capacity/stability tradeoff.
  • Updated README, usage guides, and output/README to document the --rank argument and added DeepCoder citation.

v3.1

  • Fix issue 001. The fix ensures proper handling of dequantized weight tensors in a distributed data parallel (DDP) setting, eliminating the CUBLAS error.
  • Added explicit configurable controls section in README, organized by five system blocks:
    • reward system
    • data selection
    • TinyLoRA architecture
    • GRPO optimization
    • prompt construction
  • Clarified quantization compatibility and checkpoint metadata behavior:
    • default training uses 4-bit quantized loading
    • --no_quant enables BF16 path
    • checkpoint stores is_quantized for safe loading
  • Improved README information architecture:
    • complete quick start in homepage
    • move detailed changelog and operational details to docs

v3.0

  • Upgraded core implementation to support RL training with TinyLoRA global parameters.
  • Added end-to-end path for:
    • dataset download
    • training
    • validation
    • baseline and checkpoint testing
  • Introduced verify_pipeline.py for pipeline sanity checks.

v2.5

  • Switched to Qwen2.5-Coder series for code-generation alignment.
  • Refined model wrapping behavior for TinyLoRA-injected linear layers.
  • Improved default training stability under small-sample settings.

v2.0

  • Initial public structure of TinyLoRA-Qwen-Coder project.
  • Added first runnable scripts for data processing and basic evaluation.