Skip to content

Conversation

Xing-lil
Copy link
Contributor

@Xing-lil Xing-lil commented Sep 30, 2025

PR Category

Auto Parallel

PR Types

Others

Description

问题1:shard_dataloader对非tensor数据类型的兼容性支持
解决1:增强shard_dataloader的兼容性。当输入数据为非tensor类型时,保持原始数据格式不作转换,直接存入dist_batch_data用于模型训练。

问题2:shard_dataloader迭代器重置机制不完善,部分场景下重置失败,迭代器耗尽。
解决2:将dataloader的重置逻辑统一由__next__方法调用。通过在__iter__和__call__方法中将self.iter置为None,并在调用__next__时进行重置,从而避免迭代器耗尽问题。

Copy link

paddle-bot bot commented Sep 30, 2025

你的PR提交成功,感谢你对开源项目的贡献!
请关注后续CI自动化测试结果,详情请参考Paddle-CI手册
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


lizhenxing02 seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

1 similar comment
@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


lizhenxing02 seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@codecov-commenter
Copy link

Codecov Report

❌ Patch coverage is 75.00000% with 1 line in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (develop@e584e37). Learn more about missing BASE report.

Files with missing lines Patch % Lines
python/paddle/distributed/auto_parallel/api.py 75.00% 1 Missing ⚠️

❌ Your patch status has failed because the patch coverage (75.00%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             develop   #75629   +/-   ##
==========================================
  Coverage           ?   75.00%           
==========================================
  Files              ?        1           
  Lines              ?        4           
  Branches           ?        0           
==========================================
  Hits               ?        3           
  Misses             ?        1           
  Partials           ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants