Skip to content

feat: add chunking task #54

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 19 commits into from
Sep 8, 2025
Merged

feat: add chunking task #54

merged 19 commits into from
Sep 8, 2025

Conversation

dolfim-ibm
Copy link
Contributor

Building on #53.

MagnusS0 and others added 7 commits August 26, 2025 21:42
@MagnusS0
feat: implement document chunking functionality for RAG workflows
b7f7ba4 
Signed-off-by: Magnus Samuelsen <[email protected]>
@MagnusS0
feat: improved cache key generation and seperate md table module
90cbcc6 
Also improves documentation

Signed-off-by: Magnus Samuelsen <[email protected]>
@dolfim-ibm
add tasktype and options for chunking
7b95489 
Signed-off-by: Michele Dolfi <[email protected]>
@dolfim-ibm
fix error propagation
f323812 
Signed-off-by: Michele Dolfi <[email protected]>
@dolfim-ibm
fix chunking cache management
7ea811e 
Signed-off-by: Michele Dolfi <[email protected]>
@dolfim-ibm
fix test_chunking
fb890b3 
Signed-off-by: Michele Dolfi <[email protected]>
@dolfim-ibm
test for chunking in RQ
919fe3b 
Signed-off-by: Michele Dolfi <[email protected]>
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Sep 3, 2025 •
edited
Loading

✅ DCO Check Passed

Thanks @dolfim-ibm, all your commits are properly signed off. 🎉

@mergify Mergify
Copy link

mergify bot commented Sep 3, 2025

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

@codecov Codecov
Copy link

codecov bot commented Sep 3, 2025 •
edited
Loading

Codecov Report

❌ Patch coverage is 79.37743% with 53 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
docling_jobkit/convert/chunking.py 82.57% 23 Missing ⚠️
docling_jobkit/orchestrators/rq/worker.py 0.00% 10 Missing ⚠️
docling_jobkit/orchestrators/kfp/orchestrator.py 0.00% 9 Missing ⚠️
docling_jobkit/convert/results.py 70.00% 3 Missing ⚠️
docling_jobkit/datamodel/result.py 91.66% 2 Missing ⚠️
docling_jobkit/datamodel/task.py 87.50% 2 Missing ⚠️
docling_jobkit/orchestrators/local/orchestrator.py 80.00% 2 Missing ⚠️
docling_jobkit/orchestrators/rq/orchestrator.py 81.81% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

@dolfim-ibm
fix RQ test
bbbe50a 
Signed-off-by: Michele Dolfi <[email protected]>
@dolfim-ibm
use explicit union (for the moment)
77a994a 
Signed-off-by: Michele Dolfi <[email protected]>
PeterStaar-IBM
PeterStaar-IBM previously approved these changes Sep 3, 2025
View reviewed changes
Copy link

@PeterStaar-IBM PeterStaar-IBM left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

amazing!

vagenas
vagenas reviewed Sep 3, 2025
View reviewed changes
vagenas
vagenas previously approved these changes Sep 4, 2025
View reviewed changes
@dolfim-ibm
rename and move result classes
65b9846 
Signed-off-by: Michele Dolfi <[email protected]>
@dolfim-ibm
fix test
f8ffc30 
Signed-off-by: Michele Dolfi <[email protected]>
@dolfim-ibm
add type alias
ce95f20 
Signed-off-by: Michele Dolfi <[email protected]>
@dolfim-ibm
add doc_items and captions
4bad9be 
Signed-off-by: Michele Dolfi <[email protected]>
@dolfim-ibm
use annotations
1a80a41 
Signed-off-by: Michele Dolfi <[email protected]>
@dolfim-ibm dolfim-ibm mentioned this pull request Sep 8, 2025
Merged
2 tasks
PeterStaar-IBM
PeterStaar-IBM approved these changes Sep 8, 2025
View reviewed changes
Copy link

@PeterStaar-IBM PeterStaar-IBM left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@dolfim-ibm dolfim-ibm merged commit 3b9b11c into main Sep 8, 2025
9 checks passed
@dolfim-ibm dolfim-ibm deleted the feat-add-chunking-task branch September 8, 2025 15:33
@dolfim-ibm dolfim-ibm mentioned this pull request Sep 8, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@PeterStaar-IBM PeterStaar-IBM PeterStaar-IBM approved these changes

@vagenas vagenas vagenas left review comments

@vku-ibm vku-ibm Awaiting requested review from vku-ibm

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@dolfim-ibm @vagenas @PeterStaar-IBM @MagnusS0