Skip to content

Rework mime type white list - #2198

Merged
Tschuppi81 merged 77 commits into
masterfrom
feature/ogc-2738-pentest-arbitrary-file-upload
Feb 20, 2026
Merged

Rework mime type white list#2198
Tschuppi81 merged 77 commits into
masterfrom
feature/ogc-2738-pentest-arbitrary-file-upload

Conversation

@Tschuppi81

@Tschuppi81 Tschuppi81 commented Nov 10, 2025

Copy link
Copy Markdown
Contributor

Org: Ensure mime type validation on file upload fields in form code

TYPE: Feature
LINK: ogc-2738

@linear

linear Bot commented Nov 10, 2025

Copy link
Copy Markdown

@codecov

codecov Bot commented Nov 10, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.31250% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.24%. Comparing base (51ed082) to head (b24c85f).
⚠️ Report is 3 commits behind head on master.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/onegov/form/validators.py 81.25% 3 Missing ⚠️
Additional details and impacted files
Files with missing lines Coverage Δ
src/onegov/agency/forms/agency.py 97.24% <100.00%> (-0.02%) ⬇️
src/onegov/election_day/forms/election.py 98.13% <100.00%> (-0.01%) ⬇️
src/onegov/election_day/forms/election_compound.py 97.75% <100.00%> (-0.01%) ⬇️
src/onegov/election_day/forms/subscription.py 100.00% <ø> (ø)
src/onegov/election_day/forms/upload/common.py 100.00% <ø> (ø)
src/onegov/election_day/forms/upload/election.py 100.00% <ø> (ø)
...gov/election_day/forms/upload/election_compound.py 100.00% <ø> (ø)
.../onegov/election_day/forms/upload/party_results.py 100.00% <ø> (ø)
src/onegov/election_day/forms/upload/rest.py 100.00% <ø> (ø)
src/onegov/election_day/forms/upload/vote.py 100.00% <ø> (ø)
... and 29 more

... and 9 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 51ed082...b24c85f. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread src/onegov/form/validators.py
Comment thread src/onegov/pas/forms/data_import.py Outdated
@Tschuppi81
Tschuppi81 requested a review from Daverball December 4, 2025 12:03
@Tschuppi81

Copy link
Copy Markdown
Contributor Author

I saw that files types are handled differently for view_upload_file_by_json in handle_file_upload. Basically all file types are allowed. Shall we keep this?

@Tschuppi81

Copy link
Copy Markdown
Contributor Author

Should I completely remove type application/octet-stream ? It is mostly used in conjunction with application/zip

@Daverball

Copy link
Copy Markdown
Member

I saw that files types are handled differently for view_upload_file_by_json in handle_file_upload. Basically all file types are allowed. Shall we keep this?

We can make sure to set supported_content_types on GeneralFileCollection. That's the only one that would allow anything to be uploaded currently through those views.

@Daverball Daverball left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall, but there's a couple of details we should iron out.

Comment thread src/onegov/form/validators.py
Comment thread src/onegov/form/validators.py
Comment thread src/onegov/form/validators.py
Comment thread src/onegov/org/forms/event.py Outdated
Comment thread src/onegov/pas/forms/data_import.py Outdated
Comment thread src/onegov/form/fields.py Outdated
Comment thread src/onegov/form/fields.py Outdated
Comment thread src/onegov/form/fields.py Outdated
@Daverball

Daverball commented Dec 4, 2025

Copy link
Copy Markdown
Member

Should I completely remove type application/octet-stream ? It is mostly used in conjunction with application/zip

It's probably fine to remove it for now. There may however be the rare false positive for any files that cannot be identified correctly by libmagic. Generally pdfs, zips and any other binary file formats can end up as application/octet-stream, it's a generic catch-all content type for binary data if it couldn't be detected as anything else.

Comment thread src/onegov/form/fields.py Outdated
@Tschuppi81

Tschuppi81 commented Feb 12, 2026

Copy link
Copy Markdown
Contributor Author

@Daverball I believe I am ready for a final review

@Daverball Daverball left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall, but I found some small bugs and also suggested some miscellaneous improvements.

Comment thread src/onegov/form/fields.py Outdated
Comment thread src/onegov/form/validators.py Outdated
Comment thread src/onegov/form/validators.py Outdated
Comment thread src/onegov/org/forms/newsletter.py Outdated
Comment thread src/onegov/org/views/files.py Outdated
Comment thread src/onegov/pas/forms/parliamentarian.py Outdated
Comment thread src/onegov/swissvotes/fields/dataset.py Outdated
Comment thread src/onegov/form/fields.py
Comment thread src/onegov/swissvotes/fields/metadata.py Outdated
Comment thread tests/onegov/form/test_parser.py Outdated
Tschuppi81 and others added 16 commits February 19, 2026 07:58
Co-authored-by: David Salvisberg <david.salvisberg@seantis.ch>
Co-authored-by: David Salvisberg <david.salvisberg@seantis.ch>
Co-authored-by: David Salvisberg <david.salvisberg@seantis.ch>
Co-authored-by: David Salvisberg <david.salvisberg@seantis.ch>
Co-authored-by: David Salvisberg <david.salvisberg@seantis.ch>
Co-authored-by: David Salvisberg <david.salvisberg@seantis.ch>
Co-authored-by: David Salvisberg <david.salvisberg@seantis.ch>
Co-authored-by: David Salvisberg <david.salvisberg@seantis.ch>
Co-authored-by: David Salvisberg <david.salvisberg@seantis.ch>
revert this

Co-authored-by: David Salvisberg <david.salvisberg@seantis.ch>
revert this

Co-authored-by: David Salvisberg <david.salvisberg@seantis.ch>
@Tschuppi81

Tschuppi81 commented Feb 19, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @Daverball for identifying the various bugs needed to be fixed!

@Tschuppi81
Tschuppi81 merged commit ff77bff into master Feb 20, 2026
15 checks passed
@Tschuppi81
Tschuppi81 deleted the feature/ogc-2738-pentest-arbitrary-file-upload branch February 20, 2026 08:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants