Skip to content

fix: prevent duplicate CSV header on append - #649

Open
Sidu9 wants to merge 3 commits into
kaifcodec:mainfrom
Sidu9:fix/csv-duplicate-header
Open

Sidu9 wants to merge 3 commits into
kaifcodec:mainfrom
Sidu9:fix/csv-duplicate-header

Conversation

@Sidu9

@Sidu9 Sidu9 commented Sep 14, 2026

Copy link
Copy Markdown

Description

Fixes an issue where the CSV header was duplicated every time results were appended to an existing output file.

Previously, the CSV formatter always included the header, causing repeated headers when the CLI was run multiple times against the same output file.

Changes

  • Ensure the CSV header is written only when the output file is new or empty.
  • Keep CSV formatting focused on generating result rows.
  • Preserve the existing append behavior for subsequent runs.

Testing

  • Verified that the header is written on the first run.
  • Verified that subsequent runs append results without duplicating the header.
  • Ran the existing test suite successfully.

Closes #648

@kaifcodec

Copy link
Copy Markdown
Owner

Thanks for working on this @Sidu9!

The duplicate header issue when appending is definitely real. However, removing CSV_HEADER from into_csv() entirely breaks library usage for callers importing into_csv() from Python (as documented in docs/USAGE.md), who expect a valid CSV with headers.

Could we update this to:

  1. Make include_header: bool = True an optional parameter in into_csv(results, include_header=True).
  2. In __main__.py, determine has_content first and call formatter.into_csv(t_results, include_header=not has_content).
  3. Add a test in tests/test_formatter.py verifying both include_header=True and include_header=False.

@A-S-Manoj A-S-Manoj left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Tested locally, ran the append logic across multiple writes to the same file, and confirmed the header appears only once with no duplication. Output parses cleanly with csv.DictReader.

A regression test asserting the header shows up only once across repeated writes would be a nice add-on, but it's not blocking.

Approving.

@A-S-Manoj

Copy link
Copy Markdown
Contributor

@kaifcodec Ah, missed your comment above, after checking, I agree with you. My approval was based on testing the CLI append path, which does work, but you're right that hardcoding into_csv() to drop the header breaks the documented library usage. +1 on the include_header param approach.

@kaifcodec

Copy link
Copy Markdown
Owner

Yeah no worries @A-S-Manoj, it happens.
I know the core better so I caught that up.
However thank you for opening the issue.

@Sidu9

Sidu9 commented Sep 14, 2026

Copy link
Copy Markdown
Author

Addressed your review @kaifcodec and added regression tests covering all the scenarios @A-S-Manoj. Let me know if this is Okay. Do we also need to make any changes to the docs to incorporate this new param?

@A-S-Manoj

Copy link
Copy Markdown
Contributor

Nice work @Sidu9, this addresses the concern.
Could you also update USAGE.md to mention the new include_header param?

@Sidu9

Sidu9 commented Sep 14, 2026

Copy link
Copy Markdown
Author

I've updated the documentation @A-S-Manoj. Let me know if its ok

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CSV output (-o file.csv -f csv) duplicates the header row on every append run

4 participants