Skip to content

Even faster .NET Parser#127

Open
philipmat wants to merge 95 commits intodevelopfrom
dotnet_parser_specific_xml_parsing
Open

Even faster .NET Parser#127
philipmat wants to merge 95 commits intodevelopfrom
dotnet_parser_specific_xml_parsing

Conversation

@philipmat
Copy link
Owner

@philipmat philipmat commented Sep 13, 2020

File Record Count Python C# v1 C# v2
discogs_20200806_artists.xml.gz 7,046,615 6:22 2:35 0:28 - 13x / 5x
discogs_20200806_labels.xml.gz 1,571,873 1:15 0:22 0:05 - 15x / 4x
discogs_20200806_masters.xml.gz 1,734,371 3:56 1:57 0:31 - 7x / 4x
discogs_20200806_releases.xml.gz 12,867,980 1:45:16 42:38 15:17 - 7x / 3x

That's just a part of the story.
Parallel, processing is the other.
Screen Shot 2020-09-13 at 11 36 31 AM

As the screenshot above shows, the .NET version allows processing multiple files in parallel, achieving a 20x speedup from the Python version in this case.

in part it's done to have a more accurate parser
# Conflicts:
#	alternatives/dotnet/discogs/Program.cs
the csv samples are the same as the python version uses, and the csv files are the ones produce by the `test_extract.py`
export now has parity to python version
@philipmat philipmat marked this pull request as ready for review February 8, 2026 15:32
@philipmat philipmat changed the title (WIP) Even faster .NET Parser Even faster .NET Parser Feb 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants