System diagram

[DanGraur]

Added a system diagram and linked it to the main README.

[github-actions]

✅ Result of Pytest Coverage

---------- coverage: platform linux, python 3.12.2-final-0 -----------

Name	Stmts	Miss	Cover
mixtera/core/datacollection/data_collection.py	44	10	77%
mixtera/core/datacollection/datasets/crossaint_dataset.py	16	3	81%
mixtera/core/datacollection/datasets/dataset.py	27	5	81%
mixtera/core/datacollection/datasets/jsonl_dataset.py	58	9	84%
mixtera/core/datacollection/index/index.py	47	9	81%
mixtera/core/datacollection/index/index_collection.py	95	4	96%
mixtera/core/datacollection/index/parser/metadata_parser.py	22	2	91%
mixtera/core/datacollection/index/parser/parser_collection.py	36	11	69%
mixtera/core/datacollection/local/local_collection.py	258	45	83%
mixtera/core/datacollection/property.py	7	0	100%
mixtera/core/datacollection/property_type.py	4	0	100%
mixtera/core/filesystem/filesystem.py	33	1	97%
mixtera/core/filesystem/local_filesystem.py	21	0	100%
mixtera/core/processing/execution_mode.py	4	0	100%
mixtera/core/processing/property_calculation/executor.py	22	2	91%
mixtera/core/processing/property_calculation/local_executor.py	74	5	93%
mixtera/core/query/operators/_base.py	36	6	83%
mixtera/core/query/operators/intersect.py	14	0	100%
mixtera/core/query/operators/materialize.py	13	0	100%
mixtera/core/query/operators/select.py	55	5	91%
mixtera/core/query/operators/union.py	15	0	100%
mixtera/core/query/query.py	115	6	95%
mixtera/server/server_connection.py	6	2	67%
mixtera/tests/core/datacollection/datasets/test_dataset.py	0	0	100%
mixtera/tests/core/datacollection/datasets/test_jsonl_dataset.py	53	6	89%
mixtera/tests/core/datacollection/index/parser/test_parser_collection.py	36	0	100%
mixtera/tests/core/datacollection/index/test_index_collection.py	128	0	100%
mixtera/tests/core/datacollection/local/test_local_collection.py	310	1	99%
mixtera/tests/core/datacollection/test_data_collection.py	19	0	100%
mixtera/tests/core/datacollection/test_property_type.py	7	0	100%
mixtera/tests/core/filesystem/test_filesystem.py	47	0	100%
mixtera/tests/core/filesystem/test_local_filesystem.py	37	0	100%
mixtera/tests/core/processing/property_calculation/test_executor.py	22	0	100%
mixtera/tests/core/processing/property_calculation/test_local_executor.py	75	0	100%
mixtera/tests/core/processing/test_execution_mode.py	7	0	100%
mixtera/tests/core/query/operators/test_base.py	38	0	100%
mixtera/tests/core/query/operators/test_intersect.py	34	2	94%
mixtera/tests/core/query/operators/test_materialize.py	34	0	100%
mixtera/tests/core/query/operators/test_select.py	83	0	100%
mixtera/tests/core/query/operators/test_union.py	25	0	100%
mixtera/tests/core/query/test_e2e.py	53	2	96%
mixtera/tests/core/query/test_query.py	85	1	99%
mixtera/tests/utils/test_utils.py	31	0	100%
mixtera/utils/utils.py	35	0	100%
TOTAL	2181	137	94%
Coverage	HTML	written	to

~ 124 passed in 1.39s ~

[MaxiBoether]

Thanks Dan, this looks good. I think what we might also want to emphasize is that one can use Mixtera without a server (local only) if they want, e.g., for small scale local training. But for now this is nice!

[vGsteiger]

@DanGraur or @MaxiBoether are we going to merge this at some point or is this already so far behind that it would need another overhaul? I would be more than happy to contribute to this if another overhaul is necessary (due to changes).