Skip to content

Commit 2d2afe1

Browse files
Copilotclemensv
authored andcommitted
Complete structuretocsv implementation with proper tests and documentation
- Fixed bug handling union types (type as list) in converter - Updated tests to follow proper pattern: compare with reference files in test/struct - Generated all 10 CSV reference files for test schemas - Updated README.md with s2csv command documentation - All 10 tests now passing with reference comparisons Co-authored-by: clemensv <542030+clemensv@users.noreply.github.com>
1 parent 2907e39 commit 2d2afe1

5 files changed

Lines changed: 143 additions & 256 deletions

File tree

README.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,7 @@ Generate code from JSON Structure:
8080

8181
Direct JSON Structure conversions:
8282

83+
- [`avrotize s2csv`](#convert-json-structure-to-csv-schema) - Convert JSON Structure schema to CSV schema.
8384
- [`avrotize s2x`](#convert-json-structure-to-xml-schema-xsd) - Convert JSON Structure to XML Schema (XSD).
8485

8586
Other commands:
@@ -910,6 +911,27 @@ Conversion notes:
910911
- The tool generates Markdown documentation from the Avrotize Schema. Each record type in the Avrotize Schema is converted to a Markdown section.
911912
- The fields of the record are documented in a table in the Markdown section. Nested records are documented in nested sections in the Markdown file.
912913

914+
### Convert JSON Structure to CSV Schema
915+
916+
```bash
917+
avrotize s2csv <path_to_structure_schema_file> [--out <path_to_csv_schema_file>]
918+
```
919+
920+
Parameters:
921+
922+
- `<path_to_structure_schema_file>`: The path to the JSON Structure schema file to be converted. If omitted, the file is read from stdin.
923+
- `--out`: The path to the CSV schema file to write the conversion result to. If omitted, the output is directed to stdout.
924+
925+
Conversion notes:
926+
927+
- The tool converts JSON Structure schemas to CSV Schema format.
928+
- All JSON Structure Core types are supported including primitives (string, number, boolean, null, integer), extended types (int8-128, uint8-128, float/double, decimal, date, datetime, time, duration, uuid, uri, binary), and compound types (object, array, set, map, tuple, choice).
929+
- Compound types (arrays, objects, maps) are represented as strings in CSV schema, as CSV format doesn't have native support for complex nested structures.
930+
- Required/optional properties are preserved with the `nullable` flag.
931+
- Validation constraints (maxLength, minLength, pattern, minimum, maximum, precision, scale) are preserved in the CSV schema.
932+
- Enum and const keywords are supported and preserved in the output.
933+
- JSON Structure-specific features like `$ref`, `$extends`, definitions, and namespaces are resolved during conversion.
934+
913935
### Convert JSON Structure to Protocol Buffers
914936

915937
```bash

avrotize/structuretocsv.py

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -258,6 +258,16 @@ def convert_structure_type_to_csv_type(self, field_schema: Union[Dict, str, List
258258
return 'string'
259259

260260
struct_type = field_schema['type']
261+
262+
# Handle union types when type is a list within the dict
263+
if isinstance(struct_type, list):
264+
non_null_types = [t for t in struct_type if t != 'null']
265+
if len(non_null_types) == 1:
266+
# Simple nullable type
267+
return self.map_primitive_type_to_csv(non_null_types[0])
268+
else:
269+
# Complex union - use string as fallback
270+
return "string"
261271

262272
# Handle compound types
263273
if struct_type == 'array':
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
{
2+
"fields": [
3+
{
4+
"name": "taggedChoice",
5+
"type": "string"
6+
},
7+
{
8+
"name": "inlineChoice",
9+
"type": "string",
10+
"nullable": true
11+
},
12+
{
13+
"name": "nullableString",
14+
"type": "string",
15+
"nullable": true
16+
}
17+
]
18+
}
Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
{
2+
"fields": [
3+
{
4+
"name": "emptyString",
5+
"type": "string",
6+
"nullable": true,
7+
"maxLength": 0
8+
},
9+
{
10+
"name": "veryLongString",
11+
"type": "string",
12+
"nullable": true,
13+
"maxLength": 1000000
14+
},
15+
{
16+
"name": "preciseDecimal",
17+
"type": "number",
18+
"nullable": true,
19+
"precision": 38,
20+
"scale": 18
21+
},
22+
{
23+
"name": "largeInteger",
24+
"type": "integer",
25+
"nullable": true,
26+
"minimum": -9223372036854775808,
27+
"maximum": 9223372036854775807
28+
},
29+
{
30+
"name": "emptyArray",
31+
"type": "string",
32+
"nullable": true
33+
},
34+
{
35+
"name": "nullableEverything",
36+
"type": "string",
37+
"nullable": true
38+
},
39+
{
40+
"name": "deeplyNested",
41+
"type": "string",
42+
"nullable": true
43+
}
44+
]
45+
}

0 commit comments

Comments
 (0)