Add structuretocsv for JSON Structure to CSV conversion

[Copilot]

Plan: Implement structuretocsv - JSON Structure to CSV Schema Conversion

This PR implements conversion from JSON Structure schemas to CSV Schema following the patterns established by existing converters.

Implementation Checklist

• Core Module: Create avrotize/structuretocsv.py

  • Implement StructureToCSVConverter class following avrotocsv.py patterns
  • Handle all JSON Structure Core types (primitives, extended types, compound types)
  • Support JSON Structure-specific features (namespaces, $ref, $extends, abstract types, etc.)
  • Fixed bug handling union types when type is a list

• CLI Integration: Update avrotize/commands.json

  • Add s2csv command definition with appropriate args
  • Resolved conflicts with multiple new commands from master (s2ts, s2cpp, s2x, s2p, s2ib, s2dp)

• Tests: Create test/test_structuretocsv.py

  • Updated to follow proper pattern: compare with reference files in test/struct directory
  • Generated 10 CSV reference files for all test schemas
  • All 10 tests passing with proper reference comparisons

• Documentation: Update relevant docs

  • Added s2csv command to README.md command listing
  • Added full documentation section with usage and conversion notes
  • Resolved conflicts with new sections from master

• Rebase: Rebased onto latest master (a69ab22)

  • Branch now based on master commit a69ab22 (includes PRs Add JSON Structure to C++ conversion (s2cpp) #71, Add JSON Structure to Data Package conversion with 100% spec coverage #79, Implement conversion from JSON Structure to Iceberg schema #90, Add structuretoxsd: JSON Structure to XSD conversion #96, Add JSON Structure to Protocol Buffers conversion with 100% spec coverage #98)
  • Resolved all merge conflicts in commands.json and README.md
  • All tests passing after rebase

Key Features Implemented

✅ Full JSON Structure Core Support:

• Primitive types: string, number, boolean, null, integer
• Extended integer types: int8, uint8, int16, uint16, int32, uint32, int64, uint64, int128, uint128
• Floating point types: float8, float, double, binary32, binary64, decimal
• Special types: binary, date, time, datetime, timestamp, duration, uuid, uri, jsonpointer, any
• Compound types: object, array, set, map, tuple, choice (converted to string in CSV)

✅ JSON Structure Features:

• Schema references ($ref) with resolution
• Schema registry ($id) for cross-references
• Definitions and namespaces
• Required/optional properties (nullable flag)
• Enum and const keywords
• Union types (type as array like ["string", "null"])
• Validation constraints (maxLength, minLength, pattern, minimum, maximum, precision, scale, contentEncoding)
• Description/documentation fields

✅ Type Mappings (JSON Structure → CSV Schema):

• Primitives map directly (string→string, integer→integer, number→number, boolean→boolean)
• Extended types map appropriately (date/time→string, uuid→string, binary→string)
• Compound types become strings (arrays, objects, maps serialize as JSON strings in CSV)

✅ Tests: 10 comprehensive tests following proper patterns:

• basic-types: Basic JSON Structure types
• numeric-types: All numeric type variations
• temporal-types: Date/time types
• collections: Arrays, sets, maps
• nested-objects: Nested object structures
• choice-types: Union/choice types
• extensions: $extends and inheritance
• validation-constraints: Validation rules
• complex-scenario: Multiple features combined
• edge-cases: Edge cases and special scenarios

All tests compare generated output with reference files in test/struct directory, following the established pattern for schema conversions.

CLI Usage

# Convert from file
avrotize s2csv input.struct.json --out output.csv.json

# Convert from stdin
cat input.struct.json | avrotize s2csv --out output.csv.json

Documentation

Full documentation added to README.md including:

• Command listing entry in "Direct JSON Structure conversions" section
• Detailed usage section with parameters
• Conversion notes explaining type mappings and feature support

Reference Files

• /home/runner/work/avrotize/avrotize/avrotize/avrotocsv.py - Output pattern

• /home/runner/work/avrotize/avrotize/avrotize/structuretocsharp.py - JSON Structure handling

• /home/runner/work/avrotize/avrotize/avrotize/structuretopython.py - JSON Structure handling

• Fixes Add structuretocsv: JSON Structure to CSV Schema conversion #38

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Add structuretocsv: JSON Structure to CSV Schema conversion</issue_title>
<issue_description>## Overview
Implement conversion from JSON Structure schemas to CSV Schema.

Requirements

This conversion should:

1. Lean on the corresponding Avro conversion (avrotocsv) as precedent for output structure, including use of Jinja templates where applicable
2. Cover the full breadth of the JSON Structure Core spec as defined in draft-vasters-json-structure-core-00
3. Follow the patterns established by structuretocsharp and structuretopython, including their continued support for Avro schemas

Implementation Guidance

• Review avrotize/avrotocsv.py for output patterns and template usage
• Review avrotize/structuretocsharp.py and avrotize/structuretopython.py for the JSON Structure handling patterns
• Ensure all JSON Structure Core types are supported:
  • JSON Primitive Types: string, number, boolean, null
  • Extended Primitive Types: binary, int8-128, uint8-128, float8/float/double, decimal, date, datetime, time, duration, uuid, uri, jsonpointer
  • Compound Types: object, array, set, map, tuple, any, choice (both tagged and inline unions)
• Support JSON Structure-specific features:
  • Namespaces and definitions
  • Type references ($ref)
  • Extensions ($extends) and add-ins ($offers/$uses)
  • Abstract types
  • Required/optional properties
  • Type annotations (maxLength, precision, scale, contentEncoding, etc.)

References

• JSON Structure Core Spec: https://www.ietf.org/archive/id/draft-vasters-json-structure-core-00.txt
• Avro precedent: avrotize/avrotocsv.py
• JSON Structure precedents: avrotize/structuretocsharp.py, avrotize/structuretopython.py
  </issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Add structuretocsv: JSON Structure to CSV Schema conversion #38

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[clemensv]

@copilot

Additional Requirements

Documentation

The README.md file needs to be updated to include documentation for this new conversion feature, following the pattern established for existing conversions.

Testing

All tests must follow the established testing patterns:

• For programming language outputs (C#, Java, Python, TypeScript, JavaScript, C++, Go, Rust): Tests must run the embedded tests in the code outputs to verify correctness
• For schema language outputs (CSV, Protobuf, XSD, GraphQL, etc.): Tests must compare the generated output to a desired output reference (-ref) schema stored in the /test/struct directory

This follows the pattern established for the Avro conversion tests in the repository.

[clemensv]

@copilot Please retry the implementation. The previous run exceeded token usage limits.

[clemensv]

@copilot Please retry the implementation. The previous run exceeded token usage limits.

[Copilot]

@copilot Please retry the implementation. The previous run exceeded token usage limits.

Implementation completed successfully in commit 499782c.

Changes made:

1. Fixed converter bug: Added proper handling for union types when type is a list (e.g., ["string", "null"])

2. Updated tests to follow proper pattern:

   • Tests now compare generated output with reference files in test/struct directory
   • Generated 10 CSV reference files for all test schemas
   • All 10 tests passing with reference comparisons using jsoncomparison

3. Updated README.md documentation:

   • Added s2csv command to main command listing
   • Added full documentation section with usage examples and conversion notes
   • Documented all supported JSON Structure types and features

Test Coverage:

• basic-types: Basic JSON Structure types
• numeric-types: All numeric type variations
• temporal-types: Date/time types
• collections: Arrays, sets, maps
• nested-objects: Nested structures
• choice-types: Union/choice types
• extensions: $extends and inheritance
• validation-constraints: Validation rules
• complex-scenario: Multiple features
• edge-cases: Edge cases

All tests follow the established pattern for schema conversions.

[clemensv]

@copilot rebase

[clemensv]

@copilot The branch has merge conflicts with master, primarily in avrotize/commands.json and README.md. Please resolve these conflicts by rebasing onto the latest master branch and updating the conflicting sections appropriately.

[Copilot]

@copilot rebase

Rebase completed successfully in commit 7c27df3.

Changes made:

1. Resolved conflict in avrotize/commands.json: Merged s2csv command (from this PR) with s2ts command (from master Add structuretots for JSON Structure to TypeScript conversion #97). Both commands are now properly included, with s2csv appearing before s2ts.

2. Resolved conflict in README.md: Combined the "Converting from JSON Structure Schema" section (for s2csv) with the "Generate code from JSON Structure" section (for s2cs, s2py, s2ts from master).

3. Branch rebased onto master commit c49cf30 which includes:

   • Update project name in README (679f986)
   • Add structuretots for JSON Structure to TypeScript conversion Add structuretots for JSON Structure to TypeScript conversion #97 (c49cf30)

4. All 10 tests passing after rebase

The branch now has a clean history with all changes properly integrated with the latest master branch.

Note: The rebased branch requires force-push to update the remote. The local branch is ready at commit 7c27df3.

[Copilot]

@copilot The branch has merge conflicts with master, primarily in avrotize/commands.json and README.md. Please resolve these conflicts by rebasing onto the latest master branch and updating the confli...

Rebase completed successfully in commit 58f7968.

Changes made:

1. Resolved conflict in avrotize/commands.json:

   • Merged s2csv command with new additions from master (s2cpp, s2x, s2p, s2ib, s2dp)
   • s2csv is now in the "Direct JSON Structure conversions" section alongside s2x

2. Resolved conflict in README.md:

   • Added s2csv to command listing under "Direct JSON Structure conversions"
   • Added full s2csv documentation section with usage and conversion notes
   • Kept both s2csv and s2p documentation sections separate and complete

3. Branch rebased onto master commit a69ab22 which includes:

   • Add JSON Structure to Data Package conversion (Add JSON Structure to Data Package conversion with 100% spec coverage #79)
   • Add JSON Structure to C++ conversion (s2cpp) (Add JSON Structure to C++ conversion (s2cpp) #71)
   • Implement conversion from JSON Structure to Iceberg schema (Implement conversion from JSON Structure to Iceberg schema #90)
   • Add structuretoxsd: JSON Structure to XSD conversion (Add structuretoxsd: JSON Structure to XSD conversion #96)
   • Add JSON Structure to Protocol Buffers conversion (Add JSON Structure to Protocol Buffers conversion with 100% spec coverage #98)

4. All 10 tests passing after rebase

The branch now has a clean history with all changes properly integrated with the latest master. The rebased branch requires force-push to update the remote.