Skip to content

De/serialize from/to streams as well as bytes objects

Choose a tag to compare

@nat-goodspeed nat-goodspeed released this 22 Mar 20:53
· 67 commits to main since this release
0931046

The llsd module has acquired new write_binary(), write_notation(), write_xml() and write_pretty_xml() functions. Each of these accepts a binary stream and the Python object to be serialized. format_binary(), format_notation(), format_xml() and format_pretty_xml() now internally call the corresponding write_mumble() function with an io.BytesIO instance, returning the resulting content.

Especially for large data structures, this is potentially a big win. Empirically, with the previous format_xml() implementation, the overhead of many bytes-string allocations and copies as the output string grows and grows could be startling.

Similarly, llsd.parse() can now accept a binary stream as an alternative to a bytes object. Instead of the previous idiom:

with open(somefile, 'rb') as inf:
    data = llsd.parse(inf.read())

a caller can now write:

with open(somefile, 'rb') as inf:
    data = llsd.parse(inf)

avoiding the necessity to allocate a single contiguous memory buffer capable of holding the entire content of somefile.

In addition, the Python llsd parser now has improved compatibility with the C++ LLSD serializer.

What's Changed

New Features 🎉

  • SL-18330: Support parsing direct from stream; improve C++ compatibility. by @nat-goodspeed in #4
  • SL-19314: Recast llsd serialization to write to a stream. by @nat-goodspeed in #5

Other Changes

  • Publish benchmarks in job summary by @bennettgoble in #10
  • SL-18830: Fix sporadic notation parse failure with very large input. by @nat-goodspeed in #9
  • SL-18330: Refactor notation parsing to manage a lookahead char. by @nat-goodspeed in #6

New Contributors

Full Changelog: v1.1.0...v1.2.0