feat: NDJSON elements-file mode for partition (0.46.0) (#347)

## What

Adds an opt-in NDJSON response mode to `partition()` that returns
elements as a **path to a file on disk** instead of a parsed list, and
ships it as **0.46.0**.

```python
from unstructured_client.general import PartitionAcceptEnum

res = client.general.partition(
    request=req,
    accept_header_override=PartitionAcceptEnum.APPLICATION_X_NDJSON,
)

try:
    with open(res.elements_file, encoding="utf-8") as f:
        for line in f:
            element = json.loads(line)
            ...
finally:
    os.unlink(res.elements_file)
```

`PartitionResponse.elements_file` is set instead of
`PartitionResponse.elements`. **The caller owns the file and must delete
it.** Requesting `application/json` remains the default and is entirely
unchanged.

## Why

On the split-PDF path the SDK rebuilt the whole document in memory in
order to return it: a list per chunk, a flattened list, a `json.dumps`
blob in `create_response`, and then the SDK's re-parse of that blob —
four copies live at once, with the serialization step dominating peak
usage. For documents with large `metadata.image_base64` payloads this is
the difference between a job completing and being OOM-killed.

In the new mode the per-chunk temp files are concatenated on disk and
never parsed, so peak memory is roughly one chunk rather than the whole
document.

## How

- `combine_chunk_files_to_ndjson` concatenates chunk files on disk. Each
chunk is sniffed for its first non-whitespace character, so a server
returning `application/json` still works; chunks that are already NDJSON
are copied through without parsing.
- `ndjson_mode` depends **only** on the `Accept` header, never on
`split_pdf_cache_tmp_data`. Those are set by different parties, so
gating on both let them disagree — the server would return NDJSON while
the hook took the JSON path and `res.json()` raised on a body this
client had itself requested.
- Both caching modes are handled. A cached chunk contributes its
existing temp-file path; an uncached one spills its body verbatim and
then **releases** it, since every response is retained in
`api_successful_responses` and leaving `_content` set would keep the
document resident regardless.
- The combined file is deliberately written outside the operation's
`TemporaryDirectory`, which `_clear_operation` removes as soon as
`after_success` returns.

## Temp-file ownership

Everything this path creates is accounted for:

- Spilled chunk bodies are written inside the operation's temp directory
and unlinked once combined.
- The combined file is deleted when a chunk failure means it is never
handed back to the caller.
- Recombination writes to a staging file that is atomically renamed into
place only on success, so a malformed chunk cannot orphan a partial
file.
- No combined file is created at all when every chunk failed.

## Security

The elements-file marker is an **httpx response extension**, not a
response header. Extensions are populated by the transport, so a remote
server cannot set the key. A header would be wire-controlled, and since
callers are documented to open `elements_file` and then delete it, that
would hand a hostile server an arbitrary local file to destroy. A real
server body is always copied to a file this client creates.

## Regeneration

`elements_file` is client-side only and can never come from the OpenAPI
spec, so a regeneration would silently drop it. Both `general.py` and
`models/operations/partition.py` are now in `.genignore`, and
`test_regeneration_guards.py` fails if either entry is lost.

## Known limitation

`elements_file` is set for every input, so callers need one code path.
The **memory saving**, however, applies only to split PDFs.

An input is sent whole when it is not a PDF, when
`split_pdf_page=False`, or when it has two pages or fewer —
`_before_request_unlocked` short-circuits on `split_size >= page_count`
and `get_optimal_split_size` floors at `MIN_PAGES_PER_SPLIT = 2`. For
those, the body is read fully into memory before being written to disk,
so peak is roughly 2x the body rather than bounded.

Bounding it means `stream=True` for NDJSON requests, which makes
`raw_response.content` raise on the returned closed response — a
user-visible change worth its own review. Tracked separately.

Note also that the deployed API does not currently emit
`application/x-ndjson`, so the unsplit path reaches the JSON-to-NDJSON
conversion rather than the streamed-body branch. That is not merely a
spec omission: the service does not negotiate the response format on
`Accept` at all. It selects the format from the `output_format` form
field, and consults `Accept` only to choose `multipart/mixed` and to
reject conflicting media types on multi-file uploads. NDJSON was
therefore never going to arrive via `Accept`.

The service's `406 NOT_ACCEPTABLE` on an unrecognized `Accept` is gated
on multi-file uploads. This SDK sends a single file per request —
`PartitionParameters.files` is one `Files`, and the split-PDF hook sends
one chunk per request — so that branch is unreachable from here and the
unsplit path cannot raise `SDKError` because of it. Server-side NDJSON
support is tracked separately.

## Testing

- New `_test_unstructured_client/unit/test_ndjson_elements_file.py` —
recombination across JSON-array / NDJSON / mixed chunk formats, order
preservation, byte-exact payload round-trip, non-ASCII, temp-file
lifecycle on success and failure, and regression guards for the
header-spoofing and partial-output defects.
- 235 unit tests and 64 contract tests pass; `pylint` 10.00/10; `mypy`
clean.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/Unstructured-IO/unstructured-python-client/pull/347?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: badGarnet

.genignore

@@ -23,3 +23,19 @@ src/unstructured_client/general.py
 # Custom min_attempts / absolute_max_elapsed_time_ms fields on BackoffStrategy.
 # Push upstream to Speakeasy templates to remove this entry.
 src/unstructured_client/utils/retries.py
+
+# Custom elements_file field on PartitionResponse, for the NDJSON elements-file mode.
+# The field is client-side only - the server never returns it - so it cannot come from
+# the OpenAPI spec, and regenerating would drop it. If /general/v0/general gains a new
+# response field, follow the same procedure as general.py above.
+# See test_regeneration_guards.py::test_partition_response_keeps_elements_file.
+src/unstructured_client/models/operations/partition.py
+
+# Docs for that same custom elements_file field. Generated from the spec, so a
+# regeneration would drop the row.
+#
+# Note: SDK generation is currently blocked at the Speakeasy account level
+# ("generation access blocked"), so nothing can regenerate today and gen.lock has not
+# moved since 2026-01. These entries are insurance for when that is restored, not a
+# defence against an imminent run.
+docs/models/operations/partitionresponse.md

CHANGELOG.md

@@ -1,3 +1,8 @@
+## 0.46.0
+
+### Features
+* Add an NDJSON elements-file mode to `partition()`. Pass `accept_header_override=PartitionAcceptEnum.APPLICATION_X_NDJSON` to get `PartitionResponse.elements_file` — a path to an NDJSON file with one element per line — instead of `PartitionResponse.elements`. On the split-PDF path the per-chunk temp files are concatenated on disk rather than parsed, flattened, re-serialized with `json.dumps` and re-parsed by the SDK, which held four copies of the document in memory at once; peak memory becomes roughly one chunk instead of the whole document. `elements_file` is set for every input, including ones that are not split and responses from a server that ignores the `Accept` header, so callers need only one code path — but the memory saving itself applies only to split PDFs (a PDF of more than two pages, with `split_pdf_page=True`). **The caller owns the returned file and is responsible for deleting it.** Requesting `application/json` (the default) is unchanged.
+
 ## 0.45.0
 
 ### Features

README.md

@@ -427,6 +427,37 @@ req = operations.PartitionRequest(
 )
 ```
 
+### Streaming elements to a file instead of memory
+
+For very large documents, the parsed element list can dominate the client's memory: the split-PDF path holds a list per chunk, a flattened list, a serialized blob and the SDK's re-parse of that blob. Request `application/x-ndjson` to skip all of it. The chunk responses are concatenated on disk and you get back a path in `elements_file` instead of a list in `elements`, which keeps peak memory at roughly one chunk.
+
+**You own the returned file and are responsible for deleting it.**
+
+> [!NOTE]
+> `elements_file` is always set when you pass this header, but the **memory saving** only applies to the split-PDF path. An input is only split when it is a PDF, `split_pdf_page=True` (the default), and it has more than two pages — `split_size` is floored at 2, so one- and two-page PDFs are sent whole. For those, and for non-PDFs, the response body is read fully into memory before being written to disk, so peak memory can reach roughly twice the body size.
+
+Example:
+```python
+import json
+from pathlib import Path
+
+from unstructured_client.general import PartitionAcceptEnum
+
+res = client.general.partition(
+    request=req,
+    accept_header_override=PartitionAcceptEnum.APPLICATION_X_NDJSON,
+)
+
+try:
+    with open(res.elements_file, encoding="utf-8") as f:
+        for line in f:
+            element = json.loads(line)
+            ...
+finally:
+    # missing_ok so a failure to open the file isn't masked by the cleanup.
+    Path(res.elements_file).unlink(missing_ok=True)
+```
+
 <!-- Start File uploads [file-upload] -->
 ## File uploads
 

RELEASES.md

@@ -1241,3 +1241,13 @@ Based on:
 - [python v0.45.0] .
 ### Releases
 - [PyPI v0.45.0] https://pypi.org/project/unstructured-client/0.45.0 - .
+
+## 2026-08-01 00:00:00
+### Changes
+Based on:
+- OpenAPI Doc  
+- Speakeasy CLI 1.601.0 (2.680.0) https://github.com/speakeasy-api/speakeasy
+### Generated
+- [python v0.46.0] .
+### Releases
+- [PyPI v0.46.0] https://pypi.org/project/unstructured-client/0.46.0 - .