Skip to content

Commit 8a077d7

Browse files
paulkarayanpaulkarayanclaude
authored
fix: accept the Transform Platform API URL as server_url (0.46.2) (#352)
## What & why **Problem:** The Transform Platform's API Keys page hands you `https://platform-api.transform.unstructured.io/api/v1`, and the docs tell you to pass that value as `server_url`. Do it and every Platform call in this SDK fails with a 404: listing jobs, creating a workflow, checking a connector. The same URL works with curl, so the URL looks right and the SDK looks broken, and there is nothing in the error to point at the real cause. Anyone starting from the app's own copy button hits this on their first call. **Change:** Treat hosts under `unstructured.io` as Unstructured API hosts, so a copied `/api/v1` suffix is stripped from `server_url` the way it already was for `unstructuredapp.io`. Also clean the base URL for an operation-level `server_url=` override, which bypassed the cleaning hook entirely. ## Linked ticket none Client-facing follow-up: reported while writing the Transform Python quickstart, where every SDK sample had to be written against a URL different from the one the app displays. ## The bug Every Platform operation in this SDK already carries its own path prefix. `jobs.list_jobs` requests `/api/v1/jobs/`, `workflows.create_workflow` requests `/api/v1/workflows/`, and so on. So the base URL must not carry `/api/v1` of its own. `clean_server_url` exists to strip exactly that kind of pasted-in path, but it only did so when the host contained `unstructuredapp.io`: ```python if "unstructuredapp.io" in parsed_url.netloc: ... clean_url = urlunparse(parsed_url._replace(path="", ...)) else: # For other domains, we want to keep the path clean_url = urlunparse(parsed_url._replace(params="", query="", fragment="")) ``` `platform-api.transform.unstructured.io` does not match, so the path was kept and the operation path was appended on top, giving `/api/v1/api/v1/jobs/`, which matches no route. `basesdk.py` is generated, so the customization needs protecting: it is now in `.genignore`, the mechanism this repo already uses for `general.py`, `users.py`, `retries.py` and `partition.py`, with a guard test alongside the existing ones asserting that both the `_get_url` call and the `.genignore` entry survive. Without it a regeneration silently drops the fix and the doubled prefix returns. Freezing the file freezes the generated request, retry and hook plumbing too, so the entry carries the same un-freeze procedure `general.py` documents. Three smaller problems came out of the same code while fixing it. The host test was a substring match, so `unstructuredapp.io.example.com` was treated as one of ours and had its path stripped and its scheme forced to HTTPS; it is now matched on domain boundaries and left alone. A `server_url=` passed to a single operation never reached the cleaning hook at all, because the hook runs at SDK init; that override is now cleaned in `BaseSDK._get_url`, the one point every operation's base URL passes through. And a fully qualified host carrying the terminal root dot (`api.unstructuredapp.io.`) has to be recognized explicitly, since the old substring test matched it by accident and the domain-boundary test does not; the path is stripped as before and the host keeps its dot, which changes the Host header and SNI and is the caller's choice to make. ## What the patch changes, and what it does not Every `server_url` shape the existing tests, the docs and the app use, run through `clean_server_url` on `main` and on this branch. Seven results change; sixteen are byte-identical. | `server_url` | `main` | this branch | | | --- | --- | --- | --- | | `https://platform-api.transform.unstructured.io/api/v1` | `https://platform-api.transform.unstructured.io/api/v1` | `https://platform-api.transform.unstructured.io` | changed | | `http://platform-api.transform.unstructured.io/api/v1` | `http://platform-api.transform.unstructured.io/api/v1` | `https://platform-api.transform.unstructured.io` | changed | | `platform-api.transform.unstructured.io/api/v1` | `http://platform-api.transform.unstructured.io/api/v1` | `https://platform-api.transform.unstructured.io` | changed | | `platform-api.transform.unstructured.io` | `http://platform-api.transform.unstructured.io` | `https://platform-api.transform.unstructured.io` | changed | | `https://platform-api.unstructured.io/api/v1` | `https://platform-api.unstructured.io/api/v1` | `https://platform-api.unstructured.io` | changed | | `http://unstructuredapp.io.example.com/api/v1` | `https://unstructuredapp.io.example.com` | `http://unstructuredapp.io.example.com/api/v1` | changed | | `http://myunstructuredapp.io/api/v1` | `https://myunstructuredapp.io` | `http://myunstructuredapp.io/api/v1` | changed | | `https://platform-api.transform.unstructured.io` | `https://platform-api.transform.unstructured.io` | same | | | `https://platform.unstructuredapp.io/api/v1` | `https://platform.unstructuredapp.io` | same | | | `https://api.unstructuredapp.io/general/v0/general` | `https://api.unstructuredapp.io` | same | | | `unstructured-000mock.api.unstructuredapp.io/general/v0/general` | `https://unstructured-000mock.api.unstructuredapp.io` | same | | | `http://localhost:8000` | `http://localhost:8000` | same | | | `localhost:8000` | `http://localhost:8000` | same | | | `http://localhost:8000/my/endpoint/` | `http://localhost:8000/my/endpoint` | same | | | `localhost:8000/general/v0/general` | `http://localhost:8000/general/v0/general` | same | | | `https://unstructured.example.com/api/v1` | `https://unstructured.example.com/api/v1` | same | | | `http://not-unstructured.io/api/v1` | `http://not-unstructured.io/api/v1` | same | | The first five changed rows are the reported bug. The last two are the substring-match fix: those hosts are not ours, so they keep their path and their scheme. ## Impact **Customers:** Anyone using the Python SDK against the Transform Platform can now paste the API URL shown in the app, or set it from the documented `UNSTRUCTURED_API_URL`, and have jobs, workflows, sources, destinations and templates calls work. Today that exact value 404s on every call. Users who already worked around it by passing the bare host are unaffected; that keeps working. Users on `unstructuredapp.io` are unaffected; their URLs were already cleaned. **Internal (devs / ops / other teams):** The docs can stop steering readers away from the URL the product displays. No service imports this code; it is a client library published to PyPI. **Wire contract / clients:** No request or response shape changes. The only behavior change is which URL a request is sent to, and only for base URLs that were previously producing a doubled path. The one case where a user could notice a difference is a self-hosted deployment on a host under `unstructuredapp.io` or `unstructured.io` that genuinely serves the API beneath a subpath; that path is now stripped. Hosts outside those domains keep their path exactly as before, which the existing localhost subpath tests cover. **Deployment target considerations:** This is a PyPI client library, not a deployed service, so SaaS / DI / in-VPC / on-prem / SND deploys are unaffected. Air-gapped users pointing the SDK at their own hostname keep the existing keep-the-path behavior, since their host is not under an Unstructured domain. ## A note on the diff size The last commit is `ruff format` over the files this change touches, plus seven `noqa` directives for pre-existing lint that cannot be auto-fixed without changing behaviour. It is formatting only and carries no behaviour change, so reading the first two commits on their own gives you the whole fix. Two of the `noqa`s are worth knowing about: `raise err` in `basesdk.py` re-raises whatever an after-error hook returned, which is not always the active exception, so ruff's suggested bare `raise` would be a real bug. ## Risk / rollback Low. Small changes to URL normalization plus a `.genignore` entry, revert-safe, no migration and no flag. ## How it was verified Ran the unit suite on Python 3.11, 3.12 and 3.13 and the contract suite, plus `pylint` (10.00/10) and `mypy`, all green, matching what CI runs. `uv.lock` is unchanged, so the `UV_LOCKED=1` install holds. Reproduced the bug and then the fix against the live Transform Platform API without an API key, which is enough to tell the two apart: a route that exists answers 401, a route that does not answers 404. Not exercised with a real API key end to end, and not exercised against a self-hosted deployment. ## Proof Repro, against the live API, before the fix: ``` $ curl -s -o /dev/null -w '%{http_code}\n' https://platform-api.transform.unstructured.io/api/v1/jobs/ 401 $ curl -s -o /dev/null -w '%{http_code}\n' https://platform-api.transform.unstructured.io/api/v1/api/v1/jobs/ 404 ``` Through the SDK, before the fix: ``` server_url='https://platform-api.transform.unstructured.io' request sent: https://platform-api.transform.unstructured.io/api/v1/jobs/ status: 401 server_url='https://platform-api.transform.unstructured.io/api/v1' request sent: https://platform-api.transform.unstructured.io/api/v1/api/v1/jobs/ status: 404 ``` Failing tests at `HEAD` before the fix, `_test_unstructured_client/unit/test_server_urls.py::test_platform_request_url_has_a_single_api_prefix` plus the hook tests: ``` FAILED test_custom_hooks.py::test_unit_clean_server_url_fixes_malformed_transform_platform_url[https://platform-api.transform.unstructured.io/api/v1] FAILED test_custom_hooks.py::test_unit_clean_server_url_fixes_malformed_transform_platform_url[http://platform-api.transform.unstructured.io/api/v1] FAILED test_custom_hooks.py::test_unit_clean_server_url_fixes_malformed_transform_platform_url[platform-api.transform.unstructured.io/api/v1] FAILED test_custom_hooks.py::test_unit_clean_server_url_fixes_malformed_transform_platform_url[platform-api.transform.unstructured.io] FAILED test_custom_hooks.py::test_unit_clean_server_url_leaves_lookalike_domains_alone[http://unstructuredapp.io.example.com/api/v1] E Failed: transform platform ... Expected https://platform-api.transform.unstructured.io, got https://platform-api.transform.unstructured.io/api/v1 ``` After the fix, the same live check across both ways of passing the URL: ``` client-level, bare host sent: https://platform-api.transform.unstructured.io/api/v1/jobs/ status: 401 client-level, URL from the app (/api/v1) sent: https://platform-api.transform.unstructured.io/api/v1/jobs/ status: 401 operation-level, bare host sent: https://platform-api.transform.unstructured.io/api/v1/jobs/ status: 401 operation-level, URL from the app (/api/v1) sent: https://platform-api.transform.unstructured.io/api/v1/jobs/ status: 401 ``` Every case now reaches the real route. Suites after the fix: unit and contract both pass, `pylint` 10.00/10, `mypy` clean. ## Dependencies / merge order none ## Worked Example ```python from unstructured_client import UnstructuredClient # The value the app's API Keys page gives you, pasted as-is. client = UnstructuredClient( api_key_auth="YOUR_KEY", server_url="https://platform-api.transform.unstructured.io/api/v1", ) client.jobs.list_jobs(request={}) # before: GET https://platform-api.transform.unstructured.io/api/v1/api/v1/jobs/ -> 404 {"detail":"Not Found"} # after: GET https://platform-api.transform.unstructured.io/api/v1/jobs/, the real route ``` ## Release Bumped to 0.46.2 with CHANGELOG and RELEASES entries. --------- Co-authored-by: paulkarayan <pk@unstructured.io> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
1 parent e6cd1e0 commit 8a077d7

12 files changed

Lines changed: 337 additions & 331 deletions

File tree

.genignore

Lines changed: 0 additions & 41 deletions
This file was deleted.

CHANGELOG.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,8 @@
1+
## 0.46.2
2+
3+
### Fixes
4+
* Accept the API URL the Transform Platform hands you as `server_url`. The app's API Keys page and the docs give you `https://platform-api.transform.unstructured.io/api/v1`, which works with curl but 404'd every Platform call in the SDK: the URL cleaner only stripped a path for `unstructuredapp.io` hosts, so the `/api/v1` survived and the operation's own `/api/v1/jobs/` was appended on top, producing `/api/v1/api/v1/jobs/`. Hosts under `unstructured.io` are now recognized too, and are matched on domain boundaries so a lookalike host like `unstructuredapp.io.example.com` keeps its path and scheme. A `server_url` passed to an individual operation is cleaned as well — previously only the client-level URL was, so `client.jobs.list_jobs(request={}, server_url=...)` still 404'd.
5+
16
## 0.46.1
27

38
### Fixes

RELEASES.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1261,3 +1261,13 @@ Based on:
12611261
- [python v0.46.1] .
12621262
### Releases
12631263
- [PyPI v0.46.1] https://pypi.org/project/unstructured-client/0.46.1 - .
1264+
1265+
## 2026-08-22 00:00:00
1266+
### Changes
1267+
Based on:
1268+
- OpenAPI Doc
1269+
- Speakeasy CLI 1.601.0 (2.680.0) https://github.com/speakeasy-api/speakeasy
1270+
### Generated
1271+
- [python v0.46.2] .
1272+
### Releases
1273+
- [PyPI v0.46.2] https://pypi.org/project/unstructured-client/0.46.2 - .

_test_unstructured_client/unit/test_custom_hooks.py

Lines changed: 112 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -3,14 +3,15 @@
33
import logging
44
import re
55

6+
import httpx
67
import pytest
78
import requests
8-
import httpx
9-
from httpx import Response, ConnectError
9+
from httpx import ConnectError, Response
1010

1111
from _test_unstructured_client.unit_utils import FixtureRequest, Mock, method_mock
1212
from unstructured_client import UnstructuredClient
13-
from unstructured_client.models import shared, operations
13+
from unstructured_client._hooks.custom.clean_server_url_hook import clean_server_url
14+
from unstructured_client.models import operations, shared
1415
from unstructured_client.models.errors import SDKError
1516
from unstructured_client.utils.retries import BackoffStrategy, RetryConfig
1617

@@ -34,7 +35,10 @@ def test_unit_retry_with_backoff_does_retry(caplog):
3435

3536
def mock_post(request):
3637
request_count[0] += 1
37-
if request.url == "https://api.unstructuredapp.io/general/v0/general" and request.method == "POST":
38+
if (
39+
request.url == "https://api.unstructuredapp.io/general/v0/general"
40+
and request.method == "POST"
41+
):
3842
return Response(502, request=request)
3943

4044
transport = httpx.MockTransport(mock_post)
@@ -69,7 +73,10 @@ def test_unit_backoff_strategy_logs_retries_5XX(status_code: int, caplog):
6973
)
7074

7175
def mock_post(request):
72-
if request.url == "https://api.unstructuredapp.io/general/v0/general" and request.method == "POST":
76+
if (
77+
request.url == "https://api.unstructuredapp.io/general/v0/general"
78+
and request.method == "POST"
79+
):
7380
return Response(status_code, request=request)
7481

7582
transport = httpx.MockTransport(mock_post)
@@ -83,11 +90,13 @@ def mock_post(request):
8390
partition_parameters=shared.PartitionParameters(files=files)
8491
)
8592

86-
with pytest.raises(Exception):
93+
with pytest.raises(Exception): # noqa: B017
8794
session.general.partition(request=req, retries=retries)
8895

89-
pattern = re.compile(f"Failed to process a request due to API server error with status code {status_code}. "
90-
"Attempting retry number 1 after sleep.")
96+
pattern = re.compile(
97+
f"Failed to process a request due to API server error with status code {status_code}. "
98+
"Attempting retry number 1 after sleep."
99+
)
91100
assert bool(pattern.search(caplog.text))
92101

93102

@@ -103,9 +112,11 @@ def mock_post(request):
103112
[502, True],
104113
[503, True],
105114
[504, True],
106-
]
115+
],
107116
)
108-
def test_unit_number_of_retries_in_failed_requests(status_code: int, expect_retry: bool):
117+
def test_unit_number_of_retries_in_failed_requests(
118+
status_code: int, expect_retry: bool
119+
):
109120
filename = "README.md"
110121
backoff_strategy = BackoffStrategy(
111122
initial_interval=1, max_interval=10, exponent=1.5, max_elapsed_time=300
@@ -115,17 +126,19 @@ def test_unit_number_of_retries_in_failed_requests(status_code: int, expect_retr
115126
)
116127

117128
number_of_requests = [0]
129+
118130
def mock_post(request):
119-
if request.url == "https://api.unstructuredapp.io/general/v0/general" and request.method == "POST":
131+
if (
132+
request.url == "https://api.unstructuredapp.io/general/v0/general"
133+
and request.method == "POST"
134+
):
120135
number_of_requests[0] += 1
121136
return Response(status_code, request=request)
122137

123-
124138
transport = httpx.MockTransport(mock_post)
125139
client = httpx.Client(transport=transport)
126140
session = UnstructuredClient(api_key_auth=FAKE_KEY, client=client)
127141

128-
129142
with open(filename, "rb") as f:
130143
files = shared.Files(content=f.read(), file_name=filename)
131144

@@ -166,11 +179,13 @@ def mock_post(request):
166179
partition_parameters=shared.PartitionParameters(files=files)
167180
)
168181

169-
with pytest.raises(Exception):
182+
with pytest.raises(Exception): # noqa: B017
170183
session.general.partition(request=req, retries=retries)
171184

172-
pattern = re.compile("Failed to process a request due to transport error .*? "
173-
"Attempting retry number 1 after sleep.")
185+
pattern = re.compile(
186+
"Failed to process a request due to transport error .*? "
187+
"Attempting retry number 1 after sleep."
188+
)
174189
assert bool(pattern.search(caplog.text))
175190

176191

@@ -187,31 +202,93 @@ def mock_post(request):
187202
],
188203
)
189204
def test_unit_clean_server_url_fixes_malformed_paid_api_url(server_url: str):
190-
client = UnstructuredClient(
191-
server_url=server_url,
192-
api_key_auth=FAKE_KEY,
193-
)
194205
assert (
195-
client.general.sdk_configuration.server_url
206+
clean_server_url(server_url)
196207
== "https://unstructured-000mock.api.unstructuredapp.io"
197208
)
198209

199210

211+
@pytest.mark.parametrize(
212+
"server_url",
213+
[
214+
# -- the value the Transform Platform's API Keys page hands you --
215+
"https://platform-api.transform.unstructured.io/api/v1",
216+
"http://platform-api.transform.unstructured.io/api/v1",
217+
"platform-api.transform.unstructured.io/api/v1",
218+
# -- well-formed url --
219+
"https://platform-api.transform.unstructured.io",
220+
"platform-api.transform.unstructured.io",
221+
],
222+
)
223+
def test_unit_clean_server_url_fixes_malformed_transform_platform_url(server_url: str):
224+
assert (
225+
clean_server_url(server_url) == "https://platform-api.transform.unstructured.io"
226+
)
227+
228+
229+
@pytest.mark.parametrize(
230+
"server_url,expected_url",
231+
[
232+
# -- the terminal root dot is a valid fully qualified name and still ours, so
233+
# -- the path goes; the host is left exactly as the caller wrote it, because the
234+
# -- dot is a deliberate DNS choice that changes the Host header and SNI --
235+
(
236+
"https://platform-api.transform.unstructured.io./api/v1",
237+
"https://platform-api.transform.unstructured.io.",
238+
),
239+
(
240+
"https://unstructured-000mock.api.unstructuredapp.io./general/v0/general",
241+
"https://unstructured-000mock.api.unstructuredapp.io.",
242+
),
243+
],
244+
)
245+
def test_unit_clean_server_url_handles_a_fully_qualified_host(
246+
server_url: str, expected_url: str
247+
):
248+
assert clean_server_url(server_url) == expected_url
249+
250+
251+
@pytest.mark.parametrize(
252+
"server_url,expected_url",
253+
[
254+
# -- a host that merely CONTAINS an Unstructured domain is not ours, so its
255+
# -- path and scheme are left alone --
256+
(
257+
"http://unstructuredapp.io.example.com/api/v1",
258+
"http://unstructuredapp.io.example.com/api/v1",
259+
),
260+
(
261+
"http://not-unstructured.io/api/v1",
262+
"http://not-unstructured.io/api/v1",
263+
),
264+
],
265+
)
266+
def test_unit_clean_server_url_leaves_lookalike_domains_alone(
267+
server_url: str, expected_url: str
268+
):
269+
assert clean_server_url(server_url) == expected_url
270+
271+
200272
@pytest.mark.parametrize(
201273
"server_url,expected_url",
202274
[
203275
("http://localhost:8000", "http://localhost:8000"),
204276
("localhost:8000", "http://localhost:8000"),
205-
("localhost:8000/general/v0/general", "http://localhost:8000/general/v0/general"),
206-
("http://localhost:8000/general/v0/general", "http://localhost:8000/general/v0/general"),
277+
(
278+
"localhost:8000/general/v0/general",
279+
"http://localhost:8000/general/v0/general",
280+
),
281+
(
282+
"http://localhost:8000/general/v0/general",
283+
"http://localhost:8000/general/v0/general",
284+
),
207285
],
208286
)
209-
def test_unit_clean_server_url_fixes_non_unst_domain_url(server_url: str, expected_url: str):
210-
client = UnstructuredClient(
211-
server_url=server_url,
212-
api_key_auth=FAKE_KEY,
213-
)
214-
assert client.general.sdk_configuration.server_url == expected_url
287+
def test_unit_clean_server_url_fixes_non_unst_domain_url(
288+
server_url: str, expected_url: str
289+
):
290+
assert clean_server_url(server_url) == expected_url
291+
215292

216293
@pytest.mark.parametrize(
217294
"server_url",
@@ -222,10 +299,11 @@ def test_unit_clean_server_url_fixes_non_unst_domain_url(server_url: str, expect
222299
"unstructured-000mock.api.unstructuredapp.io/general/v0/general",
223300
],
224301
)
225-
def test_unit_clean_server_url_fixes_malformed_urls_with_positional_arguments(server_url: str):
226-
client = UnstructuredClient(FAKE_KEY, server_url=server_url)
302+
def test_unit_clean_server_url_fixes_malformed_urls_with_positional_arguments(
303+
server_url: str,
304+
):
227305
assert (
228-
client.general.sdk_configuration.server_url
306+
clean_server_url(server_url)
229307
== "https://unstructured-000mock.api.unstructuredapp.io"
230308
)
231309

@@ -247,11 +325,8 @@ def mock_post(request):
247325
)
248326
with pytest.raises(SDKError, match="API error occurred: Status 401"):
249327
session.general.partition(request=req)
250-
251-
assert any(
252-
"Server responded with 401"
253-
in message for message in caplog.messages
254-
)
328+
329+
assert any("Server responded with 401" in message for message in caplog.messages)
255330

256331

257332
# -- fixtures --------------------------------------------------------------------------------

0 commit comments

Comments
 (0)