Commit 8a077d7
fix: accept the Transform Platform API URL as server_url (0.46.2) (#352)
## What & why
**Problem:** The Transform Platform's API Keys page hands you
`https://platform-api.transform.unstructured.io/api/v1`, and the docs
tell you to pass that value as `server_url`. Do it and every Platform
call in this SDK fails with a 404: listing jobs, creating a workflow,
checking a connector. The same URL works with curl, so the URL looks
right and the SDK looks broken, and there is nothing in the error to
point at the real cause. Anyone starting from the app's own copy button
hits this on their first call.
**Change:** Treat hosts under `unstructured.io` as Unstructured API
hosts, so a copied `/api/v1` suffix is stripped from `server_url` the
way it already was for `unstructuredapp.io`. Also clean the base URL for
an operation-level `server_url=` override, which bypassed the cleaning
hook entirely.
## Linked ticket
none
Client-facing follow-up: reported while writing the Transform Python
quickstart, where every SDK sample had to be written against a URL
different from the one the app displays.
## The bug
Every Platform operation in this SDK already carries its own path
prefix. `jobs.list_jobs` requests `/api/v1/jobs/`,
`workflows.create_workflow` requests `/api/v1/workflows/`, and so on. So
the base URL must not carry `/api/v1` of its own.
`clean_server_url` exists to strip exactly that kind of pasted-in path,
but it only did so when the host contained `unstructuredapp.io`:
```python
if "unstructuredapp.io" in parsed_url.netloc:
...
clean_url = urlunparse(parsed_url._replace(path="", ...))
else:
# For other domains, we want to keep the path
clean_url = urlunparse(parsed_url._replace(params="", query="", fragment=""))
```
`platform-api.transform.unstructured.io` does not match, so the path was
kept and the operation path was appended on top, giving
`/api/v1/api/v1/jobs/`, which matches no route.
`basesdk.py` is generated, so the customization needs protecting: it is
now in `.genignore`, the mechanism this repo already uses for
`general.py`, `users.py`, `retries.py` and `partition.py`, with a guard
test alongside the existing ones asserting that both the `_get_url` call
and the `.genignore` entry survive. Without it a regeneration silently
drops the fix and the doubled prefix returns. Freezing the file freezes
the generated request, retry and hook plumbing too, so the entry carries
the same un-freeze procedure `general.py` documents.
Three smaller problems came out of the same code while fixing it. The
host test was a substring match, so `unstructuredapp.io.example.com` was
treated as one of ours and had its path stripped and its scheme forced
to HTTPS; it is now matched on domain boundaries and left alone. A
`server_url=` passed to a single operation never reached the cleaning
hook at all, because the hook runs at SDK init; that override is now
cleaned in `BaseSDK._get_url`, the one point every operation's base URL
passes through. And a fully qualified host carrying the terminal root
dot (`api.unstructuredapp.io.`) has to be recognized explicitly, since
the old substring test matched it by accident and the domain-boundary
test does not; the path is stripped as before and the host keeps its
dot, which changes the Host header and SNI and is the caller's choice to
make.
## What the patch changes, and what it does not
Every `server_url` shape the existing tests, the docs and the app use,
run through `clean_server_url` on `main` and on this branch. Seven
results change; sixteen are byte-identical.
| `server_url` | `main` | this branch | |
| --- | --- | --- | --- |
| `https://platform-api.transform.unstructured.io/api/v1` |
`https://platform-api.transform.unstructured.io/api/v1` |
`https://platform-api.transform.unstructured.io` | changed |
| `http://platform-api.transform.unstructured.io/api/v1` |
`http://platform-api.transform.unstructured.io/api/v1` |
`https://platform-api.transform.unstructured.io` | changed |
| `platform-api.transform.unstructured.io/api/v1` |
`http://platform-api.transform.unstructured.io/api/v1` |
`https://platform-api.transform.unstructured.io` | changed |
| `platform-api.transform.unstructured.io` |
`http://platform-api.transform.unstructured.io` |
`https://platform-api.transform.unstructured.io` | changed |
| `https://platform-api.unstructured.io/api/v1` |
`https://platform-api.unstructured.io/api/v1` |
`https://platform-api.unstructured.io` | changed |
| `http://unstructuredapp.io.example.com/api/v1` |
`https://unstructuredapp.io.example.com` |
`http://unstructuredapp.io.example.com/api/v1` | changed |
| `http://myunstructuredapp.io/api/v1` | `https://myunstructuredapp.io`
| `http://myunstructuredapp.io/api/v1` | changed |
| `https://platform-api.transform.unstructured.io` |
`https://platform-api.transform.unstructured.io` | same | |
| `https://platform.unstructuredapp.io/api/v1` |
`https://platform.unstructuredapp.io` | same | |
| `https://api.unstructuredapp.io/general/v0/general` |
`https://api.unstructuredapp.io` | same | |
| `unstructured-000mock.api.unstructuredapp.io/general/v0/general` |
`https://unstructured-000mock.api.unstructuredapp.io` | same | |
| `http://localhost:8000` | `http://localhost:8000` | same | |
| `localhost:8000` | `http://localhost:8000` | same | |
| `http://localhost:8000/my/endpoint/` |
`http://localhost:8000/my/endpoint` | same | |
| `localhost:8000/general/v0/general` |
`http://localhost:8000/general/v0/general` | same | |
| `https://unstructured.example.com/api/v1` |
`https://unstructured.example.com/api/v1` | same | |
| `http://not-unstructured.io/api/v1` |
`http://not-unstructured.io/api/v1` | same | |
The first five changed rows are the reported bug. The last two are the
substring-match fix: those hosts are not ours, so they keep their path
and their scheme.
## Impact
**Customers:** Anyone using the Python SDK against the Transform
Platform can now paste the API URL shown in the app, or set it from the
documented `UNSTRUCTURED_API_URL`, and have jobs, workflows, sources,
destinations and templates calls work. Today that exact value 404s on
every call. Users who already worked around it by passing the bare host
are unaffected; that keeps working. Users on `unstructuredapp.io` are
unaffected; their URLs were already cleaned.
**Internal (devs / ops / other teams):** The docs can stop steering
readers away from the URL the product displays. No service imports this
code; it is a client library published to PyPI.
**Wire contract / clients:** No request or response shape changes. The
only behavior change is which URL a request is sent to, and only for
base URLs that were previously producing a doubled path. The one case
where a user could notice a difference is a self-hosted deployment on a
host under `unstructuredapp.io` or `unstructured.io` that genuinely
serves the API beneath a subpath; that path is now stripped. Hosts
outside those domains keep their path exactly as before, which the
existing localhost subpath tests cover.
**Deployment target considerations:** This is a PyPI client library, not
a deployed service, so SaaS / DI / in-VPC / on-prem / SND deploys are
unaffected. Air-gapped users pointing the SDK at their own hostname keep
the existing keep-the-path behavior, since their host is not under an
Unstructured domain.
## A note on the diff size
The last commit is `ruff format` over the files this change touches,
plus seven `noqa` directives for pre-existing lint that cannot be
auto-fixed without changing behaviour. It is formatting only and carries
no behaviour change, so reading the first two commits on their own gives
you the whole fix. Two of the `noqa`s are worth knowing about: `raise
err` in `basesdk.py` re-raises whatever an after-error hook returned,
which is not always the active exception, so ruff's suggested bare
`raise` would be a real bug.
## Risk / rollback
Low. Small changes to URL normalization plus a `.genignore` entry,
revert-safe, no migration and no flag.
## How it was verified
Ran the unit suite on Python 3.11, 3.12 and 3.13 and the contract suite,
plus `pylint` (10.00/10) and `mypy`, all green, matching what CI runs.
`uv.lock` is unchanged, so the `UV_LOCKED=1` install holds. Reproduced
the bug and then the fix against the live Transform Platform API without
an API key, which is enough to tell the two apart: a route that exists
answers 401, a route that does not answers 404. Not exercised with a
real API key end to end, and not exercised against a self-hosted
deployment.
## Proof
Repro, against the live API, before the fix:
```
$ curl -s -o /dev/null -w '%{http_code}\n' https://platform-api.transform.unstructured.io/api/v1/jobs/
401
$ curl -s -o /dev/null -w '%{http_code}\n' https://platform-api.transform.unstructured.io/api/v1/api/v1/jobs/
404
```
Through the SDK, before the fix:
```
server_url='https://platform-api.transform.unstructured.io'
request sent: https://platform-api.transform.unstructured.io/api/v1/jobs/
status: 401
server_url='https://platform-api.transform.unstructured.io/api/v1'
request sent: https://platform-api.transform.unstructured.io/api/v1/api/v1/jobs/
status: 404
```
Failing tests at `HEAD` before the fix,
`_test_unstructured_client/unit/test_server_urls.py::test_platform_request_url_has_a_single_api_prefix`
plus the hook tests:
```
FAILED test_custom_hooks.py::test_unit_clean_server_url_fixes_malformed_transform_platform_url[https://platform-api.transform.unstructured.io/api/v1]
FAILED test_custom_hooks.py::test_unit_clean_server_url_fixes_malformed_transform_platform_url[http://platform-api.transform.unstructured.io/api/v1]
FAILED test_custom_hooks.py::test_unit_clean_server_url_fixes_malformed_transform_platform_url[platform-api.transform.unstructured.io/api/v1]
FAILED test_custom_hooks.py::test_unit_clean_server_url_fixes_malformed_transform_platform_url[platform-api.transform.unstructured.io]
FAILED test_custom_hooks.py::test_unit_clean_server_url_leaves_lookalike_domains_alone[http://unstructuredapp.io.example.com/api/v1]
E Failed: transform platform ... Expected https://platform-api.transform.unstructured.io, got https://platform-api.transform.unstructured.io/api/v1
```
After the fix, the same live check across both ways of passing the URL:
```
client-level, bare host
sent: https://platform-api.transform.unstructured.io/api/v1/jobs/
status: 401
client-level, URL from the app (/api/v1)
sent: https://platform-api.transform.unstructured.io/api/v1/jobs/
status: 401
operation-level, bare host
sent: https://platform-api.transform.unstructured.io/api/v1/jobs/
status: 401
operation-level, URL from the app (/api/v1)
sent: https://platform-api.transform.unstructured.io/api/v1/jobs/
status: 401
```
Every case now reaches the real route. Suites after the fix: unit and
contract both pass, `pylint` 10.00/10, `mypy` clean.
## Dependencies / merge order
none
## Worked Example
```python
from unstructured_client import UnstructuredClient
# The value the app's API Keys page gives you, pasted as-is.
client = UnstructuredClient(
api_key_auth="YOUR_KEY",
server_url="https://platform-api.transform.unstructured.io/api/v1",
)
client.jobs.list_jobs(request={})
# before: GET https://platform-api.transform.unstructured.io/api/v1/api/v1/jobs/ -> 404 {"detail":"Not Found"}
# after: GET https://platform-api.transform.unstructured.io/api/v1/jobs/, the real route
```
## Release
Bumped to 0.46.2 with CHANGELOG and RELEASES entries.
---------
Co-authored-by: paulkarayan <pk@unstructured.io>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>1 parent e6cd1e0 commit 8a077d7
12 files changed
Lines changed: 337 additions & 331 deletions
File tree
- _test_unstructured_client/unit
- src/unstructured_client
- _hooks
- custom
This file was deleted.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
1 | 6 | | |
2 | 7 | | |
3 | 8 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1261 | 1261 | | |
1262 | 1262 | | |
1263 | 1263 | | |
| 1264 | + | |
| 1265 | + | |
| 1266 | + | |
| 1267 | + | |
| 1268 | + | |
| 1269 | + | |
| 1270 | + | |
| 1271 | + | |
| 1272 | + | |
| 1273 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
| 6 | + | |
6 | 7 | | |
7 | 8 | | |
8 | | - | |
9 | | - | |
| 9 | + | |
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
13 | | - | |
| 13 | + | |
| 14 | + | |
14 | 15 | | |
15 | 16 | | |
16 | 17 | | |
| |||
34 | 35 | | |
35 | 36 | | |
36 | 37 | | |
37 | | - | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
38 | 42 | | |
39 | 43 | | |
40 | 44 | | |
| |||
69 | 73 | | |
70 | 74 | | |
71 | 75 | | |
72 | | - | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
73 | 80 | | |
74 | 81 | | |
75 | 82 | | |
| |||
83 | 90 | | |
84 | 91 | | |
85 | 92 | | |
86 | | - | |
| 93 | + | |
87 | 94 | | |
88 | 95 | | |
89 | | - | |
90 | | - | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
91 | 100 | | |
92 | 101 | | |
93 | 102 | | |
| |||
103 | 112 | | |
104 | 113 | | |
105 | 114 | | |
106 | | - | |
| 115 | + | |
107 | 116 | | |
108 | | - | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
109 | 120 | | |
110 | 121 | | |
111 | 122 | | |
| |||
115 | 126 | | |
116 | 127 | | |
117 | 128 | | |
| 129 | + | |
118 | 130 | | |
119 | | - | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
120 | 135 | | |
121 | 136 | | |
122 | 137 | | |
123 | | - | |
124 | 138 | | |
125 | 139 | | |
126 | 140 | | |
127 | 141 | | |
128 | | - | |
129 | 142 | | |
130 | 143 | | |
131 | 144 | | |
| |||
166 | 179 | | |
167 | 180 | | |
168 | 181 | | |
169 | | - | |
| 182 | + | |
170 | 183 | | |
171 | 184 | | |
172 | | - | |
173 | | - | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
174 | 189 | | |
175 | 190 | | |
176 | 191 | | |
| |||
187 | 202 | | |
188 | 203 | | |
189 | 204 | | |
190 | | - | |
191 | | - | |
192 | | - | |
193 | | - | |
194 | 205 | | |
195 | | - | |
| 206 | + | |
196 | 207 | | |
197 | 208 | | |
198 | 209 | | |
199 | 210 | | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
200 | 272 | | |
201 | 273 | | |
202 | 274 | | |
203 | 275 | | |
204 | 276 | | |
205 | | - | |
206 | | - | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
207 | 285 | | |
208 | 286 | | |
209 | | - | |
210 | | - | |
211 | | - | |
212 | | - | |
213 | | - | |
214 | | - | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
215 | 292 | | |
216 | 293 | | |
217 | 294 | | |
| |||
222 | 299 | | |
223 | 300 | | |
224 | 301 | | |
225 | | - | |
226 | | - | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
227 | 305 | | |
228 | | - | |
| 306 | + | |
229 | 307 | | |
230 | 308 | | |
231 | 309 | | |
| |||
247 | 325 | | |
248 | 326 | | |
249 | 327 | | |
250 | | - | |
251 | | - | |
252 | | - | |
253 | | - | |
254 | | - | |
| 328 | + | |
| 329 | + | |
255 | 330 | | |
256 | 331 | | |
257 | 332 | | |
| |||
0 commit comments