Fix: Can't use curl to download a single manifest in one invocation (#5918) #6099
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dsotirho-ucsc
force-pushed
the
issues/dsotirho-ucsc/5918-manifest-post
branch
2 times, most recently
from
8b62dbc to
830a1e4
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #6099 +/- ##
===========================================
- Coverage 85.22% 85.20% -0.03%
===========================================
Files 156 156
Lines 22408 22451 +43
===========================================
+ Hits 19098 19130 +32
- Misses 3310 3321 +11 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
dsotirho-ucsc
force-pushed
the
issues/dsotirho-ucsc/5918-manifest-post
branch
2 times, most recently
from
da860a5 to
b81d052
Compare
dsotirho-ucsc
force-pushed
the
issues/dsotirho-ucsc/5918-manifest-post
branch
5 times, most recently
from
9fb4dbb to
391cedb
Compare
|
Successful use of curl to download a manifest in one invocation: (Unabridged version: 6099_manifest_invocation_unabridged.txt) |
nadove-ucsc
requested changes
Apr 1, 2024
lambdas/service/app.py
Outdated
|
|
||
| def manifest_route(*, fetch: bool, initiate: bool): | ||
| def manifest_route(*, fetch: bool, method: str): | ||
| initiate = method in ['PUT', 'POST'] |
There was a problem hiding this comment.
Suggested change
| initiate = method in ['PUT', 'POST'] | |
| initiate = method in {'PUT', 'POST'} |
lambdas/service/app.py
Outdated
|
|
||
| def _file_manifest(fetch: bool, token_or_key: Optional[str] = None): | ||
| request = app.current_request | ||
| require(request.method != 'POST' or request.raw_body.decode() == '', |
There was a problem hiding this comment.
This is the same as
Suggested change
| require(request.method != 'POST' or request.raw_body.decode() == '', | |
| require(request.method != 'POST' or request.raw_body == b'', |
Right?
|
Added #6003 as a blocker due to overlapping changes to the IT |
dsotirho-ucsc
force-pushed
the
issues/dsotirho-ucsc/5918-manifest-post
branch
from
600eb8b to
81a4905
Compare
nadove-ucsc
reviewed
Apr 4, 2024
dsotirho-ucsc
force-pushed
the
issues/dsotirho-ucsc/5918-manifest-post
branch
from
81a4905 to
ff39020
Compare
hannes-ucsc
force-pushed
the
issues/dsotirho-ucsc/5918-manifest-post
branch
from
ec5f5cc to
cb02ca7
Compare
hannes-ucsc
requested changes
Sep 4, 2024
| elif wait == '1': | ||
| time_slept = self.server_side_sleep(float(retry_after)) | ||
| retry_after = round(retry_after - time_slept) | ||
| retry_after = ceil(retry_after - time_slept) |
There was a problem hiding this comment.
The change from round to ceil is currently in a commit labeled "Add a default value for the /repository/files wait parameter". Please explain why going from round to ceil is related to adding a default or isolate that change in its own commit.
There was a problem hiding this comment.
Moved change to commit Fix rounding of /repository/file retry-after value
Comment on lines
-1060
to
+830
| as a property of a JSON object in the body of the request. This can be | ||
| useful in case the value of the `filters` query parameter causes the URL | ||
| to exceed the maximum length of 8192 characters, resulting in a 413 | ||
| Request Entity Too Large response. | ||
| as a property of a JSON object in the body of the request. This is | ||
| referred to as *parameter hoisting* and can be useful in case the value | ||
| of the `filters` query parameter causes the URL to exceed the maximum | ||
| length of 8192 characters, resulting in a 413 Request Entity Too Large | ||
| response. | ||
| The request `%s %s?filters={…}`, for example, is equivalent to `%s %s` | ||
| with the body `{"filters": "{…}"}` in which any double quotes or | ||
| backslash characters inside `…` are escaped with another backslash. That | ||
| escaping is the requisite procedure for embedding one JSON structure | ||
| inside another. | ||
| with a `Content-Type` header of `application/json` and the body | ||
| `{"filters": "{…}"}` in which any double quotes or backslash characters | ||
| inside `…` are escaped with another backslash. That escaping is the | ||
| requisite procedure for embedding one JSON structure inside another. |
There was a problem hiding this comment.
Correct me if I am wrong but it seems like these two hunks are unrelated documentation improvement that I requested at some point, but that have nothing to do with "Add[ing] support for POST requests to the manifest endpoint", as is the title of the commit the hunks are part of.
There was a problem hiding this comment.
Moved change to commit Refine parameter hoisting note
dsotirho-ucsc
force-pushed
the
issues/dsotirho-ucsc/5918-manifest-post
branch
from
cb02ca7 to
f1a03b6
Compare
dsotirho-ucsc
force-pushed
the
issues/dsotirho-ucsc/5918-manifest-post
branch
from
0d778e1 to
16f80de
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
dsotirho-ucsc
force-pushed
the
issues/dsotirho-ucsc/5918-manifest-post
branch
from
16f80de to
c923904
Compare
14 tasks
dsotirho-ucsc
force-pushed
the
issues/dsotirho-ucsc/5918-manifest-post
branch
2 times, most recently
from
e050a15 to
e69f45a
Compare
Fixes regression from fb58b01. The problem manifested as an error with the `app` property in an AppController subclass, and this wasn't noticed until now due to the `app` property not being accessed in a controller that had this problem, namely ManifestController.
…vocation (#5918) Add support for POST requests to the manifest endpoint
…vocation (#5918) Add a wait parameter option to the manifest endpoint
dsotirho-ucsc
force-pushed
the
issues/dsotirho-ucsc/5918-manifest-post
branch
from
e69f45a to
6d3c890
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Linked issues: #5918
Checklist
Author
developissues/<GitHub handle of author>/<issue#>-<slug>1 when the issue title describes a problem, the corresponding PR
title is
Fix:followed by the issue titleAuthor (partiality)
ptag to titles of partial commitspartialor completely resolves all linked issuespartiallabelAuthor (reindex)
rtag to commit title or the changes introduced by this PR will not require reindexing of any deploymentreindex:devor the changes introduced by it will not require reindexing ofdevreindex:anvildevor the changes introduced by it will not require reindexing ofanvildevreindex:anvilprodor the changes introduced by it will not require reindexing ofanvilprodreindex:prodor the changes introduced by it will not require reindexing ofprodreindex:partialand its description documents the specific reindexing procedure fordev,anvildev,anvilprodandprodor requires a full reindex or carries none of the labelsreindex:dev,reindex:anvildev,reindex:anvilprodandreindex:prodAuthor (API changes)
APIor this PR does not modify a REST APIa(A) tag to commit title for backwards (in)compatible changes or this PR does not modify a REST APIapp.pyor this PR does not modify a REST APIAuthor (upgrading deployments)
make docker_images.jsonand committed the resulting changes or this PR does not modifyazul_docker_images, or any other variables referenced in the definition of that variableutag to commit title or this PR does not require upgrading deploymentsupgradeor does not require upgrading deploymentsdeploy:sharedor does not modifydocker_images.json, and does not require deploying thesharedcomponent for any other reasondeploy:gitlabor does not require deploying thegitlabcomponentdeploy:runneror does not require deploying therunnerimageAuthor (hotfixes)
Ftag to main commit title or this PR does not include permanent fix for a temporary hotfixanvilprodandprod) have temporary hotfixes for any of the issues linked to this PRAuthor (before every review)
develop, squashed fixups from prior reviewsmake requirements_updateor this PR does not modifyrequirements*.txt,common.mk,Makefile,Dockerfileorenvironment.bootRtag to commit title or this PR does not modifyrequirements*.txtreqsor does not modifyrequirements*.txtmake integration_testpasses in personal deployment or this PR does not modify functionality that could affect the IT outcomePeer reviewer (after approval)
Note that when requesting changes, the PR must be assigned back to the author.
System administrator (after approval)
demoorno demono demono sandboxN reviewslabel is accurateOperator
reindex:…labels andrcommit title tagno demodevelopOperator (deploy
.sharedand.gitlabcomponents)_select dev.shared && CI_COMMIT_REF_NAME=develop make -C terraform/shared apply_keep_unusedor this PR is not labeleddeploy:shared_select dev.gitlab && CI_COMMIT_REF_NAME=develop make -C terraform/gitlab applyor this PR is not labeleddeploy:gitlab_select anvildev.shared && CI_COMMIT_REF_NAME=develop make -C terraform/shared apply_keep_unusedor this PR is not labeleddeploy:shared_select anvildev.gitlab && CI_COMMIT_REF_NAME=develop make -C terraform/gitlab applyor this PR is not labeleddeploy:gitlabdeploy:gitlabdeploy:gitlabSystem administrator (post-deploy of
.gitlabcomponent)dev.gitlabare complete or this PR is not labeleddeploy:gitlabanvildev.gitlabare complete or this PR is not labeleddeploy:gitlabOperator (deploy runner image)
_select dev.gitlab && make -C terraform/gitlab/runneror this PR is not labeleddeploy:runner_select anvildev.gitlab && make -C terraform/gitlab/runneror this PR is not labeleddeploy:runnerOperator (sandbox build)
sandboxlabel or PR is labeledno sandboxdevor PR is labeledno sandboxanvildevor PR is labeledno sandboxsandboxdeployment or PR is labeledno sandboxanvilboxdeployment or PR is labeledno sandboxsandboxdeployment or PR is labeledno sandboxanvilboxdeployment or PR is labeledno sandboxsandboxor this PR does not remove catalogs or otherwise causes unreferenced indices indevanvilboxor this PR does not remove catalogs or otherwise causes unreferenced indices inanvildevsandboxor this PR is not labeledreindex:devanvilboxor this PR is not labeledreindex:anvildevsandboxor this PR is not labeledreindex:devanvilboxor this PR is not labeledreindex:anvildevOperator (merge the branch)
pif the PR is also labeledpartialOperator (main build)
devanvildevdevdevanvildevanvildev_select dev.shared && make -C terraform/shared applyor this PR is not labeleddeploy:shared_select anvildev.shared && make -C terraform/shared applyor this PR is not labeleddeploy:shareddevanvildevOperator (reindex)
devor this PR is neither labeledreindex:partialnorreindex:devanvildevor this PR is neither labeledreindex:partialnorreindex:anvildevdevor this PR is neither labeledreindex:partialnorreindex:devanvildevor this PR is neither labeledreindex:partialnorreindex:anvildevdevor this PR is neither labeledreindex:partialnorreindex:devanvildevor this PR is neither labeledreindex:partialnorreindex:anvildevdevor this PR does not require reindexingdevanvildevor this PR does not require reindexinganvildevdevor this PR does not require reindexingdevanvildevor this PR does not require reindexinganvildevdevor this PR does not require reindexingdevanvildevor this PR does not require reindexinganvildevdevor this PR does not require reindexingdevdevor this PR does not require reindexingdevdeploy_browserjob in the GitLab pipeline for this PR indevor this PR does not require reindexingdevanvildevor this PR does not require reindexinganvildevdeploy_browserjob in the GitLab pipeline for this PR inanvildevor this PR does not require reindexinganvildevOperator (mirroring)
devor this PR does not require mirroringdevanvildevor this PR does not require mirroringanvildevdevor this PR does not require mirroringdevanvildevor this PR does not require mirroringanvildevdevor this PR does not require mirroringdevanvildevor this PR does not require mirroringanvildevOperator
deploy:shared,deploy:gitlab,deploy:runner,API,reindex:partial,reindex:anvilprodandreindex:prodlabels to the next promotion PRs or this PR carries none of these labelsdeploy:shared,deploy:gitlab,deploy:runner,API,reindex:partial,reindex:anvilprodandreindex:prodlabels, from the description of this PR to that of the next promotion PRs or this PR carries none of these labelsShorthand for review comments
Lline is too longWline wrapping is wrongQbad quotesFother formatting problem