[WIP - Don't review]Patch REST get_aggregated_tensor api for batched changes #1653
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ishaileshpant
changed the title
ishaileshpant
requested review from
MasterSkepticista,
aayushgaintel,
gbikkiintel,
kminhta,
noopurintel,
pasokan-intel,
payalcha,
psfoley,
rahulga1,
rajithkrishnegowda,
srikanthenugul,
tanwarsh,
teoparvanov and
theakshaypant
as code owners
ishaileshpant
force-pushed
the
batched_fetch_restapi_fix
branch
2 times, most recently
from
c13a40b to
d6c93c4
Compare
* code changes Signed-off-by: yes <[email protected]> * remove workspace Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> * code changes Signed-off-by: yes <[email protected]> --------- Signed-off-by: yes <[email protected]>
…atedai#1500) - add new AggregatorClientInterface to allow switching b/w grpc and rest - endhance existing AggregatorGRPCClient to start using AggregatorClientInterface - added new transport package for rest with AggregatorRESTClient implementing AggregatorClientInterface - added streaming api support with custom content-type - added various connection flag for streaming request - send additional header key "Sender" for better request logging at server side - aligned Rest and gRPC client for most of the init params - added AggregatorRESTServer and necesary changes in aggregator cli and federated/plan get_server method - added transport_protocol settings in defaults/network.yaml, defaulted the same to 'grpc' - reduced cyclomatic complexity of Rest Server - fixed protobuf streaming issue for v1/task/results API - added more detailed logging for task progression and metadata for each api calls - pinned Flask version to latest stable 3.1.0 - addressing review comments - 13th-May - added ping api and `collaborato` constructor hint for `AggregatorClientInterface` - added send_message_to_server in client and AggregatorClientInterface, Rest Server is already at parity - changed base uri for REST server to 'experimental/v1', adjusted the client and tests accordingly - fixed issue related to mTLS in REST server/client - disabled TLS 1.2 in both server/client rebased 21st.May.1 Signed-off-by: Shailesh Pant <[email protected]>
…ce (securefederatedai#1640) * Initial draft not tested Signed-off-by: noopur <[email protected]> * dM Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Added to wf only for testing Signed-off-by: noopur <[email protected]> * Added step to install minio and its client Signed-off-by: noopur <[email protected]> * Final changes Signed-off-by: noopur <[email protected]> * Extra comment for s3 wf Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Modified Signed-off-by: noopur <[email protected]> * Pass time_for_each_round Signed-off-by: noopur <[email protected]> * New test and yaml for verifiable dataset Signed-off-by: noopur <[email protected]> * Removed s3 test from basic e2e yamlo Signed-off-by: noopur <[email protected]> * Lint and format check applied Signed-off-by: noopur <[email protected]> * Docstring added Signed-off-by: noopur <[email protected]> * Corrected logic to call prepare_data_for_s3 Signed-off-by: noopur <[email protected]> * Added s3_bucket model Signed-off-by: noopur <[email protected]> * Steps added in the test file Signed-off-by: noopur <[email protected]> * Added no-op and s3 wf in publish dependency list Signed-off-by: noopur <[email protected]> * Fixed the comment while checking calc_hash flag Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]>
…ecurefederatedai#1638) * Add TaskRunner Analytics E2E workflow and integrate into PQ pipeline Signed-off-by: Chaurasiya, Payal <[email protected]> * review comments Signed-off-by: Chaurasiya, Payal <[email protected]> * fix rest and add models Signed-off-by: Chaurasiya, Payal <[email protected]> * add rest test Signed-off-by: Chaurasiya, Payal <[email protected]> * add rest test Signed-off-by: Chaurasiya, Payal <[email protected]> * comment restapi Signed-off-by: Chaurasiya, Payal <[email protected]> * Add Smokers health Signed-off-by: Chaurasiya, Payal <[email protected]> * remove logs Signed-off-by: Chaurasiya, Payal <[email protected]> * Add review comments Signed-off-by: Chaurasiya, Payal <[email protected]> * Review comments Signed-off-by: Chaurasiya, Payal <[email protected]> * restructure pq Signed-off-by: Chaurasiya, Payal <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]>
…FlowerTaskRunner`/`FlowerDataLoader`, and `FlowerInteropServer`/`FlowerInteropClient` into core OpenFL (securefederatedai#1633) * integrate flower collaborator into core collaborator Signed-off-by: kta-intel <[email protected]> * formatting Signed-off-by: kta-intel <[email protected]> * change task name interop -> prepare_for_interop, change references to local_grpc_server to interop_server Signed-off-by: kta-intel <[email protected]> * add prepare_interop_server as callback to run on_experiment_begin Signed-off-by: kta-intel <[email protected]> * fix typo Signed-off-by: kta-intel <[email protected]> * change to on_round_begin() Signed-off-by: kta-intel <[email protected]> * return empty dict Signed-off-by: kta-intel <[email protected]> * set self.prepare_interop_server() to run on experiment begin Signed-off-by: kta-intel <[email protected]> * remove self-explanatory comment, added another comment for clarity Signed-off-by: kta-intel <[email protected]> * fix plan parsing and automatic assignment of grpc ports Signed-off-by: kta-intel <[email protected]> * move taskrunner and dataloader to core openfl.federated Signed-off-by: kta-intel <[email protected]> * move connector and interop components to core openfl Signed-off-by: kta-intel <[email protected]> * remove comments about patch Signed-off-by: kta-intel <[email protected]> * remove collaborator number setting, it is not being used Signed-off-by: kta-intel <[email protected]> * adjust handling of flower directory. enabling user configuration with default location and use openfl's directory traversal check Signed-off-by: kta-intel <[email protected]> * exit system if error Signed-off-by: kta-intel <[email protected]> * update REAMDE.md Signed-off-by: kta-intel <[email protected]> * update docstring Signed-off-by: kta-intel <[email protected]> * override self.port Signed-off-by: kta-intel <[email protected]> * remove tasks_connector.yaml Signed-off-by: kta-intel <[email protected]> * updates Signed-off-by: kta-intel <[email protected]> * pin flwr Signed-off-by: kta-intel <[email protected]> * remove time Signed-off-by: kta-intel <[email protected]> * formatting Signed-off-by: kta-intel <[email protected]> * formatting Signed-off-by: kta-intel <[email protected]> * formatting Signed-off-by: kta-intel <[email protected]> * copyright Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> Co-authored-by: teoparvanov <[email protected]>
* Change some tests to run with restapi Signed-off-by: Chaurasiya, Payal <[email protected]> * Add grpc and rest checks Signed-off-by: Chaurasiya, Payal <[email protected]> * small fix Signed-off-by: Chaurasiya, Payal <[email protected]> * small fix Signed-off-by: Chaurasiya, Payal <[email protected]> * add some fix Signed-off-by: Chaurasiya, Payal <[email protected]> * rebase fix Signed-off-by: Chaurasiya, Payal <[email protected]> * small fix Signed-off-by: Chaurasiya, Payal <[email protected]> * commen resiliency Signed-off-by: Chaurasiya, Payal <[email protected]> * Change rest_api to rest protocol Signed-off-by: Chaurasiya, Payal <[email protected]> * Change rest_api to rest protocol Signed-off-by: Chaurasiya, Payal <[email protected]> * Add one test in analytics Signed-off-by: payalcha <[email protected]> * Add one docker test Signed-off-by: payalcha <[email protected]> * Add one windows test Signed-off-by: payalcha <[email protected]> * small fix Signed-off-by: payalcha <[email protected]> * Small check Signed-off-by: payalcha <[email protected]> * Small check Signed-off-by: payalcha <[email protected]> --------- Signed-off-by: Chaurasiya, Payal <[email protected]> Signed-off-by: payalcha <[email protected]>
Bumps [flask](https://github.com/pallets/flask) from 3.1.0 to 3.1.1. - [Release notes](https://github.com/pallets/flask/releases) - [Changelog](https://github.com/pallets/flask/blob/main/CHANGES.rst) - [Commits](pallets/flask@3.1.0...3.1.1) --- updated-dependencies: - dependency-name: flask dependency-version: 3.1.1 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Akshay Pant <[email protected]> Co-authored-by: teoparvanov <[email protected]>
…ion (securefederatedai#1649) * add function to deserialize the flower message for further introspection Signed-off-by: kta-intel <[email protected]> * format Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]>
… Framework (securefederatedai#1648) * E2E Framework: Folders cleanup and shuffling Signed-off-by: noopur <[email protected]> * Lint check and fixes Signed-off-by: noopur <[email protected]> * S3 - remove data from local oploaded to s3 or azure blob Signed-off-by: noopur <[email protected]> * After final testing of all datasources Signed-off-by: noopur <[email protected]> * Do not run resiliency in draft stage Signed-off-by: noopur <[email protected]> * Final testing Signed-off-by: noopur <[email protected]> --------- Signed-off-by: noopur <[email protected]>
…ivate attributes (securefederatedai#1605) * Delayed aggregator initialization Signed-off-by: Sachin Gupta <[email protected]> * Added test case Signed-off-by: Sachin Gupta <[email protected]> * Added test case Signed-off-by: Sachin Gupta <[email protected]> * Documentation update Signed-off-by: Sachin Gupta <[email protected]> * Incorporated review comments Signed-off-by: Sachin Gupta <[email protected]> * Incorporated review comments. Signed-off-by: Sachin Gupta <[email protected]> * Incorporated review comments Signed-off-by: Sachin Gupta <[email protected]> * Incorporated review comments. Signed-off-by: Sachin Gupta <[email protected]> --------- Signed-off-by: Sachin Gupta <[email protected]>
Signed-off-by: Shah, Karan <[email protected]>
Signed-off-by: Shah, Karan <[email protected]>
Signed-off-by: Shah, Karan <[email protected]>
This reverts commit f6ac53e. Signed-off-by: Shah, Karan <[email protected]> Signed-off-by: Shah, Karan <[email protected]>
Signed-off-by: Shah, Karan <[email protected]>
…efederatedai#1575 - rename the function to get_aggregated_tensors - adjust both grpc and rest client for name and signature change in api - for the protobuf changes of TensorSpeca and Batched response adjust both client and server in line with grpc - fix the test cases rebased 29th.May.1 Signed-off-by: Shailesh Pant <[email protected]>
ishaileshpant
force-pushed
the
batched_fetch_restapi_fix
branch
from
d6c93c4 to
46d1e95
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
For the changes in get_aggregated_tensors api as per #1575 PR we need to adjust the REST api spec and the rest client for the changes
Type of Change (Mandatory)
Patch REST get_aggregated_tensor api for batched changes as per #1575
Description (Mandatory)
Current REST api implementation and the test cases needs to be adjusted for the api changes as per #1575