Confirm that java client side data validation is sufficient for all JSON failure scenarios and hence Csharp validation is redundant #474
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ge (blacktooth#3) Co-authored-by: George Yang <[email protected]>
…latforms (blacktooth#10) * Fix get schema attribute test failure * Initial common trunk for all platforms * Common trunk changes for multi-lang GSR (native-schema-registry) * Common trunk | delete unwanted build-time file from commit * Common trunk | Update README to add full build setps order + specific instructions for AmazonLinux platform
* Add support for Kafka Connect Protobuf converter (awslabs#189) * Add support for Kafka Connect Protobuf converter * Add copyright * Add copyright to pom.xml * Fix copyright im pom.xml * Specify UTF-8 in pom.xml to fix windows build issues * Fix encoding to unblock windows build issues * Fix windows build issue Co-authored-by: Kexin Hui <[email protected]> * Upgrade version 1.1.11 (awslabs#190) * Upgrade library version to 1.1.11 * Fix typo * Add changelog.md Co-authored-by: Kexin Hui <[email protected]> * Upgrading Avro version to prevent a CVE * Upgrading Avro version to prevent a CVE (awslabs#191) Co-authored-by: George Yang <[email protected]> * Upgrade version 1.1.11 Motivation: Upgrade Avro version to prevent a CVE * Explicitly call Deflater/Inflater#close to free native resources (awslabs#203) (awslabs#204) * Upgrade kotlin-stdlib version to 1.7.10 * Upgrade all kotlin dependency versions * Upgrade version to 1.1.13 * Upgrade dependency versions to avoid potential vulnerability * Upgrade to version 1.1.14 * Fix failing KPL canary tests (awslabs#229) * Fix broken link in readme (awslabs#228) * Bump up Kafka version (awslabs#232) Co-authored-by: Mohit Paliwal <[email protected]> * Add a new module for Serde with msk-iam auth (awslabs#239) * add msk iam auth module * addressed the change requests * add readme page * add readme page * add new module for serde with MSK iam * remove the kafka connect module * remove the kafka connect module * Removed the redundant dependencies as per the code review * Add Shade plugin * Upgrade version to 1.1.15 (awslabs#249) Co-authored-by: Vanessa Pinto <[email protected]> * Fix build failure on windows runners. (awslabs#251) build-tools doesn't need to be a dependency for the plugin as plugin config is referencing the files directly using absolute path. This is causing problem when the underlying .m2 directory is ephemeral between actions steps and the installed jar is lost which causes the next step (clean install) to fail. Also removing `mvn test` step as per maven lifecycle previous `mvn install` already runs the `test` phase. * Fixes awslabs#247 (awslabs#248) * Move lombok scope from compile to provided * Remove lombok-utils as it doesn't exist in newer versions anymore * Build with JDK 17. Remove outdated 9 and 13 (awslabs#254) * Add property to provide a http proxy url to connect to glue registry (awslabs#246) * update gitignore * code changes to build http client using proxy url * updated aws sdk version * fixed indentations and moved validation logic. * Added unit test for proxy URL validation --------- Co-authored-by: Abhilash Gajarla <[email protected]> * Update wire * Update dependencies to resolve vulnerabilities from transitive dependencies Motivation: Update dependencies for security purpose * Bump to version v1.1.16 Motivation: v1.1.16 release * Bump guava from 30.0-jre to 32.0.0-jre (awslabs#270) Bumps [guava](https://github.com/google/guava) from 30.0-jre to 32.0.0-jre. - [Release notes](https://github.com/google/guava/releases) - [Commits](https://github.com/google/guava/commits) --- updated-dependencies: - dependency-name: com.google.guava:guava dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump guava from 30.0-jre to 32.0.0-jre in /common (awslabs#269) Bumps [guava](https://github.com/google/guava) from 30.0-jre to 32.0.0-jre. - [Release notes](https://github.com/google/guava/releases) - [Commits](https://github.com/google/guava/commits) --- updated-dependencies: - dependency-name: com.google.guava:guava dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update to the latest everit-json-schema (awslabs#262) * Bump com.squareup.okio:okio in /serializer-deserializer (awslabs#277) Bumps [com.squareup.okio:okio](https://github.com/square/okio) from 3.2.0 to 3.4.0. - [Changelog](https://github.com/square/okio/blob/master/CHANGELOG.md) - [Commits](square/okio@parent-3.2.0...parent-3.4.0) --- updated-dependencies: - dependency-name: com.squareup.okio:okio dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update docker-compose.yml (awslabs#284) * Use default region provider chain (awslabs#280) * Update kafka dependencies version and bump library version to v1.1.17 (awslabs#302) * Bump org.apache.avro:avro from 1.11.1 to 1.11.3 (awslabs#300) Bumps org.apache.avro:avro from 1.11.1 to 1.11.3. --- updated-dependencies: - dependency-name: org.apache.avro:avro dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump org.apache.avro:avro from 1.11.1 to 1.11.3 in /common (awslabs#299) Bumps org.apache.avro:avro from 1.11.1 to 1.11.3. --- updated-dependencies: - dependency-name: org.apache.avro:avro dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump org.apache.avro:avro from 1.11.1 to 1.11.3 in /integration-tests (awslabs#298) Bumps org.apache.avro:avro from 1.11.1 to 1.11.3. --- updated-dependencies: - dependency-name: org.apache.avro:avro dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Add a dummy class in the serializer-deserializer-msk-iam module for javadoc and source jar generation (awslabs#256) * Bump org.json:json from 20230618 to 20231013 (awslabs#304) Bumps [org.json:json](https://github.com/douglascrockford/JSON-java) from 20230618 to 20231013. - [Release notes](https://github.com/douglascrockford/JSON-java/releases) - [Changelog](https://github.com/stleary/JSON-java/blob/master/docs/RELEASES.md) - [Commits](https://github.com/douglascrockford/JSON-java/commits) --- updated-dependencies: - dependency-name: org.json:json dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update the target scala version to 2.13 for mbknor-jackson-jsonschema and kafka * Revert "Update the target scala version to 2.13 for mbknor-jackson-jsonschema and kafka" (awslabs#314) * Bump the AWS SDK version to 2.22.12 and 1.12.633 to fix all vulnerabilities * Bump library version to 1.1.18 * Update SDK version to remove ION dependencies * Remove ion dependencies from root pom * Bump library version to 1.1.19 * Remove commons compression at project level. (awslabs#344) * Bump library version to 1.1.20 (awslabs#345) * Bump org.apache.avro:avro from 1.11.3 to 1.11.4 (awslabs#357) * Bump library version to 1.1.21 (awslabs#358) Co-authored-by: Himani Haresh Patel <[email protected]> * Bump protobuf version to 3.25.5 * Bump library version to 1.1.22 * Bump json-schema version to v1.14.5 * Bump library version to 1.1.23 * Bumped wire schema version to resolve CVE https://www.cvedetails.com/cve/CVE-2024-58103/ * Bumped CI/CD github action to use v4 actions cache * Version bump to 1.1.24 --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Kexin <[email protected]> Co-authored-by: Kexin Hui <[email protected]> Co-authored-by: George Yang <[email protected]> Co-authored-by: George Yang <[email protected]> Co-authored-by: Matthias Krueger <[email protected]> Co-authored-by: Vanessa Pinto <[email protected]> Co-authored-by: Vanessa Pinto <[email protected]> Co-authored-by: Aravinthsamy Sekar <[email protected]> Co-authored-by: Mohit Paliwal <[email protected]> Co-authored-by: Mohit Paliwal <[email protected]> Co-authored-by: awsalialem <[email protected]> Co-authored-by: Ali Alemi <[email protected]> Co-authored-by: NersesAM <[email protected]> Co-authored-by: Abhilash Gajarla <[email protected]> Co-authored-by: Abhilash Gajarla <[email protected]> Co-authored-by: Luc DUZAN <[email protected]> Co-authored-by: George Yang <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Phillip Whittlesea-Clark <[email protected]> Co-authored-by: Baljit Singh <[email protected]> Co-authored-by: Junyu Chen <[email protected]> Co-authored-by: Ravindranath Kakarla <[email protected]> Co-authored-by: Himani Patel <[email protected]> Co-authored-by: Himani Haresh Patel <[email protected]> Co-authored-by: Bowen Shi <[email protected]> Co-authored-by: bowen-s <[email protected]> Co-authored-by: Michael Knox <[email protected]>
…mand structure (which needs to change)
… and add java utility for loading configurations
…on cause and update corresponding test case (awslabs#410) Originally PR'ed here: awslabs#400
…t we can build C + Java stack with a single command (awslabs#412) Originally PR'ed here: awslabs#398
Originally PRed here: awslabs#397
… use relative path (awslabs#414) Originally PRed here : awslabs#401
…y triggered on push, pr and release requests. This build is set install the require dependencies and build the C# package using maven. No integ tests are executed yet. This commit also includes a change on the datatypes class so that the header files can be automatically referenced at build time.
--------- Co-authored-by: Michael Knox <[email protected]>
--------- Co-authored-by: Michael Knox <[email protected]>
Co-authored-by: Michael Knox <[email protected]>
Co-authored-by: Michael Knox <[email protected]>
…bs#442) Co-authored-by: Michael Knox <[email protected]>
…port - Move ConfigurationFileReader to config subpackage - Add NativeGlueSchemaRegistryConfiguration for native-specific handling - Update imports and dependencies in serialization/deserialization handlers - Add comprehensive test coverage for new config classes - Configure Maven for native module testing with mockito inline support
…tive support" This reverts commit 4411455.
- Add Maven clean plugin to automatically clean C build artifacts
* Refactor configuration handling into dedicated config package - Move ConfigurationFileReader to config subpackage - Add NativeGlueSchemaRegistryConfiguration for native-specific config - Update imports and references throughout codebase - Configure Maven for proper test directory structure - Add mockito-inline dependency and disable JaCoCo coverage - Add comprehensive test coverage for config classes * Merge branch 'native-schema-registry-release' into role-assume-java-branch
Move all Go integration test files from integration-tests/ to integration-tests/tests/ subdirectory and update Makefile and build scripts to reflect the new structure. This improves project organization by clearly separating test code from configuration files. Co-authored-by: Michael Knox <[email protected]>
… tokens added as a part of PRs (awslabs#454) * Onboard git-secrets to check for unintended passwords, access keys or tokens added as a part of PRs * Remove password pattern as integration-tests/kafka-connect-json-configs/nohup.out has password pattern in the config
…their latest versions (awslabs#455)
* Automatic cleanup of go resources and added multithreading test. * Add Docker demo setup and simplify Go API usage - Add docker-compose.yml for Kafka demo with zookeeper and golang services - Remove manual Close() calls from Go API documentation examples - Disable test builds and coverage flags in C CMakeLists - Update build scripts for demo/production configuration --------- Co-authored-by: Michael Knox <[email protected]>
…d make demo producers multi-threaded (awslabs#459)
…s#461) - Move cgo_flags.go from GsrSerDe to gsrserde-go package - Remove duplicate CGO directives from individual files - Centralize CGO configuration in single cgo_flags.go file - Update package name from GsrSerDe to gsrserde for consistency Co-authored-by: Michael Knox <[email protected]>
… tests for C# (awslabs#453) * Fix getCtype errors with Protobuf parsing * Write configuration tests * Clean up the configuration validation integ tests - simplify exception flows + remove unnecessary logging + Onboard C# AWS SDK Glue library in tests module * Implement invalid region in config test * Publish configs as shared resources + write invalid endpoint test * Add setup and teardown methods + use region-specific clients --------- Co-authored-by: Ravindranath Kakarla <[email protected]>
* Adding caching and caching unit tests. * Initialize Protobuf cache during serialization handler setup Add ProtobufPreprocessor cache initialization using the configured cache size when creating the SerializerInstance. This ensures the cache is properly initialized before any serialization operations occur. --------- Co-authored-by: Michael Knox <[email protected]>
awslabs#462) * Add user_agent parameter to schema registry serializer/deserializer constructors This change adds a user_agent parameter to the constructor functions across all language bindings (C, C#, Go, Java) for both GlueSchemaRegistrySerializer and GlueSchemaRegistryDeserializer. Enabling better tracking and identification of different client applications. * Change default role session name to 'native' * Change user agent app string to 'native' --------- Co-authored-by: Michael Knox <[email protected]>
Change default roleSessionName from "native" to "native-glue-schema-registry" to provide better identification and traceability in AWS CloudTrail logs. Update corresponding test constant to maintain consistency. Co-authored-by: Michael Knox <[email protected]>
…e and other configs (awslabs#465) * Add config validation tests to test caching, compression, roleToAssume and other configs * Add more config integ tests including cachine, IAM, region and user agent configs * Clarify what we are validating in user agent tests * In new schema version test make sure serialized bytes are different due to new uuid * Refactor config file fetcher into a new method
…nce Csharp validation is redundant
isaurab007
changed the title
isaurab007
changed the title
isaurab007
changed the base branch from
native-schema-registry-release
to
master
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #, if available:
Currently, for JSON dataformat, we are performing client side schema <--> data validations in both C# AND Java, which makes it redundant.
Description of changes:
This change removes the C# checks and establishes that they are indeed redundant by showing Java throws the necessary exception(s) in all these cases.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.