Skip to content

Conversation

@KarthikReddyMaru
Copy link
Contributor

@KarthikReddyMaru KarthikReddyMaru commented Oct 29, 2025

This PR migrates the Upload task to use the standardized Data.From pattern as part of the ubiquitous from initiative.

closes kestra-io/kestra#12345

What changes are being made and why?

The Upload task previously used manual Object type handling with pattern matching for Collections, JSON arrays, and single strings. This PR standardizes it to use the Data.From facility:

Screenshot 2025-10-30 at 1 25 52 AM

Key Changes:

  • Implement Data.From interface for consistency with other plugins
  • Replace manual parsing with Data.from().readAs() reactive API
  • Update Schema annotations to use Data.From.TITLE and Data.From.DESCRIPTION
  • Remove internalStorageURI = true from @PluginProperty (now handled by Data API)

Implementation Details:

The migration uses a hybrid approach due to a Data API limitation:

@github-project-automation github-project-automation bot moved this to To review in Pull Requests Oct 29, 2025
@MilosPaunovic MilosPaunovic added kind/external Pull requests raised by community contributors area/plugin Plugin-related issue or feature request labels Oct 29, 2025
@MilosPaunovic MilosPaunovic requested review from a team and Malaydewangan09 October 29, 2025 20:51
@KarthikReddyMaru
Copy link
Contributor Author

Hi @Malaydewangan09

Quick heads up: I've posted about a Data API limitation with String.class in the sub issue kestra-io/kestra#12345. This affects the implementation approach. Would be great if you could check that before reviewing. Thank you! 🙏

@KarthikReddyMaru
Copy link
Contributor Author

Hi @Malaydewangan09, could you please review this PR when you have time? I'd appreciate any feedback or guidance on the changes. Thanks!

@Malaydewangan09
Copy link
Member

Sure @KarthikReddyMaru, thanks for the contribution! 🚀

@Malaydewangan09
Copy link
Member

Malaydewangan09 commented Nov 4, 2025

Looks quite OK to me 👍, @fdelbrayelle do you have any points to put up here or if I am overlooking something?

Copy link
Contributor

@fdelbrayelle fdelbrayelle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check my comment

@KarthikReddyMaru
Copy link
Contributor Author

Hi @fdelbrayelle

Unfortunately removing those lines breaks JSON array parsing. Data.readAs(String.class) hits the clazz.isAssignableFrom(from.getClass()) check before JSON parsing, so it returns the entire JSON string as one element instead of parsing it. I've detailed this limitation with debug evidence in kestra-io/kestra#12345 (comment). The explicit JSON handling is necessary to work around this String.class short-circuit behavior.

@fdelbrayelle fdelbrayelle merged commit 6c2c09e into kestra-io:main Nov 6, 2025
1 check passed
@github-project-automation github-project-automation bot moved this from To review to Done in Pull Requests Nov 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/plugin Plugin-related issue or feature request kind/external Pull requests raised by community contributors

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

plugin-aws/src/main/java/io/kestra/plugin/aws/s3/Upload.java: private Object from;

4 participants