Skip to content

Conversation

davidradl
Copy link
Collaborator

@davidradl davidradl commented Aug 20, 2025

Description

First commit for Flink, includes:
changing the pom in line with the standard Flink connector including

  • rat checks
  • Flink checkstyle
  • Flink spotless
  • standardize the name of the Maven artifact
  • amend the package to follow Flink standards
  • added Apache license
  • change the unit tests away from jacoco to use surefire
  • removed jetbrains annotations
  • removed the use of a google annotation it was complaining about
  • removed use of ImmutableList which it did not like.

Resolves HTTP155

PR Checklist

I am looking to get this code into Flink with the commit history as-is.

There are still things to do:

  • migrate tests away from Mockito to avoid the banned
  • separate the pom into a parent , code and SQL shading pom in line with Kafka. I think we need to separate this our so we can shade and rename dependancies we include.
  • test datastream
  • do the docs
  • amend the config option names to be flink with gid aliiases
  • notices?
  • assess maven analyse excludes
  • amend the prefix to not include gid and move to Flink

But I think it makes sense to get this initial commit in to push the commit history over. So all subsequent contributions will count as Flink contributions.

@davidradl
Copy link
Collaborator Author

fyi @grzegorz8

@davidradl
Copy link
Collaborator Author

davidradl commented Aug 21, 2025

@grzegorz8 I have split the poms out so there is now a parent , the code and sql (with the shading).

Accummulating a list of tasks I need to do before offering to Flink :

@davidradl davidradl marked this pull request as draft August 21, 2025 15:54
@davidradl
Copy link
Collaborator Author

I did not see this build breaks locally. Changing to draft while I address them.

@davidradl
Copy link
Collaborator Author

@grzegorz8 thanks for your valuable feedback - I have addressed what you have raised - so I can see the build issues again (that I do not see locally) so I can address before taking the PR out of draft. I am aware that Flink is looking to support java 17 and 21 as well as 11. 21 was failing prior to the pom split.

@davidradl
Copy link
Collaborator Author

davidradl commented Aug 22, 2025

I have fixed the convergence locally.
I am seeing that if I include flink-table-test-utils in the parent and http poms then the junit tests work locally. But I get an unused dependancy - which I can ignore in the http pom, but the ci yml does not allow me to ignore this dependancy.
If I remove this dependancy the unit tests fail as it cannot find an ExecutorFactory. I assume that the presence of the plugin means that its content is in the classpath for the test, but this is a runtime dependancy not a compile time one.

without the plugin I see errors like:
[ERROR] org.apache.flink.connector.http.internal.sink.HttpSinkConnectionTest.testServerErrorConnection Time elapsed: 0.014 s <<< ERROR!
java.lang.IllegalStateException: No ExecutorFactory found to execute the application.
at org.apache.flink.core.execution.DefaultExecutorServiceLoader.getExecutorFactory(DefaultExecutorServiceLoader.java:88)
at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getPipelineExecutor(StreamExecutionEnvironment.java:2995)
at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.executeAsync(StreamExecutionEnvironment.java:2469)
at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:2351)
at org.apache.flink.streaming.api.environment.LocalStreamEnvironment.execute(LocalStreamEnvironment.java:68)
at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:2325)
at org.apache.flink.connector.http.internal.sink.HttpSinkConnectionTest.testServerErrorConnection(HttpSinkConnectionTest.java:268)
at java.base/java.lang.reflect.Method.invoke(Method.java:572)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)

Still investigating - any thoughts ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants