Add support for `com.hazelcast:hazelcast:5.2.1` #122

linghengqian · 2022-11-22T19:31:05Z

What does this PR do?

Fixes Add support for Hazelcast #32 .
This is part of Add Support for io.vertx:vertx-core:4.3.5 #117 .

Checklist before merging

I have properly formatted metadata files (see CONTRIBUTING document)
I have added thorough tests. (see this)

...c/test/resources/META-INF/native-image/javax.cache/cache-api/1.1.1/serialization-config.json

metadata/com.hazelcast/hazelcast/5.2.1/jni-config.json

tests/src/com.hazelcast/hazelcast/5.2.1/build.gradle

wilkinsona · 2022-11-25T11:01:32Z

Like #124, this looks too broad to me. For example, there are many MBean-related entries that may create the same problem as #113 addressed.

linghengqian · 2022-11-25T11:05:22Z

Like #124, this looks too broad to me. For example, there are many MBean-related entries that may create the same problem as #113 addressed.

I misunderstood what @mhalbritter meant, I thought he was asking me to keep GraalVM reachability metadata for javax.management.**.
Looks like user-code-filter.json is not able to set restrictions in this regard, I will manually update these two PRs in the near future.

linghengqian · 2022-11-25T12:24:32Z

GraalVM reachability metadata for com.sun.management.**, sun.management.**, java.management.**, javax.management.** has been removed.

sdeleuze · 2022-11-29T15:32:21Z

Thanks for this PR.

It looks like some resources like hazelcast-default.xml and related XSD are missing, and XSD are version specific (Spring Boot 3 uses Hazelcast 5.1 for example). Maybe by default we could ship 5.x schema and not 4.x ones?

@linghengqian Could you please test the refined resources with https://github.com/spring-projects/spring-aot-smoke-tests/tree/main/framework/cache-hazelcast (../../gradlew build in this dir) to make sure it works with a typical Spring Boot use case?

linghengqian · 2022-11-29T17:07:45Z

@sdeleuze

I'll update this PR shortly to test for missing xml and xsd definitions. But is it convenient for you to open the CI of this PR?
I don't seem to be sure how to put the content of the current PR to https://github.com/spring-projects/spring-aot-smoke-tests for testing, it doesn't seem to directly configure GraalVM Native Build Tools? You mean, I hang the json file of the current PR to src/main/resources/META-INF/native-image/ folder under https://github.com/spring-projects/spring-aot-smoke-tests/tree/main/framework%2Fcache-hazelcast , and then execute ./gradlew :framework:cache-hazelcast:nativeTest in the main directory?

linghengqian · 2022-11-30T17:19:04Z

@sdeleuze

This involves a blind spot in my knowledge, I have never used Hazelcast before submitting this PR, and I don't know how to write a unit test that can detect calls to hazelcast-default.xml by conditional agent mode. I hope this gets handled by other friends in a future PR. I'm assuming referring to https://github.com/hazelcast/hazelcast/blob/v5.2.1/hazelcast/src/main/resources/hazelcast-default.xml .

Maybe by default we could ship 5.x schema and not 4.x ones?

I'm a bit confused by this statement, I don't see GraalVM reachability metadata with Hazelcast 4.x in my PR. are you mean let me put

{
         "condition": {
           "typeReachable": "com.hazelcast.config.AbstractXmlConfigHelper"
         },
         "pattern": "\\Qhazelcast-client-config-5.2.xsd\\E"
},

Replaced with the following?

{
         "condition": {
           "typeReachable": "com.hazelcast.config.AbstractXmlConfigHelper"
         },
         "pattern": "hazelcast-client-config-[0-9.-]+xsd"
 },

linghengqian · 2022-12-01T07:35:47Z

@sdeleuze

Done. After I updated the unit tests, I also removed the java.lang.management.** and jdk.management.** related GraalVM reachability metadata. Obviously, it would be great if someone knows how to write a separate PR for nativeTest for javax.cache:cache-api:1.1.1. Part of the GraalVM reachability metadata for javax.cache:cache-api:1.1.1 is located in the test/resources folder of the current PR.
I tested it at https://github.com/spring-projects/spring-aot-smoke-tests/tree/main/framework/cache-hazelcast, but I can't understand its error at all, because the current PR uses shutdown() method, and nativeTest works fine. This appears to be a problem with the Spring Framework itself. As an aside, I haven't tested it too deeply.
I am attaching a tarball of the test report generated by Gradle that is not related to this PR.
nativeAppTestReport.zip

linghengqian · 2022-12-06T01:09:13Z

Can anyone help to restart CI? It looks like the Github Action device crashed for 6 hours. Refer to https://github.com/oracle/graalvm-reachability-metadata/actions/runs/3621925163/jobs/6106007671 .
update: Does anyone know what happened to Github Action? I execute ./gradlew clean test -Pcoordinates=com.hazelcast:hazelcast:5.2.1 locally and everything works fine.

fniephaus · 2022-12-06T16:22:05Z

Can anyone help to restart CI? It looks like the Github Action device crashed for 6 hours. Refer to https://github.com/oracle/graalvm-reachability-metadata/actions/runs/3621925163/jobs/6106007671 .
update: Does anyone know what happened to Github Action? I execute ./gradlew clean test -Pcoordinates=com.hazelcast:hazelcast:5.2.1 locally and everything works fine.

It's a bit unlikely that two builds become stale so I assume that there must be something wrong with the tests (e.g., some endless loop). I've opened #144 and once that is merged, I can re-run the builds again.

fniephaus · 2022-12-06T17:46:23Z

Also, are you sure you invoked the test the same way it is invoked on GitHub actions? I don't see any build output from Native Image, so the generator may not even be running and thus may not cause the build to become stale.

linghengqian · 2022-12-07T09:40:25Z

I may need to investigate more in this PR. All three of my local devices passed ./gradlew clean test -Pcoordinates=com.hazelcast:hazelcast:5.2.1.

# Conflicts: # metadata/index.json # tests/src/index.json

linghengqian · 2023-01-09T12:51:05Z

I realize that the problem with CI has nothing to do with GraalVM Native Build Tools. CI has already caused the Github Actions device to crash due to insufficient memory when executing unit tests under JIT. This is related to the startup of the Hazelcast cluster.
I will try again in another branch Structure this PR and resubmit the PR.

linghengqian · 2023-01-09T17:13:40Z

I continue to investigate at Add support for com.hazelcast:hazelcast:5.2.1 #169 .

fniephaus · 2023-01-13T14:10:21Z

So I use action-tmate to log onto a GitHub runner and debug your build and it seems like that some test is waiting for a hazelcast connect (stack trace is below). Maybe you need to start some kind of hazelcast service before the test can run? Also, it may be worth putting a timeout in place for establishing the connect so that you don't have to wait until the GitHub runner times out.

"Test worker" #1 prio=5 os_prio=0 cpu=1591.61ms elapsed=82.09s tid=0x00007f1b480273a0 nid=0xd3e waiting on condition  [0x00007f1b4f29c000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep([email protected]/Native Method)
        at com.hazelcast.client.impl.connection.tcp.WaitStrategy.sleep(WaitStrategy.java:83)
        at com.hazelcast.client.impl.connection.tcp.TcpClientConnectionManager.doConnectToCandidateCluster(TcpClientConnectionManager.java:545)
        at com.hazelcast.client.impl.connection.tcp.TcpClientConnectionManager.doConnectToCluster(TcpClientConnectionManager.java:428)
        at com.hazelcast.client.impl.connection.tcp.TcpClientConnectionManager.connectToCluster(TcpClientConnectionManager.java:389)
        at com.hazelcast.client.impl.clientside.HazelcastClientInstanceImpl.start(HazelcastClientInstanceImpl.java:388)
        at com.hazelcast.client.HazelcastClient.constructHazelcastClient(HazelcastClient.java:472)
        at com.hazelcast.client.HazelcastClient.newHazelcastClientInternal(HazelcastClient.java:420)
        at com.hazelcast.client.HazelcastClient.newHazelcastClient(HazelcastClient.java:119)
        at com_hazelcast.hazelcast.HazelcastTest.testMap(HazelcastTest.java:55)
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0([email protected]/Native Method)
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke([email protected]/NativeMethodAccessorImpl.java:77)
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke([email protected]/DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke([email protected]/Method.java:568)
        at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725)
        at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
        at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
        at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
        at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
        at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
        at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor$$Lambda$191/0x0000000800d10cb0.apply(Unknown Source)
        at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
        at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall$$Lambda$192/0x0000000800d110d0.apply(Unknown Source)
        at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
        at org.junit.jupiter.engine.execution.ExecutableInvoker$$Lambda$343/0x0000000800d39cc0.apply(Unknown Source)
        at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
        at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
        at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
        at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
        at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
        at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
        at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:214)
        at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor$$Lambda$1664/0x0000000800fa0cf8.execute(Unknown Source)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:210)
        at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
        at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:66)
...

vjovanov · 2023-01-13T14:25:40Z

Why didn't we see this exception in the logs?

fniephaus · 2023-01-13T14:27:06Z

It's not an exception, it's part of a thread dump. Apparently, it's waiting forever, which is why I suggested using a timeout when establishing the connection.

vjovanov · 2023-01-13T14:30:32Z

@dnestoro we should investigate how to kill stuff with SIGQUIT on timeout. This should print the thread dump before exiting.

I feel this will happen more in the future.

linghengqian · 2023-01-13T14:31:58Z

@fniephaus

Hi.
I did set up an embedded Hazelcast Server in advance through Junit's @BeforeAll, which is the most surprising thing to me that it can't connect normally on Github actions, because the test of the branch is normal in my local. Maybe there's something inside Hazelcast that I don't know about.
I have tried to directly start the Docker Image of Hazelcast Server in Add support for com.hazelcast:hazelcast:5.2.1 #169, and use org.awaitility:awaitility:4.2.0 to set HazelcastClient's connection timeout test, the problem seems to be the strange judgment of HazelcastClient.
It will take some time for me to try to possibly investigate it, mainly because I'm away from my computer right now.

fniephaus · 2023-01-13T14:35:47Z

Hi @linghengqian, I don't use Hazelcast so I don't know how to proceed here. At least we now know why the build gets stuck. Feel free to let me know if you run into a similar problem again.

linghengqian mentioned this pull request Nov 22, 2022

Make ShardingSphere Proxy in GraalVM Native Image form available apache/shardingsphere#21347

Closed

7 tasks

linghengqian force-pushed the hazelcast branch from fb7fe36 to 4543851 Compare November 23, 2022 01:06

linghengqian commented Nov 23, 2022

View reviewed changes

...c/test/resources/META-INF/native-image/javax.cache/cache-api/1.1.1/serialization-config.json Show resolved Hide resolved

mhalbritter reviewed Nov 23, 2022

View reviewed changes

metadata/com.hazelcast/hazelcast/5.2.1/jni-config.json Outdated Show resolved Hide resolved

mhalbritter reviewed Nov 23, 2022

View reviewed changes

tests/src/com.hazelcast/hazelcast/5.2.1/build.gradle Outdated Show resolved Hide resolved

linghengqian force-pushed the hazelcast branch from ec3b2d4 to 639407a Compare November 23, 2022 10:59

linghengqian mentioned this pull request Nov 24, 2022

Add support for org.ehcache:ehcache:3.10.8-jakarta #124

Merged

2 tasks

linghengqian force-pushed the hazelcast branch from 639407a to 8d64079 Compare November 25, 2022 12:21

linghengqian force-pushed the hazelcast branch from 8d64079 to fb033ff Compare November 30, 2022 17:03

linghengqian force-pushed the hazelcast branch 2 times, most recently from bbfed88 to 75c023e Compare December 1, 2022 07:35

This was referenced Dec 1, 2022

In CONTRIBUTING.md point out that PR needs to remove GraalVM reachability metadata related to MBeans #133

Closed

Add support for javax.cache:cache-api:1.1.1 #136

Closed

linghengqian force-pushed the hazelcast branch from 75c023e to d532f62 Compare December 5, 2022 16:19

linghengqian force-pushed the hazelcast branch from d532f62 to 680d00d Compare December 6, 2022 03:13

linghengqian mentioned this pull request Dec 6, 2022

Is it possible to increase available memory? graalvm/setup-graalvm#6

Closed

fniephaus mentioned this pull request Dec 6, 2022

Add timeout-minutes to all build jobs. #144

Merged

linghengqian force-pushed the hazelcast branch 3 times, most recently from f353325 to 9a531ed Compare December 6, 2022 20:47

linghengqian force-pushed the hazelcast branch 3 times, most recently from 113420a to 4e86fbb Compare December 7, 2022 09:35

linghengqian marked this pull request as draft December 7, 2022 09:54

linghengqian force-pushed the hazelcast branch 2 times, most recently from 6c32dfa to 90fbb10 Compare December 7, 2022 13:15

Add support for com.hazelcast:hazelcast:5.2.1

c0743f3

linghengqian force-pushed the hazelcast branch from 90fbb10 to c0743f3 Compare December 7, 2022 21:16

linghengqian added 2 commits January 9, 2023 19:56

Fix hazelcast graalvm reachability metadata

58d954c

Merge remote-tracking branch 'origin/master' into hazelcast

1d50ea4

# Conflicts: # metadata/index.json # tests/src/index.json

linghengqian closed this Jan 9, 2023

linghengqian deleted the hazelcast branch January 9, 2023 12:51

linghengqian mentioned this pull request Jan 9, 2023

Add support for com.hazelcast:hazelcast:5.2.1 #169

Merged

2 tasks

Add support for com.hazelcast:hazelcast:5.2.1 #122

Add support for com.hazelcast:hazelcast:5.2.1 #122

Uh oh!

Conversation

linghengqian commented Nov 22, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Checklist before merging

Uh oh!

Uh oh!

Uh oh!

Uh oh!

wilkinsona commented Nov 25, 2022

Uh oh!

linghengqian commented Nov 25, 2022

Uh oh!

linghengqian commented Nov 25, 2022

Uh oh!

sdeleuze commented Nov 29, 2022

Uh oh!

linghengqian commented Nov 29, 2022

Uh oh!

linghengqian commented Nov 30, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

linghengqian commented Dec 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

linghengqian commented Dec 6, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fniephaus commented Dec 6, 2022

Uh oh!

fniephaus commented Dec 6, 2022

Uh oh!

linghengqian commented Dec 7, 2022

Uh oh!

linghengqian commented Jan 9, 2023

Uh oh!

linghengqian commented Jan 9, 2023

Uh oh!

fniephaus commented Jan 13, 2023

Uh oh!

vjovanov commented Jan 13, 2023

Uh oh!

fniephaus commented Jan 13, 2023

Uh oh!

vjovanov commented Jan 13, 2023

Uh oh!

linghengqian commented Jan 13, 2023

Uh oh!

fniephaus commented Jan 13, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Add support for `com.hazelcast:hazelcast:5.2.1` #122

Add support for `com.hazelcast:hazelcast:5.2.1` #122

linghengqian commented Nov 22, 2022 •

edited

Loading

linghengqian commented Nov 30, 2022 •

edited

Loading

linghengqian commented Dec 1, 2022 •

edited

Loading

linghengqian commented Dec 6, 2022 •

edited

Loading