Distributed System support

[jtnelson]

No description provided.

[jtnelson]

Concourse.connect methods need to take in cluster configuration and either

• connect to a random node
• connect to all nodes and do round robin
• have some location awareness and connect to the closest node?

maybe all of the above? Have some logic to detect when a node is down and try to connect to another one?

[jtnelson]

Test Transactions

[jtnelson]

findCcl error

Caused by: com.cinchapi.ensemble.core.UnknownEnsembleException: No Ensemble instance identified by f5ba9e4e-bcf1-4323-9640-4cdca16fc3dd is known to process com.cinchapi.ensemble.core.LocalProcess@2b78a1cd
[Node 1719 - warn.log]  at com.cinchapi.ensemble.core.Proces[Node 1719 - error.log]     at com.cinchapi.ensemble.cohort.ReplicatedStateMachine.handle(ReplicatedStateMachine.java:78)
s.get(Process.java:77)
[Node 1719 - error.log]         at com.cinchapi.ensemble.cohort.RaftStateMachine.query(RaftStateMachine.java:172)
[Node 1719 - warn.log]  at com.cinchapi.ensemble.cohort.ReplicatedStateMachine.lambda$handle$0(ReplicatedStateMachine.java:79)
[Node 1719 - error.log]         at org.apache.ratis.server.impl.RaftServerImpl.submitClientRequestAsync(RaftServerImpl.java:777)
[Node 1719 - warn.log]  at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4875)
[Node 1719 - error.log]         at org.apache.ratis.server.impl.RaftServerProxy.lambda$submitClientRequestAsync$9(RaftServerProxy.java:417)
[Node 1719 - warn.log]  at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)
[Node 1719 - error.log]         at org.apache.ratis.server.impl.RaftServerProxy.lambda$null$7(RaftServerProxy.java:412)
[Node 1719 - warn.log]  at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2276)
[Node 1719 - error.log]         at org.apache.ratis.util.JavaUtils.callAsUnchecked(JavaUtils.java:115)
[Node 1719 - warn.log]  at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
[Node 1719 - error.log]         at org.apache.ratis.server.impl.RaftServerProxy.lambda$submitRequest$8(RaftServerProxy.java:412)
[Node 1719 - warn.log]  at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
[Node 1719 - error.log]         at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:995)
[Node 1719 - warn.log]  ... 27 common frames omitted
[Node 1719 - error.log]         at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2137)
[Node 1719 - error.log]         at org.apache.ratis.server.impl.RaftServerProxy.submitRequest(RaftServerProxy.java:411)
[Node 1719 - error.log]         at org.apache.ratis.server.impl.RaftServerProxy.submitClientRequestAsync(RaftServerProxy.java:417)
[Node 1719 - error.log]         at org.apache.ratis.grpc.client.GrpcClientProtocolService$RequestStreamObserver.processClientRequest(GrpcClientProtocolService.java:222)
[Node 1719 - error.log]         at org.apache.ratis.grpc.client.GrpcClientProtocolService$UnorderedRequestStreamObserver.processClientRequest(GrpcClientProtocolService.java:278)
[Node 1719 - error.log]         at org.apache.ratis.grpc.client.GrpcClientProtocolService$RequestStreamObserver.onNext(GrpcClientProtocolService.java:242)
[Node 1719 - error.log]         at org.apache.ratis.grpc.client.GrpcClientProtocolService$RequestStreamObserver.onNext(GrpcClientProtocolService.java:165)
[Node 1719 - error.log]         at org.apache.ratis.thirdparty.io.grpc.stub.ServerCalls$StreamingServerCallHandler$StreamingServerCallListener.onMessage(ServerCalls.java:255)
[Node 1719 - error.log]         at org.apache.ratis.thirdparty.io.grpc.ForwardingServerCallListener.onMessage(ForwardingServerCallListener.java:33)
[Node 1719 - error.log]         at org.apache.ratis.thirdparty.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailableInternal(ServerCallImpl.java:309)
[Node 1719 - error.log]         at org.apache.ratis.thirdparty.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailable(ServerCallImpl.java:292)
[Node 1719 - error.log]         at org.apache.ratis.thirdparty.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1MessagesAvailable.runInContext(ServerImpl.java:782)
[Node 1719 - error.log]         at org.apache.ratis.thirdparty.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
[Node 1719 - error.log]         at org.apache.ratis.thirdparty.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
[Node 1719 - error.log]         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[Node 1719 - error.log]         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[Node 1719 - error.log]         at java.lang.Thread.run(Thread.java:748)
[Node 1719 - error.log] Caused by: com.cinchapi.ensemble.core.UnknownEnsembleException: No Ensemble instance identified by f5ba9e4e-bcf1-4323-9640-4cdca16fc3dd is known to process com.cinchapi.ensemble.core.LocalProcess@2b78a1cd
[Node 1719 - error.log]         at com.cinchapi.ensemble.core.Process.get(Process.java:77)
[Node 1719 - error.log]         at com.cinchapi.ensemble.cohort.ReplicatedStateMachine.lambda$handle$0(ReplicatedStateMachine.java:79)
[Node 1719 - error.log]         at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4875)
[Node 1719 - error.log]         at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)
[Node 1719 - error.log]         at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2276)
[Node 1719 - error.log]         at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
[Node 1719 - error.log]         at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
[Node 1719 - error.log]         ... 27 common frames omitted

Repro:

• add "name", jeff", 1

• add "name", foo", 2

• add "name", "bar", 3

• link "friend", 2, 1

• link "friend", 3, 1

• find "friend.name = foo"