add connection pool put back

[wuzuoliang]

No description provided.

[stevenh]

Hi @wuzuoliang could you describe what issue this is trying to fix?

[stevenh]

Just following up on this @wuzuoliang ?

[wuzuoliang]

something like this
redis/go-redis#1820

[stevenh]

something like this go-redis/redis#1820

That's an example of a similar function, but what is the benefit of that?

[TangJia025]

waiting for this pr

[stevenh]

I can't see the benefit of this I'm afraid. Are you able to explain why this is needed?

[TangJia025]

the LIFO connection get mode will lead to load inbalance
when conns in some redis proxy is more than others, the gap will get wider with time going by

[stevenh]

Sorry I don't follow what do you mean by load in balance?

[tangtj]

We use Alibaba Cloud's redis. Cloud redis consists of redis and redis proxy. When a client connects to cloud redis, conn forwards it to the corresponding redis through a proxy. A cloud Redis instance usually has multiple proxies. The current redigo conn pool uses fifo mode. It will use the most recently used conn first, and the agents connected later will be busier than other agents. Changing to lifo mode will more balance the load of multiple agents.

[stevenh]

Thanks for the details. Seems like the unbalanced nature is actually an issues with the LB which is choosing which proxy the client is connecting to.

FIFO nature of is required to ensure unused connections are freed up when not needed. Without this a spike in connections can never be reduced. FIFO should not adversely impact the behaviour of the setup you described, as long as the LB is doing its job correctly.

So the question is what is doing the load balancing between the client and the 3 proxies?

[tangtj]

The LB decides which proxy to use when creating a connection. After the establishment is successful, the proxy will not be replaced. So the fifo mode will make the conn at the front of the pool take more work.

[stevenh]

If I understand the description of the problem correctly it sounds like the issue is with the proxy, so that's where the fix should be done. This is because, as mentioned, changing the pool to LIFO breaks connection clean up so is not something we should do.

[tangtj]

Just add an option and let the user decide.

[tangtj]

I may not explain clearly, please refer to for details redis/go-redis#1819 .

[stevenh]

Just add an option and let the user decide.

The problem with that is it will break connection clean up, causing a different problem.

[stevenh]

I may not explain clearly, please refer to for details go-redis/redis#1819 .

I understand the problem, in short LIFO results in unbalanced use of connections in the pool, but this is by design to ensure that old connections are expired.

That said if the pool is seeing load which requires multiple concurrent connections and the LB created said connections across the servers evenly to start with you should see relatively consistent server use too. If the connections aren't created evenly then switching to FIFO won't solve the problem, and if the load doesn't demand multiple concurrent connections then I would argue there's no need for multiple servers anyway.

As mentioned switching to FIFO will break connection clear down and result in pool being unable to reduce its size from peak, which is a nasty side effect. This is actually something that I fixed in the database/sql too.

In your case what is doing the load balancing, and what is your access pattern?

[dreamerjackson]

@stevenh old connections are expired can be solved by have a option to set the max life conn time, then force close the connect

[dreamerjackson]

there is a special condition that the connect rise suddenly and one proxy very slowly, FIFO will always get the slowly connect and the state will continued

[stevenh]

there is a special condition that the connect rise suddenly and one proxy very slowly, FIFO will always get the slowly connect and the state will continued

I'm not sure what you're trying to say here, could you clarify?

[tangtj]

about break connection clear down and result in pool to reduce its size from peak. I think it can be resolved. #589

[seamusopen]

I may not explain clearly, please refer to for details go-redis/redis#1819 .

I understand the problem, in short LIFO results in unbalanced use of connections in the pool, but this is by design to ensure that old connections are expired.

That said if the pool is seeing load which requires multiple concurrent connections and the LB created said connections across the servers evenly to start with you should see relatively consistent server use too. If the connections aren't created evenly then switching to FIFO won't solve the problem, and if the load doesn't demand multiple concurrent connections then I would argue there's no need for multiple servers anyway.

As mentioned switching to FIFO will break connection clear down and result in pool being unable to reduce its size from peak, which is a nasty side effect. This is actually something that I fixed in the database/sql too.

In your case what is doing the load balancing, and what is your access pattern?

Slow connection processing results in a connection that is always at the end of the queue and is not eliminated because it is idle. In this way, even if the LB creates new connections evenly between server nodes, the less performing server nodes will have more connections。

[stevenh]

If you're talking about a single or small number connections then that should have little to no impact, so I'm still struggling to understand the issue?

[wuzuoliang]

cancel