feat: allow custom Redis-like broker

[asdfhuang]

This PR decouples Broker interface from redis and allows creating Server and Client directly from a broker instance. Which makes it possible for custom brokers.

For exmaple: Redis from our infrastructure doesn't support pubsub. But I can inherit RDB and implement the pub/sub in another way to make asynq work without a huge amount of work.

[asdfhuang]

There are some known issues for this PR:

• Inspector and Scheduler are locked to redis broker now, they might not work if we use custom borkers.
• Using type alias to export Broker types from internal package is a temporary solution.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 66.37%. Comparing base (4f00f52) to head (6dad2eb).
Report is 13 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #976      +/-   ##
==========================================
- Coverage   67.13%   66.37%   -0.77%     
==========================================
  Files          29       29              
  Lines        4300     5749    +1449     
==========================================
+ Hits         2887     3816     +929     
- Misses       1135     1652     +517     
- Partials      278      281       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[kamikazechaser]

What redis-like broker is this? Is it one of the forks or is it one that implements the Redis wire protocol?

[kamikazechaser]

The idea is good. Though we wouldn't land this in master immediately because:

1. We would need to move the types into a publicly importable folder e.g. pkg. I like the short term fix, but it makes this library look hacky. We have already introduced some hacky bits which are starting to show (see fix: NewScheduler incorrectly creates underlying Client, closing broker properly #977).
2. Work on supporting the scheduler and inspector to use the custom Broker (if possible at all).
3. We have the problem of go-redis which is now transferred to Redis corp. I'm not even sure if there will be guaranteed support for redis forks and clones (https://news.ycombinator.com/item?id=42239607). (Though I see you changes to Client kind of address this, but I'm not sure it addresses everywhere else in the library which ends up using go-redis, repeation of my 2nd point)

[asdfhuang]

What redis-like broker is this? Is it one of the forks or is it one that implements the Redis wire protocol?

It's a redis fork that maintained and operated by our infra team. The problem is that there's always a proxy in front of the redis instances, and the proxy doesn't support pubsub.

1. We would need to move the types into a publicly importable folder e.g. pkg. I like the short term fix, but it makes this library look hacky. We have already introduced some hacky bits which are starting to show (see fix: NewScheduler incorrectly creates underlying Client, closing broker properly #977).
2. Work on supporting the scheduler and inspector to use the custom Broker (if possible at all).
3. We have the problem of go-redis which is now transferred to Redis corp. I'm not even sure if there will be guaranteed support for redis forks and clones (https://news.ycombinator.com/item?id=42239607). (Though I see you changes to Client kind of address this, but I'm not sure it addresses everywhere else in the library which ends up using go-redis, repeation of my 2nd point)

for the 1st& 2nd point, I'll try to upload a new patch to resolve them.

[kamikazechaser]

It's a redis fork

Ah ok. It should generally be fine for now. I have started a discussion around this in #981.