[fix][proxy]Pulsar Proxy OOMs because it forgets to clear the topic name cache

[poorbarcode]

Fixes #24445

Main Issue: #24445

Motivation

PR #23052 fixed the issue caused by creating TopicName and adding items into the queue of TopicName Cache.

Issue: PR added a clarification for Pulsar Broker, but forgot to add the clarification for Pulsar Proxy, Pulsar WebSocket Service, and Pulsar Client.

Modifications

• Pulsar Proxy and Pulsar WebSocket Service: to add a scheduled task to clear the cache
• Pulsar Client: clear cache when its capacity reaches 10_000, which is the same value as the original behavior before [improve] [broker] Improve CPU resources usege of TopicName Cache #23052

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

Matching PR in forked repository

PR in forked repository: x

[poorbarcode]

/pulsarbot rerun-failure-checks

[lhotari]

This is a competing PR to fix the issue. I've already submitted #24457 and provided my argumentation in #24457 (comment) .

Unblocking this PR since we can come back to the #24457 while optimizing topic pattern related memory usage. String deduplication is mostly relevant for those use cases.

[poorbarcode]

@lhotari

Unblocking this PR since we can come back to the #24457 while optimizing topic pattern related memory usage. String deduplication is mostly relevant for those use cases.

Yes, let's first solve the problem with the original plan, and then discuss the rationality of the optimization

Thanks ❤️

[lhotari]

LGTM

[BewareMyPower]

Please hold this PR for some time. I'm working on a better solution motivated by my PR today: #24463

I will push it ASAP.

[BewareMyPower]

@lhotari @poorbarcode @coderzc I pushed a new solution that:

• has better performance with a channel based topic cache
• does not introduce any new configurations or scheduled task

PTAL #24465

[3pacccccc]

I have a question, under what scenario will evictCacheByScheduledTask be false? I see you set it to true in BrokerService、WebSocketService and ProxyService

[BewareMyPower]

Though it's already the same behavior as broker is, I'm wondering how can it address the following case?

LOOP:
   Create a topic with a random name
   Do some send and receive
   Delete this topic

The cache could easily be very large and cause OOM.

I believe the broker has the same issue as well.

[poorbarcode]

I pushed a new solution that:

has better performance with a channel based topic cache
does not introduce any new configurations or scheduled task
PTAL #24465

Close the PR