Add CacheStore, a store that uses Django's cache framework

[robhudson]

Summary

• Adds a new CacheStore that uses Django's cache framework to persist toolbar data
• Adds CACHE_BACKEND and CACHE_KEY_PREFIX settings to configure which cache and key prefix to use
• Toolbar's own cache operations are excluded from the cache panel via an _UntrackedCache proxy

Motivation

By using Django's cache framework, users can choose any configured cache backend for toolbar storage. The interesting thing about this is that the LocMemCache backend can effectively act like the MemoryStore, and the DatabaseCache backend can effectively act like the DatabaseStore.

Notes

• I haven't tested all the backends (e.g. FileBasedCache)
• I haven't tested this on a real project with real database usage patterns

Checklist:

• I have added the relevant tests for this change.
• I have added an item to the Pending section of docs/changes.rst.

AI/LLM Usage

• This PR includes code generated with the help of an AI/LLM
• I have reviewed / edited this code thoroughly

[github-actions]

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
debug_toolbar
settings.py
store.py					239, 282
debug_toolbar/panels
cache.py
profiling.py
Project Total

This report was generated by python-coverage-comment-action

[tim-schilling]

Thank you for adding this. We'll need to add some documentation around this as well. I suspect you're just looking for feedback on the approach right now though. I have a few questions.

For me, I think I would have gone with a ContextVar to control the tracking of the cache panel similar to allow_sql, but this should be fine.

[tim-schilling]

Another thought, I wonder if we can run the full test suite using DatabaseStore and CacheStore now? We could expose that as part of the tox.ini or something similar. Let me know what you think.

[robhudson]

Thanks for the review!

A couple notes:

• I did try with a context var for not recording internal cache operations but it caused a lot of boilerplate code. And you have to remember to use that boilerplate code. So I pivoted to the _UntrackedCache to make the code simpler.
• I had the thought of tweaking the test suite also but thought I'd just start off with the basic idea. I'd also be curious about testing with the various cache backends also.

I'll look into your other comments today. Good stuff.

[robhudson]

An interesting discovery:

The _UntrackedCache wrapper only prevents CachePanel tracking by setting cache._djdt_panel = None. It doesn't prevent SQL tracking. When DatabaseCache executes SQL queries (to tables like django_cache), those queries aren't filtered by SKIP_TOOLBAR_QUERIES because they're not in DDT_MODELS (which only contains debug_toolbar_historyentry).

Is this acceptable behavior, or should we also prevent these SQL queries from being tracked?

It should be possible to dynamically add the DatabaseCache table name to DDT_MODELS, but only for the cache backend configured in the toolbar's CACHE_BACKEND. This way:

• The toolbar's own cache storage queries are filtered (similar to DatabaseStore)
• Other DatabaseCache backends used by the application remain visible

[tim-schilling]

Is this acceptable behavior, or should we also prevent these SQL queries from being tracked?

It seems like we should have the ability to disable all tracking from within the toolbar temporarily, but ideally also allow the user to view those events. I suspect this means we need to shift away from SKIP_TOOLBAR_QUERIES to a more generic setting.

However, reducing our scope to just this change, I think ideally we'd prevent these SQL queries from being tracked. If it's not going to result in something horrible, I think we should do that.

[robhudson]

I refactored the tests a bit to test the various cache backends and added support for not tracking the self-generated SQL from the cache store using a database.