Cleanup anonymous user context

[dbu]

fix #198
fix #182

@andrerom @lolautruche i realized that in varnish we don't send a hash header at all for anonymous users (anonymous on http level). i tried to better document this here, and synchronize the behaviour of the symfony subscriber with the varnish configuration.

can you please check if you agree with this and if this resolves the issues you had with the hashes?

i left the option to configure the anonymous hash to something, so a perfectionist could share the cache between anonymous users and authenticated users that have no additional permissions by specifying their hash as anonymous_hash.

[andrerom]

Hmm, how would we go about clearing cache for anonymous user context hash now? As in if rights for anonymous changes, prior cache variations needs to be expired , and with no hash for no cookies anonymous users, I don't immediate see how that can be accomplished.

[dbu]

I think i am missing something here. In varnish, we don't have a hash header for anonymous user, so nothing new. For Symfony HttpCache, there is no BAN, so the header would not help with invalidating. Can you explain a bit more please?

I would assume that when permissions for user groups change, we need to reset the cache completely to be sure to catch everything.

[andrerom]

In varnish, we don't have a hash header for anonymous user, so nothing new.

there is the hard coded hash being used instead right? so cache was at least in our case using this.

the header would not help with invalidating. Can you explain a bit more please?

Implies multi tagging and what I mentioned here #182 (comment)

I would assume that when permissions for user groups change, we need to reset the cache completely to be sure to catch everything.

if by completely you imply everything: this is what I'm trying to avoid, permissions in eZ is a adminstration tags via UI, can happen quite frequently sometimes with minor changes, especially for editor roles. And then invalidation gigs of varnish cache, even that for anonymous and basically taking down the site/app is not an acceptable way to handle this for high traffic installs.

if you imply just user hash lookup url, making varnish just lookup this again, then yes that would be ok.

[dbu]

there are (at least) 3 permission-related types of changes:

1. permission on a content changes (what role is allowed to see this content). this needs a purge of that content url / ban of the tag of that content. not affected by what we do here
2. permissions of a user change. we need to invalidate the hash lookup for that user. as we probably don't know that users current session(s), we probably have to invalidate all hash lookups. as the hash will have changed, we don't need to invalidate content.
3. the application changes what permissions various content needs, not possible to identify affected content: invalidate everything

for the special case of changing the permissions of anonymous users, we always have a problem if we handle the case differently on the caching proxy. having no hash lookup is just a special case of using a hardcoded hash. either way the hash does not change when permissions change. so should we do the opposite of what i tried to do here, and do a lookup both in varnish and with symfony?

[dbu]

our varnish config never set a hash for anonymous users. and in symfony httpcache there is no ban. i think i am missing something - what exactly can you do now that will no longer work?

[ddeboer]

This sounds a little vague. What Varnish behaviour do you mean?

[ddeboer]

Perhaps: ‘as there’s no way for the caching proxy to differentiate between cookies that belong to authenticated users and session cookies that belong to anonymous sessions’.

[ddeboer]

Some small comments, 👍 otherwise.

[dbu]

ping @lolautruche @andrerom @bdunogier are you ok if we go this way?

[andrerom]

So returning to this, one concern on our side is:

permissions of a user change. we need to invalidate the hash lookup for that user. as we probably don't know that users current session(s), we probably have to invalidate all hash lookups. as the hash will have changed, we don't need to invalidate content.

This won't any longer be true for permission changes for anonymous, ideally would like to clear hash lookup to force new hash to be generated (assuming no hard coded hash is configured)

[bdunogier]

This won't any longer be true for permission changes for anonymous, ideally would like to clear hash lookup to force new hash to be generated (assuming no hard coded hash is configured)

If I'm not mistaken, the same is true if you change security rules affecting anonymous users (e.g. start protecting a route, or make a route anonymous), you do need to refresh cache items for anonymous, don't you ?

Has anybody reported scaling issues with that (like clearing gigs of cache at once) ?

[dbu]

permissions of a user change. we need to invalidate the hash lookup for that user. as we probably don't know that users current session(s), we probably have to invalidate all hash lookups. as the hash will have changed, we don't need to invalidate content.

i think there is no way around this, whatever we do.

This won't any longer be true for permission changes for anonymous, ideally would like to clear hash lookup to force new hash to be generated (assuming no hard coded hash is configured)

if anonymous requests are treated the same as everything else, they would get a calculated "real" hash and the lookup for anonymous is the only one that could reliably be forced. i think this is a strong argument to go the other way than this PR is currently going, and always do a lookup.

if you change security rules affecting anonymous users (e.g. start protecting a route, or make a route anonymous), you do need to refresh cache items for anonymous, don't you ?

if you change security on routes, you probably should purge (or ban, if you have query strings as well) that url (it will be really hard to know which roles this change affects, and for which hashes (combination of roles) a cache entry could exist. if its specific routes, this is not that heavy. general permission changes probably mean that you need to flush caches and rewarm them or whatever you do to survive that. i think this is just a limitation of the user context concept and can't see how we could avoid this.

Has anybody reported scaling issues with that (like clearing gigs of cache at once) ?

not to my knowledge. i will do an alternative pull request to this one doing the opposite, having varnish always do a hash lookup and have the symfony cache hash default to null but do a lookup if its null and an anonymous request.

[andrerom]

Has anybody reported scaling issues with that (like clearing gigs of
cache at once) ?

not to my knowledge.

we have had reports of that, but customers of that size/volume tends to just brute force disable invalidation and instead use a ttl cache with varnish to get more stable performance.

[dbu]

we have had reports of that, but customers of that size tends to just brute force disable invalidation and instead use a ttl cache with varnish to get more stable performance.

yeah that is what i would expect that you have to do in those scenarios.

[dbu]

replaced by #330