Not chosing sender identity as per configuration

[archerbarrie]

Using CI version 2.6.4 or 2.6.5 on TB 140.2.1esr the behaviour of CI seems to have changed with regard to the priority of different groups of identities, in particular it no longer adheres to the statement in the configuration under Detection that "Search terms are tried only if the "Selection" settings above don't yield a result".

In evaluating which identity to use as the sender, CI prioritises matching identities/addresses as follows

1. Identities found in Addressbooks
2. Identities found in Mailinglists
3. Addresses/names found in the "Search term" box of Detection
4. An identity that appears in the recipient list

and this applies to the ALL the entries in the To, Cc and Bcc fields.

CI sets the sender to an identity that is matched, provided that the sender has not already been set from a higher priority group.

Thus a sender identity determined by a match in a mailing list would NOT replace a sender identity from a previously matched address book, but a sender identity determined by a match in an address book WOULD replace a sender identity previously matched in a mailing list.

If a sender identity would be set from the same priority group as currently set, the sender identity may or may not be replaced. The logic for which happens is indeterminate.

One effect of this prioritisation is that the assertion in the configuration, "Detection" section that the highest priority is an identity that appears in the recipient list is no longer true.

I believe that the behaviour has changed fairly recently and that previously an identity that appeared in the recipient list WAS the highest priority but also that if there was no such match then the FIRST identity that matched was used.

Having a priority order when trying to match an identity seems like a good enhancement, however the current order is wrong in that it does not implement the specification and arguably should be inverted. This inversion would restore the priority of an identity that appears in the recipient list as the highest and also reduce the risk of CI chosing a sender identity based on an address in the Cc and Bcc field and this is very unlikely to be correct.

A use case that illustrates that inverting the priority order would be better is as follows. A company has a number of trading names as well as the parent company. Employees are instructed to use a sender identity that corresponds to the trading name relevant to the recipient. Because a recipient is in a mailing list for a particular trading company, the correct sender identity is chosen. However, if an employee of the parent company is blind copied, and that employee is in the address book for the parent company, the sender will be changed to the parent company; this is the wrong outcome. If the priority order was inverted it would not be changed and the sender would be correct.

Issue 54 also relates to this issue.

[ldreier]

Hi @archerbarrie ,

I had a look into your priority list and the implementation in the add-on.
I also added some tests to check behaviour of the add-on with respect to the flow diagram from the README.md
I can re-confirm that the add-on behaviour follows the flow diagram for the "detection" mechanism.

Actually the priorities 1., 2. and 3. from your list are not reflected in the add-on code.

The add-on iterates over the list of Thunderbird identities and checks for each identity, if we have a "match".

For a specific identity, the entries in the Detection text box are checked line by line. But this does not result in a "priority", since the result is just "match" or "not match". A "match" results in this identity to be chosen in this step.

Since the first matching identity is considered, it will depend on the order of identities in the Thunderbird configuration which identity is returned in that step if multiple identities would match.
I see no recent change in this are in the code. Maybe your order of identities changed, so that you observed a change?

Regarding your "4." priority. I think this is what is "detectable identity found..." in the flow diagram.
Note that this path is taken with first priority, directly after the "reply" button is hit, but if you later change the recipients manually the detection list in the text box takes precedence (if we have a match there)
If you later remove that matching recipients, it falls back to the initial decision as last resort (the second diamond with "detectable identity found...")

Regarding your use case ... I am not sure how to address this by changing the add-on (more below).
Without changing the the add-on, you could e.g. try to avoid having multiple matches by creating (dedicated) mailing lists or address books where only your customers are included in exactly one of it.

Ideas for changes in the add-on to help you, could be:

• allow ordering of the identities (in your use case you could move the "parent company" to the end)
• changing the way a match is searched (from what is described at beginning of comment) ... e.g. by iterating always over all identities, but taking into account an optional priority number (to-be-added) in textbox lines and choosing the identity with the highest match-prio.

[archerbarrie]

Sorry not to have been able to respond before this.

Perhaps we could just take one point at a time, looking specifically at an identity that appears in the recipient list. Irrespective of what the flowchart says, on the configuration page under Selection there is a tick box with the legend "When replying a message (sic) attempt to find and use an identity that appears in its recipient list." And then under Detection it says for the chosen identity "Specify search terms or domains that, when found in the recipient list, will resolve to this identity. Search terms are tried only if the"Selection" setting above don't yield a result".

So, let us assume there is an account which contains identities of travel@mydomain.com and friend@mydomain.com. For the identity friend@mydomain.com there is a seach term of joe@hisdomain.com. An email is received addressed to travel@mydomain with a cc of joe@hisdomain.com. A "reply all" is done to this email, so the recipient list contains travel@mydomain.com and joe@hisdomain.com. Since an identity for the account (travel@myarcher.net) appears in the recipient list, that identity should be used as the sender according to the text under Selection. However, there is also a matching search term for the identity friend@mydomain.com which matches joe@hisdomain.com but that identity should NOT be used as the sender because the Selection settings HAVE yielded a result.

I cannot read the text under configuration any way other than to affirm the above assertion. However, CI choses friend@myarcher.net which as I see is in contradiction to what the configuration text says.

Have I misunderstood what it says in the configuation text? Or is the text on the configuration page wrong? Or does what happen on my system (friend@mydomain.com is chosen) differ from what happens for everyone else?

I believe that issue #62 may relate to the above and relates to my experience in that I believe CI used to work as I describe.