Filter weirdness

Discussion of bugs in Mozilla Thunderbird
Post Reply
zoney70
Posts: 27
Joined: October 9th, 2005, 1:04 pm
Location: Baja Arizona

Filter weirdness

Post by zoney70 »

TBird 52.5.0 running on Windows 7 OS, on an Intel Core i5 processor. All email is received via IMAP. There are numerous filters applied in TBird to sort the email into IMAP folders.

Either there is a serious bug in TBird filtering or I don't understand the TBird concept of filtering.

My expectation is that filters will be run in the sequence they appear in the filter list, and that an action is taken ONLY for the first filter satisfied. However, this appears not to be the case when a message meets the criteria of more than one filter, as evidenced by the following sequence of filter log entries showing FOUR ACTIONS TAKEN on one single message. Furthermore, the "winning" filter turns out to be the LAST filter satisfied rather than the first:

--------------------- snip ---------------------
[12/3/2017 9:52:20 AM] Applied filter "Vendor Advertising" to message from "Walgreens Weekly Ad" <Walgreens@e.walgreens.com> - 'Tis the season to save big when you give! at 12/3/2017 9:52:16 AM moved message id = b2hsftvbftkvmuaucuzrpbud2my2rx.148485191.2961@mta932.e.walgreens.com to imap://~~obfuscatedaddress~~.com/Vendor/Vendor Mass Mail

[12/3/2017 9:52:20 AM] Applied filter "Everything" to message from "Walgreens Weekly Ad" <Walgreens@e.walgreens.com> - 'Tis the season to save big when you give! at 12/3/2017 9:52:16 AM moved message id = b2hsftvbftkvmuaucuzrpbud2my2rx.148485191.2961@mta932.e.walgreens.com to imap://~~obfuscatedaddress~~.com/__uncategorized

[12/3/2017 9:52:21 AM] Applied filter "Everything" to message from "Walgreens Weekly Ad" <Walgreens@e.walgreens.com> - 'Tis the season to save big when you give! at 12/3/2017 9:52:16 AM moved message id = b2hsftvbftkvmuaucuzrpbud2my2rx.148485191.2961@mta932.e.walgreens.com to imap://~~obfuscatedaddress~~.com/__uncategorized

[12/3/2017 9:52:22 AM] Applied filter "Everything" to message from "Walgreens Weekly Ad" <Walgreens@e.walgreens.com> - 'Tis the season to save big when you give! at 12/3/2017 9:52:16 AM moved message id = b2hsftvbftkvmuaucuzrpbud2my2rx.148485191.2961@mta932.e.walgreens.com to imap://~~obfuscatedaddress~~.com/__uncategorized
--------------------- snip ---------------------

The "Everything" filter criteria is/was "Age in Days is less than 10". It is intended to be a catch-all filter to remove all otherwise unfiltered messages from the SMTP inbox and store them in a catch-all IMAP inbox. Instead, every message that was already filtered and moved to a desired IMAP folder was then filtered THREE MORE TIMES and ultimately the last filter in sequence ends up as the winning filter for EVERY SINGLE MESSAGE.

This exact same pattern appears multiple times, on different accounts. My suspicion is that the fact the filter is run 4 times is because the processor has 4 cores. Beyond that, I am baffled by this behavior.
User avatar
DanRaisch
Moderator
Posts: 127188
Joined: September 23rd, 2004, 8:57 pm
Location: Somewhere on the right coast

Re: Filter weirdness

Post by DanRaisch »

1. There is no such thing as an "SMTP inbox" as SMTP servers are only for outgoing messages. IMAP or POP servers are involved with incoming messages.

2. The number of processor cores has no bearing on the number of filters impacting messages. That's controlled by the definition of the filters.

3. Rather than the filter log data above, which does not provide information on the filter criteria, please post the filter definitions, including whether the filters are set to "Match all of the following", "Match any of the following", or "Match all messages." Indicate whether those filters are each set up separately or as multiple conditions in a single filter definition with one or multiple actions to be taken. http://kb.mozillazine.org/Filters_%28Thunderbird%29
zoney70
Posts: 27
Joined: October 9th, 2005, 1:04 pm
Location: Baja Arizona

Re: Filter weirdness

Post by zoney70 »

DanRaisch wrote:1. There is no such thing as an "SMTP inbox" as SMTP servers are only for outgoing messages. IMAP or POP servers are involved with incoming messages.

Sorry if the terminology I used was confusing. An exim MTA running on a remote Linux box stores incoming email in inboxes, one per account, under var\spool\mail. Those are what I referred to as the "SMTP" inboxes. An MTA uses only SMTP to communicate, both in and out. Because these inboxes belong to the MTA, I think of them as SMTP artifacts.

My desktop system maintains a constant SSH-encrypted tunnel with the remote server. The Thunderbird MUA uses this tunnel for both SMTP (to send mail) and IMAP (to receive and to sort email). I hope that clears that up for you.
Rather than the filter log data above, which does not provide information on the filter criteria, please post the filter definitions, including whether the filters are set to "Match all of the following", "Match any of the following", or "Match all messages." Indicate whether those filters are each set up separately or as multiple conditions in a single filter definition with one or multiple actions to be taken. http://kb.mozillazine.org/Filters_%28Thunderbird%29
I posted the log entries to illustrate what happened to a single message. I thought it helpful. If not...

I also posted the exact content of the filter that cause the issue, and explained what was the intent of the filter. The account under consideration has approximately 50 filters, some quite complex, but only two filters are relevant to the issue, namely the only two that were satisfied by the message under consideration. All of them run against the inbox when an email is received. The account is set to check mail every ten minutes.

The first filter that matched, which was the "Vendor Advertising" filter, has about 30 entries where a match of ANY entry satisfies the filter. One of the entries matched the message, and the message was correctly moved from inbox to "Vendor Mass Mail" folder. I did not previously describe this filter because it worked as intended.

But the log reflects exactly what happened next. After the message had been moved to the correct IMAP folder, it was then moved (three times) to the folder indicated by the "Everything" filter, which is the very last filter on that account. I described that filter in my previous post:
The "Everything" filter criteria is "Age in Days is less than 10". I don't recall if it was set to match ALL or match ANY, but don't see a difference when there is only one entry to match. I believe it was the default. It clearly was NOt "match all messages" because that setting clears all entries.

The filter has since been replaced with a manual-only filter to move all messages. However, this risks interfering with incoming mail, so I fear to run it at all.

I don't know what more I can say about this. If anyone has a better way (a way that actually works) to move unsorted messages out of the inbox into an IMAP-owned folder, I am all ears.
Post Reply