Filter running twice

[Anamon]

Hello everyone,

I have this problem that a filter I set up seems to be executed twice per matching message. So far, I couldn't figure out why it happens and how to prevent it, and would appreciate any help or insights.

The filter I'm trying to set up is pretty simple – I want every e-mail that gets classified as Junk to be forwarded to my SpamCop submission address. The problem is that this forwarding happens twice for every message. SpamCop shows me two copies of every Junk mail, and also in the Sent folder within Thunderbird itself, I can see that it was really forwarded twice. However, it only happens when the trigger is mails being retrieved from the server. If I execute the filter manually, the forwarding happens only once.

When trying to find out more about this online, I've found other people with similar problems. From there I took the suggestion to also set a specific tag for the messages, along with forwarding them, and then extending the filter with an additional condition that said tag must not be present. So the idea is that the tag marks all messages that have already been forwarded, and prevents them from being matched by the filter again. However, this didn't make any difference. The tag is being set properly, but every e-mail still gets forwarded twice when it's received and classified as Junk.

Here are the settings I currently have (sorry for the German screenshot):


The filter is set to run when executed manually or on fetching messages (after Junk detection). The conditions are that the message has been classified as Junk and does not have the "forwarded to Spamcop" tag. The actions are to actually forward the message and tag it with the "forwarded to Spamcop" tag.

From these settings, I don't understand how it can happen that the filter is executed twice, unless the filtering is run twice in parallel. Where am I going wrong?

I'm using Thunderbird 91.9.0 (64 bit) on a Windows 10 system and with an IMAP account.

Thanks!

[tanstaafl]

The order of processing is:

Message filters
"Trust header"
"Adaptive junk" (junk mail controls)

I don't know why the filter is being executed twice on a message unless you enabled running filters automatically for the junk mail folder too and forgot about it. https://superuser.com/questions/544480/ ... olders-too

You could enable filter logging and see if that gives a clue. However, I think there are two simpler solutions you should consider:

1) Delete the filter for the inbox and just use a filter for the junk mail folder that you run manually. That filter could forward the message to SpamCop and then delete the message. Only check the "Manually Run checkbox.

2) Use the Just Report It add-on at https://addons.thunderbird.net/en-us/th ... report-it/ instead. It appears you could select multiple messages and press the button once.

[Anamon]

Thanks for the reply!

I don't have the setting referenced in the SuperUser question, so that shouldn't be the cause. Also, it sounds like in that scenario, the added tag action and condition should still prevent the double-forward, since the tag that prevents the filter from running again would have been set by the time the filter picked the e-mail up a second time from the Junk folder?

Your first suggestion is actually the workaround I set up now, in order to prevent the duplicates. Since it's immediately visible if there are Junk mails that haven't had the tag set yet, it's easy to see when the filter should be executed manually. Thank you also for the link to the add-on, I didn't think to check if something like that exists before setting up the filter.

However, both of these solutions still require at least one manual click. In the filter setup I planned, I would have to take no manual action at all, which I would prefer. So I think I'll still re-enable my previous filter configuration and enable filter logging to see if I can figure out more.

[Anamon]

I received three e-mails that got caught by the Junk filter today. Unfortunately, the filter log doesn't give a clear idea about what's happening:

Code: Select all

[25.05.2022, 17:46:15] Filter "Forward Junk to Spamcop" auf die Nachricht "sender1" <address1> - darlehen1 am 23.05.2022, 06:03:23 angewendet weitergeleitet
[25.05.2022, 17:46:15] Filter "Forward Junk to Spamcop" auf die Nachricht "sender2" <address2> - Wichtige Mitteilung; € 2,000,000'00 EUR am 23.05.2022, 19:43:34 angewendet weitergeleitet
[25.05.2022, 17:46:15] Filter "Forward Junk to Spamcop" auf die Nachricht <address3> - Ist das Ihre gültige E-Mail-Adresse? am 25.05.2022, 12:10:09 angewendet weitergeleitet
[25.05.2022, 17:46:15] Filter "Forward Junk to Spamcop" auf die Nachricht "sender1" <address1> - darlehen1 am 23.05.2022, 06:03:23 angewendet Schlagwort hinzugefügt
[25.05.2022, 17:46:15] Filter "Forward Junk to Spamcop" auf die Nachricht "sender2" <address2> - Wichtige Mitteilung; € 2,000,000'00 EUR am 23.05.2022, 19:43:34 angewendet Schlagwort hinzugefügt
[25.05.2022, 17:46:15] Filter "Forward Junk to Spamcop" auf die Nachricht <address3> - Ist das Ihre gültige E-Mail-Adresse? am 25.05.2022, 12:10:09 angewendet Schlagwort hinzugefügt
[25.05.2022, 17:46:16] Filter "Forward Junk to Spamcop" auf die Nachricht "sender1" <address1> - darlehen1 am 23.05.2022, 06:03:23 angewendet weitergeleitet
[25.05.2022, 17:46:16] Filter "Forward Junk to Spamcop" auf die Nachricht "sender2" <address2> - Wichtige Mitteilung; € 2,000,000'00 EUR am 23.05.2022, 19:43:34 angewendet weitergeleitet
[25.05.2022, 17:46:16] Filter "Forward Junk to Spamcop" auf die Nachricht <address3> - Ist das Ihre gültige E-Mail-Adresse? am 25.05.2022, 12:10:09 angewendet weitergeleitet
[25.05.2022, 17:46:17] Filter "Forward Junk to Spamcop" auf die Nachricht "sender1" <address1> - darlehen1 am 23.05.2022, 06:03:23 angewendet Schlagwort hinzugefügt
[25.05.2022, 17:46:17] Filter "Forward Junk to Spamcop" auf die Nachricht "sender2" <address2> - Wichtige Mitteilung; € 2,000,000'00 EUR am 23.05.2022, 19:43:34 angewendet Schlagwort hinzugefügt
[25.05.2022, 17:46:17] Filter "Forward Junk to Spamcop" auf die Nachricht <address3> - Ist das Ihre gültige E-Mail-Adresse? am 25.05.2022, 12:10:09 angewendet Schlagwort hinzugefügt

First, the three mails got forwarded, then the tag got applied to all of them. One second later, the same thing is repeated a second time. I was expecting that maybe the forwarding and tagging got interleaved, so the second forwarding happened before the tag was applied, but this doesn't seem to be the case. Still, what I will try next is changing the order of actions, to put the tag first and only then forward, and see if it makes a difference.

It looks as if Thunderbird sees the actual e-mail twice, and the tag is being applied to a different copy than the second time the e-mail matches. Maybe the tag is not preserved as the e-mail is moved to the Junk folder, so it matches once in Inbox, then a second time in Junk?

[mgagnonlv]

I briefly checked the Spamcop.net website and noticed that there are two ways to use their system. If you have used the "Configure Mailhost" approach, I think the email is sent to their system as soon as either your server or Thunderbird declares a message as spam.

To check this theory, I would recommend you do the following:

• Duplicate your existing rule (for safekeeping), deactivate one and modify the other.
• In the "Actions", do the following:
  – Remove the first line (Transfer to my_account@spam.spamcop.net)
  – Keep the second line as is
  – Add a third line: "Move to folder" –> "Messages to be reported"
  This will simply allow you to easily find them in a day or so and report them at that time.
  – Add a fourth line: "Stop running the filters" (to make sure no other filters interfere).
• Let it go for a day or two and check in your Spamcop account to see whether the flagged messages have been reported once or not. If they have been reported once, then it means that you don't need to send them specifically from within Thunderbird.

Good luck

[Anamon]

Thanks for the reply.

I actually do have to forward them from Thunderbird, I haven't set up any other way to get mails reported to SpamCop. I've actually been using SpamCop for a long time, but up until recently always manually copied the mail source from Thunderbird and pasted it into the text field on the site. If I don't have step number one in my filter actions, no junk mail gets sent to SpamCop automatically. As I mentioned in the OP I can also actually see the forwarded messages twice in Thunderbird's "Sent" folder, so it's actually Thunderbird forwarding them twice, not SpamCop getting the same mails a second time through some other path.

The mailhost configuration on SpamCop actually doesn't relate to how e-mails get delivered to the service, it only affects the following analysis of the e-mail (in order to avoid sending abuse reports to providers on the "receiving" end of the spam).

In the meantime I can also report that changing the order of the forwarding and tagging steps didn't make a difference. However, I took your suggestion to add the "Stop running filters" step as a final action, and will see what happens now

Thanks!

Update: The stop running filters step also didn't prevent the second forward. For now, I'll go back to just only using the manual filter trigger.