Complex Filters fail... and then disappeared

[LeoNot]

Hello all. I'm evaluating Thunderbird for a possible migration from Outlook.

I used Outlooks "Rules" heavily. This morning I spent quite some time creating equivalent filters in Thunderbird. (version 1.5 (20051201) - Windows XP Pro).

Two things....

I had two rules... if "Delivered-To" (custom header) was set to X, then COPY the message to a folder. Then later, after other rules, another that said "if "Delivered-To" was set to X, MOVE the message to a different folder. Basically the first creates an archive of all email that's delviered to a specific address, and the second catches and moves all the messages that haven't been processed by intervening rules.

The Filter Log showed that both actions had been taken.

And yet the messages did not appear in the folder referenced by the first COPY filter. Only the move.

Now, I would LOVE to give you the specific rules from the msgFilterRules.dat file. However, on a restart of Thunderbird, it decided to reset that file to its default, empty state. Thus eliminating an hour or more of my work. (I have upwards of 40 Filters/Rules - and translating from outlook-speak to thunderbird-speak was very time consuming)

Filters that work is a requirement. Conversely, filters that don't reliably work, or that disappear at random, is a deal breaker.

Are there known issues or workarounds I need to be aware of when dealing with Filters?

FWIW: I have four email accounts that receive email from various email addresses (hence my use of the delivered-to header added by my mail host). All delivered into a global Inbox for rules processing.

Thanks for any advice on this. I'm back to Outlook until I can get this resolved.

Leo

[old SKopischke]

Did you look in all possible locations for your msgFilterRules.dat file? If you have four accounts, there could be as many as five locations for that file, depending on the number of POP/IMAP servers involved.

[LeoNot]

I did indeed. "DIR /s msgFilterRules.dat" at the root of the profile directory. There were several. All empty.

Reminds me of another datapoint: I have a custom profile location. Not the default name, and all mail folders continue to show up properly. It's only Filters that I seem to be having a problem with.

Thanks,

Leo

[old SKopischke]

Is this instance of not saving your filters isolated or did it happen multiple times?

[LeoNot]

I only lost my filters once ... after the amount of time I put into creating them, I didn't want to revisit that frustration until I did reasearch here.

The issue of the filter not working properly happened several times.

Leo

[old SKopischke]

Let's address the issues one at a time. If the filters disappeared only once, it may have been a transient problem. I believe I have seen it mentioned in the forums before, but I have never experienced it myself.

Therefore, let's try to get the filters working first, then take steps to either prevent or provide a recovery point in case they disappear again.

Did you notice if the messages in question (those that should have been both moved and copied) appeared once or more than once in the destination folder?

[LeoNot]

They appeared only once, I believe.

[old SKopischke]

Can you try recreating at least subset of the first filter to test that first to make sure it works? It is possible that the second filter worked just fine and the first filter had a flaw in it, resulting in the activity you saw.

[LeoNot]

OK, here are rules, in total:
version="8"
logging="no"
name="Archive: lists<at>notenboom.org"
enabled="yes"
type="1"
action="Copy to folder"
actionValue="mailbox://nobody<at>Local%20Folders/ZZ%20Mailing%20List%20Archives"
condition="OR (\"Delivered-To\",contains,lists<at>notenboom.org)"
name="To: lists<at>notenboom.org"
enabled="yes"
type="1"
action="Move to folder"
actionValue="mailbox://nobody<at>Local%20Folders/Inbox/14%20lists%40notenboom.org"
condition="OR (\"Delivered-To\",contains,lists<at>notenboom.org)"

(I've replace "@" with "<at>" here simply to avoid spam harvesters)

Receiving an email on that account, the Filter Log shows two "Applied Filter" messages, as I would expect.

When I check folder "14 lists<at>notenboom.org" - the message is there.
When I check folder "ZZ Mailing List Archives" - the message is not there. This would be a problem.

AH HA... a clue. Deleting "ZZ Mailing List Archives.msf" and restarting TBird, and the message appears.

OK, I just received two more messages on that account.
- they show up in "14 lists"
- they don't show up in "ZZ archives"
- both show two appropriate entries in the log
- I exit, and delete the msf file and restart
- ONLY ONE of the twho shows up in "ZZ archives".

Boy am I confused.

Thoughts?

Leo

[old SKopischke]

Do you move a lot of mesages through "ZZ archives?" That is messages go in and then are either deleted or moved elsewhere?

[LeoNot]

Nope. It's a fairly static archive just sitting there collecting all the mail I get on various mailing lists. (This instance has 1544 messages right now.)

If anything, "14 lists" acts as you describe ... that's the folder I read and act on (it's a subfolder of my inbox). I read and delete from it. It has three ... *poof* ... nope, now is has zero messages.

I don't delete from the archive until I spool it off to archive it when it gets "too big". I don't know how often that will be if I use TBird ... in Outlook it happened every month or two as the entire collection of mail in Outlook's PST grew beyond a certain point.

Just downloaded two more messages, and the same behaviour as before. Yet another datapoint ... what DOES seem to appear in my "ZZ archives" folder is a blank message with a date of 12/31/1969 4pm. If I delete the .msf file, that goes away.

Leo

[old SKopischke]

Try compacting your folders to see if this impacts the issue. Check <<<b>this page</b>>> for more information on compacting folders.

[LeoNot]

Have compacted the folders in question. Several times, in fact .

Just did a full compact of everything with no change in behaviour.

Leo

[old SKopischke]

Did you upgrade to 1.5 from a previoous version? If so, did you perform the upgrade over the top of the previous version?

[LeoNot]

Nope, this was an install from scratch. (Since I'm attempting to migrate from Outlook, I blew any pre-exisitng Thunderbird side of things away and start over. ReALLY like the way Thunderbird handles profile location for that. I *do* import all the mail and contacts from Outlook, though.)