Major problems with filters and summary files

[johnmsch]

Long time loyal Thunderbird user, ever since v0.5 back in early 2004. Now having a major issue since upgrading to 38.2 last week.

Ever since the upgrade, I keep getting errors with the filters when downloading emails. I have 3 POP accounts that go to the global inbox. Below is an example of what happened just this morning.

Here is a screenshot of one of my folders, and its subfolders:



I then click on Get Messages, and get this error:



Notice in the first screenshot, the folder "Augusta" has 5 unread messages and 1711 total. But when the filter for Augusta runs, the error happens and the total and unread message counts go away. Here are but two more examples of this behavior, on two other folders. The message counts disappear when the error happens.





For the total of 133 messages downloaded in this example, roughly 100 of them threw this error. Once I'm done downloading, I then click on each folder that lost it's count and get the message that its rebuilding the summary file. All the unfiltered messages are in the global inbox. I then click on it and select Tools-> Run Filters on Folders. All the messages are then filtered to the correct folders with no errors.

FWIW, I compact my folders every day (Click on Local Folders Inbox, then File -> Compact Folders) before I shut down Thunderbird (by doing File -> Exit, not the red "X"). Sometimes I do it more than once a day. For example, if I delete any messages from any of the folders, I then do a compact. I have even done repairs on the folders that had these errors, but in the next day or two, when a message comes along that needs to go to one of them, the same error happens again. I get these errors on dozens of folders.

I can only assume that this has something to do with the new built-in feature where we have the option to display the unread message count. Before this, I used the extension Extra Folder Columns and did not have this issue.

Hoping that someone here has an idea of whats going on. In the meantime, I'd really like to go back to the previous release (38.10?) where I did not have this problem.

Thank You

[DanRaisch]

If those folders are seeing a lot of activity (adding and deleting messages) it could be an issue with the summary files or with the message files themselves. As an experiment, choose any one of the folder for which that message appeared and, with Thunderbird closed, navigate to the Profile Folder and in the Local Folders sub-directory, locate and delete the file which has the name of the folder as well as having the file name extension .msf. Do NOT delete the file with the name of the folder but no file name extension. Restart Thunderbird and see if the next message filtered to that folder triggers the problem. If it does not, you can then delete all of the .msf files in the profile.

[johnmsch]

Thanks for the quick reply Dan, but I've already tried that. Been using Thunderbird so long I'm very familiar with the msf files. During one of my tests, I deleted all the msf files for the folders in that Travel folder. I then brought TB back up and scrolled to that folder. It still showed the total and unread message counts (which I don't understand since there were no msf files for them). I then did the Get Messages and still got errors on those folders. At that point, the message totals/unread counts disappeared.

Where would I find the previous version of TB? I'm just spending way too much time cleaning up after every email download since going to 38.2.

[photolumination]

Sounds familiar. Thunderbird TB38.2.0, Windows XP Pro, SP3.

To read new mails, or even to see most of the mails in a folder (many folders are affected), I must rebuild the .msf files. However, files with attachments do appear!

This is after following the guide in mozillaZine, "Compacting folders." Deleted all .msf files. Moved all emails from main Inbox to secondary folder. etc.

Also, many files that I am sure had been deleted from the inbox reappeared!

I am using BullGuard antivirus, which scans incoming mails. Support at BullGuard says that BullGuard does not alter the emails, except to remove content or attachment found to be infected.

I have tried to run TB without any addons, and have deleted all filters.

While I am relieved that the emails seem to exist, having to do the rebuild every folder every time I want to open it is making TB untenable.

Any ideas? Has anyone filed a bug report, I want to check all the obvious fixes before filing.

[marcpare]

I also had this problem with my filters ... I get approx. 2000 emails/day. To fix, you can try:

* for the affected filter ... go to that filter, and just click on OK ... test-try sending yourself an email to that particular filtered email to see if it has sorted it.

If not, you can also re-build the filter by:

* go to the affected filter ... take a screenshot (you could also do this with pen/paper)
* delete the filter (your emails/folder hierarchy are unaffected by this)
* create a filter with the same name and use your screenshot to reassign the same filters as the previous one

I have done both and no longer have any issues with filtering ... I jumped by quite a few versions of TB and it is understandable that there may be some "growing-pains" with a large version jump.

Marc

[photolumination]

Hi again, new observations to help in diagnosis.

1) Drafts folder retains several incremental versions of a new email, rather than just the most recent revfision.

2) The only mail that displays is mail with an attachment. All mail becomes visible only after running "repair folder" to rebuild .msf This includes new, unread mail, read mail, and mail with or without attachments.

I am running Thunderbird with add-ons disabled and all filters deleted.

[photolumination]

Solution seems to be found via viewtopic.php?f=39&t=2961263

My bad! Had used the "Attachment" button on the toolbar when seeking to locate an email, and did not realise that it remained "on". Thus presenting ONLY emails with an attachment.

Does this help anyone else, or is there more to learn?

[johnmsch]

Thanks for all the replies, but I finally figured out "what" is happening, although the "why" I don't understand. Remembered that this is an OLD issue that bites me in the butt every year or two when I try to use it. Spent some time on-and-off the past few days testing.

I've had a saved search folder named "Tagged" for many years that was setup to look for all tagged messages (condition is Tags/isn't empty). I set this up when the tag feature first appeared in Thunderbird a long time ago. Whenever I would click on that folder, it would run for a while and that's when the problems started. I could always tell it was still running by clicking on some other folder and nothing would happen (focus stayed on the Tagged folder). Eventually, the focus would change to that clicked folder, but the search would not complete. I know this because I have about a dozen tagged messages, some going back as far back as 2003, and it never found more than 4 of them. After that, the next time I got mail, the problems started. Seems like the last 1/3 of my folders exhibited the behavior mentioned above. I would then go to the last folder in my list, click on it, and get the message it was "Building summary file for...". I hit the up arrow to go to the next folder above, and get the build message. I'd keep going until I hit a folder where it did not have to do the rebuild. I could then again get mail without any errors.

Thinking this was an issue with the search folder being very old, I just deleted it, then did an Edit -> Find -> Search Messages -> Search for messages in Local Folders, Tags/isn't empty. This ran for less than a minute and found the dozen tagged messages. Checked the bottom few folders in my list and they are fine (didn't have to rebuild). I did a Save as Search Folder and named it Tagged. Came back a couple hours later and clicked on Tagged. Still have the same old problem. It only finds 4 of the tagged messages and the last 1/3 of my folders have to have the summary files manually rebuilt by clicking on each one. After doing that, I can do the manual search again and it quickly finds all the tagged messages. I went through this exercise 2 more times over the weekend as time permitted, just to be sure it was re-creatable.

I just can't understand how I can do the search manually and it works as expected, but when I save that search, it just won't work. It never finds all the tagged messages and evidently, trashes the MSF files on about 1/3 of the folders and the end of my folder tree.

Thanks again for all the replies and sorry if I sent you all on a wild goose chase.