Thunderbird losing messages

User Help for Mozilla Thunderbird
Post Reply
garyfritz
Posts: 256
Joined: July 30th, 2007, 9:20 am

Thunderbird losing messages

Post by garyfritz »

I've been noticing more and more cases where a message that I KNOW I filed away -- isn't there. In fact I've seen many cases where a Gloda search finds the message, but when I try to display the message, there's nothing there.

I've tried repairing folders and that doesn't help.

Any suggestions?
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: Thunderbird losing messages

Post by tanstaafl »

Are the missing messages supposed to be in the inbox? Your symptoms sound like a corrupted folder , but usually only the inbox is at risk. Gloda builds a search index (basically the message body plus some keywords) which is stored in a SQLite database. If a message disappears behind Thunderbird's back, Gloda will find the message based on the search index, even though the actual message doesn't exist anymore.

The repair folder button doesn't repair a folder, it just rebuilds the folder listing based on the contents of the mail folder.
See the real fix section in http://kb.mozillazine.org/Compacting_folders
http://kb.mozillazine.org/Recover_messa ... upt_folder might also be useful.
garyfritz
Posts: 256
Joined: July 30th, 2007, 9:20 am

Re: Thunderbird losing messages

Post by garyfritz »

"Gloda finds the message but it doesn't exist any more" sounds exactly like what I'm seeing.

Unfortunately the KB site seems to be down this morning. I'll look at the Compacting writeup to see if there's anything there that could **prevent** these lost messages.

Because if TB is going to make a habit of losing my messages, I'm going to have to find another solution. I can't afford to have it throwing away stuff I saved.
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: Thunderbird losing messages

Post by tanstaafl »

The simplest solution is to try to keep your inbox fairly empty, move messages you want to keep to other folders, and compact the folders periodically (to physically remove the deleted and hidden messages). Its not the size of the folder or the amount of messages that is the problem, its when you combine lots of messages and lots of deleted messages in the same folder that you run the risk of corruption. This isn't unique to Thunderbird.
garyfritz
Posts: 256
Joined: July 30th, 2007, 9:20 am

Re: Thunderbird losing messages

Post by garyfritz »

I compact sporadically... probably not often enough. But I'm certain some/most of the missing files were NOT in Inbox, soo...?? Unfortunately I'm having trouble finding an example right now. I'll watch for it.

I'm also having Gloda troubles. E.g. I looked for a conversation with someone, and Gloda found at least some of their responses in Inbox. It does NOT find my responses in Sent, even though Sent is clearly marked to be indexed. Gloda finds other messages in Sent.

My Inbox usually runs around 1000 messages. That's not in the "too big" range, is it?
User avatar
DanRaisch
Moderator
Posts: 127188
Joined: September 23rd, 2004, 8:57 pm
Location: Somewhere on the right coast

Re: Thunderbird losing messages

Post by DanRaisch »

If that is a POP type account, then yes, that's about 900 too many for safety. Storing messages in the Inbox of any POP type email account is a very bad practice. As the busiest area of the system, the Inbox is most at risk of data corruption and corresponding data loss. The more messages in the Inbox, the more messages at risk of loss. Creating an organization of folders, typically under Local Folders, and moving all but the most recent messages from the Inbox to that structure, will help to minimize risk of loss and improve performance.
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: Thunderbird losing messages

Post by tanstaafl »

"My Inbox usually runs around 1000 messages. That's not in the "too big" range, is it?"

Not for a IMAP account, they seem less vulnerable. For a POP account its okay, but it would be better if you could keep it down to around 500.

"It does NOT find my responses in Sent, even though Sent is clearly marked to be indexed. Gloda finds other messages in Sent."

Try using the quick filter bar to search the Sent folder. If it finds them okay that implies the folder is fine and its a Gloda specific problem.
garyfritz
Posts: 256
Joined: July 30th, 2007, 9:20 am

Re: Thunderbird losing messages

Post by garyfritz »

It's IMAP.

Yes, the quick search (Ctrl-Shift-K) finds the messages in Sent. But there are no Sent messages in the Gloda search.
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: Thunderbird losing messages

Post by tanstaafl »

It should index messages in every folder in your IMAP account. If there is no local copy of the remote folder on your hard disk because you unchecked that folder in synchronization & storage -> advanced it fetches the messages itself from the IMAP server, using a background process. The only case where that doesn't occur that I'm aware of is if you use an add-on such as GlodaQuilla to disable indexing for that folder. So I'm puzzled why its missing every message for the sent folder, rather than just a few. Let me think about that for a while. Are there any other folders that global search ignores?

It sounds like you haven't lost any messages, but something is wrong with the database used by global search. Try exiting Thunderbird and deleting global-messages-db.* in your profile. When you restart it will take a while (anywhere from a couple of minutes to a half hour depending upon how much mail you have) to rebuild the database. Before you do that, just as a precaution, right click on your boot drive letter in windows explorer, select properties in the list box, tools, and then press the check button in Error Checking.

See if the problem returns.

http://kb.mozillazine.org/Search_-_Thunderbird
http://kb.mozillazine.org/Files_and_fol ... hunderbird
garyfritz
Posts: 256
Joined: July 30th, 2007, 9:20 am

Re: Thunderbird losing messages

Post by garyfritz »

I am indeed using GlodaQuilla, though I don't remember why. I added it for some reason years ago, and forgot about it. I've turned it off for now.
It sounds like you haven't lost any messages, but something is wrong with the database used by global search.
I have several possibly-related things going on here, and that probably confused the issue. Sorry.

The issue you're referring to, where Gloda can't find messages that are definitely there, is obviously a Gloda problem. FWIW I just tried deleting the global-messages-db.sqlite file. If anything, it's worse. Now it's not finding some of the messages it was finding yesterday. It's even totally missing messages in Inbox.

I'd definitely like to fix that so Gloda works properly. But that's a relatively minor annoyance.

The other issue is much more serious, since it's losing data. Apparently TB is losing its mind and losing my messages. I've seen cases where Gloda "finds" the message, and even displays a summary of the first few lines of the message. But the file isn't there. I can't open it from the Gloda search, I can't find it in the folders. I've also seen cases where I tried to dig back through my history to find a conversation, and I discover that messages that SHOULD be there are nowhere to be found. That's what originally caught my attention.

That's the one that really worries me, and that's the one I really need to fix.
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: Thunderbird losing messages

Post by tanstaafl »

"I've seen cases where Gloda "finds" the message, and even displays a summary of the first few lines of the message. But the file isn't there. I can't open it from the Gloda search, I can't find it in the folders. "

If you don't use global search at all, does it seem like you are missing any messages?

"I've also seen cases where I tried to dig back through my history to find a conversation, and I discover that messages that SHOULD be there are nowhere to be found. That's what originally caught my attention."

Switch to view -> sort by -> threaded and see if you can spot where there is a gap in a conversation. If you find one look at the References or In-Reply-To headers (using control-U or view -> message source) of the subsequent message and see if refers to a Message-id of a missing parent message.
garyfritz
Posts: 256
Joined: July 30th, 2007, 9:20 am

Re: Thunderbird losing messages

Post by garyfritz »

tanstaafl wrote:If you don't use global search at all, does it seem like you are missing any messages?
Yes. That's what I was referring to in the second quoted paragraph.
Switch to view -> sort by -> threaded and see if you can spot where there is a gap in a conversation. If you find one look at the References or In-Reply-To headers (using control-U or view -> message source) of the subsequent message and see if refers to a Message-id of a missing parent message.
OK. I'll keep that in mind the next time I come across this behavior.
garyfritz
Posts: 256
Joined: July 30th, 2007, 9:20 am

Re: Thunderbird losing messages

Post by garyfritz »

garyfritz wrote:FWIW I just tried deleting the global-messages-db.sqlite file. If anything, it's worse. Now it's not finding some of the messages it was finding yesterday. It's even totally missing messages in Inbox.
I spoke too soon. I thought I had given it over an hour to rebuild, but apparently it needed more. It looks like it's finding what it should find now.
Post Reply