Indexing performance after deleting messages

[BigWillyStyle42]

About 3 days ago I decided I didn't need to keep some of my old emails and ended deleting about 40,000 emails from one of my accounts via Thunderbird. The gloda indexer still has about 20K left to re-index (or I assume de-index) and while it is indexing, the performance of Thunderbird is quite slow and frustrating. It takes a noticeable amount of time to switch emails, reply to emails, move emails, open windows, etc. I'm not thrilled that it's taking so long to do this indexing, but I'd rather it took a bit longer and didn't impact the UI performance.

I'm running Thunderbird 52.7.0 on 64bit Debian Linux with an SSD. I have 8 accounts configured in my client, and I would guess there are 200,000 to 300,000 messages in total across all of those accounts.

So my questions are:

• Is this indexing taking an expected amount of time? (about 12hours per 5,000 messages)
• Are there any options for controlling how indexing works? Can I temporarily disable it when I really need Thunderbird to perform well?
• Would there have been a better way of deleting 40,000 messages?

[wsmwk]

[1]Is this indexing taking an expected amount of time? (about 12hours per 5,000 messages)
[2]Are there any options for controlling how indexing works? Can I temporarily disable it when I really need Thunderbird to perform well?
[3]Would there have been a better way of deleting 40,000 messages?

1. That seems excessively long, which makes me suspect there is something odd going on. Did you really delete them using Thunderbird UI?

2. The only open you have is to disable indexing at tools > options > advanced > general. For example you could reenable it at night after you are done working for the day.

3. Depends on your objectives and the answer to #1.

[BigWillyStyle42]

1. That seems excessively long, which makes me suspect there is something odd going on. Did you really delete them using Thunderbird UI?

Yep. The emails were all automated emails from one of my systems. I decided that there was no value in keeping the historical information, so I sorted the folder by subject, then selected the emails and hit delete. This was done in groups of about 10,000 or so. That moved them to my trash, and then I emptied the trash.

2. The only open you have is to disable indexing at tools > options > advanced > general. For example you could reenable it at night after you are done working for the day.

Does thunderbird need to be restarted after changing this setting? I just changed it, and the indexer is still going strong.

3. Depends on your objectives and the answer to #1.

My objective was to remove emails from thunderbird (and the server since these are all IMAP accounts) that I felt were no longer valuable.

[wsmwk]

1. That seems excessively long, which makes me suspect there is something odd going on. Did you really delete them using Thunderbird UI?

Yep. The emails were all automated emails from one of my systems. I decided that there was no value in keeping the historical information, so I sorted the folder by subject, then selected the emails and hit delete. This was done in groups of about 10,000 or so. That moved them to my trash, and then I emptied the trash.

2. The only open you have is to disable indexing at tools > options > advanced > general. For example you could reenable it at night after you are done working for the day.

Does thunderbird need to be restarted after changing this setting? I just changed it, and the indexer is still going strong.

3. Depends on your objectives and the answer to #1.

My objective was to remove emails from thunderbird (and the server since these are all IMAP accounts) that I felt were no longer valuable.

1. an alternative is to use shift+delete, and then they don't go to trash folder

2. yes, sorry forgot to mention that

3. Sounds like log files?? Many people don't need global search for log files. So they filter those messages to a folder, and disable indexing on that folder.

[BigWillyStyle42]

1. an alternative is to use shift+delete, and then they don't go to trash folder

Yeah, I thought about that a bit too late. Even so, they'd still have to be de-indexed, right?

3. Sounds like log files?? Many people don't need global search for log files. So they filter those messages to a folder, and disable indexing on that folder.

I didn't realize that indexing was something that could be turned off per folder. That's certainly something I'll look in to if I decide to keep emails like this again.

Thanks for the help!

[DanRaisch]

When deleting that many messages a better approach might have been to close Thunderbird after the deletions and deleting the file global-messages-db.sqlite. Restarting Thunderbird would cause it to rebuild the global search index but it would not have to deal with the thousands of deleted messages.

[wsmwk]

When deleting that many messages a better approach might have been to close Thunderbird after the deletions and deleting the file global-messages-db.sqlite. Restarting Thunderbird would cause it to rebuild the global search index but it would not have to deal with the thousands of deleted messages.

In that route you would first want to disable global search, then delete messages, then delete global-messages-db.sqlite

[DanRaisch]

Why would that be?

[BigWillyStyle42]

When deleting that many messages a better approach might have been to close Thunderbird after the deletions and deleting the file global-messages-db.sqlite. Restarting Thunderbird would cause it to rebuild the global search index but it would not have to deal with the thousands of deleted messages.

Yeah I thought about that as well. I just wasn't sure where the break-even point would be in terms of indexing time. I knew I deleted about 40,000 and my assumption was that in total I have about 300,000 across all the accounts. So I decided (possibly incorrectly) that it would be faster to allow it to de-index the 40,000 instead of re-indexing 260,000.

In any case, I was not able to turn off the indexing. Disabling it would not persist when I closed Thunderbird. So I decided that it was about time to build a new profile. Thunderbird is now happily re-downloading all my messages and re-indexing them without too much trouble. It's certainly more usable than it has been this week.

Thanks for your suggestions everyone.