Help! Big message database reindexing performance issues

[theWoosh]

Hi Folks
I am hoping I can get some help to make thunderbird usable again...
I have been using TB since about 2005 - currently with version 64.4.0 (32 bit) installed on Windows 10 with about 6 mail accounts. My local message archive folders go back as far as 2001 (from import)!

This is a pretty big database and to help speed it up i keep the profile on a separate SSD and it has worked fine up till recently.
I have been using Awesome Auto-Archive add-on to automatically archive messages locally so that my IMAP accounts don't fill up but the indexes have become innacurate presumably due to messages being moved, so I have been unable to perform global searches that return links to messages that work (find the message when clicked)...

SO I thought I would recreate my index........ I deleted the global message database and then the real problems began.
TB has become very unresponsive - taking several seconds to do anything (even move the cursor!) - making it almost unusable (and definitely unbearable).
When I switch off global search and restart TB it works fine again so it's clearly the indexing that is killing it. However when I monitor the activity manager, it is indexing incredibly slowly - about 1 message per second! At this rate it will take about a year to recreate my search index... And I don't want to lose the ability to quickly search through all archived mail on multiple accounts as well
TB is not using much CPU or memory to do this so it's not that my PC is underpowered (16GB RAM i5 CPU).
I have also tried using the 64 bit version but this is no quicker...
The original global-messages-db.sqlite was at 1½GB - Currently it has rebuilt 0.8GB of that.

Can anyone suggest any way I can get make TB more responsive and get the indexing to speed up? I would like my favourite mail client back...
I would be sooo grateful for any suggestions
Thank-you for reading

[tanstaafl]

I'd be skeptical about the accuracy of the activity manager since Thunderbird has already rebuilt half of the search index. However the GlodaQuilla add-on supposedly lets you enable/disable search indexing on specific folders and/or accounts. You could use that to tell it not to index your archived messages, finish building the index, and someday later on tell it to also index some of the archived mail.

The web page for GlodaQuilla states it supports up to version 24, so it probably works with 52.* but not 60.*. One of the comments claims there is an updated version for 60.* at https://github.com/opto/glodaquilla

[theWoosh]

thanks #tanstaafl I'll give that a try...
Wouldn't it be great to have a stand-alone tool that could just regenerate TB index nice and quick! - I could leave it running overnight and wake up to a fully working search... Sounds like this is not going to be resolved for at least a year

I did backup the database before I deleted it but that's now a few weeks out of date so doesn't contain emails that have arrived since (plus it is full of holes which is why i tried to reindex)

If anyone else has any bright ideas I'd love to hear them!

[tanstaafl]

http://kb.mozillazine.org/Search_-_Thunderbird
https://support.mozilla.org/en-US/kb/re ... l-database

I'm not aware of a stand-alone tool to do that and suspect if one existed that it would not be noticeably faster than just letting Thunderbird rebuild it. I've never read anything that stated that you couldn't restore a backup of global-messages-db.* (you want to have the corresponding journal files too, not just the database) and then let it update itself normally, rather than starting from scratch. It is normal for it to play catchup after all. I suspect the main issue is whether the backup was partially corrupted or not.

[theWoosh]

When I restored the old database file it didn't show any messages that had been sent or received since the copy was made so I reverted to the new partial one...

also the indexing is making TB unresponsive - not sure if that would improve.
Surely 1 indexed message a second + 3 seconds delay on any TB operation doesn' seem right? IfTB is spending all it's cpu generating indexes I would then expect it to be much quicker...

[theWoosh]

PS TB in safe mode doesnt' speed things up...