TB locks up when switching folders or randomly

User Help for Mozilla Thunderbird
Post Reply
aik_au
Posts: 64
Joined: November 13th, 2016, 10:13 pm

TB locks up when switching folders or randomly

Post by aik_au »

TB locks up when copying an email to a folder (for example replying to something and I have enabled copying to the folder I am replying in); or when it refreshes a folder with 20000+ mails which happens just randomly. TB does not ever say what it is doing exactly so I really have no means to fix it. Is there anything I can do to TB to make less single threaded? Randomly waiting for half a minute for an email to open is weird in 2020 :-/

The weird part is let's say I open a thread with 100k+ mails (like, linux kernel mail list), I can see TB updating the folder, the message list changes but I am still able to click on messages, switch folders so the lockups I am seeing are something else, like, IMAP timeouts.

I had a thread like this here before http://forums.mozillazine.org/viewtopic ... &t=3025233 but it never was explained why is this happening.

TB 68.11.0, Fedora 31, Corei7, 16GB RAM, lenovo x1, 3 mailboxes - 2 of gmail and one corporate, all IMAP. What number of mails per folder is considered reasonable for TB to not lock up, what is the expectation here?
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: TB locks up when switching folders or randomly

Post by tanstaafl »

I have experimented with folders that had 100K messages in the past and while there was a long delay when opening it did not lock up. You might be running into a connection timeout problem due to your ISP. However, its a poor idea to have a inbox with tens of thousands of messages. Move most of them to child folders.

Lightning is now built-in and a resource pig. "Thunderbird loads a folder's *.msf file into memory whenever it opens the folder. By default it keeps up to 30 *.msf files open for 5 minutes (and their contents in memory), unless they are "special" folders, which it never closes. It loads the inbox.msf file for every account on startup." Keeping inbox folders to a reasonable size is the easiest way to improve performance.

My normal rule of thumb is to keep my inbox under a thousand since I don't notice any delay with that limit and I like to file messages into a organized folder hierarchy (since I dislike global search and I find it awkward to use more than a dozen different tags). Given your preferences, five thousand might be a better target for you.

Have you thought about using a message filter to automate moving any new messages sent to a very active mailing list to a dedicated folder?

http://kb.mozillazine.org/Performance_-_Thunderbird
aik_au
Posts: 64
Joined: November 13th, 2016, 10:13 pm

Re: TB locks up when switching folders or randomly

Post by aik_au »

These 100K mails already sorted to a folder, I have about 20 folders for sorted mail, only one is above 100K, others between 20..60K mails, all mail is sorted and inbox is almost always empty. And even if I was not using folders, locking up with "kill or wait" from gnome is not right. ISP is fine. I am not complaining about TB being slow, lockups is what bothers me - it does that, fans go on, and I do not know when it is going to be back to normal. Reply to mail, it copies the reply to the folder with the original mail.... and locks up again. It definitely helps to have less than 50K mail in each folder though but bulk removal is another pia in gmail :-)
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: TB locks up when switching folders or randomly

Post by tanstaafl »

http://forums.mozillazine.org/viewtopic ... &t=3063338 was written by somebody whose Thunderbird installation randomly froze with 78.0 and 78.1 (but not with 68.*). That was fixed by upgrading to 78.2. Its not clear why as 78.2 didn't have a related bug fix. Perhaps you're running into a unusual configuration specific bug.

Did you have this problem with earlier versions of 68.*? http://kb.mozillazine.org/Go_back_to_an ... hunderbird

Try running Windows in safe mode with networking. If the problem doesn't occur then you probably have a interoperability problem with either the latest version of your anti-virus scanner, or a windows driver (usually either graphics or mouse).

It wouldn't hurt to also try help -> restart with add-ons disabled to see if it could have been due to a incompatible add-on.
aik_au
Posts: 64
Joined: November 13th, 2016, 10:13 pm

Re: TB locks up when switching folders or randomly

Post by aik_au »

Several earlier 68.x.x have this problem.
Disabling addons has no effect, still locks up and eats 100% CPU.
It is not Windows, it is Linux/Fedora31 but TB is not from repo, I downloaded tarball separately. No antivirus.
I guess I have to give 78 a try now...
aik_au
Posts: 64
Joined: November 13th, 2016, 10:13 pm

Re: TB locks up when switching folders or randomly

Post by aik_au »

78.2.1 has the same problem - run, open a folder which I did not open for a week = lockup. Funny enough I had TB in offline mode (new tiny icon in statusbar -> I clicked and did not unclick) - still, lockup, ~30sec later - "cannot show message in offline mode". The last message in statusbar before it locks up is often "Deleting xxx messages from folder yyy".

78.2.1 is faster otherwise, nice improvements, if only openpgp worked with yubikey (I am back to enigmail) :)
aik_au
Posts: 64
Joined: November 13th, 2016, 10:13 pm

Re: TB locks up when switching folders or randomly

Post by aik_au »

One example:

2020-09-09 04:06:14.214918 UTC - [(null) 739882: IMAP]: D/IMAP Trying auth method 0x800000000
2020-09-09 04:06:14.236537 UTC - [(null) 739882: IMAP]: D/IMAP IMAP: trying auth method 0x800000000
2020-09-09 04:06:14.236766 UTC - [(null) 739882: IMAP]: D/IMAP XOAUTH2 auth
2020-09-09 04:06:14.473846 UTC - [(null) 739882: IMAP]: D/IMAP ReadNextLine [stream=0x7f48e01838b0 nb=218 needmore=0]
2020-09-09 04:06:14.473914 UTC - [(null) 739882: IMAP]: I/IMAP 0x7f48dcb6e800:imap.gmail.com:NA:CreateNewLineFromSocket: * CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 UIDPLUS COMPRESS=DEFLATE ENABLE MOVE CONDSTORE ESEARCH UTF8=ACCEPT LIST-EXTENDED LIST-STATUS LITERAL- SPECIAL-USE APPENDLIMIT=35651584
2020-09-09 04:06:14.754347 UTC - [(null) 739882: Main Thread]: I/MsgDB nsMsgDatabase::Open(/home/aik/.thunderbird/1uyb844t.aik/ImapMail/imap.gmail-1.com/kernel.msf, FALSE, 0x7f48e888e120, FALSE)
2020-09-09 04:06:15.109892 UTC - [(null) 739882: IMAP]: D/IMAP ReadNextLine [stream=0x7f48dba7e3a0 nb=218 needmore=0]
2020-09-09 04:06:15.109924 UTC - [(null) 739882: IMAP]: I/IMAP 0x7f48da94c000:imap.gmail.com:NA:CreateNewLineFromSocket: * CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 UIDPLUS COMPRESS=DEFLATE ENABLE MOVE CONDSTORE ESEARCH UTF8=ACCEPT LIST-EXTENDED LIST-STATUS LITERAL- SPECIAL-USE APPENDLIMIT=35651584
2020-09-09 04:06:15.266816 UTC - [(null) 739882: IMAP]: D/IMAP ReadNextLine [stream=0x7f48d8916dc0 nb=218 needmore=0]
2020-09-09 04:06:15.266835 UTC - [(null) 739882: IMAP]: I/IMAP 0x7f48db050000:imap.gmail.com:NA:CreateNewLineFromSocket: * CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 UIDPLUS COMPRESS=DEFLATE ENABLE MOVE CONDSTORE ESEARCH UTF8=ACCEPT LIST-EXTENDED LIST-STATUS LITERAL- SPECIAL-USE APPENDLIMIT=35651584
2020-09-09 04:07:09.077079 UTC - [(null) 739882: Main Thread]: I/MsgDB 18 open DBs
2020-09-09 04:07:09.077102 UTC - [(null) 739882: Main Thread]: I/MsgDB /home/aik/.thunderbird/1uyb844t.aik/ImapMail/imap.gmail-1.com/watch.msf - 0 hdrs in use


55sec delay and nothing was happening :-/
The command line was:
MOZ_LOG=MsgPurge:5,MsgDB:5,MsgCopyService:5,msgcompose:5,MailDirStore:5,Mailbox:5,BayesianFilter:5,IMAP:5,IMAPOffline:5,LDAP:5,MIME:5,SMTP:5,timestamp /home/aik/thunderbird/thunderbird
aik_au
Posts: 64
Joined: November 13th, 2016, 10:13 pm

Re: TB locks up when switching folders or randomly

Post by aik_au »

this is the view of gdb when it is locked up

(gdb) info threads
Id Target Id Frame
* 1 Thread 0x7f4913766780 (LWP 739882) "thunderbird-bin" 0x00007f4909fa14a8 in ?? () from /home/aik/thunderbird/libxul.so
2 Thread 0x7f490954f700 (LWP 739919) "gmain" 0x00007f4913861a5f in poll () from /lib64/libc.so.6
3 Thread 0x7f49020ff700 (LWP 739920) "gdbus" 0x00007f4913861a5f in poll () from /lib64/libc.so.6
4 Thread 0x7f49015ff700 (LWP 739923) "Gecko_IOThread" 0x00007f49138671dd in syscall () from /lib64/libc.so.6
5 Thread 0x7f48fd5e6700 (LWP 739924) "Netlink Monitor" 0x00007f4913861a5f in poll () from /lib64/libc.so.6
6 Thread 0x7f48fd5a5700 (LWP 739925) "Socket Thread" 0x00007f4913861a5f in poll () from /lib64/libc.so.6
7 Thread 0x7f48fd564700 (LWP 739926) "Permission" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
8 Thread 0x7f490220d700 (LWP 739931) "JS Watchdog" 0x00007f4913ca407a in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
9 Thread 0x7f48fcafe700 (LWP 739932) "JS Helper" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
10 Thread 0x7f48fc8ff700 (LWP 739933) "JS Helper" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
11 Thread 0x7f48fc700700 (LWP 739934) "JS Helper" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
12 Thread 0x7f48fc501700 (LWP 739935) "JS Helper" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
13 Thread 0x7f48fbebe700 (LWP 739955) "Timer" 0x00007f4913ca407a in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
14 Thread 0x7f490183d700 (LWP 739968) "Cache2 I/O" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
15 Thread 0x7f48fbe7d700 (LWP 739969) "Cookie" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
16 Thread 0x7f48fb97d700 (LWP 739972) "Worker Launcher" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
17 Thread 0x7f48fd2ff700 (LWP 739973) "dconf worker" 0x00007f4913861a5f in poll () from /lib64/libc.so.6
18 Thread 0x7f48fae7b700 (LWP 739974) "Softwar~cThread" 0x00007f4913ca407a in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
19 Thread 0x7f48facff700 (LWP 739975) "Compositor" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
20 Thread 0x7f48fac96700 (LWP 739976) "ImgDecoder #1" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
21 Thread 0x7f48fac15700 (LWP 739977) "ImageIO" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
22 Thread 0x7f48f8897700 (LWP 739982) "ImageBridgeChld" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
23 Thread 0x7f48f8856700 (LWP 739983) "IPDL Background" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
24 Thread 0x7f48f8815700 (LWP 739984) "DataStorage" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
25 Thread 0x7f48f87d4700 (LWP 739985) "cert_storage" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
26 Thread 0x7f48ee9ff700 (LWP 739988) "DOM Worker" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
27 Thread 0x7f48ee2be700 (LWP 739996) "HTML5 Parser" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
28 Thread 0x7f48ee27d700 (LWP 739997) "mozStorage #1" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
29 Thread 0x7f48ed8a5700 (LWP 739998) "StyleThread#0" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
30 Thread 0x7f48ed864700 (LWP 739999) "StyleThread#1" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
31 Thread 0x7f48ed4ff700 (LWP 740000) "StyleThread#2" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
32 Thread 0x7f48e7d96700 (LWP 740007) "DOM Worker" 0x00007f4913861a5f in poll () from /lib64/libc.so.6
33 Thread 0x7f48ec7f7700 (LWP 740008) "localStorage DB" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
34 Thread 0x7f48dd511700 (LWP 740739) "mozStorage #4" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
35 Thread 0x7f48df7da700 (LWP 740740) "TRR Background" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
36 Thread 0x7f48df4ce700 (LWP 740742) "QuotaManager IO" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
37 Thread 0x7f48fa89b700 (LWP 740749) "ImgDecoder #2" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
38 Thread 0x7f48ddf7a700 (LWP 740753) "mozStorage #5" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
--Type <RET> for more, q to quit, c to continue without paging--
39 Thread 0x7f48dc29e700 (LWP 740755) "mozStorage #6" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
40 Thread 0x7f48dc361700 (LWP 740757) "Cache I/O" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
41 Thread 0x7f48dc2df700 (LWP 740758) "URL Classifier" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
42 Thread 0x7f48de81c700 (LWP 740759) "mozStorage #7" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
43 Thread 0x7f48de6d7700 (LWP 740794) "mozStorage #8" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
44 Thread 0x7f48ddffc700 (LWP 740796) "mozStorage #9" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
45 Thread 0x7f48dc66d700 (LWP 740798) "mozStorage #10" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
46 Thread 0x7f48df613700 (LWP 740802) "IMAP" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
47 Thread 0x7f48dbf10700 (LWP 740805) "IMAP" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
48 Thread 0x7f48de308700 (LWP 740806) "IMAP" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
49 Thread 0x7f48ddcaf700 (LWP 740807) "IMAP" 0x00007f4913ca407a in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
50 Thread 0x7f48decae700 (LWP 740871) "IMAP" 0x00007f4913ca407a in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
51 Thread 0x7f48dbcc7700 (LWP 740873) "IMAP" 0x00007f4913ca407a in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
52 Thread 0x7f48de0bf700 (LWP 740884) "mozStorage #11" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
53 Thread 0x7f48df203700 (LWP 740885) "mozStorage #12" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
54 Thread 0x7f48dc0d7700 (LWP 740886) "mozStorage #13" 0x00007f4913ca3d45 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
55 Thread 0x7f48dd142700 (LWP 745569) "IMAP" 0x00007f4913ca407a in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
56 Thread 0x7f48dbecf700 (LWP 745570) "StreamT~ns #766" 0x00007f4913ca407a in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
57 Thread 0x7f48dc118700 (LWP 745571) "StreamT~ns #767" 0x00007f4913ca407a in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
58 Thread 0x7f48dbc86700 (LWP 745572) "IMAP" 0x00007f4913ca407a in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
aik_au
Posts: 64
Joined: November 13th, 2016, 10:13 pm

Re: TB locks up when switching folders or randomly

Post by aik_au »

I keep updating the TB (now 94.0b3 (64-bit)) and it keeps locking up, annoying :-( I understand if it is slow on huge folders but it is literally the only I run which shows the unresponsive window dialog with the "wait"/"kill" choices. I am typing a message and then out of nowhere TB decides to take a break for 30..60sec :-(
aik_au
Posts: 64
Joined: November 13th, 2016, 10:13 pm

Re: TB locks up when switching folders or randomly

Post by aik_au »

95.0b3 still has the problem. The lockup delay totally depends on the number of messages in IMAP folders, cleaning up help but it does not eliminate lockups completely, still 30+ sec. I understand delays with updating the message list but I do not understand why mail editor has to freeze.
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: TB locks up when switching folders or randomly

Post by tanstaafl »

aik_au wrote:78.2.1 is faster otherwise, nice improvements, if only openpgp worked with yubikey (I am back to enigmail) :)
https://support.yubico.com/hc/en-us/art ... th-OpenPGP talks about using yubikey with OpenPGP. It requires a current version of the GnuPG. No idea if that would work with Thunderbird though.

https://support.mozilla.org/en-US/kb/op ... to-and-faq is the support article about OpenPGP in TB version 78 and later

https://blog.thunderbird.net/2020/09/op ... erbird-78/ talks about "Unlike Enigmail, OpenPGP in Thunderbird 78 does not use GnuPG software by default. This change was necessary to provide a seamless and integrated experience to users on all platforms. Instead, the software of the RNP project was chosen for Thunderbird’s core OpenPGP engine. Because RNP is a newer project in comparison to GnuPG, it has certain limitations, for example it currently lacks support for OpenPGP smartcards. As a workaround, Thunderbird 78 offers an optional configuration for advanced users, which requires additional manual setup, but which can allow the optional use of separately installed GnuPG software for private key operations."
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: TB locks up when switching folders or randomly

Post by tanstaafl »

aik_au wrote:95.0b3 still has the problem. The lockup delay totally depends on the number of messages in IMAP folders, cleaning up help but it does not eliminate lockups completely, still 30+ sec. I understand delays with updating the message list but I do not understand why mail editor has to freeze.
I think they're not paying much attention to performance yet and you aren't a typical use case. They still need to improve the situation for first time users with a large mailbox and a IMAP account. Global search/indexing can make Thunderbird unusable while its creating the search index from scratch (fetching every message in every folder) for the first 5-30 minutes.

I suggest you move most of the messages in your inbox to other folders/child folders.
Post Reply