Switching to IMAP drove Tbird insane

[Mozzied]

I did a search and didn't find my specific issues, so pardon if this is a rehash, and please link me to the solutions, if there are any.

I have numerous email addresses, for various reasons. TB used to be quick and mostly trouble-free with POP. I had all my emails set to be checked on startup and it would fly through them all, badda bing badda boom, done.

However, because I got tired of having to login to each of my email accounts individually via the web to do house cleaning and otherwise manage them occasionally, I decided to set them up as the glorious IMAP that virtually everyone touts as making one's life so much easier. Yeah, not so much it turns out, at least not with TB!
Changing to IMAP was a major PITA, since there's no easy way to convert existing accounts in TB, which of course meant I had to recreate all of those accounts and remove the old POPs. And, as everyone knows, there's no easy way to SORT the accounts later, and as most know, the Manually Sort Folders add-on is broken now and apparently abandoned by its author.

Since the change to IMAP, TB is neither quick nor trouble-free.
New issues as follows:

1.) It stalls/hangs for several minutes while it retrieves messages. CPU usage is pegged and memory usage ramps way up. I have a fast enough machine and 16GB memory. It does this with no other programs running (except the processes that Win10 Pro refuses to stop). I'm on a 100Mbps internet connection.

2.) It will not check ALL the emails on startup like it used to under POP, and the ones it decides to check with each startup varies, this variation being apparently random. How do I know it's not checking them? After it stops hanging for those several minutes, I can go and check individual accounts that it missed and there will be new mail waiting.

3.) When I had the accounts set to check on startup, I would often get a popup prompt in the lower right of my screen that indicates authentication failure for EVERY IMAP account except maybe a couple or three here and there. The few it works on also varies, apparently at random.

4.) It hangs while I'm composing a message, even when it is long done checking for new mail and the CPU and memory usage has calmed down. It takes upwards of half a minute or longer, sometimes much longer for my cursor to reappear so I can keep typing, and it will often either catch up to what I kept typing right before it decided to stall and fill that in, or in deleting after I noticed it stalling, it will backspace too much and remove what I had entered. Infuriating!

This is the only program that hangs on my desktop, so my machine is not the problem.

Switching to IMAP several versions ago has caused Tbird to lose its mind, and it continues to have these issues up to the present update.

I have resorted to disabling startup email check on all my accounts and must go through and get messages manually. That usually works, but is a pain, since I have so many. One reason I have used Tbird in the first place is to have a client that will run through and check all my accounts on autopilot. It no longer does that for me. Must I go back yet again and recreate POP accounts and remove the IMAPs just to get this functionality back? Good grief!

[tanstaafl]

There is a version of the manually sort folders add-on at https://addons.thunderbird.net/en-US/th ... rs/?src=ss that supports version 68.*.

See if some of your problems disappear if you run Thunderbird in safe mode (help -> restart with add-ons disabled). You can also start it in safe mode by holding down the Shift key while double clicking on the Thunderbird shortcut.

I'll try to help with your other problems later today. In the interim it might help to know who your email providers are and what anti-virus scanner/security suite you are using. Unfortunately I've noticed that whenever a email provider adds support for using OAuth authentication that authentication problems start appearing (not all the time, but regularly enough its a nuisance) for anybody sticking to the password based authentication. I have no problem with Fastmail (a paid account), Gmail, Outlook, Yahoo and Comcast (my ISP) IMAP accounts with 68.7.

[Mozzied]

I had tried that version of manually sort folders and it's still broken. I don't have the "can't find it in tools menu" issue that a lot of people have with it. In my case, when I engage it, a new window pops up with the list of accounts in the left pane. This window is permanently sized vertically the entire height of my desktop. The scrollbar works. The right pane is empty. There are no buttons and no other way to move the accounts around (e.g., via Shift+arrow or mouse drag). So, it shows the list of accounts, but has zero functionality beyond that. I'm not the only one with this issue, as I discovered in the bug report. Now, IF there ARE dialog buttons, then they must be scrolled off screen, due to the high number of email accounts I have. Another reviewer of the add-on mentioned this as well.
I had read another dev took it upon himself to create a working version which is on github, but it was undownloadable for me, so I haven't tried that one.

I neglected to mention that I had tried safe mode and the issues persisted.

The email providers I'm using are Gmail, Hotmail, Yahoo/Rocket/Ymail, Comcast.

I only use Windows Defender, no others are installed on my system.

[DanRaisch]

and Comcast (my ISP) IMAP accounts with 68.7.

There seem to be no problems using Thunderbird 68.7 with a POP type account with Comcast either.

[Mozzied]

I didn't have mail retrieval issues with POP. Those problems started after I started using IMAP.

The hanging while composing issue was occurring with POP as well, though, if I'm remembering correctly. It's been a good while since I went to IMAP, so I may be wrong about this one.

[DanRaisch]

IMAP has significant advantages, particularly if ones use of email involves accessing accounts from multiple locations or using multiple computers. When that's not the case, some of the advantages are not that great and some of IMAP's "features" can become problematic. I use POP for some accounts and IMAP for others, based on how I need to access the accounts.

There never was functionality in Thunderbird itself to reorganize the sequence of accounts in the left had (folder) pane. That was always provided by an extension.

I had tried that version of manually sort folders and it's still broken.

What indicates that it's broken?

So all four of those accounts are set up as IMAP? Do you have the lag problem if you get messages for one account at a time? (Click on the arrow to the right side of the "Get Messages" button and select one account for which Thunderbird should check for mail.

Regarding the hang when composing: Under Tools->Options->Composition, is Thunderbird set to "Auto Save every NN minutes"? If so, what is that numeric value?
Under menu path Tools->Account Settings->Copies & Folders, what folder is selected for storing Draft messages?

Is Thunderbird set to allow Windows to allow Windows to index your message files? Find out using menu path Tools->Options->Advanced->General. Is "Allow Windows Search to search messages" selected? What about "Enable Global Search and Indexer", is that selected?

This is the only program that hangs on my desktop, so my machine is not the problem./quote]
In this case you most likely correct but with a new installation of Thunderbird, rather than after these account changes, a security program can cause all sorts of performance problems if it doesn't recognize Thunderbird as an authorized program and/or seeks to insert itself between Thunderbird and the email servers.

[tanstaafl]

I'm using version 2.0.2 of manually sort folders with TB 68.7 under Windows 10 and re-arranging the order of accounts in the folder pane works fine. I just tested it again. Your symptoms sound like you are actually using an earlier version without realizing it.

If you want to keep auto-save enabled try configuring the drafts folder to be the one in Local Folders rather than the IMAP account. I've sometimes had to do that due to network performance problems.

[tanstaafl]

"I have resorted to disabling startup email check on all my accounts and must go through and get messages manually. That usually works, but is a pain, since I have so many. "

Check "allow immediate server notifications when new messages arrive" in each accounts server settings. That's slower checking for new mail on startup but works well once there is nothing going on in the background.

"I would often get a popup prompt in the lower right of my screen that indicates authentication failure for EVERY IMAP account except maybe a couple or three here and there. The few it works on also varies, apparently at random."

A couple of examples of the error message please. Were you using authentication = normal password, OAuth2 or something else?

Any complications such as the use of a app password (rather than the normal account password) due to two-factor authentication being enabled ?

I suggest you copy&paste the output of just the mail&news accounts section of help -> troubleshooting information.

[Mozzied]

A a picture is worth a thousand words..
This is what's happening with Manually Sort Folders, account names blurred to protect the innocent.

Version:


When I run it:

[Mozzied]

I had tried that version of manually sort folders and it's still broken.

What indicates that it's broken?

See pics above. Plus, I've heard from other people who are seeing this same behavior. We may be in the minority, but it's definitely not a fluke of one.

So all four of those accounts are set up as IMAP?

I wasn't clear. I have some POP accounts remaining. All of my accounts that are no longer POP have these problems. The ones that are affected/skipped during startup at any given running of the program varies. The same ones aren't skipped every time.
The IMAP accounts were not giving me trouble UNTIL I changed them (dumped the POP and created IMAP instead). I will have to check whether any POP accounts are missed during the startup routine — almost none of those see much activity, so I'll try sending mails to them and see if they are ever skipped.

Do you have the lag problem if you get messages for one account at a time?

Not that I can recall. I highly suspect that if I set only a few to get messages on start, there would be less of a problem. It seems to be because I set them ALL to get messages on start. There are over 100 email accounts in my Tbird — hence my need for a client that will check them all for me with one or few clicks.
I go into Task Manager and watch the CPU and memory usage ramp up while Tbird is checking them on start, then once Tbird thinks it's done (even though it isn't, because it has skipped some accounts), CPU and memory usage drops and then I can do things in Tbird. Until then, if I do anything, for example just click on an inbox email, there is tremendous lag before that email gets displayed. There is tremendous lag when I use the scrollbar in that email, etc.. The lag is sometimes to the point of me being completely locked out of Tbird until Tbird stops trying to retrieve new mails. I simply cannot do anything until it calms down.
So, instead of "hangs", I guess I should have said lags/stalls, which is perhaps more technically accurate.

Regarding the hang when composing: Under Tools->Options->Composition, is Thunderbird set to "Auto Save every NN minutes"? If so, what is that numeric value?

5 minutes

Under menu path Tools->Account Settings->Copies & Folders, what folder is selected for storing Draft messages?

They were set to "Drafts Folder on: <account>". I played around with that and tried some using "Other" with the account's Drafts folder selected from the dropdown menu. Same behavior regardless of manner of setup.

Is "Allow Windows Search to search messages" selected? What about "Enable Global Search and Indexer", is that selected?

Windows search is disabled. Global search/index is enabled.

[tanstaafl]

I used the sort accounts feature of the add-on, not sort folders. Your original post said "there's no easy way to SORT the accounts later, and as most know, the Manually Sort Folders add-on is broken now" so I assumed you were trying to sort the accounts. Which are you trying to sort, and does either (sort account or sort folders) work at all for you?

I have the "extra settings" tab set to "use a custom folder" "inbox".

I normally use "plain screen compare" but when I choose "manually sort folders" in sort folders it displays the up and down arrows and I was able to change the position of the gmail inbox in the folder hierarchy. Its tough to tell what you were doing as you don't display the top part of that popup window that specifies the account name and the sort type and has the up/down arrows. You said you blurred the accounts names. I noticed you configure the folder pane to "unread folders - compact view" so that nothing in the background would display account names. So why blur the popup window and display the portion that doesn't have the parts that you said were broken? It would help to see details of exactly what was wrong with it.

The "manually sort folders" popup doesn't list any account names (other than the one you're sorting folders for) so there was no need to blur its contents if you selected "Local Folders" as the account.

[Mozzied]

Check "allow immediate server notifications when new messages arrive" in each accounts server settings.

I enabled that with each account from the beginning (when I created the accounts). Incidentally, I have disabled checking for new messages every NN minutes with every account.

A couple of examples of the error message please. Were you using authentication = normal password, OAuth2 or something else?

It happened just now when I used Get All New Messages. This prompt lower right kept popping up rapid fire as it tried to login to each (or almost all) account:


All IMAP accounts are set to OAuth2. SOME of the POP accounts are still normal password, but some are OAuth2 — I don't remember when I changed those, but as I said, I don't have problems with the POP accounts, AFAIK.

This does not happen every time Tbird retrieves messages, and it doesn't happen to the same account every time that it does happen. I don't believe there is an inherent problem with my password setup. I don't use 2-factor authentication with any of them.

[Mozzied]

I found the problem with the add-on. It doesn't like my desktop video config. It needs to be fixed for HD compatibility.

The popup window is permanently stuck above the desktop frame (out of bounds) and there was no way to force it down once opened, IF I run the add-on with Tbird in my normal desktop mode (1920x1080).

I had tried changing desktop resolution before, but I didn't go far enough with my experiments. Turns out, there are two ways to fix this, and I'll post the FYI on the add-on page, since it should work for everyone else seeing this issue.

1.) Lower desktop vertical resolution to 1024 or below. The top bar, menu and buttons become visible. Then, drag the window down and the top portion of the window will stay in view when desktop resolution is increased to normal.

2.) This method may not work for everyone, depending on their graphics card/driver. Switch display via driver from 1080p to 1080i while the popup window is stuck out of bounds. That forces the top part of it into view. Changing from 1080i to 1080p with the window stuck out of bounds also works.

Also, the window must be maximized in order to scroll to the bottom of a long accounts list and to see the Local folders bar, Restart Thunderbird and Close buttons.

Kind of a pain to have to do this, but beggars can't be choosers.

[wsmwk]

> There are over 100 email accounts in my Tbird

This could be *part* of the issue. I know some people who have many accounts do not have trouble. But there have been cases where say 50+ was a problem. It would be interesting to profile your startup - unfortunately I think profiler is currently broken.

What happens if you start thunderbird in offline mode?

[Mozzied]

Sorry for the delay in responding.

Offline mode start helps. There is a marked decrease in lag when I take it online. Tbird logs in to each account at a much slower pace than it does when I have the accounts set to retrieve messages on start.

Btw, I also tried only having my 9 most used emails set to retrieve on start. That didn't cure the sporadic authorization failure. I also get some connection timeout prompts (same location & size as the failed authorization prompts in the previous image). I don't get a timeout for each failed authorization. More often than not, the number of auth failures is greater than the timeout messages.

I typically don't have noticeable lag or any prompts when I don't retrieve anything on start and then do them manually one by one.

The problems seem to be related to when Tbird rapidly connects to multiple accounts. When it takes its time, there seem to be fewer problems. Maybe there's something in my network adapter setup that's causing a bottleneck.