Thunderbird can't connect, but Outlook and WIN10 Mail can?

[emptee]

This has me puzzled.. I'm staying at a hotel and Thunderbird seems to be unable to connect to my Zimbra server, yet Outlook 2016 and Windows 10 Mail seem to work just fine with identical settings.

I've tried using IMAP on both 143 and 993, with and without encryption (STARTTLS and SSL) with no luck. It doesn't appear that the hotel is blocking either port, but might be using some packet inspection software.

At the office, Thunderbird continues to work fine, so it doesn't appear to be an issue with the configuration.

At any rate, has anyone encountered this before?

Thunderbird Version: 52.0.1
OS: Windows 10
IMAP Server: Zimbra
Protocols tested: IMAP unencrypted, IMAP with STARTTLS, IMAP with SSL

Kind Regards,
Michael

[DanRaisch]

What is the exact text of the error message you are getting when trying to connect?

[tanstaafl]

I assume your setup configuration depends upon how your IT dept. configured Zimbra. However, based on https://wiki.xmission.com/Zimbra_Email_ ... igurations it appears that if you use a IMAP account you'd want to select SSL/TLS on port 993 with "normal password", and use your full email address as the username. If Outlook 2016 is working fine then you'd want to mimic its account settings as much as possible.

Do you get a prompt for your password before it fails? If you don't (and you're configured to ask for a password rather than have the password wizard enter it for you) that means its a problem making a tcp-ip connection to the mail server, not a IMAP specific problem. If your password is saved I suggest you write it down and then delete it using tools -> options -> security -> passwords -> saved passwords and restart Thunderbird (since the deleted password is still in memory), to make it easier to troubleshoot.

[emptee]

Thank you both for replying

What is the exact text of the error message you are getting when trying to connect?

Doh, I should have mentioned this.. If I configure port 143 with STARTTLS, I see no error at all. Thunderbird breifly displays 'Connected to xxx' in the status bar, then nothing. If I use port 993 with SSL, I get an error "Unable to connect to your IMAP server. You may have exceeded the maximum number of connections to this server. If so, use the Advanced...". This error is displayed whether I allow 1, 5 or 20 connections (from Thunderbird). The Zimbra server is not blocking connections, the logs are clean, and other clients (Outlook/Win10 Mail) continue to work while these errors are displayed.

I assume your setup configuration depends upon how your IT dept. configured Zimbra. However, based on https://wiki.xmission.com/Zimbra_Email_ ... igurations it appears that if you use a IMAP account you'd want to select SSL/TLS on port 993 with "normal password", and use your full email address as the username. If Outlook 2016 is working fine then you'd want to mimic its account settings as much as possible.

Do you get a prompt for your password before it fails? If you don't (and you're configured to ask for a password rather than have the password wizard enter it for you) that means its a problem making a tcp-ip connection to the mail server, not a IMAP specific problem. If your password is saved I suggest you write it down and then delete it using tools -> options -> security -> passwords -> saved passwords and restart Thunderbird (since the deleted password is still in memory), to make it easier to troubleshoot.

I configured the Zimbra server and it can accept STARTTLS connections on port 143 as well. Outlook is configured identically to Thunderbird, at least for what is important here.

It's definitely not the password, keep in mind:

At my office*:
Thunderbird works
Outlook works
Windows 10 Mail works

At the hotel:
Thunderbird doesn't work
Outlook still works
Windows 10 Mail still works

Kind Regards,
MIchael

[tanstaafl]

I'm assuming that you are using the same laptop and profile at work and the hotel, and that it has only a Zimbra account. My first thought was that the hotel might have NAT enabled in its router, and several of your co-workers might have already made a connection to the same server . If that happens as far as the server is concerned, they are all connecting from the same tcp-ip address. However, I can't think of any setting that would cause a problem at the hotel but not effect Outlook, and not effect Thunderbird at work.

I suspect we need more data. You could take your laptop to some nearby coffee shop or store that provides WiFi and see if you have the same symptoms as at the hotel. If you do then we at least know the hotel isn't doing something funky, and can think in terms of not at work rather than at hotel. Or you could add a Gmail IMAP account and see if you have a similar problem. Its useful to know if a problem is account/email provider specific.

You could enable IMAP logging per http://kb.mozillazine.org/Session_logging_for_mail/news but I suspect its not worth the effort as all it would do is verify that the error message was correct.

[emptee]

I'm assuming that you are using the same laptop and profile at work and the hotel, and that it has only a Zimbra account. My first thought was that the hotel might have NAT enabled in its router, and several of your co-workers might have already made a connection to the same server . If that happens as far as the server is concerned, they are all connecting from the same tcp-ip address. However, I can't think of any setting that would cause a problem at the hotel but not effect Outlook, and not effect Thunderbird at work.

I suspect we need more data. You could take your laptop to some nearby coffee shop or store that provides WiFi and see if you have the same symptoms as at the hotel. If you do then we at least know the hotel isn't doing something funky, and can think in terms of not at work rather than at hotel. Or you could add a Gmail IMAP account and see if you have a similar problem. Its useful to know if a problem is account/email provider specific.

You could enable IMAP logging per http://kb.mozillazine.org/Session_logging_for_mail/news but I suspect its not worth the effort as all it would do is verify that the error message was correct.

Thanks for replying - it won't be related to co-workers.. it's just me and my partner. My company isn't that big, yet..

Everywhere else, the same profile on the same laptop works exactly as expected.. It's just this one hotel that seems to be an issue.

Adding a gmail account seems to work fine (IMAP on port 993 with SSL). I enabled logging and found something somewhat interesting:

Port 993/SSL:
10136[13513a70]: ImapThreadMainLoop entering [this=13541800]
12900[e12140]: 13541800:mail1.mydomain.com:NA:SetupWithUrl: clearing IMAP_CONNECTION_IS_OPEN
10136[13513a70]: 13541800:mail1.mydomain.com:NA:ProcessCurrentURL: entering
10136[13513a70]: 13541800:mail1.mydomain.com:NA:ProcessCurrentURL:imap://michael%40mydomain%2Ecom@mail1.mydomain.com:993/select%3E/INBOX: = currentUrl
10136[13513a70]: ReadNextLine [stream=1caf67e0 nb=0 needmore=1]
10136[13513a70]: 13541800:mail1.mydomain.com:NA:CreateNewLineFromSocket: clearing IMAP_CONNECTION_IS_OPEN - rv = 804b0047
10136[13513a70]: 13541800:mail1.mydomain.com:NA:TellThreadToDie: close socket connection
10136[13513a70]: 13541800:mail1.mydomain.com:NA:CreateNewLineFromSocket: (null)
10136[13513a70]: 13541800:mail1.mydomain.com:NA:ProcessCurrentURL: aborting queued urls
10136[13513a70]: ImapThreadMainLoop leaving [this=13541800]


Port 143/STARTTLS:
9968[13a11920]: ImapThreadMainLoop entering [this=1c937800]
6224[1412140]: 1c937800:mail1.mydomain.com:NA:SetupWithUrl: clearing IMAP_CONNECTION_IS_OPEN
9968[13a11920]: 1c937800:mail1.mydomain.com:NA:ProcessCurrentURL: entering
9968[13a11920]: 1c937800:mail1.mydomain.com:NA:ProcessCurrentURL:imap://michael%40mydomain%2Ecom@mail1.mydomain.com:143/select%3E/INBOX: = currentUrl
9968[13a11920]: ReadNextLine [stream=18564510 nb=18 needmore=0]
9968[13a11920]: 1c937800:mail1.mydomain.com:NA:CreateNewLineFromSocket: * OK IMAP4 ready
9968[13a11920]: 1c937800:mail1.mydomain.com:NA:SendData: 1 capability
9968[13a11920]: ReadNextLine [stream=18564510 nb=285 needmore=0]
9968[13a11920]: 1c937800:mail1.mydomain.com:NA:CreateNewLineFromSocket: * CAPABILITY ACL BINARY CATENATE CHILDREN CONDSTORE ENABLE ESEARCH ESORT I18NLEVEL=1 ID IDLE IMAP4rev1 LIST-EXTENDED LIST-STATUS LITERAL+ MULTIAPPEND NAMESPACE QRESYNC QUOTA RIGHTS=ektx SASL-IR SEARCHRES SORT THREAD=ORDEREDSUBJECT UIDPLUS UNSELECT WITHIN XLIST STARTTLS LOGINDISABLED
9968[13a11920]: ReadNextLine [stream=18564510 nb=16 needmore=0]
9968[13a11920]: 1c937800:mail1.mydomain.com:NA:CreateNewLineFromSocket: 1 OK completed
9968[13a11920]: 1c937800:mail1.mydomain.com:NA:SendData: 2 STARTTLS
9968[13a11920]: ReadNextLine [stream=18564510 nb=23 needmore=0]
9968[13a11920]: 1c937800:mail1.mydomain.com:NA:CreateNewLineFromSocket: 2 BAD invalid command
9968[13a11920]: try to log in
9968[13a11920]: IMAP auth: server caps 0x10e4c6721, pref 0x1006, failed 0x0, avail caps 0x0
9968[13a11920]: (GSSAPI = 0x1000000, CRAM = 0x20000, NTLM = 0x100000, MSN = 0x200000, PLAIN = 0x1000,
LOGIN = 0x2, old-style IMAP login = 0x4, auth external IMAP login = 0x20000000, OAUTH2 = 0x800000000)
9968[13a11920]: no remaining auth method
9968[13a11920]: login failed entirely
9968[13a11920]: 1c937800:mail1.mydomain.com:NA:ProcessCurrentURL: aborting queued urls
9968[13a11920]: 1c937800:mail1.mydomain.com:NA:TellThreadToDie: close socket connection
9968[13a11920]: ImapThreadMainLoop leaving [this=1c937800]

I've bolded the interesting part - it's actually looking like it might be some odd bug in Thunderbird...? I ran the exact same commands over a telnet session and it seemed just fine:

Telnet to port 143:
* OK IMAP4 ready
1 capability
* CAPABILITY ACL BINARY CATENATE CHILDREN CONDSTORE ENABLE ESEARCH ESORT I18NLEVEL=1 ID IDLE IMAP4rev1 LIST-EXTENDED LIST-STATUS LITERAL+ MULTIAPPEND NAMESPACE QRESYNC QUOTA RIGHTS=ektx SASL-IR SEARCHRES SORT THREAD=ORDEREDSUBJECT UIDPLUS UNSELECT WITHIN XLIST STARTTLS LOGINDISABLED
1 OK completed
2 STARTTLS
2 OK completed

I wonder if something is triggering a bug in Thunderbird..?

Kind Regards,
Michael

[tanstaafl]

https://tools.ietf.org/html/rfc2595

The STARTTLS command is valid only if you haven't logged in. I noticed Thunderbird used an existing connection (see the IMAP_CONNECTION_IS_OPEN response). Perhaps it had already made a secure connection (which you are re-using) to that server and there was no LOGOUT command sent before the prior process exited. I don't know how that would happen unless Thunderbird crashed.

Another possibility is that you ran into a regression bug where it remembered some of the capability flags from a prior command. I suggest you try again where you either reboot first or wait until netstat confirms the connections timeout.

IMAP capability flags remembered across capability responses (fixed 11 years ago).

[emptee]

https://tools.ietf.org/html/rfc2595

The STARTTLS command is valid only if you haven't logged in. I noticed Thunderbird used an existing connection (see the IMAP_CONNECTION_IS_OPEN response). Perhaps it had already made a secure connection (which you are re-using) to that server and there was no LOGOUT command sent before the prior process exited. I don't know how that would happen unless Thunderbird crashed.

Another possibility is that you ran into a regression bug where it remembered some of the capability flags from a prior command. I suggest you try again where you either reboot first or wait until netstat confirms the connections timeout.

IMAP capability flags remembered across capability responses (fixed 11 years ago).

I don't think that's what is happening here - the IMAP server is responding with capabilities which indicate that the client isn't logged in (LOGINDISABLED & STARTTLS). Besides that, I can't connect at all over the hotel connection, but all is well as soon as I connect via a SSH tunnel, 3g connection, Cafe WiFi, etc..

Update:

I started logging the data between Thunderbird and the IMAP server by using socat.. Very, very bizzare..

It looks like the hotel (or perhaps their ISP) seems to have some transparent proxy which is really messing with the communication.. Here's what I found:

Thunderbird -> socat -> IMAP

* OK IMAP4 ready\r
> 2017/04/20 03:24:00.053530 length=14 from=0 to=13
1 capability\r
< 2017/04/20 03:24:00.234860 length=301 from=18 to=318
* CAPABILITY ACL BINARY CATENATE CHILDREN CONDSTORE ENABLE ESEARCH ESORT I18NLEVEL=1 ID IDLE IMAP4rev1 LIST-EXTENDED LIST-STATUS LITERAL+ MULTIAPPEND NAMESPACE QRESYNC QUOTA RIGHTS=ektx SASL-IR SEARCHRES SORT THREAD=ORDEREDSUBJECT UIDPLUS UNSELECT WITHIN XLIST STARTTLS LOGINDISABLED\r
1 OK completed\r
> 2017/04/20 03:24:00.381523 length=12 from=14 to=25
2 STARTTLS.
< 2017/04/20 03:24:00.382146 length=23 from=319 to=341
2 BAD invalid command\r
> 2017/04/20 03:24:00.508953 length=12 from=14 to=25
2 STARTTLS.
< 2017/04/20 03:24:00.509545 length=23 from=319 to=341
2 BAD invalid command\r

Telnet -> socat -> IMAP

* OK IMAP4 ready\r
> 2017/04/20 03:35:22.798705 length=1 from=0 to=0
1> 2017/04/20 03:35:23.072391 length=11 from=1 to=11
capability> 2017/04/20 03:35:23.734813 length=2 from=12 to=13
\r
< 2017/04/20 03:35:23.735518 length=301 from=18 to=318
* CAPABILITY ACL BINARY CATENATE CHILDREN CONDSTORE ENABLE ESEARCH ESORT I18NLEVEL=1 ID IDLE IMAP4rev1 LIST-EXTENDED LIST-STATUS LITERAL+ MULTIAPPEND NAMESPACE QRESYNC QUOTA RIGHTS=ektx SASL-IR SEARCHRES SORT THREAD=ORDEREDSUBJECT UIDPLUS UNSELECT WITHIN XLIST STARTTLS LOGINDISABLED\r
1 OK completed\r
> 2017/04/20 03:35:26.784538 length=1 from=14 to=14
2> 2017/04/20 03:35:27.119490 length=9 from=15 to=23
STARTTLS> 2017/04/20 03:35:27.743136 length=2 from=24 to=25
\r
< 2017/04/20 03:35:27.743768 length=16 from=319 to=334
2 OK completed\r

Putty -> socat -> IMAP

** I've removed some nonsense from the beginning where putty attempts to negotiate terminal types)

> 2017/04/20 03:39:40.685385 length=14 from=34 to=47
1 capability\r
< 2017/04/20 03:39:40.692311 length=301 from=95 to=395
* CAPABILITY ACL BINARY CATENATE CHILDREN CONDSTORE ENABLE ESEARCH ESORT I18NLEVEL=1 ID IDLE IMAP4rev1 LIST-EXTENDED LIST-STATUS LITERAL+ MULTIAPPEND NAMESPACE QRESYNC QUOTA RIGHTS=ektx SASL-IR SEARCHRES SORT THREAD=ORDEREDSUBJECT UIDPLUS UNSELECT WITHIN XLIST STARTTLS LOGINDISABLED\r
1 OK completed\r
> 2017/04/20 03:39:43.652780 length=12 from=48 to=59
2 STARTTLS.
< 2017/04/20 03:39:43.653503 length=23 from=396 to=418
2 BAD invalid command\r

So here's my theory.. it looks like the the transparent proxy is messing things up, but ONLY if spots a valid STARTTLS command within one single packet. The extra lines in the telnet session are added because the MS telnet client doesn't buffer input at all (beyond the OS TCP stack that is), so the STARTTLS command is being broken into two packets. Since putty operates in line-buffered mode, the full command is sent in one packet and is being altered..

This doesn't quite explain what's happening when using SSL over port 993, but I'd imagine it's something similar..

Kind Regards,
Michael

[tanstaafl]

Weird. If you want to pursue this any further you could configure a free VPN and see if using Thunderbird over it bypasses the proxy's manipulations.