Is there a way to cleanly change the DNS name of the POP server without wiping out its UIDLs and other data? My mail provider is moving everything, including UIDLs, over to a new server but cannot keep the name intact. So instead of POPping oldfoo.bar.net, it'll be newfoo.bar.net. I tried this once and it wiped out the UIDLs, causing the entire server's mail holding to be seen as new and downloaded again as if it were a LOT of new messages. Ugh.
This seems to stem from the fact that the internal mail structure uses the DNS name of the server as a directory name. But I'm not sure if it's safe to just change that, especially if it does not sync with the GUI. Any suggestions? Thanks.
Changing the POP server's DNS name, while keeping it intact
-
fgoldstein
- Posts: 5
- Joined: December 28th, 2017, 10:09 am
-
tanstaafl
- Moderator
- Posts: 49647
- Joined: July 30th, 2003, 5:06 pm
Re: Changing the POP server's DNS name, while keeping it int
I'd normally expect it to download messages again from the POP server only if it changed the message-id header. My impression is that the POP3 UIDL command returns the unique Message-Id for the message. If they changed the server name used in each message id I don't see how you have any control over that unless you can figure out some way to automate changing the X-UIDL stored for each message in the popstate.dat file (in the accounts local directory).
http://kb.mozillazine.org/Popstate.dat
Why not just switch to a IMAP account? You can disable all downloading to the hard disk if you want by disabling global search and offline folders (synchronization & storage). Thunderbird will fetch the contents of a message as needed for you to read it, but only stores it in memory. It will download all of the headers needed to update the *.msf file after checking for new mail, but thats pretty quick and doesn't take much space.
http://kb.mozillazine.org/Popstate.dat
Why not just switch to a IMAP account? You can disable all downloading to the hard disk if you want by disabling global search and offline folders (synchronization & storage). Thunderbird will fetch the contents of a message as needed for you to read it, but only stores it in memory. It will download all of the headers needed to update the *.msf file after checking for new mail, but thats pretty quick and doesn't take much space.
-
fgoldstein
- Posts: 5
- Joined: December 28th, 2017, 10:09 am
Re: Changing the POP server's DNS name, while keeping it int
I really really hate "answers" that respond with "don't fix your PC problem like you asked about, throw it out and buy a Mac!" or the equivalent. S/PC/POP. It's the same mentality that buys flammable organic cat litter when the order was for vermiculite (a mineral usually sold as cat litter) to store hot nuclear waste at Los Alamos (really happened, wasn't funny at the time). So please let's try to stay responsive.
The *FOLDER* in the T'bird Profile has the server's DNS name in it. Maye it's stored elsewhere in an account settings file. I didn't think UIDL contains anything besides a numeric message ID; I didn't think it contained the server's DNS name. I don't know where else in the complex T'bird file structure the server's DNS hostname might be stored, or whether renaming the folder would magically do the trick. That's why I asked here, since I figure a T'bird expert might know more about how it works, rather than just "how to use it" under "common" circumstances (like with ISPs who don't change their server names).
And ob cat litter, T'bird's IMAP is profoundly broken. The whole purpose of POP is to download mail from the ISP, and get it off the ISP's server after enough time for all of my devices to fetch it ("delete after x days"). The IMAP protocol is perfectly capable of doing that. But instead IMAP is configured to ONLY act as a window on a mail store that, other than maybe archives, resides on the server, not the client. It doesn't download unless asked, and doesn't allow deletion from the client without deleting it from the server, and vice-versa (i.e., sync). That's just wrong. It is ONE WAY to use email, typical inside a corporate office where the system is tightly controlled by IT. It is an atrocious way to try to deal with third-party ISPs, or operate when you need the attachments NOW and are not infrequently in a location with no Internet access at all.
The *FOLDER* in the T'bird Profile has the server's DNS name in it. Maye it's stored elsewhere in an account settings file. I didn't think UIDL contains anything besides a numeric message ID; I didn't think it contained the server's DNS name. I don't know where else in the complex T'bird file structure the server's DNS hostname might be stored, or whether renaming the folder would magically do the trick. That's why I asked here, since I figure a T'bird expert might know more about how it works, rather than just "how to use it" under "common" circumstances (like with ISPs who don't change their server names).
And ob cat litter, T'bird's IMAP is profoundly broken. The whole purpose of POP is to download mail from the ISP, and get it off the ISP's server after enough time for all of my devices to fetch it ("delete after x days"). The IMAP protocol is perfectly capable of doing that. But instead IMAP is configured to ONLY act as a window on a mail store that, other than maybe archives, resides on the server, not the client. It doesn't download unless asked, and doesn't allow deletion from the client without deleting it from the server, and vice-versa (i.e., sync). That's just wrong. It is ONE WAY to use email, typical inside a corporate office where the system is tightly controlled by IT. It is an atrocious way to try to deal with third-party ISPs, or operate when you need the attachments NOW and are not infrequently in a location with no Internet access at all.
-
tanstaafl
- Moderator
- Posts: 49647
- Joined: July 30th, 2003, 5:06 pm
Re: Changing the POP server's DNS name, while keeping it int
The problem is not the mail server name used in the account settings. Changing that changes the local directory setting (which you can change back using the Browse button) and invalidates the stored password since its associated with the wrong site (enter the password again when prompted and it will store it for the new site)
The problem is how messages are identified in popstate.dat. That is the file Thunderbird uses to determine if it needs to fetch a message from the POP server. Your ISP is invalidating the existing Message Ids, replacing them with new ones. Thats why the only workaround I'm aware of is to edit popstate.dat. Its a ASCII text file. However, I just forwarded a message from my fastmail.fm account to a Gmail POP account. The Message-Id: header is Message-ID: <5a0d41b4-86fa-1a80-c293-741fceb6b28b@fastmail.fm> The corresponding entry in the Gmail popstate.dat is k GmailId160a108e01612603 1514530221 . I have no idea how to remap that since it doesn't contain a domain name. Does your popstate.dat?
Thunderbird's IMAP design is not profoundly broken. You're just not its primary use case. Get over it. Most email clients on smartphones use IMAP because they have limited storage space. ISPs were initially reluctant to support IMAP because hard disk space was expensive then. Now that its much cheaper, and free IMAP accounts from Gmail etc. became popular, most ISPs added support for IMAP. But there are always holdouts, I still come across ones that don't even support secure connections.
IMAP is designed for sharing mailboxes with multiple clients, and for letting each client access all of the webmail folders. It does that well. It is up to a implementation whether you can also read your mail later on when disconnected from the Internet. Some implementations store a local copy of the message which it uses as a cache. I wish Thunderbird did that. Instead Thunderbird has a feature where you can automatically sync a remote folder with a offline folder stored on your hard disk. If you are configured to work online it displays the contents of the remote folder. If you are configured to work offline it displays the contents of the offline folder. Normally you are only configured to work offline if you have no Internet connection.
If you impose a limit on the maximum size message it can download using the accounts disk space settings, that may prevent it from downloading the entire message. The MIME Parts on Demand settings are meant to improve performance by not fetching data unless its needed. For example why fetch a 25MB attachment unless you are going to open it. Normally only the first 30KB of a message is downloaded to the hard disk by default. But you can change that. See http://kb.mozillazine.org/Entire_messag ... AP_message
Most IMAP servers support the IDLE command, which means Thunderbird will be automatically notified of new mail without having to poll every X minutes for new mail, or wait for you to press the get messages button. Its managed by the "allow immediate server notifications when new messages arrive" checkbox in the server settings, which is normally checked by default. But some old IMAP servers don't support that feature and lie about their capabilities. Perhaps your ISPs mail server is one.
The problem is how messages are identified in popstate.dat. That is the file Thunderbird uses to determine if it needs to fetch a message from the POP server. Your ISP is invalidating the existing Message Ids, replacing them with new ones. Thats why the only workaround I'm aware of is to edit popstate.dat. Its a ASCII text file. However, I just forwarded a message from my fastmail.fm account to a Gmail POP account. The Message-Id: header is Message-ID: <5a0d41b4-86fa-1a80-c293-741fceb6b28b@fastmail.fm> The corresponding entry in the Gmail popstate.dat is k GmailId160a108e01612603 1514530221 . I have no idea how to remap that since it doesn't contain a domain name. Does your popstate.dat?
Thunderbird's IMAP design is not profoundly broken. You're just not its primary use case. Get over it. Most email clients on smartphones use IMAP because they have limited storage space. ISPs were initially reluctant to support IMAP because hard disk space was expensive then. Now that its much cheaper, and free IMAP accounts from Gmail etc. became popular, most ISPs added support for IMAP. But there are always holdouts, I still come across ones that don't even support secure connections.
IMAP is designed for sharing mailboxes with multiple clients, and for letting each client access all of the webmail folders. It does that well. It is up to a implementation whether you can also read your mail later on when disconnected from the Internet. Some implementations store a local copy of the message which it uses as a cache. I wish Thunderbird did that. Instead Thunderbird has a feature where you can automatically sync a remote folder with a offline folder stored on your hard disk. If you are configured to work online it displays the contents of the remote folder. If you are configured to work offline it displays the contents of the offline folder. Normally you are only configured to work offline if you have no Internet connection.
If you impose a limit on the maximum size message it can download using the accounts disk space settings, that may prevent it from downloading the entire message. The MIME Parts on Demand settings are meant to improve performance by not fetching data unless its needed. For example why fetch a 25MB attachment unless you are going to open it. Normally only the first 30KB of a message is downloaded to the hard disk by default. But you can change that. See http://kb.mozillazine.org/Entire_messag ... AP_message
Most IMAP servers support the IDLE command, which means Thunderbird will be automatically notified of new mail without having to poll every X minutes for new mail, or wait for you to press the get messages button. Its managed by the "allow immediate server notifications when new messages arrive" checkbox in the server settings, which is normally checked by default. But some old IMAP servers don't support that feature and lie about their capabilities. Perhaps your ISPs mail server is one.
-
fgoldstein
- Posts: 5
- Joined: December 28th, 2017, 10:09 am
Re: Changing the POP server's DNS name, while keeping it int
> The problem is how messages are identified in popstate.dat. That is the file Thunderbird uses to determine if it needs to fetch a message from the POP server. Your ISP is invalidating the existing Message Ids, replacing them with new ones.
Probably not. He was copying over the entire file system. The "new" machine will get the old DNS name once his conversion is done. He is moving over *parts* of the system at once, to a new machine, so he can't just remap the DNS at once. It's ugly. So he's copying over the same numbers that are in the popstate file. I notice that the popstate.dat file contains one instance of the hostname. I'm not sure if that does or doesn't have to change.
> Thunderbird's IMAP design is not profoundly broken. You're just not its primary use case. Get over it. Most email clients on smartphones use IMAP because they have limited storage space. ISPs were initially reluctant to support IMAP because hard disk space was expensive then. Now that its much cheaper, and free IMAP accounts from Gmail etc. became popular, most ISPs added support for IMAP. But there are always holdouts, I still come across ones that don't even support secure connections.
You've just explained the problem: T'bird confuses *protocol* with *use case*. My smartphone uses IMAP, but then I only use that as a window to see what's coming in. The real work is done on Win7Pro machines via T'bird. I have over 20 GB of T'bird mail files on my laptop's 1TB SSD and while I back those up now and then to the cloud, it is important that I have control of them. Much of the volume is attachments, which do no good on somebody else's machine. I also download everything via POP to another machine, so they're "sync'd" to that extent, but then I delete from the servers (I use more than one, but the most important one is moving) after 10 or 12 days. My point is that the "ISP download" use case *can* be accommodated with IMAP, but T'bird doesn't let it.
Ideally, I would use IMAP, which does have the immediate notification function, to fetch messages. But I want to delete them from the server when I want to, and from each client when I want to, and not necessarily have all 20 GB everywhere. Especially in someone else's rent-a-system. Instead, T'bird only uses IMAP for the "sync thin client to the corporate server" use case.
The best arrangement EVAH was the old Qualcomm server + Eudora, which extended POP with an IMAP feature to show which messages had been read already. But Qualcomm got out of that business.
Probably not. He was copying over the entire file system. The "new" machine will get the old DNS name once his conversion is done. He is moving over *parts* of the system at once, to a new machine, so he can't just remap the DNS at once. It's ugly. So he's copying over the same numbers that are in the popstate file. I notice that the popstate.dat file contains one instance of the hostname. I'm not sure if that does or doesn't have to change.
> Thunderbird's IMAP design is not profoundly broken. You're just not its primary use case. Get over it. Most email clients on smartphones use IMAP because they have limited storage space. ISPs were initially reluctant to support IMAP because hard disk space was expensive then. Now that its much cheaper, and free IMAP accounts from Gmail etc. became popular, most ISPs added support for IMAP. But there are always holdouts, I still come across ones that don't even support secure connections.
You've just explained the problem: T'bird confuses *protocol* with *use case*. My smartphone uses IMAP, but then I only use that as a window to see what's coming in. The real work is done on Win7Pro machines via T'bird. I have over 20 GB of T'bird mail files on my laptop's 1TB SSD and while I back those up now and then to the cloud, it is important that I have control of them. Much of the volume is attachments, which do no good on somebody else's machine. I also download everything via POP to another machine, so they're "sync'd" to that extent, but then I delete from the servers (I use more than one, but the most important one is moving) after 10 or 12 days. My point is that the "ISP download" use case *can* be accommodated with IMAP, but T'bird doesn't let it.
Ideally, I would use IMAP, which does have the immediate notification function, to fetch messages. But I want to delete them from the server when I want to, and from each client when I want to, and not necessarily have all 20 GB everywhere. Especially in someone else's rent-a-system. Instead, T'bird only uses IMAP for the "sync thin client to the corporate server" use case.
The best arrangement EVAH was the old Qualcomm server + Eudora, which extended POP with an IMAP feature to show which messages had been read already. But Qualcomm got out of that business.
-
fgoldstein
- Posts: 5
- Joined: December 28th, 2017, 10:09 am
Re: Changing the POP server's DNS name, while keeping it int
BTW we are going to be using a workaround: Set the HOSTS file to show the new server's IP address for the old server's DNS name. That way T'bird won't actually query DNS, but the HOSTS entry will override and point to the new machine while T'bird thinks it's the old one.
-
tanstaafl
- Moderator
- Posts: 49647
- Joined: July 30th, 2003, 5:06 pm
Re: Changing the POP server's DNS name, while keeping it int
How is that going to help? When you connect to the new server it doesn't care what DNS name you used to fetch the IP address, it's (supposedly) going to be using a new DNS name in generating the UIDL.
-
fgoldstein
- Posts: 5
- Joined: December 28th, 2017, 10:09 am
Re: Changing the POP server's DNS name, while keeping it int
No, the new server will use the old UIDL file and nobody will know its "new" DNS name. We're not querying DNS and the server will think it has the old name, since it is a replacement for the old server. (Unless for some reason the server does a reverse DNS on itself, but that would break a lot.) Just a couple of thousand miles away, and not replacing everything at once.
-
tanstaafl
- Moderator
- Posts: 49647
- Joined: July 30th, 2003, 5:06 pm
Re: Changing the POP server's DNS name, while keeping it int
If the new server will use the old UIDL file then why is there a problem?
Changing the mail server name (as long as it specifies the same mailbox) in an account only effects the local directory and the saved password. I've used different mail server names and even the ip address as the server name while getting Thunderbird to work with a vpn. The only problems I had were due to outlook.com and gmail.com being aware I wasn't at that geographic location, which I had to deal with via security exceptions and/or confirming that was my login.
Changing the mail server name (as long as it specifies the same mailbox) in an account only effects the local directory and the saved password. I've used different mail server names and even the ip address as the server name while getting Thunderbird to work with a vpn. The only problems I had were due to outlook.com and gmail.com being aware I wasn't at that geographic location, which I had to deal with via security exceptions and/or confirming that was my login.