Downloaded folders much larger than on server
-
- Posts: 17
- Joined: March 27th, 2017, 8:38 am
Downloaded folders much larger than on server
I am using a pretty standard installation, with five accounts in my setup. Three of the accounts are basically archive or storage accounts that are shared between my computers. I have noticed that these folders on my computer continue to grow after being initially downloaded though. Right now, I have a folder for the larger account on my local drive that clocks in at 58.4GB. In Thunderbird, the same account indicates that it should be about 18.9GB. In another case the account folder is 2.03GB on the hard drive, but 793MB in Thunderbird. Incidentally, the values that Thunderbird reports are quite similar to the values on the server where the IMAP store is actually kept.
I compact folders regularly, so that's not the issue. Anyone out there have any ideas?
I compact folders regularly, so that's not the issue. Anyone out there have any ideas?
- tanstaafl
- Moderator
- Posts: 49647
- Joined: July 30th, 2003, 5:06 pm
Re: Downloaded folders much larger than on server
More details please. Without a screen shot of a directory listing its tough to tell if are *.tmp files leftover due to interrupted attempts at compacting or maybe duplicates of mbox files in your accounts "local directory". It sounds like something is bloating it but without examples it's tough to guess what might be causing it.
http://kb.mozillazine.org/Posting_a_scr ... _the_forum
http://kb.mozillazine.org/Posting_a_scr ... _the_forum
-
- Posts: 17
- Joined: March 27th, 2017, 8:38 am
Re: Downloaded folders much larger than on server
Sorry it took me a while to get back on this. I was away from my computer for a few days.
Just before I made this post, I deleted the Windows folder from my computer and allowed Thunderbird to re-download the emails from the IMAP store. At the moment, all the Windows folder sizes are approximately correct. It's not clear to me what is causing things to blow up one way or another, or when it happens. However, in response to your post, I can say that there were no *.tmp files in the bloated folder.
When the problem occurs, the individual mbox files (no extension) within the Windows folder are considerably larger than the size of the IMAP folder as indicated in Thunderbird.
Just before I made this post, I deleted the Windows folder from my computer and allowed Thunderbird to re-download the emails from the IMAP store. At the moment, all the Windows folder sizes are approximately correct. It's not clear to me what is causing things to blow up one way or another, or when it happens. However, in response to your post, I can say that there were no *.tmp files in the bloated folder.
When the problem occurs, the individual mbox files (no extension) within the Windows folder are considerably larger than the size of the IMAP folder as indicated in Thunderbird.
- tanstaafl
- Moderator
- Posts: 49647
- Joined: July 30th, 2003, 5:06 pm
Re: Downloaded folders much larger than on server
I suggest you think about checking Tools -> Account Settings -> Server Settings -> Clean up ("Expunge") Inbox on Exit and using the config editor to set mail.imap.expunge_option to 2. That will expunge (physically remove) any deleted messages when there are at least 20 messages waiting to be expunged. It's not clear why your mbox files aren't syncing correctly, but it might help as expunging works differently than compacting.
http://kb.mozillazine.org/Deleting_mess ... P_accounts
http://kb.mozillazine.org/Deleting_mess ... P_accounts
-
- Posts: 17
- Joined: March 27th, 2017, 8:38 am
Re: Downloaded folders much larger than on server
I use the MultiXpunge add-in at least daily (usually a few times a day) to clean everything up. I think it basically compacts every folder. Can you explain what the difference is between expunging and compacting? I tried to look that up and couldn't find any clear explanation of the difference.
-
- Posts: 17
- Joined: March 27th, 2017, 8:38 am
Re: Downloaded folders much larger than on server
OK, the folders blew up again, so I have images of the following:
1) The folder pane on the left side of Thunderbird with the size column showing.
2) A screenshot of the properties window for the folder where my emails are stored on the local hard drive.
You can see here that Thunderbird indicates the entire DSI shared account should be 19082MB. However, on the hard drive, it is 32.5 GB.
To show a more specific example, the following two pictures show that the highlighted folder in Thunderbird should be 16MB, but the corresponding MSF file on my local hard drive is 33.1MB.
So, somehow, the individual MSF files are duplicating information (I think). Any ideas why this may be happening?
1) The folder pane on the left side of Thunderbird with the size column showing.
2) A screenshot of the properties window for the folder where my emails are stored on the local hard drive.
You can see here that Thunderbird indicates the entire DSI shared account should be 19082MB. However, on the hard drive, it is 32.5 GB.
To show a more specific example, the following two pictures show that the highlighted folder in Thunderbird should be 16MB, but the corresponding MSF file on my local hard drive is 33.1MB.
So, somehow, the individual MSF files are duplicating information (I think). Any ideas why this may be happening?
- tanstaafl
- Moderator
- Posts: 49647
- Joined: July 30th, 2003, 5:06 pm
Re: Downloaded folders much larger than on server
Even zooming to 300% I can only read the text in the first image (which I could read without zooming).
- DanRaisch
- Moderator
- Posts: 127234
- Joined: September 23rd, 2004, 8:57 pm
- Location: Somewhere on the right coast
Re: Downloaded folders much larger than on server
By the way, MSF files are not actually message files. They are, for lack of a better term, index files.
-
- Posts: 17
- Joined: March 27th, 2017, 8:38 am
Re: Downloaded folders much larger than on server
Sorry the images didn't link well. They're actually posted in another forum where I've posed this same question, so you can visit this link https://support.mozilla.org/en-US/quest ... er-1166007 to see them a bit more clearly.
The MSF files appear to be the containers for all the emails in any given folder. Is that an inaccurate assessment? In the case shown above, that container is, on my local drive, twice the size of the folder reported in Thunderbird.
The MSF files appear to be the containers for all the emails in any given folder. Is that an inaccurate assessment? In the case shown above, that container is, on my local drive, twice the size of the folder reported in Thunderbird.
- DanRaisch
- Moderator
- Posts: 127234
- Joined: September 23rd, 2004, 8:57 pm
- Location: Somewhere on the right coast
Re: Downloaded folders much larger than on server
Yes, that is inaccurate. The files that actually contain Thunderbird's messages are the ones with the same name as the .MSF files but with no file name extension.The MSF files appear to be the containers for all the emails in any given folder. Is that an inaccurate assessment?
-
- Posts: 17
- Joined: March 27th, 2017, 8:38 am
Re: Downloaded folders much larger than on server
Sorry, it is the extensionless files that correspond to each MSF that I meant to say are the containers. It appears these files are becoming larger than what is contained in the Thunderbird folder.
- tanstaafl
- Moderator
- Posts: 49647
- Joined: July 30th, 2003, 5:06 pm
Re: Downloaded folders much larger than on server
Your screen shots seem to state that e:\email is 32.5GB according to windows explorer. I don't see why Thunderbird claiming the DSI account is 19GB is a problem. That is far less than what appears to be your profile size (32.5GB). None of the screen shots appear to show a 58.4GB folder (either from Thunderbird's perspective or windows explorer's perspective).
What would help is a simple text table (not a screen shot) with three columns. It should have the size according to TB, the size according to windows explorer and the location of the profile, each account, and any extremely large folders.
In windows explorer the size of a mail folder would be the size of the file named after the folder, with no file extension. i.e. for the inbox folder, the mbox file "inbox.".
What would help is a simple text table (not a screen shot) with three columns. It should have the size according to TB, the size according to windows explorer and the location of the profile, each account, and any extremely large folders.
In windows explorer the size of a mail folder would be the size of the file named after the folder, with no file extension. i.e. for the inbox folder, the mbox file "inbox.".
-
- Posts: 17
- Joined: March 27th, 2017, 8:38 am
Re: Downloaded folders much larger than on server
The 58.4GB number was what the folder had blown up to when I originally made this post. Right after posting, I deleted the entire email folder and re-downloaded it from the IMAP server. For a week or so, the size of the folder on my local hard drive matched what was shown in Thunderbird, but when it blew up again, I posted the screen shots.
The 32.5GB is not the profile size. It is just the size of the email store on the local hard drive. I have five accounts in Thunderbird. Two of them are kept in my profile folder within %appdata%, but the other three are kept in different locations on a local drive. The local hard drive folder I show in the second screenshot corresponds to the DSI shared account. The sizes should be the same, and after deleting the folder and re-downloading everything in Thunderbird, they typically are. The problem is that after a few weeks, the local hard drive store inflates inexplicably. As I showed in the second set of screenshots, a particular example within the account shows that the S10012-0001 Shipping folder is 16MB but the corresponding file in the E drive is over 33MB. This indicates to me that the individual files within the email store folder are inflating, but I just don't understand why, which is why I'm reaching out to all you smart folks here
The 32.5GB is not the profile size. It is just the size of the email store on the local hard drive. I have five accounts in Thunderbird. Two of them are kept in my profile folder within %appdata%, but the other three are kept in different locations on a local drive. The local hard drive folder I show in the second screenshot corresponds to the DSI shared account. The sizes should be the same, and after deleting the folder and re-downloading everything in Thunderbird, they typically are. The problem is that after a few weeks, the local hard drive store inflates inexplicably. As I showed in the second set of screenshots, a particular example within the account shows that the S10012-0001 Shipping folder is 16MB but the corresponding file in the E drive is over 33MB. This indicates to me that the individual files within the email store folder are inflating, but I just don't understand why, which is why I'm reaching out to all you smart folks here
- tanstaafl
- Moderator
- Posts: 49647
- Joined: July 30th, 2003, 5:06 pm
Re: Downloaded folders much larger than on server
Why do you have offline folders enabled? Is it to read them offline or to have something that you can backup?
What version are you using? One possible workaround would be to convert the account from mbox files to maildir (single file per messages). The maildir code is not as mature (biggest risk seems to be when moving a message between accounts using maildir and accounts using mbox or vice versa) but eliminates many potential problems because you no longer have multiple messages in a single file.
https://wiki.mozilla.org/Thunderbird/Maildir (out of date but useful background)
Have you verified the global search/indexing database (global-messages-db.sqlite) isn't growing out of control too? It should be no larger (usually smaller) than the combined size of the remote folders and whatever POP accounts/Local Folders you have (though I think it has a minimum size of about 10MB).
What version are you using? One possible workaround would be to convert the account from mbox files to maildir (single file per messages). The maildir code is not as mature (biggest risk seems to be when moving a message between accounts using maildir and accounts using mbox or vice versa) but eliminates many potential problems because you no longer have multiple messages in a single file.
https://wiki.mozilla.org/Thunderbird/Maildir (out of date but useful background)
Have you verified the global search/indexing database (global-messages-db.sqlite) isn't growing out of control too? It should be no larger (usually smaller) than the combined size of the remote folders and whatever POP accounts/Local Folders you have (though I think it has a minimum size of about 10MB).
-
- Posts: 17
- Joined: March 27th, 2017, 8:38 am
Re: Downloaded folders much larger than on server
Do you mean the "Local Folders" at the bottom of my list? I don't actually use this folder. If you are instead asking why do I keep my email stored locally, the answer is that we have set up our system so that all emails are shared across our organization. We, of course, receive emails at our own individual email addresses, through the corresponding account, but an email that is related to a given project is then moved to a folder in the DSI shared account IMAP so that everyone in the company has access to it. The DSI partnership and Personal IMAPs that you see in my screenshot are similar. Every computer in our organization has all of the emails from the DSI shared account saved to their local system as a redundant backup.
I'm running two different versions (52.9.1 on my laptop and 60.2.1 on my desktop). The problem occurs on both installations, and has been occurring through multiple upgrades.
I used to have all our systems on maildir, but as you mentioned, moving messages around was causing too many problems. And as I recall, I had similar issues with the local stores inflating over time. I also found that offline access didn't work well with maildir.
My global-messages-db.sqlite file is 214.6MB on both computers. Far smaller than the combined size of all my IMAPs.
I'm running two different versions (52.9.1 on my laptop and 60.2.1 on my desktop). The problem occurs on both installations, and has been occurring through multiple upgrades.
I used to have all our systems on maildir, but as you mentioned, moving messages around was causing too many problems. And as I recall, I had similar issues with the local stores inflating over time. I also found that offline access didn't work well with maildir.
My global-messages-db.sqlite file is 214.6MB on both computers. Far smaller than the combined size of all my IMAPs.