Backing up email shared accounts in different machines.

User Help for Mozilla Thunderbird
Post Reply
Budgie3
Posts: 97
Joined: March 24th, 2018, 11:43 am

Backing up email shared accounts in different machines.

Post by Budgie3 »

I have read many posts on how to backup, store and recover emails but none quite offer what I need. My problem is that I need to store emails and be able to recover them if required but....

For example we have an email account for a team member (say "Sally") which resides on the same IMAP server domain (ourdomain.co.uk) and is used on two computers. In other words sally@ourdomain.co.uk is used on an office machine but also has an account on her laptop. This means that sally@ourdomain.co.uk has the same account on several profiles on different machines along with still active accounts belonging to other team members.

Sally now leaves the team but we must keep her email in secure storage as we may need to refer to it in the future. Meanwhile we need to minimise the IMAP storage volumes so sally@ourdomain.co.uk has been archived by year and subdirectory using the T'bird tool and moved to the Local Storage folder in the folder, say Sally_Archive and the emails deleted from the IMAP server.

We now need to be able to backup the Sally_Archive file off site, possibly in the Cloud because Sally has moved on with her laptop and the office machine could fail as Local Folders is just that and could be lost.

My question is how do move the Sally_Archive folder to a backup location and how may I access Sally_Archive some time later when the original profile may not still exist.

Most of the suggestions I have found propose that I backup the whole profile but if I take a backup of the whole profile which may include several email accounts other than sally@ourdomain.co.uk all of which remain active and then try and restore this backed up profile several years later my IMAP account could become very confused and possibly corrupted.

Sorry to make such a meal of this but any advice would be appreciated.
Budgie3
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: Backing up email shared accounts in different machines.

Post by tanstaafl »

http://webcache.googleusercontent.com/s ... =1&vwsrc=0 (cache of KB article about archiving)
http://webcache.googleusercontent.com/s ... =1&vwsrc=0 (cache of KB article about mail utilities)

Two suggestions:

(1) Export Sally's mail folders as mbox files using the ImportExportToolsNG add-on. Save them somewhere safe. The add-on can also import the mbox files into a child folder in "Local Folders" later on if you need to look at the contents of the mail folders. Or you could use a utility such as MboxViewer at http://sourceforge.net/projects/mbox-viewer/ to view the contents of the mbox files. That viewer has full MIME support and can view the contents of attachments. i.e. good chance you can see everything in that message that you saw using Thunderbird.

Note: If you are using version 60.* or earlier you'd use the ImportExportTools add-on instead. See http://webcache.googleusercontent.com/s ... ng_folders for more information.

(2) Use the Mbox2XML tool at http://tools.elit.nl/mbox2xml.php to create a .XML file that can be displayed using a modern browser such as Microsoft Edge, Firefox or Chrome. You'd use a menu command like file -> open file in Firefox to display it. You can add more content to that file. It lets you view the mail messages (including any attachments) without requiring Thunderbird. The main disadvantage is it doesn't provide any way to import the mail folders back into Thunderbird.

Or you could use both methods. That would let you use a browser to view the archived mail folders, but have a way to import the mail folders back into a Thunderbird profile if you ran into some sort of problem.
Budgie3
Posts: 97
Joined: March 24th, 2018, 11:43 am

Re: Backing up email shared accounts in different machines.

Post by Budgie3 »

Hi tanstaafl and very many thanks for your helpful and authoritative reply. Much to look at in more detail but glad you have understood my problem. Should I now go to the Local Folders Sally_Archive or put these, still archived, back onto IMAP server for these procedures?

There is only one supplementary question I would add. (There may be more when I try it out!) I recall reading somewhere that it was also possible to convert the mbox files to the windows ?outlook? format. Is this relevant and worth adding? Meanwhile I am inclined to take your advice and adopt both approaches.

Many thanks again.
Budgie3
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: Backing up email shared accounts in different machines.

Post by tanstaafl »

You can export sally_archive from Local Folders using both the ImportExportToolsNG add-on or Mbox2XML.

Thunderbird defaults to storing the mail in a mail folder using the mbox file format. That stores all of the messages for the folder in one file, named after the folder, with no file extension. For example, the mbox file for a inbox folder is called "inbox.". You can configure Thunderbird to use the maildir format instead. That stores each message as a separate file. Its basically a unix-centric version of using .eml files.

Mbox is more portable. The implementation in Thunderbird is more mature. However, mbox files have file locking issues if they are on a file share (especially if you want several people to access the same profile at the same time), they're prone to corruption, the consequences of corruption are far worse, and they don't support incremental backup. If you add one message to a 2GB mbox file you need to copy/backup the whole file.

The reason mbox files are so vulnerable to corruption is that is just a plain text 7bit file with messages stored sequentially. Thunderbird has to parse the file's contents to find messages in it, and if part of it becomes corrupted Thunderbird gets confused where a message ends. You need to periodically compact them to keep them healthy (not just to reduce wasted space) but if it gets too bad when you compact a folder you can lose many more messages because its confused where the deleted message ends.

The *.msf file is sometimes called a index file but it does not have a index to the messages location within the mbox file. Its really just a cache of the folder listing plus the ability to store some flags/keywords for messages. That's useful if you have a IMAP server that doesn't support keywords, it falls back to storing the tags in the folders *.msf file.

A maildir file can get corrupted but that will only effect that one message. Its also far less likely to occur since that's not influenced by the number of deleted messages in a mail folder. Its safer for an anti-virus program to scan the mail folder for viruses since you can remove the infected message without any risk of removing the other messages in the folder. Incremental backups are much easier and quicker. Minimal to no file locking is needed, eliminating many issues with file shares.

Maildir's main disadvantage is its only recently that the maildir support has gotten pretty good in Thunderbird, and it creates many many small files, which can put a strain on the file system and/or be inefficient depending upon the cluster size.

Thunderbird's maildir implementation is sometimes called maildir-lite because it only stores messages - no message flags. Normally there are new, tmp and cur directories , Thunderbird appears to omit the new subdirectory so its not clear that "An MUA doesn't have to worry about partially delivered mail: each message is safely written to disk in the tmp subdirectory before it is moved to new." applies to Thunderbird (its an example of a MUA).

In general maildir not supporting compacting is a good thing. However, I read messages using the message pane, and my IMAP accounts are configured to "just mark messages as deleted" when deleting them. So I'm used to reading new mail, deleting some of them, quickly checking that I still want to delete those messages (which I can still read, they just have a lines drawn through their entry in the folder listing ) and then compacting the folder to remove the visual clutter. I can't do that with maildir, though file -> compact folders will expunge the deleted messages. Instead I set mail.imap.expunge_threshold_number to 2, which will automatically expunges (physically remove) deleted messages once there are 20 of them waiting to be expunged. Expunging is a IMAP specific alternative to compacting.

Thunderbird doesn't have any support for converting an account from mbox to maildir. You could configure Thunderbird to use maildir for all new accounts , and replace the existing accounts. If you're using IMAP that's not a big deal since the mail is still stored on the IMAP server.

http://webcache.googleusercontent.com/s ... =1&vwsrc=0 (cache of KB article about deleting message sin IMAP accounts)
Budgie3
Posts: 97
Joined: March 24th, 2018, 11:43 am

Re: Backing up email shared accounts in different machines.

Post by Budgie3 »

Hi tanstaafl,
Wishing you a happy new year and many thanks for your authoritative explanation of the options and problems with what I am planning.
I have some time to return to this and shall accept your suggestion that I adopt both of the methods you suggest. I am still planning how to proceed and the backup strategy for this, possibly using something like pCloud and also a NAS drive.
One question which occurs to me concerns the T'bird Archive folders. Should I work with the normal messages or should I work on the archived files in their respective archive folders? I have them set up by year and with their respective filtered subdirectories intact, including the Sent messages. I would prefer to work with the archived messages as I shall keep the two years of messages active on the IMAP server for easy access but older files will only rarely need access.
Will let you know how I get on but please comment further if you have time.
Many thanks for all your help.
Regards,
Budgie3
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: Backing up email shared accounts in different machines.

Post by tanstaafl »

"One question which occurs to me concerns the T'bird Archive folders. Should I work with the normal messages or should I work on the archived files in their respective archive folders?"

All that archiving (per the built-in feature in Thunderbird) does is move messages to another folder and organize them differently. It doesn't change how they are physically stored unless you're configured to archive messages for a IMAP account in Local Folders. If the former team member(s) archived some of their messages it indicates that those message are even less likely to be needed by you later on. Knowing that may save you time if you eventually have to search their messages for something.

If I understand you correctly you want to keep 2 years of those messages on the IMAP server and store the rest elsewhere (pCloud and/or NAS drive). I suggest you archive any of their messages that are more than 2 years old, move the remaining normal messages to the IMAP server, move the archived messages to the other storage, and then copy their normal messages to the other storage too. That reduces the risk of losing that information if your normal IMAP server backup process turns out to have flaws, and might make it simpler to get rid of former team members data on the IMAP server later on.
Budgie3
Posts: 97
Joined: March 24th, 2018, 11:43 am

Re: Backing up email shared accounts in different machines.

Post by Budgie3 »

Hi tanstaafl,
I am still working on this problem and almost there i I have one question however concerning the archive feature in T'bird. What are the differences between messages in their normal directory or subdirectories and those which have been archived?
This may be relevant if I need to export messages in .eml format from my IMAP server.
Grateful for your advice on this please.
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: Backing up email shared accounts in different machines.

Post by tanstaafl »

"What are the differences between messages in their normal directory or subdirectories and those which have been archived?"

None, if they are in the same account. Any differences will be due to archiving messages in a different account (not due to them being archived).
Post Reply