GMAIL backup to massive MBOX, Thunderbird to parse backup

Discussion of bugs in Mozilla Thunderbird
Locked
radcomtech
Posts: 1
Joined: October 27th, 2016, 7:51 am

GMAIL backup to massive MBOX, Thunderbird to parse backup

Post by radcomtech »

More of a quick tutorial comment of discovery

I used a Google (Alphabet?) tool to extract more than a decade of GMAIL into a massive 6GB MBOX file.
Basically you request one huge file, Google compiles it, and then eventually sends you the MBOX file.
It took almost a day for google to export this 6GB file (I think 15hrs) .

But what can I do with one huge MBOX file?
The great and powerful Internet (which is really just one short man behind the curtain)
procliamed THUNDERBIRD as the answer.

(1) Side effect:
I observe the native GMAIL storage, in addition to this MBOX backup
can potentially consume or exceed the free allowance of 15GB
capacity in Google Drive which was allotted by Google long ago.
So 6+ GB is now 12+ GB. For my case, the result was GDRIVE very near the capacity threshold.

I imagine that if additional cloud storage (Picasa images) consumed the remainder of the free consignment ,
Google would likely start to ask for $3 per month,
therefore,
the "cloud" would no longer be free( aka "Advertising supported";
but I bet/guess that even if I subscribed to/paid for/ extra G-storage, Adverts would still pursue me).

This is one reason to extract my GMAILS prior to 2010,
save them to my home archive, delete them,
and free more capacity to store cute cat videos in the cloud,
so that when she(cat) runs out of 9th life, she can reminisce there.

(2) ?What do you do with one big MBOX file from GMAIL?
The issue is that I needed to back up this unwieldy large cumbersome collage somewhere ,
but I wanted to separate the mails into Year directories.
Online tutorials indicated THUNDERBIRD was the best client to open MBOX files.
And I find that it is.

Other users describe the MBOX importation method into Thunderbird Mail Client
as a method of migration (away from GMAIL, to some other email service provider).

(3) Thunderbird IMPORT to ARCHIVE brought the MBOX contents into a searchable format.
However,
Thunderbird started to slow down
despite the advanced uProcessing power of Intel i5 and SSD and adequate 4GB DDR3 .

*The objective was to save or export all individual emails to EML files in directories parsed by year

I have been online since the early 1990's in BBS and other methods ENVY AOL etc
so my legacy mails and correspondances may be atypical.

The issue is that during an operation to highlight 11,000 or so emails per a specific year,
a right-click to SAVE resulted in Thunderbird 45.4.0 freezing/halting, and the Client become unresponsive.
I opine that the FILENAME of the EML saved file is just too long
when the original EMAIL SUBJECT is concatenated with date to form a long string.
I discovered this is a common issue, and then sought to find a solution without a forum query.
There is a limit to the underlying DOS filename length, likely around 255 characters
and special symbols would also drive SAVE operation to a OS-pass halt.(but not catch fire)

(incomplete pass-next down)

Solution
- - - - -

SEARCH and EXPORT (brilliant feature)
I used DATE is BEFORE mm/dd/yy (+) DATE is AFTER mm/dd/yy
in example: DATE is BEFORE 1/1/2012 + DATE is AFTER 12/31/2010
results in all email within the MBOX in year 2011.

EXPORT {ALL} (at the bottom of this dialog) can export to EML (or TEXT)

Result:
GEEmails from ()years separated by year into directories, in EML , HTML or TXT format
that are searchable, backed up to redundant drives.

:D thanks Thunderbird
I knew I could come back in time to find the magic Mozilla Dinosaur to help me
User avatar
DanRaisch
Moderator
Posts: 127234
Joined: September 23rd, 2004, 8:57 pm
Location: Somewhere on the right coast

Re: GMAIL backup to massive MBOX, Thunderbird to parse backu

Post by DanRaisch »

I used a Google (Alphabet?) tool to extract more than a decade of GMAIL into a massive 6GB MBOX file.
Just be aware that Thunderbird has a 4GB limit for mbox files ( http://kb.mozillazine.org/Limits_-_Thunderbird ) While it may be possible to access larger files it is likely that problems will develop. The first order of business would be to move messages out of that single large folder/file into multiple folders to prevent data loss.
mgagnonlv
Posts: 848
Joined: February 12th, 2005, 8:33 pm

Re: GMAIL backup to massive MBOX, Thunderbird to parse backu

Post by mgagnonlv »

I haven't tried it with Google exports and certainly not with a huge mbox file, but I would suggest you take a look at the following extension: ImportExportTools. Amongst other things, this extension allows you to open and explore an Mbox file.

I would use the extension to upload your data to a local folder. Then split the content between two or more local folders (ex.: 2010-2012 ; 2013-2016, etc.), because smaller folders are more efficient and are less prone to corruption. Once you have sorted out what you want to keep and what you want to delete, decide whether you want to archive your messages locally, load them back onto Google, another cloud provider, etc.

***************************
That being said, are you keeping the same Gmail account that you have had for the last 10 years? If so, let me suggest a different approach.

1. Use Thunderbird to access your gmail account in IMAP mode. If it's already the case (you may have to reconfigure your account server's Advanced properties), check the "Subscribed folder" to make sure you see the "All Mail" folder.
If you have been accessing your gmail account in POP mode, either make another profile (it's safer) and/or reconfigure your Gmail access to use it in IMAP.

2. You will see that all the mail you have received in your archive is already in the "All mail" folder. Even if you have been accessing via Thunderbird (Pop mode) or only the Inbox via the web interface, Google kept a copy of your messages archived in the "All mail" folder.
So once you see your "All Mail" folder synchronized in Thunderbird, you can delete messages from that box and they will be moved in the Trash, from which they will be deleted automatically after 30 days (or sooner if you do "Empty Trash" in Thunderbird).

Warning. If you delete a message from the "All mail" folder, it will also be deleted from any other folder that you have (inbox, important, family, reports, etc.). In Google's terms, if a message is not in "All mail", it doesn't exist anymore. Messages that you see in the Inbox and other folders are essentially messages from "All Mail" tagged to also be visible in other folders.

*********************************

Assuming you still use the same Gmail account, another option is to do the same in Gmail's web interface.

In the left panel, you see the list of folders, followed by "More".
Click on "More", select "All messages" and work from there.

The same warning as above applies here: email deleted from "All messages" disappears from all other views.
Michel Gagnon
Montréal (Québec, Canada)
Locked