Reading an mbox file - Linux

User Help for Mozilla Thunderbird
Post Reply
charlesfinley
Posts: 11
Joined: April 27th, 2016, 10:15 am

Reading an mbox file - Linux

Post by charlesfinley »

Is it possible for Thunderbird to read an active mbox file? I know you can import mbox files, but I'm not seeing where it can be made to read an mbox file.

I use fetchmail and procmail to filter mail. I use mutt for reading a lot of mbox file through the command line.

I would like to be able to use fetchmail and procmail to output some mail into an INBOX file and have Thunderbird actively read that file. Instead of checking a server for new mail, when that INBOX file is updated, Thunderbird will show the updated INBOX mbox file.

I haven't been able to get this to work, and it may not be possible. When I open an mbox file with Thunderbird, Thunderbird seems to convert that into an msf (? I think, been a couple of days sense I messed with it) file that it reads. Changes to the mbox file don't get picked until Thunderbird is restarted.
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: Reading an mbox file - Linux

Post by tanstaafl »

Thunderbird supports movmail accounts. They're just not supported in the menu anymore. You need to edit the settings in either the prefs.js file or using the config editor, for a POP account to create one.

See http://www-archive.mozilla.org/mailnews/movemail/ . Ignore the statement about it not being officially supported. My impression is its used by a significant number of users under Linux, and that if it broke, that they would fix it. There don't appear to be any open movmail bugs in the Bugzilla bug database.

Thunderbird stores messages in the mbox file, and uses the corresponding *.msf file as a cache of the folder listing to increase performance. It also stores some message attributes there such as whether or not a message has been replied to or forwarded.

http://lifehacker.com/235207/geek-to-li ... -fetchmail might also be useful though its written for somebody using cygwin.
charlesfinley
Posts: 11
Joined: April 27th, 2016, 10:15 am

Re: Reading an mbox file - Linux

Post by charlesfinley »

This still doesn't work. The only way to get the Inbox in Thunderbird to update is to Repair Folder.

I was hoping to find a solution that would just automatically populate the Inbox when procmail delivers a new message into the Inbox file on the file-system. But this just may not be possible with Thunderbird. It seems Thunderbird is intent on using the msf file instead of reading the mbox file directly.
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: Reading an mbox file - Linux

Post by tanstaafl »

No. Other folks in this forum have used movmail accounts with procmail/fetchmail. I've also read articles and blog postings that describe how to configure Thunderbird to use a spool file. I don't have time right now to search for them.

http://www.linuxquestions.org/questions ... 653-print/
charlesfinley
Posts: 11
Joined: April 27th, 2016, 10:15 am

Re: Reading an mbox file - Linux

Post by charlesfinley »

I've had some time to look into this some more. I still can't get it to work.

I found where a user on the Ubuntu forums was also having this same issue:

http://askubuntu.com/questions/301988/u ... -on-ubuntu
1. When new mail is received in the mailbox, Thunderbird won't see this until after I restart Thunderbird.
I'm thinking all of the tutorials and how this used to work were all based on older versions of Thunderbird. Perhaps before the summary file became a core part of Thunderbird. Thunderbird 45.7.0 does not appear to be able to read these mbox files in real-time.

It's a shame because this would be a real nice feature for Thunderbird to have.
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: Reading an mbox file - Linux

Post by tanstaafl »

That link is 3 years old. Its possible something broke, though I would have expected numerous posts if that happened. However, the design/use of *.msf files has not changed AFAIK.

See http://forums.mozillazine.org/viewtopic ... &t=2998751 , especially rsx11m's posts near the end. Look at your /isp/movemail.rdf file

http://forums.mozillazine.org/viewtopic ... 9&t=577528 and https://bugzilla.mozilla.org/show_bug.cgi?id=365879
http://forums.mozillazine.org/viewtopic ... &t=2728949
https://www.linuxquestions.org/question ... ot-482777/
charlesfinley
Posts: 11
Joined: April 27th, 2016, 10:15 am

Re: Reading an mbox file - Linux

Post by charlesfinley »

Well, a lot of those links are 10 years old or older.

I was able to get this to work a tiny bit if you don't procmail into Inbox. You have to output it into a different filename. But as soon as you do anything to that mbox file - it's back to not updating until Thunderbird has been restarted.

I really think this functionality is broke. But I'm also guessing that nobody is using Thunderbird as an mbox reader, so nobody ever noticed and/or cared. If I'm the only one wanting this functionality then I can hardly blame them for not really caring.

I've started to move on to just using my own IMAP service on the local machine to serve up my procmail'd mail and using Thunderbird as an IMAP client to that local service. It's a bit of a round about way of doing it. If Thunderbird would just act as an mbox reader, that'd be ideal. I may just stick to using mutt.
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: Reading an mbox file - Linux

Post by tanstaafl »

In https://www.linuxquestions.org/question ... 544-print/ somebody "grabbed the tarball from the mozilla download site, extracted just the movemail.rdf file from there to the proper location mentioned above. Restarted Thunderbird" and that added the option to create a movemail account.

The source for the mailnews.rdf file is at https://dxr.mozilla.org/comm-central/so ... vemail.rdf . Do you have that file in your installation directory?
charlesfinley
Posts: 11
Joined: April 27th, 2016, 10:15 am

Re: Reading an mbox file - Linux

Post by charlesfinley »

Well, that post is nearly 8 years old.

Creating the Movemail account isn't the issue.

The issue is that if you set the Movemail directory to say:

/home/user/mail

And have procmail set to deliver mail into:

/home/user/mail/Inbox

If you start up Thunderbird - everything is fine. It reads the Inbox file without issue. Leave Thunderbird open. Get a new message that fetchmail/procmail gets and writes to /home/user/mail/Inbox and Thunderbird just sits there. The Inbox doesn't update. The only way to read that new message in Thunderbird is to close Thunderbird and start it again OR repair the Inbox file from within Thunderbird.

Inbox appears to be a special file for Thunderbird.

If you set procmail to deliver mail into:

/home/user/mail/Special

Start Thunderbird, the mail is listed in the Special mailbox. Get a new message that fetchmail/procmail gets and writes to /home/user/mail/Special and the message list will update. BUT, if you read any messages in the Special mailbox in Thunderbird or do anything to disrupt the Special mailbox, then when fetchmail/procmail gets a new message and writes to /home/user/mail/Special then Thunderbird just sits there again. Close and Restart or Repair the Special mailbox and the new message shows.

This is not the intended behavior of an mbox reader - or at least not my definition of an mbox reader.

I want procmail to be able to write mail to /home/user/mail/Inbox. I want Thunderbird to be able to read /home/user/mail/Inbox and when changes are made to /home/user/mail/Inbox I want Thunderbird to be able to pick up on them. That doesn't not presently seem possible. Perhaps it was in ancient versions of Thunderbird, but it doesn't seem possible with modern day Thunderbird.

Perhaps Thunderbird was never able to read an active mbox file. It can read mbox files, but only static mbox files.
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: Reading an mbox file - Linux

Post by tanstaafl »

Static is a relative term. I've edited mbox files with a text editor (including adding new messages) and Thunderbird had no problem reading them when I opened the folder. I didn't even have to exit Thunderbird to edit the mbox file. Thunderbird will also discover a mbox file if its in the right directory and treat it as a mail folder, and create the corresponding *.msf file.

See if https://www.jwz.org/doc/movemail.html helps. Yes, I know its ancient.
solitone
Posts: 3
Joined: September 15th, 2017, 10:35 pm

Re: Reading an mbox file - Linux

Post by solitone »

charlesfinley wrote:Creating the Movemail account isn't the issue.

The issue is that if you set the Movemail directory to say:

/home/user/mail

And have procmail set to deliver mail into:

/home/user/mail/Inbox

If you start up Thunderbird - everything is fine. It reads the Inbox file without issue. Leave Thunderbird open. Get a new message that fetchmail/procmail gets and writes to /home/user/mail/Inbox and Thunderbird just sits there. The Inbox doesn't update. The only way to read that new message in Thunderbird is to close Thunderbird and start it again OR repair the Inbox file from within Thunderbird.

[...]

This is not the intended behavior of an mbox reader - or at least not my definition of an mbox reader.

I want procmail to be able to write mail to /home/user/mail/Inbox. I want Thunderbird to be able to read /home/user/mail/Inbox and when changes are made to /home/user/mail/Inbox I want Thunderbird to be able to pick up on them. That doesn't not presently seem possible. Perhaps it was in ancient versions of Thunderbird, but it doesn't seem possible with modern day Thunderbird.

Perhaps Thunderbird was never able to read an active mbox file. It can read mbox files, but only static mbox files.
This is *exactly* the same behaviour I'm also experiencing. To me this is a bug, and as charlesfinley noted, it was already described in point #1 here [1].

I've got Thunderbird 52.3.0, and I use postfix to deliver local mail to a mailbox file named mail/Inbox. In Thunderbird I've created a movemail account pointing to ~/mail.

As charlesfinley decribes, when I startup Thunderbird, it reads all the mailbox content. However, when a new email is delivered to the mailbox, Thunderbird doesn't get it, unless I restart it or I repair the Inbox folder (right click on the Inbox folder -> Properties -> Repair Folder).

Another issue I have is that when:
  1. I delete a message;
  2. a new message is delivered to my mailbox;
  3. I restart Thunderbird to get the new message;
the deleted message reappears as new. This does not happen if I compact the folder after deleting the message. This is related to point #2 described here [1]

[1] https://askubuntu.com/questions/301988/ ... -on-ubuntu
solitone
Posts: 3
Joined: September 15th, 2017, 10:35 pm

Re: Reading an mbox file - Linux

Post by solitone »

BTW, I've just reported this bug here:
https://bugzilla.mozilla.org/show_bug.cgi?id=1400507
solitone
Posts: 3
Joined: September 15th, 2017, 10:35 pm

Re: Reading an mbox file - Linux

Post by solitone »

charlesfinley wrote:The issue is that if you set the Movemail directory to say:

/home/user/mail

And have procmail set to deliver mail into:

/home/user/mail/Inbox

If you start up Thunderbird - everything is fine. It reads the Inbox file without issue. Leave Thunderbird open. Get a new message that fetchmail/procmail gets and writes to /home/user/mail/Inbox and Thunderbird just sits there. The Inbox doesn't update. The only way to read that new message in Thunderbird is to close Thunderbird and start it again OR repair the Inbox file from within Thunderbird.
OK charlesfinley, I've found out what's going on here. Thunderbird is designed to read new mail from /var/spool/mail/<user> file and move it to its internal Inbox (e.g. at ~/.thunderbird/<profile>/Mail/localhost/Inbox) [1].

So your procmail should deliver mail into /var/spool/mail/<user>, and in Thunderbird you may well leave untouched the local directory setting in Server Settings (e.g. ~/.thunderbird/<profile>/Mail/localhost). Thunderbird will automatically read new mail from /var/spool/mail/<user>, and move it to its mailbox file located at e.g. ~/.thunderbird/<profile>/Mail/Inbox.

AFAIK, there's no way to point Thunderbird to an input mailbox different from /var/spool/mail/<user>, so you should configure your MTA (e.g. procmail) accordingly.

Hope this helps.

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1400507#c3
charlesfinley
Posts: 11
Joined: April 27th, 2016, 10:15 am

Re: Reading an mbox file - Linux

Post by charlesfinley »

I just happened to look for this thread to see if there were any updates.

Thanks @solitone for confirming that I'm not crazy.

My solution was to just set up an IMAP service on localhost to read my mbox file and configure Thunderbird to connect to that IMAP service. Kind of a roundabout way to do this, but it was the only solution I could come up with at the time. That has worked well for me ever since.

But thanks for going the extra mile to figuring out what this issue was. This will be helpful if I ever decide to do away with the local IMAP service.
Post Reply