double feed 1 RSS

Discussion about official Mozilla Thunderbird builds
Post Reply
Posts: 358
Joined: September 2nd, 2003, 3:47 pm

double feed 1 RSS

Post by fbu924 »

Hi, I don't know where this belongs, but I use aurora/earlybird daily (auto updated daily). Using it, I subscribe to about 40 feeds (showing the article summary checkboxed, not loading the web page). One (only one) shows 2 entries/day (the same summary/entry, so it's a duplicate & never used to happen till I r & r'd to w81 recently). How do I remedy that? Do I need to completely unsubscribe & resubscribe? (I like having the old feeds there, which I may come back to weeks later & download, & don't know if unsubscribing will delete those). Thank you.
Paul L
User avatar
Posts: 19979
Joined: December 7th, 2004, 8:51 pm
Location: Indiana

Re: double feed 1 RSS

Post by smsmith »

Delete the duplicates, then compact the folder and see if it stops.

If you wanted to try unsubscribing and then resubscribing, you could archive the old messages, unsubscribe, remove the folder (which I do not think happens when you unsubscribe), resubscribe and then drop the old messages that aren't downloaded again back into the folder.
Give a man a fish, and he eats for a day. Teach a man to fish, and he eats for a lifetime.
I like poetry, long walks on the beach and poking dead things with a stick.
Please do not PM me for personal support. Keep posts here in the Forums instead and we all learn.
Posts: 1029
Joined: January 28th, 2006, 3:08 pm

Re: double feed 1 RSS

Post by alta88 »

This is how it works:
1. Atom feeds (mandatory) have a unique id; Rss feeds (not mandatory) usually have a unique guid. For Rss feeds without a guid, an attempt is made to create a unique id from mandatory parts of the feed item.
2. All downloaded feed messages have a record with this id stored in feeditems.rdf and exist there as long as they exist in the publisher's file, with that id. If the publisher removes a message with the id from their file, after 24 hours the feeditems.rdf cache is also purged (on get messages biff).
3. If a publisher reuses an id after it has been purged, you will get a dupe (if the content is identical). This is an abuse of the intent behind unique ids and the publisher's error.
4. If a publisher reuses an id before it is purged, and the content is different, you will not see the new content, as it will be treated as a duplicate. Thunderbird does not use the <updated> tag currently and its misuse by publishers may make it difficult to implement.
5. If you view the source (Ctrl-U) of two apparent dupes, you will note the Message-Id header. If two apparent dupes have different Message-Id values, then they are not dupes regardless of potential identical content. Tb does not distinguish duplicate content.

If you want extreme debugging, change the Feeds.logging.console pref to debug or trace and restart, to see what happens during feed processing.

If you unsubscribe a feed url, this will clear the feeditems.rdf cache for that feed. If you subsequently resubscribe you will get dupes of all current items in the publisher's file that also exist in your feed folder.

Compaction has no effect on feed processing, it just removes marked for deletion items from the file. If you delete a folder/move it to trash, it is unsubscribed. Starting with Tb29, if you drag/drop a folder from one feed acount to another feed account, the subscription is retained (but not feeditems). For very old profiles/feed accounts (pre Tb17), it can be a good idea to create a new feed account and drag folders there (Tb29 and up), as a fresh feeds.rdf database is created; the penalty is a one time dupe possibility.

As fyi, the most recent definitive guide is here:
Post Reply