filter operations / issues

Discussion of features in Mozilla Thunderbird
Post Reply
TomTB
Posts: 7
Joined: February 20th, 2016, 1:36 pm

filter operations / issues

Post by TomTB »

like so many people, one presumes, I get a ton of junk on Hotmail and other MS-associated accounts - and "other-other" email accounts.

for the types that send from the same email address/domain, one can "block" those addresses via Microsoft/etc.
so far so good.

but then there's the ever popular subject lines of
damaged message
deferred message
delayed email
delayed message
missed message
which come from a different spoofed email every time.

along with the usual and customary viagra/vaigra/cialis - and other popular spam From: and Subj:

and the spammers who know me as Todd - or Theresa.
and why am I so bombarded with stuff about Roslynn King?
and the voice mail/voicemail/voicemessage from every WhatsApp/Whats App in the world - and ain't it loverly - including all the social web sites I don't belong to / function in.

stop:
no suggestions about how to protect my email identity. got that part.
thanks.

what would be splendidly superior is a filter option to "permanently delete" a message.
action "Delete" i.e. moved to the Deleted folder is a lame exit. computers are here to serve people. when I have identified a From or Subj to be deleted, I do not ever wish to deal with it ever again; ever.
regrets, that is not an option - but it should be - it is an option in MS Office Outlook - but Office Outlook filters also only operate on the "InBox" - so stuff I wish to incinerate that lands in the Junk folder - I have to deal with again. like, seriously,,, I have to delete the stuff in the Junk box I wanted to delete in the first place....

it is to be noted that real life valid emails do get classified as Junk/Spam by email provides - so, no...one cannot simply ignore "junk" emails - especially in a business scenario - or if you're expecting a package from Amazon, for example.

so I set up a filter to move messages with specifically tagged From and Subj words to a user generated folder "Mist" - that's German for... well, one can look it up....
This executes a marvelous ingenious theory that anything in the Mist folder only got there by my filter rule and I can completely / safely ignore it until the cows come home and die of old age.

there is however a problem.
some/most/all email providers screen incoming email and 'classify' the msgs spam/junk/pick-a-label.
with the result that I get emails to my
"Inbox" and to my "Junk"box.
the filters I have constructed run - automatically - but only on the Inbox by ?intent/default?

yes, I can call up the filters function for that account, select "Run on (folder)" and it will mark as read and move to Mist folder. but all that only reminds me of the sci-fi quote: "To Serve Man" - am I here to keep the computer happy, or is the computer here to make my life better?

I would seriously like a filter that automatically runs on both Inbox and Junk.
The basic hugely simple bottom line is: I can/have identified certain emails by words in their Subj or From field which I wish to make evaporate into outer space and never see or deal with them again.

There is the distinct possibility that I am not doing things "right" in Thunderbird.
If anyone has ideas about how to "permanently delete" an email landing in my Inbox or Junk folder for Hotmail, LiveMail, gmail accounts - I'm all ears.

got the memo about msgFilterRules.dat - I'm so old I can even create a batch file to copy the prime .dat file to all the other accounts before launching Thunderbird. given tools of existence, I can do most any techie stuff..... I intentionally installed Thunderbird on a non-C: drive, hoping it would contain itself - but I see the msgFilterRules.dat files are buried in C:\Users . . . sigh.... one could of course keep the entire program out of Windows foul clutches - just read the environment.
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: filter operations / issues

Post by tanstaafl »

TomTB wrote:what would be splendidly superior is a filter option to "permanently delete" a message.
action "Delete" i.e. moved to the Deleted folder is a lame exit. computers are here to serve people. when I have identified a From or Subj to be deleted, I do not ever wish to deal with it ever again; ever.
My impression is that what you want has nothing to do with message filters, you just don't like how Thunderbird deletes a message.

IMAP accounts provide several options to delete messages that aren't available in POP accounts. Tools -> Account Settings -> Server Settings -> "When I delete a message" has choices for "Move it to the Trash folder", "Mark it as deleted" and "Remove it immediately". If you used a IMAP account and configured it to use "Remove it immediately" and set mail.imap.expunge_option to 1 (always expunge the deleted message) using the config editor whenever you delete a message it would just permanently disappear, with no trace of it anywhere. You wouldn't even have to periodically compact the folders as the deleted messages would already have been physically deleted.

Many ISPs do not support IMAP accounts, though some of the largest (such as Comcast, Verizon and Cox) do. The major free email providers such as Gmail, Yahoo, Outlook, and many commercial email providers support IMAP.

http://kb.mozillazine.org/Deleting_mess ... P_accounts
TomTB
Posts: 7
Joined: February 20th, 2016, 1:36 pm

Re: filter operations / issues

Post by TomTB »

thanks - actually what I'm wishing for is a filter that will check both Inbox and Junk / Spam folders and delete messages based on word(s) in either the subject line or the from line.

I understand about setting up the "Remove it immediately" - what I would prefer is a filter to delete it vs. manually plodding through the spam.

Setting Thunderbird to Remove Immediate on user action and/or empty Deleted folder on exit does entail the risk of "...oops, I didn't mean to delete that and now its gone for good." the filter has to be carefully constructed - but wouldn't make silly mistakes, one hopes.

the Thunderbird filters work in the sense they do find/flag/identify the stuff to delete but....
- most email provides "flag" messages as spam/suspicious/'this could be fake' - etc. some stuff gets past their classifications with the result that one gets "junk" in both the Inbox and the Junk folder.

- I have not found a way to have Thunderbird run the filter - automatically - on both Inbox and Junk folder

nor, as an alternative:

have Thunderbird to move *.* / i.e. all messages into the Inbox so the filter acts on "everything"

I realize not everyone has such problems. mine stem from people's computers /social sites being compromised and their contacts list 'stolen' - my email among them.

all my accounts are set up as IMAP, btw.
TomTB
Posts: 7
Joined: February 20th, 2016, 1:36 pm

Re: filter operations / issues

Post by TomTB »

so, any ideas how to make Thunderbird's filter rules work automatically on the Junk folders?

or other email clients that have better filters?
for the mods - I'm not trying to be an *ss - I'm looking for an email client will filters that operate on all incoming mail, regardless of how the email provider has "classified" it - and permanently delete a message using keywords in the From or Subj field.
mgagnonlv
Posts: 848
Joined: February 12th, 2005, 8:33 pm

Re: filter operations / issues

Post by mgagnonlv »

I actually have found that the Spam filtering done by Thunderbird is actually very good. Almost as good as the one done by gmail (my throw-away account). Once you activate "junk filters", it will seem crazy for a couple of days, but after that it's great.

To give you an idea, my account uses SpamAssasin with a score of 4, so the most blatant spams are thrown away by the server itself. I then let TB do the job automatically. I also added my own spam rule for a few writers that write what look like legitimate emails (GalleryGora sends me invitations to art shows twice a month, even though I don't want them. So they are sent in Spam too.)

I receive about 30-80 spam messages per day. There are 1-2 spam messages per day that are not detected as such, and maybe one message per week that is wrongly classified as spam (usually a password reset message). I find that a very good performance and I wish I could say the same about my phone email!

There is one added configuration I have changed: the conservation period.
– In the folder list, right-click on the "Junk Mail" folder and select "Properties".
– In the second tab (conservation rules), remove the checkmark at "use account parameters", and select "Suppress messages that are more than 5 days old".

That way, new "junk" messages are always classified by date (most recent last) and the folder stays at a manageable size.
Michel Gagnon
Montréal (Québec, Canada)
TomTB
Posts: 7
Joined: February 20th, 2016, 1:36 pm

Re: filter operations / issues

Post by TomTB »

seriously?
here's a newsflash: computers are required to make my life easier. I do not exist to keep the computer happy.

I set out on this adventure to reduce my dependence on Microsoft products. Microsoft is the #2 company of "What customers?" - IBM is the Number 1 idiot company of "we don't do customer service, we are IBM, we don't need customers." and Dell ate their lunch.....

I've reviewed any number of email clients. only the guy at iScribe/et.al. had any reaction to immediately/finally/totally deleting a user identified spam. unfortunately the iScribe product is a hobby effort and one would be well advised to avoid getting too deep into that....

so after all the sturm&drang, I find that Microsoft Live Mail product offers the option to just simply delete "obvious" spam - versus everyone else who wants to move only user identified spam in the IN box to a folder.

I hammered MS for years - like yo' Duide' - why you transmitting all the crap you've identified as spam, pfishing, etc?
Microsoft is the only place I've discovered that will, on user selection option, simply drop the trash into the bit bucket without bothering me.

I'm "test driving" Live Mail on my most spammed account; it's impressive. it looks like I'll be dumping the others who think I'm seriously crazy to immediately deleted Froms and Subjs I can specify as spam.

who'da thunk it, Microsoft actually responding to customer concerns. . . .
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: filter operations / issues

Post by tanstaafl »

I used iScribe a long time ago, I think its one of the better flyweight email clients.

"I find that Microsoft Live Mail product offers the option to just simply delete "obvious" spam - versus everyone else who wants to move only user identified spam in the IN box to a folder. "

That is because most junk filters have false positives. The developers probably figure its not worth the grief of having to regularly deal with really mad users who lose important messages because they made an error in judgement, in order to add a feature that only a small percentage of their users would use.

If your email provider adds a custom header or a string to the Subject to indicate that they think its spam you can test for that with a message filter and immediately delete it. My fastmail account for example uses SpamAssassin and adds both a X-Spam-score: and a X-Spam-hits: header (tells me what tests it failed). I can define custom headers for them so that message filters can recognize them.

A lot of people turn their nose up at SpamPal because its no longer maintained, but it still works. It will tag messages with a custom header (X-SpamPal: contains SPAM) based on DNS blacklists of mail servers. You can chose whose DNS blacklist to use.

You can configure the junk mail controls to automatically delete mail older than 1 day. You can test for really obvious junk using the message filters, though I don't recommend it as you're re-inventing the wheel. But if you do that you can have the message filter immediately delete the spam message. The FiltaQuilla add-on adds support for regular expressions in searching the headers, and executing javascript while searching.

Personally, I think the best solution is to chose a email provider that does a good job of detecting spam. I pay for a inexpensive account with fastmail.fm for example. They use SpamAssassin and support server side Sieve scripts. Between those two features I regularly go over a year between spam in my inbox. I've had an account with them for over a decade.
TomTB
Posts: 7
Joined: February 20th, 2016, 1:36 pm

Re: filter operations / issues

Post by TomTB »

or, of course, something like Thunderbird could simply have filters that operated on both Inbox and Spam/Junk Box.

see, I set up a custom folder: FilterDelete

since the email clients refuse to do a permanent delete lest they be sued, I could in theory see what was being moved - by filter - into FilterDelete.
since the email clients have nadda' clue about FilterDelete, _only_ those messages that I have identified by words in From or Subj would have any possibility of appearing there.

except of course all the stuff Hotmail and Gmail and Yahoo "identify" as spam/junk goes into my junk/spam folder and no filters apply. And as the quaintness goes, "Be sure to check your Spam folder in case our message got misplaced"

so after one goes to the trouble of constructing a filter, the email providers pre-classify it as junk, put in in the junk folder, and then:
"Be sure to check your Spam folder in case our message got misplaced"

one can sing, one can dance. one can say "Oh, use MailWasher" - lemme see, log into MailWasher and click on the non-spam/junk I want to download....really? what is the purpose of computers and filters? why not just download all the trash and garbage and click-delete it in "real life" vs. doing double duty?

the whole situation is so incredibly simple it defies mis-understanding:
- the user sets up "filters" to get rid of email they do not want to see
- the email client runs those filters on every single email that winds up on the user's computer - regardless of "folder"

the theory that
- well if you use this email provider
and
- well if you use this add on
and
- etc el barfo
you can overcome our short-sighted approach to email filtering.

would it not make more sense to take the approach:
for a filter constructed by the user, any email, from any email provider, routed to any "box," is checked and acted on by the filter.

this ain't rocket email.....
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: filter operations / issues

Post by tanstaafl »

"You can test for really obvious junk using the message filters, though I don't recommend it as you're re-inventing the wheel. But if you do that you can have the message filter immediately delete the spam message"

See http://www.stroppykitten.com/filterSubFolders/ for how to automatically also run the filters on child-folders of the Inbox. You need to create a "mail.server.default.applyIncomingFilters" string and set it to "true" using the config editor even though it seems like it should be a boolean. Another way is by setting the folder property "applyIncomingFilters" which supposedly you can do using the FiltaQuilla add-on. See the Folder Name section in http://mesquilla.com/extensions/filtaquilla/ . Its unclear whether you can use the folder property "applyIncomingFilters" to filter new mail that shows up in folders that are not a child of the inbox.

https://bugzilla.mozilla.org/show_bug.cgi?id=428574
http://superuser.com/questions/544480/h ... olders-too
http://mxr.mozilla.org/comm-central/sou ... er.cpp#629

I never got around to updating http://kb.mozillazine.org/Message_Filters about this. I need to test how well it works with the latest beta before doing that. A new major version is due soon (possibly later this month, based on the Firefox 45 schedule).
TomTB
Posts: 7
Joined: February 20th, 2016, 1:36 pm

Re: filter operations / issues

Post by TomTB »

"You can test for really obvious junk using the message filters, though I don't recommend it as you're re-inventing the wheel. But if you do that you can have the message filter immediately delete the spam message"

yup. if I am forced to "see" / "deal" with it, I can tell Thunderbird to permanently delete it.
you've missed the point.
I have constructed a filter. I expect anything that filter identifies to be deleted or moved to my specified folder.
I do not wish to plod through spam after spam hitting delete.
this is why computers and filters were invented - they are supposed to do my bidding, I am not here to justify the computer's existence.

to my point, Thunderbird has gone to great lengths to provide all kinds of filtering capability. and then applied it only to the Inbox.
as to reinventing the wheel - no. the wheel you propose exists only in limited circumstances, provided by limited email services and/or limited add-on functionality. and, only those that "integrate" with Thunderbird. what happens when these external sources go away?

/quote
See http://www.stroppykitten.com/filterSubFolders/ for how to automatically also run the filters on child-folders of the Inbox. You need to create a "mail.server.default.applyIncomingFilters" string and set it to "true" using the config editor even though it seems like it should be a boolean. Another way is by setting the folder property "applyIncomingFilters" which supposedly you can do using the FiltaQuilla add-on. See the Folder Name section in http://mesquilla.com/extensions/filtaquilla/ . Its unclear whether you can use the folder property "applyIncomingFilters" to filter new mail that shows up in folders that are not a child of the inbox.
/unquote

of 100,000 Thunderbird users, how many got past the first sentence?
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: filter operations / issues

Post by tanstaafl »

You're preaching to the choir about the flaws in the message filters design. If you want my help with a workaround I'm glad to help. Otherwise, I suspect you're just wasting your time in this thread. Most developers don't visit this forum and they have bigger fish to fry.
TomTB
Posts: 7
Joined: February 20th, 2016, 1:36 pm

Re: filter operations / issues

Post by TomTB »

you're right.

except for the little itty bitty part about the biggest fish is meeting the users' expectations.
Post Reply