whitelist filtering

Discussion of features in Mozilla Thunderbird
petrarch
Posts: 56
Joined: January 16th, 2003, 10:07 am

Re: Spamassassin

Post by petrarch »

comomolo wrote:Spamassassin is IMHO precisely the kind of bloatware


Spamassassin is absolutely not bloatware. It provides a tremendous service to thousands of people. If you're going to accuse it of being bloatware, you'll need to defend your assertion

no one but geeks would like to use. It's a server side solution, thought for administrators and computer geeks, not "for the rest of us".


Where does it say that? Anybody is free to use it. The idea that only geeks would want to reduce the amount of spam they need to deal with is ludicrous. As to its being a server side solution, I'm not running a mail server, yet it's working fine here.

I had it as an option in my hosting package and have disabled it a couple of weeks after I started using it. It's complicated, error prone, _does_ yield false positives (not to mention the many false negatives)


What version were you running? Spam and spamassassin are both constantly morphing. If you want it to be effective, you need to use the latest version. I never claimed it didn't yield false positives or negatives. I will say that it's extremely efficient at mininimizing both. They calibrate the scores for each release to do just that, so there's quantitative evidence to prove this. Perhaps you were using an outdated version?

and I simply don't want it to manage my mail, because I CAN'T UNDERSTAND ITS MANY TWEAKS, and I DON'T WANT TO LEARN THEM


You don't need to tweak it to use it. My only tweak? The use of a whitelist; anybody that can use a computer can stick email addresses in a text file.

What I want to see and can't find is a software that would reply to an unknown sender the classic blurry image of a word only readable by humans. If the "human" who sent me his/her message replies properly they'll be included in my whitelist and never asked again (this process should be automated and, essentially, it would be the core of the software I'm talking about). This should run at server level (the whole point is avoiding the download of tons of garbage) and -here comes my suggestion- have an interface through the mail client, in this case, Thunderbird, or if it's so difficult (I'm not a developer) through a web interface. The whitelist would reside in the server but managed through the client. Easy and clean.


Such software already exists. It's not what the original poster of this thread is talking about though, so I suggest you start a new thread if you'd like to discuss it.

My mailbox isn't a public place.


Fine, but that's only going to be true as long as you don't publicly post it in the future.

You can write me, but only if I like you you can keep writing me. I can't see how this hurts "innocent people". All antispam solutions I've seen so far -including Spamassassin- force me to download spam messages from my server in order to check if the spam filtering has been done right. As long as this isn't changed in the anti-spam tools designers' minds, so called "anti spam solutions" are simplistic (albeit very complicated algorithm wise) patches to a serious problem: megabytes downloaded for nothing and time spent checking the work done by the anti spam filter.

I just want to verify who's trying to write me, and since most spam is machine originated and reply addresses used by them are fake I would never see the spam messages and not a single human being would be prevented from reaching my mailbox (as long as he/she can identify a few letters inside a blurry image...). Everybody is aware about the spam problem. Nobody will feel insulted if you explain clearly why authorization to write you is needed.


Your idea is riddled with problems. Please start a different thread so we don't sully this one.

Besides, if you believe YOUR mailbox is a public place, you can always keep using Spamassassin and other inelegant solutions like it... :-)


If I didn't want my email address to be public, it wouldn't appear in public. And you haven't successfully explained how Spamassassin is inelegant.
comomolo
Posts: 80
Joined: May 1st, 2003, 7:35 pm

Re: Spamassassin

Post by comomolo »

petrarch wrote:
comomolo wrote:Spamassassin is IMHO precisely the kind of bloatware


Spamassassin is absolutely not bloatware. It provides a tremendous service to thousands of people. If you're going to accuse it of being bloatware, you'll need to defend your assertion


You probably need to defend your assertion too then. What is "thousands" of people? Were do you get your statistics from?

no one but geeks would like to use. It's a server side solution, thought for administrators and computer geeks, not "for the rest of us".


Where does it say that? Anybody is free to use it. The idea that only geeks would want to reduce the amount of spam they need to deal with is ludicrous. As to its being a server side solution, I'm not running a mail server, yet it's working fine here.


Well, I know my English is not very good, but I'm pretty sure your understanding is very slanted: everyone wants to reduce their spam. Only geeks can use Spamassassin because that software forgot about a simple issue: computer users are to computer geeks what car drivers are to mechanics... You guys (I assume you're a geek) think everyone should know certain things about computers, but frankly, nobody has entitled you to make such assumption. Sorry if they have released a windows client side version of Spamassassin, I wasn't aware of that, but taken the "usability" of the software I tried, I can't think of any reason to try it again.


I had it as an option in my hosting package and have disabled it a couple of weeks after I started using it. It's complicated, error prone, _does_ yield false positives (not to mention the many false negatives)


What version were you running? Spam and spamassassin are both constantly morphing. If you want it to be effective, you need to use the latest version. I never claimed it didn't yield false positives or negatives. I will say that it's extremely efficient at mininimizing both. They calibrate the scores for each release to do just that, so there's quantitative evidence to prove this. Perhaps you were using an outdated version?


Antoher geek issue: be prepared to check for newer versions, download them, install them, configure them, every time you find your software is not working... or every other week. You don't really want me to check what version I was using six months ago when I tested the software, will you? Again think of me as a user, not as a computer expert, no matter how hard it is for you to understand the difference, there is one.

and I simply don't want it to manage my mail, because I CAN'T UNDERSTAND ITS MANY TWEAKS, and I DON'T WANT TO LEARN THEM


You don't need to tweak it to use it. My only tweak? The use of a whitelist; anybody that can use a computer can stick email addresses in a text file.


Besides the fact that you have talked here about changing sensitiviy levels -i.e.: twekaing- anybody, you're right, CAN put addresses in a text file. But only those few Linux users and the like out there (please go to the statistics if you feel Linux users and the like are "a mass") actually WANT to do that. We like point and click, we lo ve GUIs, we are fond of being treated like people, we regular users...

What I want to see and can't find is a software that would reply to an unknown sender the classic blurry image of a word only readable by humans. If the "human" who sent me his/her message replies properly they'll be included in my whitelist and never asked again (this process should be automated and, essentially, it would be the core of the software I'm talking about). This should run at server level (the whole point is avoiding the download of tons of garbage) and -here comes my suggestion- have an interface through the mail client, in this case, Thunderbird, or if it's so difficult (I'm not a developer) through a web interface. The whitelist would reside in the server but managed through the client. Easy and clean.


Such software already exists. It's not what the original poster of this thread is talking about though, so I suggest you start a new thread if you'd like to discuss it.


Oh, yes it is: he's talking about whitelists and I'm talking about whitelists, only with a different approach. He's talking about one solution (Qurb) which I don't completely like beacuse it's client based and I'm proposing a mixed client-server solution. If you know about such a solution (you seem to) and don't want to tell, I will suspect you have some sort of religious commitment to Spamassassin... Would you be so kind to point me to a solution like the one I propose. Believe me, I'd be rally happy if someone has already had the idea and implented it.

My mailbox isn't a public place.


Fine, but that's only going to be true as long as you don't publicly post it in the future.


My postal address is public. My mailbox is not, neither the home behind the address. If you don't know the difference it'll be hard to follow up this discussion. I may public my email address but that doesn't grant you permission to put something inside it. Got it? Because its content is private and the access to it is equally private. Ever heard of private clubs? You can find them in phone directoies and guides, but you won't enter unless allowed, because they're private.

You can write me, but only if I like you you can keep writing me. I can't see how this hurts "innocent people". All antispam solutions I've seen so far -including Spamassassin- force me to download spam messages from my server in order to check if the spam filtering has been done right. As long as this isn't changed in the anti-spam tools designers' minds, so called "anti spam solutions" are simplistic (albeit very complicated algorithm wise) patches to a serious problem: megabytes downloaded for nothing and time spent checking the work done by the anti spam filter.

I just want to verify who's trying to write me, and since most spam is machine originated and reply addresses used by them are fake I would never see the spam messages and not a single human being would be prevented from reaching my mailbox (as long as he/she can identify a few letters inside a blurry image...). Everybody is aware about the spam problem. Nobody will feel insulted if you explain clearly why authorization to write you is needed.


Your idea is riddled with problems. Please start a different thread so we don't sully this one.


I'm sorry but I believe this is exactly the place to discuss my ideas. If you disagree you can stop posting and start a new thread to praise Spamassassin. As simple as that.

Besides, if you believe YOUR mailbox is a public place, you can always keep using Spamassassin and other inelegant solutions like it... :-)


If I didn't want my email address to be public, it wouldn't appear in public. And you haven't successfully explained how Spamassassin is inelegant.


I'm sure you've had already understood the diference between a public address and the fact that the place pointed by that address might be private or public, regardless of the public nature of the address itself. Otherwise please let me know where you live (your address is public, after all), I'd like to plan a wild party for next weekend at your place... ;-)

Sorry for not explaining why Spamassassin in particular is inelegant, but I'm referring to all these well intentioned efforts made by many developers in order to get around spammers techniques. Very complicated solutions to tackle simple problems is what I call inelegant (a famous philosopher named Ockham thought pretty much the same...). But since you can't get through the easy point that my mailbox is not a public place, I'd find it terribly hard to explain why all these Bayesian filters and the like are so unlikely to succeed ever; when it's only a matter of closing the door to those who won't accept the simple idea that I don't want to be indiscriminatedly disturbed.

By the way, I find the example of Instant Messaging acurate, precise and absolutely appropriate. And the whitelist in some cases can be held at the server level, which makes it especially useful as an example for my proposal too. (BTW: AOL usually asks me if I want to grant permission for someone unknown to talk to me, which I usually deny.)

Maybe you should try to calm down in your evangelist fervor for what's just a piece of software. We're discussing about a new mail client (or "MUA" like geeks and acronym freaks like to name it), called Thunderbird, and both the poster and I are suggesting ways of getting rid of spam using the whitelists approach. I believe your point on Spamassassin is taken. Now can we move on?

Thanks


Regards,

C
petrarch
Posts: 56
Joined: January 16th, 2003, 10:07 am

Re: Spamassassin

Post by petrarch »

comomolo wrote:You probably need to defend your assertion too then. What is "thousands" of people? Were do you get your statistics from?


Thousands of people was a minimum estimate for order of magnitude, based upon posts to the spamassassin-talk mailing list. I've now got concrete proof that this can safely be considered millions

Well, I know my English is not very good, but I'm pretty sure your understanding is very slanted: everyone wants to reduce their spam. Only geeks can use Spamassassin because that software forgot about a simple issue: computer users are to computer geeks what car drivers are to mechanics... You guys (I assume you're a geek)


What the hell? Since when does understanding something about computers equate to being a geek?

think everyone should know certain things about computers, but frankly, nobody has entitled you to make such assumption.


I don't think you'll find this a very defendable statement. Your computer is plugged into the Internet, which means you're expected to maintain a modicum of understanding of the way things work.

Sorry if they have released a windows client side version of Spamassassin


It looks like SAproxy exists for windows. I sort of doubt it is easily plugged in to Thunderbird though. We're not really at that point.

I wasn't aware of that, but taken the "usability" of the software I tried, I can't think of any reason to try it again.
What software did you try and what's to say that its TB implementation wouldn't be different?

Antoher geek issue: be prepared to check for newer versions, download them, install them, configure them, every time you find your software is not working... or every other week.


How is this any different from downloading new nightlies, something you're doing by testing a pre 0.1 version of TB?

You don't really want me to check what version I was using six months ago when I tested the software, will you?


If you're going to complain that SA isn't doing its job, you need to be willing to speak to what version you were using. Otherwise it's the same as complaining that your pre 1.0 release of Mozilla is slow.

Again think of me as a user, not as a computer expert, no matter how hard it is for you to understand the difference, there is one.


It's not unreasonable to ask someone who is getting his website hosted to talk about the version of the software he's running. This information should have been provided to you by the hosting company!

Besides the fact that you have talked here about changing sensitiviy levels -i.e.: twekaing- anybody, you're right, CAN put addresses in a text file. But only those few Linux users and the like out there (please go to the statistics if you feel Linux users and the like are "a mass") actually WANT to do that. We like point and click, we lo ve GUIs, we are fond of being treated like people, we regular users...


I've never suggested a GUI is bad to do this stuff. A config box to change the threshold, and a GUI already exists for managing whitelists. No need to touch text files.

Oh, yes it is: he's talking about whitelists and I'm talking about whitelists, only with a different approach. He's talking about one solution (Qurb) which I don't completely like beacuse it's client based and I'm proposing a mixed client-server solution.


Right. If I understand the solution you're advocating for, it's wholly different from the client side approach, because it fundamentally alters the way email works. It deserves its own thread

If you know about such a solution (you seem to) and don't want to tell, I will suspect you have some sort of religious commitment to Spamassassin... Would you be so kind to point me to a solution like the one I propose. Believe me, I'd be rally happy if someone has already had the idea and implented it.


http://tmda.net/

You can also find complaints about this software on mailing lists.

I may public my email address but that doesn't grant you permission to put something inside it. Got it?


No, I don't think you get it. By posting your email address in a public place, you are making an implicit assertion that you desire correspondence. If you don't like the way the system works, you don't post it on the internet.

Because its content is private and the access to it is equally private.


And sending you email does nothing to violate your privacy. If you post your email address, you've already waived the privacy of the address.

Ever heard of private clubs? You can find them in phone directoies and guides, but you won't enter unless allowed, because they're private.


And the Internet is beautiful because it supports the free flow of information, not a private club. What you are suggesting would fundamentally alter the Internet, to the ire of many of its inhabitants.

I just want to verify who's trying to write me, and since most spam is machine originated and reply addresses used by them are fake I would never see the spam messages and not a single human being would be prevented from reaching my mailbox (as long as he/she can identify a few letters inside a blurry image...).


I'll address these points in the new thread you should create about this software.

I'm sorry but I believe this is exactly the place to discuss my ideas. If you disagree you can stop posting and start a new thread to praise Spamassassin. As simple as that.


I reference spamassassin as a solution that already solved the original poster's problem. What your advocating is fundamentally different, and I'm not going to further violate netiquette by filling this thread up with different software. It makes more sense to start a new thread, allowing for extensive discussion.

Sorry for not explaining why Spamassassin in particular is inelegant, but I'm referring to all these well intentioned efforts made by many developers in order to get around spammers techniques. Very complicated solutions to tackle simple problems is what I call inelegant (a famous philosopher named Ockham thought pretty much the same...).


If you've got The Solution, please share it with the rest of us, because for many people, spamassassin is among the best tools out there.

But since you can't get through the easy point that my mailbox is not a public place, I'd find it terribly hard to explain why all these Bayesian filters and the like are so unlikely to succeed ever; when it's only a matter of closing the door to those who won't accept the simple idea that I don't want to be indiscriminatedly disturbed.


This one is simple. If you don't want random people emailing you, don't post the address on the Internet, and don't pretend to participate on the Internet, thereby removing yourself from the unspoken rules that govern the place.

(IM stuff removed as I accidentally mixed quotes)

Maybe you should try to calm down in your evangelist fervor for what's just a piece of software. We're discussing about a new mail client (or "MUA" like geeks and acronym freaks like to name it), called Thunderbird, and both the poster and I are suggesting ways of getting rid of spam using the whitelists approach.


All I'm pointing out is that superior software that does this both exists, and would do well to be plugged in to TB.

Why do you insist upon belittling those who use the term MUA? Can you see how that can be construed as being offensive?

I believe your point on Spamassassin is taken. Now can we move on?


yes, to a new thread, if you care to further discuss TMDA like software.
Last edited by petrarch on May 8th, 2003, 8:54 am, edited 1 time in total.
comomolo
Posts: 80
Joined: May 1st, 2003, 7:35 pm

Post by comomolo »

I can only say that since you can't understand pretty simple things (like privacy issues) I won't take it any further. Besides, you've mixed quotes from me and from other people as if I had said them, which is enough to stop responding you.

If you need to start a new thread about anything it's none of my business, but please don't try to tell others what to do. If anyone has a constructive point of view about whitelisting software and how to integrate it in Thunderbird I'll be pleased to hear it right here. If you just want to "sell" your Spamassassin thing you're still welcome to try here, but I can only suggest you move on and open your own "Spamassasin is great" thread and don't be surprised if you find no echo to your propaganda. You won't find me there, be sure of that.

C

PS: Thanks for the link anyway, I'll check the TDMA site.
petrarch
Posts: 56
Joined: January 16th, 2003, 10:07 am

Post by petrarch »

comomolo wrote:I can only say that since you can't understand pretty simple things (like privacy issues) I won't take it any further. Besides, you've mixed quotes from me and from other people as if I had said them, which is enough to stop responding you.


An honest mistake. Thanks for pointing this out; I've corrected it.

Just because I don't agree with your point of view doesn't mean you're successfully argued your points about privacy.

If you need to start a new thread about anything it's none of my business, but please don't try to tell others what to do. If anyone has a constructive point of view about whitelisting software and how to integrate it in Thunderbird I'll be pleased to hear it right here.


Honestly, it should really be up to the original poster. Lacking his commentary, the best course of action seems to be starting a new thread.

I hope you're not implying my criticism hasn't been constructive, btw.


If you just want to "sell" your Spamassassin thing you're still welcome to try here


As I said before, I only brought up spamassassin as an example of software that already does what the original poster desired, plus much much more. I'm not here to sell spamassassin specifically, as there is plenty of other software out there. Don't establish this false dichotomy.

but I can only suggest you move on and open your own "Spamassasin is great" thread


I think you missed the point, which is that I used spamassassin to point out that the original poster's idea is inferior to solutions that exist today. That doesn't mean spamassassin is the One True Way.

and don't be surprised if you find no echo to your propaganda.


I'm arguing technical points about software. Propaganda distorts facts and appeals to the emotional state of people. I'm not attempting to do any of this. Please step back from the situation and examine who is becoming emotional.

PS: Thanks for the link anyway, I'll check the TDMA site.


You're welcome.
ataferner
Posts: 44
Joined: November 6th, 2002, 5:31 am

i agree

Post by ataferner »

comomolo:

I can only say that since you can't understand pretty simple things (like privacy issues) I won't take it any further. Besides, you've mixed quotes from me and from other people as if I had said them, which is enough to stop responding you.


Its unfortunate that it has come to this although I completely agree with you.

On another note I agree with you on the weakness with qurb that the whitelist is client based. I like your suggestion for the whitelist to be server based. I myself use thunderbird on multiple computers to connect to an IMAP account and can see how a client based whitelist wouldn't be very elegant.
tgnb
space m0nkey
Posts: 13
Joined: December 18th, 2002, 5:09 pm
Location: Scotland

Post by space m0nkey »

I might have missed something but why do you need something like qurb when a filter will do the job just fine:

Image
If you try to build something idiot proof, someone builds a better idiot.
comomolo
Posts: 80
Joined: May 1st, 2003, 7:35 pm

Post by comomolo »

Oufff...

OK, let's imagine for a second that a filter might do the job just fine. In that utopic world we wouldn't need anything else. But in an utopic world there would be no spam, so let's get back to Earth... I guess I can safely say that what you've missed is that filters aren't 100% accurate and so they don't eliminate the two consequences of spam: bandwidth being devoured by it and my time still wasted by checking and telling spam from ham (in the inbox for false negatives and in the spam box for false positives). If you're happy with these two "minor" annoyances, stop reading here and go for a filter (Bayesians recommended).

No one filter -NO ONE SINGLE FILTER OUT THERE- claims to be 100% effective at stopping spam. Nobody would believe such a statement anyway. So complaint number one (and the only one): if a filter can't guarantee total success so I can blindly trust it, it isn't even worth the pain using it. The only antispam solution worth of consideration would be the one I can't see working behind the scenes and can be trusted 100%. Not even those fanatics working at Spamassassin claim that (at least they're honest...).

To make things worse with filters, there's no way to guess beforehand what sort of mistakes will be made; they are random by the very nature of the approach. I've been able to test a number of filters and most of them claim and accomplish more or less the same figures: they'll catch around 90%-95% of the spam, make *ALMOST* no false positives and that's pretty it. Now if I have to check my spam folder for false positives and also manually delete that 10% of false negatives, the solution is useless: my bandwidth is still being stolen by spammers and my time too, the two main issues with spam.

Whitelists approach has an inherent 100% effectiveness in blocking spam, although no one can claim ever that it will be 100% good at false positives, since not all "ham" comes from humans. You will get absolutely no false positives coming from humans (if a human doesn't want to reply to you in order to identify himself or herself, you'll get much better off without his or her messages, so this is also a pretty nice "attitude" filter :-)). That means your business and personal emails won't be lost as long as you don't chat with robots or do business with machines, both presumably unable to reply to a confirmation message.

The tricky thing with whitelist software is how to deal with mailing lists and other automated messages that should get trough. That's why I'm proposing a mixed client and server solution. If your client is at the same time the control center for what happens at the server level, there's little or no chance that you will miss any single good message. Every time you subscribe to a list the client will notify your server that messages coming from the list's server should pass through. For lists you subscribe via web browser you might have to do it manually. If you're going to, say, buy at amazon.com or ebay.com, you just put that names beforehand in your whitelist (if they write to you without you having got in touch with them, i.e., without your permission, their mail SHOULD be blocked).

As you can see the problem with whitelists is very well isolated and identified. I know what kind of email I can lose using whitelists and confirmation messages, something that doesn't happen with filters, no matter how "clever" they are. I just have to be careful about putting my mailing lists or ecommerce domains into the whitelist and that's all. On the other hand, I just can use a temporary mail account for mailing lists and change it every now and then, something I can't do with a business or personal account.

But since the ONLY problem with whitelists are legitimate machine-originated mail, those clever folks out there making mailing lists, shopping carts and other automated mail generators, would help quite a bit by allowing their software to interact with whitelists properly (a simple web button saying: "Add to my whitelist" is pretty everything they'd need to have their email allowed).

Now I would be happy to listen to just ONE reasonable complaint about this.

Returning to the main point, ThunderBird might as well start a new generation of whitelist antispam solutions, based on what I've called a client & server (both working together) approach. That is my feature request and I believe it properly seconds the original poster's own request.

C

PS: I've been visiting the link posted by someone here. The TMDA software hasn't been designed with users in mind (so frequent with software offered "as is"...) I think everyone calling himself or herself a developer should be asked to pass some test on user interfaces before he or she writes a single line of code... Anyway, their effort is worth a try.

SORRY FOR THE LONG POST, I just tried to cover in advance minor questions that might arise.

--
Post Reply