Filter on html in body not working

[ForestCat]

Hi,
I'm at the point that if I get another "dating" spam from Russia, I'm going to steal a B2 bomber & nuke the entire country. Just barely kidding.

Why aren't Thunderbird's filters able to parse embedded urls like "http://www.datinglehup.ru/" such that a filter like:

Body contains "dating"
AND
Body contains ".ru"

will flag this crap?

I've found other people w/ the same problem, but no answers. Is there any hope?

[rais]

is your Junk Mail Controls enabled and working ?
is it filtering any junk mail at all?

[ForestCat]

I have a _lot_ of filters enabled, and they are ALL working, with the exception of any filters designed to trap embedded html urls as described above.

I realize now that this is a much bigger issue, since Thunderbird can't even search on this criteria. For example:

Search for "Body Contains datinglehup.ru"

yields no results, when I can plainly see the text does exist in the embedded url...

No one else seeing this?

[Lee_Dailey]

howdy ForestCat,

i just made a filter with this criteria & action ...
match all: [body] [contains] [theregister.co.uk]
action: [tag msg] [important]

when i ran it, all my msgs from TheReg were tagged with "important". this is a plain text msg.

i also ran the same filter with "informationweek.com" to test how links under text are selected by the filter in a full-bore HTML msg. it worked just as well as the one for TheReg did.

have you looked at the raw msg source? i get one newsletter that is entirely in base64, so searches on things that i can see in the reader window are NOT detectable via filters since they don't exist in the format i am seeing until the darned thing is displayed.

take care,
lee

[ForestCat]

Lee,

Thanks for your reply. Here's a div copied & pasted from the raw source:

*******************

<div class="content">
<p style="margin: 10px 0; padding: 10px 0px;">There are millions
of singles inside our site. Our goal is to help you meet
succesful singles, go out on great dates, and maybe even find
the love of your life.</p>
<p style="margin: 10px 0; padding: 10px 0px;">Today we have
picked <strong>8</strong> new matches for you.
</p>
<table class="button button_wrapper" align="center" style="text-align: center; padding-bottom: 15px; padding-top: 15px; border-collapse: separate; margin: 0 auto;"><tr><td style="height: 100%; color: #ffffff; border-radius: 10px; -webkit-box-shadow: inset 0 0 0px 1px #ffffff, 0 0 4px 2px #81a6cb; box-shadow: inset 0 0 0px 1px #ffffff, 0 0 4px 2px #81a6cb; background-color: #81a6cb; padding: 10px;" bgcolor="#81a6cb">
<a href="http://www.datinglehup.ru/?2AA5" style="color: white; text-decoration: none; display: block; width: 100%; border-radius: 10px; height: 100%; font-style: normal; font-variant: normal; font-weight: bold; font-size: medium; line-height: 16px; font-family: Verdana, Arial, Helvetica, sans-serif; background-color: #81a6cb; margin: auto; padding: 0;">
Click here to meet them now!</a></td></tr></table>
<p style="margin: 10px 0; padding: 10px 0px;">We recomend that you save this email for future reference.</p>
</div>

*******************************


As you can see, the the text "datinglehup" is there, but a thunderbird folder search for that text in the body does not find this message. I'm not sure why.

Any thoughts?

[Lee_Dailey]

howdy ForestCat,

[1] as rais pointed out, you are likely to have better results via the built in anti-spam system. [*grin*]

[2] you can make your code easier to read & add scroll bars by enclosing it in CODE tags in the advanced editor. you can get there via the preview button.

[3] please post the entire msg
when i run a filter against that snippet - pasted into a new blank HTML msg - the filter finds and tags it.
i'm afraid i'm not properly testing that sample since it is not valid html. you can open the email, save it as EML, open that, edit the source to replace any private info with something like "someone@example.com". the example.com addresses are reserved for ... examples. [*grin*]

then post it in a new msg with CODE tags around it.

[4] can you filter for anything else in that DIV?

*****
it's beginning to seem like you have found a bug in tbird. [*grin*]

take care,
lee

[ForestCat]

Lee,

[1] I'm aware, but prefer to use my own filters

[2] See [3]

[3]

Code: Select all

X-Account-Key: account24
X-UIDL: UID30722-1307022665
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
X-Mozilla-Keys:                                                                                 
Return-path: <BB0C6707@andertest.com>
Envelope-to: no@body.com
Delivery-date: Wed, 17 Apr 2013 11:01:33 -0400
Received: from 184-94-15-48.dedicated.allstream.net ([184.94.15.48])
   by host.foobar.com with esmtp (Exim 4.69)
   (envelope-from <BB0C6707@andertest.com>)
   id 1USTrZ-0001vW-6a
   for no@body.com; Wed, 17 Apr 2013 11:01:33 -0400
Message-ID: <20130417110135.A41E8AA17618564BB61A.5AA511@184-94-15-48.dedicated.allstream.net>
Date: Wed, 17 Apr 2013 11:01:35 -0400
From: "Notification" <BB0C6707@andertest.com>
To: <no@body.com>
Subject: Alert - New message received
List-Unsubscribe: <mailto:75CF68AE72682@amtaorg.com>
MIME-Version: 1.0
Content-Type: text/html; charset=iso-8859-1; format=flowed;
Content-Transfer-Encoding: 8bit

<!doctype html>
<html>
<head>
<title>Welcome weborders100 you are now a member</title>
<style type="text/css">
email{color:black;font:16px Arial, Helvetica, sans-serif;margin:0 auto;padding:0;min-width:290px;max-width:690px;}
email p{margin:10px 0;padding:0;}
email a{color:#1664D5;}
logo{padding-bottom:2px;}
logo img{border-style:none;height:46px;width:155px;}
header{margin-bottom:20px;}
header table{border-collapse:collapse;margin:0;padding:0;}
header td { padding: 1em 0;}
header .left{background-color:#000000;background-position:top;background-repeat:repeat-x;color:#fff;font-size:15px;font-weight:bold;padding:0 10px;}
header .right{background-color:#1664d5;background-position:top;background-repeat:repeat-x;color:#fff;font-size:20px;padding-left:17px;width:100%;}
header .right a{color:#ffffff;font-size:20px;font-weight:bold;}
header .highlight{color:#83DBFF;}

button{text-align:center;padding-bottom:15px;padding-top:15px;}
table.button { border-collapse: separate; margin: 0 auto;}
button td { height: 100%; padding: 10px; background-color:#990000; color:#ffffff; border-radius:10px;
-webkit-box-shadow: inset 0 0 0px 1px #FFFFFF, 0 0 4px 2px #990000;  box-shadow: inset 0 0 0px 1px #FFFFFF, 0 0 4px 2px #990000;
}
button a{background-color:#990000; color:#ffffff;font:bold 9px/16px Verdana, Arial, Helvetica, sans-serif;margin:auto; padding:3px; text-decoration:none;display:inline-block;width:100%;border-radius:10px;}
button td>a { display: block;  height: 100%; padding: 0;}
td table.button { min-width: 250px; margin: 0 5px; }
two_col { margin: 0 auto; }

keyline{background-color:#E5E5E5;height:1px;line-height:1px;margin:auto;width:100%;}
content p{padding:10px 0px;}
unsubscribe{padding:10px 0px;font-size:12px;}
clear {clear:both;}
photo {width:90px;height:100px;}
gift {width:80px;height:80px;}
medium_photo{width:126px;height:140px;}
cropped_photo {width:100%;text-align:center;}
cropped_photo img{width:180px;height:200px;}
cropped_photo a img{border:2px solid #1664cd !important;}
cropped_photo h2 {padding:0px;margin-top:0px;}
multiple {width:310px;float:left;}
br {line-height:20px;}
photo a img{border:1px solid #1664cd !important;}
label {font-weight:bold;}
h2 {font-size:22px;color:#990000;margin:10px 0px;}
h4 {font-size:18px;color:#990000;margin:10px 0px 0px 0px;}
h5 {font-size:16px;color:#333333;margin:0px;}
note {font-size:10px;}

innerbox{border:1px solid #E5E7E8;padding:15px 20px;}

toolbar{padding-top:10px;padding-bottom:10px;}
toolbar .p1{color:#990000;font-size:25px;font-weight:normal;}
toolbar .p2{color:#050505;font-size:16px;font-weight:bold;}
toolbar .image_wrapper{padding-bottom:24px;padding-top:24px;}
toolbar img{display:block;height:20px;width:633px;}
toolbar a{display:block;font-size:14px;font-weight:bold;}

gift_ad{padding-top:10px;padding-bottom:10px;}
gift_ad .p1{color:#990000;font-size:20px;font-weight:normal;padding-bottom:8px;}
gift_ad a{display:block;font-size:14px;font-weight:bold;}

profile_mini{margin-top:14px;padding-left:2px;padding-right:2px;float:left; width: 98%; max-width:600px}
profile_mini a{font-family:Verdana, Arial, Helvetica, sans-serif;font-weight:bold;}
profile_mini .name{border:1px solid #e5e7e8;padding-left:4px;height:23px;}
profile_mini .name a{font-size:14px;}
profile_mini .photo{border-left:1px solid #e5e7e8;}
profile_mini .profile_info{line-height:20px;vertical-align:top;font-size:12px;border-right-width:1px;border-right-style:solid;border-right-color:#e5e7e8;}
profile_mini .action{text-align:center;height:26px;border: 1px #e5e7e8 solid;}
profile_mini .action a{font-size:9px;}

profile_big {border:1px solid #E5E7E8;margin-top:16px;padding:15px; width: 98%; max-width:600px}
profile_big img{margin-right:14px;}
profile_big table{font-size:13px;line-height:26px;}
profile_big a{font-weight:bold;}
profile_big .photo{padding:1px;}

percentage-percentage{color:#990000;}
percentage-body{border:1px solid #E5E7E8;padding:5px;background-color:#e6f2ff;}
percentage-body td{line-height:24px;vertical-align:top;font-size:17px;font-weight:bold;}
percentage-body a{font-size:15px;}

progress_bar_container{background-color:#FFFFFF;border:solid 1px #7F7F7F;padding:1px;width:130px;}
progress_bar{background-color:#990000;height:5px;}

cupid_logo {font-size:12px;}
cupid_logo img{height:25px;width:107px;margin-top:20px;}

ul.profile_rejection_reasons_list{padding-top:0px;margin-top:0px;}

ul {margin-top:2px;}

order {padding-top:20px;margin:20px, auto;}
order td {padding: 0px 10px;}
order .headerRow {font-weight:bold;}
order .summaryRow {border-top: 1px solid black;}

receipt td.subEntry {padding-left:25px;}
receipt td.summaryRow {border-top:1px solid black;}

header .right{background-color:#19426d;}
email a{color:#9b0da7;}
button a{background-color:#81a6cb;color:white;}
button td {background-color:#81a6cb; color:#ffffff;
-webkit-box-shadow: inset 0 0 0px 1px #ffffff, 0 0 4px 2px #81a6cb;
        box-shadow: inset 0 0 0px 1px #ffffff, 0 0 4px 2px #81a6cb;
}
photo a img{border:1px solid #9b0da7;}
percentage-percentage{color:#1664D5;}
progress_bar{background-color:#1664D5;}
toolbar .p1{color:#9b0da7;}
h2 {color:#81a6cb;}
h4 {color:#81a6cb;}
</style>
</head>
<body>

   <div class="email" style="color: black; font-style: normal; font-variant: normal; font-weight: normal; font-size: 16px; line-height: normal; font-family: Arial, Helvetica, sans-serif; min-width: 290px; max-width: 690px; margin: 0 auto; padding: 0;">


      <div class="email_body">
         <div class="header" style="margin-bottom: 20px;">
            <table style="border-collapse: collapse; margin: 0; padding: 0;">
               <tr>
                  <td class="left" style="color: #fff; font-size: 15px; font-weight: bold; background-repeat: repeat-x; background-color: #000000; background-position: top; padding: 0 10px;" bgcolor="#000000">
                  weborders100</td>
                  <td class="right" style="color: #fff; font-size: 20px; width: 100%; background-repeat: repeat-x; background-color: #19426d; background-position: top; padding: 1em 0 1em 17px;" bgcolor="#19426d">Congratulations! You've just taken the first step in your new dating life.</td>
               </tr>
            </table>
         </div>
         <div class="content">
            <p style="margin: 10px 0; padding: 10px 0px;">There are millions
            of singles inside our site. Our goal is to help you meet
            succesful singles, go out on great dates, and maybe even find
            the love of your life.</p>
            <p style="margin: 10px 0; padding: 10px 0px;">Today we have
            picked <strong>8</strong> new matches for you.
            </p>
            <table class="button button_wrapper" align="center" style="text-align: center; padding-bottom: 15px; padding-top: 15px; border-collapse: separate; margin: 0 auto;"><tr><td style="height: 100%; color: #ffffff; border-radius: 10px; -webkit-box-shadow: inset 0 0 0px 1px #ffffff, 0 0 4px 2px #81a6cb; box-shadow: inset 0 0 0px 1px #ffffff, 0 0 4px 2px #81a6cb; background-color: #81a6cb; padding: 10px;" bgcolor="#81a6cb">
               <a href="http://www.datinglehup.ru/?222222222222222222222222" style="color: white; text-decoration: none; display: block; width: 100%; border-radius: 10px; height: 100%; font-style: normal; font-variant: normal; font-weight: bold; font-size: medium; line-height: 16px; font-family: Verdana, Arial, Helvetica, sans-serif; background-color: #81a6cb; margin: auto; padding: 0;">
               Click here to meet them now!</a></td></tr></table>
            <p style="margin: 10px 0; padding: 10px 0px;">We recomend that you save this email for future reference.</p>
         </div>
         <div class="ad">



         </div>
      </div>
      <div class="unsubscribe" style="font-size: 12px; padding: 10px 0px;">If you no longer wish to receive this or other messages please <a href="http://www.datinglehup.ru/?3A82D6E=550301C59C8DF35211">adjust your mail settings</a></div>

   </div>
</body>
</html>

[4] Don't need to filter. Just put the message in a folder by itself and search the folder for:

Body contains datinglehup

My t-bird misses it completely.

Thanks again for the help/testing.

[Lee_Dailey]

howdy ForestCat,

well, i am obviously doing something wrong. [*sigh ...*]

if i save that text as an EML and then open it in tbird, it has all kinds of problems. the subject doesn't display and the body shows only the code for the HTML - not the rendered web display.

can you post a copy of a working EML or mailbox file to someplace that we can access? perhaps a dropbox link or something similar.

take care,
lee

[ForestCat]

Lee,

Try this:
http://www.filedropper.com/spam

Grey button halfway down the page, "Download This File"

The link should be good for a week.

[Lee_Dailey]

howdy ForestCat,

got it. thanks! [*grin*]

well, i tried the filter and - as with you - it fails. i tried stripping out all of the STYLE stuff in the HEAD section and i still can't find the URL with either a filter or tbirds quick filter. interestingly, testing for "We recomend that you save this email for future reference" works.

how very, very odd!

i suggest you create a new bug in bugzilla. the "file a bug" button walks you thru a fairly simple process. you can ignore most of the settings and simply include ...
- your O/S version
- your tbird version
- a clear STR [Steps To Reproduce]
- a copy of the msg [they support uploading attachments]

if any of the addresses or other info in the msg is private, you can edit the saved EML file or mailbox file to remove that info.

please, if you do make a bug entry, post a link to it back here. i'm _very_ curious about what is going on ...

take care,
lee

[LP_TB]

Any news on this bug? Filtering HTML content still not working properly in 52.3.0.
I'd like to filter every e-mail that contains a link with

Code: Select all

 .php? 

It seems everything inside

Code: Select all

<a href=

tags isn't read by the filter.
Almost 100% of phishing e-mails have this.