urlclassifier3.sqlite and excessive I/O and CPU
-
- Posts: 235
- Joined: May 26th, 2003, 2:50 am
Don't know if it's worth mentioning that Bug 402469 is one of only three bugs still marked as 'blocking' which means that the Firefox team intend to hold off the release of Firefox 3 until it's fixed:
https://bugzilla.mozilla.org/show_bug.cgi?id=402469
This bug concerns itself with the slow rate at which the urlclassifier3.sqlite database is built. Its natural size is over 30 megabytes, but this is downloaded gradually from Google, meaning that full protection from attack websites will not be achieved until it has fully downloaded.
If this bug is fixed, the urlclassifier3.sqlite database will grow much faster, which will offer better protection from attack websites.
If you don't want this file to grow at all you will need to disable the "suspected attack site" warning in your Firefox preferences.
https://bugzilla.mozilla.org/show_bug.cgi?id=402469
This bug concerns itself with the slow rate at which the urlclassifier3.sqlite database is built. Its natural size is over 30 megabytes, but this is downloaded gradually from Google, meaning that full protection from attack websites will not be achieved until it has fully downloaded.
If this bug is fixed, the urlclassifier3.sqlite database will grow much faster, which will offer better protection from attack websites.
If you don't want this file to grow at all you will need to disable the "suspected attack site" warning in your Firefox preferences.
- Bluefang
- Posts: 7857
- Joined: August 10th, 2005, 2:55 pm
- Location: Vermont
- Contact:
Another bug that I just recently saw is this:
Firefox 3 uses fsync excessively
Which has the potential to bring a Linux system to its knees with very large databases. I think this could potentially affect both the urlclassifier and bookmarks/history.
Firefox 3 uses fsync excessively
Which has the potential to bring a Linux system to its knees with very large databases. I think this could potentially affect both the urlclassifier and bookmarks/history.
There have always been ghosts in the machine... random segments of code that have grouped together to form unexpected protocols. Unanticipated, these free radicals engender questions of free will, creativity, and even the nature of what we might call the soul...
- a;skdjfajf;ak
- Posts: 17002
- Joined: July 10th, 2004, 8:44 am
-
- Posts: 34
- Joined: July 18th, 2007, 9:06 pm
Morac wrote:Well I wiped out the urlclassifier3.sqlite file and it recreated itself, jumped to about 6 MB and is now at about 12 MB and appears to be holding steady. I'm not sure what triggers it to grow or how it decides when to do so.
Considering it grew to > 54 MB in less than an hour I think something went wrong the first time.
That's the correct size.
55894016 2008-05-23 12:33 urlclassifier3.sqlite
There are a <I>lot</I> of malicious sites on the Internet.
-
- Posts: 182
- Joined: January 22nd, 2003, 10:45 pm
various replies...
Bluefang wrote:Another bug that I just recently saw is this:
Firefox 3 uses fsync excessively
Which has the potential to bring a Linux system to its knees with very large databases. I think this could potentially affect both the urlclassifier and bookmarks/history.
You're just a bit confused about 421482-- the SIZE of the database doesn't matter, merely the fact that writes are being "confirmed" by flushing to disk frequently is the cause of the problem. We're doing that <b>way, WAY more often</b> on "places.sqlite" than we do it on "urlclassifier3.sqlite". (Basically, every time you touch a new page or even reload a current one, we update your "history".) Also the "fix" which LittleMutt refered to isn't really a "fix", it just creates a preference so that us testers can flip the flushing frequency between the current default "FULL", the new default "NORMAL", and the ultra-efficient but probably dangerous value of "OFF". With a pref, we can do this on-the-fly, instead of loading up a whole new Frefox build for each test. (And of course, having the pref allows the normal Tinderboxen and Nightlies to include the capability, instead of us building custom jobbies over and over and over.)
But testing so far has seen very little difference between "FULL" and "NORMAL", that's not really a solution IMO.
402469 is actually "done", according to the text in the bug. (As you may have noticed, I was the first end-user to pounce on a Tinderbox build which had both 402469 and 430530.) Dave Camp has opened other bugs for more recent issues with the Google Servers (e.g., all the 400s we were getting a while back), but there's nothing happening on this one, I'm not sure why it's still open.
No one ever answered my question about "why is it 50+MB, when Norton Internet Security can fit all their programs AND data in less? They're not merely doing phishing and malware sites in your browsers-- they're also scanning your emails as they arrive, scanning your entire filesystems, running a pretty comprehensive firewall....)
Last edited by rickst29 on May 24th, 2008, 1:34 pm, edited 1 time in total.
Make a fire FOR a man, he'll be warm for the rest of the night. Set a man ON fire, he'll be warm for the rest of his life!
- Bluefang
- Posts: 7857
- Joined: August 10th, 2005, 2:55 pm
- Location: Vermont
- Contact:
My bad. I didn't read the entire bug, and it started with discussion of the problem becoming noticeable after the history got significantly large.
There have always been ghosts in the machine... random segments of code that have grouped together to form unexpected protocols. Unanticipated, these free radicals engender questions of free will, creativity, and even the nature of what we might call the soul...
-
- Posts: 51
- Joined: November 5th, 2002, 12:48 am
I thought I enabled safe browsing in security options by setting both:
- Tell me if the site I'm visiting is a suspected attack site
- Tell me if the site I'm visiting is a suspected forgery
However, on my system I cannot find the file urlclassifier3.sqlite, only a urlclassifier2.sqlite file of 0KB which was last updated on 17/08/07 11:45. Any ideas?
Seems I'm definitely suffered from the places.sqlite (24421KB) problem.
- Tell me if the site I'm visiting is a suspected attack site
- Tell me if the site I'm visiting is a suspected forgery
However, on my system I cannot find the file urlclassifier3.sqlite, only a urlclassifier2.sqlite file of 0KB which was last updated on 17/08/07 11:45. Any ideas?
Seems I'm definitely suffered from the places.sqlite (24421KB) problem.
-
- Posts: 512
- Joined: March 28th, 2007, 3:54 pm
- ehume
- Posts: 6743
- Joined: November 17th, 2002, 12:33 pm
- Location: Princeton, NJ, USA
Very interesting blog indeed, coming as it does from Shaver.
I've already made my comments on incorporating sqlite into Firefox's bookmarks and history. This is just more of the same.
I'm on WinXP with a 53.4MB url classifiier file. The Firefox installation directory for 3.0RC1 is only 23.3 MB.
I've already made my comments on incorporating sqlite into Firefox's bookmarks and history. This is just more of the same.
I'm on WinXP with a 53.4MB url classifiier file. The Firefox installation directory for 3.0RC1 is only 23.3 MB.
Firefox: Sic transit gloria mundi.
-
- Posts: 1
- Joined: May 10th, 2009, 2:57 am
Re: urlclassifier3.sqlite and excessive I/O and CPU
did any of you think of this before:
1. Delete urlclassifier3.sqlite
2. Recreate it as a 0 byte txt file with the same name
3. Deny Write/Delete Permissions to everyone
Works for me
1. Delete urlclassifier3.sqlite
2. Recreate it as a 0 byte txt file with the same name
3. Deny Write/Delete Permissions to everyone
Works for me
- malliz
- Folder@Home
- Posts: 43796
- Joined: December 7th, 2002, 4:34 am
- Location: Australia
Re: urlclassifier3.sqlite and excessive I/O and CPU
The world has been waiting around for a year for that
What sort of man would put a known criminal in charge of a major branch of government? Apart from, say, the average voter.
"Terry Pratchett"
"Terry Pratchett"
- toods
- Posts: 255
- Joined: December 22nd, 2004, 9:33 am
- Location: Cornwall, UK
Re: urlclassifier3.sqlite and excessive I/O and CPU
Irongrip wrote:did any of you think of this before:
1. Delete urlclassifier3.sqlite
2. Recreate it as a 0 byte txt file with the same name
3. Deny Write/Delete Permissions to everyone
Works for me
If you had read some of the Forum Posts over the last 12 months since this thread was a active you would have realised that there are ways of avoiding 'urlclassifier3.sqlite' usage by making 'conventional' preference setting changes.
Bill.
Slackware, 12.1
- ehume
- Posts: 6743
- Joined: November 17th, 2002, 12:33 pm
- Location: Princeton, NJ, USA
Re: urlclassifier3.sqlite and excessive I/O and CPU
Please. The guy comes up with a workable solution and you dump on him. He's not claiming he had the only solution, or the best. In fact, framing his question the way he did, he's not even saying he's the first to think of it.
Notice that this is his very first post. With responses like yours, it might be his last. Lighten up, please.
Notice that this is his very first post. With responses like yours, it might be his last. Lighten up, please.
Firefox: Sic transit gloria mundi.
-
- Posts: 960
- Joined: February 25th, 2003, 10:11 pm
- Location: USA
Re: urlclassifier3.sqlite and excessive I/O and CPU
ehume wrote:Please. The guy comes up with a workable solution and you dump on him. He's not claiming he had the only solution, or the best. In fact, framing his question the way he did, he's not even saying he's the first to think of it.
Notice that this is his very first post. With responses like yours, it might be his last. Lighten up, please.
No kidding. I'm trying to get MORE people posting on MozillaZine instead of SUMO and this is counter-productive to my efforts.