urlclassifier3.sqlite and excessive I/O and CPU
- Aus
- Posts: 183
- Joined: September 6th, 2004, 1:59 pm
urlclassifier3.sqlite and excessive I/O and CPU
I've had some problems recently with 3.0pre nightly builds on Windows XP. Shortly after opening fx, I get excessive disk I/O and high CPU usage. I finally tracked the problem down to the urlclassifier3.sqlite database, which on my system is almost 32Mb. If I delete the file, a new file is created 9kb in size and the I/O and CPU problems go away.
I noticed there have been reports of problems in this area from people using linux builds, but I couldn't find anything specific about Windows. Is this a known problem?
Aus
I noticed there have been reports of problems in this area from people using linux builds, but I couldn't find anything specific about Windows. Is this a known problem?
Aus
- a;skdjfajf;ak
- Posts: 17002
- Joined: July 10th, 2004, 8:44 am
-
- Posts: 182
- Joined: January 22nd, 2003, 10:45 pm
There's been multiple problems/fixes in this area-- the big for Linux people was bug 430530. pytechd was right to ask exactly which Build, because the fix for 430530 only landed last Sunday AM.
402469 and it's follow-up "relatives" (bug 432490 bug 431563) are all still open, and they're all blockers. These effect all safebrowsing users-- but they're issues with the Google-based Servers, we won't know what Google has done until they post, and the timing of the fix(es) won't be an exact match one of our FF BuildIDs.
Because we (now I'm talking you and I, not mozilla.org) are using these builds as 'volunteer testers', it wouldn't be nice to turn off phishing protection-- Mozilla needs our bug reports. So please leave it active. I think that bug 402469 is the most general of the "urlclassifier3.sqlite" bugs which is still open, so post any misbehavior in that one-- and let Dave Camp split it off if it turns out to be a specific issue still present in our code on Windows.
402469 and it's follow-up "relatives" (bug 432490 bug 431563) are all still open, and they're all blockers. These effect all safebrowsing users-- but they're issues with the Google-based Servers, we won't know what Google has done until they post, and the timing of the fix(es) won't be an exact match one of our FF BuildIDs.
Because we (now I'm talking you and I, not mozilla.org) are using these builds as 'volunteer testers', it wouldn't be nice to turn off phishing protection-- Mozilla needs our bug reports. So please leave it active. I think that bug 402469 is the most general of the "urlclassifier3.sqlite" bugs which is still open, so post any misbehavior in that one-- and let Dave Camp split it off if it turns out to be a specific issue still present in our code on Windows.
Make a fire FOR a man, he'll be warm for the rest of the night. Set a man ON fire, he'll be warm for the rest of his life!
-
- Posts: 8
- Joined: May 10th, 2008, 9:22 pm
Firefox 3 RC1 still has this issue.
OS: Windows XP Pro SP3
For someone, who uses notebooks on batteries, there is no other solution but to turn off phishing protection, because this excessive CPU/IO activity is draining the battery. Also urlclassifier3.sqlite file is growing bigger, last time I saw it, it was about 53MB. How large it can grow(?) I'm beginning to think that Firefox 3 is not intended for mobile users.
OS: Windows XP Pro SP3
For someone, who uses notebooks on batteries, there is no other solution but to turn off phishing protection, because this excessive CPU/IO activity is draining the battery. Also urlclassifier3.sqlite file is growing bigger, last time I saw it, it was about 53MB. How large it can grow(?) I'm beginning to think that Firefox 3 is not intended for mobile users.
-
- Posts: 182
- Joined: January 22nd, 2003, 10:45 pm
I agree about the file size-- why should FF3 need more disk space than an ENTIRE installation of Norton Internet Security? (That's *all* of NIS-- the programs, the data, the updates, everything.) It sounds like you've got the entire file, though-- mine is also 53.2 MB, and I've been running FF3 for many months.
Do you see lots of I/O even when you have phishing protection turned off? I've got a bad feeling that maybe, you're not killing the battery just via "urlclassifier3.sqlite". Could also be bug 421482, a places/history database fiasco. I just scribbled a note into that bug (on your behalf) that battery life is a BIG issue. They've so far been focusing primarily on GUI "lock-up".
Do you see lots of I/O even when you have phishing protection turned off? I've got a bad feeling that maybe, you're not killing the battery just via "urlclassifier3.sqlite". Could also be bug 421482, a places/history database fiasco. I just scribbled a note into that bug (on your behalf) that battery life is a BIG issue. They've so far been focusing primarily on GUI "lock-up".
Make a fire FOR a man, he'll be warm for the rest of the night. Set a man ON fire, he'll be warm for the rest of his life!
-
- Posts: 8
- Joined: May 10th, 2008, 9:22 pm
rickst29 wrote:Do you see lots of I/O even when you have phishing protection turned off? I've got a bad feeling that maybe, you're not killing the battery just via "urlclassifier3.sqlite". Could also be bug 421482, a places/history database fiasco. I just scribbled a note into that bug (on your behalf) that battery life is a BIG issue. They've so far been focusing primarily on GUI "lock-up".
With phishing protection turned off I/O activity is low, almost none. After that, some small I/O activity appears only while browsing, editing/removing/adding bookmarks, clearing history. But CPU load is also noticeable at these moments. I guess excessive I/O and also CPU load (as stated in the post title) both at the same time appear only while updating/working with urlclassifier3.sqlite, it was periodically updated (while browsing or while browser was idle) with large amounts of data: after update file size increased by 2-3MB or more. I guess SQLite engine is not suitable for such kind of transactions, or it is not tuned for best performance at the moment, or update operations are made not in an optimal way.
- SickSadWorld
- Posts: 116
- Joined: February 23rd, 2006, 8:15 pm
- Bluefang
- Posts: 7857
- Joined: August 10th, 2005, 2:55 pm
- Location: Vermont
- Contact:
You have to disable both the phishing filter and the malicious site filter.
There have always been ghosts in the machine... random segments of code that have grouped together to form unexpected protocols. Unanticipated, these free radicals engender questions of free will, creativity, and even the nature of what we might call the soul...
- Morac
- Posts: 2519
- Joined: February 9th, 2004, 8:20 pm
- Contact:
My urlclassifier3.sqlite in Firefox 3.0 RC1 on Windows was at 53,440,512 and it was being rebuilt in the temp directory I shutdown Firefox when it reached around 90 MB. This seems a lot larger than it should have been.
I did try to shut down Firefox when the urlclassifier3.sqlite was first being build (Firefox didn't shut down for a few minutes and was using 99% of the CPU during that time).
Is it possible the urlclassifier3.sqlite file could have become corrupted?
I did try to shut down Firefox when the urlclassifier3.sqlite was first being build (Firefox didn't shut down for a few minutes and was using 99% of the CPU during that time).
Is it possible the urlclassifier3.sqlite file could have become corrupted?
- MechR
- Posts: 1286
- Joined: July 30th, 2003, 4:13 pm
- Location: Earth
- Morac
- Posts: 2519
- Joined: February 9th, 2004, 8:20 pm
- Contact:
Well I wiped out the urlclassifier3.sqlite file and it recreated itself, jumped to about 6 MB and is now at about 12 MB and appears to be holding steady. I'm not sure what triggers it to grow or how it decides when to do so.
Considering it grew to > 54 MB in less than an hour I think something went wrong the first time.
Considering it grew to > 54 MB in less than an hour I think something went wrong the first time.