Why safebrowsing folder contents can cause a big slow down

User Help for Mozilla Firefox
Post Reply
User avatar
c627627
Posts: 641
Joined: April 3rd, 2005, 12:58 pm
Location: Kansas City, Missouri
Contact:

Why safebrowsing folder contents can cause a big slow down

Post by c627627 »

On my home computer I have multiple Firefox profiles, some have their home page set to a Yahoo! email account so I can just check email by starting that Firefox profile.
Since it took seven to eight seconds to show the email home page, I created a new profile and immediately noticed that it took less than a couple of seconds on it.

So I meticulously went through all extensions, through all about:config settings to look for the culprit and still did not find it, so I resorted to file comparing a new profile with existing profile.
Found out the hard way that only after deleting all files inside profile's safebrowsing folder, only then would the startup speed increase five fold or more (!)

So...would appreciate your learned opinion on cause/effect and why the safebrowsing folder's contents might have caused this problem, especially since my attempt to originally fix this included recreation of all Firefox profiles a month ago, so this was not caused by age old profile contents.
Thanks for posting.
Open the pod bay doors, Cortana.
User avatar
Frank Lion
Posts: 21173
Joined: April 23rd, 2004, 6:59 pm
Location: ... The Exorcist....United Kingdom
Contact:

Re: Why safebrowsing folder contents can cause a big slow do

Post by Frank Lion »

"The only thing necessary for the triumph of evil, is for good men to do nothing." - Edmund Burke (attrib.)
.
User avatar
c627627
Posts: 641
Joined: April 3rd, 2005, 12:58 pm
Location: Kansas City, Missouri
Contact:

Re: Why safebrowsing folder contents can cause a big slow do

Post by c627627 »

Thank you.
Looking at your post in the other thread it looks like I may be facing the same problem after a while.

In your post you suggest
A. ...manual comparison of the settings in Options/Preferences and then those in Help> Trouble Shoot.
and
B. ...I would go straight for the Safe Browsing options and turn them off, as I reckon that is what is causing this.

First of all, how is B. accomplished. Is it through Options > Security > then UNCHECK: Warn me when sites try to install add-ons and UNCHECK: Block dangerous and deceptive content
Those two and that's it?

And as for A.: I uninstalled all extension, no change. Restored all about:config options, no change, deleted the entire chrome folder and no change.
Only after deleting the actual contents of the safebrowsing folder did things immediately speed up fivefold.


But the most important question is if UNCHECKING only those two options would effectively switch off Safe Browsing?
Open the pod bay doors, Cortana.
User avatar
Frank Lion
Posts: 21173
Joined: April 23rd, 2004, 6:59 pm
Location: ... The Exorcist....United Kingdom
Contact:

Re: Why safebrowsing folder contents can cause a big slow do

Post by Frank Lion »

c627627 wrote:Thank you.
Looking at your post in the other thread it looks like I may be facing the same problem after a while.
Yep, the problem I have on that thread is people saying they have this problem and then vanishing. Plus, the one guy who remains (OP?) isn't trying what I suggested and the thread is going nowhere.

This is a problem that not everyone has (I don't) but for those that do it's a very repeatable cycle of - new profile>OK>24hrs>not OK>rinse>repeat. I'm not saying it is safebrowsing causing the problem, that's just my guess as you're looking for something that would change/increase immediately after a new profile is used. However, your OP does suggest this is pretty likely the reason.

I'm reckoning it's similar to the problem adblockers used to have - huge lists loaded up at startup and huge lists checked for every single site you open - slows things down. My uBlock Origin doesn't do this and maybe all present day adblockers don't, so it can be fixed.

B. ...I would go straight for the Safe Browsing options and turn them off, as I reckon that is what is causing this.

First of all, how is B. accomplished. Is it through Options > Security > then UNCHECK: Warn me when sites try to install add-ons and UNCHECK: Block dangerous and deceptive content
Those two and that's it?
Not the first one and certainly the second. In SeaMonkey I have 2 entries under safebrowing, one for attack sites, one for phishing. Firefox might have only 1 entry, I can't remember, I haven't got Firefox one the OS I'm using atm.
But the most important question is if UNCHECKING only those two options would effectively switch off Safe Browsing?
No, it would appear.....which is the odd thing.

I can't help you much on this, simply because I don't have this problem. So, what I'm going to do is to give you the tools to start to figure it out yourself. However, I have some things to check out on all this and afterwards I'd like you to do a series of simple steps, after which I'm confident will show exactly what is causing this and exactly what can be done about it.

What hits Firefox today, is surely going to hit SeaMonkey tomorrow, as we've already seen.

Meantime, you'll need these test site links (from Mozilla) -

https://itisatrap.org/firefox/its-an-attack.html
https://itisatrap.org/firefox/its-a-trap.html

Here's the odd thing, disabling SafeBrowsing in Options/Preferences only has the effect of not stopping access to those test sites. Meantime, the safebrowing side carries on as before, it seems - it populates and updates every 30 minutes!

For those who are not aware, the safebrowing folder is in the same folder as your cache. -

Local store is in:

~/.cache/mozilla/firefox/XXXX/safebrowsing/ on Linux
~/Library/Caches/Firefox/Profiles/XXXX/safebrowsing/ on Mac
C:\Users\XXXX\AppData\Local\mozilla\firefox\profiles\XXXX\safebrowsing\ on Windows
"The only thing necessary for the triumph of evil, is for good men to do nothing." - Edmund Burke (attrib.)
.
User avatar
c627627
Posts: 641
Joined: April 3rd, 2005, 12:58 pm
Location: Kansas City, Missouri
Contact:

Re: Why safebrowsing folder contents can cause a big slow do

Post by c627627 »

Just a side question Frank, why would you switch to Sea Monkey so soon when there is half a year+ left on Fx ESR, is it because of a features it has or you just want to do it early?

I do have the problem profile backed up. It takes seven seconds to start and auto login display Yahoo email Inbox.
The moment safebrowsing folder contents are deleted (and nothing else is done) that drops to well under 2 seconds. So it's a perfect test setup to experiment with this problem.

Doing this controls access to your test links (thanks for posting those):
Options > Security >
UNCHECK: Warn me when sites try to install add-ons
UNCHECK: Block dangerous and deceptive content


However, you are saying that safebrowsing contents, the culprit of this problem, that's not affected?
I will UNCHECK those options then wait a week to retest. If the problem returns, is some sort of auto deleting script the only cure, something that would periodically delete contents of safebrowsing folder every few days?
Open the pod bay doors, Cortana.
User avatar
Frank Lion
Posts: 21173
Joined: April 23rd, 2004, 6:59 pm
Location: ... The Exorcist....United Kingdom
Contact:

Re: Why safebrowsing folder contents can cause a big slow do

Post by Frank Lion »

c627627 wrote:Just a side question Frank, why would you switch to Sea Monkey so soon when there is half a year+ left on Fx ESR, is it because of a features it has or you just want to do it early?
I rarely mention it when I'm posting in Firefox related forums, but I've been using SeaMonkey since spring 2014.
c627627 wrote: I do have the problem profile backed up. It takes seven seconds to start and auto login display Yahoo email Inbox.
The moment safebrowsing folder contents are deleted (and nothing else is done) that drops to well under 2 seconds. So it's a perfect test setup to experiment with this problem.
Good. First, light general reading - https://wiki.mozilla.org/Security/Safe_Browsing

Next, here's how it seems to work - with a new profile the safebrowsing folder is quickly filled with files, all prefixed by 'test'. They are all around 1kb in size and I reckon these come with the program, i.e. locally stored. Without safebrowsing enabled, they do 'update' every 30 mintes but only with copies of themselves. Small as they are, they are good enough to block the test attack and trap sites, once safebrowsing options are checked in Options/Preferences.

Once safebrowsing is enabled, the folder is quickly filled with non-local files, prefixed with 'goog' or 'moz' and range in size from a few kb up to 3mb.

The following tests have, obviously, been tested by me first and are safe.

#Test 1 -

#1. Uncheck safebrowsing options.
#2. New profile > about:config > search for safe > clear all http/s urls that show as safebrowsing entries, by right clicking each, select Modify and deleting the existing entry. There are around 20 of these.
#3. Close Firefox and delete existing safebrowsing folder, then restart.
#4. Enable the safebrowsing options in Options/Preferences, wait 5 minutes and then check test sites.
#5. Check startup page loading times at regular intervals over a period of, say, 6 hours.

#Test 2. -

#1. Close Firefox and delete existing safebrowsing folder, then restart.
#2. Enable the safebrowsing options in Options/Preferences and then leave Firefox running for a few hours (to fill the safebrowsing folder with 'goog' prefixed, etc files.)
#3. Do Step #2. from #Test 1. above.
#4. Do Step #5.. from #Test 1. above.

The test order can be reversed, if preferred.
"The only thing necessary for the triumph of evil, is for good men to do nothing." - Edmund Burke (attrib.)
.
User avatar
c627627
Posts: 641
Joined: April 3rd, 2005, 12:58 pm
Location: Kansas City, Missouri
Contact:

Re: Why safebrowsing folder contents can cause a big slow do

Post by c627627 »

***EDIT:***
Upon further experimenting, I have removed Firefox as the only culprit.
As I said, the test setup consisted of Yahoo Classic email being the home page [with Firefox remembering passwords enabled] and all Extensions removed and all about:config settings clicked back to default.

Everything is as posted below, however:
If I log out of Yahoo email or if I click on "Switch to the newest Yahoo Mail" - the problem goes away.

So when people have this problem, it appears that web sites themselves are (also) the cause.
Is Firefox (also) to blame? Other browsers like Chrome and IE do not exhibit the same problem with Yahoo Email Classic.

But this ↑ is definitely crucial information when diagnosing this problem.
Before I knew that, I posted this:


Thank you for test suggestions.
Additional piece of information: I tried to see if I could isolate an individual file inside the safebrowsing folder that is causing this and I did. It is goog-phish-shavar.sbstore
I don't know if that helps but *if* that file and that file alone is deleted on my system, the problem immediately goes away. It is a 4.31MB file and here it is:
http://www.c627627.com/goog-phish-shavar.sbstore

My test setup is Firefox ESR 52.5.3 32-Bit with Home page set to https://mail.yahoo.com/ and since I set it to remember passwords, it automatically logs into Yahoo Email classic and displays the Inbox.


Regarding your tests: The reoccurrence of the problem may take as much a week to reappear. It certainly took at least maybe 3 days for me to notice a slow down of maybe 2 extra seconds.
My test setup has a situation where extra 5 seconds are shaved off once file goog-phish-shavar.sbstore is deleted from the safebrowsing folder.
I would await response to this new information before commencing a week long (at least) test process of your suggestions.
Open the pod bay doors, Cortana.
User avatar
Frank Lion
Posts: 21173
Joined: April 23rd, 2004, 6:59 pm
Location: ... The Exorcist....United Kingdom
Contact:

Re: Why safebrowsing folder contents can cause a big slow do

Post by Frank Lion »

If all you want to do is to stop this slow startup loading, then disable safebrowsing options, delete the safebrowsing folder and you're done. Upon investigation, once the safebrowsing options are unchecked then the downloading of remote files ceases and it is only those local 'test' prefixed files that get updated every 30 minutes and they are not really being updated at all.

However, if you want to keep safebrowsing enabled (or you want to help other people who do) and still have Yahoo mail as your startup on Firefox, then further testing will need to happen.

Before starting the proposed tests, you could try taking some archive file (a xpi or similar) renaming it to goog-phish-shavar.sbstore and putting it in the safebrowsing folder to replace the original one and setting this fake one to 'Read Only'. I've had success with that technique in the past when people had trouble with malware exes that kept coming back. It will only pass a name check, not a hash midsum one, but worth a try.

The tests will then take as long as the tests take until you're satisfied that the test has succeeded or failed, i.e. as soon as the startup page loading time is as long as it was when all this first caused a problem, then that test has failed.

I'm trying to figure out if it's the size of the files (like Spybot's massive, massive hosts file fiasco from years back), the loading of them, the reading of them, or the downloading of them....that is causing the problem.
"The only thing necessary for the triumph of evil, is for good men to do nothing." - Edmund Burke (attrib.)
.
User avatar
c627627
Posts: 641
Joined: April 3rd, 2005, 12:58 pm
Location: Kansas City, Missouri
Contact:

Re: Why safebrowsing folder contents can cause a big slow do

Post by c627627 »

This:
Frank Lion wrote:If all you want to do is to stop this slow startup loading, then disable safebrowsing options, delete the safebrowsing folder and you're done. Upon investigation, once the safebrowsing options are unchecked then the downloading of remote files ceases and it is only those local 'test' prefixed files that get updated every 30 minutes and they are not really being updated at all.
Thanks. That actually is a solution for me. So on latest ESR, this would permanently solve the problem:
Options > Security >
UNCHECK: Warn me when sites try to install add-ons
UNCHECK: Block dangerous and deceptive content

Thank you again for that information.

I will however instead start with your second suggestion:
Create an empty text file inside the safebrowsing folder and rename it to goog-phish-shavar.sbstore and set it to Read-only.
It may take a week or weeks but I look forward to finding out if fake file resolves the problem and report back in a month or so.
Open the pod bay doors, Cortana.
Stereodoc
Posts: 76
Joined: July 23rd, 2011, 12:34 pm

Re: Why safebrowsing folder contents can cause a big slow do

Post by Stereodoc »

Frank,

Yep, the problem I have on that thread is people saying they have this problem and then vanishing. Plus, the one guy who remains (OP?) isn't trying what I suggested and the thread is going nowhere.
you made one post in that thread.

I work 50 plus hours a week and do not transport my desktop computer whenever I'm on business travel. And yes, I did try want you suggested and it did not make any difference. Evidently, I was not clear on that, my bad. Additionally, as I have stated a couple of times, my SM settings are default just like my XP box that is more than 8 years old and another Win7 box that's 2 years old that use the same exact settings and SM version. This particular desktop was reloaded with win7 8 months ago. Maybe SM is configuring itself differently for this particular box or a particular website I frequent is changing something. I don't know what it is and how to narrow it down. Is that possible?
User avatar
Frank Lion
Posts: 21173
Joined: April 23rd, 2004, 6:59 pm
Location: ... The Exorcist....United Kingdom
Contact:

Re: Why safebrowsing folder contents can cause a big slow do

Post by Frank Lion »

Stereodoc wrote:Frank...

...you made one post in that thread.
That's one more than most people get from me. You are not the only one who is busy with other things.

Glad your problem is now solved.
"The only thing necessary for the triumph of evil, is for good men to do nothing." - Edmund Burke (attrib.)
.
Post Reply