Large files no longer get cached

Discussion of bugs in Seamonkey
Post Reply
barbaz
Posts: 1504
Joined: October 1st, 2014, 3:25 pm

Large files no longer get cached

Post by barbaz »

I have set about:config > browser.cache.disk.max_entry_size to -1, so that large files don't keep getting re-downloaded just because they're large. :-k
Recently noticed that on pages with large files included, such as embedded video, that SeaMonkey keeps re-downloading large files anyway every time I close/reopen the page, despite that about:config setting. Clean profile doesn't help, nor does reverting to SeaMonkey 2.39 or 2.35 where I'm certain large files definitely used to get cached.

I'm testing with these STR:
1) put a large video file (~450 MB) on a local server, create a simple html page to embed it
2) set SeaMonkey to use cache2 (not needed to reproduce the problem, but makes it easier to see what's going on):
about:config > set browser.cache.use_new_backend;1 and browser.cache.use_new_backend_temp;true
I restart SeaMonkey after this
3) SeaMonkey Preferences > Advanced > Cache, set cache size to manually managed, at least 2 GB (2048)
4) navigate to the page with the embedded video, play the video
5) check what's in the disk cache

Large video is intermittently freezing the browser as it's downloading, and it's not kept in cache. Cache size can even be increased to 10+ GB (at least 10240) and it still won't cache the video. (There also seems to be some limit on cache size, such that it doesn't work to set it at 20 GB?)
Repeating these STR with small webm video instead, and everything works as expected.

I've checked the cache2 source on mxr, and it seems part of this behavior is by design (there is a hard limit on cache entry size is 1/8 the total cache size), but part of what I'm experiencing still inexplicable to me, so I'm posting in SeaMonkey Bugs.

Not caching unchanging large files will unnecessarily burn up bandwidth, and this stuff matters on a metered connection.
Any ideas what changed or how to get SeaMonkey caching large files again?
User avatar
therube
Posts: 21714
Joined: March 10th, 2004, 9:59 pm
Location: Maryland USA

Re: Large files no longer get cached

Post by therube »

And the associated version of FF does what when doing the same?

If anything, aren't you more likely to get "chunked" files with /cache2/ (compared to /cache/)?
Fire 750, bring back 250.
Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.19) Gecko/20110420 SeaMonkey/2.0.14 Pinball CopyURL+ FetchTextURL FlashGot NoScript
barbaz
Posts: 1504
Joined: October 1st, 2014, 3:25 pm

Re: Large files no longer get cached

Post by barbaz »

therube wrote:And the associated version of FF does what when doing the same?
Forgot to add, I tested both Firefox 45.1.1esr and 46.0.1 as well, and got the same behavior.
therube wrote:If anything, aren't you more likely to get "chunked" files with /cache2/ (compared to /cache/)?
IIRC cache2 doesn't split up large files. But the issue here is that large files aren't getting stored in cache at all.
barbaz
Posts: 1504
Joined: October 1st, 2014, 3:25 pm

Re: Large files no longer get cached

Post by barbaz »

I'm thinking this is likely not a bug. Some playing around with Firefox Nightly shows there is actually a 2 GB cap on cache size, even though the pref for disk cache capacity can be set higher. That would explain it.

Now where (or why) that's being set...? :-k

(Thanks frg for the help over IRC!)
frg
Posts: 1361
Joined: December 15th, 2015, 1:20 pm

Re: Large files no longer get cached

Post by frg »

Search for EntryisTooBig and mSkipSizeCheck:

https://dxr.mozilla.org/mozilla-central ... rect=false

I just looked over the source and I think there is a hard limit of 4GB for a file and a maximum file size of 1/8 of the ram and also disk size.

FRG
barbaz
Posts: 1504
Joined: October 1st, 2014, 3:25 pm

Re: Large files no longer get cached

Post by barbaz »

barbaz wrote:there is actually a 2 GB cap on cache size
Based on some more discussion on IRC, it looks like this is not a "hard limit" but based on some property of my machine...
Post Reply