browser.cache.disk.capacity >4G?

[GrampaWildWilly]

I have looked high & low. Googled the subject. Searched Bugzilla. Searched Mozillazine. I can't find my answer. I have changed the necessary about:config settings (advice which I DID find via Google) to move my disk cache to a non-default place. But I cannot find a straight answer about the size of disk cache. I currently have my browser.cache.disk.capacity set to 4194303. That gives me a disk cache size of 4G-1K. But the size reverts to 0 when I try browser.cache.disk.capacity=4194304 This tells me that the disk cache management logic of Firefox is written using 32-bit APIs, despite the fact that I am running Firefox 66.0.3 64-bit, if I can believe Help->About Firefox. Please. Somebody in the know. Tell me if it is possible to set disk cache size above 4G. Are there plans to bring the disk cache management logic into the modern world using 64-bit APIs? Are we ever going to get a proper GUI interface in the Preferences dialog to allow relocating & resizing disk cache? This is something we should have & for the life of me I can't figure out why we don't. Didn't we used to have such a facility many releases back? Why would such an obviously useful feature be removed? Why are we forced to use the most user-unfriendly, most non-intuitive possible interface (about:config) for doing something so routine?

[malliz]

As far as I can remember you are the only one in ten years with this request so hardly a common problem

[kerft]

If one does not exist after searching, you can file a bug report at bugzilla.mozilla.org. I have no reason to doubt that your testing is correct and that 4GB is the current limit for disk cache. Note that most heavy websites such as webmail do not seem to store most of their data in the disk cache anyway. They use DOM storage. I do not know what the limit for DOM storage is, I think it may be a limit per site. Actually I heard something that the limit for storage is not yet even being enforced so extensions for example can use as much as they want. I don't know if that is also true for website DOM storage.

[therube]

Why would such an obviously useful feature be removed?

What feature is that, to have a cache size > 4 GB?

And with that, just what is the benefit of such a large /cache/?

[GrampaWildWilly]

Why would such an obviously useful feature be removed?

What feature is that, to have a cache size > 4 GB?

No, the ability to set disk cache size within the standard GUI for setting Preferences. I'm fairly sure the Preferences dialog used to include a way to set the maximum disk cache size some unknown number of years ago but that's no longer available. We must resort to the arcane technique of tweaking about:config.

And with that, just what is the benefit of such a large /cache/?

I regularly visit web pages that include 100, 150, 200 large images. And by large I mean in the 5M-10M range. I'm finding a 4G disk cache does help but it's really not enough. I have relocated my Firefox disk cache to a multi-terabyte drive, so a mere 4G is pitifully small. I'd like to experiment with a 16G disk cache for starters to see how that improves things.

If one does not exist after searching, you can file a bug report at bugzilla.mozilla.org.

Not sure I understand what you are suggesting I post on Bugzilla. A suggestion to add back the Preferences dialog for setting disk cache size? A suggestion to document the maximum disk cache size? I thought they wanted to focus on actual program failures, not perceived shortcomings or design flaws. Do clarify please. If they take suggestions on Bugzilla, I'll suggest this.

Note that most heavy websites such as webmail do not seem to store most of their data in the disk cache anyway. They use DOM storage. I do not know what the limit for DOM storage is, I think it may be a limit per site.

I'm pretty sure I'm not encountering this issue, but it is something I'll keep in the back of my mind should it ever become an issue for me. Thanks for mentioning it.

Actually I heard something that the limit for storage is not yet even being enforced so extensions for example can use as much as they want. I don't know if that is also true for website DOM storage.

Again, thanks but I don't believe this is relevant to me at the moment.

As far as I can remember you are the only one in ten years with this request so hardly a common problem

Now I'm basking in the glow that only a true pioneer can feel. Thank you! Smartassness aside, I'm frankly surprised. With the advent of cheap, large disk storage, I'd have thought somebody would have wanted to use a large disk cache at some point. Perhaps the silent majority is truly silent. Perhaps certain performance issues are not being connected to disk cache size. I have stumbled across a number of reports from people who wish to shrink or disable disk cache in an attempt to improve performance. I think the implied criticism is that the logic for managing disk cache is not particularly efficient. Bunping my disk cache up from whatever it was (don't rightly remember now what I had it set at before) to 4G has improved things in the way I expected. I have not encountered performance degradation due to inefficient disk cache memory management. Revisiting those image-intensive pages now takes less time because the large pools of images are conveniently saved in my disk cache, which I do not empty when I close the browser. Of course, there is a limit & some of the 4G worth of images does incrementally cycle out of cache gradually, which brings me to my conclusion that I'd like to try a bigger disk cache to see whether I can improve things even further.

I am encouraged by the discussion here. I'm hoping it results in a change to the product.

[Brummelchen]

browser.cache.disk.capacity=4194304
means 4meg, not gig.
and it depends on your hard drive, varies each system, mine is browser.cache.disk.capacity;696320 and that is not set by me.

[kerft]

Mozilla / Firefox does take suggestions at bugzilla. There are rules, and they don't like content-free replies to existing bugs.

[GrampaWildWilly]

browser.cache.disk.capacity=4194304
means 4meg, not gig.
and it depends on your hard drive, varies each system, mine is browser.cache.disk.capacity;696320 and that is not set by me.

Brummelchen, you are quite simply wrong. The value of the preference in about:config is multiplied by 1K=1024 to calculate the total maximum size of the browser cache. You should try a dir command on your cache. My cache is located on my N: drive where there is a directory named cache2. I did a dir on that & at the bottom it says (cut/pasted):

21671 File(s) 4,265,044,832 bytes

4G=4x(1Kx1Kx1K)=4x(1024x1024x1024)=4x(1,073,741,824)=4,294,967,296

So I'm using not quite 4G, which is totally expected. I mean, it would be pretty coincidental if your cache entries happened to add up to exactly 4G on the number. Besides, the maximum disk cache size is actually 4G-1K=4,294,966,272.

Alternatively, I went into Windows Explorer & popped up the properties dialog on that directory. It reports a size of 3.97GB.

Yet another alternative is to use Firefox itself. Visit "site" about:cache. In the Disk section, it reports (cut/pasted)

Maximum storage size: 4194303 KiB

Note the suffix of KiB. The preference in about:config is multiplied by 1K. If you try what I did, you'll find a preference value of 4194303 gives you 4G-1K maximum cache. But add 1 & you'll find 4194304 does not at all give you what you want.

[dickvl]

This could also be a limitation for Integer prefs that do not support Integer values beyond 32 bit.
Larger values would have to be string values just like is done for floating point values although the latter sometimes use integer values that have been shifted two digits (xxx.xx -> xxxxx).

[GrampaWildWilly]

This could also be a limitation for Integer prefs that do not support Integer values beyond 32 bit.

Now that's an interesting thought. But I don't think that's the issue. 2**22=4,194,304, which is 1 greater than the largest value that actually works as a value for browser.cache.disk.capacity. So the prederence value fits comfortably in 22 bits. Since the preference value is interpreted as being multiplied by 1K, that would be another 10 bits (1024=1K=2**10), taking it up to 32 bits. But the full 2**32=4G is not the maximium disk cache size. The parameter value has a maximum of 4,194,303, making the effective maximum cache size (2**32)-(2**10)=4,294,966,272. This maximum parameter value is something I have determined by trial & error. It should have been something I could look up in the Firefox documentation but like I said, I haven't found that documentation. This 4G-1K maximum size is what led me to suspect that despite the browser overall supposedly being 64-bit, the cache logic must still be written with 32-bit APIs. I admit that's a bit of . . . well, maybe not quite reverse engineering but certainly jumping to a conclusion that fits the observations. I very well could be wrong. That's why I've asked my question. I'm still hoping that somebody who has read the code & understood it comes in here & sets us all straight.

Larger values would have to be string values just like is done for floating point values although the latter sometimes use integer values that have been shifted two digits (xxx.xx -> xxxxx).

More interesting ideas. I had no idea about this. Can you give an example of a preference that works this way? Of course, I would need assurance that browser.cache.disk.capacity is one such preference. My experimenting seems to contradict that conclusion but I've been just going by trial & error. I would love it if, as I said above, somebody who knows the code came in here & spoke truth.

[atlanto]

>I currently have my browser.cache.disk.capacity set to 4194303. That gives me a disk cache size of 4G-1K. But the size reverts to 0 when I try browser.cache.disk.capacity=4194304

I'm still wondering what "reverts to 0" means.

prefs.js
>user_pref("browser.cache.disk.capacity", 4194304);
>user_pref("browser.cache.disk.parent_directory", "D:\\firefox\\cachedir");
>user_pref("browser.cache.disk.smart_size.enabled", false);

about:cache
>disk
>Number of entries: 4749
>Maximum storage size: 4194304 KiB
>Storage in use: 315660 KiB
>Storage disk location: D:\firefox\cachedir\cache2
(...it's hard to fill such large cache...)

[GrampaWildWilly]

>about:cache
>disk
>Maximum storage size: 4194304 KiB

Oh now really . . . You have inspired me. I now have this:

Number of entries: 18630
Maximum storage size: 16777216 KiB
Storage in use: 3733216 KiB

16G. Just like I always wanted. I will confess that the last time I tried to do this, setting browser.cache.disk.capacity to 4194304 caused my cache to go to 0. In effect, it disabled disk caching because the value was invalid. Was. When I tried it last, several months ago. In the interim, I have not tried it. I didn't want to lose the 4G of stuff I did have in the disk cache. But when I tried it tonight, things seem to have worked fine & my existing cache contents have evidently been retained. It appears we have received a stealth upgrade to the disk cache logic at some point during the last several releases of Firefox. I'm on 66.0.3, which says it's up to date.

(...it's hard to fill such large cache...)

Oh trust me. I'll have no problem filling it. If you visit any site that has lots of large images -- photos & whatever -- you can fill it up no problem.

I wonder if it really would support a disk cache of size 2**64. That's actually way bigger than my entire hard drive. But that's idle curiosity. I'm going to run with this 16G cache for a while & see how it goes, see if it's enough, see if I think I can use more. I'll post back. I guess there's no reason to post anything to Bugzilla.

I am most grateful to you, atlanto, for posting. You have justified my posting here. I don't think I would have gotten the knowledge I needed any other way.

[Brummelchen]

about:cache
well, you are right, ist measured in KiB (1024, KB=1000)
Maximum storage size: 696320 KiB
Storage in use: 146905 KiB

and yes, not on drive C here. as you can see firefox set max a lot more as it needs. i would not count on that. here is a special drive with current 8gig free.

sorry.

[therube]

So to do that, you did what ?

Something like:

>user_pref("browser.cache.disk.capacity", 16000000);
>user_pref("browser.cache.disk.smart_size.enabled", false);

And with that, your wanted size stuck?

In your earlier testing, did you also change browser.cache.disk.smart_size.enabled (to false) ?

[GrampaWildWilly]

So to do that, you did what ?

Sorry I didn't answer sooner. Even though I am subscribed to this thread, for some reason I did not get any E-mail notification that you had posted. I've just looked through my profile & I can't see why I didn't get the notification. Oh well. Better late than never, I suppose.

Here's every relevant setting I have as reported to me by about:config:

browser.cache.disk.capacity;16777216
browser.cache.disk.enable;true
browser.cache.disk.filesystem_reported;1
browser.cache.disk.free_space_hard_limit;1024
browser.cache.disk.free_space_soft_limit;5120
browser.cache.disk.hashstats_reported;1
browser.cache.disk.max_chunks_memory_usage;40960
browser.cache.disk.max_entry_size;51200
browser.cache.disk.max_priority_chunks_memory_usage;40960
browser.cache.disk.metadata_memory_limit;250
browser.cache.disk.parent_directory;N:
browser.cache.disk.preload_chunk_count;4
browser.cache.disk.smart_size.enabled;false
browser.cache.disk.smart_size.first_run;false
browser.cache.disk.smart_size.use_old_max;false
browser.cache.disk_cache_ssl;true

The ones in bold are the ones that are not at their default values. Some of the ones that are not at their default values, I do not believe I changed. I think they may have been a result of some activity that Firefox performed unbeknownst to me. But some of them I did change myself after reading things I found via Google searches. All I know is this is what I am currently running with. At the moment, about:cache is telling me this:

disk
Number of entries: 55571
Maximum storage size: 16777216 KiB
Storage in use: 12827456 KiB
Storage disk location: N:\cache2

Looks to me like 16G is a good size for my disk cache. I was toying with the idea of bumping it to 32G but it's taking rather longer than I expected for me to fill my disk cache to capacity. I'm sure I will get there in a few weeks but it's working fine so far.