Javascript Performance Thread

Discussion about official Mozilla Firefox builds
Post Reply
Srap
Posts: 290
Joined: July 18th, 2012, 11:57 am

Re: Javascript Performance Thread

Post by Srap »

KWierso wrote:
jiangiang wrote:v8 take the crown from ionMonkey in kraken test

http://arewefastyet.com/?a=b&view=regress

20 ms faster? Oh the humanity!

Wouldn't be surprised if they did this on purpose. Just like when the cat plays with the mouse.
Sorry for my bad English. Even if there wasn't a mistake.
User avatar
Grantius
Posts: 1545
Joined: June 28th, 2011, 4:14 pm
Contact:

Re: Javascript Performance Thread

Post by Grantius »

Remember, fast Javascript benchmarks is the only thing that matters - just look at toms hardware!

Seriously, any little bit counts I guess.
Micro gaming box: AMD A10-7800 APU, 8gb RAM M350 ITX case (size of a book), Windows 10/Ubuntu
Tablet/Laptop: Asus Transformer T100, Intel Atom 2GB RAM, Windows 10 x86
Mobile:Xiaomi Redmi Note 3 Pro
dutchguy
Posts: 199
Joined: October 14th, 2003, 12:39 am

Re: Javascript Performance Thread

Post by dutchguy »

Someone at Apple has been working hard (if I look at AWFY)?
User avatar
Zlip792
Posts: 1340
Joined: May 7th, 2011, 1:47 pm
Location: Pakistan

Re: Javascript Performance Thread

Post by Zlip792 »

dutchguy wrote:Someone at Apple has been working hard (if I look at AWFY)?

Their engine blog post: http://wingolog.org/archives/2011/10/28 ... ementation
User avatar
cyko_01
Posts: 77
Joined: August 2nd, 2011, 6:52 pm

Re: Javascript Performance Thread

Post by cyko_01 »

Zlip792 wrote:
dutchguy wrote:Someone at Apple has been working hard (if I look at AWFY)?

Their engine blog post: http://wingolog.org/archives/2011/10/28 ... ementation

ummm that blog post is almost a year old
User avatar
Zlip792
Posts: 1340
Joined: May 7th, 2011, 1:47 pm
Location: Pakistan

Re: Javascript Performance Thread

Post by Zlip792 »

cyko_01 wrote:
Zlip792 wrote:
dutchguy wrote:Someone at Apple has been working hard (if I look at AWFY)?

Their engine blog post: http://wingolog.org/archives/2011/10/28 ... ementation

ummm that blog post is almost a year old


Yes, I do know but this is about their DFG JIT, which they added and improving.
bksening
Posts: 26
Joined: December 22nd, 2009, 5:05 pm

Re: Javascript Performance Thread

Post by bksening »

Zlip792 wrote:Yes, I do know but this is about their DFG JIT, which they added and improving.

But what's the relevance to dutchguy's comment about Apple's Javascript improvement this month from AWFY with a posting 1 year old?
coolg1024
Posts: 12
Joined: June 28th, 2012, 5:09 pm

Re: Javascript Performance Thread

Post by coolg1024 »

On another note, chrome might be the fastest to run the GB-EMU benchmark on octane, but it sure as hell can't render the web app correctly where the code came from: http://i.imgur.com/epRYo.png
Tametomo
Posts: 42
Joined: April 1st, 2011, 10:03 am

Re: Javascript Performance Thread

Post by Tametomo »

bksening wrote:But what's the relevance to dutchguy's comment about Apple's Javascript improvement this month from AWFY with a posting 1 year old?


They enabled it on 32-bit finally, while they've had it enabled on 64-bit for a while.

For instance, from the article:

JavaScriptCore Blog wrote:As far as I can tell, the DFG JIT is shipping in Mac OS X Lion's Safari browser, but it is not currently enabled on any platform other than 64-bit Mac. (I am working on getting that fixed.)
User avatar
cyko_01
Posts: 77
Joined: August 2nd, 2011, 6:52 pm

Re: Javascript Performance Thread

Post by cyko_01 »

what is the difference between ss-aes and kraken-crypto-aes? why do we do so well on kraken and so poorly on ss? this could be a nice win on sunspider if we can fix it
Srap
Posts: 290
Joined: July 18th, 2012, 11:57 am

Re: Javascript Performance Thread

Post by Srap »

Chrome engine (V8, if I am not mistaken) seems to be missing from AWFY. Any reasons behind this?
Sorry for my bad English. Even if there wasn't a mistake.
User avatar
cyko_01
Posts: 77
Joined: August 2nd, 2011, 6:52 pm

Re: Javascript Performance Thread

Post by cyko_01 »

It was making us look bad, so they nuked it :P
Tametomo
Posts: 42
Joined: April 1st, 2011, 10:03 am

Re: Javascript Performance Thread

Post by Tametomo »

Srap wrote:Chrome engine (V8, if I am not mistaken) seems to be missing from AWFY. Any reasons behind this?


While I don't know for sure, I can think of a couple of reasons why they might have done it (assuming that d8 didn't break, move repository locations, or that the local build system for d8 didn't break. Also going to call it d8 as well, like Mozilla does internally, so as to not confuse it with the benchmark of the same name):

1. If I am not mistaken, I could have sworn I've seen some accusations internally of d8 trying to game benchmarks a bit and getting increasingly frustrated with it, even to the point where while some code might work within a benchmark, and which might make them look good, they then aren't always doing full test coverage and end up broken with functionality or uses outside of benchmarks on newer javascript features. For instance, while I don't have a link handy, I think when Octane was first announced, there was a claim made that one of the benchmarks, if compiled with Emscripten instead of Mandreel, caused a compile failure when it was valid code. Don't remember which though, and could always be making something rather minor appear to be bigger than it was, and might even be remembering it wrong. I also wouldn't quote me on this as a source, as I'm not involved with any browser team, nor do I do a lot of work with javascript itself. If I'm wrong on this, then someone please let me know, and I'll gladly strike this (although I don't check here too often. Sorry about that).

2. Likewise, going along with number 1, but not requiring it to be true, there do seem to have been some concerns about their recent benchmarks seemingly being geared towards making chrome look good, but not necessarily being representative of advancing the web as a whole. For instance, I remember seeing comments about how while some of the ideas in Octane probably weren't all that bad, it seemed too geared towards being a testing suite for marketing apps from the Chrome store, and that benchmarks based on bananabread or emscripten, for instance, might be better suited for real web performance purpose.

3. Dead code elimination being added to d8, which then removes some reproducability as it retains past information about what was used and what wasn't, and also brings some concerns into correctness issues for the code generated (not that any new compiler optimization technique doesn't already do this, but dead code elimination is somewhat controversial in some respects, since it can be rather easy to get it wrong).

4. An increasing frustration with d8's team relying on AWFY too much for their own testing resources, when the purpose is to help Mozilla better visualize and track their own regressions and improvements internally, even going as far as requesting that they remove V8 and replace it with Octane, so that they could track how they've been making progress on it a bit more easily, when the developers (at least from what I've seen) haven't been quite as enthusiastic about it, since there appear to be doubts about whether it's really designed for what they think they need for real web performance.

5. Other benchmarks like Robohornet being developed, which then bring into question even more about whether the right things are even being tested at all, and whether or not a lot of these newer benchmarks being pushed by Google now are even measuring the right things any more (for instance, see the open bugs by kevingadd). Then again, this is essentially an extension of #2, just for a different benchmark.

6. d8 becoming increasingly irrelevant to benchmark against, as it's not too different from JSC in performance now on V8, where Mozilla is the farthest behind at the moment. This was the original reason why JSC was originally removed IIRC (that, and it having been broken for a day or two at one point, making it so that it wasn't showing results), since at one point last year, they beat JSC by a rather large margin, and they didn't appear to be improving fast enough to warrant them being tracked any more. But then they landed quite a few changes to the compiler, and it started to get close to d8's performance and exceeded it on sunspider, which then prompted them to add it back again, as it became a relevant comparison again. In fact, it might be that last part which might have prompted dropping d8, since as far as I know, they also don't have a generational garbage collector yet, but are able to get close to d8 speeds on V8. And since JSC's compiler is now currently closer to what they are aiming to be, and that it's similar enough in benchmarking performance to be a good baseline to chase, then there's not much of a need for d8 to be profiled against at this time.


Could be any combination of these or none of these. Just my own wild mass speculations that I don't think are too far off the mark.

Also remember that AWFY was originally meant to be a developer tool to help them improve their own engine, and not so much a marketing tool, even though it has kind of turned into one. It should first suit their own internal needs rather than what the community might want. So if this is not an accident, and they decided internally that it's not helping them much any more to profile d8, then I don't see why they should continue to have to do so if the community asks for it. The code for the site is open sourced, so if someone else decides that they want to track things differently, then it's completely possible to run your own private copy to display what you want to see.
User avatar
KilliK
Posts: 612
Joined: June 18th, 2004, 7:11 am

Re: Javascript Performance Thread

Post by KilliK »

simply put, you can't beat a multi-billion dollar company..
rkl
Posts: 70
Joined: November 5th, 2002, 9:08 am

Can you benchmark the browser binary's JS performance?

Post by rkl »

I know this might sound like a naive question, but can you benchmark the JS performance automatically using just the browser binaries (i.e. without having to build from source)? In other words, can all the data points from AWFY be gathered from shipped binary releases (whether they're nightly, alpha, beta or final)?

If that's the case, then I'd have thought that a) you wouldn't keep having to build releases to produce AWFY data points and b) you could easily benchmark nightlies of Firefox against, say, the latest finals of all the other major browsers (including IE!), which is probably what you should be doing in AWFY...trying to compare how your latest JS fares against what the public already has out there (since very few users relatively speaking use anything other than final versions of browsers).

(I'd also include the latest final of Firefox vs. nightly Firefox, since that's an important comparison (ideally, you want it the same or better - within error tolerances - between the two).
Post Reply