Javascript performance: 3.1 nightly vs Chrome 0.2.149.30

[unquietwiki]

Test ran on 10-12-2008: "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1b2pre) Gecko/20081012 Minefield/3.1b2pre" vs Chrome 0.2.149.30.
Test machine: Athlon X2 4200+; Vista x64 SP1; 4GB RAM
Test ran with Sunspider: http://www2.webkit.org/perf/sunspider-0 ... pider.html


TEST COMPARISON FROM TO DETAILS

=============================================================================

** TOTAL **: *1.82x as slow* 2380.8ms +/- 1.3% 4330.4ms +/- 0.7% significant

=============================================================================

3d: *3.46x as slow* 163.0ms +/- 1.4% 564.2ms +/- 0.5% significant
cube: *4.77x as slow* 41.2ms +/- 8.4% 196.6ms +/- 0.6% significant
morph: *2.98x as slow* 69.2ms +/- 5.9% 206.2ms +/- 0.7% significant
raytrace: *3.07x as slow* 52.6ms +/- 2.7% 161.4ms +/- 2.1% significant

access: *6.22x as slow* 123.6ms +/- 4.1% 768.2ms +/- 1.2% significant
binary-trees: *8.60x as slow* 8.0ms +/- 11.0% 68.8ms +/- 1.5% significant
fannkuch: *8.28x as slow* 44.2ms +/- 2.4% 365.8ms +/- 0.7% significant
nbody: *5.06x as slow* 40.6ms +/- 10.1% 205.6ms +/- 5.3% significant
nsieve: *4.16x as slow* 30.8ms +/- 5.3% 128.0ms +/- 3.6% significant

bitops: *7.91x as slow* 89.8ms +/- 5.0% 710.2ms +/- 0.8% significant
3bit-bits-in-byte: *21.2x as slow* 6.2ms +/- 9.0% 131.6ms +/- 1.1% significant
bits-in-byte: *15.1x as slow* 12.2ms +/- 4.6% 183.8ms +/- 1.3% significant
bitwise-and: *4.99x as slow* 32.4ms +/- 5.8% 161.8ms +/- 0.3% significant
nsieve-bits: *5.97x as slow* 39.0ms +/- 9.6% 233.0ms +/- 0.8% significant

controlflow: *21.6x as slow* 3.4ms +/- 20.0% 73.6ms +/- 1.5% significant
recursive: *21.6x as slow* 3.4ms +/- 20.0% 73.6ms +/- 1.5% significant

crypto: *4.12x as slow* 75.2ms +/- 3.4% 310.0ms +/- 1.1% significant
aes: *3.93x as slow* 29.8ms +/- 3.5% 117.0ms +/- 1.3% significant
md5: *3.98x as slow* 24.0ms +/- 8.2% 95.6ms +/- 1.7% significant
sha1: *4.55x as slow* 21.4ms +/- 3.2% 97.4ms +/- 1.5% significant

date: 1.79x as fast 491.0ms +/- 7.0% 274.8ms +/- 1.5% significant
format-tofte: 1.48x as fast 247.8ms +/- 1.7% 167.0ms +/- 1.7% significant
format-xparb: 2.26x as fast 243.2ms +/- 12.8% 107.8ms +/- 1.5% significant

math: *3.65x as slow* 150.0ms +/- 2.8% 547.8ms +/- 3.6% significant
cordic: *3.11x as slow* 83.0ms +/- 4.4% 258.4ms +/- 1.0% significant
partial-sums: *3.47x as slow* 47.2ms +/- 5.7% 163.6ms +/- 10.9% significant
spectral-norm: *6.35x as slow* 19.8ms +/- 5.3% 125.8ms +/- 1.1% significant

regexp: 1.91x as fast 539.8ms +/- 0.4% 282.4ms +/- 5.8% significant
dna: 1.91x as fast 539.8ms +/- 0.4% 282.4ms +/- 5.8% significant

string: *1.07x as slow* 745.0ms +/- 1.5% 799.2ms +/- 0.7% significant
base64: 1.07x as fast 97.2ms +/- 7.5% 90.6ms +/- 1.2% significant
fasta: *2.48x as slow* 81.2ms +/- 2.3% 201.6ms +/- 1.2% significant
tagcloud: 1.39x as fast 205.0ms +/- 1.9% 148.0ms +/- 1.3% significant
unpack-code: 1.03x as fast 256.0ms +/- 2.4% 247.8ms +/- 1.6% significant
validate-input: *1.05x as slow* 105.6ms +/- 2.1% 111.2ms +/- 3.0% significant

[Omega X]

Just to meet apples with apples:

Chromium 0.3.155.0 vs Gecko/20081012 Minefield/3.1b2pre

=============================================================================

** TOTAL **: 1.29x as fast 4370.0ms +/- 4.9% 3381.4ms +/- 8.5% significant

=============================================================================

3d: - 347.0ms +/- 8.2% 317.0ms +/- 16.4%
cube: *1.41x as slow* 84.8ms +/- 2.4% 119.8ms +/- 2.9% significant
morph: 2.12x as fast 165.0ms +/- 9.4% 77.8ms +/- 10.3% significant
raytrace: ?? 97.2ms +/- 13.9% 119.4ms +/- 46.9% not conclusive: might be *1.23x as slow*

access: *2.31x as slow* 222.6ms +/- 11.1% 513.2ms +/- 31.5% significant
binary-trees: *7.57x as slow* 17.6ms +/- 16.3% 133.2ms +/- 46.4% significant
fannkuch: *3.83x as slow* 69.4ms +/- 8.3% 265.6ms +/- 57.3% significant
nbody: ?? 80.2ms +/- 16.7% 84.8ms +/- 3.2% not conclusive: might be *1.06x as slow*
nsieve: 1.87x as fast 55.4ms +/- 9.7% 29.6ms +/- 18.5% significant

bitops: 1.97x as fast 153.2ms +/- 4.1% 77.6ms +/- 7.0% significant
3bit-bits-in-byte: 3.67x as fast 11.0ms +/- 24.0% 3.0ms +/- 0.0% significant
bits-in-byte: 1.18x as fast 18.0ms +/- 8.5% 15.2ms +/- 3.7% significant
bitwise-and: 8.19x as fast 50.8ms +/- 4.7% 6.2ms +/- 9.0% significant
nsieve-bits: 1.38x as fast 73.4ms +/- 6.6% 53.2ms +/- 9.6% significant

controlflow: *19.0x as slow* 6.0ms +/- 0.0% 114.0ms +/- 1.7% significant
recursive: *19.0x as slow* 6.0ms +/- 0.0% 114.0ms +/- 1.7% significant

crypto: *1.26x as slow* 135.0ms +/- 5.9% 170.4ms +/- 2.2% significant
aes: *1.42x as slow* 51.8ms +/- 7.1% 73.4ms +/- 7.4% significant
md5: *1.62x as slow* 44.4ms +/- 6.1% 71.8ms +/- 3.3% significant
sha1: 1.54x as fast 38.8ms +/- 11.4% 25.2ms +/- 8.1% significant

date: 2.18x as fast 1112.8ms +/- 8.7% 510.6ms +/- 7.7% significant
format-tofte: 1.53x as fast 432.4ms +/- 2.4% 282.8ms +/- 6.8% significant
format-xparb: 2.99x as fast 680.4ms +/- 13.9% 227.8ms +/- 8.9% significant

math: 1.87x as fast 333.0ms +/- 8.3% 178.2ms +/- 7.1% significant
cordic: 1.64x as fast 191.4ms +/- 14.0% 117.0ms +/- 9.4% significant
partial-sums: 2.48x as fast 103.8ms +/- 4.9% 41.8ms +/- 23.6% significant
spectral-norm: 1.95x as fast 37.8ms +/- 6.3% 19.4ms +/- 11.6% significant

regexp: 1.49x as fast 800.4ms +/- 2.3% 538.6ms +/- 3.7% significant
dna: 1.49x as fast 800.4ms +/- 2.3% 538.6ms +/- 3.7% significant

string: 1.31x as fast 1260.0ms +/- 6.9% 961.8ms +/- 7.1% significant
base64: 3.29x as fast 127.0ms +/- 10.9% 38.6ms +/- 6.7% significant
fasta: *1.39x as slow* 133.2ms +/- 17.4% 185.0ms +/- 8.9% significant
tagcloud: 1.20x as fast 287.2ms +/- 13.2% 238.4ms +/- 7.7% significant
unpack-code: 1.33x as fast 510.6ms +/- 4.7% 384.4ms +/- 5.1% significant
validate-input: 1.75x as fast 202.0ms +/- 1.4% 115.4ms +/- 16.3% significant


The latest Minefield hourly is faster than Chromium's hourly.

Minefield: http://www2.webkit.org/perf/sunspider-0 ... sults.html?{%223d-cube%22:[118,119,124,121,117],%223d-morph%22:[74,76,89,77,73],%223d-raytrace%22:[98,99,102,98,200],%22access-binary-trees%22:[110,114,112,108,222],%22access-fannkuch%22:[205,205,484,207,227],%22access-nbody%22:[83,82,87,86,86],%22access-nsieve%22:[24,28,36,29,31],%22bitops-3bit-bits-in-byte%22:[3,3,3,3,3],%22bitops-bits-in-byte%22:[16,15,15,15,15],%22bitops-bitwise-and%22:[6,6,7,6,6],%22bitops-nsieve-bits%22:[51,54,60,50,51],%22controlflow-recursive%22:[115,114,112,113,116],%22crypto-aes%22:[68,75,70,75,79],%22crypto-md5%22:[71,70,75,71,72],%22crypto-sha1%22:[27,24,26,26,23],%22date-format-tofte%22:[273,277,274,280,310],%22date-format-xparb%22:[218,221,222,221,257],%22math-cordic%22:[111,112,118,112,132],%22math-partial-sums%22:[56,38,39,38,38],%22math-spectral-norm%22:[21,21,17,18,20],%22regexp-dna%22:[521,533,539,535,565],%22string-base64%22:[40,41,36,37,39],%22string-fasta%22:[180,179,175,183,208],%22string-tagcloud%22:[232,232,246,222,260],%22string-unpack-code%22:[373,391,376,373,409],%22string-validate-input%22:[102,119,110,106,140]}

Chromium: http://www2.webkit.org/perf/sunspider-0 ... ,205%5D%7D

[dutchguy]

So the latest chrome build is a lot slower than older one?

[Stifu]

So the latest chrome build is a lot slower than older one?

Or Chrome is faster than Chromium.
Or cwolfsheep's Firefox profile is bloated and he needs a new one... Could be anything.

[Ted Mielczarek]

You can't compare sunspider results between different computers. You can only compare results between different browsers running on the same computer. All that indicates is that cwolfsheep has a faster computer than Omega X.

[teoli2003]

And does cwolfsheep has JIT enabled for his tests?

[sciguyryan]

You can't compare sunspider results between different computers. You can only compare results between different browsers running on the same computer. All that indicates is that cwolfsheep has a faster computer than Omega X.

That and it's still not a fair comparison at this point anyway as recursion still isn't traced yet. Will be more interesting to see the performance when that is done (should be done before b2 hopefully).

[Gobd]

Heres my test. Score is just directly related to time and is just an easier ad smaller number to compare. Smaller score means faster time. So using the score you can say Chromium build 3286 is about 2x as fast as Firefox 20081013 with JIT disabled. The TM build used is 20081001 the most recent I saw. The Chrome build is 0.2.149.30

The TM build is just slightly slower than Chromium. I'd say by release JIT will be faster than Chromium but we're already beating the release version of Chrome. Now if only JIT would quit crashing on such popular websites.

[Omega X]

Now if only JIT would quit crashing on such popular websites.

They are still working on it, so I don't expect the crashing to stay long. It rarely crashes on any of the sites that I visit.

[sciguyryan]

Now if only JIT would quit crashing on such popular websites.

They are still working on it, so I don't expect the crashing to stay long. It rarely crashes on any of the sites that I visit.

Me too. It seems pretty stable. The best thing anyone can do is report the crashes so that the devs see them to take notice at this point.

[sciguyryan]

It looks like another merge with tracermonkey into the trunk was done, may be worth retesting these in one of the later hourlies.

[Omega X]

OK, got the latest hourly once again.

Minefield/3.1b2pre ID:20081013153204 vs Chromium 0.3.155.0 Build: 3328

Sunspider Benchmark:

=============================================================================

** TOTAL **: 1.24x as fast 3920.8ms +/- 5.5% 3174.4ms +/- 0.8% significant

=============================================================================

3d: - 297.0ms +/- 9.3% 283.8ms +/- 1.7%
cube: *1.53x as slow* 73.6ms +/- 11.2% 112.6ms +/- 3.0% significant
morph: 1.91x as fast 138.6ms +/- 9.9% 72.6ms +/- 3.6% significant
raytrace: *1.16x as slow* 84.8ms +/- 9.1% 98.6ms +/- 2.9% significant

access: *2.17x as slow* 197.4ms +/- 7.0% 427.4ms +/- 1.2% significant
binary-trees: *6.69x as slow* 15.4ms +/- 9.2% 103.0ms +/- 2.3% significant
fannkuch: *3.65x as slow* 64.8ms +/- 8.1% 236.6ms +/- 1.3% significant
nbody: 1.20x as fast 68.4ms +/- 8.0% 57.0ms +/- 4.1% significant
nsieve: 1.58x as fast 48.8ms +/- 10.9% 30.8ms +/- 14.9% significant

bitops: 1.96x as fast 143.0ms +/- 7.7% 72.8ms +/- 4.7% significant
3bit-bits-in-byte: 3.93x as fast 11.0ms +/- 35.7% 2.8ms +/- 19.9% significant
bits-in-byte: 1.17x as fast 18.0ms +/- 10.9% 15.4ms +/- 4.4% significant
bitwise-and: 10.1x as fast 44.4ms +/- 5.8% 4.4ms +/- 15.5% significant
nsieve-bits: 1.39x as fast 69.6ms +/- 12.7% 50.2ms +/- 4.4% significant

controlflow: *17.0x as slow* 6.4ms +/- 10.6% 109.0ms +/- 1.6% significant
recursive: *17.0x as slow* 6.4ms +/- 10.6% 109.0ms +/- 1.6% significant

crypto: *1.21x as slow* 121.2ms +/- 6.0% 147.0ms +/- 1.6% significant
aes: *1.56x as slow* 43.8ms +/- 11.2% 68.2ms +/- 0.8% significant
md5: *1.37x as slow* 41.4ms +/- 9.2% 56.8ms +/- 2.9% significant
sha1: 1.64x as fast 36.0ms +/- 8.1% 22.0ms +/- 10.6% significant

date: 2.00x as fast 963.4ms +/- 5.7% 482.2ms +/- 2.2% significant
format-tofte: 1.37x as fast 385.4ms +/- 6.6% 280.6ms +/- 4.2% significant
format-xparb: 2.87x as fast 578.0ms +/- 5.3% 201.6ms +/- 2.5% significant

math: 1.62x as fast 278.0ms +/- 7.0% 171.4ms +/- 7.2% significant
cordic: 1.33x as fast 149.8ms +/- 8.7% 112.8ms +/- 3.6% significant
partial-sums: 2.19x as fast 91.4ms +/- 4.6% 41.8ms +/- 28.8% significant
spectral-norm: 2.19x as fast 36.8ms +/- 14.2% 16.8ms +/- 6.2% significant

regexp: 1.46x as fast 743.8ms +/- 7.4% 509.0ms +/- 5.0% significant
dna: 1.46x as fast 743.8ms +/- 7.4% 509.0ms +/- 5.0% significant

string: 1.20x as fast 1170.6ms +/- 6.2% 971.8ms +/- 0.8% significant
base64: 3.18x as fast 108.0ms +/- 21.3% 34.0ms +/- 4.5% significant
fasta: *1.52x as slow* 115.6ms +/- 5.9% 175.8ms +/- 1.8% significant
tagcloud: 1.07x as fast 250.4ms +/- 4.9% 233.6ms +/- 2.2% significant
unpack-code: 1.21x as fast 507.4ms +/- 5.7% 418.8ms +/- 2.9% significant
validate-input: 1.73x as fast 189.2ms +/- 6.0% 109.6ms +/- 3.9% significant

Links:

Minefield/TM

Chromium



The latest Chromium hourly has actually picked up speed. TraceMonkey shaved some points off as well and is still on top according to my tests on this machine.

[Cyberbeing]

Firefox 3.1 with JIT enabled is still faster overall in SunSpider when compared to Chrome on my 3 year old box.
AMD X2 4800+ @2.64Ghz, 2GB DDR433 2-3-3-6, WinXP SP3


Minefield Build 20081011094850 1347.4ms VS. Chromium 0.3.155.0 Build 3340 1844.0ms

Code: Select all

TEST                   COMPARISON            FROM                 TO             DETAILS

=============================================================================

** TOTAL **:           1.37x as fast     1844.0ms +/- 0.5%   1347.4ms +/- 1.2%     significant

=============================================================================

  3d:                  1.05x as fast      128.2ms +/- 1.4%    121.6ms +/- 1.2%     significant
    cube:              *1.69x as slow*     29.4ms +/- 2.3%     49.8ms +/- 1.1%     significant
    morph:             1.81x as fast       57.6ms +/- 3.6%     31.8ms +/- 3.3%     significant
    raytrace:          1.03x as fast       41.2ms +/- 1.3%     40.0ms +/- 2.2%     significant

  access:              *2.04x as slow*     92.8ms +/- 1.1%    189.0ms +/- 1.9%     significant
    binary-trees:      *7.79x as slow*      6.8ms +/- 8.2%     53.0ms +/- 1.7%     significant
    fannkuch:          *2.63x as slow*     33.4ms +/- 2.0%     88.0ms +/- 3.9%     significant
    nbody:             *1.17x as slow*     30.0ms +/- 0.0%     35.0ms +/- 2.5%     significant
    nsieve:            1.74x as fast       22.6ms +/- 3.0%     13.0ms +/- 11.7%    significant

  bitops:              1.83x as fast       70.0ms +/- 1.3%     38.2ms +/- 3.6%     significant
    3bit-bits-in-byte: 2.87x as fast        4.6ms +/- 14.8%     1.6ms +/- 42.6%    significant
    bits-in-byte:      1.02x as fast        9.0ms +/- 0.0%      8.8ms +/- 18.4%    significant
    bitwise-and:       7.00x as fast       26.6ms +/- 2.6%      3.8ms +/- 14.6%    significant
    nsieve-bits:       1.24x as fast       29.8ms +/- 3.5%     24.0ms +/- 0.0%     significant

  controlflow:         *19.5x as slow*      3.0ms +/- 0.0%     58.6ms +/- 1.2%     significant
    recursive:         *19.5x as slow*      3.0ms +/- 0.0%     58.6ms +/- 1.2%     significant

  crypto:              *1.10x as slow*     54.4ms +/- 4.2%     59.6ms +/- 2.4%     significant
    aes:               *1.48x as slow*     20.2ms +/- 5.1%     29.8ms +/- 3.5%     significant
    md5:               *1.15x as slow*     17.8ms +/- 5.8%     20.4ms +/- 3.3%     significant
    sha1:              1.74x as fast       16.4ms +/- 4.2%      9.4ms +/- 7.2%     significant

  date:                2.62x as fast      513.0ms +/- 0.5%    196.0ms +/- 0.6%     significant
    format-tofte:      1.86x as fast      196.6ms +/- 0.3%    105.8ms +/- 0.5%     significant
    format-xparb:      3.51x as fast      316.4ms +/- 0.8%     90.2ms +/- 1.5%     significant

  math:                1.79x as fast      110.0ms +/- 1.4%     61.6ms +/- 1.1%     significant
    cordic:            1.38x as fast       55.2ms +/- 1.9%     40.0ms +/- 0.0%     significant
    partial-sums:      2.54x as fast       36.0ms +/- 2.4%     14.2ms +/- 3.9%     significant
    spectral-norm:     2.54x as fast       18.8ms +/- 9.8%      7.4ms +/- 9.2%     significant

  regexp:              1.99x as fast      401.6ms +/- 0.3%    201.8ms +/- 3.8%     significant
    dna:               1.99x as fast      401.6ms +/- 0.3%    201.8ms +/- 3.8%     significant

  string:              1.12x as fast      471.0ms +/- 0.8%    421.0ms +/- 2.1%     significant
    base64:            3.16x as fast       48.0ms +/- 1.8%     15.2ms +/- 3.7%     significant
    fasta:             *1.44x as slow*     57.4ms +/- 1.9%     82.4ms +/- 1.3%     significant
    tagcloud:          1.12x as fast      111.4ms +/- 0.6%     99.4ms +/- 3.3%     significant
    unpack-code:       1.03x as fast      182.4ms +/- 1.1%    177.4ms +/- 3.4%     significant
    validate-input:    1.54x as fast       71.8ms +/- 2.6%     46.6ms +/- 10.3%    significant

(Note: Please remember to share your system specs whenever you post a benchmark score)

[UrbenLegend]

@Gobd. What's the difference between Firefox TM JIT chrome and content vs Firefox JIT chrome and content? I am guessing the nightly builds are without all of TM?

[teoli2003]

Firefox TM JIT chrome and content was from the tracemonkey development branch, the Firefox JIT chrome and content is from the mozilla-central integration branch. The TM dev branch was syncronized yesterday with mozilla-central. TM branch is where the TM dev happens.

The nightly builds from m-c have an about:config option to enable JIT, which may (but not today) lag one or two weeks behind the TM branch in tracemonkey development.