It is generic, but the graphs on arewefastyet.com suggest that it isn't actually making most regular benchmarks go faster, just some of the asm.js-style ones.
All the graphs are missing the keys to identify which coloured line represents which machine/setup. And the y-axes aren't labelled, so you can't tell whether higher is better or not :(
V8 supports asm.js as any other ES3-compliant JavaScript engine. It also has many optimizations in place which benefit from asm.js-style code. Asm.js-style code also simply doesn't do those expensive JavaScript-y things. That's why that Citadel demo runs just fine in Chrome, too.
However, V8 doesn't have a separate mode. The "'use asm';" pragma is ignored.
The Octane benchmark suite does contain Emscripten and Mandreel benchmarks though. So, they are tracking it and they also try to make this kind of code run faster.
Not only that, a few months after asm.js was included in Firefox, Chrome had optimizations done that helped a great bit with asm.js (http://blog.chromium.org/2013/05/chrome-28-beta-more-immersi...). Possibly a coincidence, but they certainly seem to care about asm.js performance above and beyond generic JS performance, even if they're not going to support the pragma.
> but they certainly seem to care about asm.js performance above and beyond generic JS performance
That doesn't follow. They certainly care about it; it's now in the Octane benchmark suite, which means they think it's representative of code that should be optimized for, but it's only part of the suite, which means it's just one more style of code that should be able to run efficiently in a modern JS engine.
You're oth saying the same thing for the most part, I dont think he's trying to imply that its their focus, just that its important enough to optimise specifically.
Nobody will see any benefits from JSCore being "faster than V8 on asm.js".
In fact, if JSC is slower on non asm.js code, the comparison would also be extremely misleading too, because 99.9999% of web code is NOT asm.js.
At the moment it's like saying "JSC is faster than v8 on a synthetic benchmark almost nobody really uses in the actual world, and v8 hasn't bothered optimizing for".
Obviously, people using asm.js will benefit from JavaScriptCore being faster than V8 on asm.js.
Whether it matters is an interesting question, but V8 developers seem to disagree with you, because they included asm.js benchmark in Octane benchmark suite, which V8 developers use to tune V8.
>Obviously, people using asm.js will benefit from JavaScriptCore being faster than V8 on asm.js
Also obviously I covered that, when I wrote "99.9999% of web code is NOT asm.js". So who are these "people"? All 10 of them?
>Whether it matters is an interesting question, but V8 developers seem to disagree with you, because they included asm.js benchmark in Octane benchmark suite, which V8 developers use to tune V8.
If anything, it shows that they agree with me. See how I compared asm.js to a "synthetic benchmark" in my comment?
V8 doesn't have any specific optimizations for asm.js, opting instead to focus on optimizing JavaScript as a whole (which asm.js also benefits from). That's their choice, and this is the result of that.
Because nobody (for large values of "nobody") cares for asm.js. 99.999% of Javascript code out there is not asm.js, and nobody really uses it at the moment for production/business. Some PoC games mainly.
So the fact that it beats V8 on something:
a) V8 does not implement,
b) requires specifically written (actually "compiled" from things like emscripten) JS code,
c) is irrelevant at the moment for JS performance on the web,
> Because nobody (for large values of "nobody") cares for asm.js.
The most important games companies - Unity, Epic, etc. - care very much about asm.js. Unity is going into beta very soon. Expect to see games after that.
> So the fact that it beats V8 on something: a) V8 does not implement
jsc also does not "implement" asm.js, in the sense that both v8 and jsc do not detect asm.js code specifically using the asm.js type system. Yet, jsc beats v8.
asm.js code is going to matter because games matter. Those games are using plugins now, but plugins are going away, and asm.js is the best option after that - that's why the games industry is adopting it.
Games are not "using plugins" today. They are either shipped as console games on DVDs/BluRays, or as native apps you download and run either for PC/Mac/tablet OS's.
asm.js is a bizarre hack. You're sending someone assembly code for a CPU that doesn't exist, as text.
It would have been far superior to just figure out CPU family specific sandboxes as NaCL did, and then accept that only two CPU architectures matter these days: x86 and ARM. If you can sandbox both, you're done and if someone wants to invent some super-duper new ISA they can damn well make code easily sandboxed when they design it.
I don't think that's a fair way to describe things - for one thing, NaCl is not portable, and PNaCl (which is) took years to materialize, at a point in time when it is not clear which option is better between PNaCl and JS.
Maybe irrelevant today, but maybe not irrelevant tomorrow.
JavaScript is what the web "runs on", and if Firefox might in the future provide the fastest way to run JavaScript. Obviously for most cases that isn't going to be that relevant, but I can imagine a situation where you have Photoshop-in-the-browser and it runs better on Firefox due asm.js.
If you care a lot about 99.9999% of web code, as in code found on real webpages, try JSBench <http://jsbench.cs.purdue.edu>. It's the only JS benchmark made from JS found in actual real-world production webpages. JavaScriptCore clocks all the other JS engines on it by a ridiculous margin.
It's neat that V8 is fast on long-running code that's not asm.js, but it's wrong to extrapolate that to most real-world web code.
ASM enables developers to run code at an order-of-magnitude as fast as native. This in-and-of-itself is important. While 99.999% of JavaScript may not be transcompiled from LLVM, that doesn't mean it isn't a powerful tool.
Apple may not care about such use cases. The fact that they could get ASM speedups for free is pretty cool. : )
Does anyone have numbers handy on the amount of size linking LLVM would add to a browser? Installing llvm-3.2-dev gives me over 75mb of static libraries, but I don't know how many of those would need to be shipped to support such a JIT.
It may be that the LLVM core parts are already included in the OSX distribution, at least it was used a while ago for dynamic code generation in OpenGL (http://lists.cs.uiuc.edu/pipermail/llvmdev/2006-August/00649...), pure speculation on my side though.
To have a meaningful "victory", you want to have one of those:
1. A generic JS JIT that runs generic JS faster than another generic JS JIT.
2. ASM.JS JIT that runs ASM.JS faster than another ASM.JS JIT.
What we have here is something else:
3. A generic JS JIT that runs ASM.JS faster than another generic JS JIT.
If people are not familiar with ASM.JS, the somewhat misleading (to the uninitiated) headline sounds more interesting than it is actually.
It's not a significant milestone, as SpiderMonkey's ASM.JS implementation outperforms this one, and V8 isn't implementing ASM.JS in any way. As in at all. So of course it'll be slower.
The purpose of ASM.JS is to allow a separate ASM.JS runtime to run ASM.JS code. The reason it happens to be a JS subset, is so you can have a fallback for backwards compatibility.
Not to say this LLVM engine doesn't show potential. But implementing ASM.JS in LLVM is not the hard part of implementing a generic JS JIT.
ASM.JS is entirely static, so it's comparatively trivial to JIT, it's literally an ASM to ASM transpiler.
Edit: the particular quote I objected to was edited out by parent.
I object to "all the dynamicness of the rest of the JS is what V8 excels at". V8 excels at a subset of dynamic JavaScript (namely those benefit from Crankshaft) and is not very good outside of the subset.
Typescript benchmark included in Octane benchmark suite is a good example. In Google's own words: "This test measures the time TypeScript takes to compile itself and is a proxy of how well a VM handles complex and sizable Javascript applications".
In particular, Crankshaft is not effective for this benchmark. (Check yourself by running it with Crankshaft disabled: the score actually improves because you save compilation effort.) And as you can check yourself or check on AreWeFastYet, JavaScriptCore beats V8 by 2x on this benchmark.
I admit Crankshaft subset of JavaScript is large and interesting (certainly compared to asm.js subset) but it is not true that V8 excels at dynamic JavaScript, for sufficiently large value of dynamic. JavaScriptCore does.
TypeScript benchmark is one of the Octane benchmarks that does not do warm up, so Crankshaft does not have chance to fully shine [which is totally fair given how TypeScript compiler is used: you don't warm it up, you just spawn it again and again]. It also has an extremely flat profile. So you can't simply say that "Crankshaft is not effective for it". Though potentially you can say "benefits of Crankshafting do not compensate for the cost of Crankshafting". If you enable warmup for it then score jumps by a factor of 2 for V8.
JSC might be warming faster or simply have a better cost / benefit ratio at lower tiers. I doubt FTL kicks at all for this benchmark given how short running it is. V8 also deopts quite a bit on it, so it might be some pathological case... This is definitely something worth exploring and fixing on V8 side.
> V8 excels at a subset of dynamic JavaScript (namely those benefit from Crankshaft) and is not very good outside of the subset
As I understand it Crankshaft's claim to fame is the runtime type specialization, which does well if the code is hot and ends up using same types in all cases. Is this what you are referring to?
I don't think there is a "victory" here. Just an important milestone for Safari/JavaScriptCore. They have done some amazing engineering here!
In the broader context, this shows that the JSC approach of taking LLVM and using it as a JIT - called FTL - can beat Chrome/v8 on large long-running computational code. That means that LLVM is significantly better than v8's backend JIT at some things. This is not too surprising since LLVM is much larger and sophisticated - it has a more powerful register allocator, supports things like float32, etc.
Those two examples weren't random - Firefox has added an LLVM-like register allocator and float32 support (not just for asm.js code, in general). So there are two paths here: Safari takes LLVM, and Firefox implements some bits from LLVM in its own JIT. v8 so far has done neither, but hopefully has plans to do one of the two, or some other cool thing.
In addition there is the issue of asm.js. Safari doesn't detect it, and beats Chrome but still trails Firefox. But it's far from clear that is the last word - Safari's new JIT just landed, it can be tuned a lot more, presumably. So it might still end up showing that a general JIT can beat custom asm.js optimizations, time will tell.
What is clear though is that the Safari's FTL JIT took far more work than Firefox's asm.js optimizations. If the LLVM-based JIT mostly helps asm.js code anyhow - it shows little benefits in other benchmarks so far - then that raises interesting questions (if the benefit is mostly for asm.js, why not focus on asm.js specifically, if it's simpler and more effective?). But again, FTL just landed, perhaps with more tuning it will show general benefits.
IIRC, V8 had implemented optimizations that specifically target asm.js code patterns in the past (the performance gap for asm.js code between FF and Chrome definitely got a lot smaller in 2013), they just don't rely on the "use asm" keyword or do AOT compilation like Odinmonkey.
Your comment contradicts itself: "Asm.js is a JS subset" and "V8 isn't implementing Asm.js in any way.".
It is fully implemented, of course, it is only the "use asm.js" directive that is not. That prevents some control to signal non asm.js code in a JS file, and only that.
V8 doesn't explicitly support asm.js so it's easier to beat than Firefox.