LuaJIT 2.0 is an amazing achievement. On the programming language shootout's "flawed benchmarks", LuaJIT outperforms almost every other dynamic language (including JavaScript V8, Python, Ruby 1.9, JRuby, VisualWorks Smalltalk, CPython, PLT Scheme), and is competetive on many benchmarks with highly-optimized language implementations like Sun's JVM, and GHC:
In my experience, the 1.x releases of LuaJIT have typically made whole programs (not just benchmarks) run 2-3+x faster than standard Lua. Without any other changes. :)
I haven't tried LuaJIT 2 yet, but I'm eagerly waiting for it to get ported to amd64.
Most of the VM is written in C. Only the core of the interpreter is written in assembler (generates ~13K of machine code).
And the port of the interpreter to x64 is already done, so it was certainly practical. Porting the rest of the VM and the JIT compiler is what takes most of the time.
I was abusing a footnote package, sorting footnotes alphabetically per page, and that took about a second a page on my laptop, but for everything else disk seeks take the majority of TeX's time; once you LaTeX something, and cache the font files in memory, it usually takes under a second or two for anything under 50 pages.
Our Lua usage isn't too widespread at the moment; it's really one infrastructure project in particular that uses Lua to allow user-defined functions to run within a tightly controlled container. Lua was the best choice, because of its low overhead, fast execution, and the ability to set limits on execution time.
Unfortunately I cannot be more specific, since the project is not public.
Interesting to read this. I thought Google only used C and JVM-based languages as a company policy. Perhaps that's still the case except for this one example?
That's something I've also been wondering lately, since the Kepler and Orbit projects haven't seen activity for ages (at least according to their webpages).
I found this Sinatra clone for Lua called Mercury (http://github.com/nrk/mercury), but it lacks documentation and does not seem to be very mature at the moment.
LuaForge's ftp server is broken since the big LuaForge crash a few months ago, so those pages are badly out of date. Browse http://github.com/keplerproject for more recent information.
Ruby might just need a better implementation with more time spent on optimization. http://maglev.gemstone.com/ is going in this direction.
But also, Lua was always designed with cleanness and simplicity of implementation in mind. Its developers have strong backgrounds in both theory and engineering, which you can see in their interview in the book Masterminds of Programming. They've been very conservative about the design of the language. In contrast, Ruby feels much messier and more "evolved" - look at its Yacc grammar[1], for example, compared to the grammar of JavaScript or Lua[2]. Even the Ruby lexer is full of weird edge cases. Much of MRI is like that, and for years it was the only "spec" for the language; now its decisions have been reverse-engineered and inherited by all other implementations.
[By the way, I've used Ruby for years and like it at lot, while I've written only a few lines of Lua code and like it but am not deeply familiar with it.]
Also, Ruby has much more complicated rules for method dispatch. Lua just has functions, and if you want to implement any more complicated dispatch, you have to do it yourself. Once I implemented Ruby-like dispatch on top of Lua, and it was terribly slow:
If you want to learn Lua, Ierusalimschy's _Programming in Lua_ (http://www.inf.puc-rio.br/~roberto/pil2/) is by far the best book. Lua is one of my favorite languages. While it's good on its own, you'll get a lot more out of it if you're comfortable with C. Lua+C is an excellent combination.
Just chiming in as another Lua fan; I really like it because it's like the Scheme of imperative scripting languages. And while we're passing around links to papers, I've found the following the be good for additional insight:
It'd be interesting to see how fast Ruby would run "cached" (i.e. with the AST already assembled in memory,) to see how much of the time is really spent in the parsing and lexing phases.
Note that I'm not claiming Ruby's parser and lexer are slow, or that they're the cause of any greater problems. I'm just picking on them as one example of deeper differences between the Ruby and Lua designs/implementations.
Ruby might just need a better implementation with more time spent on optimization...But also, Lua was always designed with cleanness and simplicity of implementation in mind
Ease of implementation has a multiplier effect with Open development effort. Not only do you get more of "more eyeballs," new contributors can ramp up faster. There tends to be a better effort/ fun ratio.
Even the Ruby lexer is full of weird edge cases.
Actually, the lexer sets state that changes the operation of the parser and vice versa.
Some of Ruby's more excentric semantics are really hard to optimize away by a compiler. It takes a much greater effort to compile Ruby source code to machine code of similar quality than for Lua. Since time and money are not infinite resources, especially for compiler development, you'll get worse results in practice.
> Some of Ruby's more excentric semantics are really hard to optimize away by a compiler.
Any examples in particular? Besides name resolution and method dispatch (and callcc, but I think that may have even been removed from Ruby 1.9) none of Ruby's semantics seem all that different from Lua. Am I missing something?
Read it, you will find it interesting. The guy also wrote a small language called Duby which used Ruby like syntax but is much more faster, because he didn't include any of those slowing down features.
Mostly resolution and dispatch. Some of this needs very clever caching or the overhead can't be eliminated or hoisted. The control-flow constructs and iterators come to mind, too.
The Rubyists certainly know about these issues, so you'll have to ask them for details.
There's also the over-featured thread implementation (especially before 1.9)
Dispatch is worse than you realize due to the hackish optimizations made in MRI: if you monkeypatch a natively-method from Ruby, other native code will call the original by default.
Ruby was just designed and implemented with all the attention paid to sugary syntax/semantics and very little thought given to how well it could be implemented. MRI is a really poor implementation of a (formerly) poorly-specified, highly irregular, unorthogonal language.
MRI's garbage collector is abysmal, use Phusion's REE fork to get one that at least doesn't mark every single object on every collection and actually bothers to perform them in normal situations. It didn't do any desugaring at all until 1.9, and YARV only has bytecodes for the most primitive accessors, assignments, stackops, jump/condjump, and returns; with optimizations for -, +, <. <<, and some regexes. Your normal code just gets passed through verbatim.
Every once in a while I'll be reading docs or sitting in irb and suddenly scream at the ceiling, spin in my chair, and get the urge to throttle a Japanese person.
No need to wonder, have a look at the source code for them both and it will become clear exactly why there is such a performance difference. Lua gives you raw C numeric types which makes a big difference right there. Other people have mentioned a few other good reasons too (no costly method dispatch).
I haven't looked at LuaJIT yet but considering how small it is I'm sure it meets or exceeds the quality of the code behind Lua.
Standard Lua (the one from PUC-Rio) doesn't really, it uses boxed numeric values, as I imagine ruby (and almost every other dynamic language) does as well. "Boxed" meaning that each number is stored on the heap with some kind of tag, and when you pass such a value to a function you're really passing a pointer to it and that indirection costs cycles and puts pressure on the cache. LuaJIT's most significant (IMO) feature is that it stores numeric values directly, like C does, so when you call a function the actual bits representing it go right on the stack, not the address of where the bits can be found, and it uses some cool tricks of floating point representation to prevent the garbage collector from trying to follow these stack values as if they were pointers. This (combined with the trace optimization) is why when you write a numeric for-loop in LuaJIT it runs (nearly?) as fast a C.
Lua numbers are not boxed in the usual sense; they live in the Lua data stack (which leaves in the C heap, but this is not isomorphic to the Lua heap), are not accessed through a pointer, and do not put pressure on Lua's memory allocator and garbage collector.
http://shootout.alioth.debian.org/u32q/which-programming-lan...
Mike Pall wrote LuaJIT and now LuaJIT 2 as essentially solo efforts. Thanks to Google and the other sponsors for helping him continue!