Two different approaches are emerging in this effort. The first, proposed in a paper called Incremental Dynamic Code Generation with Trace Trees by Andreas Gal and Michael Franz of the University of California, Irvine, establishes how trace trees can be used to locate and compile looped content within an interpreted language as the program is running, in essence compiling and optimizing the program in real time. This algorithm (with some variations to handle edge cases not initally covered by Gal and Franz) is now being used within the new Trace Monkey application as part of Mozilla's Tamarin, now being incorporated into the current Mozilla 3.1 builds.
The second approach, developed by the V8 team for Google's new Chrome browser, differs in that it performs a compilation process prior to execution, then optimizes the compiled code with hooks for evaluations and imported code. There are some indications that it may in fact utilize a variation of the Trace Trees algorithm as well.
In both cases, however, the results even now are fairly stunning. For instance, when TraceMonkey enabled code was tested against the Sun Spider uBench suite, it ran 22.5 times faster than the equivalent pre Trace-Tree spider monkey code. Image manipulation code ran 6.5 times faster, matrix manipulation 6.3 times. These aren't percentage changes compared to existing code - this code is easily an order of magnitude faster across the board. Google's V8 code is testing at similar levels (indeed, the irony here is that the speed improvements for both systems are now within a few percentage of one another).
Just In Time Compilation
Put another way, in most cases, if a programmer initially assigns a string to a variable, it is unlikely, though not completely unheard of, for that programmer to reassign the variable as a number or date. Even with numbers, it's generally fairly easy to determine whether or not a variable is in fact an integer or a floating point number, and to use that information to set the variable to that type explicitly within compiled code - until some evidence comes along to disprove that assertion.
Many of the fundamental precepts of compiler design are rooted in philosophies that emerged at a time when computers themselves were considerably slower, and hence less much less sophisticated. They used a very deterministic model where the worst case scenarios about type and performance were the ones coded to, unless the programmer specifically established optimization hints (such as explicitly declaring type). The idea that you can do performance enhancements in optimizing code by using stochastic methods - looking at the most likely scenarios rather than the worst case ones - were in general considered to be expensive and inefficient, as it required that the interpreter itself run optimizations on the fly (with the associated performance hits and memory overhead).
Recently, however, this attitude has been changing. Emerging "interpreters" are now keeping track of how often a given set of instructions is called or how often the implicit type of a variable gets changed, and when it becomes evident that a given pattern is in fact emerging, it will then attempt to optimize the sequence in question or data assignment being given as compiled code.
The sequences being watched in the Mozilla case are what are called "trace trees", where a tree in this case indicates a directional graph (or block of commands) within a loop. This works fine if the sequence in question is just one set of instructions after another, but if a conditional expression (an if-then-else statement) for instance occurs and both true and false conditions for that expression are given, the trace tree is essentially rebuilt with the new assumptions in place, and the trace engine goes back to watching the process. Similarly, if a variable does get reassigned to a different data type on the fly, the engine rebuilds the appropriate state trace with the assumption being that the variable in question is of a more generalized type.
You might expect that this process of tearing down and rebuilding trace trees would prove hideously slow and have a major impact upon performance, but as it turns out this is where the stochastic nature of programming comes into play. Such events are comparative outliers - a trace tree compiler may have to perform such actions nine or ten times over the course of running the script, but each time it can take advantage of the existing optimizations to make the process faster and more streamlined.
The trace tree approach works especially well with loops within loops, in essence by optimizing upon the inner loop while simultaneously holding enough state in the outer loop to switch to it as an optimization path if necessary. This technique is also used in order to better support recursive calls, which can be computationally expensive.
The trace tree engine also keeps track of potentially overlong or complex trees, and when such a condition emerges, the engine "unwinds" the tree at that stage and rebuilds it with the current state conditions. This keeps the trace trees manageable in memory, at the cost of a fairly small performance hit. Finally, the technique is also able to identify those particular programmatic constructs that just shouldn't be optimized, and keeps those blocks separate - they run at the interpreted speed, not the compiled speed.
Most of the next generation of compilers are far better at being able to track and trace such closure variables, and during garbage collection are usually much better at identifying these variables when it is time to clean up the allocated memory. Not only does this mean that the applications themselves take up somewhat less memory (and are less likely to crash due to a memory overrun) but it also insures that it is easier to create larger contiguous memory blocks, reducing the amount of overhead necessary to process data held within these blocks.
Building Web Super Apps
One upshot of this is that there should be between a four and sixfold increase in the graphics API speed, both for raster and vector operations. While this likely won't end up putting Firefox at a point where it can replace Adobe Photoshop for sheer processing power, it is very likely to give Adobe Flash a run for its money (though to make matters even more interesting, Adobe is also looking at incorporating many of the same algorithms into their ActionScript engine).
Finally, all of these things should be seen in the context of other innovations going on in all of these browsers, especially the incorporation of offline databases such as Google Gears and Firefox's SQLLite database. This makes it possible to persist information between sessions more effectively than can be handled with cookies and makes it possible to offload potentially large data structures safely.