The Ever-Dynamic John Lam on Iron Ruby, Open Source and Microsoft

By James Turner
September 29, 2008

James Turner: James Turner with O'Reilly News here at OSCON 2008; I'm talking to John Lam who is the Iron Ruby Guy for Microsoft or as his badge says, the Evil Empire for Redmond, so clearly you're taking things with a little bit of a wry sense of humor there?

John Lam: That's the thing; if you don't have a sense of humor --what do you got in life? So you got to, especially when you get a chance to talk to people it's a lot about trying to approach things on a less confrontational way and that kind of stuff and making fun of yourself is a great way of doing it.

James Turner: Well you certainly are in a role where you're bridging two groups of people who traditionally have been a little bit adversarial--

John Lam: Sure.

James Turner: --to each other.

John Lam: Yeah; on both ways, yeah.

James Turner: Yeah; so tell me how you got to Microsoft and--.

book coverbook coverbook cover
For a complete list of Ruby books, visit the
Ruby Topic page in the O'Reilly Store.

John Lam: Sure; so the story was kind of fun. To kind of cut it so that the stories over like the last three years, I just--I gave a talk at Ruby Fringe last week where I kind of covered the longer talk, which was--or the longer history which is actually a little bit more interesting but the short history is I built an Open Source bridge between the Ruby Interpreter, the existing Matz's Ruby Interpreter MRI and the CLR back--started working on that maybe about three years ago--three and a half years ago. And as part of my promotional tour for this thing I went around and spoke at a bunch of conferences, including this one here at OSCON back in I think it was '05 and I really wanted to kind of present my work to the world because it was cool. It lets you use .net objects and Ruby objects side-by-side inside of the same process. So for all intents and purposes a .net object would look and feel and smell and taste just like a Ruby object. And so that work was done here. That was--at the same time this is when Iron Python was going 1.0 here at the conference and so I knew Jim Hugunin from a long time ago; he was the guy who created Iron Python and he works for Microsoft as well now. And so what happened was while they took me in the backroom, they shown the bright lights in me, it was a little bit of mind-conditioning and got me to at least go for an interview with the Microsoft in a few months and I did that. And they made me an offer I couldn't refuse which was an awful lot of fun.

I got a chance to go off and build a Ruby implementation from scratch and build a team from scratch, build it as an Open Source project, and also had this opportunity to create this platform for building Ruby applications on top which is this thing we call Silverlight today. Back then it was called a crazy bunch of code names. And the fascinating thing about that was it was an opportunity to build a truly cross-platform version of Ruby in a sense that Ruby on Windows is kind of treated as a second-class citizen, but through the Silverlight platform we have the ability to run Windows as well as on Mac as well as on Linux via the Moonlight Project. So we have this consistent platform everywhere where folks go off and build Ruby applications will have this opportunity to go off and use these interesting scenarios.

James Turner: Why don't you talk a little bit about the CLR and exactly what that is and what it brings because for a lot of people I think they hear bytecode, they hear .net, they hear Mono and their eyes glaze?

John Lam: So the CLR is just like the JVM, since people certainly know an awful lot more about Java in the Open Source world. It's a run time; it is a virtual machine. It is--offers a lot of the same kinds of services that you'll find on the JVM. There's a JCompiler ; there's a byte-code verifier, what we call IL and it's a fast platform essentially where it tries to abstract away details of the underlying platform. It also presents a bunch of services that make it easier for people to write applications like garbage collection.

James Turner: And a technology like Iron Ruby, how does that leverage the CLR and what benefits does that bring to Ruby developers?

John Lam: So what's interesting is that designing a language or building and implementing a language is actually the easy part. The much more difficult part is getting it to be secure, getting it to run in multiple places, putting in things like a world-class garbage collector or Just In Time Compiler; those are things that take an awful lot of work and it seems like every single programming language reinvents some kind of virtual machine environment for them to go off and do these things. Ruby is no different; so today there's two acronymed implementations of Ruby that are kind of like the original Ruby. MRI which is Matz's Ruby implementation and KRI which is Koichi's Ruby implementation; KRI is otherwise known as Ruby 1.9 where Koichi created his own VM and just like all the other people who have gone before him, they've gone off and created their own VM and VM(s) as it turns out are all very, very difficult tasks. Not to say that I don't think that they're doing is awesome, but the reality is there's an awful lot of engineering work that has to go into building a Virtual Machine, in the order of thousand of person years, certainly in the mature Virtual Machines we have in the world like a JVM or the CLR. And it's just not realistic for a small two--three man team like what Koichi has there to go off and build an implementation of the same robustness and performance. So really what Iron Ruby does is--so there's actually another piece of technology that my team builds in Microsoft and it's this thing that we call the Dynamic Language Run Time or the DLR.

The DLR is --it started off originally as the Python Language Run Time, so the Iron Python was this implementation of Python that was built from scratch on top of the CLR. And building it from scratch turned out to be harder than it should have been--now not harder in that oh my god it was impossible; harder in the sense that in order to build a high performance implementation you had to make lots and lots and lots of little decisions and make them all correctly. And as it turns out if you make that set of things--those sets of decisions correctly you'll wind up with a very fast implementation. So certainly in terms of through-put, Iron Python is the fastest implementation of Python today. So given that oh gee it was awesome that we could do this but gosh it was really hard to do this, we thought that it would be a really great idea to abstract out the hard stuff, things like Dynamic Method Dispatch into a set of libraries that could be used by other compiler implementers so that--that was the idea around the DLR. To vet the idea, we decided to go off and build three other languages outside of Python, so we built an internal prototype of, a version of Visual Basic we called a VBX; we built an implementation of JavaScript and we also built Iron Ruby; all right so those were the three projects that built on top of the DLR. And effectively what that happened was is it kind of forced all the Pythonisms out of the DLR since it originally really started off as a Python language.

James Turner: Well that's what happens when you factor the entire little nitchy things come out into the calling.

John Lam: Yes; yes. So --so that was a lot of the things that we wound up doing and it was I think that we're vetting the idea quite well. The DLR is actually going to wind up being part of shipping Microsoft stuff, so in a sense that DLR is going to ship with Windows and sometime in the future it's going to ship with our tools and our platforms and that kind of stuff so that any one--it effectively becomes just like the CLR, so folks that want to go build a language can expect on it being available and deployed very widely.

James Turner: One of the concerns that people had a lot when Java was first implemented by Microsoft was oh they're going to poison it in a way that it will no longer be a compatible language and I think there's similar concerns with Iron Python and Iron Ruby that they're going--and going to end up with non-portable programs.

John Lam: I'm also trying to come up with a pitchy quote around talk is cheap and certainly anyone can go off and talk about things and say oh my god. Well this could happen and that can happen. The only way you can judge us and the way--the only way anyone can judge us is based on what our actions are as opposed to what we say and what other people say. So if you look at the actions of Iron Python and you look inside of the Python community at large, Iron Python is a very well-respected implementation of Python and nobody has seen any evidence of all of some G# thing or something else. So and it's the same thing with Iron Ruby; Iron Ruby had exactly the same set of doubts in the beginning. I tend to clip out those old articles and stuff. That's a good motivational thing for the team. A lot of people doubted us but as I said; what you have to do is you have to go off and you got to develop thick skin and you got to go off and do your work and you got to show what you're doing and you got to show up again and you got to show up again and you got to show up again. And you got to go show your progress and the rest of that? Anything I say I wouldn't expect anybody to believe me outside of like judge us based on where we are right now. And judge us based on our kind of passages even.

If you look at what's going on in the Ruby community there are today seven different funded implementations of Ruby. And most of them have corporate backing of some sort and --so there's already this called balkanization of Ruby that's happening in the sense that each one of these things has the potential to go off and veer off in some direction. Now some of these things are due to the fact that like a lot of other Open Source languages and scripting languages in particular--not just Open Source; that's too broad of a category--scripting languages generally don't have a specification, . Java Script is the exception to the rule where there is actually the specification for how it's supposed to behaving and the only things that really matter to implementers which are in the weird corner cases where the features combine together.

James Turner: Right; like in Perl 6, Perl 6 is everything that fit into the bucket.
John Lam: Sure; and so we're--so right now there's rumors of a standardization process for Ruby ticking off out of Japan and going to ISO and that's--Matz has said that he would like to see this happen. That's awesome; but that's years away as anyone that's ever done any standards work knows. So in the short-term what we have in the community is a projected called the Ruby Spec Project, which grew up out of one of the seven implementations called Rubinius and so the Ruby Spec Project is an attempt by the community to go off and find a set of unit tests that define what is and what isn't Ruby.
James Turner: That's like the Java standard; I don't remember what they call it but they've got a standard test suite you have to run--

John Lam: Yeah.

James Turner: --in order to get the cert.

John Lam: Yeah; and there's--unfortunately there's not going to be a certification--I don't know. I don't know what's going to happen long-term around this stuff.

James Turner: But the same kind of idea--if it walks like a duck and talks like a duck it's a duck.

John Lam: Yeah; exactly.

James Turner: So it's interesting to watch what seems to be a transition inside of Microsoft from the, evil cancer days of Open Source and Microsoft to what we see today where you got Sarah Ford over at the Open Source Lab and Iron Python and Iron Ruby, and what seems to be a much more, I don't know if accepting--maybe accepting is the right word--attitude by Microsoft toward--or maybe resigned--toward the fact that Open Source is going to be a part of the landscape. What is the feel that you get as kind of an Open Source(y) guy inside of the corporation?

John Lam: So there are several parts of that--that I find really interesting about that question. So one of them is the fact that the reality is for all of our enterprise customers --enterprise being large organizations, Open Source is a fact of life there. The Fortune 500 virtually every single last one of those guys have some amount of Open Source inside of their IT infrastructure. So certainly the things that we have to do with respect to interop, playing nice with the other things are just things that help our customers do better and then ultimately keep them as customers. So there's this thing where we certainly have to do this interoperability thing. The second thing really is--is that if you think about programming languages, IDEs to a certain extent or to perhaps even a larger extent these days in frameworks, almost all of that stuff is being commoditized; you will find it's very, very hard to go off and find say--let's take programming languages--somebody selling a compiler today, .

James Turner: Right.

John Lam: I'm sure there are a few and each year--

James Turner: I think Sun was the last one to really try to make a living off the C Compiler and gcc ate their lunch.

John Lam: Yeah; almost every one--we still technically make money off of our c++ but there's so many different ways that you can get it for free that, somebody is going to pay for it but most people who care to can get the thing for free. And so the reality now is since it's commoditized generally, things like frameworks and languages; well how do you monetize this stuff, ? Some people have service models, but we're not a services company really; we're a platforms company, so the question that you generally have to ask is--does this thing that we are Open Sourcing somehow contribute value or increase the value on some underlying platform that we make money off of? And that's just the way we do business. And if that's--so at least where I sit at Microsoft in the Developer Division that's a really easy thing to make the argument for since virtually everything we do at Developer Division is all about our underlying platforms. So for our folks at other parts of the company it's a very different set of decisions like things like you're not going to see Windows or Office Open Sourcing for that same set of reasons. The auto platform we make money off of.

James Turner: Right; it's the core competency.

John Lam: Right; yeah.

James Turner: I'm not going to make any jokes. But I've always thought it was kind of ironic that you had this very negative toward Open Source for a while at Microsoft because practically from Windows 3.1 as soon as they put the Winsock Stack in you had Berkeley traceroute in there and all these other Open Source that were allowed to be because they were under a Berkeley license technology that were in Windows. So it always seemed a little weird to me.

John Lam: Well and in some ways I also find it interesting in the sense that a lot of programmers believe they understand intellectual property , and I think the bigger--my part-time job at Microsoft is understanding intellectual property because so much of what I do is all around dealing with lawyers and dealing with stuff to get our stuff out there. And I understand--so there are a perfectly rational and reasonable set of arguments that kind of define our position around intellectual property, things that you'll hear people talking about and there are perfectly sound and rational arguments of why it doesn't make sense at all. Like there are two sides to this thing and I think what we're seeing is we're seeing an opportunity where we're going to push the ball a little bit harder each time or a little bit higher each time. We're going to look around after taking that step and say did that work? Was there some positive benefit by doing that? If so yeah, great; that's awesome. The company is still around and still making money--check; that looks good. So if we are able to continually push the envelope on a lot of this stuff and we have certainly come an awful long way if you kind of look at where we came from originally we'll see more and more of this stuff coming out of the company in the future.

James Turner: So apart from being an Open Source guy in what was until recently very commercial software space--

John Lam: We are still a very commercial--

James Turner: Yeah; less than it was--uh-huh. What's Microsoft like these days?

John Lam: It's an awesome--so I'm a guy that--well I'm 40 years-old; I've got two kids, wife, mortgage, dog all that kind of stuff and and for a guy like me, Microsoft is an awesome place to work . It's--certainly have phenomenal benefits for the family; Seattle--we live in Redmond--is a stunningly beautiful place to live. As a company for me personally it's been an awesome experience. I also get to go to work every day with some really, really smart people. My team is fantastic, easily the best set of developers I've ever worked with--period. So that kind of, experience where you get to go to work each day and work with awesome people and do cool things --what more could you really ask for?

James Turner: Is Iron Ruby still very open to community contributions or--?

John Lam: We've--since last year when we announced that we were accepting our community contributions here at OSCON; we've gotten community contributions. We're as open as anyone wants us to be. Folks just simply need to go off and submit a patch to our thing and we'll review it. If it's good we'll check it in.

James Turner: All right; well John it's been great talking to you and thanks for some insight into Open Source inside of Microsoft or the Evil Empire as your badge says. We've been talking to John Lam, who is the Iron Ruby guy at Microsoft and here at OSCON 2008.

You might also be interested in:

News Topics

Recommended for You

Got a Question?