Mongo Boston: fast progress, with hitches in the cloud, Map/Reduce

By Andy Oram
September 21, 2010 | Comments: 1

I joined a couple hundred developers yesterday for a MongoDB conference, pacing through Microsoft's R&D Center side by side with staff engaged in Microsoft activities, like sets of Escher woodcut characters occupying parallel worlds.

Version 1.0 of MongoDB was introduce just over a year ago, and its popularity has shot up like a rocket over the past several months. This conference came just after the release of O'Reilly's MongoDB book, and co-author Michael Dirolf spoke. The talks fell into three main categories:

  • Overviews of particularly interesting features, which mostly were the replication sets and auto-sharding announced in August
  • Introductions to using MongoDB from various languages (C#, Java, PHP, Python, Ruby)
  • Discussions of MongoDB cloud services

Regarding clouds, a talk on MongoHQ drew an overflow audience, while a much smaller group came to hear about MongoDB on Azure. It's not surprising, because Azure doesn't really support MongoDB yet (you can run a single instance, but that's a rare configuration). The Azure situation is an interesting study in how design choices made in different systems keep them from working well together. The Azure team is sincere, I'm sure, about bridging the gap and will eventually get there, but the incompatibilities run pretty deep.

Most significantly, Azure does not let you choose an IP address for a service and doesn't let a service return to the same IP address when it restarts. This makes sense in cloud computing, where the whole design is based on non-persistence and on how to recover from it. But MongoDB replication depends on systems knowing each other's IP addresses. The current Azure solution is to create a special manager host that knows where all the servers are and directs requests to them--an extra layer of management that administrators won't find welcome.

The use of ports also lies outside server control, so you can't depend on the Web interface being available where MongoDB likes to put it (at a port 1000 higher than the port for its native interactions).

Upcoming features in MongoDB 1.8 and beyond were announced at the end. Map/Reduce, a valuable feature that MongoDB built in, but 10gen managers admit it doesn't offer as much speed-up as users expect. They are aiming for hundred-fold speed-ups in the future. At the same time, they plan to extract popular functions such as GROUP BY that currently are available through Map/Reduce but that should be easier to code.

Finer-grained locking will also be introduced. Currently, access to all collections are coordinated with one big lock. Soon MongoDB will have a separate lock for each collection, followed by intracollection locking.


You might also be interested in:

1 Comment

I was at the MongoDB Boston event yesterday, and it was quite excellent! The speakers totally knew their stuff. When asked difficult questions, they explained the answers, and were quite open about the costs and benefits (rather than just promoting their own stuff). This is precisely what I was hoping for. I now have immense respect for 10gen, and MongoDB looks like a great product. I intended to learn a lot more about it.

News Topics

Recommended for You

Got a Question?