Managing Open Source Complexity

By Eric Larson
November 19, 2008 | Comments: 1

Sean McGrath wrote a great article on the struggle for Open Source to manage complexity. It's a great insight because much of programming is managing complexity. This requirement has fostered "high level" languages and continues to be powerful concept that pushes the state of the art.

Open Source does have one tool it uses to manage complexity and that is competition. For a very long time, Apache was essentially synonymous with "web server". Now we have Lighttpd, Nginx and Cherokee. All these projects have their reasons for existing, but I'm sure the fact they couldn't simply mold Apache to their needs played a role. Starting from scratch is so appealing because it removes the complexity.

It is arguable whether starting or forking a project is the most efficient way of removing complexity. At the very least though it is one tactic. As Sean mentioned, the Open Source development process will end up increasing complexity. This is not an invalidation of the process, but rather a simple trade off. The question in my mind is what other tactics can mature Open Source projects use to help manage complexity? Is it a tooling issue or an organizational problem? I'd wonder if things like Distributed Version Control Systems and Bug Trackers might help in terms of reducing/increasing complexity. Are there other tools on the horizon to help calm the storm of complexity or is the issue just a fact of Open Source software? Hopefully bringing up the issue can help begin turning some of the many eyes towards finding the solution this "bug" of complexity.

You might also be interested in:

1 Comment

There is also the question of complexity versus perceived complexity. You perceive complexity because you are aware of the multitude of available web servers. This is arguably the result of better distribution of knowledge.

In fact, when the Apache web server was released in 1995, there were many other web servers in use. Eg. the NCSA httpd (c1993 ?), CERN httpd (1991), NaviServer (1994), Sun (pre-1995) and many others including various Perl modules and custom implementations. In fact according to the Netcraft Survey, in 1995, nearly 40% of all servers were a bundle of miscellaneous "Other" servers.

Today, "Other" is less than 10%, so it seems that the number of different servers in use has in fact dropped since then as Apache and Microsoft have come to hold over 80% of the server market between them.

News Topics

Recommended for You

Got a Question?