Standards, formal or not, make the Internet, the Web, and much much more run. Unfortunately, the process of creating standards can make both sausage factories and political processes look like excellent places to visit.
Standards processes face a wide variety of challenges: competing vendors, hidden agendas, technology that shifts during the standards creation process, too many participants for a coherent conversation, and the difficulty of combining the invention and standardization process. These problems are compounded by many different transparency and legal issues, but even in the best of times standardization is difficult.
While I don't expect to reach standards utopia, standards are definitely a field that could use some refactoring. The suggestions below reflect more than a little of the pain I've felt as a participant, critic, reporter, and user of standards, combined with the success I've seen when standardization actually worked. Though lots of people say "just give me the results I want", I've learned that the process used to create a specification really matters.
As is true in many situations, it seems like the best way to create technology is by having a phase of experimentation followed by a phase of culling and cleaning. This avoids many of the pitfalls of "design by committee" and allows for some additional process tweaks I think would produce better results.
Many standards processes, though not as many as I'd like, actually are culling and cleaning prior work. XML was culling and cleaning SGML. XSL was culling and cleaning DSSSL. XLink was culling and cleaning HyTime. Both of those processes, though imperfect, succeeded in producing something smaller and more usable. In the case of XML, that smaller and more usable transformed the way computing works. In the case of XLink, they produced a spec, but it's rarely used. XSL wound up somewhere in the middle.
There's a political problem here, however. While we need software developers to experiment in real code, letting us figure out which things work in reality, those same developers generally want to maximize the return on their work. Often that means the reason they would invest in a standards process to steer it to do what they want. On the brightest days, enlightened compromises between competitors can produce a usable specification that makes its users happy while not requiring too much extra investment from participants. Other times, deadlock, conflict, inefficiency, inconsistency, and just plain awfulness prevail. It's not all vendors' fault, but they definitely shift priorities.
My proposal combines the necessary role of software developers with a standardization process run by the direct consumers of that software, not its creators. That means two phases of development:
- Invention - A very loosely-directed phase which opens with a call for proposals, possibly a meeting that generates a loose description of the work to be done. Developers can band together and form alliances to build work that answers to that description. Hopefully, multiple groups will take up the challenge, producing alternatives for exploration.
- Selection - A formal group of customers - customers who don't work for the implementers and inventors - evaluates the results of the development phase to figure out what pieces work most easily. They may be able to standardize in a single round, or they may have to select some parts while leaving others open for further development and later standardization.
Separating these two pieces won't solve all of the politics of standards. Different customer groups can certainly have different needs and priorities, and personalities won't vanish. I'm reasonably certain that implementers will still look for any way they can to influence the people in the selection process. Choosing selectors will be difficult, as it would be a major time commitment for questionable reward.
There are pieces of this idea already in the works. The early XML crowd was definitely built around people whose livings depended on using markup rather than creating software. In the CSS world, vendors experiment pretty freely with new possibilities as clearly-marked extensions, allowing developers to try them out and determine how well they work before committing to their broad use. The W3C makes a point of trying to include companies that use their specifications as well as companies that build software around their specifications. Boeing has been a canonical example of that in the XML world, for example.
I recognize that restricting the people who get to make decisions may be controversial, though I'd want all of their deliberations to be public. My hope, though, is that by changing the model to reflect a much more formalized producer-consumer model, we can have the innovative benefits of competition while maximizing the user-friendliness of the results. It's a model - we'll see if anyone is brave enough to implement it.