Through A Glass (Very) Darkly: XML 2009 (Part 1 of 2)

By Kurt Cagle
December 23, 2008 | Comments: 5

In December 1999, at an XML Conference in New York, I gave an hour long talk about the economic dangers that faced the programming community to a room full of engineers, programmers, and IT managers. More than a few left the room angry because I had broken one of the unspoken rules that seems to be the norm in programming - don't talk economics or politics, because it doesn't have anything to do with programming.

A year later, the IT industry was in the worst recession that it had faced in fifteen years, a time that became known as the Tech Nuclear Winter. Senior programmers with thirty years of experience and post graduate degrees - people who sat on standards committees boards and often served to shape the industry - could be found at coffee shops "working on their next projects" while waiting for a job to open up.

Dot-com became synonymous with failed business. Enrollment in colleges for computer science degrees, once a sure ticket to become rich and possible famous, dried up overnight, leading to a labor shortage that still hasn't been full assuaged.

My publisher at the time, Wrox Books, dissolved into a morass of accounting nightmares as the book publishing industry collapsed along with the broader market - the current Wrox book imprint (now owned by Wiley) consisted for some time of the few remaining A-Listers of that once wildly popular publishing line.

I started writing an annual prediction letter on the XML industry in particular (touching on the broader tech industry on occasion) that year - sent out initially just to personal contacts, then over the years expanding this as the venue permits, with most of them having appeared in O'Reilly at for the last several years.

This year is particularly portentious and ominous as the global economy looks to be entering its first major recession of the twenty-first century, though ironically, it may actually have less of an impact on IT than the Tech Winter did. Part of the reason for this is that IT itself has evolved considerably over the last decade, becoming more distributed and decentralized - while at the same time, there are fewer skilled people for the jobs that do exist.

What this means over the next few years will be discussed in depth in this report, but overall what will happen is likely just a continuation of existing trends - the rise of services and virtualization, the increasing distribution of IT workers, the move towards power-user friendly technologies rather than dedicated programming environments and so forth.

I've traditionally covered how I did in the last year, both to provide some context for this report and just to see how well I did. As per usually the results were something of a mixed bag:

1. Vertical XML Standards become pervasive. Overall, I'd say this one was pretty much on the mark ... this was the year that standards such as XBRL, DITA, S1000D, HL7 and so forth began taking a much bigger lion's share of press coverage than "pure" XML topics. I expect that XBRL in particular will play a huge role in the upcoming years, though its association with Chris Cox, currently embattled chair of the SEC, could blunt that some. We've definitely entered the realm where people no longer refer to these as XML standards, which means the XML-ness is taken for granted. +1.

2. The New XML Professional. Professional data modellers have entered the IT space in much the same way that architects did a couple of years ago, and XML has become a big part of their toolkit. There are a number of Semantics conferences and Semantic technologies seem to finally be hitting mainstream in meaningful ways, XQuery is beginning to emerge as a must-have skill for people who develop data applications (and I've begun seeing XQuery DBAs listed as titles on business cards). Overall I see this trend continuing as well. +1.

3. Sea Change to XML Data Systems. All of a sudden, nearly every major IT vendor now has an XML Database on the market, and even those that don't, such as Sun, have begun providing XML support into their databases. MarkLogic released their new v.4 XML Server, IBM their IBM DB2 XML server, Oracle's integration with Sleepy Cat is complete, Microsoft has radically expanded their XML capabilities with SQL Server 2008. EMC/Documentum recently purchased the X-Hive XML database and will be releasing upgraded versions of that early in 2009, while the open source eXist database has become the dominant low-end XML database in the FOSS community. +1

4. XForms Gets WiCD. XForms had a mixed time of it this year. The Mozilla XForms project continues to improve, but more slowly, and XForms at this stage seems to be just that much farther from being implemented on a browser than it was in 2007 (although there are some very interesting things happening with the IBM Ubiquity project).

On the other hand, server side XForms is becoming much more solid. Orbeon has solidified its standing in the field, though X-Hive has also come out with a strong XForms solution and with MarkLogic hiring XForms guru Micah Dubinko, I suspect that XForms may not be completely out of the picture for MarkLogic. Meanwhile, JustSystems fXy server provides a similar XForms like interface model for building applications, and they are spending aggressively in order to be seen as one of the dominant players in that space.

I'll claim this one as a partial, though I'm not going to rule out broader XForms support in the browsers via AJAX any time soon. (I felt that XForms probably would end up maturing more in 2009 originally, and I think that continues to be the case). + 1.

5. The Semantic Web Becomes Real. Wow! This one came true in spades this year. Content enrichment, automated taxonomy and classification, Wikipedia being analysed and tripletized, and so on. My suspicion that RDF was going to be a smaller and smaller part of the whole was also pretty much spot-on - RDF is one way of encoding semantics, but not all semantics is necessarily tied up to the atomic level triplet processing that seems so intrinsic to RDF. Rather, what's happening now is the realization that there's a very short line between semantics and navigation in an information sphere, and a lot of the more intriguing tools in this space seem to be focusing far more on this aspect of the semantic web than on notational tagged triplets. +1

6. Atomic Power. AtomPub has definitely gained a fair amount of interest in the last year, as part of a broader trend that I'll discuss shortly which I call Restful services. It's definitely become a big factor - Microsoft and IBM both now have syndication servers that are AtomPub driven, and I've given talks on AtomPub for a number of highly-interested audiences - but it makes far more sense when discussed in the XRX/Restful services context. +1

7.JavaScript 4.0 Debuts. Owwww!! That hurt. After a very contentious several months, the ECMAScript committee basically shelved Javascript 4 in favor of creating a more harmonized 3.5 version in June. I'd had high hopes for this one, but realistically, it looks increasingly like JavaScript will probably be frozen at a level about that of where Firefox is now. However, with Google entering the Browser fray with Chrome, I think that the dynamics of both JavaScript and AJAX have become much more ... interesting. 0.

8.A Standards Compliant Internet Explorer? Hope may spring eternal, but Microsoft's Internet Explorer 8.0 was a disappointment - it added a couple of new features that nobody wanted (because increasingly they are not writing for IE exclusively or even as a primary browser), it did gain better CSS 2.0 compliance, but its JavaScript changes were weak and almost nothing was done in the XML space (and support for XHTML seems to be fading away as well).

The big story in 2008 was Google Chrome, which has a whipsaw fast JavaScript engine, a foundation based upon WebKit, separate processes for each browser tab that will let you kill an unresponsive page in one tab while still being able to function normally in other tabs, moderate SVG support, and some of the coolest documentation on the planet, thanks to geek-comic artist superstar Scott McCloud.

Microsoft for years has been poo-pooing Firefox as being beneath their concern. However, a Google sponsored browser with Firefox-esque internal capabilities likely must have the IE team sweating bullets right now. I don't see the browser wars slowing down anytime soon. +1/2.

9. MS Linq, XQuery and XSLT2. This was a negative prediction - I predicted that Microsoft would probably continue trying to hew off in its own direction rather than follow the W3C standards, and absolutely nothing has changed there. XQuery 1.0 is still not adequately supported in MS SQL Server 2008 (it's essentially unchanged from the 2005 working draft level), while Redmond has continued to push Linq. XQuery traffic has picked up some, however Linq traffic has exploded to about 5 times that of XQuery over the last year, though it seems to also have leveled off, while the XQuery curve is still growing. Unless it appears in a service pack, I don't see XQuery compliance appearing in MS SQL Server until their next version around 2012 or so. +1.

10. XML/JSON Accords. Messaging is becoming a more dominant theme in general (cf. Distributed Computing, below), and I've rather changed my own thoughts on this since I wrote this particular entry. What I see happening now is that most places that expose services are generally exposing three or four different flavors of service - two or three for XML content, one for JSON. JSON has become the dominant language of AJAX, of course, and the indifferent adoption of E4X has done nothing to significantly change that.

On the other hand, its becoming obvious that at the level of the server, the language of communication is increasingly XML, with probably about twenty percent of that being SOAP, forty percent being some syndication format, and the balance being raw XML. If RESTful services take off the way I think they will, expect that percentage to change, but the reality here seems to be that XML/JSON bridges are becoming far more common, though not yet standardized. +1/2.

11. Mozilla Retrenches. 2008 was a challenging year for Mozilla, especially after the rather exhilarating growth of the browser in the previous year. The 3.0 version didn't manage to get all of the changes that had been intended into the browser, though the 3.1 upgrade represents the incorporation of the new JavaScript engine and better support for graphics. The struggles of maintaining a platform that supported plugins meant that the browser struggled with memory issues (though many of these were ameliorated with the 3.0/3.1 releases) that tarnished Mozilla's reputation. The changes planned for JavaScript 4 had to be scaled back with the ECMA Harmony meeting, and Google's announcement of Chrome strained the relationship between the two formerly close companies.

2009 should actually be a fairly good year for the browser maker, however. Refractor provides a stand-alone capability for the browser that makes it possible to create applications that use the browser windows but lets authors determine the underlying chrome - this should make Mozilla more attractive as a pure development platform. Ubiquity provides a command line capability for Firefox that is developing to an intriguing project - what may amount to a command-line system for the web itself.

Performance improvements in 3.1 keeps the competition between Firefox, Chrome/WebKit and Internet Explorer hot and the developments coming, and after a couple of years in which the development community was not strong enough there is now a solid developer core in place that are treating Mozilla as their preferred system. +1

12. Google Grumbles. Google's stock value has tumbled from its lofty valuations, and the crumbling advertising market has adversely affected the company's revenue significantly, to the extent that a hiring freeze was instituted, a number of contractors were laid off and several real estate purchases were either stalled or came undone. Google has also gained the ire of right wing pundits, given the close relationship that Eric Schmidt had with the Obama campaign (although Schmidt announced shortly after the election that he was not interested in a potential Administration CTO cabinet position).

While its likely that this will continue into 2009, there are a number of positive trends within the company as well. Google has strongly hinted that it will be expanding its own cloud computing initiatives, which are already fairly advanced (albeit, like most Google properties, very understated), placing itself in direct competition with Amazon's EC2 services and Microsoft's recently announced Azure system. It has recently settled a number of lawsuits with authors concerning its Google Books properties, which now puts it in a much better position to expand in that space, and has started doing the same with newspaper archives.

The key areas to watch, I believe, with Google in 2009 are going to be in health care, business financial transparency, green technologies and its continued heavy GIS investment, in addition to expanding its cloud computing base. The integration of KML with the Geographic Markup Language (GML) means that Google has developed an extensive delivery platform for annotated spatial projects, which means that it may play a major IT part in the infrastructure revamp coming online with the Obama administration. Google spent some time trying (unsuccessfully) to develop a health care/bio-informatics unit; a different administration with a much stronger focus on pushing IT integration technologies into hospitals may make this worthwhile in revisiting. I'll address the other two areas - financial transparency and green technologies - later in this article. +1

13. Yahoo, AOL, Amazon. I was wrong on Yahoo, more or less correct on AOL and very right on Amazon for 2008. Yahoo is now among the walking wounded. The Microsoft/Yahoo/Google soap opera that has occupied Silicon Valley for the last year has left Yahoo as a zombie company - with a mass exodus of employees after the Microsoft bid was so ineptly handled followed by the layoff of an additional 1500 employees announced in December 2008.

At this stage, Yahoo and AOL are currently discussing a potential merger in 2009, though this may break down if Jerry Yang is finally brought down by irate shareholders. AOL had a moderately good start of the year (with strong advertising revenues) but as the economy has deteriorated so too has the advertising market, and AOL's value has consequently also declined dramatically. Moreover, Microsoft is probably going to be reducing its own cashflow in the face of the credit crunch, and this will likely translate into an unwillingness to take on Yahoo again any time soon, regardless of how open they may be to a takeover. My guess is that either Yahoo or AOL will not be around as an independent entity (though AOL is technically still tied to the flailing Time/Warner entertainment juggernaut) by this time next year.

I had a feeling that Amazon was going to end up looking good this year, and was spot on - EC2 has become the poster child for cloud computing in many people's minds, and that segment of the company will likely continue to expand as a result. Moreover, Jeff Bezos' strategy of diversifying is now paying huge dividends, as it has definitely benefited in the last few years (and almost certainly this year) selling consumer goods as a general store.

On the book front, the introduction of the Kindle was off to a slow start, but is beginning to develop a core audience now, even at the rather high base cost for the unit. If they can get the unit cost of a Kindle down below $200, I suspect that its usage should soar, and with it the rise of consumer e-books. (indeed, my suspicion is that what might prove most cost effective for them would be to "give away" the Kindle and make their money purely on subscription, in much the same way that mobile phones are sold now - premium models would cost above the basic plan, but a baseline Kindle given away as a freebie should pay for itself in four to five months.

Amazon has also become a major electronic and POD self-publisher in a move that enraged many publishing houses - and one that may ultimately radically reduce the total number of "old school" publishers in favor of online publishing "companies" that effectively handle editorial and promotional costs while coordinating with authors and Amazon to handle printing and delivery (or e-conversion and delivery). More on this later ... +1

14. The Coming Social Networks Meltdown. I'll concede this one for timing, but I'll still hold onto it. With the credit freeze, startup companies that had been able to survive on VC or bridge loans while trying to reach profitability are suddenly having to be profitable right now, and social networks in particular are likely to start dropping in droves within the next two to three months as funding runs out. MySpace has definitely jumped the shark (again a victim of ad revenues as well as a general migration to Facebook) and is now probably becoming a stage 6 social network - decaying but not yet dead.

FaceBook, on the other hand, has become the dominant social networking site on the web for now. I suspect that it will weather the coming storm simply by attrition - it'll be the last man standing. Part of this has to do with the use of an extensive and easy to use community API, but part of it too has been a very clear understanding of their target demographics and how to market to them. I think that as job cuts continue into 2009, LinkedIn will likely end up holding out for a while, but its going to be a racing battle - people who are unemployed generally are less likely to be interested in buying goods and services. If they can hold on until the economy recovers, they will be in a very strong position, but I think its going to be close. +1/2

I'm going to carry 15 (Open Source and Open Standards) and 16 (An Economic Credit Crunch and IT) forward, the first because it has a number of different predictions (most of which I called right), the second because this has patently become true and want to discuss it in my predictions for 2009.

Overall, I did pretty well - 11 1/2 out of 14, with only the JavaScript 4.0 being completely wrong. I'm not quite as confident for 2009 - my feeling is that we are in for, at a minimum, a multi-year recession that will be seen in retrospect as a once in a generation event, and with that level of economic chaos, its difficult to see what upsets may be coming down the line.

These predictions will be covered next week in the second installment of this look on where XML and the IT industry in general, is going over the next year.

Kurt Cagle is an author, developer, and online editor for O'Reilly Media, living in Victoria, BC, Canada. You can subscribe to his published articles here or follow him on Twitter.

You might also be interested in:


I think you are spot on with regard to social networking companies dropping like flies, but I have a feeling Facebook will be doing the same. The reason being that they do not provide public profiles and that is a killer for many organizations that might actually pay for a service. The music industry has come to depend on MySpace to a certain extent as many bands use it at their electronic presskit (despite sites like SonicBids owning major conferences such as SXSW), while major labels have managed to effectively promote their artists image and pr. Facebook, on the other hand, does absolutely nothing to gather public attention. The apps platform is nice from a developer perspective, but the value for users just isn't there.

Also, I think Facebook might just be the of Web 2.0. There was an opportunity to sell that was passed up and while it is still a valuable commodity, social networking sites (with the exception of LinkedIn) seem to be entirely about image. MySpace became tagged with a youthfulness that pushed many users to Facebook. Unfortunately, these users are not immature kids writing in "txt speak" and "pimping" their profiles. No, Facebook gained traction with a slightly more mature audience, that I believe has a much better chance of recognizing the lack of value Facebook offers.

LinkedIn, on the other hand, appeals to an Enterprise 2.0 market that actually the ability to make money. LinkedIn could be a huge force in the talent management field, which is a huge topic in Enterprise 2.0 communities. They have the pool of users that help to define common terms everyone can agree on and facilitate actual distributed talent management. I'm not usually a fan of Enterprise 2.0 topics as my impression is that it is a way for small companies to make money implementing consumer trends poorly for large companies, but in this case, LinkedIn has a real opportunity to solve a real need created by distributing the workforce.

"unspoken rules that seems to be the norm in programming - don't talk economics or politics, because it doesn't have anything to do with programming."

Yeah, but the libertarianism of the spoiled brats (white-male boy-men) is pretty conspicuous.


I didn't say that ;-) (May have thought it once or twice, though).

-- Kurt

If xml should become the datastore of record; welcome to the 1960s, it's a poor man's ripoff of IMS. Dr. Codd will return and smite you all. :)

With regards to the XML standards. One of the biggest issues we are seeing is poor conformance the complex standards like HL7. The XSD standard is very complex, and without good XML development and validation tools its all to easy to end up with invalid XML.

We are seeing a lot of clients moving towards XML Data Binding as it forces developers to create compliant XML.

A move to some kind of XML Firewall may be a good approach for companies to take, ie all XML messages must validate against there respective schemas before there consuming application will even consider processing them. This start to ensure better conformance to the standards. The only fly in this ointment is many of the tools sets produce conflicting reports when validating XML documents against an XML Schema.

News Topics

Recommended for You

Got a Question?