IBM marketing guy Rob Weir has half of a new series of blogs The Final OOXML Update. up.
Readers may be surprised that I agree with many of the points he makes.
The Importance of a Balance of Interests
In Part I he mentions issues about the domination of standards committees by a limited number of players. This is an issue I certainly think is important, and of concern, and I have blogged about it multiple times. The main difference between our views is that I think Weir sees things primarily in terms of competitive corporations, while I see vendors and users as opposed: to Weir a balance of interests is basically satisfied by a multiplicity of vendors being present (even if most of them are using the same code-base) if I understand him correctly, while to me a balance of interests is satisfied by having adequate non-vendor involvement. (He thinks a committee is balanced if it has people like him on it, while I think it is unbalanced if it has no people like me on it!)
I have raised these issues many times before, and my answer is still the same: at the end of the day, what is needed is more participation especially by governments and non-vendors. Vendors are not credible when speaking for their users, because they don't: they speak for their interests in trying to get, satisfice and keep users. Microsoft, IBM, Sun, Apple, none of them can speak for their users let alone the broader community of their non-users.
About six months ago, I put the boot into the ODF TC for not being representative enough. The laudable participation by a couple of core vendors was not being matched by participation by other kinds of stakeholders, in such a way that would make it difficult for those vendors not to get their way on any issues that came up.
You don't have to see this as some kind of conspiracy by the vendors (Sun and IBM); it is perhaps more like the old "Carry On" movie gag of the hapless soldier volunteering by having everyone else in line step back when volunteers are called for.
The solution is more participation by stakeholders other than software developers, and more participation by a broader range of vendors, and more participation by non-commercial interests, and more participation by end-users: any of the above!
I have no objection to the idea that if a group is in fact over-dominated by a particular set of stakeholders, even though this may be caused innocently because the other stakeholders are disinterested in participating, that it alters how successful the standardization process was. (Indeed, I wrote about it two years ago.) The BRM is so compelling because of the hundreds of participants and multiple National Bodies, for example: it's resolutions must be taken very seriously.
The Need for Continued Participation
In his Part II Weir mentions how unsatisfactory the ISO procedure was that meant that the OOXML BRM (Ballot Resolution Meeting) findings were not first incorporated into a final text that could be seen before voting. I agree, and Weir notes glumly that the ISO procedures have been changed in this regard.
However, as I noted at the time in The Perpetual Ballot Resolution Meeting, the idea that the BRM was the last chance for changes, or the only opportunity for review, is one that causes unnecessary panic. But there is a maintenance process that sifts through these issues. So again, what is the solution? Participation
I don't know why, if you think your opponents are tricky bastards, you would leave them alone for a year, then suddenly complain that they have too much influence? Go figure.
The Need for Follow Through on the BRM Decisions
In Part III Weir looks at some particular issues that have come up as part of maintenance, and looks at them in the light of the BRM. I think this is a very good and important thing to do, though of course our views of the BRM are rather different.
I am also quite concerned about some aspects of the forthcoming Corrigenda and Amendments. Of course, I am not concerned that they exist, that is the point of the Perpetual Ballot Resolution meeting. National Bodies are currently discussing their ballot positions. I believe the next SC34 meeting is in Paris in the first week of December. I hope that National Bodies and Liaison Groups, especially those who attended the BRM, will be able to participate.
To be clear, I think most all the individual items in the Corrigenda and Amendments are non-controversial and deserve support. I don't know that I have any trouble with any item in the Corrigenda. But some amendment items are definitely tricky (not in the sense of someone trying to trick you, in the sense of them being difficult to think through): JTC1's Yes with Comments and No with Comments mechanism is available for individual item improvements or vetoes, and WG4 is an ongoing group so objections of a fine-tuning nature are better handled in the immediately following round of maintenance.
And to be concrete, here are the Corrigenda and Amendments: they are limited in scope particularly to issues arising from the BRM (there are a ton of other defects of course):
But I do think about three big background issues. (I hope without the same hysterical tendency that brands every mistake as a lie, every cockup a conspiracy, every criticism an attack, and every divergence from party line as a betrayal, as the reader may see in some commentators' writings.)
The first is that the initial ballot, the BRM and the subsequent ballot on IS29500 clearly did not accept the idea that it was good enough for IS29500 to merely replicate Microsoft's OOXML: not for Strict nor even for Transitional. Similarly, the idea that IS29500 should be entirely "aspirational" was not accepted. Instead, the BRM came up with the notion that the future was with a "Strict" version of OOXML that had several changes in syntax (and dropped VML and so on) and that other features would be noted in a Transitional section, a superset of the Strict.
Some of the Amendments would change this: I don't actually have any objection to any bogus parts of Transitional being removed or corrected (where DIS29500 says one thing in the name of compatibility, but it was inaccurate against what Office 2007 does) but changing the subset/superset relationship is a big step. SC34 feels it is a technical necessity (because of a report from a vendor, by the way, no Microsoft.) What changes is the subset/superset relationship.
The second issue is related. Any tendency to see Transitional as something with a life, or something to be standardized by national standards bodies needs to be thumped on the head. Some NBs need to get out their priests.
I wrote about this earlier Should OOXML be a national standard? The really regrettable thing that has come out, is that the editor failed to incorporate the text from the BRM that said that Transitional was, err, Transitional. This is fixed in the Part 4 Technical Corrigendum. And I would imagine it would take a bit of chutzpah to make any argument that Transitional should hang around longer just because the consolidated text left it out.
But I think any National Body that has made a National Standard of IS29500 may need to re-visit the decision, as far as Part 4 Transitional features, goes, in the light of that missing text. As it says The intent of this Part is to enable a transitional period during which existing binary documents being migrated to ISO/IEC 29500 can make use of legacy features to preserve their fidelity, while noting that new documents should not use them. Part 4, if anything, is a list of feature that should be avoided and to mandate them for new documents in any way goes against the standard as voted for.
To me, any National Body that did not go through the published text of IS29500 and verify that their issues as resolved at the BRM had not been satisfactorily implemented (and therefore raised a Defect Report) have not done their job. I certainly checked on our Australian issues.
The third is the change in namespace for Strict. For a couple of years now, I have been making little warning noises that I thought we were at the limits of what our schema languages conveniently allow us to represent. I blame W3C XML Schemas to some extent, but the biggest reason is the blind desire on the part of schema-makers and infrastructure makers to avoid any issue of schema evolution apart from the most trivial.
For example, I have said in the past that I think that both ODF and OOXML should have both DSRL and MCE built-in, and I have given some ideas on schema langages for namespace superset/subset relationships. I think the issue of OOXML Strict changing namespace, and therefore becoming incompatible with any existing OOXML software even if it does use MCE (unless there is a total duplication in Transitional) is an example of this crisis. But no-one else regards it as a crisis in our infrastructure. They would prefer to treat it as a challenge to be overcome or somehow unavoidable. Or non-existent.
So what should be done? The answer now, again, is participation. If you are concerned about any of these issues, make sure your national body is participating in the current ballots: if you are not a direct participant, send in you view to the technical committee via your national standards body. (I mean a real view on some of the technical issues, not a diatribe on issues not under discussion.)
The hare and the tortoise
Weir's final point is also one that I am happy to second:
OOXML will be extended by Microsoft much faster than it will be standardized and corrected by ISO. This will make the ISO version of OOXML, currently not supported by Microsoft, even more irrelevant in the future.
This is certainly a possibility, and one that is in fact being discussed already. Of course, IS29500 has a good story for handling extension features with graceful degradation to standards: the MCE features that allow multiple parallel versions. And new features on the market is headache for ODF too, of course: think SmartArt. Or think of the all the changes recently announced for the new formula functions for Excel: how does will WG4 cope with them? How will OpenFormula cope with them? How will governments who want to enforce standards for a level playing-field cope with them?
And here participation is only part of the answer. The NBs and governments who supported IS29500 need to follow through on their goals for adopting IS29500, since they also supported IS26300.
Was the lack of documentation just an excuse?
For example, one of the big selling points for IS29500 was that the Open Source developers had long said they were being hampered by a lack of documentation. But in the two or so years that the ECMA and ISO specifications have been out, have many Open Source project actually used the information much? (I was looking at recent drafts of Open Formula, and found no mention of IS29500, for example, which I found incredible.)
If opening the format floodgates has been met with disinterest, does it suggest that the FOSS people were just bleating to make excuses for product shortfalls? That, for documents at least, the critical path or congestion point has not been the availability of information or IP, but the lack of manpower or adequate processes to harness the million eyeballs?
IS29500 removes that excuse.
But if the primary purpose of IS29500 was to provide documentation, whether or not that documentation was in fact so critical (autospacelikeWindows95 important? pulleeeeze!) then that has been done, with some gaps that can be addressed. If an NB voted in order to get the information out, then now that it is done, it is not so important that IS29500 continue.
In fact, that is exactly what I think should happen to Part 4 of IS29500, the Transitional Migration features. People can still reference the 2008 version of the standard, as amended. But I don't see any value in Part 4 remaining an ISO standard in the medium term. I think WG4 should get a plan together to remove Part 4 entirely. ECMA can maintain it separately if it is important for them. Part 4 is just a burden and distraction on the standards process.
The trouble I see with Part 4 is that it will confuse Microsoft executives. Lead us not into temptation they will undoubtedly be praying by their bedsides each night for we love wriggling and we are atop a cliff. What the BRM and the final IS29500 ballot did not do was say that the existing OOXML format was acceptable as-is as the basis for future documents: only after the transitional features had been removed was it OK.
Is SC34 WG4 an effective mechanism for instructing Microsoft on necessary format and feature changes?
Another reason that a NB might have supported IS29500 was because they wanted to have a more direct input in the features, schemas and notations generated by Microsoft's products. They wanted more input, and less dictation from Microsoft.
I have not seen any indication that this is something that Microsoft is vigorously pursuing: where is the consultation on the new features in Office 14 for example? To be fair, I believe Microsoft did give various briefings to sc34 WG4 participants at various times, and the WG4 was focused on getting the BRM corrections out the door.
But Weir's comment about catch-up is one that I have raised myself privately. Though, again, the availability of MCE means that if a new feature is always used so that its elements are accompanied by an MCE fragment in the standard format, the new features can be down-converted to standard applications or simpler ones like ODF without difficulty. (Just not round-tripped.)
MCE changes a lot, I think. It provides a way for Microsoft to outrun ODF systems on features, while at the same time keeping readable by standard systems.
But I think that NBs that voted for IS29500 because they wanted to make sure that Microsoft cleaned up its formats and feature sets will not get what they want from SC34 WG4 unless they use the only gun and only bullet that it provides: participation.
So where does this leave us?
I have deliberately avoided trialing Office 14, or looking at its feature set in toto. So I am saying this purely out what I believe was the intent of the BRM. I don't know what file formats Office 14 will provide.
The instruction of the BRM, that Transitional is not to be used for new documents, should be taken seriously by any government that wants to adopt open standards. An application that generates OOXML Transitional for new documents (by default) should not be acceptable, because it goes against the clear instructions of Part 4 of the Standard. (These instructions are being restored by the current Corrigenda.)
That this information was missing from the Part 4 as published, due to a serious mistake by the editor, by ITTF and perhaps by a lack of diligence by the Canadian NB to follow it up, changes nothing. The wide participation and publicity of the standard robs that excuse of any power. (Indeed, the use of the word "Transitional" should be enough of a give-away.)
To put it bluntly, if Office 14 (or a future OpenOffice ) generates OOXML Transitional for new documents but not OOXML Strict, the documents may conform to the standard, however the application is non-conforming.
To be clear, the specific text of Part 4 of IS29500 s1 "Scope" (as corrected) says of the transitional features that new documents should not use them. In particular: The features described in this Part shall only be used by documents of conformance class WML Transitional (§2.1), SML Transitional (§2.1), or PML Transitional (§2.1). These features are sometimes needed for high quality migration of existing binary documents to ISO/IEC 29500. The standard does not approve of Part 4 features except for these kind of import scenarios. The should is a little weak, and the shall not is directly tied to the scenario.
Of course, there are several other issues like this. It will be interesting to double check the issue of whether an Excel document using the new Excel functions is saved to OOXMl, it would still conform to the standard. I would presume not. Unless there is an MCE workaround, IS29500 conformance does restrict the kinds of changes that Microsoft can make to its formats without putting them through international scrutiny.
It will be very interesting to see if the European authorities clarify what their intent was when they called for Microsoft to standardize their file formats: was it just to put the information out there to create a level-playing field, or is there an expectation that the reputable standards body will take over stewardship of the format to actively reduce the control that Microsoft is allowed to have on the format. I suspect that Microsoft thinks the former is the case, while many other people might be hoping for the later.
But the one thing that Microsoft cannot be accused of, and that is a lack of participation, a fact its competitors and detractors need to treat seriously, rather than dismiss with a blast of insults.