SAMBA's Jeremy Allison has a great post Why writing a Windows compatible file server is (still) hard.
What leaps out to me? First, that the method of requiring complete documentation outside a formalized QA process doesn't work real well:
The document describing the precise behavior of an NTFS filesystem as seen over the wire from an SMB/CIFS server is yet to be finished. They're still working on it.
Which is not to say that working within a formalized QA process (I mean an international standardization organization, as a standard or technical specification) will necessarily fare better. But at least the formalized process has timetables, reviews, and responses: legs but no teeth.
The second thing is that even if there is documentation, some incompatibilities come down to capability mismatches:
A POSIX system is very different from Windows, so one of the challenges we have is to be able to emulate the different Windows features on top of standard POSIX.
So to get interoperability, Allison has to do some kind of capability matching: implementing a Windows file system emulator on top of the POSIX file system. (There is a school of thought that if I have a beautiful, powerful, elegant system, and you have a complex, carbuncular, scarified system, then it should not be difficult for us to interact well: I think it is a story people tell themselves to cheer themselves up: you may get the advantage my elegance, but I may still have to cope with your complexity.)
The third thing, is that even when the capability matching, it turns out that some of what Allison was assuming/hoping was protocol-determined may in fact have been application-determined:
One packet being returned at the wrong time. One single mis-timed packet caused a ripple effect in the Windows client file system software that was seen all the way up in the complex user interface of only that particular version of Excel, when interacting with the "Offline Files" feature, only on Windows Vista.
If some timing issue is only used by one version of one application on one OS, is it really part of the general protocol? An answer is not necessary, my point is not to split hairs, but to suggest that to some extent getting interoperability may even have to involve layer matching, where you need specific, detailed documentation about the particular super-protocols that applications implement using the networking primitives. (And, if documentation is not forthcoming in a timely fashion, I don't think it is far-fetched that Microsoft runs the risk, ultimately, of being forced to show the appropriate source code to the SAMBA developers, presumably under NDA and with interesting impact on copyright or IP licensing.)