Regulatory Transparency and XBRL

By Kurt Cagle
October 15, 2008 | Comments: 6

The last couple of weeks have seen some of the most turbulent action ever in the financial markets, in which trillions of dollars of wealth have disappeared from the stock market, credit remains incredibly tight, and banks and other businesses are beginning to disappear at an alarming rate.

While there are many culprits to blame in this (and its easy to blame anyone and everyone) one group of individuals that are getting especially heavy scrutiny are the banking regulators. It is very likely that some corruption exists here - suborning the regulatory mechanism is the very first step necessary in order to make the kind of deals that ultimately led to the financial collapse possible, and there is no doubt that more than a few regulators should probably be wearing prison orange jumpsuits right now.

However, at the same time it's worth noting that part of the problem with the regulatory system as it exists right now is that it has to do a very nearly impossible task - the regulators have to be able to monitor the flow of more than one hundred billion transactions involving hundreds of trillions of dollars in order to determine whether any specific transactions are taking place illegal (outside the regulatory framework).

Given that in most cases, companies only have to provide financial statements and show their books once a year, and these books in turn are often highly edited and worked over in order to show the companies in the best possible light in terms of both investors and taxes, it's perhaps not surprising that regulation has proved to be such a difficult proposition.

Embedding XML Processing Into Java Applications

Java and XSLTJava and XSLT — Learn how to use XSL transformations in Java programs ranging from stand-alone applications to servlets. This book introduces XSLT and then shows you how to apply transformations in real-world situations, such as developing a discussion forum, transforming documents from one form to another, and generating content for wireless devices.


It's an election year, people are opening up their 401K and dividend statements and fainting, and the cries for more regulation are being increasingly heard in the halls of political power. Yet the irony is that in general, there is plenty of adequate regulation that exists today. The trouble is that is virtually impossible to enforce this regulation in a meaningful fashion, because most corporations are about as transparent as concrete.

Transparency has been one of those goals that has been given a lot of lip service by many corporations, but in point of fact such transparency is a difficult thing for most companies to achieve, even when the will is there ... and often times transparency can be seen as running counter to the goal of the corporation, which is to maximize profits while at the same time minimizing the ability of competitors to do the same.

There is, additionally, another reason for corporations to be less than forthcoming about the actual state of their finances. Public corporations in particular make money in two ways - by creating and selling services and products, and by having investors purchase shares of the company.

Transparency within companies mean that people may be able to see when a company that is representing itself as being profitable is in fact playing fast and free with accounting tricks that show net income to be far smaller than may be presented (or one-time write-offs actually occur with startling regularity). This can affect stock values (especially important to CEOs with large stock option positions) and dividend structures, and thus, until something bad happens, the shareholders typically are just as uninterested in the company revealing their inner workings as the senior management are.

Something bad has happened.

The amount of money being poured into the global financial community in particular by governments (and by extension, by taxpayers) in the last month is staggering - enough to completely revamp the US transportation infrastructure, rebuild the educational system, support the development of a whole host of alternative fuels technologies and fund the entirety of another Iraq war from the US side alone. Couple that with the losses from the level of the worker saving for retirement to the level of the countries of Iceland and Ireland, and what emerges is the fact that the lack of transparency in these dealings made it possible for trillions of dollars of "imaginary" money to enter into the system that collapsed the moment that Heisenberg's box was opened.

In an economy where billions of transactions and hundreds of trillions of dollars move around the globe every day, there is no way that humans by themselves can regulate such systems. Instead, one of the most significant reforms that could take place would be the widespread adoption of the Extensible Business Reporting Language (XBRL) at all levels.

The purpose of XBRL is to define and establish a set of standards that can be used to describe, at a programmatic modeling level, common factors of business accounting, as used by the SEC and other regulatory agencies. Last year, Chris Cox, current head of the SEC, endorsed XBRL as an acceptable format for submitting annual reports - and other countries are moving to the point of not just accepting, but requiring, that an XBRL document be submitted as part of quarterly or annual filings.

It may seem hyperbole to expect that a single XML document will radically transform the finance industry, but it has the potential to have a significant impact. One of the central problems that regulators face currently is that most reports and filings are hard copy text rather than electronic form, and even the form of such filings varies dramatically from one company to the next. While it's possible to scan such documents and apply OCR or similar mechanisms, the amount of work necessary to get even one such company's filings into a searchable form is not trivial, and the sheer variety of such form data means that printed documents are essentially unsearchable as so much of the relevant content is not data but metadata (context).

The SEC has accepted the use of electronic filings using both text and html through its EDGAR (Electronic Data Gathering, Analysis, and Retrieval) system, and does in fact have an older XML format as part of the EDGAR specification - XFDL (Extensible Forms Description Language). However, XFDL is, as the name implies, a forms presentation language. It makes it possible to precisely lay out form content for later print production, but it does not in fact contain any formal business semantics about the information contained within this form.

XBRL, on the other hand, is a standard intended to model business entities rather than providing presentation. With XBRL you could lay out the assets that you have and their associated asset classes, you could give information about profits and losses achieved, you could provide taxation information. Because XBRL focuses on the business objects rather than on the presentation, XBRL is far better suited for providing context to the data that a given company produces as part of its reports, and because of this it also makes XBRL far more ideal to be searched and analyzed.

In the current financial crisis, such an XBRL could have far more clearly identified potential problem areas such as untenable hedge fund bets, understatement of capitalization compared to outstanding debts drawn against this capital, potential misclassification of risk vehicles and so forth. What's perhaps more important, however, is that if the XBRLs were available on a quarterly basis it would have been possible to run analytics on them that would have clearly demonstrated places where inequalities and systemic pressures were building - and would have allowed both regulators and analysts a much better view of the real health of a company.

Moreover, XBRL has gone past both the hype and negative phases and is now emerging as a solid technology with a well-financed and sophisticated product base. Companies such as Rivet Software, JustSystems, RR Donnelley, UBMatrix, Price Waterhouse Coopers, Edgar Online, Deloitte Consulting and others have not only developed sophisticated XBRL tools and workflows in their own right, but have also formed strategic partnerships with Microsoft, Oracle, Adobe, Fujitsu, IBM and other major software consortia in order to offer XBRL creation and analytics solutions to companies and governments worldwide.

This market is being driven in part by pilot electronic systems in governments worldwide. Currently XBRL has either been implemented or is the primary candidate for regulatory organizations in the United States (voluntary filings), Canada (voluntary filings) the United Kingdom (mandatory use of XBRL by 2010), France (mandated), Germany, the Netherlands (mandated), Spain (mandated), Japan (mandated), China (mandated), Australia (voluntary), New Zealand (voluntary) and others that are currently evaluating XBRL based regulatory infrastructures.

One of the more intriguing aspect of this level of development and cooperation is that the XBRL standard makes it much easier to compare apples with apples when dealing with companies in different countries (and which consequently makes it easier for companies that have an XBRL audit trail to clear potential regulatory issues more quickly when wishing to expand into other countries).

Additionally, within the US, the XBRL standard is an integral part of a program unveiled by the SEC on August 19, 2008 called the Interactive Data Electronic Applications system (IDEA), which is intended to move beyond the single document presentation of annual reports to a fully supported, services-oriented data system built on an entirely new architecture. This means that XBRL, either through IDEA or using IDEA-based data, will become the foundation for most of the business analytics performed in the United States.

Because of that, it is likely that the biggest beneficiary in all of this is the investor or financial analyst, what could be seen as a shadow regulatory environment. All too often, an analyst is forced to predict financial behavior for companies based upon very incomplete data such as annual reports, documents that are of course intended more as sales vehicles for investors and existing shareholders. By being able to load in what amounts to a well-defined financial data model for a given company, analysts can actually perform far more comprehensive computational metrics upon a company, with the consequence being a much clearer idea about the true viability of a company when placed in a given competitive environment.

The benefit for companies themselves, however, is somewhat more mixed. XRBL is still a document produced from data that a financial officer provides into the model, so the data within that document is only going to be as accurate as the inputs. This means that if someone is deliberately cooking the books, XBRL by itself will likely not reflect that ... immediately. However, one side effect that may emerge with XBRL is the fact that, because you can run analytics on the data, inconsistencies in the data model over time usually become much more apparent, especially once a large enough base of "rogue patterns" emerge within a regulator's (or analyst's) repository.

Moreover, these are not ordinary times. Faith in financial companies in particular, and with the current capitalistic system in general, has taken a severe blow - to the extent that trust, one of the key hallmarks necessary for a true "free market" to survive, no longer really exists. Companies that would have fought such transparency measures even a year ago are likely to adopt them without much complaint (if not necessarily enthusiastically) if this transparency will restore faith in the companies and they way that they operate.

What's more, as more honestly run businesses make their organizations marginally more transparent via something like XBRL, this will put pressure on those companies that refuse to adopt XBRL, or try to provide the absolute minimum information required by law, and this in turn will make such companies more suspect in the eyes of regulators, analysts, investors, and even customers. Moreover, as the XBRL users grow, so too do tools that make XBRL fit more readily into the electronic accounting workflow that companies use, minimizing the overall impact upon XBRL adopters.

A tectonic shift is taking place in the economy right now, one that is punishing those that have been most abusive of the trust of customers, investors, governments and the taxpayers in those governments. XBRL has the potential to help renew that trust.

Kurt Cagle is Online Editor for O'Reilly Media. He lives in Victoria, British Columbia.


You might also be interested in:

6 Comments

Hi Kurt


- 10 years after the birth of XML and a celebrity excel snafu of considerable proportion emerges:
http://www.theregister.co.uk/2008/10/15/lehman_buyout_excel_confusion/

Ps: I love the material you publish here!

Oh, that's priceless (or at least very, very costly). It's errors like this that I think will ultimately spell the demise of the Microsoft era - not because of anything inherently wrong with Microsoft itself in this case, but because we're reaching a stage where interoperability of documents - whether corporate, financial or legal - has reached a stage where semantics matter, and free-form spreadsheets and the like lack a sufficient amount of metadata to support these semantics in a meaningful fashion

We need XML - need to establish publishing infrastructures that utilize XML for the entire pipeline, rather than relying upon ad-hoc transformations that may introduce unintended consequences on the documents being published.

Thanks for the link (and the kind words) - I'm going to be watching that case closely now.

Perfect analysis. The only thing I'd like to throw in here is that in Germany XBRL isn't there yet. Instead, companies have to file some strange xml which of course is better than pdf & co. It will still take some time to have a XBRL-based filing system in Germany.

The International 2008 XBRL Conference was in Germany this year, which I suspect was partially driven by a desire to showcase the technology to both the German regulatory agencies and corporations that may be interested in working with the technology. One nice thing about the state of XRBL right now is that it is generally easier to perform adoption every time some other government launches their own XBRL initiatives, as a lot of headaches and incompatibilities get aired out.

Getting the content into XML in the first place is a big win. Getting it into contextual XML (i.e., XBRL) is a comparatively small step thereafter.

Regarding the note by one commenter, about the end of the Microsoft era, I think that is premature. One XBRL software vendor uses Excel and Word to hide the complexity of XBRL, and enhance validation. Best of both worlds.

Now is the time to merge and centralize all the financial regulators in the US. We have 3 bank regulators, 1 thrift regulator, 1 credit union regulator, no (repeat NO) centralized insurance company regulator, and the SEC. Other countries have done it, why not us? Answer – politics. The result is that we have some companies reporting similar data to multiple regulators, and the financial mess we are in now.

FYI, the FDIC has been using XBRL to capture bank financial data on quarterly basis for a few years. Result - data has few errors and more timely.
Just imagine what we could do if we had 1 (one) centralized financial regulator, we could merge many, many reports, capture it sooner and analyze it in no time.

This is the perfect opportunity to get past the political roadblocks and merge them all. Call it the Financial Homeland. We have one for physical security, why not financial security?

My comment on the end of the Microsoft Era - you're right about the use of Excel and Word here (I believe that was the Price Waterhouse Cooper product, actually, though I think there are a couple of other vendors that are going that way), though the comparison I brought up here has more to do with the fact that ultimately it is not the Word and Excel documents themselves that are important here, but rather the underlying data structures. As a vector for providing that data, MS Office has some interesting (and useful) benefits, but ultimately I'm more inclined to suspect that this is a late transitional form at best to a more formal web services orientation for XBRL.

As to the question about creating a central Department of Finance - interesting idea, though I suspect that your commentary about politics being behind the current proliferation of regulatory agencies is spot on, though there is another factor - from the outside, a commercial bank does not look like an investment bank does not look like an insurance agency. Internally, of course, the similarities are quite striking, especially if you can move them to a common standard for reporting. With the nationalization of the banks in the US and the quasi-nationalization of the insurance companies, consolidating these entities (at least from a reporting standpoint) makes a lot of sense. On the other hand, power of the purse has long been the basis of power in government, and its unlikely that people will easily (or voluntarily) hand that power over without a lot of friction.

News Topics

Recommended for You

Got a Question?