Schema coverage report

SVRL to XSL and Ant

By Rick Jelliffe
October 22, 2010 | Comments: 3

You have a large or complex Schematron schema and it produces no errors. How do you know it is working? A coverage report lets you see how many of each Schematron rule was fired when checking the document(s).

The report can be quite simple but still useful:
Coverage2.PNG

Here is a little XSLT stylesheet to produce this (the blue section is optional: I just add jquery tablesorter automatically to everything technical with tables nowadays): the input is an SVRL validation report (or a collection of these in some wrapper element):


<xsl:stylesheet version="2.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:svrl="http://purl.oclc.org/dsdl/svrl" >
<xsl:output method="html" />
<xsl:template match="/">
<html>
<head>
<title>Rules Coverage Report</title>
<link rel="stylesheet"
href="/js/jquery.tablesorter/themes/blue/style.css"
type="text/css" media="print, projection, screen" />
<script type="text/javascript" src="/js/jquery-1.4.2.min.js"></script>
<script type="text/javascript"
src="js/jquery.tablesorter/jquery.tablesorter.min.js"></script>
<script type="text/javascript"
src="js/jquery.tablesorter/addons/pager/jquery.tablesorter.pager.js"
></script>
<script type="text/javascript">

$(document).ready(function() {
$("#t1").tablesorter();
$("#t2").tablesorter();
} );
</script>

</head>
<body>
<h1>Coverage Report For SBR Round-Trip Tests</h1>
<table border="1" id="t1" class="tablesorter" cellspacing="1">
<thead>
<tr>
<th>Schematron Rule Id</th> <th>Count</th>
</tr>
</thead>
<tbody>
<xsl:for-each-group select="/properties//svrl:fired-rule" group-by="@id">
<xsl:sort order="ascending" />
<tr>
<td><xsl:value-of select="current-grouping-key()"/></td>
<td><xsl:value-of select="count(current-group())"/></td>
</tr>
</xsl:for-each-group>
</tbody>
</table>
</body>
</html>
</xsl:template>

</xsl:stylesheet>

One good tip is to hypothesize before you run the report about what it should contain: that way you may be primed to spot oddities. Are some rules that should be firing never fired, or with unexpected counts? This may be a bug in your Xpaths or your understanding of rule contexts.

Another use of this kind of report is that when you run it over many files, it tells you what cases your test set actually has. You may think you have many examples of some kind of data, and spent time writing lots of nice rules to handle it, only to find that the test data set you have been provided with has none. Good to find out.

Below I have a little ANT script which trawls a directory tree for SVRL files and creates a large file suitable for input into the XSLT above. I am not super proud of this code, but this clunky method was the only thing that would work for me (yes, I tries appending to buffers; yes, I know generating start and end tags as text is a sign of mental and moral defectiveness, etc.) The only trick in this code is a line to remove the XML header from the concatenated files; I also strip them back, in the interests of file sizes. (The green code below is just to pull in the ant-contrib library, which you will have to install.)

<project name="your project"
   xmlns:svrl="http://purl.oclc.org/dsdl/svrl"  >
...
<!-- use ant-contrib task library for for-each task --> <taskdef resource="net/sf/antcontrib/antcontrib.properties"/> <!-- use xmltask library to read xml files --> <taskdef name="xmltask" classname="com.oopsconsultancy.xmltask.ant.XmlTask"/>
... <!-- ============================================================= Generate rules coverage report =============================================================--> <target name="makeRulesCoverageReport"> <!-- Generate coverage report: which comparator rules have been fired --> <delete quiet="true"> <file name="${resultsDir}/Tests/ruleCoverage.xml"/> </delete> <ac:for param="svrlFile" xmlns:ac="antlib:net.sf.antcontrib" > <path> <fileset dir="${resultsDir}/Tests/" includes="**/*svrl"/> </path> <ac:sequential> <echo>Processing @{svrlFile}</echo> <xmltask source="@{svrlFile}" dest="@{svrlFile}.strip" encoding="utf-8"> <remove path="/processing-instruction()" /> <remove path="//svrl:failed-assert" /> <remove path="//svrl:successful-report" /> <remove path="//svrl:text" /> <remove path="//text()" /> <remove path="/*/*/@title" /> <remove path="/*/*/comment()" /> <remove path="//svrl:fired-rule/@context" /> </xmltask> <!-- remove XML encoding declaration--> <copy file="@{svrlFile}.strip" toFile="@{svrlFile}.strip1"> <filterchain> <headfilter skip="1" lines="-1" /> </filterchain> </copy> <delete quiet="true" file="@{svrlFile}.strip}" /> </ac:sequential> </ac:for > <concat encoding="utf-8" destfile="${resultsDir}\Tests\ruleCoverage.xml" > <fileset dir="${resultsDir}/Tests/" includes="**/*strip1" /> <header><![CDATA[<properties initial="yes">]]></header> <footer><![CDATA[</properties>]]></footer> </concat> <antcall target="runTransform"> <param name="xsltFile" value="${ruleCoverageXslt}"/> <param name="instanceFile" value="${resultsDir}\Tests\ruleCoverage.xml"/> <param name="outFile" value="${resultsDir}\Tests\ruleCoverage.html"/> </antcall> </target>

You might also be interested in:

3 Comments

Where has the /properties element come from Rick?

Not in the ISO standard app D, SVRL?

Dave

It is just a wrapper to allow multiple SVRL files to be concatenated by the Ant.

It is redundant in the XSLT code (I thought I had removed it) so I'll take it out. Thanks for finding that!

Thank you for this report Rick.

News Topics

Recommended for You

Got a Question?