Olivier Mengué – Code & rando

Aller au contenu | Aller au menu | Aller à la recherche

Tag - testing

Fil des billets - Fil des commentaires

samedi 27 septembre 2008

Playing with timezones and building my own XSLT test framework

Par Olivier Mengué le samedi 27 septembre 2008, 00:22 - Code

I'm currently playing with XSLT to improve my XMLTV grabbing solution. Much fun playing with a functionnal language available in most web browsers.

I had the need to handle timezone with daylight savings to display date/times in my timezone, Europe/Paris (same as most of western Europe), which has currently the following definition (I'm not interested in history for this project):

  • Winter is UTC + 01:00
  • Summer is UTC + 02:00
  • Transition from winter to summer occurs on the last sunday of March at 01:00 UTC
  • Transition from summer to winter occurs on the last sunday of October at 01:00 UTC

So I wrote the following template date-time-Paris which is very specialized as it handles only one specific timezone, but it is very fast. I'm using only two extensions from EXSLT: date:day-in-week() and date:add().

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:tap="test:tap"
  xmlns:date="http://exslt.org/dates-and-times"
  extension-element-prefixes="date"
  >

  <!-- Transformation de l'heure UTC en heure de Paris -->
  <!-- Copyright (c) 2008 Olivier Mengué -->
  <xsl:template name="date-time-Paris">
    <!--
      Daylight savings transitions:
      - last sunday of March at 1:00 UTC
      - last sunday of October at 1:00 UTC
      Time offsets:
      - winter: +01:00
      - summer: +02:00
    -->
    <xsl:param name="date-time-Z"/><!-- UTC -->
    <xsl:variable name="len" select="string-length($date-time-Z)"/>
    <xsl:variable name="seps" select="translate($date-time-Z, '0123456789', '')"/>
    <xsl:if test="$len &lt; 11 or substring($date-time-Z, $len, 1) != 'Z' or ($seps != '--T::Z' and $seps != '--T::.Z' and $seps != '--Z' and $seps != '--TZ' and $seps != '--T:Z')">
      <xsl:message terminate="yes">date-time-Paris: date '<xsl:value-of select="$date-time-Z"/>' invalide.</xsl:message>
    </xsl:if>
    <xsl:variable name="month" select="number(substring($date-time-Z, 6, 2))"/>
    <xsl:variable name="offset">
      <xsl:choose>
        <xsl:when test="$month &lt; 3 or $month &gt; 10">1</xsl:when>
        <xsl:when test="$month &gt; 3 and $month &lt; 10">2</xsl:when>
        <xsl:otherwise>
          <xsl:variable name="transition-day" select="32 - date:day-in-week(concat(substring($date-time-Z, 1, 7), '-31Z'))"/>
          <xsl:variable name="less" select="translate(substring($date-time-Z, 8), '-T:Z', '') &lt; $transition-day*1000000+10000"/>
          <xsl:value-of select="1+number(($less and $month = 10) or (not($less) and $month = 3))"/>
        </xsl:otherwise>
      </xsl:choose>
    </xsl:variable>
    <xsl:value-of select="concat(translate(date:add($date-time-Z, concat('PT', $offset, 'H')), 'Z', ''), '+0', $offset, ':00')"/>
  </xsl:template>

The implementation is really short, however it is quite complex as XSLT 1.0 requires to uses many tricks to workaround languages limitations (no native date type, no string order, no xor operator). Here are some tips to follow the code:

  • (a xor b) is equivalent to ((a and not(b) ) and (not(a) and b)).
  • number(true()) is 1, number(false()) is 0.
  • March and October have the same number of days: 31.
  • $transition-day*10000000+10000 for october 2008 (26010000) is an number representation of 2008-10-26T01:00:00Z.
  • the second argument to date:add() is a ISO 8601 duration. Ex: PT2H (2 hours).

In fact I present here the fourth version of the code, the first one was much longer (5x) as I was using intermediate templates for computations (last-sunday-of-month, previous-sunday). Of course the key to debug, and then optimise and refactor the implementation was to start with a good test suite. The test suite is of course written in XML and embedded in the stylesheet so it can evolve with the code.



  <tap:suite name="date-time-Paris" xmlns:t="test:date-time-Paris">
    <t:t dt="2008-09-24T21:37:52Z"     r="2008-09-24T23:37:52+02:00"/>
    <t:t dt="2008-09-24T23:37:52Z"     r="2008-09-25T01:37:52+02:00"/>
    <t:t dt="2008-09-24T23:37:52.325Z" r="2008-09-25T01:37:52.325+02:00"/>
    <t:t dt="2008-10-01T00:00:00Z"     r="2008-10-01T02:00:00+02:00"/>
    <t:t dt="2008-10-26T00:59:59Z"     r="2008-10-26T02:59:59+02:00"/>
    <t:t dt="2008-10-26T01:00:00Z"     r="2008-10-26T02:00:00+01:00"/>
    <t:t dt="2008-10-31T12:00:00Z"     r="2008-10-31T13:00:00+01:00"/>
    <t:t dt="2008-11-24T21:37:52Z"     r="2008-11-24T22:37:52+01:00"/>
    <t:t dt="2008-12-31T23:30:00Z"     r="2009-01-01T00:30:00+01:00"/>
    <t:t dt="2009-01-01T00:00:00Z"     r="2009-01-01T01:00:00+01:00"/>
    <t:t dt="2009-03-01T12:00:00Z"     r="2009-03-01T13:00:00+01:00"/>
    <t:t dt="2009-03-29T00:59:59Z"     r="2009-03-29T01:59:59+01:00"/>
    <t:t dt="2009-03-29T01:00:00Z"     r="2009-03-29T03:00:00+02:00"/>
    <t:t dt="2009-03-30T12:00:00Z"     r="2009-03-30T14:00:00+02:00"/>
    <t:t dt="2009-07-01T00:00:00Z"     r="2009-07-01T02:00:00+02:00"/>
  </tap:suite>

A more complete testsuite can be generated from a local dump of the zoneinfo database (directly works on Ubuntu Hardy, however the zdump tool is missing on RHEL/CentOS):

zdump -v Europe/Paris | perl -ne 'm/ (Mar|Oct) (\d{2}) (\d{2}:\d{2}:\d{2}) (20[0-5]\d) UTC = ... (?:Oct|Mar) (\d{2}) (\d{2}:\d{2}:\d{2}) / && do { my $mm = sprintf "%02d", 3+7*($1 eq 'Oct'); print "    <t:t dt=\"$4-$mm-$2T$3\" r=\"$4-$mm-$5T$6\"/>\n"; }'

Now, we need is a way to run the test suite. In XSLT terms, we say: "to apply a template to the test data". This template will just apply a date-time-Paris call to each of the tests.

  <xsl:template match="t:t" xmlns:t="test:date-time-Paris">
    <xsl:call-template name="tap:is">
      <xsl:with-param name="got">
        <xsl:call-template name="date-time-Paris">
          <xsl:with-param name="date-time-Z" select="@dt"/>
        </xsl:call-template>
      </xsl:with-param>
      <xsl:with-param name="expected" select="@r"/>
      <xsl:with-param name="name" select="@dt"/>
    </xsl:call-template>
  </xsl:template>

</xsl:stylesheet>

We now have a complete stylesheet that can be used as a library in other stylesheet. But we have not yet run the tests. Also you probably wonder what are this XML namespace test:tap and the template called tap:is.

test:tap is just a small XSLT test framework that I wrote and the report test results following the Test Anything Protocol that is well known by Perl programmers. tap:is is part of the test:tap API and just check for equality and reports in the TAP format.

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:tap="test:tap"
  >

  <!-- Run all the tests in the test suite -->
  <xsl:template match="tap:suite">
    <xsl:message>1..<xsl:value-of select="count(node()[name()!=''])"/></xsl:message>
    <xsl:message># Suite: <xsl:value-of select="@name"/></xsl:message>
    <!-- TODO fix this expression -->
    <xsl:apply-templates select="node()[name()!='']"/>
  </xsl:template>

  <!-- Test Anything Protocol style reporting of tests -->
  <xsl:template name="tap:is">
    <xsl:param name="got"/>
    <xsl:param name="expected"/>
    <xsl:param name="name"/>
    <xsl:choose>
      <xsl:when test="$got = $expected">
        <xsl:message>ok <xsl:value-of select="position()"/> - <xsl:value-of select="concat($name, ' -&gt; ', $got)"/></xsl:message>
      </xsl:when>
      <xsl:otherwise>
        <xsl:message>not ok <xsl:value-of select="position()"/> - <xsl:value-of select="$name"/></xsl:message>
        <xsl:message>#          got: <xsl:value-of select="$got"/></xsl:message>
        <xsl:message>#     expected: <xsl:value-of select="$expected"/></xsl:message>
      </xsl:otherwise>
    </xsl:choose>
  </xsl:template>

</xsl:stylesheet>

We now have two libraries. Let's add a bit more XSLT to glue them together. The main stylesheet applies tap:suite templates to all tap:suite from all imported stylesheets.

<?xml version="1.0" encoding="UTF-8"?>
<xsl:transform version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  >

  <!-- Test framework -->
  <xsl:import href="tap.xslt"/>

  <!-- All stylesheets to test -->
  <xsl:import href="date-time-Paris.xslt"/>

  <!-- Launch the tests embedded in libraires, whatever the input is -->
  <xsl:template match="/">
    <xsl:for-each select="document('')/xsl:transform/xsl:import/@href">
      <xsl:apply-templates select="document(.)//tap:suite" xmlns:tap="test:tap"/>
    </xsl:for-each>
  </xsl:template>

</xsl:transform>

We now have everything to run the test suite:

$ echo '<x/>' | xsltproc tap-run.xslt -
1..15
# Suite: date-time-Paris
ok 1 - 2008-09-24T21:37:52Z -> 2008-09-24T23:37:52+02:00
ok 2 - 2008-09-24T23:37:52Z -> 2008-09-25T01:37:52+02:00
not ok 3 - 2008-09-24T23:37:52.325Z
#          got: 2008-09-25T01:37:52.3249999999998+02:00
#     expected: 2008-09-25T01:37:52.325+02:00
ok 4 - 2008-10-01T00:00:00Z -> 2008-10-01T02:00:00+02:00
ok 5 - 2008-10-26T00:59:59Z -> 2008-10-26T02:59:59+02:00
ok 6 - 2008-10-26T01:00:00Z -> 2008-10-26T02:00:00+01:00
ok 7 - 2008-10-31T12:00:00Z -> 2008-10-31T13:00:00+01:00
ok 8 - 2008-11-24T21:37:52Z -> 2008-11-24T22:37:52+01:00
ok 9 - 2008-12-31T23:30:00Z -> 2009-01-01T00:30:00+01:00
ok 10 - 2009-01-01T00:00:00Z -> 2009-01-01T01:00:00+01:00
ok 11 - 2009-03-01T12:00:00Z -> 2009-03-01T13:00:00+01:00
ok 12 - 2009-03-29T00:59:59Z -> 2009-03-29T01:59:59+01:00
ok 13 - 2009-03-29T01:00:00Z -> 2009-03-29T03:00:00+02:00
ok 14 - 2009-03-30T12:00:00Z -> 2009-03-30T14:00:00+02:00
ok 15 - 2009-07-01T00:00:00Z -> 2009-07-01T02:00:00+02:00

So now I know that I have a failing test due to rounding occuring. For my XMLTV project it is not important as I will not have to handle decimal seconds.

aucun commentaire aucun rétrolien

jeudi 12 avril 2007

Simple Perl wrapper for non-Perl TAP programs

Par Olivier Mengué le jeudi 12 avril 2007, 18:51 - Code

TAP (Test Anything Protocol) is a simple protocol to report the result of a testsuite. It was initially designed for test suites of Perl modules, but developers of the Test::Harness modules have seen the potential for a more generic usage of the protocol outside of the Perl ecosystem and libraries are in development for other programming languages.

I'm writing C code and contrary to most programming langages today, there is no library to build test suites in a standard way. Java has JUnit, Perl has Test::Simple and Test::More...

Someone has developped libtap, but no one uses it (except FreeBSD developers as advertised on the site). Also, I do not like the library interface because it exports functions such as ok() or diag() which could conflict with existing code. So, I wrote my own library that I will publish one day. But that is not the point of this post.

When you have written a program conforming to the TAP, you want tools to analyse the result of the tests. And the main advantage of using a standardized protocol is to be able to use generic tools and avoid to implement/test/debug them yourself. The problem with TAP is that the only tool that is available is prove and it suffers of a major flaw: it is designed to run only Perl tests.

Here is a simple TAP-compliant C program that outputs the result of a static testsuite:

#include <stdio.h>

int main(int argc, char *argv[])
{
  puts("1..1\nok 1");
  return 0;
}

And the output:

$ gcc a.c
$ ./a.out
1..1
ok 1

When trying to run it with prove, perl complains it did not found Perl code:

$ prove a.out
a....Unrecognized character \x7F at a.out line 1.
a....dubious
        Test returned status 9 (wstat 2304, 0x900)
FAILED--1 test script could be run, alas--no output ever seen

So I wrote the following generic Perl wrapper a.out.t that just runs a.out:

# vim:set ft=perl:

use strict;
use File::Spec;
my $exe = File::Spec->rel2abs($0);
$exe =~ s/\.t$//;

exec $exe $exe or die "$exe: $!";

And now, success is achieved:

$ prove a.out.t
a.out....ok
All tests successful.
Files=1, Tests=1,  0 wallclock secs ( 0.01 cusr +  0.01 csys =  0.02 CPU)

aucun rétrolien