Olivier Mengué – Code & rando - Tag - XSLT
2024-03-25T15:25:14+01:00
urn:md5:57e646ab8ca85028168daaaa985aa995
Dotclear
xmltv-fr.appspot.com est en ligne !
urn:md5:bad73d92d457285b615d07698ff1b5a8
2008-09-28T00:10:00+00:00
2013-08-10T16:21:11+00:00
Olivier Mengué
XMLTV
AppEngineBeautifulSoupPythonwebdevXMLTVxmltv-frXSLT
<p>J'annonce <a href="http://xmltv-fr.appspot.com/">xmltv-fr.appspot.com</a>, mon grabber XMLTV qui diffuse la grille de programmes de télévision au format XMLTV.</p>
<p>J'avais déjà développé <a href="http://o.mengue.free.fr/blog/2008/02/11/51-les-programmes-tele-de-telerama-en-xmltv">une solution à base de scripts shell et de feuille de style XSLT</a>, puis un <a href="http://o.mengue.free.fr/xmltv/telerama.xmltv" type="application/xml" rel="noindex nofollow">proxy en PHP</a> il y a 6 mois, mais j'avais des problèmes de fiabilité en raison de l'hébergement chez Free.fr : un appel sur trois au service échouait. J'ai donc fait en une journée le portage en Python pour l'héberger sur Google AppEngine. Grâce à l'infrastructure de Google je ne devrait pas avoir de soucis du même type.</p>
<p>Pour l'instant, le flux récupéré ne correspond qu'à un seul appel au site Télérama et donc ne récupère que 3h de programmes des 6 chaînes analogiques nationales. Mais je compte bien poursuivre le portage pour arriver à l'équivalent de la version PHP puis éventuellement utiliser le stockage Google pour améliorer les performances grâce à un cache.</p>
<p>Pour vous faire patienter, voici un peu de doc sur l'interface du grabbber PHP :</p>
<ul>
<li><code>/xmltv/telerama.xmltv</code> : 3h de programme à partir de maintenant sur les 6 chaînes (c'est la seule chose qui fonctionne sur xmltv-fr.appspot.com pour l'instant).</li>
<li><code>/xmltv/telerama.xmltv?start=20080927220000</code> : 3h de programme à partir du samedi 27 à 22:00 (heure de Paris)</li>
<li><code>/xmltv/telerama.xmltv?channels=192,4,80,34,47,118,111,445,119,195,446,444,234,78,226,481,458,482</code> : 3h de programme de la TNT</li>
</ul>
<p>J'ai écrit cette semaine une feuille de style XSLT pour faire plusieurs appels au service et combiner les résultats en un seul fichier XMLTV :</p>
<pre class="terminal">
<samp>xsltproc -o tv.xml http://o.mengue.free.fr/xmltv/telerama-full.xslt http://o.mengue.free.fr/xmltv/TNT.xml</samp>
</pre>
<p>Dès que j'aurais implémenté les paramètres <code>start</code> et <code>channels</code> la même feuille de style pourra aussi fonctionner pour <code>xml-fr.appspot.com</code>.</p>
<p class="update"><strong>MàJ 2008-09-29 :</strong> le portage est complet, l'API complète (<code>start</code>, <code>channels</code>) est disponible, donc vous pouvez utiliser la même feuille de style simplement en changeant l'URL du service. Voici un exemple où je récupère le programme des 5 prochains jours (<a href="http://www.w3.org/TR/xmlschema-2/#duration">voir le format du paramètre <code>duration</code></a>) :</p>
<pre class="terminal">
<samp>xsltproc -o tv.xml -stringparam xmltv-url http://xmltv-fr.appspot.com/telerama.xmltv -stringparam duration P5D http://o.mengue.free.fr/xmltv/telerama-full.xslt http://o.mengue.free.fr/xmltv/TNT.xml</samp>
</pre>
<p>Malheureusement, je m'aperçois que je rencontre les mêmes problèmes d'accès au service : je ne peux qu'en déduire que cela vient en fait du site Télérama. À suivre…</p>
Playing with timezones and building my own XSLT test framework
urn:md5:bf1609529cefeef2c332c708ec216212
2008-09-27T00:22:40+00:00
2008-10-21T00:44:52+00:00
Olivier Mengué
Code
programmingTAPTDDTest Anything Protocoltest driven developmenttestingtimezonewebdevXSLT
<p>I'm currently playing with <a href="http://www.w3.org/TR/xslt" title="XSL Transformations">XSLT</a> to improve my <a href="http://o.mengue.free.fr/blog/2008/03/02/53-telerama-xmltv-mise-a-jour">XMLTV grabbing solution</a>. Much fun playing with a functionnal language available in most web browsers.</p>
<p>I had the need to handle timezone with daylight savings to display date/times in my timezone, <code>Europe/Paris</code> (same as most of western Europe), which has currently the following definition (I'm not interested in history for this project):</p>
<ul>
<li>Winter is UTC + 01:00</li>
<li>Summer is UTC + 02:00</li>
<li>Transition from winter to summer occurs on the last sunday of March at 01:00 UTC</li>
<li>Transition from summer to winter occurs on the last sunday of October at 01:00 UTC</li>
</ul>
<p>So I wrote the following template <code>date-time-Paris</code> which is very specialized as it handles only one specific timezone, but it is very fast. I'm using only two extensions from <a href="http://exslt.org/">EXSLT</a>: <a href="http://exslt.org/date/functions/day-in-week/index.html"><code>date:day-in-week()</code></a> and <a href="http://exslt.org/date/functions/add/index.html"><code>date:add()</code></a>.</p>
<pre class="code vim vimft-xslt">
<span class="Comment"><?</span><span class="Type">xml</span><span class="Type"> </span><span class="Type">version</span>=<span class="Constant">"1.0"</span><span class="Type"> </span><span class="Type">encoding</span>=<span class="Constant">"UTF-8"</span><span class="Comment">?></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">stylesheet</span><span class="Identifier"> </span><span class="Type">version</span>=<span class="Constant">"1.0"</span>
<span class="Identifier"> </span><span class="Type">xmlns</span><span class="Comment">:</span><span class="Type">xsl</span>=<span class="Constant">"<a href="http://www.w3.org/1999/XSL/Transform">http://www.w3.org/1999/XSL/Transform</a>"</span>
<span class="Identifier"> </span><span class="Type">xmlns</span><span class="Comment">:</span><span class="Type">tap</span>=<span class="Constant">"test:tap"</span>
<span class="Identifier"> </span><span class="Type">xmlns</span><span class="Comment">:</span><span class="Type">date</span>=<span class="Constant">"<a href="http://exslt.org/dates-and-times">http://exslt.org/dates-and-times</a>"</span>
<span class="Identifier"> </span><span class="Type">extension-element-prefixes</span>=<span class="Constant">"date"</span>
<span class="Identifier"> ></span>
<span class="Comment"><!</span><span class="Comment">-- Transformation de l'heure UTC en heure de Paris --</span><span class="Comment">></span>
<span class="Comment"><!</span><span class="Comment">-- Copyright (c) 2008 Olivier Mengué --</span><span class="Comment">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"date-time-Paris"</span><span class="Identifier">></span>
<span class="Comment"><!</span><span class="Comment">--</span>
<span class="Comment"> Daylight savings transitions:</span>
<span class="Comment"> - last sunday of March at 1:00 UTC</span>
<span class="Comment"> - last sunday of October at 1:00 UTC</span>
<span class="Comment"> Time offsets:</span>
<span class="Comment"> - winter: +01:00</span>
<span class="Comment"> - summer: +02:00</span>
<span class="Comment"> --</span><span class="Comment">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">param</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"date-time-Z"</span><span class="Identifier">/></span><span class="Comment"><!</span><span class="Comment">-- UTC --</span><span class="Comment">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">variable</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"len"</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"string-length($date-time-Z)"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">variable</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"seps"</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"translate($date-time-Z, '0123456789', '')"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">if</span><span class="Identifier"> </span><span class="Type">test</span>=<span class="Constant">"$len </span><span class="Type">&</span><span class="Statement">lt</span><span class="Type">;</span><span class="Constant"> 11 or substring($date-time-Z, $len, 1) != 'Z' or ($seps != '--T::Z' and $seps != '--T::.Z' and $seps != '--Z' and $seps != '--TZ' and $seps != '--T:Z')"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">message</span><span class="Identifier"> </span><span class="Type">terminate</span>=<span class="Constant">"yes"</span><span class="Identifier">></span>date-time-Paris: date '<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"$date-time-Z"</span><span class="Identifier">/></span>' invalide.<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">message</span><span class="Identifier">></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">if</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">variable</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"month"</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"number(substring($date-time-Z, 6, 2))"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">variable</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"offset"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">choose</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">when</span><span class="Identifier"> </span><span class="Type">test</span>=<span class="Constant">"$month </span><span class="Type">&</span><span class="Statement">lt</span><span class="Type">;</span><span class="Constant"> 3 or $month </span><span class="Type">&</span><span class="Statement">gt</span><span class="Type">;</span><span class="Constant"> 10"</span><span class="Identifier">></span>1<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">when</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">when</span><span class="Identifier"> </span><span class="Type">test</span>=<span class="Constant">"$month </span><span class="Type">&</span><span class="Statement">gt</span><span class="Type">;</span><span class="Constant"> 3 and $month </span><span class="Type">&</span><span class="Statement">lt</span><span class="Type">;</span><span class="Constant"> 10"</span><span class="Identifier">></span>2<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">when</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">otherwise</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">variable</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"transition-day"</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"32 - date:day-in-week(concat(substring($date-time-Z, 1, 7), '-31Z'))"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">variable</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"less"</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"translate(substring($date-time-Z, 8), '-T:Z', '') </span><span class="Type">&</span><span class="Statement">lt</span><span class="Type">;</span><span class="Constant"> $transition-day*1000000+10000"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"1+number(($less and $month = 10) or (not($less) and $month = 3))"</span><span class="Identifier">/></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">otherwise</span><span class="Identifier">></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">choose</span><span class="Identifier">></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">variable</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"concat(translate(date:add($date-time-Z, concat('PT', $offset, 'H')), 'Z', ''), '+0', $offset, ':00')"</span><span class="Identifier">/></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Identifier">></span>
</pre>
<p>The implementation is really short, however it is quite complex as XSLT 1.0 requires to uses many tricks to workaround languages limitations (no native date type, no string order, no xor operator). Here are some tips to follow the code:</p>
<ul>
<li>(a xor b) is equivalent to ((a and not(b) ) and (not(a) and b)).</li>
<li>number(true()) is 1, number(false()) is 0.</li>
<li>March and October have the same number of days: 31.</li>
<li><code>$transition-day*10000000+10000</code> for october 2008 (26010000) is an number representation of 2008-10-26T01:00:00Z.</li>
<li>the second argument to <code>date:add()</code> is a <a href="http://www.w3.org/TR/xmlschema-2/#duration">ISO 8601 duration</a>. Ex: PT2H (2 hours).</li>
</ul>
<p>In fact I present here the fourth version of the code, the first one was much longer (5x) as I was using intermediate templates for computations (last-sunday-of-month, previous-sunday). Of course the key to debug, and then optimise and refactor the implementation was to start with a good test suite. The test suite is of course written in XML and embedded in the stylesheet so it can evolve with the code.</p>
<pre class="code vim vimft-xslt">
<span class="Identifier"><</span><span class="Special">tap</span><span class="Comment">:</span><span class="Identifier">suite</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"date-time-Paris"</span><span class="Identifier"> </span><span class="Type">xmlns</span><span class="Comment">:</span><span class="Type">t</span>=<span class="Constant">"test:date-time-Paris"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">t</span><span class="Comment">:</span><span class="Identifier">t</span><span class="Identifier"> </span><span class="Type">dt</span>=<span class="Constant">"2008-09-24T21:37:52Z"</span><span class="Identifier"> </span><span class="Type">r</span>=<span class="Constant">"2008-09-24T23:37:52+02:00"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">t</span><span class="Comment">:</span><span class="Identifier">t</span><span class="Identifier"> </span><span class="Type">dt</span>=<span class="Constant">"2008-09-24T23:37:52Z"</span><span class="Identifier"> </span><span class="Type">r</span>=<span class="Constant">"2008-09-25T01:37:52+02:00"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">t</span><span class="Comment">:</span><span class="Identifier">t</span><span class="Identifier"> </span><span class="Type">dt</span>=<span class="Constant">"2008-09-24T23:37:52.325Z"</span><span class="Identifier"> </span><span class="Type">r</span>=<span class="Constant">"2008-09-25T01:37:52.325+02:00"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">t</span><span class="Comment">:</span><span class="Identifier">t</span><span class="Identifier"> </span><span class="Type">dt</span>=<span class="Constant">"2008-10-01T00:00:00Z"</span><span class="Identifier"> </span><span class="Type">r</span>=<span class="Constant">"2008-10-01T02:00:00+02:00"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">t</span><span class="Comment">:</span><span class="Identifier">t</span><span class="Identifier"> </span><span class="Type">dt</span>=<span class="Constant">"2008-10-26T00:59:59Z"</span><span class="Identifier"> </span><span class="Type">r</span>=<span class="Constant">"2008-10-26T02:59:59+02:00"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">t</span><span class="Comment">:</span><span class="Identifier">t</span><span class="Identifier"> </span><span class="Type">dt</span>=<span class="Constant">"2008-10-26T01:00:00Z"</span><span class="Identifier"> </span><span class="Type">r</span>=<span class="Constant">"2008-10-26T02:00:00+01:00"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">t</span><span class="Comment">:</span><span class="Identifier">t</span><span class="Identifier"> </span><span class="Type">dt</span>=<span class="Constant">"2008-10-31T12:00:00Z"</span><span class="Identifier"> </span><span class="Type">r</span>=<span class="Constant">"2008-10-31T13:00:00+01:00"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">t</span><span class="Comment">:</span><span class="Identifier">t</span><span class="Identifier"> </span><span class="Type">dt</span>=<span class="Constant">"2008-11-24T21:37:52Z"</span><span class="Identifier"> </span><span class="Type">r</span>=<span class="Constant">"2008-11-24T22:37:52+01:00"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">t</span><span class="Comment">:</span><span class="Identifier">t</span><span class="Identifier"> </span><span class="Type">dt</span>=<span class="Constant">"2008-12-31T23:30:00Z"</span><span class="Identifier"> </span><span class="Type">r</span>=<span class="Constant">"2009-01-01T00:30:00+01:00"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">t</span><span class="Comment">:</span><span class="Identifier">t</span><span class="Identifier"> </span><span class="Type">dt</span>=<span class="Constant">"2009-01-01T00:00:00Z"</span><span class="Identifier"> </span><span class="Type">r</span>=<span class="Constant">"2009-01-01T01:00:00+01:00"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">t</span><span class="Comment">:</span><span class="Identifier">t</span><span class="Identifier"> </span><span class="Type">dt</span>=<span class="Constant">"2009-03-01T12:00:00Z"</span><span class="Identifier"> </span><span class="Type">r</span>=<span class="Constant">"2009-03-01T13:00:00+01:00"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">t</span><span class="Comment">:</span><span class="Identifier">t</span><span class="Identifier"> </span><span class="Type">dt</span>=<span class="Constant">"2009-03-29T00:59:59Z"</span><span class="Identifier"> </span><span class="Type">r</span>=<span class="Constant">"2009-03-29T01:59:59+01:00"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">t</span><span class="Comment">:</span><span class="Identifier">t</span><span class="Identifier"> </span><span class="Type">dt</span>=<span class="Constant">"2009-03-29T01:00:00Z"</span><span class="Identifier"> </span><span class="Type">r</span>=<span class="Constant">"2009-03-29T03:00:00+02:00"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">t</span><span class="Comment">:</span><span class="Identifier">t</span><span class="Identifier"> </span><span class="Type">dt</span>=<span class="Constant">"2009-03-30T12:00:00Z"</span><span class="Identifier"> </span><span class="Type">r</span>=<span class="Constant">"2009-03-30T14:00:00+02:00"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">t</span><span class="Comment">:</span><span class="Identifier">t</span><span class="Identifier"> </span><span class="Type">dt</span>=<span class="Constant">"2009-07-01T00:00:00Z"</span><span class="Identifier"> </span><span class="Type">r</span>=<span class="Constant">"2009-07-01T02:00:00+02:00"</span><span class="Identifier">/></span>
<span class="Identifier"></</span><span class="Special">tap</span><span class="Comment">:</span><span class="Identifier">suite></span>
</pre>
<p>A more complete testsuite can be generated from a local dump of the <a href="http://www.twinsun.com/tz/tz-link.htm">zoneinfo database</a> (directly works on <a href="http://www.ubuntu.com/">Ubuntu</a> Hardy, however the <code>zdump</code> tool is missing on RHEL/CentOS):</p>
<pre class="terminal unix">
<kbd>zdump -v Europe/Paris | perl -ne 'm/ (Mar|Oct) (\d{2}) (\d{2}:\d{2}:\d{2}) (20[0-5]\d) UTC = ... (?:Oct|Mar) (\d{2}) (\d{2}:\d{2}:\d{2}) / && do { my $mm = sprintf "%02d", 3+7*($1 eq 'Oct'); print " <t:t dt=\"$4-$mm-$2T$3\" r=\"$4-$mm-$5T$6\"/>\n"; }'</kbd>
</pre>
<p>Now, we need is a way to run the test suite. In XSLT terms, we say: "to apply a template to the test data". This template will just apply a <code>date-time-Paris</code> call to each of the tests.</p>
<pre class="code vim vimft-xslt">
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Identifier"> </span><span class="Type">match</span>=<span class="Constant">"t:t"</span><span class="Identifier"> </span><span class="Type">xmlns</span><span class="Comment">:</span><span class="Type">t</span>=<span class="Constant">"test:date-time-Paris"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">call-template</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"tap:is"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">with-param</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"got"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">call-template</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"date-time-Paris"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">with-param</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"date-time-Z"</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"@dt"</span><span class="Identifier">/></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">call-template</span><span class="Identifier">></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">with-param</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">with-param</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"expected"</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"@r"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">with-param</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"name"</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"@dt"</span><span class="Identifier">/></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">call-template</span><span class="Identifier">></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Identifier">></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">stylesheet</span><span class="Identifier">></span>
</pre>
<p>We now have a complete stylesheet that can be used as a library in other stylesheet. But we have not yet run the tests. Also you probably wonder what are this XML namespace <code>test:tap</code> and the template called <code>tap:is</code>.</p>
<p><code>test:tap</code> is just a small XSLT test framework that I wrote and the report test results following the <a href="http://en.wikipedia.org/wiki/Test_Anything_Protocol">Test Anything Protocol</a> that is well known by Perl programmers. <code>tap:is</code> is part of the <code>test:tap</code> API and just check for equality and reports in the TAP format.</p>
<pre class="code vim vimft-xslt">
<span class="Comment"><?</span><span class="Type">xml</span><span class="Type"> </span><span class="Type">version</span>=<span class="Constant">"1.0"</span><span class="Type"> </span><span class="Type">encoding</span>=<span class="Constant">"UTF-8"</span><span class="Comment">?></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">stylesheet</span><span class="Identifier"> </span><span class="Type">version</span>=<span class="Constant">"1.0"</span>
<span class="Identifier"> </span><span class="Type">xmlns</span><span class="Comment">:</span><span class="Type">xsl</span>=<span class="Constant">"<a href="http://www.w3.org/1999/XSL/Transform">http://www.w3.org/1999/XSL/Transform</a>"</span>
<span class="Identifier"> </span><span class="Type">xmlns</span><span class="Comment">:</span><span class="Type">tap</span>=<span class="Constant">"test:tap"</span>
<span class="Identifier"> ></span>
<span class="Comment"><!</span><span class="Comment">-- Run all the tests in the test suite --</span><span class="Comment">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Identifier"> </span><span class="Type">match</span>=<span class="Constant">"tap:suite"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">message</span><span class="Identifier">></span>1..<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"count(node()[name()!=''])"</span><span class="Identifier">/></span><span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">message</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">message</span><span class="Identifier">></span># Suite: <span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"@name"</span><span class="Identifier">/></span><span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">message</span><span class="Identifier">></span>
<span class="Comment"><!</span><span class="Comment">-- </span><span class="Todo">TODO</span><span class="Comment"> fix this expression --</span><span class="Comment">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">apply-templates</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"node()[name()!='']"</span><span class="Identifier">/></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Identifier">></span>
<span class="Comment"><!</span><span class="Comment">-- Test Anything Protocol style reporting of tests --</span><span class="Comment">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"tap:is"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">param</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"got"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">param</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"expected"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">param</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"name"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">choose</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">when</span><span class="Identifier"> </span><span class="Type">test</span>=<span class="Constant">"$got = $expected"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">message</span><span class="Identifier">></span>ok <span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"position()"</span><span class="Identifier">/></span> - <span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"concat($name, ' -</span><span class="Type">&</span><span class="Statement">gt</span><span class="Type">;</span><span class="Constant"> ', $got)"</span><span class="Identifier">/></span><span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">message</span><span class="Identifier">></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">when</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">otherwise</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">message</span><span class="Identifier">></span>not ok <span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"position()"</span><span class="Identifier">/></span> - <span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"$name"</span><span class="Identifier">/></span><span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">message</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">message</span><span class="Identifier">></span># got: <span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"$got"</span><span class="Identifier">/></span><span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">message</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">message</span><span class="Identifier">></span># expected: <span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"$expected"</span><span class="Identifier">/></span><span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">message</span><span class="Identifier">></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">otherwise</span><span class="Identifier">></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">choose</span><span class="Identifier">></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Identifier">></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">stylesheet</span><span class="Identifier">></span>
</pre>
<p>We now have two libraries. Let's add a bit more XSLT to glue them together. The main stylesheet applies <code>tap:suite</code> templates to all <code>tap:suite</code> from all imported stylesheets.</p>
<pre class="code vim vimft-xslt">
<span class="Comment"><?</span><span class="Type">xml</span><span class="Type"> </span><span class="Type">version</span>=<span class="Constant">"1.0"</span><span class="Type"> </span><span class="Type">encoding</span>=<span class="Constant">"UTF-8"</span><span class="Comment">?></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">transform</span><span class="Identifier"> </span><span class="Type">version</span>=<span class="Constant">"1.0"</span>
<span class="Identifier"> </span><span class="Type">xmlns</span><span class="Comment">:</span><span class="Type">xsl</span>=<span class="Constant">"<a href="http://www.w3.org/1999/XSL/Transform">http://www.w3.org/1999/XSL/Transform</a>"</span>
<span class="Identifier"> ></span>
<span class="Comment"><!</span><span class="Comment">-- Test framework --</span><span class="Comment">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">import</span><span class="Identifier"> </span><span class="Type">href</span>=<span class="Constant">"tap.xslt"</span><span class="Identifier">/></span>
<span class="Comment"><!</span><span class="Comment">-- All stylesheets to test --</span><span class="Comment">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">import</span><span class="Identifier"> </span><span class="Type">href</span>=<span class="Constant">"date-time-Paris.xslt"</span><span class="Identifier">/></span>
<span class="Comment"><!</span><span class="Comment">-- Launch the tests embedded in libraires, whatever the input is --</span><span class="Comment">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Identifier"> </span><span class="Type">match</span>=<span class="Constant">"/"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">for-each</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"document('')/xsl:transform/xsl:import/@href"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">apply-templates</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"document(.)//tap:suite"</span><span class="Identifier"> </span><span class="Type">xmlns</span><span class="Comment">:</span><span class="Type">tap</span>=<span class="Constant">"test:tap"</span><span class="Identifier">/></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">for-each</span><span class="Identifier">></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Identifier">></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">transform</span><span class="Identifier">></span>
</pre>
<p>We now have everything to run the test suite:</p>
<pre class="terminal unix">
<samp class="prompt unix">$ </samp><kbd>echo '<x/>' | xsltproc tap-run.xslt -
</kbd><samp class="TAP">1..15
# Suite: date-time-Paris
ok 1 - 2008-09-24T21:37:52Z -> 2008-09-24T23:37:52+02:00
ok 2 - 2008-09-24T23:37:52Z -> 2008-09-25T01:37:52+02:00
not ok 3 - 2008-09-24T23:37:52.325Z
# got: 2008-09-25T01:37:52.3249999999998+02:00
# expected: 2008-09-25T01:37:52.325+02:00
ok 4 - 2008-10-01T00:00:00Z -> 2008-10-01T02:00:00+02:00
ok 5 - 2008-10-26T00:59:59Z -> 2008-10-26T02:59:59+02:00
ok 6 - 2008-10-26T01:00:00Z -> 2008-10-26T02:00:00+01:00
ok 7 - 2008-10-31T12:00:00Z -> 2008-10-31T13:00:00+01:00
ok 8 - 2008-11-24T21:37:52Z -> 2008-11-24T22:37:52+01:00
ok 9 - 2008-12-31T23:30:00Z -> 2009-01-01T00:30:00+01:00
ok 10 - 2009-01-01T00:00:00Z -> 2009-01-01T01:00:00+01:00
ok 11 - 2009-03-01T12:00:00Z -> 2009-03-01T13:00:00+01:00
ok 12 - 2009-03-29T00:59:59Z -> 2009-03-29T01:59:59+01:00
ok 13 - 2009-03-29T01:00:00Z -> 2009-03-29T03:00:00+02:00
ok 14 - 2009-03-30T12:00:00Z -> 2009-03-30T14:00:00+02:00
ok 15 - 2009-07-01T00:00:00Z -> 2009-07-01T02:00:00+02:00</samp>
</pre>
<p>So now I know that I have a failing test due to rounding occuring. For my XMLTV project it is not important as I will not have to handle decimal seconds.</p>
http://o.mengue.free.fr/blog/post/2008/09/27/61-playing-with-timezones-and-building-my-own-small-xslt-test-framework#comment-form
http://o.mengue.free.fr/blog/feed/atom/comments/61
Les programmes télé de Télérama en XMLTV
urn:md5:847ff4c10a314f8538e43d2889043dbe
2008-02-11T00:19:00+00:00
2011-06-05T21:14:20+00:00
Olivier Mengué
XMLTV
curlEPGgrille téléHTMLJavaScriptPerlprogramme téléservice webTVtéléTéléramawebdevwebserviceXHTMLXMLXMLTVXSLTxsltproc
<p><a href="http://xmltv.org/wiki/" hreflang="en">XMLTV</a> est un format informatique pour les programmes de télévision. Ce format est utilisable dans un nombre grandissant d'applications. Le magazine <a href="http://www.telerama.fr/">Télérama</a> propose sur son site une <a href="http://television.telerama.fr/tele/grille.php">grille des programmes télé</a>, mais juste consultable sur le web. Pas de XMLTV.</p>
<p>Je vous propose donc de combler ce manque avec un petit cours de reverse engineering sur le web et la démonstration de quelques outils pour arriver rapidement à vos fins. La grille de Télérama est une excellente cible pour ce genre d'exercice parce que, vous le verrez, la tâche n'est pas simple, mais possible. Elle donne l'ocassion de montrer l'usage de plusieurs outils (curl, XSLT, Perl) et c'est un exemple de ma démarche de prototypage rapide en utilisant le meilleur de chacun.</p>
<p><strong>MàJ 2008-02-15 :</strong> j'ai profondément remanié l'introduction et la conclusion suite à quelques commentaires de lecteurs.</p> <p style="text-align: center;"><a href="http://o.mengue.free.fr/xmltv/xsltv/tv.html"><img src="http://o.mengue.free.fr/dolmen.blog/public/20080214-grille-telerama-488.jpg" alt="La grille du site Télérama.fr" /></a></p>
<p>Cette grille fonctionne avec des <a href="http://fr.wikipedia.org/wiki/Service_web">services web</a>. Cela se voit en cliquant sur les flèches à gauche ou à droite du bandeau des heures : seule la grille est rechargée, et non pas la page entière. Un service web est par définition exploitable pour récupérer des données structurées, plus légères que la page entière affichée dans le navigateur.</p>
<p>La première étape : un survol rapide du source HTML de la page. On repère les éléments suivants :</p>
<ol>
<li>La page est très grosse : 208 381 octets. Toute la grille initiale est donc fournie en HTML. Ce sont les mises à jour qui sont fournies en service web.</li>
<li>La page est déclarée comme <a href="http://fr.wikipedia.org/wiki/XHTML">XHTML</a> d'après l'entête :
<pre class="code vim vimft-html"><span class="Comment"><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"</span>
<span class="Comment"> "<a href="http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd</a>"></span>
<span class="Identifier"><</span><span class="Statement">html</span><span class="Identifier"> xmlns=</span><span class="Constant">"<a href="http://www.w3.org/1999/xhtml">http://www.w3.org/1999/xhtml</a>"</span><span class="Identifier"> xml:</span><span class="Type">lang</span><span class="Identifier">=</span><span class="Constant">"fr"</span><span class="Type">lang</span><span class="Identifier">=</span><span class="Constant">"fr"</span><span class="Identifier">></span>
</pre></li>
<li>Plusieurs balises <SCRIPT> en majuscules. Or, en XHTML ces balises doivent être en minuscules. Ceci indique déjà que malgré la déclaration le XHTML n'est pas valide et donc ne sera pas utilisable directement dans un analyseur XML.</li>
<li>Beaucoup de références à des fichiers JavaScript additionnels. Les familiers d'<a href="http://www.ajaxian.com/">Ajaxian.com</a> détectent vite lesquels sont des librairies JavaScript bien connues. Pour les autres une petite recherche sur Google sur le nom du fichier donne rapidement la réponse.</li>
<li>Un bloc de JavaScript révélant l'adresse du service web et le framework utilisé pour l'appeler :
<pre class="code vim vimft-html"><span class="Identifier"><</span><span class="Statement">script</span><span class="Type">type</span><span class="Identifier">=</span><span class="Constant">"text/javascript"</span><span class="Identifier">></span>
<span class="Identifier">var</span><span class="Special"> xajaxRequestUri=</span><span class="Constant">"<a href="http://television.telerama.fr/tele/grille.php">http://television.telerama.fr/tele/grille.php</a>"</span><span class="Special">;</span>
<span class="Identifier">var</span><span class="Special"> xajaxDebug=</span><span class="Constant">false</span><span class="Special">;</span>
<span class="Identifier">var</span><span class="Special"> xajaxStatusMessages=</span><span class="Constant">true</span><span class="Special">;</span>
<span class="Identifier">var</span><span class="Special"> xajaxWaitCursor=</span><span class="Constant">true</span><span class="Special">;</span>
<span class="Identifier">var</span><span class="Special"> xajaxDefinedGet=</span>0<span class="Special">;</span>
<span class="Identifier">var</span><span class="Special"> xajaxDefinedPost=</span>1<span class="Special">;</span>
<span class="Identifier">function</span><span class="Special"> xajax_chargerProgramme</span>()<span class="Identifier">{</span><span class="Statement">return</span><span class="Special"> xajax.call</span>(<span class="Constant">"chargerProgramme"</span><span class="Special">, </span><span class="Identifier">arguments</span><span class="Special">, </span>1)<span class="Special">;</span><span class="Identifier">}</span>
<span class="Identifier"></</span><span class="Statement">script</span><span class="Identifier">></span>
</pre></li>
<li>Des champs de formulaire tel que celui-ci :
<pre class="code vim vimft-html"><span class="Identifier"><</span><span class="Statement">select</span><span class="Type">style</span><span class="Identifier">=</span><span class="Constant">"width: 120px;"</span><span class="Special">onchange="</span><span class="Special">ajouterChaine</span>(<span class="Identifier">this</span><span class="Special">.options</span><span class="Identifier">[</span><span class="Identifier">this</span><span class="Special">.options.selectedIndex</span><span class="Identifier">]</span><span class="Special">.value</span>)<span class="Special">;</span><span class="Special">"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Statement">option</span><span class="Type">value</span><span class="Identifier">=</span><span class="Constant">""</span><span class="Identifier">></span>Ajoutez une cha<span class="Special">&icirc;</span>ne<span class="Identifier"></</span><span class="Statement">option</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Statement">OPTGROUP</span><span class="Type">label</span><span class="Identifier">=</span><span class="Constant">"Groupes de cha</span><span class="Special">&icirc;</span><span class="Constant">nes"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Statement">option</span><span class="Type">value</span><span class="Identifier">=</span><span class="Constant">"192,4,80,34,47,111,118"</span><span class="Identifier">></span>Chaînes hertziennes<span class="Identifier"></</span><span class="Statement">option</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Statement">option</span><span class="Type">value</span><span class="Identifier">=</span><span class="Constant">"192,4,80,34,47,111,118,445,119,195,446,444,78,234,481,226,458,482"</span><span class="Identifier">></span>Chaînes TNT<span class="Identifier"></</span><span class="Statement">option</span><span class="Identifier">></span>
</pre>
On voit qu'il s'agit d'une liste de codes pour les différentes chaînes de télé à afficher.
</li>
<li>Des appels de fonction JavaScript :
<pre class="code vim vimft-html"><span class="Identifier"><</span><span class="Statement">a</span><span class="Type">href</span><span class="Identifier">=</span><span class="Constant">"javascript:void(0);"</span><span class="Special">onclick="</span><span class="Statement">return</span><span class="Special"> changerJour</span>(0)<span class="Special">;</span><span class="Special">"</span><span class="Identifier">></span><span class="Underlined">Aujourd'hui</span><span class="Identifier"></</span><span class="Statement">a</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Statement">a</span><span class="Type">href</span><span class="Identifier">=</span><span class="Constant">"javascript:void(0);"</span><span class="Special">onclick="</span><span class="Statement">return</span><span class="Special"> changerMaintenant</span>(0)<span class="Special">;</span><span class="Special">"</span><span class="Identifier">></span><span class="Underlined">Maintenant</span><span class="Identifier"></</span><span class="Statement">a</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Statement">a</span><span class="Type">href</span><span class="Identifier">=</span><span class="Constant">"javascript:void(0);"</span><span class="Special">onclick="</span><span class="Statement">return</span><span class="Special"> changerCeSoir</span>(0)<span class="Special">;</span><span class="Special">"</span><span class="Identifier">></span><span class="Underlined">Ce soir</span><span class="Identifier"></</span><span class="Statement">a</span><span class="Identifier">></span>
</pre>
On note ainsi quelques noms de fonctions qu'on pourra chercher plus tard.</li>
<li>Le code HTML correspondant aux éléments de la grille. Par exemple :
<pre class="code vim vimft-html"><span class="Identifier"><</span><span class="Statement">div</span><span class="Type">class</span><span class="Identifier">=</span><span class="Constant">"emission magazine"</span><span class="Type">style</span><span class="Identifier">=</span><span class="Constant">"width:16px;left:677px;background-image: url('<a href="http://icon-telerama.sdv.fr/iconsv2/bg_grille_genre_vide.gif">http://icon-telerama.sdv.fr/iconsv2/bg_grille_genre_vide.gif</a>');height:51px; cursor: pointer; z-index: 10;"</span><span class="Type">id</span><span class="Identifier">=</span><span class="Constant">"emission_8069449"</span><span class="Special">onclick="</span><span class="Statement">return</span><span class="Special"> afficherEmission</span>(<span class="Constant">'8069449'</span><span class="Special">, </span><span class="Constant">'226'</span>)<span class="Special">;</span><span class="Special">"</span><span class="Type">alt</span><span class="Identifier">=</span><span class="Constant">"Journal de la nuit - Vendredi 08 février de 02h30 à 02h34"</span><span class="Type">title</span><span class="Identifier">=</span><span class="Constant">"Journal de la nuit - Vendredi 08 février de 02h30 à 02h34"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Statement">div</span><span class="Type">class</span><span class="Identifier">=</span><span class="Constant">"conteneur"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Statement">span</span><span class="Type">class</span><span class="Identifier">=</span><span class="Constant">"genre"</span><span class="Identifier">></span><span class="Identifier"></</span><span class="Statement">span</span><span class="Identifier">></span><span class="Identifier"><</span><span class="Statement">br</span><span class="Identifier"> /></span>
<span class="Identifier"><</span><span class="Statement">span</span><span class="Type">class</span><span class="Identifier">=</span><span class="Constant">"titre"</span><span class="Identifier">></span>...<span class="Identifier"></</span><span class="Statement">span</span><span class="Identifier">></span><span class="Identifier"><</span><span class="Statement">br</span><span class="Identifier"> /></span>
<span class="Identifier"><</span><span class="Statement">span</span><span class="Type">style</span><span class="Identifier">=</span><span class="Constant">"resume"</span><span class="Identifier">></span><span class="Identifier"></</span><span class="Statement">span</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Statement">div</span><span class="Type">id</span><span class="Identifier">=</span><span class="Constant">"data_8069449"</span><span class="Type">style</span><span class="Identifier">=</span><span class="Constant">"display:none;"</span><span class="Identifier">></span>{"Id_Diffusion":"8069449","Id_Emission":"9648923","Id_Chaine":"226","Date_Debut":"2008-02-08 02:30:00","Date_Fin":"2008-02-08 02:34:00","Titre":"Journal de la nuit","Sous_Titre":"","ShowViewFr":"46271175","note_T":"0","Id_Rubrique":"8108","Rubrique_Libelle":"Journal","Rubrique_Niveau":"3","Type":"Magazine","Chaine_Nom":"i Télé","Logo":"226.gif","Id_Hierarchie":"040801","DureeEnSecondes":"240","resume_court":"","resume_long":"","dateheurechaine":"Vendredi 08 février de 02h30 à 02h34 sur i Télé","intervenant":""}<span class="Identifier"></</span><span class="Statement">div</span><span class="Identifier">></span>
<span class="Identifier"></</span><span class="Statement">div</span><span class="Identifier">></span>
</pre>
On remarque en particulier l'élement <code>data_8069449</code> qui n'est pas affiché et qui ressemble à des données au format <a href="http://fr.wikipedia.org/wiki/JSON">JSON</a>.
</li>
</ol>
<p>L'étape suivante consiste à récupérer une copie de la page et de tous les scripts additionnels, feuilles de style et images. Il suffit dans Firefox de choisir le menu Fichier → Enregistrer sous... → Page complète.</p>
<p>On obtient les fichiers suivants :</p>
<pre class="terminal"><samp class="prompt">$ </samp><kbd>ls</kbd>
<samp>grille.php.html grille.php_fichiers/</samp>
<samp class="prompt">$ </samp><kbd>ls grille.php_fichiers/</kbd>
<samp>1520033938Top1Bottom2x01x10.js mochikit_packed_tele_min.js
AC_RunActiveContent_min.js notes.js
ajax-loader.gif oas.js
alerte.js ong_montel_off.gif
builder.js ong_podcast_off.gif
carrousel_min.js ong_telerama_on.gif
controls.js ong_wiz_off.gif
couverture.gif pied.js
dragdrop.js prototype.js
effects.js rss.gif
empty.gif scriptaculous.js
favoris.js scriptaculous_packed_tele_min.js
fleche_droite_bottom1.gif scripts_2007.js
fleche_gauche_bottom1.gif slider.js
grille_croix.gif sound.js
grille_heures_fleche_dro.gif styles_2007.css
grille_heures_fleche_gau.gif T1_p.gif
heureFloatEntete.js T2_p.gif
hit.gif T3_p.gif
horsserie.js T5_p.gif
js_tra_authentification.html trans.gif
M5447.jpg tra_tele_v2.js
M5456.jpg urchin.js
M5495.jpg xajax.js
menu.js xtroi.js
milieu.js</samp>
<samp class="prompt">$ </samp><kbd>du -b -c grille.php.html grille.php_fichiers/</kbd>
<samp>210394 grille.php.html
546599 grille.php_fichiers/
756993 total</samp></pre>
<p>On peut alors les trier en trois catégories :</p>
<ol>
<li>les librairies : Scriptaculous, Prototype, Xajax, MochiKit. Repérer les librairies permet d'aller sur le web chercher la doc des fonctions et des objets et d'éviter de plonger le nez dans du JavaScript compressé.</li>
<li>les pubs et les pisteurs : on les reconnaît au faible nombre de lignes et à la piètre qualité du code écrit une fois il y a dix ans et jamais mis à jour en fonction des nouvelles normes (XHTML, DOM). Exemple : <a href="view-source:http://statique-telerama.sdv.fr/scripts/oas.js">oas.js</a></li>
<li>les scripts spécifiques au site : non compressé, avec un nom de fichier, des commentaires ou des noms de fonctions en français, ou contenant des noms de fonctions repérés dans des gestionnaires d'évènements attachés à un tag HTML (par exemple, <code>changerMaintenant()</code> vu plus haut). Chercher le nom du site (<code>grep telerama *.js</code>) permet aussi de les isoler. Ce sont bien sûr les plus importants puisque c'est là que l'on trouve les adresses des services web et la manière de les appeler.</li>
</ol>
<p>Ce premier aperçu permet de se donner une idée de la compétence du/des développeurs. En l'occurence :</p>
<ul>
<li>Malgré la déclaration de l'entête, <a href="http://validator.w3.org/check?verbose=1&uri=http%3A%2F%2Ftelevision.telerama.fr%2Ftele%2Fgrille.php">la page n'est pas du XHTML valide</a>. La validation avec <a href="http://validator.w3.org/">l'outil du W3C</a> révèle ainsi 384 erreurs !</li>
<li>Le fichier HTML est énorme (200 Ko), le nombre de fichiers également. Le poids total de la page est de plus de 800 Ko ! On a beau être à l'heure de l'ADSL en France, ce n'est pas une raison.</li>
<li>De nombreuses librairies JavaScript sont utilisées. Leurs fonctions se recouvrent et sont redondantes. Il existe au moins 3 façons différentes de faire un appel AJAX. Est-ce que plusieurs développeurs ont travaillé sur cette page successivement ? En tout cas le manque de coordination est flagrant.</li>
<li>Les tags XHTML sont détournés pour stocker des données JSON contenant elles-même du HTML (et non du XHTML). Pas étonnant que cela ne valide pas !</li>
<li>Beaucoup de code mort laissé en commentaires dans le JavaScript.</li>
</ul>
<p>En somme, pas brillant.</p>
<p>Nous avons repéré plus haut l'adresse d'un service web. Il est temps de le tester, en dehors du navigateur pour examiner les données retournées. Ce service est appellé avec <a href="http://xajaxproject.org/">Xajax</a>. En fouinant dans xajax.js, on trouve vite (cherchez "postData") la façon dont Xajax envoie les données au serveur. On peut alors reconstruire une requête avec <a href="http://curl.hacxx.se/">curl</a> :</p>
<pre class="terminal"><samp class="prompt">$ </samp><kbd>curl --data "xajax=chargerProgramme&xajaxr=$(date +%s)&xajaxargs[]=$(date '+%Y-%m-%d %H:00:00')&xajaxargs[]=192,4,80,34,47,111,118" http://television.telerama.fr/tele/grille.php</kbd>
</pre>
<p>On obtient cette fois du beau XML valide dont voici la structure :</p>
<pre class="code vim vimft-xml"><span class="Identifier"><</span><span class="Identifier">xjx</span><span class="Identifier">><</span><span class="Identifier">cmd</span><span class="Type">n</span>=<span class="Constant">"as"</span><span class="Type">t</span>=<span class="Constant">"leprogramme"</span><span class="Type">p</span>=<span class="Constant">"innerHTML"</span><span class="Identifier">></span><span class="Type"><![</span><span class="Statement">CDATA</span><span class="Type">[</span><var>...</var><span class="Type">]]></span><span class="Identifier"></cmd></span><var>...</var><span class="Identifier"></xjx></span>
</pre>
<p>Le contenu de <code>leprogramme</code> est le même que ce que l'on trouve dans la page HTML de base. On retrouve notamment la description des programmes à la fois en HTML et en JSON. Ce n'est pas vraiment efficace en terme de quantité de données transférées sur le réseau. Les données JSON aurait été suffisantes, et je l'avoue, cela m'aurait facilité la tâche.</p>
<p>Il suffit d'appliquer le filtre XSLT suivant (<a href="http://o.mengue.free.fr/xmltv/leprogramme.xslt" type="text/xsl">leprogramme.xslt</a>) pour extraire le contenu de <code>leprogramme</code> :</p>
<pre class="code vim vimft-xslt"><span class="Comment"><?</span><span class="Type">xml</span><span class="Type">version</span>=<span class="Constant">"1.0"</span><span class="Type">encoding</span>=<span class="Constant">"UTF-8"</span><span class="Comment">?></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">stylesheet</span><span class="Type">version</span>=<span class="Constant">"1.0"</span><span class="Type">xmlns</span><span class="Comment">:</span><span class="Type">xsl</span>=<span class="Constant">"<a href="http://www.w3.org/1999/XSL/Transform">http://www.w3.org/1999/XSL/Transform</a>"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">output</span><span class="Type">method</span>=<span class="Constant">"text"</span><span class="Type">encoding</span>=<span class="Constant">"UTF-8"</span><span class="Identifier"> /></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">strip-space</span><span class="Type">elements</span>=<span class="Constant">"*"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Type">match</span>=<span class="Constant">"/xjx/cmd[@n='as']"</span><span class="Identifier">></span><span class="Type">&</span><span class="Statement">lt</span><span class="Type">;</span>!DOCTYPE html [
<span class="Type">&</span><span class="Statement">lt</span><span class="Type">;</span>!ENTITY nbsp "<span class="Type">&</span><span class="Statement">amp</span><span class="Type">;</span>#160;"<span class="Type">&</span><span class="Statement">gt</span><span class="Type">;</span>
]<span class="Type">&</span><span class="Statement">gt</span><span class="Type">;</span>
<span class="Type">&</span><span class="Statement">lt</span><span class="Type">;</span>html<span class="Type">&</span><span class="Statement">gt</span><span class="Type">;&</span><span class="Statement">lt</span><span class="Type">;</span>body<span class="Type">&</span><span class="Statement">gt</span><span class="Type">;</span><span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Type">select</span>=<span class="Constant">"text()"</span><span class="Identifier">/></span><span class="Type">&</span><span class="Statement">lt</span><span class="Type">;</span>/body<span class="Type">&</span><span class="Statement">gt</span><span class="Type">;&</span><span class="Statement">lt</span><span class="Type">;</span>/html<span class="Type">&</span><span class="Statement">gt</span><span class="Type">;</span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Type">match</span>=<span class="Constant">"/xjx/cmd[@n!='as']"</span><span class="Identifier">/></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">stylesheet</span><span class="Identifier">></span>
</pre>
<p>Notez la génération d'un DOCTYPE définissant l'entité "nbsp", définie en HTML mais pas en XML.</p>
<p>Pour l'appliquer :</p>
<pre class="terminal"><samp class="prompt">$ </samp><kbd>xsltproc <a href="http://o.mengue.free.fr/xmltv/leprogramme.xslt" type="text/xsl">leprogramme.xslt</a> a.xml > b.xml</kbd>
</pre>
<p>On peut maintenant ouvrir ce fichier dans un navigateur pour vérifier que le document XML est valide. Ce n'est pas le cas :</p>
<ul>
<li>Il y a des entités mal formées (voir par exemple l'émission D&CO le dimanche soir sur M6)</li>
<li>Les blocs JSON contiennent contiennent des tags <code><br></code> non fermés.</li>
</ul>
<p>De plus, il serait bien utile d'extraire les données JSON sous forme de XML pour pouvoir continuer notre prototypage avec XSLT. Appellons Perl et le module JSON à la rescousse !</p>
<pre class="terminal"><samp class="prompt">$ </samp><kbd>xsltproc leprogramme.xslt a.xml | perl -M<a href="http://search.cpan.org/%7Emakamaka/JSON-2.06/lib/JSON.pm">JSON</a> -npe '
<span class="code vim vimft-perl"><span class="Statement">if</span> (m@^ *<div id=<span class="Constant">"</span><span class="Constant">(data_[^</span><span class="Constant">"</span>]*)<span class="Constant">"</span><span class="Constant"> style=</span><span class="Constant">"</span>display:none;<span class="Constant">"</span><span class="Constant">>([^}]*})</div>@) {</span>
<span class="Constant"> (</span><span class="Identifier">$id</span><span class="Constant">,</span><span class="Identifier">$data</span><span class="Constant">)=(</span><span class="Identifier">$1</span><span class="Constant">,jsonToObj(</span><span class="Identifier">$2</span><span class="Constant">));</span>
<span class="Identifier">$data</span><span class="Constant"> = join(</span><span class="Constant">"</span>\n<span class="Constant">"</span><span class="Constant">, map { qq|<div class=</span><span class="Constant">"</span><span class="Identifier">$_</span><span class="Constant">"</span><span class="Constant">></span><span class="Identifier">$data->{$_}</span><span class="Constant"></div>| } keys %</span><span class="Identifier">$data</span><span class="Constant">);</span>
<span class="Identifier">$_</span><span class="Constant"> = </span><span class="Constant">"</span><div id=\<span class="Constant">"</span><span class="Identifier">$id</span><span class="Special">\"</span><span class="Constant">></span><span class="Identifier">$data</span><span class="Constant"></div></span><span class="Constant">"</span>;
s@<span class="Identifier"><br></span>@<br/><span class="Identifier">@g</span>
}
<span class="Statement">s/</span><span class="Constant">&</span><span class="Special">([^;]{8})</span><span class="Statement">/</span><span class="Constant">&amp;</span><span class="Identifier">$1</span><span class="Statement">/g</span>;</span>' > c.xml</kbd>
</pre>
<p>Lançons un petit test d'extraction XSLT : nous allons lister les chaînes avec <a href="http://o.mengue.free.fr/xmltv/chaines.xslt" type="text/xsl"><code>chaines.xslt</code></a>.</p>
<pre class="code vim vimft-xslt"><span class="Comment"><?</span><span class="Type">xml</span><span class="Type">version</span>=<span class="Constant">"1.0"</span><span class="Type">encoding</span>=<span class="Constant">"UTF-8"</span><span class="Comment">?></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">stylesheet</span><span class="Type">version</span>=<span class="Constant">"1.0"</span><span class="Type">xmlns</span><span class="Comment">:</span><span class="Type">xsl</span>=<span class="Constant">"<a href="http://www.w3.org/1999/XSL/Transform">http://www.w3.org/1999/XSL/Transform</a>"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">output</span><span class="Type">method</span>=<span class="Constant">"text"</span><span class="Type">encoding</span>=<span class="Constant">"UTF-8"</span><span class="Identifier"> /></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Type">match</span>=<span class="Constant">"/html/body/div"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Type">select</span>=<span class="Constant">"substring(@id, 6)"</span><span class="Identifier">/></span>: <span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Type">select</span>=<span class="Constant">"div[@class='chaine']/@title"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">text</span><span class="Identifier">></span><span class="Type">&</span><span class="Statement">#10</span><span class="Type">;</span><span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">text</span><span class="Identifier">></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Identifier">></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">stylesheet</span><span class="Identifier">></span>
</pre>
<p>Et voici le résultat :</p>
<pre class="terminal"><samp class="prompt">$ </samp><kbd>xsltproc <a href="http://o.mengue.free.fr/xmltv/chaines.xslt" type="text/xsl"><code>chaines.xslt</code></a> b.xml</kbd>
<samp>192: TF 1
4: France 2
80: France 3
34: Canal+
47: France 5
111: Arte
118: M6</samp></pre>
<p>OK. Nous avons le champ libre pour transformer notre document en XMLTV avec <a href="http://o.mengue.free.fr/xmltv/xmltv.xslt"><code>xmltv.xslt</code></a> :</p>
<pre class="code vim vimft-xslt"><span class="Comment"><?</span><span class="Type">xml</span><span class="Type">version</span>=<span class="Constant">"1.0"</span><span class="Type">encoding</span>=<span class="Constant">"UTF-8"</span><span class="Comment">?></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">stylesheet</span><span class="Type">version</span>=<span class="Constant">"1.0"</span><span class="Type">xmlns</span><span class="Comment">:</span><span class="Type">xsl</span>=<span class="Constant">"<a href="http://www.w3.org/1999/XSL/Transform">http://www.w3.org/1999/XSL/Transform</a>"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">output</span><span class="Type">method</span>=<span class="Constant">"xml"</span><span class="Type">indent</span>=<span class="Constant">"yes"</span><span class="Type">encoding</span>=<span class="Constant">"UTF-8"</span><span class="Identifier"> /></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">strip-space</span><span class="Type">elements</span>=<span class="Constant">"*"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Type">match</span>=<span class="Constant">"/html"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Identifier">tv</span><span class="Type">generator-info-name</span>=<span class="Constant">"Télérama → XMLTV"</span>
<span class="Type">source-info-url</span>=<span class="Constant">"<a href="http://television.telerama.fr/tele/grille.php">http://television.telerama.fr/tele/grille.php</a>"</span>
<span class="Type">source-info-name</span>=<span class="Constant">"Télérama"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">apply-templates</span><span class="Type">mode</span>=<span class="Constant">"channel"</span><span class="Type">select</span>=<span class="Constant">"body/div/div[@class='chaine']"</span><span class="Identifier"> /></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">apply-templates</span><span class="Type">mode</span>=<span class="Constant">"programme"</span><span class="Type">select</span>=<span class="Constant">"body/div/div/div/div[contains(@class, 'emission')]/div/div[substring(@id, 0, 6)='data_']"</span><span class="Identifier"> /></span>
<span class="Identifier"></tv></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Type">match</span>=<span class="Constant">"text()"</span><span class="Type">priority</span>=<span class="Constant">"-1"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Type">mode</span>=<span class="Constant">"channel"</span><span class="Type">match</span>=<span class="Constant">"div"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Identifier">channel</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">attribute</span><span class="Type">name</span>=<span class="Constant">"id"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Type">select</span>=<span class="Constant">"substring(../@id, 6)"</span><span class="Identifier">/></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">attribute</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Identifier">display-name</span><span class="Type">lang</span>=<span class="Constant">"fr"</span><span class="Identifier">><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Type">select</span>=<span class="Constant">"./@title"</span><span class="Identifier">/></span><span class="Identifier"></display-name></span>
<span class="Identifier"><</span><span class="Identifier">icon</span><span class="Type">width</span>=<span class="Constant">"40"</span><span class="Type">height</span>=<span class="Constant">"40"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">attribute</span><span class="Type">name</span>=<span class="Constant">"src"</span><span class="Identifier">></span><a href="http://icon-telerama.sdv.fr/tele/imedia/images_chaines_tra/Transparent/40x40/">http://icon-telerama.sdv.fr/tele/imedia/images_chaines_tra/Transparent/40x40/</a><span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Type">select</span>=<span class="Constant">"substring(../@id, 6)"</span><span class="Identifier">/></span>.gif<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">attribute</span><span class="Identifier">></span>
<span class="Identifier"></icon></span>
<span class="Identifier"></channel></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Type">mode</span>=<span class="Constant">"programme"</span><span class="Type">match</span>=<span class="Constant">"div"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Identifier">programme</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">attribute</span><span class="Type">name</span>=<span class="Constant">"channel"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Type">select</span>=<span class="Constant">"div[@class='Id_Chaine']"</span><span class="Identifier">/></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">attribute</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">attribute</span><span class="Type">name</span>=<span class="Constant">"start"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">apply-templates</span><span class="Type">mode</span>=<span class="Constant">"xmltv-time-from-iso-8601"</span><span class="Type">select</span>=<span class="Constant">"div[@class='Date_Debut']"</span><span class="Identifier">/></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">attribute</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">attribute</span><span class="Type">name</span>=<span class="Constant">"stop"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">apply-templates</span><span class="Type">mode</span>=<span class="Constant">"xmltv-time-from-iso-8601"</span><span class="Type">select</span>=<span class="Constant">"div[@class='Date_Fin']"</span><span class="Identifier">/></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">attribute</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">attribute</span><span class="Type">name</span>=<span class="Constant">"showview"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Type">select</span>=<span class="Constant">"div[@class='ShowViewFr']"</span><span class="Identifier">/></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">attribute</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Identifier">title</span><span class="Type">lang</span>=<span class="Constant">"fr"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Type">select</span>=<span class="Constant">"div[@class='Titre']"</span><span class="Identifier">/></span>
<span class="Identifier"></title></span>
<span class="Identifier"><</span><span class="Identifier">sub-title</span><span class="Type">lang</span>=<span class="Constant">"fr"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Type">select</span>=<span class="Constant">"div[@class='Sous_Titre']"</span><span class="Identifier">/></span>
<span class="Identifier"></sub-title></span>
<span class="Identifier"><</span><span class="Identifier">desc</span><span class="Type">lang</span>=<span class="Constant">"fr"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Type">select</span>=<span class="Constant">"div[@class='resume_long']"</span><span class="Identifier">/></span>
<span class="Identifier"></desc></span>
<span class="Identifier"><</span><span class="Identifier">category</span><span class="Type">lang</span>=<span class="Constant">"fr"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Type">select</span>=<span class="Constant">"div[@class='Type']"</span><span class="Identifier">/></span>
<span class="Identifier"></category></span>
<span class="Identifier"><</span><span class="Identifier">length</span><span class="Type">units</span>=<span class="Constant">"seconds"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Type">select</span>=<span class="Constant">"div[@class='DureeEnSecondes']"</span><span class="Identifier">/></span>
<span class="Identifier"></length></span>
<span class="Identifier"></programme></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Type">mode</span>=<span class="Constant">"xmltv-time-from-iso-8601"</span><span class="Type">match</span>=<span class="Constant">"*|@*"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Type">select</span>=<span class="Constant">"concat(substring(.,0,5),substring(.,6,2),substring(9,2),substring(.,12,2),substring(.,15,2))"</span><span class="Identifier">/></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Identifier">></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">stylesheet</span><span class="Identifier">></span>
</pre>
<p>Il n'y a plus qu'à enchaîner toutes les étapes dans un script shell (<a href="http://o.mengue.free.fr/xmltv/tv.sh"><code>tv.sh</code></a>) :</p>
<pre class="terminal unix"><samp class="prompt">$ </samp><kbd>./<a href="http://o.mengue.free.fr/xmltv/tv.sh">tv.sh</a> tv.2008-02-14T23:00:00.xml 2008-02-14T23:00:00</kbd>
</pre>
<p>Et voici quelques extraits du résultat :</p>
<pre class="code vim vimft-xml"><span class="Comment"><?</span><span class="Type">xml</span><span class="Type">version</span>=<span class="Constant">"1.0"</span><span class="Type">encoding</span>=<span class="Constant">"UTF-8"</span><span class="Comment">?></span>
<span class="Identifier"><</span><span class="Identifier">tv</span><span class="Type">generator-info-name</span>=<span class="Constant">"Télérama → XMLTV"</span><span class="Type">source-info-url</span>=<span class="Constant">"<a href="http://television.telerama.fr/tele/grille.php">http://television.telerama.fr/tele/grille.php</a>"</span><span class="Type">source-info-name</span>=<span class="Constant">"Télérama"</span><span class="Identifier">></span>
<span class="Comment"><!</span><span class="Comment">-- […] --</span><span class="Comment">></span>
<span class="Identifier"><</span><span class="Identifier">channel</span><span class="Type">id</span>=<span class="Constant">"111"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Identifier">display-name</span><span class="Type">lang</span>=<span class="Constant">"fr"</span><span class="Identifier">></span>Arte<span class="Identifier"></display-name></span>
<span class="Identifier"><</span><span class="Identifier">icon</span><span class="Type">width</span>=<span class="Constant">"40"</span><span class="Type">height</span>=<span class="Constant">"40"</span><span class="Type">src</span>=<span class="Constant">"<a href="http://icon-telerama.sdv.fr/tele/imedia/images_chaines_tra/Transparent/40x40/111.gif">http://icon-telerama.sdv.fr/tele/imedia/images_chaines_tra/Transparent/40x40/111.gif</a>"</span><span class="Identifier">/></span>
<span class="Identifier"></channel></span>
<span class="Comment"><!</span><span class="Comment">-- […] --</span><span class="Comment">></span>
<span class="Identifier"><</span><span class="Identifier">programme</span><span class="Type">channel</span>=<span class="Constant">"111"</span><span class="Type">start</span>=<span class="Constant">"2008020000"</span><span class="Type">stop</span>=<span class="Constant">"2008020220"</span><span class="Type">showview</span>=<span class="Constant">"2779370"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Identifier">title</span><span class="Type">lang</span>=<span class="Constant">"fr"</span><span class="Identifier">></span>Arizona Dream<span class="Identifier"></title></span>
<span class="Identifier"><</span><span class="Identifier">sub-title</span><span class="Type">lang</span>=<span class="Constant">"fr"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Identifier">desc</span><span class="Type">lang</span>=<span class="Constant">"fr"</span><span class="Identifier">></span>Axel, un orphelin de 20 ans, vit davantage dans un monde rempli de rêves et de poissons volants qu'à New York, où il habite. Par l'esprit, il se «transporte» souvent sur la banquise lointaine et ...<span class="Identifier"></desc></span>
<span class="Identifier"><</span><span class="Identifier">category</span><span class="Type">lang</span>=<span class="Constant">"fr"</span><span class="Identifier">></span>Film<span class="Identifier"></category></span>
<span class="Identifier"><</span><span class="Identifier">length</span><span class="Type">units</span>=<span class="Constant">"seconds"</span><span class="Identifier">></span>8400<span class="Identifier"></length></span>
<span class="Identifier"></programme></span>
<span class="Comment"><!</span><span class="Comment">-- […] --</span><span class="Comment">></span>
<span class="Identifier"></tv></span>
</pre>
<p>Pour visualiser le résultat obtenu sous une forme plus lisible pour un humain, je vous recommande d'essayer <a href="http://www.ericandchar.com/xsltvgrid/" hreflang="en">XSLTv</a>. Vous enregistrez le fichier XMLTV sous <code>tv.xml</code> dans le répertoire de XSLTv et vous chargez <code>tv.html</code> dans votre navigateur. Vous obtenez <a href="http://o.mengue.free.fr/xmltv/xsltv/tv.html" rel="noindex nofollow">ceci</a> :</p>
<p style="text-align: center;"><a href="http://o.mengue.free.fr/xmltv/xsltv/tv.html" rel="noindex nofollow"><img src="http://o.mengue.free.fr/blog.dc/images/20080214-grille-telerama-xsltv-488.jpg" alt="Le fichier XMLTV visualisé avec XSLTv" /></a></p>
<p>Ce n'est pas visuellement aussi beau que le site Télérama, mais pensez que ce n'est que l'une des multiples utilisations du format XMLTV : ces données peuvent aussi être utilisées par exemple pour piloter un magnétoscope numérique.</p>
<p>L'étape suivante sera de programmer un proxy en PHP, mais je vous en parlerais plus tard.</p>
<hr>
<p>Fichiers joints :</p>
<ul>
<li><a href="http://o.mengue.free.fr/xmltv/leprogramme.xslt" type="text/xsl"><code>leprogramme.xslt</code></a></li>
<li><a href="http://o.mengue.free.fr/xmltv/chaines.xslt" type="text/xsl"><code>chaines.xslt</code></a></li>
<li><a href="http://o.mengue.free.fr/xmltv/xmltv.xslt" type="text/xsl"><code>xmltv.xslt</code></a></li>
<li><a href="http://o.mengue.free.fr/xmltv/tv.sh" type="application/x-shellscript"><code>tv.sh</code></a></li>
</ul>
<p><strong>MàJ 2008-03-02 :</strong> J'ai mis à jour tv.sh et xmltv.xslt. Voir <a href="http://o.mengue.free.fr/blog/2008/03/02/53-telerama-xmltv-mise-a-jour">ce billet</a>.</p>
http://o.mengue.free.fr/blog/post/2008/02/11/51-les-programmes-tele-de-telerama-en-xmltv#comment-form
http://o.mengue.free.fr/blog/feed/atom/comments/51
Stations Vélib’ : v2.0
urn:md5:2e4ce0720926ff2f6a656953fbe250f0
2007-08-12T23:15:52+00:00
2009-02-08T22:28:43+00:00
Olivier Mengué
Code
cartecodehackMaemoMaemo MappermapNokia N800ParisPOIVélib'XSLT
<p>Voici une nouvelle version de mon script de conversion des stations Vélib’ en base POI pour Maemo Mapper sur Nokia N800 (voir <a href="http://o.mengue.free.fr/blog/2007/08/02/32-une-carte-des-stations-velib-sur-nokia-n800">mon post précédent</a>).</p>
<p>Parmi les améliorations :</p>
<ul>
<li>un script pour automatiser le processus de transformation </li>
<li>le renseignement du champ <code>poi_id</code>, ce qui corrige le problème d'édition des POI </li>
<li>un index sur la table <code>POI</code> sur les champs latitude, longitude en espérant que cela améliore les temps d'accès. En pratique, je n'ai constaté aucune amélioration ;</li>
<li>4 catégories de POI au lieu d'une seule, pour créer votre propre base des stations : Station Vélib’, Station Vélib’ corrigée, Station Vélib’ validée, Station Vélib’ erronée ;</li>
<li>des modifications d'implémentaiton XSLT : l'utilisation de <a href="http://www.w3.org/TR/xslt#for-each"><code>xsl:foreach</code></a> et un traitement exhaustif des caractères de l'adresse de la station.</li>
</ul>
<p>Lisez la suite pour le nouveau code...</p> <p><strong>Relisez d'abord le <a href="http://o.mengue.free.fr/blog/2007/08/02/32-une-carte-des-stations-velib-sur-nokia-n800">post précédent</a> comme introduction.</strong></p>
<p>Voici le script pour Windows <code>stations2mm.cmd</code> :</p>
<pre class="code vim vimft-dosbatch">
@<span class="Identifier">echo</span><span class="Operator"> off</span>
xsltproc -o <span class="Identifier">%~dpn1</span>.sql <span class="Identifier">%~dpn0</span>.xslt <span class="Identifier">%1</span> || <span class="Statement">goto</span><span class="Label"> :EOF</span>
<span class="Statement">if</span><span class="Operator"> exist</span> velib-poi.db <span class="Identifier">del</span> velib-poi.db
<span class="Identifier">echo</span><span class="Constant"> .q</span>| sqlite3 -batch -init <span class="Identifier">%~dpn1</span>.sql velib-poi.db
</pre>
<p>Pour le lancer :</p>
<pre class="terminal cmd">
<samp class="prompt">C:\Users\Moi\Documents\Vélib'></samp><kbd>stations2mm stations.xml</kbd>
</pre>
<p>Voici la nouvelle feuille de style <code>stations2mm.xslt</code>. Le nombre de lignes a doublé simplement pour l'échappement correct des apostrophes.</p>
<pre class="code vim vimft-xslt">
<span class="Comment"><?</span><span class="Type">xml</span><span class="Type"> </span><span class="Type">version</span>=<span class="Constant">"1.0"</span><span class="Type"> </span><span class="Type">encoding</span>=<span class="Constant">"UTF-8"</span><span class="Comment">?></span>
<span class="Comment"><!</span><span class="Comment">-- stations2mm.xslt --</span><span class="Comment">></span>
<span class="Comment"><!</span><span class="Comment">-- Copyright (c) 2007 Olivier Mengué <a href="http://o.mengue.free.fr/">http://o.mengue.free.fr/</a> --</span><span class="Comment">></span>
<span class="Comment"><!</span><span class="Comment">--</span>
<span class="Comment"> Maemo Mapper POI format: <a href="http://eko.one.pl/index.php?page=Nokia770_software#Details">http://eko.one.pl/index.php?page=Nokia770_software#Details</a></span>
<span class="Comment">--</span><span class="Comment">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">stylesheet</span><span class="Identifier"> </span><span class="Type">version</span>=<span class="Constant">"1.0"</span><span class="Identifier"> </span><span class="Type">xmlns</span><span class="Comment">:</span><span class="Type">xsl</span>=<span class="Constant">"<a href="http://www.w3.org/1999/XSL/Transform">http://www.w3.org/1999/XSL/Transform</a>"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">output</span><span class="Identifier"> </span><span class="Type">method</span>=<span class="Constant">"text"</span><span class="Identifier"> </span><span class="Type">encoding</span>=<span class="Constant">"UTF-8"</span><span class="Identifier"> /></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">strip-space</span><span class="Identifier"> </span><span class="Type">elements</span>=<span class="Constant">"*"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Identifier"> </span><span class="Type">match</span>=<span class="Constant">"/markers"</span><span class="Identifier">></span>-- Script SQLite pour construire une base POI des stations Vélib pour Maemo
-- Mapper <a href="http://gnuite.com:8080/nokia770/maemo-mapper/">http://gnuite.com:8080/nokia770/maemo-mapper/</a> (Nokia 770, N800).
-- See <a href="http://eko.one.pl/index.php?page=Nokia770_software#Details">http://eko.one.pl/index.php?page=Nokia770_software#Details</a>
--
-- Créé avec stations2mm.xslt (<a href="http://o.mengue.free.fr/)">http://o.mengue.free.fr/)</a>
--
-- Pour céer le fichier:
-- echo .q | sqlite3 -batch -init stations.sql poi.db
PRAGMA encoding = "UTF-8";
CREATE TABLE IF NOT EXISTS category (cat_id INTEGER PRIMARY KEY, label TEXT NOT NULL, desc TEXT, enabled INTEGER NOT NULL);
CREATE TABLE IF NOT EXISTS poi (poi_id INTEGER PRIMARY KEY, lat REAL NOT NULL, lon REAL NOT NULL, label TEXT NOT NULL, desc TEXT, cat_id INTEGER NOT NULL);
CREATE INDEX IF NOT EXISTS poi_idx_latlon ON poi (lat ASC, lon ASC);
-- Placez 'Velib.jpg' dans un sous-répertoire 'poi' de votre dépôt de cartes
-- Pas d'accent dans label
INSERT INTO category (cat_id, label, desc, enabled) VALUES (30001, 'Velib', 'Station Vélib''', 1);
INSERT INTO category (cat_id, label, desc, enabled) VALUES (30002, 'Velib2', 'Station Vélib'' corrigée', 1);
INSERT INTO category (cat_id, label, desc, enabled) VALUES (30003, 'Velib3', 'Station Vélib'' validée', 1);
INSERT INTO category (cat_id, label, desc, enabled) VALUES (30004, 'Velib4', 'Station Vélib'' erronée', 1);
<span class="Comment"><!</span><span class="Comment">--</span>
<span class="Comment">Ignore les stations dont les coordonnées ne sont pas définies</span>
<span class="Comment"> lat=27.1409733745, lng=-3.40456062425</span>
<span class="Comment">--</span><span class="Comment">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">for-each</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"marker[@lat!='27.1409733745']"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">sort</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"@lat"</span><span class="Identifier"> </span><span class="Type">data-type</span>=<span class="Constant">"number"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">sort</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"@lng"</span><span class="Identifier"> </span><span class="Type">data-type</span>=<span class="Constant">"number"</span><span class="Identifier">/></span><span class="Comment"><!</span><span class="Comment">--</span>
<span class="Comment"> --</span><span class="Comment">></span>INSERT INTO poi (poi_id, lat, lon, label, desc, cat_id) VALUES (<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"30000+position()"</span><span class="Identifier">/></span>, <span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"@lat"</span><span class="Identifier">/></span>, <span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"@lng"</span><span class="Identifier">/></span>, '<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"@number"</span><span class="Identifier">/></span>', '<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">call-template</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"globalReplace"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">with-param</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"input"</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"@address"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">with-param</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"search"</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"</span><span class="Type">&</span><span class="Statement">quot</span><span class="Type">;</span><span class="Constant">'</span><span class="Type">&</span><span class="Statement">quot</span><span class="Type">;</span><span class="Constant">"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">with-param</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"replacement"</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"</span><span class="Type">&</span><span class="Statement">quot</span><span class="Type">;</span><span class="Constant">''</span><span class="Type">&</span><span class="Statement">quot</span><span class="Type">;</span><span class="Constant">"</span><span class="Identifier">/></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">call-template</span><span class="Identifier">></span>', 30001);
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">for-each</span><span class="Identifier">></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Identifier">></span>
<span class="Comment"><!</span><span class="Comment">-- Voir <a href="http://www.xml.com/pub/a/2002/06/05/transforming.html">http://www.xml.com/pub/a/2002/06/05/transforming.html</a> --</span><span class="Comment">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"globalReplace"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">param</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"input"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">param</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"search"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">param</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"replacement"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">choose</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">when</span><span class="Identifier"> </span><span class="Type">test</span>=<span class="Constant">"contains($input, $search)"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Identifier"> </span><span class="Type">select</span>=
<span class="Identifier"> </span><span class="Constant">"concat(substring-before($input, $search),</span>
<span class="Constant"> $replacement)"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">call-template</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"globalReplace"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">with-param</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"input"</span>
<span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"substring-after($input, $search)"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">with-param</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"search"</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"$search"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">with-param</span><span class="Identifier"> </span><span class="Type">name</span>=<span class="Constant">"replacement"</span>
<span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"$replacement"</span><span class="Identifier">/></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">call-template</span><span class="Identifier">></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">when</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">otherwise</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Identifier"> </span><span class="Type">select</span>=<span class="Constant">"$input"</span><span class="Identifier">/></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">otherwise</span><span class="Identifier">></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">choose</span><span class="Identifier">></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Identifier">></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">stylesheet</span><span class="Identifier">></span>
</pre>
Une carte des stations Vélib’ sur Nokia N800
urn:md5:9a8a308a1b23081a458724a88bdbdce7
2007-08-02T23:17:00+00:00
2011-06-05T21:12:20+00:00
Olivier Mengué
Code
cartecodeMaemoMaemo MappermapN800Nokia N800ParisPOIVélib'XSLT
<p><a href="http://www.velib.paris.fr/">Vélib’</a>, c'est génial ! Abonné depuis deux jours et déjà accro ! Vive le vélo Plug'n Play ! Vive le vélo jetable !</p>
<p>Néanmoins, un outils devient vite indispensable : la carte des stations.</p>
<p>Je suis l'heureux propriétaire d'un <a href="http://www.nokia.fr/A4363032">Nokia N800</a> : c'est une « tablette Internet », avec connexions wifi, Bluetooth et un écran 800x480. Le tout fonctionne avec un système ouvert : Linux. Et ça tient dans la poche.</p>
<p>Le logiciel <a href="http://gnuite.com:8080/nokia770/maemo-mapper/">Maemo Mapper</a> pour le N800 permet de télécharger des fonds de carte (Google Maps notamment) et de les afficher. Ces cartes peuvent aussi être récupérées dans un cache pour consultation hors-ligne. Ainsi on peut avoir tous les plans de Paris de Google Maps dans la poche, sans connexion Internet. Maemo Mapper peut aussi afficher les informations d'un GPS Bluetooth.</p>
<p>J'ai donc créé une base de points d'intérêt (POI) pour afficher les <a href="http://www.velib.paris.fr/les_stations/trouver_une_station">stations Vélib’</a> dans Maemo Mapper.</p>
<p>Mais ma source des données pour les emplacements des stations n'est pas libre : il s'agit du fichier XML publié sur le site officiel (http://www.velib.paris.fr/service/carto, voir <a href="http://www.gpspassion.com/forumsen/topic.asp?TOPIC_ID=56151&whichpage=4">ce forum</a>). Je préfère donc éviter de distribuer le fichier terminé. Je vais par contre vous expliquer comment recréer vous-même ce fichier.</p>
<p><img src="http://o.mengue.free.fr/dolmen.blog/public/DSCN4779-520.JPG" alt="Photo de mon N800 avec Maemo Mapper affichant un plan de Paris avec les stations Vélib'" height="310" width="520" /></p>
<p>Lisez <a href="http://o.mengue.free.fr/blog/2007/08/02/32-une-carte-des-stations-velib-sur-nokia-n800">la suite de ce post</a> pour voir le code...</p> <p><strong>Mise à jour : j'ai publié une <a href="http://o.mengue.free.fr/blog/2007/08/12/36-stations-velib-v20">v2.0</a>.</strong> Mais lisez d'abord ce post.</p>
<p>Voici les outils que j'ai utilisés :</p>
<ol>
<li><a href="http://curl.haxx.se/">curl</a> pour récupérer le fichier XML des stations (mais vous pouvez simplement utiliser votre navigateur web)</li>
<li><a href="http://xmlsoft.org/XSLT/xsltproc2.html">xsltproc</a> pour transformer les données en script SQL. Sous Windows, il vous faut <code>xsltproc.exe</code>, <code>libxml2.dll</code>, <code>libxslt.dll</code>, <code>libexslt.dll</code>, <code>iconv.dll</code>, <code>zlib1.dll</code>, à piocher dans les <a href="http://www.zlatkovic.com/pub/libxml/">divers zips</a>.</li>
<li><a href="http://www.sqlite.org/">sqlite3</a> pour créer la base de POI. Téléchargez les "Precompiled binaries For Windows" <a href="http://www.sqlite.org/download.html">ici</a>.</li>
</ol>
<p>Pour transformer les données des stations, j'utilise une transformation <a href="http://fr.wikipedia.org/wiki/Extensible_Stylesheet_Language_Transformations">XSLT</a> :</p>
<pre class="code vim vimft-xslt"><span class="Comment"><?</span><span class="Type">xml</span><span class="Type">version</span>=<span class="Constant">"1.0"</span><span class="Type">encoding</span>=<span class="Constant">"UTF-8"</span><span class="Comment">?></span>
<span class="Comment"><!--
stations2mm.xslt
Copyright © 2007 Olivier Mengué <a href="http://o.mengue.free.fr/">http://o.mengue.free.fr/</a>
Maemo Mapper POI format: <a href="http://eko.one.pl/index.php?page=Nokia770_software#Details">http://eko.one.pl/index.php?page=Nokia770_software#Details</a>
--></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">stylesheet</span><span class="Type">version</span>=<span class="Constant">"1.0"</span><span class="Type">xmlns</span><span class="Comment">:</span><span class="Type">xsl</span>=<span class="Constant">"<a href="http://www.w3.org/1999/XSL/Transform">http://www.w3.org/1999/XSL/Transform</a>"</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">output</span><span class="Type">method</span>=<span class="Constant">"text"</span><span class="Type">encoding</span>=<span class="Constant">"UTF-8"</span><span class="Identifier"> /></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">strip-space</span><span class="Type">elements</span>=<span class="Constant">"*"</span><span class="Identifier">/></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Type">match</span>=<span class="Constant">"/markers"</span><span class="Identifier">></span>-- Script SQLite pour construire une base POI des stations Vélib pour Maemo
-- Mapper <a href="http://gnuite.com:8080/nokia770/maemo-mapper/">http://gnuite.com:8080/nokia770/maemo-mapper/</a> (Nokia 770, N800).
-- See <a href="http://eko.one.pl/index.php?page=Nokia770_software#Details">http://eko.one.pl/index.php?page=Nokia770_software#Details</a>
--
-- Créé avec stations2mm.xslt (<a href="http://o.mengue.free.fr/%29">http://o.mengue.free.fr/</a>)
--
-- Pour céer le fichier:
-- sqlite3 -init velib.sql poi.db
PRAGMA encoding = "UTF-8";
CREATE TABLE IF NOT EXISTS poi (poi_id INTEGER PRIMARY KEY, lat REAL, lon REAL, label TEXT, desc TEXT, cat_id INTEGER);
CREATE TABLE IF NOT EXISTS category (cat_id INTEGER PRIMARY KEY, label TEXT, desc TEXT, enabled INTEGER);
-- Placez 'Velib.jpg' dans un sous-répertoire 'poi' de votre dépôt de cartes
INSERT INTO category (label, desc, enabled) VALUES ('Velib', NULL, 1);
<span class="Comment"><!</span><span class="Comment">--</span>
<span class="Comment">Ignore les stations dont les coordonnées ne sont pas définies</span>
<span class="Comment"> lat=27.1409733745, lng=-3.40456062425</span>
<span class="Comment">--</span><span class="Comment">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">apply-templates</span><span class="Type">select</span>=<span class="Constant">"marker[@lat!='27.1409733745']"</span><span class="Identifier">/></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Identifier">></span>
<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Type">match</span>=<span class="Constant">"marker"</span><span class="Identifier">></span>INSERT INTO poi (lat, lon, label, desc, cat_id) VALUES (<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Type">select</span>=<span class="Constant">"@lat"</span><span class="Identifier">/></span>, <span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Type">select</span>=<span class="Constant">"@lng"</span><span class="Identifier">/></span>, "VELIB <span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Type">select</span>=<span class="Constant">"@number"</span><span class="Identifier">/></span>", "<span class="Identifier"><</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">value-of</span><span class="Type">select</span>=<span class="Constant">"@address"</span><span class="Identifier">/></span>", 1);
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">template</span><span class="Identifier">></span>
<span class="Identifier"></</span><span class="Special">xsl</span><span class="Comment">:</span><span class="Statement">stylesheet</span><span class="Identifier">></span>
</pre>
<p>Voici les commandes à lancer :</p>
<pre class="terminal unix"><samp class="prompt shell">$ </samp><kbd class="shell">curl http://www.velib.paris.fr/service/carto > stations-velib.xml</kbd>
<samp class="prompt shell">$ </samp><kbd class="shell">xsltproc -o stations-velib.sql stations2mm.xslt stations-velib.xml</kbd>
<samp class="prompt shell">$ </samp><kbd class="shell">sqlite3 -init stations-velib.sql velib-poi.db</kbd>
<samp class="sqlite">Loading resources from stations-velib.sql
SQLite version 3.4.1
Enter ".help" for instructions</samp>
<samp class="prompt sqlite">sqlite> </samp><kbd class="sqlite sqlite3">.q</kbd>
</pre>
<p>Il suffit alors de transférer sur le N800 la base de données <code>velib-poi.db</code>. Dans le menu <em>Settings...</em>, onglet <em>POI</em>, sélectionnez enfin le chemin de votre <em>POI database</em>.</p>
<p>Pour afficher une icône spécifique comme symbole de station, placez une petite image JPEG nommée <code>Velib.jpg</code> (même nom que la catégorie) dans un sous-répertoire <code>poi</code> du répertoire de cache des fonds de carte.</p>
<p><strong>Mise à jour : j'ai publié une <a href="http://o.mengue.free.fr/blog/2007/08/12/36-stations-velib-v20">v2.0</a>.</strong></p>