XML Tools

I wrote a small utility for xml data extraction. It works from the command line and acts as "standard output filter". This "standard output" can be a file or redirected output of another program.

LP - input preprocessor that can extract data from xml files

Syntax:

LP [[/?] | [/A] | [[drive:][\path\]driver.xml [/C=x][/N=<..>] [[/Q] | [/V]]]]

driver.xml:     XML file that will be used for
                processing of input data
/?              Shows this help message

/Q              Quiet - does not show any messages
                on the screen
/V              Verbose - shows more information on
                the screen
/A              About - shows info on dll file used
/N=<xml tag>    Macro that will be used to process
                the new line
/C=x            Set the number of counters (default
                5)

Switches /V and /Q are mutually exclusive, if used both verbose will be used

Examples:
        LP driver.xml < test.txt > LPT1:
        LP /q driver.xml < test.txt | more
        LP /v driver.xml < test.txt > output.txt
        LP driver.xml < test.txt
        LP /?
        LP

Input data:

Input can be redirected from a file or from another program. Any character found will be send to output.

Macros:

Input can contain "macros". Macro starts with "$"

Example of a macro: $(macro)

There are some builtin macros:

• $($) This macro will print "$".
  
• $(ff) This macro will print form feed character (ascii 12).
  
• $(cl) This macro will print current line (could be used to do some line numbering for example).
• $(localdatetime) This macro will print system date and time.
  
• $(gmtdatetime) This macro will print gmt date and time. TZ system variable must be set for this macro to work correctly.
• $(counter) This macro will count how many times it was called. There are multiple counters.

Counters:

Counter macros can be used to count ... By default there are 5 counters. You can access them like this:

$(counter) = $(counter[0])
$(counter[1])
$(counter[2])
$(counter[3])
$(counter[4])

If you need more than 5 counters, you can specify how many you need on a command line with option /C=x where x is the number of counters. X can be between 0 and (theoretically) 4294967295. How many counters can actually be created, depends on the amount of free memory. If you specify to high number, error will be reported.

XML macros:

This is best explained on an example:

This is our xml file:

--------test.xml--------------
<?xml version="1.0"?>
<document>
 <item var="first item">First item text</item>
 <item var="second item">Second item text</item>
 <item var="third item">Third item text</item>
 <item var="fourth item">Fourth item text</item>
 <item var="fifth item">Fifth item text</item>
 <!-- this is comment -->
 <item>
  <subitem>
   <sub-subitem myVar="some text">
    <sub-sub-subitem>
     This is a text of sub-sub-subitem
    </sub-sub-subitem>
   </sub-subitem>
  </subitem>
 </item>
</document>

---------eof------------------

We want to extract the text from the first item in te document. This is done like this:

$(document\item)

It will print:

First item text

Xml document can have multiple items of the same type. We can access all the items by using "index". Index has to be written in the square brackets. That is the way to get the fourth item text:

$(document\item[3])

Returns:

Fourth item text

Index of the first item is 0, that is why fourth item has index 0.

It is easy to extract data from deeply nested xml tags. This example shows how to access 4th level:

$(document\item[5]\subitem\sub-subitem\sub-sub-subitem)

This will print:

This is a text of sub-sub-subitem

Data in xml can be stored as "parameter" like this:

<item parameter="text of parameter" />

It is easy to access it. If we want to access first item's "var", we do it like this:

$(document\item<var>)

This will print:

first item

We can write this as:

$(document\item[0]<var>)

This will print:

first item

If we need to access deeply nested myVar in sub-subitem, we do it like this:

$(document\item[5]\subitem\sub-subitem<myVar>)

This prints:

This is a text of sub-sub-subitem

Each item has an index. If you do not write it explicitly, index is assumed to be 0. Each item can have a parameter. Only the parameter of the last item in a path is used.

Errors in macros:

We are all human. Errors happen. If the item does not exist, nothing will be written to the output.

If input contains dollar sign "$", it MUST be followed by "(". $ means  "this is a beginning of a macro". If you want to print "$" to the output, use $($).

If you want to extract data from xml and you make a mistake like this:

$(document\item[0)

(there is no right square bracket) nothing will be printed. The same goes for parameters (<>).

Comments in xml file:

Comments in xml file are ignored.

Output:

Output can be redirected to a file or to another program.

How to build from the source code:

I used OpenWatcom C++ to build the source code. You need OpenWatcom installation on path. Issue command: 

WMAKE

to build.

To clean:

WMAKE clean

To rebuild from scratch:

WMAKE /a

It is possible to make DOS, WINDOWS and OS/2 binary. You need to edit the "HOST" variable in the "makefile" accordingly.

It is possible to optimize for specific processor, you need to edit "MYCPU" variable in the "makefile".

DOS version:

Dos version uses HXDOS Extender by Japheth. It is available here: http://www.japheth.de/HX.html

Examples:

I created a couple of examples to show how LP preprocessor could be used. Net folder (these need a working internet connection and use WGET utility to download xml file from the internet):

• a simple RSS reader (RSS.BAT)
  
• a weather "gadget" (WEATHER.BAT)
  
• yahoo! answers (ASK.BAT)
  

Local folder (these do not need the internet):

• a simple RSS reader (RSS.BAT)
  
•  a weather "gadget"  (WEATHER.BAT)

License information:

Standard zlib License:

  Copyright (C) 2007-2008 Matjaz Mesnjak

  This software is provided 'as-is', without any express or implied   warranty.  In no event will the authors be held liable for any damages   arising from the use of this software.

Permission is granted to anyone to use this software for any purpose,   including commercial applications, and to alter it and redistribute it freely, subject to the following restrictions:

  1. The origin of this software must not be misrepresented; you must not

     claim that you wrote the original software. If you use this software

     in a product, an acknowledgment in the product documentation would be

     appreciated but is not required.

  2. Altered source versions must be plainly marked as such, and must not be

     misrepresented as being the original software.

  3. This notice may not be removed or altered from any source distribution.

Portitions of code written by others:

All versions of this program use TinyXml library: http://www.grinninglizard.com/tinyxml/ 
(I had to rename the library files to match DOS 8.3 rule)

DOS version includes some code from HXDEV package: http://www.japheth.de/HX.html 
(HXDLL.CPP (original file name: HXDLL.C) and HXDLL.H)

Version information:

• Version 1,2 were not publicly released.
  
• 16:42 2.11.2008 Version 3.0 is the first public release.

To do:

Similar program to write/edit/delete xml items.

Contact:

Matjaz Mesnjak:
matjaz85 at gmail dot com