Java Abstract Syntax Trees

DTD Validation Guide

DTD Validation

DTD Validation is where you check the syntax of a DOM-tree against a Document Type Definition, which is a grammar of the expected XML elements and attributes. A DTD may also contain rules for expanding XML entity references.

A Document Type Definition may be:

internal - specified as a Doctype node containing grammar rules within the XML file;
external - specified as a Doctype node referring to an external file with the extension ".dtd"

An external DTD may be shared by many XML files. An XML file may include both internal and external DTDs, in which case the internal rules take precedence over the external rules.

DTD Validation is enabled simply by setting the validation-level of the XMLReader to DOCTYPE before you read in the XML file (see below).

Validation Levels

Document validation is controlled through the values of an enumerated type Validation:

Validation.IGNORE - is the default setting. This turns off all entity reference expansion, DTD validation and XSD validation.
Validation.EXPAND - is the lowest setting, which reads DTDs only in order to expand entity references.
Validation.DOCTYPE - is the middle setting, which expands entity references and checks the XML document against a Doctype.
Validation.SCHEMA - is the highest setting, which expands entity references and checks the XML document against an XML Schema.

Validation is found in the package:

uk.ac.sheffield.jast.valid

Reading a DTD

Reading a DTD is handled internally - you don't have to request this explicitly; but for interest, DTDReader is used to read both internal and external DTDs. Its constructors are:

DTDReader(File file,
Lexicon lexicon, String encoding)

DTDReader(URL url,
Lexicon lexicon, String encoding)

DTDReader(Reader reader,
Lexicon lexicon, String encoding)

These allow JAST to read from different kinds of source (from a file, from a web URL, or from another stream). They all require an initial Lexicon of entity references and the character encoding (e.g. "UTF-8") as a String.

DTDReader is in the package:

uk.ac.sheffield.jast.valid

Validating a File

Validating a local XML file is easily done using the following Java code.

File file =
new File("my/xml/input.xml");

XMLReader reader =
new XMLReader(file, "UTF-8");
reader.setValidation(Validation.DOCTYPE);

Document document =
reader.readDocument();

reader.close();

When XMLReader reads an XML file with a Doctype declaration, by default it stores this just as text (to save processing). However,

if DOCTYPE Validation is requested, the text is parsed and compiled into grammar rules;
and then the whole DOM-tree is checked against the compiled grammar.

If there are any XML grammar faults, these are reported by raising a SemanticError. Otherwise, the check passes silently.

Validating from a URL

Validating XML read from a URL on the web is also easily done using the following Java code:

URL url = new URL(
"https://www.my.site/input.xml");

XMLReader reader =
new XMLReader(url, "ISO-8859-1");

reader.setValidation(Validation.DOCTYPE);

Document document =
reader.readDocument();

reader.close();

This shows that you can specify a different character encoding Latin-1, which used to be standard for the HTTP protocol (probably UTF-8 now).

Once the DOM-tree has been read, it will then be validated against whatever Doctype was specified in the XML data.

Entity References

Certain characters and punctuation cannot easily be included as data within an XML document. This is because they form part of the defining syntax for XML. However, XML has a way of escaping these characters as Entity References.

An Entity Reference is a label that starts with & (ampersand) and finishes with ; (semicolon). The following Entity References are predefined:

< < less than

> > greater than

& & ampersand

" " quotation

' ' apostrophe

Further Entity References may be specified by the user, for example, the following may be declared as entity rules within a Doctype:

<!ENTITY copy "©">

<!ENTITY ajhs "Anthony Simons">

These define the Entity References:

&ajhs; Anthony Simons full text

Entity References may be defined for special characters (e.g. the copyright symbol), or simply to abbreviate longer boilerplate text (e.g. the author's full name).

The mapping from each Entity Reference to its full expansion is stored in a Lexicon. All of the JAST readers and builders have a Lexicon, which can be accessed in programs. You may choose to:

add business characters to the lexicon
add Latin characters to the lexicon

See the Java documentation for further details. Lexicon is found in the top-level package:

uk.ac.sheffield.jast

Expanding Entities

Any reader may choose to leave Entity References unexpanded, or may choose to expand all Entity References to their escaped text. This is useful if your program is going to process the original text outside of XML.

Entity Expansion is enabled simply by setting the validation-level of the XMLReader or ASTReader to EXPAND before you read in the XML file (see above), using the following code:

File file =
new File("my/xml/catalogue.xml");

ASTReader reader =
new ASTReader(file, "UTF-8");

reader.useDomain("project.org");
reader.usePackage("org.project.cat");

reader.setValidation(Validation.EXPAND);

Catalogue catalogue =
(Catalogue) reader.readDocument();

reader.close();

This will expand any Entity References in the XML attribute and text content, before this is stored as text in your own Java AST classes.

Entity Expansion happens automatically if you set the validation level to DOCTYPE or SCHEMA, but note that only XMLReader supports these higher levels of validation. This is because they need a full DOM-tree to check.

Encoding Entities

When writing a DOM-tree, or your own Java AST back to XML, you will want to convert all special characters and boilerplate text that should be escaped in XML back to their Entity References.

The best way to do this is to use the same Lexicon that expanded the entities when reading, and give this Lexicon to the writer:

File in = new File("my/xml/input.xml");

ASTReader reader = new ASTReader(in);

Lexicon lexicon = reader.getLexicon();

lexicon.addLatinCharacters();

lexicon.addBusinessCharacters();

Catalogue catalogue = (Catalogue)
reader.readDocument();

reader.close();

// do something with catalogue

File out = new
File("my/xml/output.xml");

ASTWriter writer = new ASTWriter(out);

writer.setLexicon(lexicon);

writer.writeDocument(catalogue);

writer.close();

This example adds two extra sets of entities to the ASTReader's default Lexicon, so that it may decode more kinds of entity when it reads the document.

This same Lexicon is passed to the ASTWriter, so that it may encode the necessary text back to entity references in the output file. XMLWriter and ASTWriter encode escaped characters and text automatically; this need not be requested.

Internal DTD

We show an example Document Type Definition below. This is an internal Doctype, which should appear after the XML declaration and processing instructions, and before the root element in the related XML file:

<!DOCTYPE Catalogue [

<!ELEMENT Catalogue (Film | TVShow)*>

<!ELEMENT Film (Title, Director)>

<!ELEMENT TVShow (Title, Director)>

<!ELEMENT Title (#PCDATA)>

<!ELEMENT Director (#PCDATA)>

<!ATTLIST Film

date CDATA #REQUIRED

rating (U|PG|12|15|18) #IMPLIED>

<!ATTLIST TVShow

date CDATA #REQUIRED

rating (U|PG|12|15|18) #IMPLIED>

<!ENTITY copy "©">

This defines the grammar of the Catalogue root element in the XML file. In this DTD:

DOCTYPE specifies the name of the root element and a list of rules in [ ] brackets;
ELEMENT specifies the legal children of each named element;
ATTLIST specifies the attributes of each named element and the range of values each may take;
ENTITY defines an entity reference for the copyright symbol.

Grammar Rules

The grammar defined in this DTD expresses the following rules:

Catalogue - may have multiple children, where each child is either a Film or a TVShow, in any order.
Film and TVShow - must have exactly two children Title and Director, in that order.
Title and Director contain parseable character data (plain text which may embed more XML).
Film and TVShow - have a REQUIRED attribute date, which is character data (plain text). This is compulsory.
Film and TVShow - have an IMPLIED (optional) attribute called rating, which must take one of the values U|PG|12|15|18.

The syntax of a DTD is different from usual XML syntax, and uses a kind of BNF (Backus-Naur Form) to express the grammar:

a comma separates items in a sequence
a vertical bar separates alternatives
parentheses group a set of elements
an asterisk denotes arbitrary repetition

External DTD

The above DTD definitions could be supplied as the text of a separate external file, which we will assume is called Catalogue.dtd.

In this case, the Doctype in the XML file only needs to refer to the location of the DTD file.

<!DOCTYPE Catalogue SYSTEM "Catalogue.dtd">

This assumes that the DTD file is locally available on the same filesystem, in the same folder as the XML file. A longer relative path could be given.

If an external DTD is meant to be published more widely on the web, it is usually given what is known as a Formal Public Identifier:

<!DOCTYPE Catalogue PUBLIC "-//AJHS//DTD Catalogue 1.0//EN" "Catalogue.dtd">

This is a standard way of advertising the web address of the DTD. See this explanation of Formal Public Identifiers on Wikipedia.

Doctype Node

The Doctype node can be accessed in programs, using Document's method getDoctype().

The checking methods of Doctype may also be called explicitly in programs. There are four checking methods, which accept either a whole Document or the root Element of a DOM-tree as argument:

public boolean accept(Document doc);
public boolean accept(Element root);

public void validate(Document doc);

public void validate(Element root);

The accept() methods return true, if the DOM-tree is valid, or false otherwise. The validate() methods succeed silently if the DOM-tree is valid, or raise exceptions otherwise.

Doctype is found in the package:

uk.ac.sheffield.jast.xml

Visualising a DTD

It is possible to visualise the grammar compiled by a Doctype. Every top-level ElementRule in the grammar is capable of writing a pretty-printed representation of itself, using a toString() method. This will print out one production from the grammar.

To see the whole grammar, call the access method getProductions() on the top-level rule, and this will return a list of all the productions in the grammar. You can iterate through the list and print out each rule as a production in BNF format:

Doctype doctype = ... // you fetch

ElementRule topRule =
doctype.getGrammar();

for (ElementRule rule :
topRule.getProductions()) {

System.out.println(rule);

}

W3C Compliance

JAST supports the creation of DOCTYPE nodes that contain a sequence of ELEMENT, ATTLIST and ENTITY definitions.

The full BNF syntax for ELEMENT definitions is supported, including sequence, selection and iteration of single items or bracketed () structures.
Multiplicity markers may specify optional ?, zero-to-many *, or one-to-many + occurrences.
ELEMENT definitions may contain the EMPTY category marker to indicate empty content, or ANY category marker to indicate arbitrary content.

ATTLIST definitions may declare single, or multiple attributes for each element within the same declaration.

Each defined attribute has a name, an attribute type, and either an occurrence specifier or a default value.
Attribute types may be symbolic types such as ID, NMTOKEN or CDATA; or they may be enumerated selections.
An occurrence specifier is either #REQUIRED (compulsory), #IMPLIED (optional), or #FIXED which must be followed by a fixed value. Any other value is interpreted as a default value.

ENTITY definitions may declare general entities (but not parameter entities, which are not supported). General entities may be either:

a character entity reference (for an escaped character);
or a string entity reference (for boilerplate text).

An ENTITY may declare an internal entity as shown above, or through SYSTEM or PUBLIC identifiers, it may refer to an external entity, whose text expansion is given in a separate file. As a security restriction, the expansion-text must be stored in a simple text file ending with the extension .txt, to prevent the malicious use of external entities to read secret password files.

See the W3C tutorial on DTD for more information.

Page updated

Google Sites

Report abuse