4.1 The difference between a hardcoded generator, a template interpretator and a template compiler
We already discussed the different approaches in text generation. LetТs review the problem in more detail. The straightforward approach to generate a text is to write a program using the standard programming language. The classic "Hello World" program is a good example. Here is slightly extended version -
void main(char** args, int argc) {
printf("Hello, world!\n");
printf("My name is ");
printf(args[1]);
printf("\nGood bye!\n");
}
Even in such a primitive example we can see that the construction of the programming language obscures the structure of the generated text. To solve the problem, text templates are used. Structure is described in a special text file which resembles the desirable result. For our example it would be something like
Hello, world!
My name is %name%
Good bye!
It is a very clean representation of a text we want to produce. The generator program has to load the template, process some dynamic elements like %name% and return the result. Sometimes it is necessary to use dynamic elements that are more complex than simple substitutions, such as repetitive blocks. It can be templated as the following example -
Hello, world!
My name is %name%<foreach x in %languages%>
I can speak %x.name%<endfor>
The result would be
Hello, world!
My name is Sam
I can speak German
I can speak French
Still simple enough to understand the result structure, but a new problem is arising - control structures require special markers to be separated from static text. It is a question of taste, but in general, marker style syntax is more disturbing and less convenient than straight commands in traditional programming languages. To find the golden median, the TEAL language was designed. TEAL can be decoded as TEmplAte Language. Next is our example in TEAL syntax -
"Hello, world!
My name is ".name
for x .languages begin "
I can speak " x.name
endfor
TEAL combines some benefits from both approaches - templates and hard coded generators -
- like templates, TEAL doesn't require special constructions like commands or operators to produce a result. Most of TEAL language elements are "values" automatically placed into result text. Such elements include string literals (static text), variables and function calls. The rest of the language elements do not produce text and are used only to control the generation process.
- like hardcoded generators, TEAL templates are compilable, not interpretable. The result of compilation is a dynamic library which can be loaded by the host program to generate the text. As result, the generation process is much faster then traditional template interpretation. Also, the program doesn't need to include the interpretation code.
- like hardcoded generators, TEAL provides simple and powerful syntax to control the generation process. A programmer can use FOR and IF commands, dot notation in traditional object-oriented style, function and file decomposition of template.
4.2 TEAL syntax
4.2.1 TEAL elements. Values and directives. Template decomposition. Data types.
As was mentioned before, TEAL syntax consists of an "elements". Each element can be a "value" or a "directive". Value is an element producing a text and adding it to the result of generation. Directives don't produce any text, but somehow affects the generation process.
To simplify and organize template development and simplify its maintenance, it can be decomposed by files and functions. A function is a named template block which can be called from other places. Using functions reduces code duplications. Each template file can include several function definitions. Files are included into a template project using an "include" directive (see below). File decomposition allows the reuse of files in different projects. It also simplifies the template code development and maintenance.
TEAL supports only two data types - strings and objects. It has no mechanism of type checking (which is main subject for next version development).
4.2.2 Literals.
Text literals are pieces of static text placed in the result as is (with the exception of special constant codes). Literals are identified by double quotes at the text start and end. It is also possible to specify a different character as a literal identifier (using pragma "literal" expression).
Literals execute a skeleton role in the template structure. A literal represents a piece of text inserted in a result as is. A literal starts and ends with a marker (") - double quote char. Two sequential literal markers generate a single character inside of the literal. For example,
"Horactio said, ""Is't possible?"""
will generate
Horactio said, "Is't possible?"
If a double quote character is used widely in your target syntax it, makes sense to change the literal marker to a different character, say a single quote, or tilde. It is possible to use the "pragma liter" directive. For example,
pragma liter ~~
will force the parser to use the tilde character as a literal marker starting immediately after the pragma expression. Pragma takes effect until next marker definition, so different parts of the template can use different markers. Note that the pragma parameter requires to specify the new marker value twice (see pragma syntax description below).
Here is a list of constant codes supported by TEAL syntax -
%cr% - carriage return character
%comma% - comma character
So, the next code
"O, I die, Horatio;
The potent poison quite o'er-crows my spirit..."
is an equivalent to
"O%comma% I die%comma% Horatio;%cr%The potent poison quite o'er-crows my spirit..."
4.2.3 FUNC command
The "FUNC" directive is the topmost element of the Teal syntax. It defines a named template code callable from any place in the template. The directive has syntax as follow -
[external|internal] func function_name [ ( [obj] param1, [obj] param2, Е ) ] [begin body endfunc] | [;]
Where
a. Without "external" or "internal" keywords, the FUNC command declares and defines the function at once. Definition is provided by a code between "begin" and "endfunc" keywords (body block). With "external" or "internal" keywords, the command only declares the function and relies on implementation outside of the TEAL template - in a data processor or in the C++ library. In this case, the body block is not used and the command must be terminated with a semicolon. Read more about external and internal functions in the following sections.
b. Function_name gives a name to the function.
c. Param1 is a formal parameter name. A parameter list is not required if the function doesnТt use parameters.
d. With the "obj" keyword, the parameter identifies an object identified by the parameter name. All associations between objects and their names can be done only inside the FOR command (see below) -
func map(obj AnObject, obj AContainer) begin ... endfunc
...
for x .classes begin
for a c.attributes begin a.name "=" map(a, c) "%cr%" endfor
// or in OO-style form
for a c.attributes begin a.name "=" a.map(c) "%cr%" endfor
endfor
The generator will call the function "map" with two parameters, the first one is an object reference to the class attribute, and the second one is an object reference to the class in context of which the mapping was done.
4.2.3.1 Internal functions.
"Internal" keyword specifies if the function is defined in the C++ library.
Internal functions can be defined in additional C++ modules and serve more generic purposes, like text manipulation. To make a C++ function useful in the TEAL template, it has to be defined in a C++ file and comply with the following rules -
1) It has to be a function of the void type.
2) The first parameter is required, it is used to add a result of function execution to a result string of a template. It must be a reference to std::string.
3) Any number (or none at all) of parameters of std:string type.
The file has to be compiled with C++ compiler and the result in the form of an object file or a library has to be linked at template compilation time (see "Template Compilation").
In addition, the function has to be declared in the TEAL file using the syntax
internal func <function_name>(function parameters) ;
Internal functions cannot work with object references.
For example,
// C++
void substr(std::string& result, std::string text, std::string start, std::string length) {
result += text.substr(atoi(start.c_str()), atoi(length.c_str()));
}
// TEAL
internal substr(text, start, length);
4.2.3.2 External functions
External functions are implemented by Data Processor. They donТt need to be defined using TEAL syntax, only declared. Usually external functions are used to access object properties. They can also provide access to some information about the data processor or the host application.
To be useful in template code, external functions must be declared using TEAL syntax -
external func function_name(function parameters) ;
For example, suppose we have a "name" function -
external func name(obj object) ;
So, the template can use this function like
for c in p.classes begin
name(c) "%cr%"
endfor
An example of a function not accessing an object is an "exever" function implemented by the UML processor. It returns a version of the host application running the generator -
external func exever();
All external functions supported by the UML processor are defined in the uml_externals.tl2 file. They provide access to UML object names, types, stereotypes, etc. In addition, some of them give access to mapping information.
4.2.4 MACRO command
Macros in TEAL language are similar to ones in C++ language. First, they generate a code in each point of call. Second, macro parameters are not values, but actual peaces of code inserted at each point where they are sued inside of the macro. For example, the code:
macro MacroTest(p1, p2, p3) begin
for p1 p2 begin p3 endfor
endmacro
MacroTest(x, c.attributes, x.name "%cr%")
MacroTest(x, c.roles, x.type "%cr%")
is equivalent to
for x c.attributes begin x.name "%cr%" endfor
for x c.roles begin x.type "%cr%" endfor
4.2.5 Function call
FUNC directive does not produce a value, it only defines a function. The value can be retrieved using function call. Function call has a simple syntax - function name followed by optional parenthesis with list of actual parameters, like
substr("QWERTY",2,4)
To make template syntax more convenient, TEAL supports a short form of function call if the first parameter is an object reference -
func myfunc(obj TheRef) begin ... endfunc
for c in p.classes begin
myfunc(c) "%cr%"
endfor
is identical to
for c in p.classes begin
c.myfunc "%cr%"
endfor
Such dot-notation resembles object-oriented syntax in languages like C++ and Object Pascal. Dot-notation is still useful if a function has more than one parameter. For example, if we have a declaration like
external func MyFunc2(obj theRef, obj theRef2, txt) ;
then the following two calls will be semantically identical -
MyFunc2(c1, c2, "abc")
c1.MyFunc2(c2, "abc")
4.2.6 Object properties
To access object properties, the TEAL template generates text for, it uses an external function implemented by Data Processor and declared as externals. For example, UML Processor supports many such functions, all of them are listed in uml_externals.tl2 -
external func map_(obj object, mapname) ;
external func mapval_(obj object, mapname) ;
external func maptype_(obj object, mapname) ;
external func exe() ;
external func exever() ;
external func exetime() ;
external func parent(obj object, obj container) ;
external func metatype(obj object, obj container) ;
external func stereotype(obj object, obj container) ;
external func attrs(obj object, obj container) ;
external func uname(obj object, obj container) ;
external func name(obj object, obj container) ;
external func fname(obj object, obj container) ;
external func name1(obj object, obj container) ;
external func type(obj object, obj container) ;
external func type1(obj object, obj container) ;
external func realtype(obj object, obj container) ;
external func realtype1(obj object, obj container) ;
external func version(obj object, obj container) ;
external func phase(obj object, obj container) ;
external func guid(obj object, obj container) ;
external func attrCount(obj object, obj container) ;
external func typekind(obj object, obj container) ;
external func iskey(obj object, obj container) ;
external func isderived(obj object, obj container) ;
external func ref(obj object, obj container) ;
external func reftarget(obj object, obj container) ;
external func typecode(obj object, obj container) ;
external func visibility(obj object, obj container) ;
external func return(obj object, obj container) ;
external func abstract(obj object, obj container) ;
external func retarray(obj object, obj container) ;
external func synchronized(obj object, obj container) ;
external func static(obj object, obj container) ;
external func const(obj object, obj container) ;
external func pure(obj object, obj container) ;
external func isquery(obj object, obj container) ;
external func dir(obj object, obj container) ;
external func cardinality(obj object, obj container) ;
external func cardinality1(obj object, obj container) ;
external func aggregation(obj object, obj container) ;
external func qualifier(obj object, obj container) ;
external func typeStereotype(obj object, obj container) ;
external func map(obj object, obj container) ;
external func mapval(obj object, obj container) ;
external func maptype(obj object, obj container) ;
external func mapinfo(obj object, obj container) ;
external func mapuser(obj object, obj container) ;
external func load(obj object, obj container) ;
external func add(obj object, obj container) ;
external func del(obj object, obj container) ;
external func tag(obj object, tagname) ;
Most of the functions supported by Data Processor take object reference as a first parameter. So, as discussed before, they can be used with dot notation.
As was already mentioned before, TEAL doesnТt know the nature of objects it generates a text for. It can work with data structures representing the file system hierarchy, UML class model, data feed for HTML pages, etc. The TEAL compiler has no way to check the data types. So, at compile time the TEAL compiler doesnТt know if a function is applicable in each particular case. For example, the external function "attrCount(obj object)" returns a number of class attributes. But if it is called with package object it will fail. The template programmer has the responsibility to use only applicable functions.
Note: in future versions of TEAL language, a data type checking mechanism is expected to be implemented.
4.2.7 FOR
The "FOR" command is the most significant element of the TEAL syntax. The command iterates through a collection of objects, and for each object generates a code specified in the commandТs body. The command syntax is
for variable [owner_name].collection_name [where condition] [asc value] [desc value] [div value] [wrap value] [set set_expression] begin body endfor
Where
a. Owner_name is a variable name representing an object - owner of the collection. Owner_name references to the variable in an outer "for" command. If the "for" command is the topmost one, it can use the "root" variable with the name "#". Owner_name can be omitted, in which case the current object (provided by the closest outer "for" iteration) is used as the owner.
b. Collection_name identifies a collection of objects. Actually it identifies not a collection, but an iterator used to access the collection members. As well as objects, collections are provided by Data Processor (see UML Data Processor). Because TEAL has no dependency on data types it serves, it doesnТt know about possible collection names supported by the data processor. To help the TEAL compiler to avoid a compilation of templates with incorrectly spelled collection names, the template developer has to explicitly list all possible names using the "iterator" directive, like
iterator packages;
iterator classes;
If the compiler finds an undeclared collection name it outputs a warning message.
c. Variable is a variable name associated with the current object - member of the collection.
d. "Where" element specifies a boolean expression used to filter out some members of the collection.
e. Asc and Desc are values used to order the collection members.
f. Div is a value to be placed between iterations.
g. Wrap is a value to be placed between iterations if the generated string is too long. Obsolete.
h. Set is a set command (without a trailing semicolon) to be executed at the beginning of each iteration. The only difference between embedded and standalone "set" commands is the fact that the variable value is restored to the original one after the last iteration.
i. Body is a value generated at each iteration.
4.2.7.1 Support of object iterators in UML Processor
As well as external functions providing access to object properties, collection iterators are supported by Data Processor. Here is list of iterators supported by UML Processor and declared in the "uml_exetrnals.tl2" -
iterator packages;
iterator classes;
iterator parent;
iterator attributes;
iterator attributes-;
iterator associations;
iterator roles;
iterator associations-;
iterator roles-;
iterator methods;
iterator parameters;
iterator type;
iterator refattribute;
iterator opposite;
4.2.8 Boolean expression
A Boolean expression.
Boolean expressions are used in the "FOR" and "IF" commands. Several expressions can be combined with the use of OR, AND, NOT operators. The standard boolean expression uses comparison operators. Comparison operators return a result of two string comparison. Supported comparisons are "=", "<", ">", "<=" and ">=". For example,
if a.isderived = "true"
begin
"Attribute is derived"
else
"Regular attribute"
endif
4.2.8.1 Coded (short) Boolean expressions
In addition to standard Boolean expression Teal syntax supports coded or short Boolean expressions. Coded expressions return a result of the comparison between object attributes and the attribute filter provided by the expression. The filter expression has positive and negative sections. The positive section is prefixed with a plus ("+") sign and lists attributes the object must to have to match the criteria. The negative section is prefixed with a minus ("-") sign and lists attributes the object must not to have to match the criteria. The association between attributes and their codes is set in special filter initialization template. For example, "D" can mean "derived" attribute, "S" mean "attribute of simple type", etc. Using these codes, we can write an "IF" command like the following -
if +SD begin "Attribute of simple type, derived" endif
The main purpose of simple filter expressions is a simplification of Boolean expressions. In addition, they work faster than standard expressions because object codes are cached and need no repetitive calculations.
4.2.9 IF command
The "IF" command provides the conditional execution of the code depending on the Boolean expression. The command returns the value of its block if the expression returns a positive result. Otherwise it can return an optional alternative (following after "else" keyword) block. The full syntax is
if <boolean expression> begin ... [ else ... ] endif
4.2.10 Variables
TEAL allows to define and use a variables. All variables in TEAL language are global variables. To define a variable, use "set" directive -
set var_name = var_value ;
like
set "note" = "Today is my birthday";
After that, the variable value can be accessed simply by a variable name. The following code
"The note is: " note
will generate
The note is: Today is my birthday
Please note, the "set" directive uses a value to get a variable name, while access to a variable value is possible only through a constantly expressed name. Using values such as names in the "set" directive allows dynamic variable definitions like
set s = "Var";
set s "#1" = "123" ;
set s "#2" = "ABC" ;
...
if Var#1 <> "123" then ...
if Var#2 <> "ABC" then ...
Current TEAL syntax does not support the possibility to get variable value using dynamic name resolution.
4.2.11 Include directive
The directive allows splitting the template source code between several files. It actually informs the compiler to parse the named file. The syntax is
include "<file_name>"
It allows to split the template on different files and hence simplifies template development and especially maintenance. The template developer can put the file inclusion in each file where it is necessary, but also all inclusions can be collected in single place - at the beginning of the main file (see the "Template Project Structure" discussion).
4.2.12 Iterator directive
The Iterator directive is used to declare an iterator name supported by the data processor. It helps the TEAL compiler to identify incorrect (misspelled) collection names used inside of the FOR command. The syntax is
iterator <collection name list> ;
where <collection name list> is a space- or comma- delimited list of
The data processor should come with the TEAL file where all supported collection names are listed, as well as all supported functions. For example, the UML processor comes with a "uml_externals.tl2" file where the iterator portion looks like
iterator packages classes parent;
iterator attributes attributes-;
iterator associations, roles, associations-, roles-;
iterator methods parameters type;
iterator refattribute;
iterator opposite;
Note that all names can be listed in a single iterator directive or be split into several ones.
4.2.13 Pragma directive
Pragmas are used to control the compiler process. The syntax is
pragma <pragma_name> [pragma parameters]
4.2.13.1 CodeCalc pragma
The "codeCalc" pragma lets the compiler know which function should be used to generate a list of object codes for coded (short) Boolean expressions. The syntax is
pragma codeCalc <function_name>
4.2.13.2 Liter pragma
The "liter" pragma commands the compiler to use a character different from the default double-quote to identify the literals. The syntax is
pragma liter <Ch><Ch>
The character should to be repeated twice - this way, you will avoid the problem with syntax highlighting in the text editor used for TEAL files modification. For example,
pragma liter '' // use a single quote as a literal identifier
'Hello, "Alex"!'
pragma liter "" // use a double quote as a literal identifier
"Hello, 'Michael'!"
4.2.13.3 Export pragma
The "export" pragma defines a list of an exported functions - functions available by the host application for generation. The syntax is
pragma export <list of functions> ;
where <list of fucntions> is a comma- or space- delimited list of function names.
You can use the individual pragma expression for each exported function, or you can enumerate all of them in a single directive. For example, the code:
pragma export my_main ;
pragma export func_a funct_b func_c ;
will export four functions.
4.2.14 Resource directive
The resource directive defines a name-value string pair which is placed inside of the compiled template. It is usually used to store information about a template accessible by the host application. The syntax is
resource "<name>" = "<value>"
4.2.15 Comments
The TEAL compiler supports one line comments only. Text located outside of the literal element, beginning with a double-slash-space sequence and ending at the end of the current line is ignored by the compiler. For example -
for x in .classes begin
x.name ":" // printing name and stereotype of each class
x.stereotype "%cr%"
endfor
4.3 Template project structure
To make a complete template, the TEAL program should provide the following -
- Set of functions used for text generation. All functions can be defined in one file, but in most cases it is better to split the template code into several files and use them with the help of an "include" directive. The include directives can be used in each file where necessary. It creates a hierarchy of files - the topmost, main file includes the set of next level files, each one can optionally include more files. The alternative approach is to include all files used in the project in one place - at the top of main file. It works fine until you decide to reuse some of the files in a different project. In that case, you have to remember all dependencies and include all files required by the shared one in the main file.
- Mark at least one function as exported, i.e. available for call from the host application using the "pragma export" directive.
- Optionally identify a function used for short codes calculation. If short Boolean expressions are not used, this step is not required.
4.4 Template compilation
The TEAL compiler does not produce the machine code directly. Instead, it generates a C++ code which can be compiled into the machine code in the form of a dynamically loaded library. To make it possible, the TEAL package includes a Generator.h C++ header file and a Generator.lib library file.
4.4.1 C++ compiler configuration
The final dynamical library (ready to use template) can be compiled manually, but to simplify the process, the TEAL compiler can invoke the C++ compiler and librarian automatically if a special command line option is provided. The TEAL configuration file has to contain full information necessary to compile and link the result. It allows to use the C++ compiler available in your system. The configuration file can contain profiles for several C++ compilers.
4.4.2 Genarator.lib and Generator.h
The direct result of template compilation is a single C++ file. It has to be compiled and linked with Generator.lib to produce the final result - dynamic library. For compilation, the generated file requires some declarations for generator runtime support. Such definitions are provided by the Generator.h file. Generator.lib includes compiled code for generator runtime and the UML processor. If Data Processor is used, the appropriate library has to be linked also.
4.4.3 TEAL2 command line options
The command line syntax is
teal2.exe [-v N] [-p] [-c:profile] input_file [output_file]
-v N set verbosity level in a range 0..10
-p make a pause after an execution
-c=profile select C++ compiler configuration