Discovering more with an example

The first time, we recommend to read the precedent chapter, more approachable, before reading this one.

Let imagine that we dispose of a design expressed in a simple modeling language, like it:

      // file "GettingStarted/SolarSystem0.sml":
      1 class Planet {
      2     double diameter;
      3     double getDistanceToSun(int day, int month, int year);
      4 }
      5
      6 class Earth : Planet {
      7     string[] countryNames;
      8 }
      9
      10 class SolarSystem {
      11     aggregate Planet[] planets;
      12 }

line 1: a class is declared with keyword class
line 2: declaration of attributes in a syntax close to C++ or JAVA
line 3: declaration of methods in a syntax close to C++ or JAVA
line 6: a class may inherit from an other ; the syntax looks like C++, see ':'
line 7: an attribute may be an array ; the syntax looks like JAVA
line 11: an attribute may be an object or an array of objects, and an object may be an aggregation (meaning that it belongs to the instance),
This simple modeling language conforms to a BNF grammar (see paragraph
BNF syntax to obtain information about the elements of a BNF syntax):
world ::= [class_declaration]*
class_declaration ::= "class" IDENT [':' IDENT]? class_body
class_body ::= '{' [attribute_decl | method_decl]* '}'
attribute_decl ::= type_specifier IDENT ';'
method_decl ::= type_specifier IDENT '(' [parameters_decl]? ')' ';'
parameters_decl ::= parameter [',' parameters_decl]*
parameter ::= [parameter_mode]? type_specifier IDENT
parameter_mode ::= "in" | "inout" | "out"
type_specifier ::= basic_type ['[' ']']?
basic_type ::= "int" | "double" | "string" |
"boolean" | class_specifier
class_specifier ::= ["aggregate"]? IDENT
IDENT ::= ['a'..'z'|'A'..'Z'|'_'] ['a'..'z'|'A'..'Z'|'_'|'0'..'9']*

Starting from the desing file "SolarSystem0.sml" seen before, which conforms to the Simple Modeling Language described just above, we propose to implement the source code for classes and a light documentation.

1 The parse tree

CodeWorker doesn't belong to the category of typed languages. It recognizes only the tree as structured type and the string as basic type (that may however represent an integer or a boolean, ...). Each node may contain a string as a value, and/or an array of nodes. The main tree is called project, which is the name of its root node, accessible everywhere into scripts.

Now, the best way to understand how to handle the tree is to run the console, and to practice some examples.

Type CodeWorker to the shell to set the console mode. A cursor is waiting for your commands.

Type set a = "little"; and press enter. Don't forget the semi-colon at the end of the line. If absent, the console wait for more input: type the expected semi-colon, and it should be right.

What is the impact of the line you typed? You assigned "little" to the variable a, which doesn't exist. So, a node named 'a' has been added into the main parse tree (called project, remember), to which the variable a points. You noticed that a varning has occurred. It means that you assigned a value to a node that doesn't exist yet. In fact, the instruction set supposes that the variable to assign already exists, and a warning has been thrown to prevent you of a spelling error (perhaps do you intended to type another variable that already exists?) or a logic mistake (at this point of the program, the variable should exist, so what?). It is important to offer this protection, because the language isn't typed, and so, a lot of errors may be reported during the runtime.

The variable a has been added, even if the warning has occurred, but we prefer the instruction insert to add a new node properly : type insert b = "big"; and press enter. No warning was displayed. Now, the root project node contains two sub-nodes, called 'a' and 'b', and we control it by typing traceObject(project);. The following lines are displayed:


Tracing variable 'project':
        a = "little"
        b = "big"
End of variable's trace 'project'.

Let's go further. What about storing a list of items?
Type insert classes["Planet"].name = "Planet";. A node node called 'classes' has been added to project, and then an array entry called "Planet" has been pushed. This entry points to a node, to which 'name' is added, and node 'name' is worth "Planet".

Type insert classes["Earth"].name = "Earth"; and then ask for tracing node 'project'. The following lines are displayed:


Tracing variable 'project':
        a = "little"
        b = "big"
        classes = ""
        classes["Planet", "Earth"]
End of variable's trace 'project'.

Notice that the node 'classes' has no value (but could have!) and contains an array of nodes where entries are "Planet" and "Earth".

To iterate items of array 'classes', type foreach i in classes traceLine("handling class '" + i.name + "'..."); and see the result:


handling class 'Planet'...
handling class 'Earth'...

Variable 'i' is an iterator and is declared locally for processing the foreach instruction. We'll see further that the statement local allows declaring a tree to the stack.

What you know about the parse tree in CodeWorker is sufficient to tackle the next section.

2 Parsing our design

CodeWorker provides two different approaches for parsing files.

2.1 The parsing scripts that read tokens

Those that aren't familiar with a BNF representation will perhaps be more self-assured in using a procedure-driven parsing, where control resides within the implementation and where all tokens are explicitly read by a devoted operation. But it means for instance that ignoring blanks and comments must be indicated explicitly between reading of tokens.

The parsing scripts that read tokens are the oldest way to parse into CodeWorker and are the fastest mode too. But it doesn't offer the same flexibility as BNF scripts, which are syntax-oriented.

Below is an example of what a script that reads tokens looks like:

      // file "GettingStarted/SimpleML-token-reading.cws":
      1 declare function readType();
      2
      3 while skipEmptyCpp() {
      4     if !readIfEqualToIdentifier("class") error("'class' expected");
      5     skipEmptyCpp();
      6     local sClassName = readIdentifier();
      7     if !sClassName error("class name expected");
      8     skipEmptyCpp();
      9     if readIfEqualTo(":") {
      10         skipEmptyCpp();
      11         local sParentName = readIdentifier();
      12         if !sParentName error("parent name expected for class '" + sClassName + "'");
      13         skipEmptyCpp();
      14     }
      15     if !readIfEqualTo("{") error("'{' expected");
      16     skipEmptyCpp();
      17     while !readIfEqualTo("}") {
      18         skipEmptyCpp();
      19         readType();
      20         skipEmptyCpp();
      21         local sMemberName = readIdentifier();
      22         if !sMemberName error("attribute or method name expected");
      23         skipEmptyCpp();
      24         if readIfEqualTo("(") {
      25             skipEmptyCpp();
      26             if !readIfEqualTo(")") {
      27                 do {
      28                     skipEmptyCpp();
      29                     local iPosition = getInputLocation();
      30                     local sMode = readIdentifier();
      31                     if !sMode error("parameter type or mode expected");
      32                     if (sMode != "in") && (sMode != "out") && (sMode != "inout") {
      33                         setInputLocation(iPosition);
      34                         set sMode = "";
      35                     }
      36                     skipEmptyCpp();
      37                     readType();
      38                     skipEmptyCpp();
      39                     local sParameterName = readIdentifier();
      40                     if !sParameterName error("parameter name expected");
      41                     skipEmptyCpp();
      42                 } while readIfEqualTo(",");
      43                 if !readIfEqualTo(")") error("')' expected");
      44             }
      45             skipEmptyCpp();
      46         }
      47         if !readIfEqualTo(";") {
      48             error("';' expected to close an attribute, instead of '" + readChar() + "'");
      49         }
      50         skipEmptyCpp();
      51     }
      52 }
      53 traceLine("the file has been read successfully");
      54
      55 function readType() {
      56     local sType = readIdentifier();
      57     if !sType error("type modifier or name expected, instead of '" + readChar() + "'");
      58     if sType == "aggregate" {
      59         skipEmptyCpp();
      60         sType = readIdentifier();
      61         if !sType error("aggregated class name expected");
      62     }
      63     skipEmptyCpp();
      64     if readIfEqualTo("[") {
      65         skipEmptyCpp();
      66         if !readIfEqualTo("]") error("']' expected to close an array declaration");
      67     }
      68 }

line 1: forward declaration of method readType(), so as to start explanations about how to implement BNF clause world ::= [class_declaration]*,
line 3: do a loop while the end of file hasn't been reached, skipping blanks and C++ comments: skipEmptyCpp() returns false only if an error occurs while reading the stream or the file has completed,
line 4: waiting for token "class" as an identifier (doesn't accept "class" as the beginning of another identifier, such as "classes"). If not found, an error occurs. This token announces a class declaration.
line 5: a disadvantage of writing a procedure-driven reading/parsing: don't forget to skip explicitly blanks and comments by yourself,
line 6: populates a local variable with an identifier token that represents the name of the class
line 7: if an identifier token hasn't been found (token is empty), an error is thrown,
line 9: if the file location points to ":", announcing the inheritance, function readIfEqualTo(":") returns true, and the location moves after the matched expression. If it fails, the file location remains the same.
line 15: body of the class declaration expected
line 17: while inside the class body, reading of attribute and method members,
line 19: we don't conform exactly to the BNF: beginning of method and attribute declaration is factorized,
line 21: name of the attribute or method member,
line 24: not any more ambiguity : it starts by a parenthesis when the members is a method,
line 27: the method expects at least one parameter,
line 29: we keep the current file position, to be able to come back if the next token isn't an access mode ("in", "out" or "inout"),
line 33: we were reading a basic type, instead of a parameter access mode: we come back to the beginning of this token and the mode is set as empty (no mode). Of course, it is possible not to waste time like this, and to optimize function readType() by passing the token as a parameter. But here is the occasion of discovering how to handle the file position.
line 37: type of the current parameter is expected,
line 39: name of the current parameter is expected,
line 42: parameters are separated by commas,
line 47: both attributes and methods must finish with a semi colon,
line 48: function readChar() reads just one character, or returns an empty string if the end of file has been reached,
line 53: once the read of file has completed, a message of success is written,
line 55: user-defined function ; may return a value or not. The declaration always starts with keyword function, even if it announces a procedure (no return value). Reading a type is called at several points of the grammar, so the code is factorized in the procedure readType(). It doesn't return any value about success or failure, because an error is thrown in case of syntax mismatch.
line 58: does the keyword is a modifier? If not sType contains a basic type or a class name
line 60: reads the name of the aggregated class
line 64: perhaps that the type is an array, represented by [],
This script seems quite far from the BNF of our simple modeling language, while it implements it in a procedural way. It is able to read a well-formed design file, as our solar system presented at the beginning of the chapter. It doesn't care about populating a parse tree yet, but produces contextual error messages when the design file doesn't conform to the BNF.

Let apply the script on the design file:


parseFree("GettingStarted/SimpleML-token-reading.cws",
        project, "GettingStarted/SolarSystem0.sml");

Output:

the file has been read successfully

Now, let improve the script to allow populating a parse tree:

      // file "GettingStarted/SimpleML-token-parsing.cws":
      1 declare function readType(myType : node);
      2
      3 while skipEmptyCpp() {
      4     if !readIfEqualToIdentifier("class") error("'class' expected");
      5     skipEmptyCpp();
      6     local sClassName = readIdentifier();
      7     if !sClassName error("class name expected");
      8     insert project.listOfClasses[sClassName].name = sClassName;
      9     skipEmptyCpp();
      10     if readIfEqualTo(":") {
      11         skipEmptyCpp();
      12         local sParentName = readIdentifier();
      13         if !sParentName error("parent name expected for class '" + sClassName + "'");
      14         insert project.listOfClasses[sClassName].parent = sParentName;
      15         skipEmptyCpp();
      16     }
      17     if !readIfEqualTo("{") error("'{' expected");
      18     skipEmptyCpp();
      19     local myClass;
      20     ref myClass = project.listOfClasses[sClassName];
      21     while !readIfEqualTo("}") {
      22         skipEmptyCpp();
      23         local myType;
      24         readType(myType);
      25         skipEmptyCpp();
      26         local sMemberName = readIdentifier();
      27         if !sMemberName error("attribute or method name expected");
      28         skipEmptyCpp();
      29         if readIfEqualTo("(") {
      30             insert myClass.listOfMethods[sMemberName].name = sMemberName;
      31             if myType.name != "void" {
      32                 setall myClass.listOfMethods[sMemberName].type = myType;
      33             }
      34             skipEmptyCpp();
      35             if !readIfEqualTo(")") {
      36                 local myMethod;
      37                 ref myMethod = myClass.listOfMethods[sMemberName];
      38                 do {
      39                     skipEmptyCpp();
      40                     local iPosition = getInputLocation();
      41                     local sMode = readIdentifier();
      42                     if !sMode error("parameter type or mode expected");
      43                     if (sMode != "in") && (sMode != "out") && (sMode != "inout") {
      44                         setInputLocation(iPosition);
      45                         set sMode = "";
      46                     }
      47                     skipEmptyCpp();
      48                     local myParameterType;
      49                     readType(myParameterType);
      50                     skipEmptyCpp();
      51                     local sParameterName = readIdentifier();
      52                     if !sParameterName error("parameter name expected");
      53                     insert myMethod.listOfParameters[sParameterName].name = sParameterName;
      54                     setall myMethod.listOfParameters[sParameterName].type = myParameterType;
      55                     if sMode {
      56                         insert myMethod.listOfParameters[sParameterName].name = sMode;
      57                     }
      58                     skipEmptyCpp();
      59                 } while readIfEqualTo(",");
      60                 if !readIfEqualTo(")") error("')' expected");
      61             }
      62             skipEmptyCpp();
      63         } else {
      64             insert myClass.listOfAttributes[sMemberName].name = sMemberName;
      65             setall myClass.listOfAttributes[sMemberName].type = myType;
      66         }
      67         if !readIfEqualTo(";") error("';' expected to close an attribute, instead of '" + readChar() + "'");
      68         skipEmptyCpp();
      69     }
      70 }
      71 traceLine("the file has been parsed successfully");
      72
      73 function readType(myType : node) {
      74     local sType = readIdentifier();
      75     if !sType error("type modifier or name expected, instead of '" + readChar() + "'");
      76     if sType == "aggregate" {
      77         insert myType.isAggregation = true;
      78         skipEmptyCpp();
      79         sType = readIdentifier();
      80         if !sType error("aggregated class name expected");
      81     }
      82     insert myType.name = sType;
      83     if (sType != "int") && (sType != "double") && (sType != "boolean") && (sType != "string") {
      84         insert myType.isObject = true;
      85     }
      86     skipEmptyCpp();
      87     if readIfEqualTo("[") {
      88         skipEmptyCpp();
      89         if !readIfEqualTo("]") error("']' expected to close an array declaration");
      90         insert myType.isArray = true;
      91     }
      92 }

line 8: about parsing, classes are modeled into node project.listOfClasses[sClassName]. Its attribute name contains the value of sClassName.
line 14: this class inherits from a parent, so the optional attribute parent of the class is populated with the value of sParentName,
line 19: to work easier with the current class node project.listOfClasses[sClassName], we define a reference to it, called myClass,
line 23: the class is populated with the characteristics of the member once its declaration has finished. Otherwise, it may confuse between an attribute or a method declaration. So, we should have factorized the type declaration and the name of the member into a common clause, for example.
line 30: about parsing, methods are modeled into node myClass.listOfMethods[sMemberName],
line 31: attribute name is compulsory into a type node, so if myType.name returns "void", there is no return type,
line 36: to work easier with the current class node myClass.listOfMethods[sMemberName], we define a reference to it, called myMethod,
line 53: about parsing, parameters are modeled into node myMethod.listOfParameters[sParameterName],
line 64: about parsing, attributes are modeled into node myClass.listOfAttributes[sMemberName],
line 65: the type is allocated on the stack, so it is copied into branch type (no node reference) integrally,
line 71: once the parsing of file has achieved, a message of success is written,
line 73: function readType() requires a node into which description of type will be populated,
line 77: about parsing, myType.isAggregation contains true if type is an array,
line 82: about parsing, myType.name contains the name of basic type,
line 83: check whether the type is a basic one or a class specifier,
line 84: about parsing, myType.isObject contains true because we suppose that this type is a class specifier (by default: it isn't a basic type),
line 90: about parsing, myType.isArray contains true if type is an array,
The first version of the script was just able to read a well-formed design file written in the simple modeling language. The second version validates the file and populates the parse tree:


parseFree("GettingStarted/SimpleML-token-parsing.cws",
        project, "GettingStarted/SolarSystem0.sml");

Output:

the file has been parsed successfully

2.2 The parsing scripts that describe a BNF syntax

A BNF is more flexible and more synthetic than a procedural description of parsing. CodeWorker accepts parsing scripts that conform to a BNF.

For more information about elements of syntax for a BNF, let have a look to paragraph BNF syntax.

Below is an example of what a BNF script looks like:

      // file "GettingStarted/SimpleML-reading.cwp":
      1 // syntactical clauses:
      2 world ::= #ignore(C++) [class_declaration]* #empty
      3             => { traceLine("file read successfully"); };
      4 class_declaration ::= IDENT:"class" IDENT [':' IDENT]? class_body;
      5 class_body ::= '{' [attribute_decl | method_decl]* '}';
      6 attribute_decl ::= type_specifier IDENT ';';
      7 method_decl ::= [IDENT:"void" | type_specifier] IDENT
      8                 '(' [parameters_decl]? ')' ';';
      9 parameters_decl ::= parameter [',' parameters_decl]*;
      10 parameter ::= [parameter_mode]? type_specifier IDENT;
      11 parameter_mode ::= IDENT:{"in", "inout", "out"};
      12 type_specifier ::= basic_type ['[' ']']?;
      13 basic_type ::= "int" | "boolean" | "double" | "string" | class_specifier;
      14 class_specifier ::= ["aggregate"]? IDENT;
      15
      16 // lexical clauses:
      17 IDENT ::= #!ignore ['a'..'z'|'A'..'Z'|'_']
      18                     ['a'..'z'|'A'..'Z'|'_'|'0'..'9']*;

line 2: the world to model is composed of classes ; some special commands are used:

line 4: a class declaration begins with identifier "class", and IDENT:"class" means that an identifier is expected, and that this identifier is worth "class". This instruction isn't identical to "class" IDENT that validates the expression "classes", where IDENT matches to "es". A class has a name, read by the first IDENT clause call, and may inherit from a parent, read by the second IDENT
line 5: the body of a class is composed of attributes and methods
line 6: the attribute is preceded by its type, and IDENT reads the name of the attribute
line 7: the method has a return type or expects void keyword, and may expect some parameters ; IDENT reads the name of the method
line 9: a comma separates parameters
line 10: an access mode may be specified to the parameter ; the type is then specified, and IDENT reads the name
line 11: a parameter may be passed:

The pattern IDENT:{"in", "inout", "out"} means that the identifier must match with one of the constant strings listed between brackets. It isn't identical to the pattern "in" | "inout" | "out" that validates the beginning of "int".
line 12: a type is a basic type or an array of basic types
line 13: some basic types, including object types
line 14: IDENT reads the class name, and the object may be aggregated
line 17: this clause reads an identifier, such as pretty_pig1 ; #!ignore means that no character is ignored, even if it matches C++ comment or a blank. If we forget clause #!ignore, then IDENT will validate pretty/*comment*/_pig 1 as an identifier.
This BNF script is very close to the BNF of our simple modeling language, and is able to read a well-formed design file, as our solar system presented at the beginning of the chapter. It doesn't care about populating a parse tree yet, and doesn't produce a contextual error message when the design file doesn't conform to the BNF.

Let apply the BNF script on the design file:


parseAsBNF("GettingStarted/SimpleML-reading.cwp",
        project, "GettingStarted/SolarSystem0.sml");

Output:

file read successfully

About differences, note that each BNF rule must end with a semi colon, and that they have to indicate what is their behaviour while encountering blanks and comments.

Now, let improve the BNF script to allow populating a parse tree, or throwing an error when a syntax error has occurred:

      // file "GettingStarted/SimpleML-parsing.cwp":
      1 // syntactical clauses:
      2 world ::= #ignore(C++) [class_declaration]* #empty
      3             => {
      4                 traceLine("file parsed successfully");
      5                 saveProject("Scripts/Tutorial/SolarSystem0.xml");
      6             };
      7 class_declaration ::= IDENT:"class" #continue
      8             IDENT:sClassName
      9                 => insert project.listOfClasses[sClassName].name = sClassName;
      10             [':' #continue IDENT:sParentName
      11                 => insert project.listOfClasses[sClassName].parent = sParentName;
      12             ]?
      13             class_body(project.listOfClasses[sClassName]);
      14 class_body(myClass : node) ::= '{'
      15         [attribute_decl(myClass) | method_decl(myClass)]* '}';
      16 attribute_decl(myClass : node) ::=
      17             => local myType;
      18             type_specifier(myType) IDENT:sAttributeName ';'
      19             => {
      20                 insert myClass.listOfAttributes[sAttributeName].name = sAttributeName;
      21                 setall myClass.listOfAttributes[sAttributeName].type = myType;
      22             };
      23 method_decl(myClass : node) ::=
      24             => local myType;
      25             [IDENT:"void" | type_specifier(myType)]
      26             IDENT:sMethodName '('
      27             #continue
      28                 => {
      29                     insert myClass.listOfMethods[sMethodName].name = sMethodName;
      30                     if myType.name
      31                         setall myClass.listOfMethods[sMethodName].type = myType;
      32                 }
      33             [parameters_decl(myClass.listOfMethods[sMethodName])]? ')' ';';
      34 parameters_decl(myMethod : node) ::=
      35                 parameter(myMethod)
      36                 [',' #continue parameters_decl(myMethod)]*;
      37 parameter(myMethod : node) ::=
      38             [parameter_mode]?:sMode
      39             => local myType;
      40             type_specifier(myType)
      41             IDENT:sParameterName
      42                 => {
      43                     insert myMethod.listOfParameters[sParameterName].name = sParameterName;
      44                     setall myMethod.listOfParameters[sParameterName].type = myType;
      45                     if sMode {
      46                         insert myMethod.listOfParameters[sParameterName].name = sMode;
      47                     }
      48                 };
      49 parameter_mode ::= IDENT:{"in", "inout", "out"};
      50 type_specifier(myType : node) ::=
      51     basic_type(myType)
      52     ['[' #continue ']' => insert myType.isArray = true; ]?;
      53 basic_type(myType : node) ::=
      54     ["int" | "boolean" | "double" | "string"]:myType.name
      55         |
      56     class_specifier(myType);
      57 class_specifier(myType : node) ::=
      58     ["aggregate" => insert myType.isAggregation = true; ]?
      59     IDENT:myType.name => {insert myType.isObject = true; };
      60
      61 IDENT ::= #!ignore ['a'..'z'|'A'..'Z'|'_']
      62                     ['a'..'z'|'A'..'Z'|'_'|'0'..'9']*;

line 2: the pattern [class_declaration]* always matches with the parsed file, so the rule will continue in sequence in any case (supposing that no error has occurred into clause class_declaration) and the end of file will be checked. If not reached, it doesn't write the message "file read successfully",
line 7: once keyword "class" has been matched, there is no ambiguity : we are handling a class declaration and the rule must continue in sequence. To require that, instruction #continue is written after pattern "class". If a pattern of the sequence doesn't match the parsed file, the parser throws a syntax error automatically.
line 8: the identifier that matches with clause call IDENT is assigned to the local variable sClassName : on contrary of other types of script, a new variable is considered as local, instead of an new attribute added to the current node this,
line 9: about parsing, classes are modeled into node project.listOfClasses[sClassName]. Its attribute name contains the value of sClassName.
line 10: if the class inherits from a parent, ':' is necessary followed by an identifier (pattern #continue), and the identifier that matches with clause call IDENT is assigned to the local variable sClassName,
line 11: this class inherits from a parent, so the optional attribute parent of the class is populated with the value of sParentName,
line 14: clause class_body expects an argument: the class node into which the class members must be described (myClass : node),
line 16: the class is populated with the characteristics of the attribute once its declaration has finished. Otherwise, it may confuse with the beginning of a method declaration. To avoid this ambiguity, we should have factorized the type declaration and the name of the member into a common clause, for example.
line 20: about parsing, attributes are modeled into node myClass.listOfAttributes[sAttributeName],
line 21: the type is allocated on the stack, so it is copied into branch type (no node reference) integrally,
line 23: the class is populated with the characteristics of the method once the opened parenthesis is recognized,
line 27: from here, there is no doubt that we are parsing a method declaration,
line 29: about parsing, methods are modeled into node myClass.listOfMethods[sMethodName],
line 30: attribute name is compulsory into a type node, so if condition myType.name returns false, there is no return type (void),
line 36: a parameter declaration is expected after the comma,
line 43: about parsing, parameters are modeled into node myMethod.listOfParameters[sParameterName],
line 52: about parsing, myType.isArray contains true if type is an array,
line 54: about parsing, myType.name contains the name of basic type,
line 58: about parsing, myType.isAggregation contains true if the object is aggregated,
line 59: about parsing, myType.isObject contains true because this type is a class specifier,
line 61: the lexical clause IDENT recognizes identifiers and might be replaced by the predefined clause #readIdentifier, which does the same work,
The first version of the script was just able to read a well-formed design file written in the simple modeling language. The second version validates the file and populates the parse tree:


parseAsBNF("GettingStarted/SimpleML-parsing.cwp",
        project, "GettingStarted/SolarSystem0.sml");

Output:

file parsed successfully

3 Decorating the parse tree

Once our design file has been parsed (either procedure-driven or BNF-driven, we don't care), there is sometimes a little more work to acomplish on the parse tree. It may be verifying consistency of the whole, as checking existence of each class referenced as association or parent. It may also be reorganizing the graph differently, so as to simplify tasks of source code generation. We call it decorating the parse tree in the CodeWorker vocabulary.

The next script proposes to check the existence of each class specifier types and to keep a reference to the node that describes this class specifier. Some nodes change their nature (myClass.parent becomes a reference to the parent node, for example), some other are added (for object types, the new node myType.class keeps a reference to the class):

      // file "GettingStarted/TreeDecoration.cws":
      1 foreach myClass in project.listOfClasses {
      2     if myClass.parent {
      3         if !findElement(myClass.parent, project.listOfClasses)
      4             error("class '" + myClass.parent + "' doesn't exist while class '"
      5                   + myClass.name + "intends to inherit from it");
      6         ref myClass.parent = project.listOfClasses[myClass.parent];
      7     }
      8     foreach myAttribute in myClass.listOfAttributes {
      9         local myType;
      10         ref myType = myAttribute.type;
      11         if myType.isObject {
      12             if !findElement(myType.name, project.listOfClasses)
      13                 error("class '" + myType.name + "' doesn't exist while attribute '"
      14                       + myClass.name + "::" + myAttribute.name + "' refers to it");
      15             ref myType.class = project.listOfClasses[myType.name];
      16         }
      17     }
      18     foreach myMethod in myClass.listOfMethods {
      19         if existVariable(myMethod.type) && myMethod.type.isObject {
      20             localref myType = myMethod.type;
      21             if !findElement(myType.name, project.listOfClasses)
      22                 error("class '" + myType.name + "' doesn't exist while method '"
      23                       + myClass.name + "::" + myMethod.name + "' refers to it");
      24             ref myType.class = project.listOfClasses[myType.name];
      25         }
      26         foreach myParameter in myMethod.listOfParameters {
      27             localref myType = myParameter.type;
      28             if myType.isObject {
      29                 if !findElement(myType.name, project.listOfClasses)
      30                     error("class '" + myType.name
      31                           + "' doesn't exist while method '"
      32                           + myClass.name + "::" + myMethod.name
      33                           + "' refers to it");
      34                 ref myType.class = project.listOfClasses[myType.name];
      35             }
      36         }
      37     }
      38 }

line 1: we iterate all classes,
line 2: if field parent is filled, we check its existence and then, we change it as a reference to the parent class,
line 8: we iterate all attributes of each class,
line 11: only object attributes are interesting,
line 12: check whether the class exists or not into the array node that contains all classes: does the key myType.name exist as an array entry of node project.listOfClasses?
line 15: to optimize navigating into the parse tree later, we keep a reference to the class into new node myType.class,
line 18: we iterate all methods of each class,
line 26: we iterate all parameters of each method,
Now, we dispose of a parsing script that loads well-formed Simple-Modeling designs, and a script that decorates the parse tree. It is time to write a leader script that will take in charge calling tasks of parsing, tree decoration and source code generation:

CodeWorker command line to execute:
-I Scripts/Tutorial/GettingStarted -define DESIGN_FILE=SolarSystem0.sml -script LeaderScript0.cws

      // file "GettingStarted/LeaderScript0.cws":
      1 if !getProperty("DESIGN_FILE")
      2     error("'-define DESIGN_FILE=file' expected on the command line");
      3 traceLine("'Simple Modeling' design file to parse = \""
      4           + getProperty("DESIGN_FILE") + "\"");
      5 parseAsBNF("SimpleML-parsing.cwp",
      6            project, getProperty("DESIGN_FILE"));
      7 #include "TreeDecoration.cws"

line 1: we expect the design as a file that conforms to our Simple-Modeling Language ; the file name is given to the definition preprocessor DESIGN_FILE on the command line by typing -define DESIGN_FILE=SolarSystem0.sml,
line 5: the file is parsed thanks to our previous BNF script,
line 7: the source code for decorating tree is included here, and its content will be executed just after the parsing,

4 Generating code

A script that is intended to source code generation is called a pattern script in the CodeWorker vocabulary. The output file is rewritten completely after the protected areas of user's source code have been preserved.

Such a script begins with a sequence of characters exactly like they must be written into the output file, up to it encounters special character '@' or JSP-like tag '<%'. Then it swaps into script mode, and everything is interpreted as script instructions, up to special character '@' or JSP-like tag '%>' are encountered. Content of the script file is again understood as a sequence of characters to write into the output file, up to the next special character. And it continues swapping from a mode to another...

For convenience, the script mode may be just restrained to an expression (often the name of a variable) whose value is written into the output file.

To do source code generation, we'll need some useful functions, such as converting a Simple-Modeling type to its C++ representation. These functions might be included into the leader script, so as to be shared by all pattern scripts.

We'll discover a new type of functions, called template functions that bring a little generic programming in the language: let imagine that we need function getType(myType : node), to decline for every language we could have to generate (C++ and JAVA in this chapter). You plan to generate an object library from the design you have written in the Simple Modeling Language. This object library will be delivered both in C++ and JAVA, and a technical documentation will come with each of these implementations. This technical documentation will give the signature of methods and the type of attributes in the language the developer will choose. So the C++ documentation will be slightly different from the JAVA one, just at the level of type's spelling. Normally, you'll write the following lines to recover the type depending on the language for which you are producing the documentation:


if doc_language == "C++" {
    sType = getCppType(myParameterType);
} else if doc_language == "JAVA" {
    sType = getJAVAType(myParameterType);
} else {
    error("unrecognized language '" + doc_language + "'");
}

Thanks to the template functions, you may replace the precedent lines by the next one:


sType = getType<doc_language>(myParameterType);
...
function getType<"JAVA">(myType : node) {
    ... // implementation for returning a Java type
}

function getType<"C++">(myType : node) {
    ... // implementation for returning a C++ type
}

During the execution, the function getType<T>(myType : node) resolves on what instantiated function it has to dispatch: either getType<"JAVA">(myType : node) or getType<"C++">(myType : node), depending on what value is assigned to variable doc_language.

Trying to call an instantiated function that doesn't exist, raises an error at runtime. However, one might imagine an implementation by default. For instance:


function getType<T>(myType : node) {
    ... // implementation for any unrecognized language
}

For those that know generic programming with C++ templates, here is a classical example of using template functions:


function f<1>() { return 1; }
function f<N>() { return $N*f<$N - 1$>()$; }
local f10 = f<10>();
if $f10 != 3628800$ error("10! should be worth 3628800");
traceLine("10! = " + f10);

Output:

10! = 3628800

We'll find below all useful functions we'll need for source code generation, including the template function getType<T>(myType : node) we spoke about:

      // file "GettingStarted/SharedFunctions.cws":
      1 function normalizeIdentifier(sName) {
      2     if sName {
      3         if startString(sName, "_")
      4             return "_" + normalizeIdentifier(subString(sName, 1));
      5         set sName = toUpperString(charAt(sName, 0))
      6                     + subString(sName, 1);
      7         local iIndex = findFirstChar(sName, "_.");
      8         if !isNegative(iIndex) {
      9             local sNext = subString(sName, add(iIndex, 1));
      10             return leftString(sName, iIndex)
      11                     + normalizeIdentifier(sNext);
      12         }
      13     }
      14     return sName;
      15 }
      16
      17 function getType<"C++">(myType : node) {
      18     local sType;
      19     if myType.isObject set sType = myType.name + "*";
      20     else if myType.name == "boolean" set sType = "bool";
      21     else if myType.name == "string" set sType = "std::string";
      22     else set sType = myType.name;
      23     if myType.isArray set sType = "std::vector<" + sType + ">";
      24     return sType;
      25 }
      26
      27 function getParameterType<"C++">(myType : node, sMode) {
      28     local sType = getType<"C++">(myType);
      29     if endString(sMode, "out") set sType += "&";
      30     else if (sMode == "in") set sType = "const " + sType + "&";
      31     return sType;
      32 }
      33
      34 function getType<"JAVA">(myType : node) {
      35     local sType;
      36     if myType.name == "string" set sType = "String";
      37     else set sType = myType.name;
      38     if myType.isArray set sType = "java.util.ArrayList/*<" + sType + ">*/";
      39     return sType;
      40 }
      41
      42 function getParameterType<"JAVA">(myType : node, sMode) {
      43     return getType<"JAVA">(myType);
      44 }
      45
      46 function getVariableName(sName, myType : node) {
      47     local sPrefix;
      48     if myType.isArray set sPrefix = "t";
      49     if myType.isObject set sPrefix += "p";
      50     else {
      51         switch(myType.name) {
      52             case "int": set sPrefix += "i";break;
      53             case "double": set sPrefix += "d";break;
      54             case "boolean": set sPrefix += "b";break;
      55             case "string": set sPrefix += "s";break;
      56         }
      57     }
      58     return sPrefix + normalizeIdentifier(sName);
      59 }
      60
      61 function getMethodID(myMethod : node) {
      62     local sMethodID = myMethod.name;
      63     foreach i in myMethod.listOfParameters {
      64         set sMethodID += "." + i.type.name;
      65         if i.type.isArray set sMethodID += "[]";
      66     }
      67     return sMethodID;
      68 }

line 1: this function normalizes identifiers, so as to capitalize the first letter and to suppress '_' or dots after capitalizing the letter that follows: average_speed becomes AverageSpeed, for example. This function is applied on attribute names for instance.
line 3: if the identifier starts with an underscore, it is preserved,
line 7: points to the first character encountered among an underscore and a dot,
line 17: this function returns the C++ type of a Simple-Modeling type node:

line 27: this function returns the C++ type of a Simple-Modeling type node as expected when passed to a method as a parameter type (sMode is worth "in", "out", "inout" or empty string),
line 34: this function returns the JAVA type of a Simple-Modeling type node:

line 42: this function returns the JAVA type of a Simple-Modeling type node as expected when passed to a method as a parameter type (sMode is worth "in", "out", "inout" or empty string, but we don't care about "inout" or "out" for the moment),
line 46: this function returns a variable name whose nomenclature depends on its type,
line 51: the switch statement allows selection among multiple sections of code, depending on the value of expression myType.name, enclosed in parentheses. If no controlling expression (announced by label case) matches with the value, and no default label is present, CodeWorker throws an error.
line 61: this function returns a unique method ID, which is composed from the name of the method and the type of parameters, to avoid confusing protected areas from a method to another,
The next two examples both implement same functionalities, but in different languages (C++ and JAVA). They describe the skeleton of our objects.

4.1 C++ classes

A pattern script may be launched thanks to the procedure generate that expects three parameters:

The next pattern script describes the pattern of a C++ header file:

      // file "GettingStarted/CppObjectHeader.cwt":
      1 #ifndef _@this.name@_h_
      2 #define _@this.name@_h_
      3
      4 @
      5 newFloatingLocation("include files");
      6 @
      7 // this line separates the two insertion points, so as to distinguish them!
      8 @
      9 newFloatingLocation("class declarations");
      10
      11 function populateHeaderDeclarations(myType : node) {
      12     if myType.isObject insertTextOnce(getFloatingLocation("class declarations"), "class " + myType.name + ";" + endl());
      13     if myType.isArray insertTextOnce(getFloatingLocation("include files"), "#include <vector>" + endl());
      14     if myType.name insertTextOnce(getFloatingLocation("include files"), "#include <string>" + endl());
      15 }
      16
      17 @
      18 class @this.name@ @
      19 if existVariable(this.parent) {
      20     insertTextOnce(getFloatingLocation("include files"), "#include \"" + this.parent.name +".h\"" + endl());
      21     @: public @this.parent.name@ @
      22 }
      23 @{
      24     private:
      25 @
      26 foreach i in this.listOfAttributes {
      27     populateHeaderDeclarations(i.type);
      28     @ @getType<"C++">(i.type)@ _@getVariableName(i.name, i.type)@;
      29 @
      30 }
      31 @
      32     public:
      33         @this.name@();
      34         ~@this.name@();
      35
      36         // accessors:
      37 @
      38 foreach i in this.listOfAttributes {
      39     local sVariableName = getVariableName(i.name, i.type);
      40     %> inline <%getType<"C++">(i.type)%> get<%normalizeIdentifier (i.name)%>() const { return _<%sVariableName%>; }
      41         inline void set<%normalizeIdentifier(i.name)@(<%getType <"C++">(i.type)%> <%sVariableName@) { _<%sVariableName%> = <%sVariableName%>; }
      42 @
      43 }
      44 @
      45         // methods:
      46 @
      47 foreach i in this.listOfMethods {
      48     @ virtual @
      49     if existVariable(i.type) {
      50         populateHeaderDeclarations(i.type);
      51         @@getType<"C++">(i.type)@@
      52     } else {
      53         @void@
      54     }
      55     @ @i.name@(@
      56     foreach j in i.listOfParameters {
      57         if !first(j) {
      58             @, @
      59         }
      60         populateHeaderDeclarations(j.type);
      61         @@getParameterType<"C++">(j.type, j.mode)@ @getVariableName(j.name, j.type)@@
      62     }
      63     @);
      64 @
      65 }
      66 @
      67     private:
      68         @this.name@(const @this.name@&);
      69         @this.name@& operator =(const @this.name@&);
      70 };
      71
      72 #endif

line 1: the value of attribute this.name is written to the output file, where this points to a node that describes the current class. Note that this is facultative, and is assigned by the caller of procedure generate that runs this script.
line 5: put one anchor for including all files that we'll encounter as compulsory, while iterating attributes or methods. Example: if an attribute is an array, we'll need to include the STL header vector at this position of the file: #include <vector>. This insertion point is called "include files".
line 6: to avoid that the two floating locations "include files" and "class declarations" (described just below) point to the same file position, an empty line is added,
line 9: put one anchor for announcing all classes that we'll encounter as referenced, while iterating attributes or methods. Example: if an attribute is an object Planet, we'll need to write class Planet; at this position of the file. This insertion point is called "class declarations".
line 11: this function is called on every type encountered while iterating attributes and methods. Its role is to populate the "include files" and "class declarations" areas.
line 12: the type of an object must be declared at the beginning of the header, otherwise the compiler will not recognize it : the class is declared once only in the insertion point called "class declarations". Use of function insertTextOnce assures that if this class has already been inserted before, it will not be twice.
line 13: this type is an array, so the declaration of std::vector must be included to the insertion point called "include files",
line 14: this type is a string, so the declaration of std::string must be included to the insertion point called "include files",
line 19: if the class inherits from a parent class, this relationship must be written,
line 20: the parent class must be declared,
line 26: declaration of all attributes,
line 27: does the type of the attribute need some backward declarations?
line 38: accessors to each attribute,
line 40: there are two symbols to swap between writing a sequence of characters and interpreting script ; we have used the symbol '@', and now we illustrate the use of tags '<% and '%>,
line 41: you can melt the two swapping symbol, but it is more difficult to read, so not very interesting!
line 47: declaration of all methods,
line 48: each method might be overloaded by subclasses,
line 49: the return type of the method is translated to C++,
line 50: does the return type of the method need some backward declarations?
line 51: expression getType<"C++">(i.type) to evaluate is embedded between double '@'. The first one allow swapping to the sequence of characters mode, but there is no characters to write. The second one allows swapping to the script mode, which is reduced just to evaluate the expression. The two final '@' take the same role as seen before.
line 56: parameters of the method are iterated to be written in C++
line 57: if iterator j doesn't point to the first parameter, a comma makes a separation with the precedent,
line 60: does the type of the parameter need some backward declarations?
Let's continue with the pattern that describes the skeleton of a C++ body file:

      // file "GettingStarted/CppObjectBody.cwt":
      1 #ifdef WIN32
      2 #pragma warning(disable : 4786)
      3 #endif
      4
      5 @
      6 setProtectedArea("include files");
      7 @
      8 #include "@this.name@.h"
      9
      10 @this.name@::@this.name@()@
      11 local bAtLeastOne = false;
      12 foreach i in this.listOfAttributes {
      13     if !i.type.isArray && (i.type.name != "string") {
      14         if bAtLeastOne {
      15             @, @
      16         } else {
      17             @ : @
      18             set bAtLeastOne = true;
      19         }
      20         @_@getVariableName(i.name, i.type)@(@
      21         if i.type.isObject {
      22             @0L@
      23         } else {
      24             switch(i.type.name) {
      25                 case "int":
      26                     @0@
      27                     break;
      28                 case "double":
      29                     @0.0@
      30                     break;
      31                 case "boolean":
      32                     @false@
      33                     break;
      34             }
      35         }
      36         @)@
      37     }
      38 }
      39 @ {
      40 }
      41
      42 @this.name@::~@this.name@() {
      43 @
      44 foreach i in this.listOfAttributes {
      45     if i.type.isAggregation && i.type.isObject {
      46         local sAttributeName = "_" + getVariableName(i.name, i.type);
      47         local sIndex = "iterate" + normalizeIdentifier(i.name);
      48         if i.type.isArray {
      49             @ for (std::vector<@i.name@*>::const_iterator @sIndex@ = @sAttributeName@.begin(); @sIndex@ != @sAttributeName@.end(); ++@sIndex@) {
      50         delete *@sIndex@;
      51     }
      52 @
      53         } else {
      54             @ delete @sAttributeName@;
      55 @
      56         }
      57     }
      58 }
      59 @}
      60
      61 @
      62 foreach i in this.listOfMethods {
      63     if existVariable(i.type) {
      64         @@getType<"C++">(i.type)@@
      65     } else {
      66         @void@
      67     }
      68     @ @this.name@::@i.name@(@
      69     foreach j in i.listOfParameters {
      70         if !first(j) {
      71             @, @
      72         }
      73         @@getParameterType<"C++">(j.type, j.mode)@ @getVariableName(j.name, j.type)@@
      74     }
      75     @) {
      76 @
      77         setProtectedArea(getMethodID(i));
      78 @}
      79 @
      80 }

line 1: Visual C++-specific pragma must be added to prevent from intempestive warnings about template class instantiation of std::vector<T> in DEBUG mode!
line 6: the developer will add here all include files he will need for implementation of methods,
line 8: the header of this body is compulsory,
line 11: this part concerns the initialization of attributes. Some attributes, such as strings and vectors of the STL don't require to be initialized explicitly. It justifies the declaration of variable bAtLeastOne that is worth false as long as no attribute has been initialized yet. We'll see why below.
line 13: arrays and strings are skipped,
line 15: if it isn't the first attribute to be initialized, a comma make a separation with the precedent,
line 17: if it is the first attribute to be initialized, a colon is expected to announce the beginning of initializations
line 18: now, there is at least one attribute to be initialized,
line 21: attribute is populated with the default value corresponding to its type,
line 44: aggregated objects must be deleted before leaving this instance,
line 49: all elements of an aggregated array must be deleted
line 54: the aggregated object is deleted
line 62: implementation of all methods,
line 63: the return type of the method is translated to C++,
line 69: parameters of the method are iterated to be written in C++
line 70: if iterator j doesn't point to the first parameter, a comma makes a separation with the precedent,
line 77: a protected area is inserted, whose key is the method ID,
The leader script has to be improved to reclaim generation of C++ files:

CodeWorker command line to execute:
-I Scripts/Tutorial -path . -define DESIGN_FILE=GettingStarted/SolarSystem0.sml -script GettingStarted/LeaderScript1.cws

      // file "GettingStarted/LeaderScript1.cws":
      1 if !getProperty("DESIGN_FILE")
      2     error("'-define DESIGN_FILE=file' expected on the command line");
      3 traceLine("'Simple Modeling' design file to parse = \""
      4           + getProperty("DESIGN_FILE") + "\"");
      5 parseAsBNF("GettingStarted/SimpleML-parsing.cwp",
      6            project, getProperty("DESIGN_FILE"));
      7 #include "TreeDecoration.cws"
      8
      9 #include "SharedFunctions.cws"
      10 foreach myClass in project.listOfClasses {
      11     traceLine("generating class '" + myClass.name + "' ...");
      12     generate("GettingStarted/CppObjectHeader.cwt", myClass,
      13              getWorkingPath() + "Scripts/Tutorial/GettingStarted/Cpp/"
      14              + myClass.name + ".h");
      15     generate("GettingStarted/CppObjectBody.cwt", myClass,
      16              getWorkingPath() + "Scripts/Tutorial/GettingStarted/Cpp/"
      17              + myClass.name + ".cpp");
      18 }

line 9: all useful functions for source code generation are loaded here,
line 10: all classes are iterated and their C++ header and body are generated
line 12: instruction generate is applied on a pattern script and its second argument expects a node that will be seen as variable 'this' into the pattern script,
line 13: getWorkingPath() is worth the output path passed to the command line via the option '-path',

Output:

'Simple Modeling' design file to parse = "GettingStarted/SolarSystem0.sml"
file parsed successfully
generating class 'Planet' ...
generating class 'Earth' ...
generating class 'SolarSystem' ...

Let have a look on some generated files:

      // file "GettingStarted/Cpp/SolarSystem.h":
      #ifndef _SolarSystem_h_
      #define _SolarSystem_h_
     
      #include <vector>
      #include <string>
     
      // this line separates the two insertion points, so as to distinguish them!
      class Planet;
     
      class SolarSystem {
          private:
              std::vector<Planet*> _tpPlanets;
     
          public:
              SolarSystem();
              ~SolarSystem();
     
              // accessors:
              inline std::vector<Planet*> getPlanets() const { return _tpPlanets; }
              inline void setPlanets(std::vector<Planet*> tpPlanets) { _tpPlanets = tpPlanets; }
     
              // methods:
     
          private:
              SolarSystem(const SolarSystem&);
              SolarSystem& operator =(const SolarSystem&);
      };
     
      #endif

      // file "GettingStarted/Cpp/Planet.cpp":
      1 #ifdef WIN32
      2 #pragma warning(disable : 4786)
      3 #endif
      4
      5 //##protect##"include files"
      6 //##protect##"include files"
      7
      8 #include "Planet.h"
      9
      10 Planet::Planet() : _dDiameter(0.0) {
      11 }
      12
      13 Planet::~Planet() {
      14 }
      15
      16 double Planet::getDistanceToSun(int iDay, int iMonth, int iYear) {
      17 //##protect##"getDistanceToSun.int.int.int"
      18 //##protect##"getDistanceToSun.int.int.int"
      19 }

line 1: Visual C++-specific pragma must be added to prevent from intempestive warnings about template class instantiation of std::vector<T> in DEBUG mode!

4.2 JAVA classes

Some modelers don't separate clearly the design and its implementation, but theoretically, no language-dependent data has to be included into the design. The modeling language should be improved to take into account some finer modeling aspects that lead to choose a mapping (for parameter types, for example) to the implementation language. The logic of a source code generation process is to factorize as most as possible the knowledge at the design level. We'll speak longer about it further.

Our design is totally independent from the implementation : a string isn't explicitly a const std::string& or a std::string in C++, but the pattern script decides according to the context whether it is more judicious to choose the first C++ mapping or the second one.

This independence allows us implementing the same functionalities as in C++, but in JAVA now:

      // file "GettingStarted/JAVAObject.cwt":
      1 package solarsystem;
      2
      3 public class @this.name@ @
      4 if existVariable(this.parent) {
      5     @extends @this.parent.name@ @
      6 }
      7 @{
      8 @
      9 foreach i in this.listOfAttributes {
      10     @ private @getType<"JAVA">(i.type)@ _@getVariableName(i.name, i.type)@;
      11 @
      12 }
      13 @
      14     public @this.name@() {}
      15
      16     // accessors:
      17 @
      18 foreach i in this.listOfAttributes {
      19     local sVariableName = getVariableName(i.name, i.type);
      20     @ public @getType<"JAVA">(i.type)@ get@normalizeIdentifier(i.name)@() { return _@sVariableName@; }
      21     public void set@normalizeIdentifier(i.name)@(@getType<"JAVA">(i.type)@ @sVariableName@) { _@sVariableName@ = @sVariableName@; }
      22 @
      23 }
      24 @
      25         // methods:
      26 @
      27 foreach i in this.listOfMethods {
      28     @ public @
      29     if existVariable(i.type) {
      30         @@getType<"JAVA">(i.type)@@
      31     } else {
      32         @void@
      33     }
      34     @ @i.name@(@
      35     foreach j in i.listOfParameters {
      36         if !first(j) {
      37             @, @
      38         }
      39         @@getParameterType<"JAVA">(j.type, j.mode)@ @getVariableName(j.name, j.type)@@
      40     }
      41     @) {
      42 @
      43     setProtectedArea(getMethodID(i));
      44 @ }
      45
      46 @
      47 }
      48 @}

line 4: if the class inherits from a parent class, this relationship must be written,
line 9: declaration of all attributes,
line 18: accessors to each attribute,
line 27: declaration of all methods,
The leader script has to be improved to reclaim generation of JAVA files:

CodeWorker command line to execute:
-I Scripts/Tutorial -path . -define DESIGN_FILE=GettingStarted/SolarSystem0.sml -script GettingStarted/LeaderScript2.cws

      // file "GettingStarted/LeaderScript2.cws":
      1 if !getProperty("DESIGN_FILE")
      2     error("'-define DESIGN_FILE=file' expected on the command line");
      3 traceLine("'Simple Modeling' design file to parse = \""
      4           + getProperty("DESIGN_FILE") + "\"");
      5 parseAsBNF("GettingStarted/SimpleML-parsing.cwp",
      6            project, getProperty("DESIGN_FILE"));
      7 #include "TreeDecoration.cws"
      8
      9 #include "SharedFunctions.cws"
      10 foreach myClass in project.listOfClasses {
      11     traceLine("generating class '" + myClass.name + "' ...");
      12     generate("GettingStarted/CppObjectHeader.cwt", myClass, getWorkingPath() + "Scripts/Tutorial/GettingStarted/Cpp/" + myClass.name + ".h");
      13     generate("GettingStarted/CppObjectBody.cwt", myClass, getWorkingPath() + "Scripts/Tutorial/GettingStarted/Cpp/" + myClass.name + ".cpp");
      14     generate("GettingStarted/JAVAObject.cwt", myClass, getWorkingPath() + "Scripts/Tutorial/GettingStarted/JAVA/solarsystem/" + myClass.name + ".java");
      15 }

line 14: generates the JAVA implementation of the current design class,

Output:

'Simple Modeling' design file to parse = "GettingStarted/SolarSystem0.sml"
file parsed successfully
generating class 'Planet' ...
generating class 'Earth' ...
generating class 'SolarSystem' ...

Let have a look on some generated files:

      // file "GettingStarted/JAVA/solarsystem/SolarSystem.java":
      package solarsystem;
     
      public class SolarSystem {
          private java.util.ArrayList/*<Planet>*/ _tpPlanets;
     
          public SolarSystem() {}
     
          // accessors:
          public java.util.ArrayList/*<Planet>*/ getPlanets() { return _tpPlanets; }
          public void setPlanets(java.util.ArrayList/*<Planet>*/ tpPlanets) { _tpPlanets = tpPlanets; }
     
              // methods:
      }

      // file "GettingStarted/JAVA/solarsystem/Planet.java":
      package solarsystem;
     
      public class Planet {
          private double _dDiameter;
     
          public Planet() {}
     
          // accessors:
          public double getDiameter() { return _dDiameter; }
          public void setDiameter(double dDiameter) { _dDiameter = dDiameter; }
     
              // methods:
          public double getDistanceToSun(int iDay, int iMonth, int iYear) {
      //##protect##"getDistanceToSun.int.int.int"
      //##protect##"getDistanceToSun.int.int.int"
          }
     
      }

5 Expanding a file

Expanding a file consists of generating code to some determined points of the file. These points are called markups and are noted ##markup##"name-of-the-markup", surrounded by comment delimiters.

For example, a valid markup inlayed in a C++ file could be:
//##markup##"factory"
and a valid markup inlayed in an HTML file could be:
<!- -##markup##"classes"- ->

Some data may accompany the markup. The block of data is put between tags ##data##:
//##markup##"switch(sText)"
//##data##
//Customer
//Videostore
//##data##
You obtain the data attached to the current markup key by calling the function getMarkupValue() (see
getMarkupValue()). This example extends the C++/Java functionalities with a switch statement working on a string expression.

A pattern script intended to expand code is launched thanks to the procedure expand that expects three parameters:

Each time CodeWorker will encounter a markup, it will call the pattern script that will decide how to populate it. The code generated by the pattern script for this markup is surrounded by tags ##begin##"name-of-the-markup" and ##end##"name-of-the-markup", automatically added by the interpreter. If some protected areas were put into the generated code, they are preserved the next time the expansion is required.

Note that CodeWorker doesn't change what is written outside the markups and their begin/end delimiters.

Starting from a (very simple) HTML canvas, we'll generate an HTML documentation to our project SolarSystem. Here is the canvas that we would like to keep for all our projects:

      // file "GettingStarted/defaultDocumentation.html":
      <HTML>
          <HEAD>
              <TITLE>some title...</TITLE>
          </HEAD>
          <BODY>
              <H1>some title...</H1>
              some global documentation...
      <!--##markup##"classes presentation"-->
          </BODY>
      </HTML>

We'll copy it to "GettingStarted/SolarSystem0.html" to populate it with the characteristics of our current project. The pattern script that will be launched to expand "GettingStarted/SolarSystem0.html" is:

      // file "GettingStarted/HTMLDocumentation.cwt":
      1 @
      2 if getMarkupKey() == "classes presentation" {
      3     foreach i in project.listOfClasses {
      4         @
      5         <H2><A href="#@i.name@">@i.name@</A></H2>
      6 @
      7         setProtectedArea(i.name + ":presentation");
      8         if !isEmpty(i.listOfAttributes) {
      9             @
      10         <TABLE border="1" cellpadding="3" cellspacing="0" width="100%">
      11             <TR BGCOLOR="#CCCCFF">
      12                 <TD><B>Type</B></TD>
      13                 <TD><B>Attribute name</B></TD>
      14                 <TD><B>Description</B></TD>
      15             </TR>
      16 @
      17             foreach j in i.listOfAttributes {
      18                 @ <TR>
      19                 <TD>@composeHTMLLikeString(getType<this.language> (j.type))@</TD>
      20                 <TD>@j.name@</TD>
      21                 <TD>
      22 @
      23                 setProtectedArea(i.name + "::" + j.name + ":description");
      24                 @
      25                 </TD>
      26             </TR>
      27 @
      28             }
      29             @ </TABLE>
      30 @
      31         }
      32         if !isEmpty(i.listOfMethods) {
      33             @
      34         <UL>
      35 @
      36             foreach j in i.listOfMethods {
      37                 @ <LI>@
      38                 if existVariable(j.type) {
      39                     @function @composeHTMLLikeString(getType <this.language>(j.type))@ @
      40                 } else {
      41                     @procedure@
      42                 }
      43                 @<B>@j.name@</B>(@
      44                 foreach k in j.listOfParameters {
      45                     if !first(k) {
      46                         @, @
      47                     }
      48                     @@composeHTMLLikeString(getParameterType <this.language>(k.type, k.mode))@ <I>@getVariableName(k.name, k.type)@</I>@
      49                 }
      50                 @)
      51                 <BR>
      52 @
      53                 setProtectedArea(i.name + "::" + getMethodID(j) + ":description");
      54                 @
      55             </LI>
      56 @
      57             }
      58             @ </UL>
      59 @
      60         }
      61     }
      62 }

line 2: the predefined function getMarkupKey() returns the name of the markup to expand,
line 3: the markup is worth "classes presentation", and so, we'll describe all classes
line 7: a protected area is embedded here, which has to be populated by hand into the expanded file for describing the class,
line 9: attributes are presented into a table,
line 18: the language into which types have to be expressed is given by this.language, and is worth "C++" or "JAVA" ; don't forget to convert the type to the HTML syntax, because of '<' or '>' to convert respectively to '&lt' or '&gt' for instance. Use the predefined function composeHTMLLikeString() to do this process.
line 23: a protected area is embedded here, which has to be populated by hand into the expanded file for describing the attribute,
line 37: methods are presented into unordered lists,
line 53: a protected area is embedded here, which has to be populated by hand into the expanded file for describing the method,
Now, we have to change the leader script, so as to take into account the generation of the documentation:

CodeWorker command line to execute:
-I Scripts/Tutorial -path . -define DESIGN_FILE=GettingStarted/SolarSystem0.sml -script GettingStarted/LeaderScript3.cws

      // file "GettingStarted/LeaderScript3.cws":
      1 if !getProperty("DESIGN_FILE")
      2     error("'-define DESIGN_FILE=file' expected on the command line");
      3 traceLine("'Simple Modeling' design file to parse = \""
      4           + getProperty("DESIGN_FILE") + "\"");
      5 parseAsBNF("GettingStarted/SimpleML-parsing.cwp",
      6            project, getProperty("DESIGN_FILE"));
      7 #include "TreeDecoration.cws"
      8
      9 #include "SharedFunctions.cws"
      10 foreach myClass in project.listOfClasses {
      11     traceLine("generating class '" + myClass.name + "' ...");
      12     generate("GettingStarted/CppObjectHeader.cwt", myClass, getWorkingPath() + "Scripts/Tutorial/GettingStarted/Cpp/" + myClass.name + ".h");
      13     generate("GettingStarted/CppObjectBody.cwt", myClass, getWorkingPath() + "Scripts/Tutorial/GettingStarted/Cpp/" + myClass.name + ".cpp");
      14     generate("GettingStarted/JAVAObject.cwt", myClass, getWorkingPath() + "Scripts/Tutorial/GettingStarted/JAVA/solarsystem/" + myClass.name + ".java");
      15 }
      16 if !existFile("Scripts/Tutorial/GettingStarted/SolarSystem0.html") {
      17     copyFile("Scripts/Tutorial/GettingStarted/defaultDocumentation.html", "Scripts/Tutorial/GettingStarted/SolarSystem0.html");
      18 }
      19
      20 local myDocumentationContext;
      21 insert myDocumentationContext.language = "C++";
      22 traceLine("generating the HTML documentation...");
      23 setCommentBegin("<!--");
      24 setCommentEnd("-->");
      25 expand("GettingStarted/HTMLDocumentation.cwt",
      26         myDocumentationContext, getWorkingPath()
      27         + "Scripts/Tutorial/GettingStarted/SolarSystem0.html");

line 16: copy the default empty HTML documentation to "SolarSystem0.html" if it doesn't exist yet,
line 20: the myDocumentationContext variable will be passed to the procedure expand(),
line 21: an attribute language is added to the myDocumentationContext variable, which specifies whether types must be expressed in C++ or in JAVA into the HTML documentation,
line 23: don't forget to specify comment delimiters that are expected by an HTML file,
line 25: the procedure expand() allow populating "SolarSystem0.html" with the characteristics of the project automatically,

Output:

'Simple Modeling' design file to parse = "GettingStarted/SolarSystem0.sml"
file parsed successfully
generating class 'Planet' ...
generating class 'Earth' ...
generating class 'SolarSystem' ...
generating the HTML documentation...

After executing this script, we obtain the following HTML documentation, where protected areas have to be populated, so as to describe classes and attributes and methods:

      // file "GettingStarted/SolarSystem0.html":
      <HTML>
          <HEAD>
              <TITLE>some title...</TITLE>
          </HEAD>
          <BODY>
              <H1>some title...</H1>
              some global documentation...
      <!--##markup##"classes presentation"--><!--##begin##"classes presentation"-->
              <H2><A href="#Planet">Planet</A></H2>
      <!--##protect##"Planet:presentation"--><!--##protect##"Planet:presentation"-->
              <TABLE border="1" cellpadding="3" cellspacing="0" width="100%">
                  <TR BGCOLOR="#CCCCFF">
                      <TD><B>Type</B></TD>
                      <TD><B>Attribute name</B></TD>
                      <TD><B>Description</B></TD>
                  </TR>
                  <TR>
                      <TD>double</TD>
                      <TD>diameter</TD>
                      <TD>
      <!--##protect##"Planet::diameter:description"--><!--##protect##"Planet::diameter:description"-->
                      </TD>
                  </TR>
              </TABLE>
     
              <UL>
                  <LI>function double <B>getDistanceToSun</B>(int <I>iDay</I>, int <I>iMonth</I>, int <I>iYear</I>)
                      <BR>
      <!--##protect##"Planet::getDistanceToSun.int.int.int:description"--><!--##protect##"Planet::getDistanceToSun.int.int.int:description"-->
                  </LI>
              </UL>
     
              <H2><A href="#Earth">Earth</A></H2>
      <!--##protect##"Earth:presentation"--><!--##protect##"Earth:presentation"-->
              <TABLE border="1" cellpadding="3" cellspacing="0" width="100%">
                  <TR BGCOLOR="#CCCCFF">
                      <TD><B>Type</B></TD>
                      <TD><B>Attribute name</B></TD>
                      <TD><B>Description</B></TD>
                  </TR>
                  <TR>
                      <TD>std::vector&lt;std::string&gt;</TD>
                      <TD>countryNames</TD>
                      <TD>
      <!--##protect##"Earth::countryNames:description"--><!--##protect##"Earth::countryNames:description"-->
                      </TD>
                  </TR>
              </TABLE>
     
              <H2><A href="#SolarSystem">SolarSystem</A></H2>
      <!--##protect##"SolarSystem:presentation"--><!--##protect##"SolarSystem:presentation"-->
              <TABLE border="1" cellpadding="3" cellspacing="0" width="100%">
                  <TR BGCOLOR="#CCCCFF">
                      <TD><B>Type</B></TD>
                      <TD><B>Attribute name</B></TD>
                      <TD><B>Description</B></TD>
                  </TR>
                  <TR>
                      <TD>std::vector&lt;Planet*&gt;</TD>
                      <TD>planets</TD>
                      <TD>
      <!--##protect##"SolarSystem::planets:description"--><!--##protect##"SolarSystem::planets:description"-->
                      </TD>
                  </TR>
              </TABLE>
      <!--##end##"classes presentation"-->
          </BODY>
      </HTML>

We'll suppose that the skeleton of the HTML documentation is acceptable for us. It will evolve with our design "SolarSystem0.sml": if some classes or some members are added or removed, the skeleton will take these changes into account. When the reference to a protected area disappears, because the member it was linked to changes its name or is removed, the protected area is kept up at the end of the file.

Now, we have to populate protected areas and parts of text that are put outside the markups, so as to complete our documentation. This work has been done to "SolarSystem1.html".

6 Translating a file

Up to now, we discovered parsing on one side and source code generation on the other side. The translation mode merges the two: it offers to parse a file conforming to a BNF and to translate it into another format, all in the same translation script.

A translation script looks like a BNF-driven parsing script, but where:

are allowed into compound statements that are announced by '=>'.

Outputs are written into another file, so the input file is preserved. The procedure that takes the translation in charge is called translate().

Little practical example: all our documentation has been written in HTML, but we would like to translate it to LaTeX, into our own format. Why not?

First step, we must be able to read an HTML file according to a BNF representation. The corresponding BNF-driven script we have to write is restricted to be able to write our file "SolarSystem1.html":

      // file "GettingStarted/HTML-parsing.cwp":
      1 #noCase
      2
      3 HTML ::= #ignore(HTML) #continue '<' "HTML" '>' HTMLHeader HTMLBody '<' '/' "HTML" '>' #empty;
      4 HTMLHeader ::= '<' #continue "HEAD" '>' [~['<' '/' "HEAD" '>']]* '<' '/' "HEAD" '>';
      5 HTMLBody ::= '<' #continue "BODY" '>' HTMLText '<' '/' "BODY" '>';
      6 HTMLText ::=
      7         [
      8             ~'<'
      9                 |
      10             !['<' '/'] #continue '<'
      11                 #readIdentifier:sTag HTMLNextOfTag<sTag>
      12         ]*;
      13 HTMLNextOfTag<"H1"> ::= #continue '>' HTMLText '<' '/' "H1" '>';
      14 HTMLNextOfTag<"H2"> ::= #continue '>' HTMLText '<' '/' "H2" '>';
      15 HTMLNextOfTag<"A"> ::= [HTMLAttribute]* #continue '>' HTMLText '<' '/' 'A' '>';
      16 HTMLNextOfTag<"TABLE"> ::= [HTMLAttribute]* #continue '>' [HTMLTag("TR")]* '<' '/' "TABLE" '>';
      17 HTMLTag(sTag : value) ::= '<' #readText(sTag) #continue HTMLNextOfTag<sTag>;
      18 HTMLNextOfTag<"TR"> ::= [HTMLAttribute]* #continue '>' [HTMLTag("TD")]* '<' '/' "TR" '>';
      19 HTMLNextOfTag<"TD"> ::= [HTMLAttribute]* #continue '>' HTMLText '<' '/' "TD" '>';
      20 HTMLNextOfTag<"UL"> ::= [HTMLAttribute]* #continue '>' [HTMLTag("LI")]* '<' '/' "UL" '>';
      21 HTMLNextOfTag<"LI"> ::= [HTMLAttribute]* #continue '>' HTMLText '<' '/' "LI" '>';
      22 HTMLNextOfTag<"B"> ::= #continue '>' HTMLText '<' '/' "B" '>';
      23 HTMLNextOfTag<"I"> ::= #continue '>' HTMLText '<' '/' "I" '>';
      24 HTMLNextOfTag<"FONT"> ::= [HTMLAttribute]* #continue '>' HTMLText '<' '/' "FONT" '>';
      25 HTMLNextOfTag<"BR"> ::= ['/']? #continue '>';
      26 HTMLAttribute ::= #readIdentifier ['=' #continue [STRING_LITERAL | WORD_LITERAL]]?;
      27
      28
      29 STRING_LITERAL ::= #!ignore '\"' [~'\"']* '\"';
      30 WORD_LITERAL ::= #!ignore [~['>' | '/' | ' ' | '\t']]+;

line 1: we don't care about the case: <BODY> and <Body> must be recognized as identical for instance,
line 6: the clause HTMLText reads the value between tags,
line 11: the best way to assure an easy extension of the grammar: to declare a template clause for describing the reading of a tag,
line 17: a clause to read a determined tag: the token #readText matches the input stream to the evaluated expression passed in parameter and the rest is read by the template clause that describes the reading of a tag,
Second step, we have to improve the BNF-driven script to add some features for generating the LaTeX code properly. Don't be afraid about the length of the source code, but go forward to the notes directly:

      // file "GettingStarted/HTML2LaTeX.cwp":
      1 #noCase
      2
      3 HTML2LaTeX ::= #ignore(HTML) #continue '<' "HTML" '>' HTMLHeader HTMLBody '<' '/' "HTML" '>' #empty;
      4 HTMLHeader ::= '<' #continue "HEAD" '>' [~['<' '/' "HEAD" '>']]* '<' '/' "HEAD" '>';
      5 HTMLBody ::= '<' #continue "BODY" '>' HTMLText '<' '/' "BODY" '>';
      6 HTMLText ::= #!ignore
      7         [
      8             '&' #continue #readIdentifier:sEscape HTMLEscape<sEscape> ';'
      9                 |
      10             ~'<':cChar => writeText(cChar);
      11                 |
      12             !['<' blanks '/']
      13             [
      14                 "<!--" #continue [~"-->"]* "-->"
      15                     |
      16                 '<' #continue #ignore(HTML) #readIdentifier:sTag HTMLNextOfTag<sTag>
      17             ]
      18         ]*;
      19 HTMLEscape<"lt"> ::= => {@<@};
      20 HTMLEscape<"gt"> ::= => {@>@};
      21 HTMLTag(sTag : value) ::= '<' #readText(sTag) #continue HTMLNextOfTag<sTag>;
      22 HTMLNextOfTag<"H1"> ::=
      23         #continue '>' => {@\subsection{@}
      24         HTMLText
      25         '<' '/' "H1" '>' => {@}@};
      26 HTMLNextOfTag<"H2"> ::=
      27         #continue '>' => {@\subsubsection{@}
      28         HTMLText
      29         '<' '/' "H2" '>' => {@}@};
      30 HTMLNextOfTag<"A"> ::= [HTMLAttribute]* #continue '>' HTMLText '<' '/' 'A' '>';
      31 HTMLNextOfTag<"TABLE"> ::=
      32         [HTMLAttribute]* #continue '>' => {
      33             @\begin{table@
      34             newFloatingLocation("table PDF suffix");
      35             @}{@
      36             newFloatingLocation("table columns");
      37             @}{.5}@
      38         }
      39         => local sPDFTableSuffix;
      40         HTMLTableTitle(sPDFTableSuffix)
      41         [HTMLTableLine(sPDFTableSuffix)]*
      42         '<' '/' "TABLE" '>' => {@\end{table@sPDFTableSuffix@}
      43 @};
      44 HTMLTableTitle(sPDFTableSuffix : node) ::=
      45     '<' "TR" [HTMLAttribute]*
      46     #continue '>'
      47     [HTMLTableCol(sPDFTableSuffix)]*
      48     '<' '/' "TR" '>' => {
      49         insertText(getFloatingLocation("table PDF suffix"), sPDFTableSuffix);
      50         writeText(endl());
      51     };
      52 HTMLTableCol(sPDFTableSuffix : node) ::=
      53     '<' "TD" [HTMLAttribute]* #continue '>' => {
      54         @{@
      55         if !sPDFTableSuffix insertText(getFloatingLocation("table columns"), "l");
      56         else insertText(getFloatingLocation("table columns"), "|l");
      57         set sPDFTableSuffix += "i";
      58     }
      59     '<' 'B' '>' [#!ignore [~'<':cChar => writeText(cChar);]*] '<' '/' 'B' '>'
      60     '<' '/' "TD" '>' => {@}@};
      61 HTMLTableLine(sPDFTableSuffix : value) ::=
      62         '<' "TR" [HTMLAttribute]* #continue '>' => {@\line@sPDFTableSuffix@@}
      63         [HTMLTag("TD")]* '<' '/' "TR" '>' => {writeText(endl());};
      64 HTMLNextOfTag<"TD"> ::=
      65         [HTMLAttribute]* #continue '>' => {@{@}
      66         HTMLCellText '<' '/' "TD" '>' => {@}@};
      67 HTMLCellText ::= #!ignore
      68         [
      69             '&' #continue #readIdentifier:sEscape HTMLEscape<sEscape> ';'
      70                 |
      71             ['\r']? ['\n'] => {@ @}
      72                 |
      73             ~'<':cChar => writeText(cChar);
      74                 |
      75             !['<' blanks '/']
      76             [
      77                 "<!--" #continue [~"-->"]* "-->"
      78                     |
      79                 '<' #continue #ignore(HTML) #readIdentifier:sTag HTMLNextOfTag<sTag>
      80             ]
      81         ]*;
      82 HTMLNextOfTag<"UL"> ::=
      83         [HTMLAttribute]* #continue '>' => {@\begin{itemize}
      84 @}
      85         [HTMLTag("LI")]*
      86         '<' '/' "UL" '>' => {@\end{itemize}
      87 @};
      88 HTMLNextOfTag<"LI"> ::=
      89         [HTMLAttribute]* #continue '>' => {@\item @}
      90         HTMLText
      91         '<' '/' "LI" '>' => {writeText(endl());};
      92 HTMLNextOfTag<"B"> ::=
      93         #continue '>' => {@\textbf{@}
      94         HTMLText
      95         '<' '/' "B" '>' => {@}@};
      96 HTMLNextOfTag<"I"> ::=
      97         #continue '>' => {@\textbf{@}
      98         HTMLText
      99         '<' '/' "I" '>' => {@}@};
      100 HTMLNextOfTag<"FONT"> ::= [HTMLAttribute]* #continue '>' HTMLText '<' '/' "FONT" '>';
      101 HTMLNextOfTag<"BR"> ::= ['/']? #continue '>' => { writeText(endl());};
      102 HTMLAttribute ::= #readIdentifier ['=' #continue [STRING_LITERAL | WORD_LITERAL]]?;
      103
      104
      105 blanks ::= [' '| '\t' | '\r' | '\n']*;
      106 STRING_LITERAL ::= #!ignore '\"' [~'\"']* '\"';
      107 WORD_LITERAL ::= #!ignore [~['>' | '/' | ' ' | '\t']]+;

line 6: blank characters are interesting, so we refuse to ignore HTML blanks and comments,
line 8: handling of HTML escape sequences, announced by character '&',
line 10: if not the beginning of a tag, the current character of the input stream is put to the output stream,
line 12: token operator '!' doesn't move the position of the input stream, and it continues in sequence only if the token expression that follows doesn't match; here, we check whether we have reached an end of tag or not,
line 14: we do not ignore comments anymore, so we have to do it my ourselves,
line 16: an embedded tag has been encountered,
line 19: template clauses HTMLEscape<T> are always valid and just convert special characters to their LaTeX representation,
line 22: in the real life, HTML tag <H1> could represent a chapter, but the LaTeX output file is intended to be included into the reference manual of CodeWorker as an illustration ; it will be a part of a section, so chapters are translated as sub sections!
line 26: in the real life, HTML tag <H2> could represent a section, but for the same reason as above, it will be translated as a sub-sub section,
line 34: with HTML, the number of columns the table expects is deduced later. However, a latex table (well-formed for a PDF conversion) must know explicetly of how many columns it is composed. So, a floating position is attached to the current position of the output file. While discovering columns, text will be inserted here and further.
line 36: the format of each column is specified at this place,
line 39: we consider that the first line of the table gives the name of the columns, and we'll take the PDF table suffix ('ii' for 2 columns, 'iii' for 3 columns, ...) to write lines of the table correctly,
line 41: we translate as many lines of the table as we can read, knowing the PDF suffix,
line 52: the clause is intended to read the name of a column of a table, and to translate it to LaTeX, knowing that some text must be inserted into the declarative part of the LaTeX table,
line 67: the text into a cell of a table shouldn't contain paragraph jumps (empty line in LaTeX),
line 71: the simplest way to avoid empty lines is to ignore end of lines, and to replace it to a space,
Last step, we have to change the leader script, so as to take into account the translation of the HTML documentation to the LaTeX one:

CodeWorker command line to execute:
-I Scripts/Tutorial -path . -define DESIGN_FILE=GettingStarted/SolarSystem0.sml -script GettingStarted/LeaderScript4.cws

      // file "GettingStarted/LeaderScript4.cws":
      1 if !getProperty("DESIGN_FILE")
      2     error("'-define DESIGN_FILE=file' expected on the command line");
      3 traceLine("'Simple Modeling' design file to parse = \""
      4           + getProperty("DESIGN_FILE") + "\"");
      5 parseAsBNF("GettingStarted/SimpleML-parsing.cwp",
      6            project, getProperty("DESIGN_FILE"));
      7 #include "TreeDecoration.cws"
      8
      9 #include "SharedFunctions.cws"
      10 foreach myClass in project.listOfClasses {
      11     traceLine("generating class '" + myClass.name + "' ...");
      12     generate("GettingStarted/CppObjectHeader.cwt", myClass, getWorkingPath() + "Scripts/Tutorial/GettingStarted/Cpp/" + myClass.name + ".h");
      13     generate("GettingStarted/CppObjectBody.cwt", myClass, getWorkingPath() + "Scripts/Tutorial/GettingStarted/Cpp/" + myClass.name + ".cpp");
      14     generate("GettingStarted/JAVAObject.cwt", myClass, getWorkingPath() + "Scripts/Tutorial/GettingStarted/JAVA/solarsystem/" + myClass.name + ".java");
      15 }
      16
      17 local myDocumentationContext;
      18 insert myDocumentationContext.language = "C++";
      19 traceLine("generating the HTML documentation...");
      20 setCommentBegin("<!--");
      21 setCommentEnd("-->");
      22 expand("GettingStarted/HTMLDocumentation.cwt",
      23         myDocumentationContext, getWorkingPath()
      24         + "Scripts/Tutorial/GettingStarted/SolarSystem1.html");
      25 translate("GettingStarted/HTML2LaTeX.cwp", project, "GettingStarted/SolarSystem1.html", getWorkingPath() + "Scripts/Tutorial/GettingStarted/SolarSystem.tex");

line 22: the procedure expand() will allow populating "SolarSystem1.html" with the characteristics of the project,
line 25: a context of execution (project here) is given as a this variable, although no parsing will be processed: reading and writing only, no data to keep,

Output:

'Simple Modeling' design file to parse = "GettingStarted/SolarSystem0.sml"
file parsed successfully
generating class 'Planet' ...
generating class 'Earth' ...
generating class 'SolarSystem' ...
generating the HTML documentation...

It generates the LaTeX file that composes the next sub section:

6.1 Design of a solar system

We dispose of some classes both in C++ and JAVA that allow building applications working on notions of planets, stars and solar systems.

6.1.1 Planet

This class represents the characteristics of a planet.

TypeAttribute nameDescription
double diameter the average diameter of the planet

6.1.2 Earth

This class represents our planet, for instantiating our particular solar system for instance, and working on geopolitical data perhaps!

TypeAttribute nameDescription
std::vector<std::string> countryNames the name of all countries are put into

6.1.3 SolarSystem

This class represents the solar system, with its constituents, the sun excluded for the moment.

TypeAttribute nameDescription
std::vector<Planet*> planets the planets that compose the solar system.

7 The debugger

The -debug option passed to the command line allows running the interpreter in debug mode. See chapter the integrated debugger for more information about its functionalities. We'll apply it on our precedent leader script:

CodeWorker command line to execute:
-I Scripts/Tutorial -path . -define DESIGN_FILE=GettingStarted/SolarSystem0.sml -script GettingStarted/LeaderScript5.cws -stdin GettingStarted/Debugger.cmd -debug

      // file "GettingStarted/LeaderScript5.cws":
      if !getProperty("DESIGN_FILE")
          error("'-define DESIGN_FILE=file' expected on the command line");
      traceLine("'Simple Modeling' design file to parse = \""
                + getProperty("DESIGN_FILE") + "\"");
      parseAsBNF("GettingStarted/SimpleML-parsing.cwp",
                 project, getProperty("DESIGN_FILE"));
      #include "TreeDecoration.cws"
     
      #include "SharedFunctions.cws"
      foreach myClass in project.listOfClasses {
          traceLine("generating class '" + myClass.name + "' ...");
          generate("GettingStarted/CppObjectHeader.cwt", myClass, getWorkingPath() + "Scripts/Tutorial/GettingStarted/Cpp/" + myClass.name + ".h");
          generate("GettingStarted/CppObjectBody.cwt", myClass, getWorkingPath() + "Scripts/Tutorial/GettingStarted/Cpp/" + myClass.name + ".cpp");
          generate("GettingStarted/JAVAObject.cwt", myClass, getWorkingPath() + "Scripts/Tutorial/GettingStarted/JAVA/solarsystem/" + myClass.name + ".java");
      }
     
      local myDocumentationContext;
      insert myDocumentationContext.language = "C++";
      traceLine("generating the HTML documentation...");
      setCommentBegin("<!--");
      setCommentEnd("-->");
      expand("GettingStarted/HTMLDocumentation.cwt",
              myDocumentationContext, getWorkingPath()
              + "Scripts/Tutorial/GettingStarted/SolarSystem1.html");
      translate("GettingStarted/HTML2LaTeX.cwp", project, "GettingStarted/SolarSystem1.html", getWorkingPath() + "Scripts/Tutorial/GettingStarted/SolarSystem.tex");

Output:

"LeaderScript5.cws" at 5: if !getProperty("DESIGN_FILE")
// The controlling sequence stops on the first statement of the leader script.
// We go the next instruction:
n
"LeaderScript5.cws" at 7: traceLine("'Simple Modeling' design file to parse = \""
// twice more:
n2
'Simple Modeling' design file to parse = "GettingStarted/SolarSystem0.sml"
"LeaderScript5.cws" at 11: parseAsBNF("GettingStarted/SimpleML-parsing.cwp",
//let plunge into the BNF-driven script:
s
parsed file is "c:/Projects/generator/Scripts/Tutorial/GettingStarted/SolarSystem0.sml":1,1
"SimpleML-parsing.cwp" at 6: world ::= #ignore(C++) [class_declaration]* #empty
//We are pointing to the beginning of the rule. Let execute '#ignore(C++)':
s
parsed file is "c:/Projects/generator/Scripts/Tutorial/GettingStarted/SolarSystem0.sml":1,1
"SimpleML-parsing.cwp" at 6: world ::= #ignore(C++) [class_declaration]* #empty
//Let go to the unbounded expression '[class_declaration]*':
s
parsed file is "c:/Projects/generator/Scripts/Tutorial/GettingStarted/SolarSystem0.sml":1,1
"SimpleML-parsing.cwp" at 6: world ::= #ignore(C++) [class_declaration]* #empty
//Now, we have a look to 'class_declaration':
s
parsed file is "c:/Projects/generator/Scripts/Tutorial/GettingStarted/SolarSystem0.sml":2,1
"SimpleML-parsing.cwp" at 16: class_declaration ::= IDENT:"class" #continue
//We visit 'INDENT:"class"' and we step over immediatly. Into a BNF-driven script, tokens of a
//sequence are iterated step by step, and 'next' runs all the sequence in one shot:
s
parsed file is "c:/Projects/generator/Scripts/Tutorial/GettingStarted/SolarSystem0.sml":2,1
"SimpleML-parsing.cwp" at 112: IDENT ::= #!ignore ['a'..'z'|'A'..'Z'|'_']
n
parsed file is "c:/Projects/generator/Scripts/Tutorial/GettingStarted/SolarSystem0.sml":2,6
"SimpleML-parsing.cwp" at 21: IDENT:sClassName
//We visit 'INDENT:sClassName' and we step over immediatly:
s
parsed file is "c:/Projects/generator/Scripts/Tutorial/GettingStarted/SolarSystem0.sml":2,7
"SimpleML-parsing.cwp" at 112: IDENT ::= #!ignore ['a'..'z'|'A'..'Z'|'_']
n
parsed file is "c:/Projects/generator/Scripts/Tutorial/GettingStarted/SolarSystem0.sml":2,13
"SimpleML-parsing.cwp" at 25: => insert project.listOfClasses[sClassName].name = sClassName;
//What about all local variables available on the stack?
l
sClassName
//What is the value of 'sClassName'?
t sClassName
Planet
//Now, we are looking at a classical statement of the language, an 'insert' assignment. But
//it might be more convenient to see more source code:
d 4
parsed file is "c:/Projects/generator/Scripts/Tutorial/GettingStarted/SolarSystem0.sml":2,13
21: IDENT:sClassName
22: //note: about parsing, classes are modeled into node
23: //note: \textbf{project.listOfClasses[}\textit{sClassName}\textbf{]}. Its attribute
24: //note: \samp{name} contains the value of \textit{sClassName}.
25: => insert project.listOfClasses[sClassName].name = sClassName;
26: //note: if the class inherits from a parent, \samp{\textbf{':'}} is necessary followed by
27: //note: an identifier (pattern \samp{\#continue}), and the identifier that matches with
28: //note: clause call \textit{IDENT} is assigned to the local variable \samp{sClassName},
29: [':' #continue IDENT:sParentName
//What about the call stack?
stack
parsed file is "c:/Projects/generator/Scripts/Tutorial/GettingStarted/SolarSystem0.sml":2,13
"SimpleML-parsing.cwp" at 25: => insert project.listOfClasses[sClassName].name = sClassName;
parsed file is "c:/Projects/generator/Scripts/Tutorial/GettingStarted/SolarSystem0.sml":2,13
"SimpleML-parsing.cwp" at 6: world ::= #ignore(C++) [class_declaration]* #empty
parsed file is "c:/Projects/generator/Scripts/Tutorial/GettingStarted/SolarSystem0.sml":2,13
"LeaderScript5.cws" at 11: parseAsBNF("GettingStarted/SimpleML-parsing.cwp",
//Exiting the debug session:
q
file parsed successfully
generating class 'Planet' ...
generating class 'Earth' ...
generating class 'SolarSystem' ...
generating the HTML documentation...

8 Scripts coverage and time consuming

The -quantify option passed to the command line allows running the interpreter with the profiling mode. See chapter quantifying scripts for more information about its functionalities. We'll apply it on our precedent leader script:

CodeWorker command line to execute:
-I Scripts/Tutorial -path . -define DESIGN_FILE=GettingStarted/SolarSystem0.sml -script GettingStarted/LeaderScript6.cws -quantify Scripts/Tutorial/GettingStarted/quantify.html

      // file "GettingStarted/LeaderScript6.cws":
      if !getProperty("DESIGN_FILE")
          error("'-define DESIGN_FILE=file' expected on the command line");
      traceLine("'Simple Modeling' design file to parse = \""
                + getProperty("DESIGN_FILE") + "\"");
      parseAsBNF("GettingStarted/SimpleML-parsing.cwp",
                 project, getProperty("DESIGN_FILE"));
      #include "TreeDecoration.cws"
     
      #include "SharedFunctions.cws"
      foreach myClass in project.listOfClasses {
          traceLine("generating class '" + myClass.name + "' ...");
          generate("GettingStarted/CppObjectHeader.cwt", myClass, getWorkingPath() + "Scripts/Tutorial/GettingStarted/Cpp/" + myClass.name + ".h");
          generate("GettingStarted/CppObjectBody.cwt", myClass, getWorkingPath() + "Scripts/Tutorial/GettingStarted/Cpp/" + myClass.name + ".cpp");
          generate("GettingStarted/JAVAObject.cwt", myClass, getWorkingPath() + "Scripts/Tutorial/GettingStarted/JAVA/solarsystem/" + myClass.name + ".java");
      }
     
      local myDocumentationContext;
      insert myDocumentationContext.language = "C++";
      traceLine("generating the HTML documentation...");
      setCommentBegin("<!--");
      setCommentEnd("-->");
      expand("GettingStarted/HTMLDocumentation.cwt",
              myDocumentationContext, getWorkingPath()
              + "Scripts/Tutorial/GettingStarted/SolarSystem1.html");
      translate("GettingStarted/HTML2LaTeX.cwp", project, "GettingStarted/SolarSystem1.html", getWorkingPath() + "Scripts/Tutorial/GettingStarted/SolarSystem.tex");

Output:

'Simple Modeling' design file to parse = "GettingStarted/SolarSystem0.sml"
file parsed successfully
generating class 'Planet' ...
generating class 'Earth' ...
generating class 'SolarSystem' ...
generating the HTML documentation...

Profiling results:

-- quantify session --
quantify execution time = 427ms
User defined functions:
  populateHeaderDeclarations(...) file "c:/Projects/generator/Scripts/Tutorial/GettingStarted/CppObjectHeader.cwt" at 29: 7 occurences in 0ms
  getMethodID(...) file "c:/Projects/generator/Scripts/Tutorial/GettingStarted/SharedFunctions.cws" at 98: 3 occurences in 0ms
  getParameterType(...) file "c:/Projects/generator/Scripts/Tutorial/GettingStarted/SharedFunctions.cws" at 44: 3 occurences in 0ms
  getType(...) file "c:/Projects/generator/Scripts/Tutorial/GettingStarted/SharedFunctions.cws" at 31: 13 occurences in 0ms
  getVariableName(...) file "c:/Projects/generator/Scripts/Tutorial/GettingStarted/SharedFunctions.cws" at 76: 26 occurences in 1ms
  normalizeIdentifier(...) file "c:/Projects/generator/Scripts/Tutorial/GettingStarted/SharedFunctions.cws" at 5: 39 occurences in 0ms
Predefined functions:
  charAt(...): 39 occurrences
  composeHTMLLikeString(...): 7 occurrences
  endString(...): 9 occurrences
  endl(...): 19 occurrences
  executeStringQuiet(...): 1 occurrences
  existVariable(...): 11 occurrences
  findElement(...): 2 occurrences
  findFirstChar(...): 39 occurrences
  first(...): 12 occurrences
  getFloatingLocation(...): 23 occurrences
  getMarkupKey(...): 1 occurrences
  getProperty(...): 3 occurrences
  getWorkingPath(...): 11 occurrences
  isEmpty(...): 6 occurrences
  isNegative(...): 39 occurrences
  newFloatingLocation(...): 12 occurrences
  not(...): 72 occurrences
  startString(...): 39 occurrences
  subString(...): 39 occurrences
  toUpperString(...): 39 occurrences
Procedures:
  __RAW_TEXT_TO_WRITE(...): 498 occurrences
  clearVariable(...): 1 occurrences
  expand(...): 1 occurrences
  generate(...): 9 occurrences
  insertTextOnce(...): 24 occurrences
  parseAsBNF(...): 1 occurrences
  setCommentBegin(...): 1 occurrences
  setCommentEnd(...): 1 occurrences
  setProtectedArea(...): 19 occurrences
  traceLine(...): 5 occurrences
  translate(...): 1 occurrences
  writeText(...): 325 occurrences
Covered source code: 83%
-- end of quantify session --

When the -quantify option isn't followed by an HTML file name, the synthetic profiling results are reported to the console:

If a file name was specified, the HTML output file highlights all visited script, so as to show parts of the code that are executed a lot and those that are less executed. Each visited line is prefixed by the number of times the controlling sequence has run on it.

Some points to notice:

9 Translating interpreted scripts to C++ source code

Once the scripts are considered as stable, it might be interesting to convert the interpreter and all necessary scripts to an executable, for many reasons:

The executable is built starting from the corresponding C++ source codes of the script files. It exists two ways to ask for compiling the CodeWorker script files to C++:
Compiling the project to C++ will convert the leader script and all its dependencies (meaning that all scripts that may be required by the leader will be compiled to C++) and then two makefiles will be created (a DSP for Visual C++ and a classical makefile intended to LINUX). The project takes the name of the leader script.

To compile our Simple Modeling Language project to C++, we may choose to proceed as one of the following:

The directory called "Scripts/Tutorial/GettingStarted/bin" contains the C++ source files and the makefiles:

    Scripts/Tutorial/GettingStarted/bin/CGExternalHandling.h
    Scripts/Tutorial/GettingStarted/bin/CGRuntime.h
    Scripts/Tutorial/GettingStarted/bin/CppObjectBody_cwt.cpp
    Scripts/Tutorial/GettingStarted/bin/CppObjectBody_cwt.h
    Scripts/Tutorial/GettingStarted/bin/CppObjectHeader_cwt.cpp
    Scripts/Tutorial/GettingStarted/bin/CppObjectHeader_cwt.h
    Scripts/Tutorial/GettingStarted/bin/CppParsingTree.h
    Scripts/Tutorial/GettingStarted/bin/DynPackage.h
    Scripts/Tutorial/GettingStarted/bin/HTML2LaTeX_cwp.cpp
    Scripts/Tutorial/GettingStarted/bin/HTML2LaTeX_cwp.h
    Scripts/Tutorial/GettingStarted/bin/HTMLDocumentation_cwt.cpp
    Scripts/Tutorial/GettingStarted/bin/HTMLDocumentation_cwt.h
    Scripts/Tutorial/GettingStarted/bin/JAVAObject_cwt.cpp
    Scripts/Tutorial/GettingStarted/bin/JAVAObject_cwt.h
    Scripts/Tutorial/GettingStarted/bin/LeaderScript6.dsp
    Scripts/Tutorial/GettingStarted/bin/LeaderScript6_cws.cpp
    Scripts/Tutorial/GettingStarted/bin/LeaderScript6_cws.h
    Scripts/Tutorial/GettingStarted/bin/Makefile
    Scripts/Tutorial/GettingStarted/bin/SimpleML-parsing_cwp.cpp
    Scripts/Tutorial/GettingStarted/bin/SimpleML-parsing_cwp.h
    Scripts/Tutorial/GettingStarted/bin/UtlException.h

The main C++ source file is "LeaderScript6.cpp" and the executable will be called "LeaderScript6.exe".