CodeWorker

A parsing tool & a source code generator


Since 19jan2003

Software+Documentation+Website Release: 3.2
Last update: november 24, 2003

Distributed under the terms of the GNU Lesser General Public License.

Presentation

CodeWorker is a parsing tool and a source code generator, available in Open Source (distributed under the GNU Lesser General Public License) devoted to cover many aspects of the generative programming. Generative programming is a software engineering approach for producing reusable, tailor-made, evolvable and reliable IT systems with a high level of automation.

The tool interprets a scripting language that drives the processing of parsing and source code generation in a quite familiar syntax for developers. It insists on adapting the syntax of the language to the particularities of tasks to resolve (parse, code generation, procedural) and on offering powerful functionalities to do intensive source code generation.

Please do not hesitate to contact us (questions, criticisms, suggestions, ...).

The parsing

CodeWorker provides two distinct methods for performing a parsing:
  • the extended-BNF notation is declarative, and conforms to a kind of BNF (the Backus-Naur Form represents a grammar in a particular syntax) extended with regular expressions, predefined non-terminals and useful directives. Something close to javacc in the JAVA world or to ANTLR.
  • the reading of tokens is procedural and a little obsolete now that BNF parsing scripts in CodeWorker have evolved in-depth.
During the parsing of files, CodeWorker feeds an appropriate data structure that is called a tree, a parse tree. A tree is a convenient structure to represent a hierarchical set of nodes, as in XML for instance.

The parse tree is shared both by the parse task, which takes in charge of populating the tree, and by the source code generation that will walk through it for generating text.

The source code generation

Given a specification provided in any kind of format, CodeWorker will generate source code or text as required in template-based scripts.

The source code generation can use three modes: generation, expansion or translation.

  • generation mode is used to let the script produce the most part of the output file, processing a kind of template-based generation as it exists for a JSP or PHP script. Only some areas called protected areas in the vocabulary of CodeWorker are preserved in the file. This philosophy has been adopted by some modeling tools that generate a skinny skeleton copiously interspersed with areas intended to the developer.
  • expansion mode is used when the file is mainly written by hand, but small portions need to be generated. The points where to insert the code are called markers in the vocabulary of CodeWorker. The Class Wizard of Visual C++ changes source code following this principle.
  • translation mode is used when both parsing and source code generation are required to process a file. It arrives for processing:
    • a source-to-source translation: a file must be rewritten in a different syntax. For example, a LaTeX file might have to be translated in HTML.
    • a program transformation: a source file has to change for optimizing, refactoring, instrumenting or rewriting some portions. For example, a script could add a trace at the beginning of each function body of a JAVA or C++ source code. To do that, parsing will serve to discover function bodies, and source code generation will insert the C++ or JAVA code that implements the trace.

Parse-to-Generate: a straightforward process

Tasks for parsing specifications and generating code are executed in a straightforward process presented in the figure below. As CodeWorker proposes a scripting language with expressiveness enough, there is no need of a gluing language to join parse and generation tasks.

The figure describes the classical approach used in a leader script interpreted by CodeWorker. It shows the script, which calls first a BNF-parse script, before continuing in sequence and calling a template-based script.

It exists some variants, for example when CodeWorker runs as a CGI program.

 CodeWorker is maintained by Cedric Lemaire. Please send a mail to Submit a bug or feature