TextFilt

SourceForge.net Logo    Valid HTML 4.01!

News

Contents

Rationale

Have you ever seen error-messages from your compiler that look like this:

   example.cc:21: conversion from `double' to non-scalar type `map<vector<basic_string<char,string_char_traits<char>,
   __default_alloc_template<true,0> >,allocator<basic_string<char,string_char_traits<char>,__default_alloc_template<t
   rue,0> > > >,set<basic_string<char,string_char_traits<char>,__default_alloc_template<true,0> >,less<basic_string<c
   har,string_char_traits<char>,__default_alloc_template<true,0> > >,allocator<basic_string<char,string_char_traits<c
   har>,__default_alloc_template<true,0> > > >,less<vector<basic_string<char,string_char_traits<char>,__default_alloc
   _template<true,0> >,allocator<basic_string<char,string_char_traits<char>,__default_alloc_template<true,0> > > > >,
   allocator<set<basic_string<char,string_char_traits<char>,__default_alloc_template<true,0> >,less<basic_string<char
   ,string_char_traits<char>,__default_alloc_template<true,0> > >,allocator<basic_string<char,string_char_traits<char
   >,__default_alloc_template<true,0> > > > > >' requested

Have you ever dreamed of a simple tool that translates these messages to their basic form like this:

   example.cc:21: conversion from `double' to non-scalar type `map<vector<string>,set<string>>' requested

TextFilt's root is the need for simplified STL-error-messages. During its development, it became a general purpose text processing tool.

Concept

The basic idea of TextFilt was to use regular expressions to simplify the output of the compiler. Originally being a perl script, the concept changed slightly after a small discussion with Dave Abrahams towards a C++-version (Thanks, Dave!). The regular expression used are John Maddock's Regex++, found in the Boost-libraries.

The tool was designed to work on streams to support on-the-fly processing of compiler output. TextFilt therefore works by reading the standard input stream, processes it through one or more rulesets and write the output to the standard output stream. The rulesets are simple text files that can be modified without rebuilding TextFilt itself. This also separates the technical part of processing streams through rulesets from the actual rules. This allows new rules for different compilers, STLs, etc. to be added easily and also allows you to use TextFilt for various other tasks.

Requirements

You need to install the Boost-library. Please refer to their documentation for building and installing the library.

Installation

Download and compile TextFilt.cc . Remember to give it the include directory for Boost and link against Boost's regular expression library. Copy it to some directory which is in your $PATH. Here's how I compile it:

   g++ -I$WORK/include -ansi -pedantic -Wall -O2 -c TextFilt.cc -o TextFilt.o
   g++ -L$WORK/lib -lboost_regex TextFilt.o -o TextFilt
   strip TextFilt

Where $WORK is the path where I have the boost libraries installed. The strip reduces the size of the executable by 50%.

There is also a precompiled binary available, linked statically with the boost regex library. It will not work for all systems, but you may want to try it anyway. Here it is: TextFilt.

For the most convenience, here is everything in one package: TextFilt.tgz.

Usage

TextFilt is platform independend and very simple. All command line parameters are interpretated as filenames to specify the rulesets to be used. After parsing the rulesets, the standard input is parsed line-by-line. For every line, TextFilt tries to apply all rules until the line becomes stable, e.g. a run through all rules does not change the line any more.

Rulesets

All rulesets are parsed line-by-line in the order you supplied them as command-line-arguments. Each line is either

All other lines are considered an error and will be ignored

Comments

A comment starts with // and terminates with the next new-line character.

Variables

A variable definition looks like this:

   $identifier = "value";

Where identifier must match [a-zA-Z_][a-zA-Z0-9_]* and value can be anything you like. Within value you can use other variables, which will be replaced immediately, by inserting ${var}.

Rules

A rule is build by a regex to match and a replace-pattern. It has the form

   "regex-search-pattern" = "replacement";

Variables are expanded like for value above for both sides of a rule.

Please refer to Boost's RegEx++ documentation for more information on the regular expressions and have a look at the rulesets below for some examples.

Example rulesets

Here are some example rulesets.

For the GCC 2.95.x, use base.rs, gcc_2_95_x.rs and stl.rs. For the GCC 3.x or the Intel 6.0, use base.rs and stl.rs, don't forget to supply -fmessage-length=0 to the GCC 3.x command line to prevent spreading error messages through multiple lines. I haven't tested other compilers, but if you have problems with your compiler, feel free to mail me.

If you are using STLport, you can use the rulesets base.rs, stlport.rs and stl.rs. You don't need gcc_2_95_x.rs in this case.

Tips for using TextFilt

Example compiler invocation

When your previous invocation of the compiler looked like this:

   g++ -O2 example.cc -o example

than you should modify your Makefile to call the compiler this way:

   g++ -O2 example.cc -o example 2>&1 | TextFilt rulesets/base.rs rulesets/gcc_2_95_x.rs rulesets/stl.rs && [ ${PIPESTATUS[0]} -eq 0 ]

That way, make receives the correct error code from the compiler.

Profiler-output

Some profilers produce files which contain function signatures which can be run through the STL-rulesets to shorten them.

CVS

CVS-output can be improved by cutting away unwanted clutter and highlighting and colorizing important parts. The rules can easily be adapted to match your personal needs. Try using aliases or shell-functions to ease the invocation of TextFilt.

Compiler warnings for foreign libraries

When you need to use a foreign library like xalan/xerces or RogueWave that produces a lot of warnings when compiled with -ansi -pedantic, add rulesets to remove these warnings from your compiler's output.

Typedefs

When your project uses general typedefs, you might want to see your typedef'd name instead of the real type. As an example, we use RWDecimal< RWMultiPrecisionInt< 3 > > though our typedef Base::decimal and it's far easier to read error messages which match the name you have written.

Feedback

Any feedback can be send to Daniel Frey. As you can see easily from this page, I am a software-developer - not a web-designer. If you enjoy web-design and writing documentation and if you want to offer your help, I'd be happy to hear from you :)

Future

STLFilt

If you came here, you may also be interested in Leor Zolman's STLFilt. It is similar to TextFilt in it's roots, but has a longer history and more features wrt STL error decryption. Both TextFilt and STLFilt have their strengths, it depends on your needs which tool suits better.


Last modified: Thu Sep 05 14:25:50 CEST 2002