Bazsi's blog

The 3.2 git tree received yet another feature I still wanted to shove into the 3.2 final: template functions. A template function is a transformation: it is able to modify the way macros or name-value pairs are expanded.

For example, let’s assume that you have a log file where you want to write CSV formatted data, which you could do with the following template:

template d_csv { file(“/var/log/data.csv” template(“$DATE,$HOST,$PROGRAM,$PID,$MSGn”)); };

This works fine until you discover that the CSV format can be broken if one of the expanded values contain a ‘,’ which is normally used to separate values. The CSV format solves this by quoting the values or by escaping the offending characters, e.g. you’d have to write something like this:

template d_csv { file(“/var/log/data.csv” template(‘”$DATE”,”$HOST”,”$PROGRAM”,”$PID”,”$MSG”n’)); };

This would solve the comma problem, but introduces another: the quote character can also be present in the macros expanded above, which if happens to be the case will also spoil the nice CSV format. Lucky us, the authors of syslog-ng had one little hack for this problem for a long time now: template-escape:

template d_csv { file(“/var/log/data.csv” template(‘”$DATE”,”$HOST”,”$PROGRAM”,”$PID”,”$MSG”n’) template-escape(yes)); };

This causes syslog-ng to automatically escape the apostrophe, the quote character and the backspace whenever it is encountered in the expanded string. But please note that it is only escaped when escaping and not in the literal portion of the string. However what happens if you don’t like the CSV format or you have different escaping rules? Or perhaps you only want to increase the size of your output file when the quotes are absolutely necessary? You could perhaps use rewrite rules, but that has some drawbacks: it increases the size of the log message and it is inconvinient to use. Until now, those were to only options. With a template-function however, the solution is easy:

template d_welf { file(“/var/log/data.welf” template(“time=$(escape-welf $DATE) host=$(escape-welf $HOST) program=$(escape-welf $PROGRAM) pid=$(escape-welf $PID) msg=$(escape-welf $MSG)n”)); };

Of course it is possible to add new template functions via the new plugin mechanism. Right now it is more of a framework and a syntax embedded in templates, and only the “echo” function is available, but I expect growth in this area.

In case you are interested this patch in the git implements template functions and this one adds plugin support on top of it. Adding a new function is simple, just look at the file named builtin-tmpl-func.c in the source tree.