Bazsi's blog

The other day someone wanted a special syslog-ng macro that would expand into digit changing every 5 seconds (e.g. R_UNIXTIME % 5) and although I couldn’t give an exact solution to his problem, I’ve came up with this configuration snippet:

rewrite p_date_to_values {
set(“$R_DATE”, value(“rdate”));
};

filter f_get_second_chunk {
match(‘^… .. [0-9]+:[0-9]+:(?[0-9])[0-9]$’
type(pcre) value(‘rdate’));
};

The way it works is as follows:

• the rewrite statement sets the name-value pair named “rdate” to $R_DATE (the macro)
• the filter statement uses Perl Compatible Regular Expressions to parse the value of the “rdate” value and uses a named subpattern on the tens of seconds position to store that character in a value named “rdate.second_tens”
• Later on in the configuration you can use “rdate.second_tens” just like any other macro/value.

This proves that the current rewrite/parser/filter subsystems are really powerful, however even though this proved to be possible, there are some lessons learned from this example:

• the macro and name-value space should really converge to each, this would mean that the match() filter could directly match against the macro value $R_DATE without the need for the separate rewrite statement
• when you are after a given goal, you don’t really want to differentiate rewrite/parser/filter rules at all. The current syntax of using separate blocks for separate type of log processing elements is a pain.

So I’m thinking about inventing yet another block, which simply wouldn’t care what kind of processing element is added to it, something along the lines:

pipeline rdateseconds {
set(“$R_DATE”, value(“rdate”));
match(‘^… .. [0-9]+:[0-9]+:(?[0-9])[0-9]$’
type(pcre) value(‘rdate’));

};

And then:

log {
source(src);
pipeline(rdateseconds);
destination(dst);
};

Maybe I should even allow the creation of rewrite/parser/filter elements right there in the log statement:

log {
source(src);
filter(facility(mail));
destination(dst);
};

What do you think?