Bazsi's blog

I wanted to let you know that a new alpha version for the upcoming syslog-ng version has been published on our website. It is by all means an important step forward, because the new release brings us:

• It uses multiple threads to do its work, thus capable of scaling to multiple CPUs. Add ‘threaded(yes)’ to your option block and you’re good to go.
• Built in mongodb destination for folks who prefer NoSQL over traditional SQL.
• This version has a new and improved “pdbtool patternize” functionality

You can find the latest release here:

http://www.balabit.com/downloads/files/syslog-ng/open-source-edition/3.3.0alpha2

Alternatively, you can also check the git repository at:

git://git.balabit.hu/bazsi/syslog-ng-3.3.git (browse online)

Known issues

I know for certain that it will not work on FreeBSD unless the following environment variable is set:

IV_EXCLUDE_POLL_METHOD=kqueue

Otherwise syslog-ng will crash at startup, immediately.

Stability

Certainly you wouldn’t want to run your production system on this version, however this code base had endured a couple of test rounds, so if it compiles and starts up for you (which can be a challenge I can admit), then the basic functions will most certainly work.

Features:

• Support for systemd activation added.
• Add support for customizable token delimiters in “pdbtool patternize”, which makes the resulting patterns to have much better quality.
• Added support for a –no-parse command line option for “pdbtool patternize” to avoid parsing the input as normal syslog messages.
• Added a new flag ‘ignore-errors’ to LogWriter based destinations (file, pipe, tcp, udp, syslog)
• Added support for specifying the suppress() option globally.
• Added a new macro named $CONTEXT_ID that expands to the current context-id in correllated rules.

Bugfixes

• Fixed a tcp()/udp()/syslog() destination driver issue that caused aborts during startup.
• Handle non-epollable devices like /dev/null normally by failling back to the regular file-like handling. Earlier these caused an abort inside ivykis.
• Makefile fixes to make it possible to compile syslog-ng from a “make dist” tarball.
• Added error messages about libdbi initialization failures, that would prevent the SQL destination from working.
• Make it possible to compile against PCRE not in a standard location (caused compilation failures on FreeBSD).
• Fixed several $(grep) related bugs:
  • not to cause a segmentation faule when the filter expression supplied contains syntax error
  • the template parsing code removed quotes required by filter expressions, with the current change it is possible to use quotes in the filter expressions directly:
    $(grep (‘$username’ == ‘root’) $MSG)
  • fixed the handling of template function invocations within correllation that had multiple messages in its context.
• Fixed a segmentation fault in pdbtool match –debug-pattern in case there was no matching rule.
• Fixed “pdbtool test”.

Other changes

• Changed the default syslog-ng.conf version number to 3.3 to match the current version.
• The pipe() destination used to override the default value of flush_lines() by explicitly setting it to 0. This behaviour has been removed, now the pipe() driver will properly use the default, unless overridden explicitly.
• Ported the 3.2 linking changes to 3.3, which means that:
  • libsyslog-ng-patterndb.so is gone, both the command line pdbtool command and the patterndb plugin (libpatterndb.so) links its contents statically, to improve portability on Cygwin.
  • unit tests & command line utilities link properly in –enable-mixed-linking mode
• Added debug messages in the correllation code to make it easier to debug correllation rules.
• Added debug/verbose options to pdbtool.

Credits

Code, bugreports, testing, documentation suggestions and other improvements were contributed by:

• Arkadiusz Miśkiewicz (PLD Linux)
• Balazs Scheidler (BalaBit)
• Balint Kovacs (BalaBit)
• Corinna Vinschen (RedHat)
• Dalibor Toman (fortech.cz)
• Gergely Nagy (BalaBit)
• Laszlo Boszormenyi (lsc.hu)
• Marius Tomaschewski (Novell)
• Peter Czanik (BalaBit)
• Peter Gyongyosi (BalaBit)
• Tom Gundersen  (jklm.no)
• Valentijn Sessink  (blub.net)
• Zoltan Pallagi (BalaBit)