ABOUT
multi-threading coming in syslog-ng OSE 3.3
As posted on the mailing list already, I’m planning to turn syslog-ng into a fully-multi-threaded application in order to improve performance on multi-core systems. Since I don’t want to start destaibilizing 3.2 (rather to push dub that as stable soon), this will become part of OSE 3.3. During my holidays I’ve worked a little on [...]
[.] Továbbpdbtool test improvements
I have added some more functionality to “pdbtool test” which I needed while working on the official syslog-ng patterndb patterns. It now can process several pdb files in a single invocation and also it is now able to validate the patterndb XML files against the official schema. This is the shell command I’ve used: $ [...]
[.] Továbbsyslog-ng & open core, why we think it is different
As you may have seen in my last post, syslog-ng was quoted as an example of “open core” development model, which has problems in the eyes of some Open Source purists. I was trying to do my homework and understand their point of view and see what we can do about it. If you are [...]
[.] TovábbLWN: syslog-ng rotten to the (Open) Core?
This was first posted as a comment under an article on lwn.net, but I thought it was important enough to post it here for others not reading lwn. Please go ahead and read the original article which is about the “Open Core” business model and its problems from the Free Software community point of view. [...]
[.] Továbbsyslog-ng 3.2alpha2 released
I’ve just uploaded syslog-ng 3.2alpha2 to the release directory. The last alpha release didn’t compile on all supported platforms and the automatic test-suite was disabled, because it only worked if syslog-ng got installed first. These obstacles have been overcome and together with some fixes and a couple of new features, 3.2alpha2 is now available. I’ve [...]
[.] Továbbsyslog-ng name-value pair naming
I was giving a lot of thought recently to the topic of naming name-value pairs in syslog-ng. Until now the only documented rule is stating somewhat vaguely that whenever you use a parser you should choose a name that has at least one dot in it, and this dot must not be the initial character. [...]
[.] Továbbsyslog-ng & distributions
syslog-ng 1.6.x and 2.0.x versions had lived quite long. A lot of distributions used these versions and never upgraded to the newer ones. This has changed recently, Peter Czanik was busy to help maintainers get to the latest versions. Already available in the latest release: openSUSE FreeBSD ports Mandriva Gentoo portage OpenBSD ports In development [...]
[.] Továbbsyslog-ng and process accounting
In one of my previous posts, I’ve mentioned that syslog-ng is not for syslog anymore, we aim to support other log formats too, preferably those that have some kind of structure. In fact syslog-ng is trying to convert all incoming messages (be them unstructured syslog messages, process accounting messages or SNMP traps) into the same, [...]
[.] Továbbpatterndb status update
I thought I’d post a quick update on the patterndb project status. Our first aim was to draft a basic policy which governs how patterns should be created. This is available in the patterndb git repository as a README.txt file. Although not completely finished, I feel the current description is enough for some basic work [...]
[.] Továbbpatterndb: grep on steroids
You may have heard of my last project to collect log samples from various applications, in order to convert log data from free-form human readable strings into structured information. The first round to collect login/logout messages from sshd is now complete. You could ask: ok, but what is the immediate benefit? You supposedly have a [...]
[.] Továbbsyslog-ng OSE 3.2 caveats
Starting with syslog-ng OSE 3.2, syslog-ng became plugin based, which has some consequences that even experienced syslog-ng users may stumble into. The most obvious one, is that syslog-ng now produces a series of .so files loaded at runtime, instead of being a monolithic executable. If a given .so is not not or not loaded, some [...]
[.] Továbbsyslog-ng contributions redefined
syslog-ng has been around for about 12 years now, but I think the biggest change in the project’s life is imminent: with the upcoming release of syslog-ng OSE 3.2, syslog-ng will become an independent entity. Until now, syslog-ng was primarily maintained & developed by BalaBit, copyrights needed to be reassigned in order to grant BalaBit [...]
[.] Továbbpatterndb project
By now probably most of you know about patterndb, a powerful framework in syslog-ng that lets you extract structured information from log messages and perform classification at a high speed: http://www.balabit.com/dl/html/syslog-ng-ose-v3.1-guide-admin-en.html/concepts_pattern_databases.html Until now, syslog-ng offered the feature, but no release-quality patterns were produced by the syslog-ng developers. Some samples based on the logcheck database were [...]
[.] Továbbsmall incompatible change for 3.1
I’ve just commited a small incompatible change for syslog-ng 3.1, even though theoreticaly I shouldn’t have. The change is not big, simply the ‘store-legacy-msghdr’ flag became default for all sources, whereas earlier you had to specify that explicitly. In order to understand why I did that, a short description of the flag follows below. syslog-ng [...]
[.] Továbbsyslog-ng 3.2 changes
I’ve just pushed a round of updates to the syslog-ng 3.2 repository, featuring some interesting stuff, such as: SQL reorganization: Patrick Hemmer sent in a patch to implement explicit transaction support instead of the previous auto-commit mode used by syslog-ng. I threw in some fixes and refactored the code somewhat. Configuration parser changes: the syntax [...]
[.] TovábbExplicit transaction support in SQL
The SQL destination in syslog-ng so far assumed that databases automatically start a new transaction for each INSERT statement that syslog-ng issues. This works fine, however there is a significant overhead of starting new transactions, with sqlite I’ve measured about 20 times performance increase on my development notebook and my debug build. With explicit-commits: bazsi@bzorp:~/.zwa/install/syslog-ng-ose-3.2$ [...]
[.] Továbbsyslog-ng 3.2 opened, experimental “blocks” branch opened
After last the stable syslog-ng 3.1.0 release last week, I’ve opened the 3.2 branch to receive the new stuff. The first bits are already in the repository: the basic plugin framework and the conversion of the socket related stuff (tcp, udp, unix-dgram, unix-stream, syslog drivers) into a separate plugin. The reason of the afsocket plugin [...]
[.] Továbbsyslog-ng 3.1 final release
I’m proud to announce that both the Open Source and the Premium editions of syslog-ng 3.1 was published and are available on our website. This is an important milestone in multiple ways: the new feature/stable release schema is making its debut the patterndb got significant improvements: new parsers, pdbtool, tagging support the ability to change/add [...]
[.] Továbbplugins branch updated
Since the last post, I could hack a couple of hours on the plugins branch, which now compiles. The plugin framework is capable for supporting a quite important core functionality: all socket like sources/destinations are now found in an external plugin called “afsocket”. The reason I’ve started with afsocket is to make syslog-ng a bit [...]
[.] Tovább