Bazsi's blog

I’ve achieved an important milestone on the current threading stuff and I’m happy to tell you that multi-processing and epoll related performance improvements work is progressing nicely. The current master branch of the syslog-ng-3.3 tree runs the testsuite (make check) and performs much better than earlier releases.

The only performance data was measured on my laptop, there it grew from about 60k to 180k msg/sec. While doing fixes and adding locking here and there, it went down to 160k and I didn’t investigate why that happened. But anyway, 160k msg/sec is not really bad either, from a single client and my guess is that adding more clients (and CPUs) to the picture will scale syslog-ng to several hundred thousand messages per second range.

I still have some locking job to do, I’ve just found problems with the udp() destination driver, so it is currently quite fragile, but I’d appreciate any kind of feedback you could have by installing it on your test systems. Production is of course out of the question.

Until now, the work was available in the “wip/epoll” branch, which I rebased it regularly, so that fixes of problems that I’ve found were incorporated into the original “threading” patchset. However that patch grew quite large by now and I now feel it’d be easier to track changes as individual patches instead of folding them back into the original series. Therefore I merged it back to “master”, and from now on the wip/epoll branch will be removed, further fixes will be published on the “master” branch.

In order to compile this stuff you’ll need one dependency library: ivykis. Ivykis is written by Lennert Buytenhek and encapsulates an epoll based event loop. It also supports other systems like FreeBSD’s kqueue, Solaris’s /dev/poll and of course the traditional select/poll system calls. I needed a couple of modifications against ivykis, those are hosted on git.balabit.hu, more specifically at: git://git.balabit.hu/bazsi/ivykis.git. I’m working with Lennert to incorporate my changes, so that hopefully no changes to upstream ivykis will be necessary.

In the coming days, I’m trying to fix up things that broke, and then quite possibly do a 3.3alpha1 release once I feel that it is getting stable enough for anyone to try.

Stay tuned!