Stefan Bodewig's Weblog

Java Client Library for QMQP

In a project at innoQ we're using QMQP to quickly queue mail to an MTA for delivery.

Even though - or maybe because - the protocol looks rather simple, we didn't find any open source library for this. We've decided to open source our own implementation QMQP Java, version 0.1 is available from Maven central (com.innoq.qmqp:qmqp-client:0.1) under the Apache License 2.0.

This initial release is strongly tailored to our project's needs. If you want to use it and find it lacking anything, don't hesitate and use a pull request or open an issue at github.

Apache Ant 1.8.4 and Commons Compress 1.4.1 Security Bugfix Releases

In case you have missed it, there is a flaw in the code that writes bzip2 archives in both Ant and Commons Compress. There are new releases for both of them, so go grab them: Ant, Commons Compress.

As part of the process of creating bzip2 compressed blocks the input data (usually in chunks of 900kb) is sorted (during the Burrows-Wheeler transformation, if you want to know). The only sorting algorithm present in the bzip2 classes prior to the security release is very efficient for the average case but shows extraordinarily bad performance for very repetitive inputs. For certain inputs the bzip2 task took two hours on my really fast work notebook (at 100% CPU for a single core) while it finishes in less than two seconds with Ant 1.8.4.

These inputs have to be specially crafted, it is very unlikely you will face them in the wild. The flaw turns into a security issue if you are providing a public service that compresses input created by arbitrary users - maybe a public build server or an archiving solution.

The bzip2 code in Ant (and all forks that stem from it, like Commons Compress) was derived from an early version of Julian Seward's libbzip2. Starting with 0.9.5 libbzip2 detects if sorting is taking too long because of bad inputs and switches to a different sorting strategy in such cases. The fix in the two releases now consists of porting this fallback sorting algorithm from C to Java.

While porting this I learned a lot. I read several academic papers in order to understand what was actually going on. It felt like I was back in University again and it felt good.

Many thanks to David Jorm of the Redhat Security Team who uncovered the issue.

ROCA - Resource-Oriented Client Architecture

Some of my colleagues at innoQ have put together a bunch of rules about what makes up a web application that actually uses the web rather than hides it.

There is more on Stefan Tilkov's blog and the ROCA website. Discussion (there, not here) is more than welcome.

Apache Ant 1.8.3 Released

yesterday we released Ant 1.8.3, go grab it from the download page. By pure coincidence it was released on a leap-day.

This release really mostly is a bug fix release, see the release notes for a complete list. There isnt anything major sticking out to me, but I know people have been bitten by some of the bugs - like forked Java processes hanging when they read from System.in - so for them the new release was important.

The dev team has decided to drop Java 1.4 support (as Ant's runtime) for trunk, so this may likely be the last release supporting Java 1.4. We have prepared a branch so we may be able to create more 1.8.x releases if a major bug raises its head. For trunk this means we'll be able to start using "modern" features like generics. It also means I can merge some improvements like Zip64 support from Commons Compress into Ant.

One of the fixes introduced a new class in order to better multiplex between System.out and System.err when forking a new process. This allows Ant 1.8.3 to be detected by either

  <antversion property="Ant-1.8.3-or-later"
              atleast="1.8.3"/>

  <available property="Ant-1.8.3-or-later"
             classname="org.apache.tools.ant.util.LineOrientedOutputStreamRedirector"/>

It's been the first time I acted as Ant's release manager since Ant 1.1 more than eleven years ago, quite a bit has changed WRT process but also automation since then. It wasn't as painful as I feared it to be, largely because we no longer ship optional tasks that require third party jars that cannot be downloaded freely.

Moved my Tiny OSS Libs to GitHub

I've been keeping a small Antlib for GWT and a DLL with NAnt and MSBuild tasks to run Ant in a local darcs repository. Now I've decided to not maintain them here anymore but rather move them to github, so they now are at https://github.com/bodewig/gwttasks and https://github.com/bodewig/Ant4NantAndMSBuild respectively.

For the migration of my - trivial - darcs repos I used https://github.com/purcell/darcs-to-git and it worked like a charm.