In case you have missed it, there is a flaw in the code that writes bzip2 archives in both Ant and Commons Compress. There are new releases for both of them, so go grab them: Ant, Commons Compress.

As part of the process of creating bzip2 compressed blocks the input data (usually in chunks of 900kb) is sorted (during the Burrows-Wheeler transformation, if you want to know). The only sorting algorithm present in the bzip2 classes prior to the security release is very efficient for the average case but shows extraordinarily bad performance for very repetitive inputs. For certain inputs the bzip2 task took two hours on my really fast work notebook (at 100% CPU for a single core) while it finishes in less than two seconds with Ant 1.8.4.

These inputs have to be specially crafted, it is very unlikely you will face them in the wild. The flaw turns into a security issue if you are providing a public service that compresses input created by arbitrary users - maybe a public build server or an archiving solution.

The bzip2 code in Ant (and all forks that stem from it, like Commons Compress) was derived from an early version of Julian Seward's libbzip2. Starting with 0.9.5 libbzip2 detects if sorting is taking too long because of bad inputs and switches to a different sorting strategy in such cases. The fix in the two releases now consists of porting this fallback sorting algorithm from C to Java.

While porting this I learned a lot. I read several academic papers in order to understand what was actually going on. It felt like I was back in University again and it felt good.

Many thanks to David Jorm of the Redhat Security Team who uncovered the issue.

path: /en/Apache/Ant | # | Writebacks

yesterday we released Ant 1.8.3, go grab it from the download page. By pure coincidence it was released on a leap-day.

This release really mostly is a bug fix release, see the release notes for a complete list. There isnt anything major sticking out to me, but I know people have been bitten by some of the bugs - like forked Java processes hanging when they read from System.in - so for them the new release was important.

The dev team has decided to drop Java 1.4 support (as Ant's runtime) for trunk, so this may likely be the last release supporting Java 1.4. We have prepared a branch so we may be able to create more 1.8.x releases if a major bug raises its head. For trunk this means we'll be able to start using "modern" features like generics. It also means I can merge some improvements like Zip64 support from Commons Compress into Ant.

One of the fixes introduced a new class in order to better multiplex between System.out and System.err when forking a new process. This allows Ant 1.8.3 to be detected by either

  <antversion property="Ant-1.8.3-or-later"
              atleast="1.8.3"/>

or

  <available property="Ant-1.8.3-or-later"
             classname="org.apache.tools.ant.util.LineOrientedOutputStreamRedirector"/>

It's been the first time I acted as Ant's release manager since Ant 1.1 more than eleven years ago, quite a bit has changed WRT process but also automation since then. It wasn't as painful as I feared it to be, largely because we no longer ship optional tasks that require third party jars that cannot be downloaded freely.

path: /en/Apache/Ant | # | Writebacks

A few days after Apache Commons Compress 1.3 has been released the Compress Antlib has seen a new release as well. This gives Ant support for Zip64 extensions, the Unix dump format (read-only) and the Pack200 format.

Prior to this release Pack200 support has already been available via a a task at java.net but the Compress Antlib also adds a pack200resource as well as a pack200normalize task that can be used to "normalize" a JAR so that it can be signed, packed and unpacked with the signature remaining valid.

path: /en/Apache/Ant | # | Writebacks

Ant releases have become more frequent than my blog posts by now, oh my. I won't promise anything.

While most of the world was on vacation Antoine (again) put together a new relase of Apache Ant and announced 1.8.2 late December. As usual a full list of fixed issues can be found in Bugzilla. This time it not only contains bug fixes (quite a few of them, though) but also some new features like improved support for Apache Harmony, gcj and OpenJDK7 (where javac likes to break command line backwards compatibility again).

The issues that came up most frequently and have been fixed are "xslt no longer uses the specified classpath" and "propertyfile with prefix doesn't work as expected" IIRC.

We've added a new collection so that the order of files inside <copy> has become predictable. Detecting Ant 1.8.2 means either

  <antversion property="Ant-1.8.2-or-later"
              atleast="1.8.2"/>

or

  <available property="Ant-1.8.2-or-later"
             classname="org.apache.tools.ant.util.LinkedHashtable"/>

path: /en/Apache/Ant | # | Writebacks

Last weekend Antoine announced Ant 1.8.1. This is more or less a pure bugfix release which addresses a few important issues. The fix with the biggest impact likely is that <extension-point> and <import> finally work together as intended - and even documented.

The full list of issues raised against Ant that have been fixed with this release can be found in Bugzilla. We've adressed almost all issues raised against Ant 1.8.0 so far.

Thanks to the new <augment> task, detecting Ant 1.8.1 means either

  <antversion property="Ant-1.8.1-or-later"
              atleast="1.8.1"/>

or

  <available property="Ant-1.8.1-or-later"
             classname="org.apache.tools.ant.taskdefs.AugmentReference"/>

path: /en/Apache/Ant | # | Writebacks