Creating Debian Packages with Maven 2

I spent some time researching different approaches to integrate Debian packaging with Maven 2. A number of Maven plugins promised to take care of the hard work with some simple XML configuration. A closer look, however, proved that the plugins were immature, deprecated or not ready for production use as of December 2009.

I summarize my findings:

• deb-maven-plugin: The Maven plugin deb-maven-plugin no longer exists and cannot be found on Maven central repository. It seems that it was deprecated in favor of the Unix Maven Plugin.
  
• unix-maven-plugin: as of December 2009, this plugin is in Alpha. Documentation is inconsistent, ran into a issues with artifact resolution, and problems with the dpkg plugin mode.
  
• jdeb Ant/Maven plugin: documentation does not cover how to configure the Maven plugin but instead focuses on ant.
  

In light of the limited plugin support, I opted for implementing a solution based on the debuild tool. This path seems to be popular among some open-source projects, including MuleSource. So, without further ado, here is my recipe for integrating Debian packaging into your application's Maven build lifecycle.

Introducing Debian Packages

A debian package is a collection of files along with instructions on where those files should reside in the filesystem, what libraries or other programs the contents of the package are dependent on (if any), setup instructions, and basic configuration scripts. Packages usually contain precompiled software, but you can also package source code. This document describes how you can enable Debian packaging for your Maven project, making your packages easily installable using the APT and dpkg tools.

Prerequisites

In order to build your own debian packages using the debuild tool, you will need the Debian repackaging utilities which you can install with the following command:

sudo apt-get install devscripts build-essential fakeroot 

Required Files

Debian packages require certain files to be packaged in the .deb archive. These files are contained within the debian folder and include the control file, changelog and optional installation and uninstallation scripts. These files, along with their contents, are covered in the next sections.

An example directory layout is given below:

distributions/deb-package/build.sh
distributions/deb-package/debian/changelog
distributions/deb-package/debian/compat
distributions/deb-package/debian/control
distributions/deb-package/debian/dirs
distributions/deb-package/debian/postinst
distributions/deb-package/debian/postrm
distributions/deb-package/debian/preinst
distributions/deb-package/debian/rules

Control File

Control files consist of key/value pairs in the format key: value. Most of these fields are optional, but a few are required. You can find the full list of fields in the Debian Policy Manual Chapter 5 - Control files and their fields.

An example control file for the mail-service is given below:

Source: acme-mail-service
Priority: optional
Section: devel
Build-Depends: debhelper (>= 7)
Maintainer: Acme Corp 
Homepage: http://www.acmecorp.com
Package: acme-mail-service
Architecture: all
Depends:
Description: Debian package for the Acme mail-service

Changelog

Changes in the Debian version of the package should be briefly explained in the Debian changelog file debian/changelog. This should include any changes and updates to the package. The format of the debian/changelog allows the package building tools to discover which version of the package is being built and find out other release-specific information. You can find additional information in the Debian Policy Manual Chapter 4 - Source packages.

An example changelog is given here for the mail-service:

acme-mail-service (9.11.18) unstable; urgency=low

  * initial revision

 -- Acme Corp   Fri, 20 Nov 2009 17:21:37 -0400

Scripts

It is possible to supply scripts as part of a package which the package management system will run for you when your package is installed, upgraded or removed. These scripts are the files preinst, postinst, prerm and postrm in the control area of the package. They must be proper executable files; if they are scripts (which is recommended), they must start with the usual #! convention. They should be readable and executable by anyone, and must not be world-writable.

The package management system looks at the exit status from these scripts. It is important that they exit with a non-zero status if there is an error, so that the package management system can stop its processing. It is also important, of course, that they exit with a zero status if everything went well. Broadly speaking the preinst is called before (a particular version of) a package is installed, and the postinst afterwards; the prerm before (a version of) a package is removed and the postrm afterwards. More information is available in the Debian Policy Manual Chapter 6 - Package maintainer scripts and installation procedure.

An example of the postinst script follows:

#!/bin/bash

SERVICE_NAME=mail-service

# set ownership permissions
chown -R acme:acme /home/acme/packaging

echo "Installing symlink for ${SERVICE_NAME} in /home/acme"
ln -s /home/acme/packaging/${SERVICE_NAME} /home/acme/${SERVICE_NAME}

Similary for postrm:

#!/bin/bash

SERVICE_NAME=mail-service

# remove site symlink here
if [ -L /home/acme/${SERVICE_NAME} ]; then
  echo "Removing ${SERVICE_NAME} symlink from /home/acme"
  rm /home/acme/mail-service
fi

and preinst:

#!/bin/bash

# verify that the required installation directory is present
if [ ! -d /home/acme ]; then
  echo "Directory /home/acme was not found. Aborting!"
  exit 1
fi

Maven Integration

With the required Debian control files in place, we are ready to integrate the debian packaging process into the Maven build lifecycle. To this end, we introduce a helper build script which we will invoke via the Maven ant-run plugin.

Build Script

The helper build script build.sh kicks off the building of the Debian package by invoking the debuild packaging tool. An example for the mail-service is shown below:

#!/bin/sh

MODULE_NAME=mail-service

echo "Building Debian package for ${MODULE_NAME}"
echo

rm -rf ../../target/deb-pkg
mkdir -p ../../target/deb-pkg

# Extract the tarball to the package workspace
tar xfz ../../target/${MODULE_NAME}.tar.gz --directory ../../target/deb-pkg
# Add the Debian control files
cp -r debian ../../target/deb-pkg

# Build the package
cd ../../target/deb-pkg
debuild --check-dirname-level 0 -us -uc -b

Changes to pom.xml

We can easily integrate the build script above into the Maven build lifecycle with the help of the Maven ant-run plugin. In addition, we introduce a new Maven profile deb-pkg to active the building of Debian packages. This prevents Debian packages from being built by default on invoking mvn package. The following pom.xml snippet from the mail-service demonstrates this process:

<profiles>
      <profile>
         <id>deb-pkg</id>
         <build>
            <plugins>
               <plugin>
                  <groupId>org.apache.maven.plugins</groupId>
                  <artifactId>maven-antrun-plugin</artifactId>
                  <configuration>
                     <tasks>
                        <echo
                           message="Creating deb package">
                        </echo>
                        <exec
                           dir="${basedir}/distributions/deb-package"
                           executable="${basedir}/distributions/deb-package/build.sh"
                           failonerror="true">
                        </exec>
                     </tasks>
                  </configuration>
                  <executions>
                     <execution>
                        <id>deb-pkg</id>
                        <phase>package</phase>
                        <goals>
                           <goal>run</goal>
                        </goals>
                     </execution>
                  </executions>
               </plugin>
            </plugins>
         </build>
      </profile>
   </profiles>

Finally, in order to build your Debian package using Maven, you would simply invoke:

maven package -Pdeb-pkg

Advanced Topics

Setting up a Local Apt Repository

Creating a local apt repository is a great way to test your newly created Debian packages. You can do this easily with the following steps:

(i). create a directory to contain your local repo:

mkdir /tmp/apt-repo

(ii). add the following line to your /etc/apt/sources.list:

deb file:///tmp/apt-repo/ binary/

After building a new Debian package, you can add it to your repository and refresh the repository inventory by invoking:

cd /tmp/apt-repo
cp /home/acme/mail-service/target/*.deb binary/
dpkg-scanpackages binary | gzip -9c > binary/Packages.gz
apt-get update

Once you have setup your local apt repository, you can easily test your new debian packages by invoking apt-get install your-package for installing/updating your package, and apt-get remove your-package for uninstallation.

Adding a Service to Startup

On Linux systems, you can add your service to the appropriate run-level so that it automatically starts at bootup and stops on shutdown. As part of your Debian package, you should install a script to /etc/init.d/ with start/stop operations. An example script is given below:

#! /bin/sh
# /etc/init.d/blah
#

# Some things that run always
touch /var/lock/blah

# Carry out specific functions when asked to by the system
case "$1" in
  start)
    echo "Starting script blah "
    echo "Could do more here"
    ;;
  stop)
    echo "Stopping script blah"
    echo "Could do more here"
    ;;
  *)
    echo "Usage: /etc/init.d/blah {start|stop}"
    exit 1
    ;;
esac

exit 0

Once your script is copied to /etc/init.d/, you can install it so that it is invoked at the appropriate run-level using the update-rc.d command:

update-rc.d myscript defaults

where myscript should be replaced with the name of your service script. This command will make links to start the service in runlevels 2345, and stop the service in runlevels 016.

Similarly, to remove a service from startup you must first remove the script from /etc/init.d/ and only then invoke:

update-rc.d myscript remove

Using the update-rc.d command, we can install and uninstall our service from the run-level using the Debian package's postinst and postrm maintainer scripts.

Further Reading

Basics of the Debian package management system
Debian Policy Manual
IBM DeveloperWorks: Create Debian Linux packages
Making scripts runt at boot time with Debian
An introduction to run-levels
Ubuntu Bootup Howto

Blogging Under Internationalized Domain Names

Back in 2007, I didn't pay too much attention to the launch of internationalized domain names (IDNs) in Spanish. After starting my blog, I started to think that there might be a few hidden gems waiting to be discovered. And it turned out to be true. I just registered the domain name soñador.org (dreamer in Spanish) with 101domains.com and it works like a charm. I even set it up so that it redirects to this blog! I haven't had a chance to experiment with internationalized email addresses yet.

Static Mocks for Groovy

After experimenting with Groovy's excellent support for mock objects, I was a little disappointed to find out that mocking of static methods was not supported. I decided to write my own implementation and share it with others.

First, let's start with the enhanced StaticMockFor class itself:

/**
 * Use this class to mock static methods in a similar way to 
 * Groovy's MockFor implementation (which does not support
 * mocking of static methods).
 * 
 * Note that this implementation does not support demand
 * ranges, unlike Groovy's MockFor.
 * 
 * @author Gerardo Viedma
 */
class StaticMockFor {
   
   final Class clazz
   //final Demand demand = new Demand()
   final OrderedDemand demand = new OrderedDemand()
   
   StaticMockFor(Class clazz) {
      this.clazz = clazz
   }

   def use(Closure clo) {
      try {
         // override with mock behavior
         clazz.metaClass.static.invokeMethod = { String name, args ->
            demand.invoke(name, args)
         }         
         // execute the closure
         clo()
         // verify that we satisfied all the demands
         demand.verify()
      }
      finally {
         // reset to the original invokeMethod after verifying mocks
         clazz.metaClass.static.invokeMethod = { String name, args ->
            def original = clazz.metaClass.getStaticMetaMethod(name, args)
            if (original)
               original.invoke(name, args)
            else
               throw new RuntimeException("Method $name not found!")
         }
      }
   }
}

Note that the StaticMockFor implementation relies on the OrderedDemand type to verify cardinality and ordering of method calls to the mocked class. OrderedDemand leverages Groovy's Expando mechanism to add dynamic behavior allowing us to override the mocked static methods.

/**
 * Encapsulates demands for instances of StaticMockFor.
 * The implementation is based on a queue of demanded
 * method invokations that have to be met in order
 * and with the specified cardinality.
 * 
 * @author Gerardo Viedma
 */
class OrderedDemand {

   final Map closures = [:]
   // maintain calls in order 
   final Queue calls = [] as Queue
   // maps method name to an integer count
   final Map actualCount = [:]
   // maps method name to a range
   final Map expectedCount = [:]
   
   // add dynamic method behavior
   static {
      OrderedDemand.metaClass.invokeMethod = { String name, args ->
         def metaMethod = OrderedDemand.metaClass.getMetaMethod(name, args)
         // pass on calls to invoke and verify
         if(metaMethod) {
            return metaMethod.invoke(delegate,args)
         }
         // add methods dynamically and keep track of their counts
         def range
         Closure clo
         if (args.size() == 1) {
            range = 1..1
            clo = args[0]
         }
         else {
            range = args[0]
            if (!(range instanceof Range))
               range = range..range
            clo = args[1]
         }
         // repeat the methods as many times as necessary
         calls.add(name)
         actualCount.put(name, 0)
         expectedCount.put(name, range)
         closures.put(name, clo)
      }
   }
   
   /*
    * Invokes a method, removing it from the demand queue
    * if it was the next invokation in the demand queue.
    * Otherwise, throws an assertion failure.
    */
   def invoke(String name, args) {
      if (calls.isEmpty())
         throw new RuntimeException("Did not expect any calls to $name")
      def head = calls.peek()   
      if (name == head) {
         def rslt = closures[name](args)
         // update the count
         def actual = actualCount.get(name) + 1
         actualCount.put(name, actual)
         return rslt
      }
      else {
         verify()
         // updated the head, so call recursively
         invoke(name, args)
      }
   }

   /*
    * Verifies that all demands have been met by ensuring
    * that all demanded methods were invoked.
    */
   def verify() {
      def head = calls.remove()
      def expected = expectedCount.get(head)
      def actual = actualCount.get(head)
      if (!expected.contains(actual)) {
         throw new RuntimeException("Incorrect number of calls to $head")
      }      
   }
}

Finally, we can demonstrate usage of the StaticMockFor class and verify its behavior by writing some unit tests:

/**
 * Tests functionality of StaticMockFor instances.
 * 
 * @author Gerardo Viedma
 */
class StaticMockForTest extends GroovyTestCase {
   
   static final NOT_SO_RANDOM = 0.5
   
   void testSingleCall() {
      def mathMock = new StaticMockFor(Math)
      mathMock.demand.random { 
         println 'Mocking Math.random()'
         NOT_SO_RANDOM
      }
      mathMock.use {
         assertEquals Math.random(), NOT_SO_RANDOM
      }
      assertFalse Math.random() == NOT_SO_RANDOM
   }
   
   void testMultipleCalls() {
      def mathMock = new StaticMockFor(Math)
      mathMock.demand.random(3) { 
         println 'Mocking Math.random()'
         NOT_SO_RANDOM
      }
      mathMock.use {
         assertEquals Math.random(), NOT_SO_RANDOM
         assertEquals Math.random(), NOT_SO_RANDOM
         assertEquals Math.random(), NOT_SO_RANDOM
      }
      assertFalse Math.random() == NOT_SO_RANDOM
   }
   
   void testCallRange() {
      def mathMock = new StaticMockFor(Math)
      mathMock.demand.random(2..4) { 
         println 'Mocking Math.random()'
         NOT_SO_RANDOM
      }
      mathMock.use {
         assertEquals Math.random(), NOT_SO_RANDOM
         assertEquals Math.random(), NOT_SO_RANDOM
         assertEquals Math.random(), NOT_SO_RANDOM
      }
      assertFalse Math.random() == NOT_SO_RANDOM
   }
   
   void testMissingCalls() {
      def mathMock = new StaticMockFor(Math)
      mathMock.demand.random(4) { 
         println 'Mocking Math.random()'
         NOT_SO_RANDOM
      }
      shouldFail {
         mathMock.use {
            assertEquals Math.random(), NOT_SO_RANDOM
            assertEquals Math.random(), NOT_SO_RANDOM
            assertEquals Math.random(), NOT_SO_RANDOM
         }
      }
      assertFalse Math.random() == NOT_SO_RANDOM
   }
   
   void testMissingCallsInRange() {
      def mathMock = new StaticMockFor(Math)
      mathMock.demand.random(2..4) { 
         println 'Mocking Math.random()'
         NOT_SO_RANDOM
      }
      shouldFail {
         mathMock.use {
            assertEquals Math.random(), NOT_SO_RANDOM
         }
      }
      assertFalse Math.random() == NOT_SO_RANDOM
   }
   
   void testExceededCalls() {
      def mathMock = new StaticMockFor(Math)
      mathMock.demand.random(2) { 
         println 'Mocking Math.random()'
         NOT_SO_RANDOM
      }
      shouldFail {
         mathMock.use {
            assertEquals Math.random(), NOT_SO_RANDOM
            assertEquals Math.random(), NOT_SO_RANDOM
            assertEquals Math.random(), NOT_SO_RANDOM
         }
      }
      assertFalse Math.random() == NOT_SO_RANDOM
   }
   
   void testExceededCallsInRange() {
      def mathMock = new StaticMockFor(Math)
      mathMock.demand.random(1..2) { 
         println 'Mocking Math.random()'
         NOT_SO_RANDOM
      }
      shouldFail {
         mathMock.use {
            assertEquals Math.random(), NOT_SO_RANDOM
            assertEquals Math.random(), NOT_SO_RANDOM
            assertEquals Math.random(), NOT_SO_RANDOM
         }
      }
      assertFalse Math.random() == NOT_SO_RANDOM
   }   
   
   void testCorrectCallOrder() {
      def mathMock = new StaticMockFor(Math)
      mathMock.demand.random { 
         println 'Mocking Math.random()'
         NOT_SO_RANDOM
      }
      mathMock.demand.abs { a ->
         println 'Mocking Math.abs()'
         (a > 0) ? a : -a
      }      
      mathMock.use {
         assertEquals Math.random(), NOT_SO_RANDOM
         assertEquals Math.abs(NOT_SO_RANDOM), NOT_SO_RANDOM
      }
   }
   
   void testWrongCallOrder() {
      def mathMock = new StaticMockFor(Math)
      mathMock.demand.random { 
         println 'Mocking Math.random()'
         NOT_SO_RANDOM
      }
      mathMock.demand.abs { a ->
         println 'Mocking Math.abs()'
         (a > 0) ? a : -a
      }   
      shouldFail {
         mathMock.use {
            assertEquals Math.abs(NOT_SO_RANDOM), NOT_SO_RANDOM
            assertEquals Math.random(), NOT_SO_RANDOM
         }   
      }
   }
   
}