Chris Read

While we were still working on Buildix 2, people started asking about an AMI for Buildix on Amazons EC2. This didn’t seem to be such a big ask, but now that I’ve finally gotten around to working on this I’ve found it can be a bit fiddly! While there is a lot of good documentation in the various sections of the EC2 site, I still had a quite a few head scratching moments trying to create my own Ubuntu 7.04 Server image to load Buildix into.

The Buildix image is now available for public use as ami-e4ca2f8d.

Here’s a couple of tips to keep in mind when rolling your own:

Read the rest of this entry »

I’ve been at JAOO for the past few days, and while here I had a chance to do a presentation with Erik Doernenburg on Continuous Integration and CruiseControl. We used the new Beta version of Buildix 2 to show people the new CruiseControl Dashboard, and quite a few people were impressed with it. Favourite features were the CCTray integration, and the ability to see the status of a large number of projects at a glance.

As always, there were also people who were interested in hearing about how it can be used for non Java projects. I had a good chat to one person who is interested in using it on a mixed Common LISP and Erlang project he’s working on. I’m looking forward to hearing how it goes for him. Due to lack of experience on my part I could unfortunately not help him much with the darcs problems he’s having though. Some people have all the fun…

It was also quite useful to speak to people about the problems they’re currently facing when trying to use CruiseControl. A common theme is people trying to manage large numbers of builds, or trying to build products across large numbers of different platforms. These are problems the dedicated ThoughtWorks development team are currently working on, so it’s great to get the validation that we’re putting effort into the things people care about now.

Up until now I’ve managed to avoid ranting here, but I can no longer resist! After all these years of experience, why are the error messages in windows still generally meaningless? Is it because there are so many of them?

The problem I faced was trying to patch a game. Downloaded the patch, tried to apply it, but got the message “The application failed to initialize properly (0xc0000135)”. Much confusion. Downloaded from another source, tried again, same error. Tried on my laptop, and this time it worked perfectly. What’s the problem?

Turns out the patch is written in .Net, and I don’t have the correct version of the runtime installed on my desktop, but I do have it on my laptop. I only had .Net 1.0, and it needs a newer version. Good thing the error message told me that in the first place then, isn’t it…

A lot of people are under the impression that CruiseControl is solely intended for Java projects. This is a common misconception. At my current client we’re using CruiseControl to build a 15 year old, 2.5 million line mixed C/C++ app using GNU make. We’re also using it to build and functionally test a new C app we’re writing as part of the project. I’ve also used it in the past on Python apps.

There are a number of ways to do this, and they all really depend on what you’re using Cruise for. If you’re simply using it for compilation, then you can simply use <exec> to call make (or whatever you’re building with). By default it will break the build if the command returns a non-zero return code, so it should be smart enough to do this out of the box. I know that some of our Ruby projects that use this method too.

My personal preference is to get CruiseControl to do an Ant build, which has some additional logic, including taging the source repository with the build label if the build passes. The approach is very similar in this case, and has the following steps:

• Have ant do any additional work/checks you want to ensure the environment is how you want it.
• Update my working copy to an exact known version that can be tagged at a later stage if the build is good (primarily a CVS problem, not really needed if you’re using Subversion).
• Use the <exec> task from ant to run the build
• If you have additional tests to run, use an <exec> within a <parallel> block to start your newly built binary in the background, then run the tests
• If everything is good and passes, tag your source code with the build label.

So, although it is fairly easy to add build plugins for various other projects, you can save yourself a bit of pain simply using the one <exec> or shelling out of an <ant> builder.

One of the things I spend a lot of my life doing is looking after Continuous Integration environments. As I work for ThoughtWorks, and am one of the guys who created Buildix, it should come as no surprise that this is all done with CruiseControl. However, in the past few months there a few new Continuous Integration tools that have crept out onto the scene. The two that I hear the most about are TeamCity and Bamboo. I finally managed to get a few hours to test them out, and here’s my impressions.

One of the things I wanted to see was how quick and easy it was to get these tools up and running. With this in mind, I specifically decided in advance to pull down the .war bundle of each one and try that out. In my experience, this would be the one that most of the clients I’ve been at would have gone for.

Notes:

• This is very much a first impressions run of the products - I had a limited time window (2 hours total) to do this in.
• People could quite fairly say that I have a biased view, but I’ve tried to keep this as fair as possible.
• I only tested doing an Ant build of a simple sample webapp from a subversion repository.
• You have to buy a license for TeamCity and Bamboo (I got evaluation licenses for both), but CruiseControl is still free…

Test Environment:

• Fedora Core 4
• Via C3 800 server with 512Mb RAM
• Sun Java 1.6.0-b105
• Apache Ant 1.7.0
• Jetty 6.1.1
• Subversion 1.2.3

Read the rest of this entry »

Part of what we’re doing at my current client is writing an HTTP interface to allow read only access to a Tangosol cache. For the first month or so we’ve been using Jetty (6.0.2) for testing and development, because it’s free and easy. Everything has been going well, and we finally got the official ruling that our webapp will run in production under JBoss (4.0.3SP1). We did expect this, as current webapps they have in production run under JBoss, but we were not sure of which version we were going to end up on so we stuck with Jetty. We’ve been very carefull to keep our app as vanilla as possible, but when we deploy our happy app under JBoss, we start getting showered with NullPointerExceptions like these:

java.lang.NullPointerException
at org.apache.xalan.transformer.SerializerSwitcher.switchSerializerIfHTML(SerializerSwitcher.java:153)
at org.apache.xalan.transformer.TransformerIdentityImpl.startElement(TransformerIdentityImpl.java:991)

After a bit of poking around I found that the only nulls that existed in that area of our code was when we were calling startElement() and endElement() with nulls. A quick look at the JavaDocs for these methods says that you should pass “the empty string” if there is no Namespace URI or Namespace. Simply replacing calls like:

transformerHandler.startElement(null, null, ERROR_TAG_NAME, errorAttributes)

with

transformerHandler.startElement(”", “”, ERROR_TAG_NAME, errorAttributes)

sorted out the problem!

We’re only due to go live with this app in July, but one of the patterns that has saved me many times is to test early and often. The trick is not to limit your testing, but push it as far as you can go. In this case, testing in a production like environment 5 months before we’re due to go live has meant we only needed to change a few lines of code in two classes. Imagine how much more we would have to change in 5 months time…

For those of you in the London area who would like to know a bit more about Buildix, see it in action or just ask questions - I’ll be showing it off (so to speak) at London 2.0 RC6. For more info on where and when, check out Sam Newman’s blog entry. I think we even have a few CD’s left from Agile 2006 for those who are interested…

Ever since I started working for ThoughtWorks, I have heard people saying things along the lines of “Wouldn’t it be nice if we had some kind of Cruise-in-a-box to help us get projects up and running quickly?”. After about 6 months I was thinking the same thing, and started tinkering around with various options. Nothing really happened, until the first week of January this year when a group of us who often fill “Build Master” type rolls were all in the office together with a few days unassigned to clients. We were all sitting around a desk together catching up, when somehow the topic once again emerged. With the critical mass in place, this sparked off the birth of Buildix.

The whole point of Buildix is to help any Java based Agile Development Project get up and running as quickly as possible buy providing them with pre-configured and integrated version control system, continuous integration framework, wiki and issue tracking system. We chose our favourite products in each of these areas - Subversion, CruiseControl and Trac. Another common difficulty faced by our development teams, especially in the early stages of a project, is network access. Sometimes all we get from our clients when we arrive on site is a switch to allow us to get our laptops to talk to each other - no DNS, file shares, anything. Buildix can also help in situations like this, as it also runs Samba, and will run as a DNS and DHCP server if given the correct kernel boot parameters.

So, six months after we started, and after a few internal releases, we decided to give something back to the community that helps us do our job, and make Buildix available to everyone. Enjoy!

At one of the clients I’m assigned to at the moment, we’re moving our development environment to Solaris 10 on Sun x4100 servers. We have two physical machines, one for our CruiseControl environments, and one for all our testing. To make good use of the resources we have (Dual Core CPU’s, lots of RAM) I’ve been carving them into zones. I’ve tinkered with zones in Solaris 10 ever since the first beta build that featured them, but it was always for little things and never anything serious. Consequently I thought they were quick and painless. Note the use of the word “thought”. Don’t get me wrong, they are the (almost) perfect solution for what we need, it’s just that if you’re planning on doing anything serious with them, here’s a list of gotchas you need to take in to consideration.

Read the rest of this entry »

A few of us are using Trac for an internal project at work, and at some stage in the process we decided to use the wiki as the distribution documentation. Everything has been going wonderfully, until we realized we that we wanted the documentation to be available without having to run Trac. After some digging around, I found this ticket on the Trac dev site where Alec Thomas put up a diff to allow you to download Trac wiki pages as HTML. After applying the patch, I found that it worked well, but there was still no easy way to pull the whole wiki down as HTML. So I wrote a python script to do it, and the script is available here for those who are interested.