Hashbang.ca

At Pythian, we have one application that is composed of several components, the deployment of which needs to conform to our slightly peculiar server setup. Until recently, this required manually deploying each component. I did this a couple weeks ago, and it took me something like 40 hours to figure out and complete. As I went, I started reading up on Module::Build, trying to figure out how to automate as much as possible. It turns out that this core module gives us a surprisingly powerful tool for customized deployment. First, it will help to understand a few aspects of how our code is deployed.

This application is a collection of old-school CGI scripts running under mod_perl, plus modules from external software packages, and some modules which are used in both the server and client of this application. To make upgrade and rollback easier, we use a symlink that points to the current version of the application. To switch versions, just switch the symlink and restart apache. Unfortunately, we have one such symlink for every component – the consolidated configuration, the shared libraries, the three web endpoints, the external dependencies. These should be merged so only one symlink need be changed, but I wanted to modify the deployment process as little as possible, just automating what we already do manually.

As I spent over 40 hours working on this build-out, I made notes on what components went where, and started to see how a standard build tool like Module::Build could be of use. These components can’t be installed like normal CPAN dists, but there are features to permit a customized install process that does what we want.

First, we want to install several dists into several custom locations which have a versioned name. Today, I want to install them into /var/www/app/C1/C1_v1.2.3 and /var/www/app/C2/C2_v1.2.3. So, let’s add a configurable install target (the /var/www/app part), and a configurable install version (the v1.2.3 part). We’ll want to accept these as parameters: perl Build.PL --install-version v2.3.4 and via environment variables. This makes it easier to specify them once in the environment, and then install each component individually, without having to remember what today’s version number is, or forgetting to set it, or making a typo.

use Module::Build;
my $build = Module::Build->new(
    ...
    get_options => {
        'base-dir' => {
            type => '=s',
            default => $ENV{PYTHIAN_BASE_DIR} || '/var/www/app',
        'install-version' => {
            type => '=s',
            default => $ENV{PYTHIAN_INSTALL_VERSION} || do {
                chomp(my $date = `date +%Y-%m-%d`); $d };
                $date;
            },
    },
);

Next, we can use those options to specify custom install locations:

my $base_dir = $build->args('base-dir');
my $install_version = $build->args('install-version');

$build->install_base( "${base_dir}/libs_shared/lib_${install_version}" );

# Remember, even if they're in the "bin" directory, your files will be
# scanned, and if they're actually scripts, they'll go in "blib/script"!
$build->install_path( script => "${base_dir}/product_server/product_srv_${install_version}" );

Finally, we can add new components, like configuration files:

my $build = Module::Build->new(
    ...
    config_files => { map { $_ => $_ } qw( config/component_A.conf ) },
);
$build->add_build_element('config');
$build->install_path( config => "${base_dir}/config_shared/config_${install_version}" );

$build->create_build_script;

Lastly, I wanted to give users a hint for how to switch the symlink to point to the just-deployed version. I could have done this in the Build.PL script itself, but I wanted to present a message only after the install action, and I wanted to experiment with subclassing Module::Build.

my $mb_subclass = Module::Build->subclass(
    class => 'Module::Build::ProductServer',
    code  => join '', map { sprintf < <'SUBCLASS', $_, $_ } ('', 'fake') ); # Add this code for install and fakeinstall

sub ACTION_%sinstall {
    my $self = shift;
    $self->SUPER::ACTION_%sinstall;

    my $base_dir = $self->args('base-dir');
    my $install_version = $self->args('install-version');
    my $new = "${base_dir}/product_server/product_srv_${install_version}";
    my $tmp = "${base_dir}/product_server/product_srv_${install_version}.tmp";
    my $sym = "${base_dir}/product_server/api";
    # ln -s new-thing link-tmp && mv -Tf link-tmp link
    print < <"END";
To switch the symlinks, do:
    ln -s $new $tmp && mv -Tf $tmp $sym
END
}

SUBCLASS

(As a side-note, the normal way of giving a symlink a new target is not atomic – you need to use a rename call, which is guaranteed to be atomic under POSIX; see File::Symlink::Atomic.)

Once we do this custom setup in all the components required for the server build-out, the install process becomes much simpler:

export PYTHIAN_INSTALL_VERSION=`date +%Y-%m-%d-%I`
cd C1
perl Build.PL && ./Build && ./Build install
# Copy the provided command at the end of the output for later
cd ../C2
# Wash, rinse, repeat
# At the end,
ln -s ... && mv -Tf ... # once for each component

Much faster. And once this automated method of installing the components is actually used, we can more easily change how the deployment works, because humans won’t have to learn a new way - only the Build.PL scripts need to change.