Repeating Beats

A State of Refactoring

2013-02-18T00:00:00Z

I was recently working on a feature with requirements and constraints that can be boiled down to:

Listen for filesystem change events and initiate an asynchronous worker process. The simplified watcher API:

:::javascript
 var watcher = require('./watcher');

 watcher.on('change', function () {
   // Do something useful
 });

The worker process cannot handle concurrency, so ensure only one worker is running at a time. The simplified worker API:

:::javascript
 var worker = require('./worker');

 worker.run(function (err) {
   // Invoked when the worker is done
 });

If a change event is emitted while the process is running, mark the output as stale and re-run the process when it finishes.
While stale, any additional change events can be ignored, and the process only needs to be re-run once.
In JavaScript.

Branching Out

As I was exploring the problem space, my first cut at a solution that passed my tests looked a lot like:

:::javascript
var worker = require('./worker')
  , watcher = require('./watcher')
  , working = false
  , stale = false;

watcher.on('change', function () {

  function handler(err) {
    if (err) {
      console.error(err.stack);
      return process.exit(1);
    }

    if (stale) {
      stale = false;
      worker.run(handler);
    }
    else {
      working = false;
    }
  }

  if (!working) {
    working = true;
    worker.run(handler);
  }
  else { // working
    stale = true;
  }

});

This solution uses two variables at module scope (working and stale) to keep track of system state. Change events and worker callbacks branch on the current state to determine the necessary state modifications and worker strategy. The worker callback needs to reference itself to handle the stale case.

This code works, but it’s not very easy to read. There is a lot of bookkeeping around the two state variables. A reader would be very hardpressed to infer the problem requirements from the code itself. In other words, it’s the kind of code that is just asking for bugs the next time an unfamiliar developer dives into it.

State, Meet Machine

While attempting to refactor my initial implementation into something cleaner and more maintainable, I noted that I was effectively building a machine with three states (ready, working, stale) that responds to two events (change, done).

The machine’s transitions can be described with a simple JavaScript object:

:::javascript
{
  ready: {
    change: 'working'
  }
, working: {
    change: 'stale'
  , done: 'ready'
  }
, stale: {
    change: 'stale'
  , done: 'working'
  }
}

Given this declarative state machine definition, all I needed was a way to proxy watcher change events to the machine and observe when the machine entered the working state. This yields a considerably more straightforward solution:

:::javascript
var statem = require('statem')
  , worker = require('./worker')
  , watcher = require('./watcher')
  , machine;

machine = statem.machine({
  initial: 'ready'
, states: {
    ready: {
      change: 'working'
    }
  , working: {
      change: 'stale'
    , done: 'ready'
    }
  , stale: {
      change: 'stale'
    , done: 'working'
    }
  }
});

machine.onEnter('working', function () {
  worker.run(function (err) {
    if (err) {
      console.error(err.stack);
      return process.exit(1);
    }
  
    machine.send('done');
  });
});

watcher.on('change', function () {
  machine.send('change');
});

One Little Problem

➜ ~ npm info statem
npm http GET https://registry.npmjs.org/statem
npm http 404 https://registry.npmjs.org/statem
npm ERR! 404 'statem' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it

I often write code to APIs that don’t exist, if only to fully understand what the perfect API looks like from the consumer perspective. There is a limit to this, but I definitely don’t like pushing complexity to the consumer when it can be avoided.

In this case, I found myself staring at a wall of JavaScript state machines, none of which really provided the semantics and simplicity I was looking for. Pragmatism won the day, and I ended up shipping the feature with javascript-state-machine, but I eventually found time to poke at the problem a bit myself.

I now have a mostly complete implementation of statem built on top of Node’s EventEmitter. I haven’t published it yet, mostly because I need to explore how I want the libary to behave in error cases (invalid states, invalid transitions, etc.). I’m also a bit wary of adding to the proliferation of state machine options without a compelling differentiator. Though it seems doing so is The Node Way™.

In the meantime, consider it a work in progress.

Ship It

2013-02-05T00:00:00Z

It has been slightly over two years since I have written in this space. My ”I’m going to have a blog” movement of early 2011 did not last very long. I’m a very busy man.

I started getting the itch to write again late last year. By that point, I was both displeased by my previous design attempts and subtly discouraged by the richness of the Wordpress authoring experience. I wanted to whip up some Markdown, publish, and call it a day. And the rewrite began.

Goodbye Wordpress, PHP, MySQL, CSS, and WYSIWYG.

Hello Middleman, Ruby, S3, SASS, and Markdown.

It’s all to easy for me to fall into the trap where I don’t ship because I’m tweaking and polishing to perfection. Or at least to the closest approximation to perfection that I might stand a chance at achieving. The irony is not lost on me that I have spent far more hours hacking on the implementation of this site than I have actually writing content for it.

Current status? It’s far from perfect. Some of the code is downright sloppy. My stylesheets and layouts are disorganized. The list of user-facing TODOs is far from complete.

So I’m shipping it.

One Thread to Rule Them All

2011-03-08T00:00:00Z

Among other assertions, the Node.js home page declares, “Because nothing blocks, less-than-expert programmers are able to develop fast systems.” It’s tempting to jump to the conclusion that Node.js is a panacea for the difficulties of developing complex systems. Counterpoint: I love async, but I can’t code like this. Perhaps there is a degree of nuance to the aysynchronous programming model.

The techniques described in the remainder of this post were not invented by me. There are numerous libraries dedicated to unwinding asynchronous spaghetti callback hell, one of which (async.js) does things very similar to what I’m going to describe. But that’s not the point. The objective here is to identify some common async coding patterns, exercise the javascripty side of my brain to drum up solutions, and hopefully illuminate a point or two in the process.

The code presented in this post can be examined in its fully glory over at Github. This script can be invoked with node on the command line.

$ node asynchronicity.js

One final caveat. This post is quite long, and I decided not to make it quite longer by dealing with errors. All of my examples, in reality, need to handle errors. I’ll save that as an exercise for the reader.

Our First Contrived Example

Imagine we want to retrieve some data over a very slow network, and we only have one thread on which to do it. Let’s look at the uber-simple synchronous model:

:::javascript
// Slow network getters
var get_first_part = function() {
    // Pretend this blocks for three seconds.
      return 'hello';
};
var get_second_part = function() {
    // Pretend this blocks for one second.
      return 'world';
};
// Let's get some data!
console.log(get_first_part() + ' ' + get_second_part());

Simple. Easy. Wonderful. Other than the fact that our lone thread is locked up for four seconds while we sit around waiting to print ‘hello world’.

Asynchronize It

Instead of waiting around doing nothing while we wait for the slow network to respond, we can pass callback functions that will be invoked when the data is available. Now we have getters that look something like this:

:::javascript
var slow_network = {

  get_first_part: function (callback) {
    setTimeout(function() { callback('hello'); }, 3000);
  },

  get_second_part: function (callback) {
    setTimeout(function() { callback('async'); }, 1000);
  },

  get_third_part: function (callback) {
    setTimeout(function() { callback('world'); }, 500);
  },

};

At this point, I’ll also introduce a quick helper that will be used to report results and timing information for async operations.

:::javascript
var exhibit = function (title) {
  var start_time = Date.now();

  return {
    log: function (msg) {
      console.log(title + ': ' + msg);
    },
    report: function (result) {
      console.log('report for ' + title +
                  ':\n\tresult: ' + result +
                  '\n\telapsed: ' +
                  (Date.now() - start_time) + ' ms');
    }
  };
};

It is common to use nested callbacks to for asynchronous functions. Let’s retrive our data over the slow async network:

:::javascript
var exhibit_a = exhibit('basic nested callbacks');

slow_network.get_first_part(function (first_part) {
  slow_network.get_second_part(function (second_part) {
    slow_network.get_third_part(function (third_part) {
      exhibit_a.report([first_part, second_part, third_part].join(' '));
    });
  });
});

This doesn’t look all that scary, but it’s trending considerably toward spaghetti when compared to our original synchronous version. When each callback requires additional logic before carrying on to the next invocation, it can be very difficult to retain a clear mental context for what is happening.

More importantly, there’s absolutely no reason to invoke these independent functions sequentially. It took four and a half seconds to get to ‘hello async world’ despite the fact that no individual call took longer than three seconds. We should be able fire off all three methods at once and carry on as soon as all three have responded. First, the ugly way:

:::javascript
var exhibit_b = exhibit('parallel polling');
var first, second, third;

slow_network.get_first_part(function (first_part) {
  first = first_part;
});

slow_network.get_second_part(function (second_part) {
  second = second_part;
});

slow_network.get_third_part(function (third_part) {
  third = third_part;
});

var interval_id = setInterval(function () {
  if (first && second && third) {
    clearInterval(interval_id);
    exhibit_b.report([first, second, third].join(' '));
  }
}, 100);

Yes, That Polling Timer is Gross

Speaking more generally, we have a number of asynchronous functions executing in parallel, and we have code that cannot be executed until all of the parallel functions have called back. This is a common pattern in async programming, so we certainly need a better solution than polling. The following helper method allows us to specify our async functions as a series of arguments. The final argument is the callback that will be invoked once all of the parallel functions have called back. This callback provides an array of arbitray result objects that correspond to each of the parallel functions.

:::javascript
var async_helper = {

  parallel: function () {

    var args = Array.prototype.slice.call(arguments),
        callback = args.pop(),
        results = [],
        in_progress = args.length;

    args.forEach(function (async_call, index) {
      async_call(function (result) {
        results[index] = result;
        if (--in_progress == 0) {
          callback(results);
        }
      });
    });
  }

};

Aided by the parallel helper, our slow network example now looks like this:

:::javascript
var exhibit_c = exhibit('async.parallel');

async_helper.parallel(
  slow_network.get_first_part,
  slow_network.get_second_part,
  slow_network.get_third_part,
  function (results) {
    exhibit_c.report(results.join(' '));
  }
);

This form is more efficient than simple nested callbacks, and in my opinion also more readable.

Avoiding Sequential Callback Hell

We don’t always have the luxury of calling functions in parallel. If each function in a sequence needs to pass its results to the next function, we can’t improve in efficiency over the nested async callbacks. We can, however, still provide a helper to keep sequential async calls clean and readable:

:::javascript
var async_helper = {

  parallel: function () {

  // See implementation above ...

  },

  sequence: function () {

    var args = Array.prototype.slice.call(arguments),
        callback = args.pop(),
        results = [];

    function next() {
      var func = args.shift();

      func(function (result) {
        results.push(result);
        args.length > 0 ? next() : callback(results);
      }, results);
    }

    next();
  }

};

Note that the sequential function calls take two arguments: a function callback and a results object. This allows the results to be passed down the sequence. If we pretend that ‘hello async world’ can’t be retrieved in parallel, the sequential helper could be used like this:

:::javascript
var exhibit_d = exhibit('async_helper.sequence');

async_helper.sequence(
  slow_network.get_first_part,
  slow_network.get_second_part,
  slow_network.get_third_part,
  function (results) {
    exhibit_d.report(results.join(' '));
  }
);

Mix It Up

The sequence and parallel helpers can, of course, be combined to create more complex sets of async function calls. For my next contrived example, assume that we have a disk that is extremely slow to read. Fortunately, our contrived slow disk allows us to do some reads in parallel. (Yes, I’m completely making something up to suit the problem I want to solve).

Our contrived disk has two kinds of reads. Direct reads are invoked with a getter and call back with a numeric value. Indirect reads are also invoked with a getter, but indirect reads call back with the name of a direct getter that then must be called to retrieve a value.

Pretend that our awesome slow disk has the following getters:

'get_a' -> 'get_e' (indirect)
'get_b' -> 'get_c' (indirect)
'get_c' ->    2    (direct)
'get_d' ->    5    (direct)
'get_e' ->    1    (direct)

First, I’ll build an object that implements my slow disk’s behavior.

:::javascript
var slow_disk = (function () {

  var getter_factory = function(getters) {

    var widget = {},
        params = null,
        name = null;

    function build_getter(val, delay) {
      return function (callback, results) {
        setTimeout(function() { callback(val); }, delay);
      };
    }

    for (name in getters) {
      params = getters[name];
      widget[name] = build_getter(params.val, params.delay);
    }

    return widget;
  }

  return getter_factory({
    get_a: { val: 'get_e', delay: 2000 },
    get_b: { val: 'get_c', delay: 3000 },
    get_c: { val: 2, delay: 1000 },
    get_d: { val: 5, delay: 1500 },
    get_e: { val: 1, delay: 5000 },
  });

}());

Now that we have a contrived slow disk, let’s invent a problem to solve with async helpers. I want to know the answer to:

a + b + c + d + e

We need to get five values in parallel, and for the indirect values we need to make two sequential calls. We’ll hold off on cleverness for the moment and implement that directly:

:::javascript
var exhibit_e = exhibit('parallel/sequence mix');

async_helper.parallel(

  // Get a, then get the value of what a points to
  function (callback) {
    async_helper.sequence(
      slow_disk.get_a,
      function (callback, results) {
        slow_disk[results[0]](function (val) { callback(val); });
      },
      function (results) {
        // We want the result of the second function
        callback(results[1]);
      }
    );
  },

  // Get b, then get the value of what b points to
  function (callback) {
    async_helper.sequence(
      slow_disk.get_b,
      function (callback, results) {
        slow_disk[results[0]](function (val) { callback(val); });
      },
      function (results) {
        // Again, we want the result of the second function
        callback(results[1]);
      }
    );
  },

  slow_disk.get_c,

  slow_disk.get_d,

  slow_disk.get_e,

  function (results) {
    var sum = results.reduce(function (sum, val) { return sum + val; });

    exhibit_e.report('sum = ' + sum);
  }
);

That does the job, but I have two issues with the solution.

First, the sequential indirect getters are messy, and they clearly repeat code. That issue can be solved with another helper function:

:::javascript
function get_indirect(indirect_method) {
  var self = smarter_slow_disk;

  return function (callback) {
    async_helper.sequence(
      self[indirect_method],
      function (callback, results) {
        self[results[0]](function (val) { callback(val); });
      },
      function (results) {
        callback(results[1]);
      }
    );
  };
};

The second issue with the parallel/sequence mix solution arises from the fact that we’re wasting time. If I can safely assume (of course we can; I defined the problem) that the values on my slow disk are static enough to be cached, we don’t need to make the same slow calls multiple times. But simple result caching only gets us half way.

My problem was carefully designed to include a second call to get_e that is invoked while the first call to get_e is in the middle of its roundtrip. At the time the second function is called, we don’t have a cached response. We need a way to call the second callback when we receive the first result.

:::javascript
var async_cache = function(obj) {

  var responses = {},
      waiting = {},
      caching_obj = {},
      func = null;

  caching_obj.prototype = obj;

  function wrap_method(method, func) {
    return function(callback) {
      if (responses[method]) {
        // We already have this result, invoke callback immediately.
        callback(responses[method]);
      } else if (waiting[method]) {
        // Someone else is waiting for this result. Just add
        // ourself to the waiting list.
        waiting[method].push(callback);
      } else {
        // No one has asked for this yet. Store the caller's
        // callback on the waiting list and handle the response
        // callback by looping through the waiting list and
        // invoking all waiting callbacks.
        waiting[method] = [callback];
        func(function (val) {
          responses[method] = val;
          waiting[method].forEach(function(callback) {
            callback(val);
          });
          delete waiting[method];
        });
      }
    }
  }

  for (var method in obj) {
    func = obj[method];
    // Don't try cache magic for non-functions
    if (typeof func === 'function') {
      caching_obj[method] = wrap_method(method, func);
    }
  }

  return caching_obj;
}

The function above extends the object passed into it with async caching capabilities. It ensures that a slow asynchronous function call is only dispatched once. Repeated calls to the method either use a stored response or add their callback to a list of callbacks that will be invoked when the initial method calls back.

Using both our indirect helper and the caching extension, the solution to the slow disk problem is now:

:::javascript
var smarter_slow_disk = async_cache(slow_disk);

var exhibit_f = exhibit('sequence/parallel mix with caching');
async_helper.parallel(
  get_indirect('get_a'),
  get_indirect('get_b'),
  smarter_slow_disk.get_c,
  smarter_slow_disk.get_d,
  smarter_slow_disk.get_e,
  function (results) {
    var sum = results.reduce(function (sum, val) { return sum + val; });

    exhibit_f.report('sum = ' + sum);
  }
);

Node Doesn’t Solve Your Problems

At no point in this post did I write any complex, multi-threaded code. Such is the promise of Node. I will allow the reader to decide whether or not this exercise involved any complex, single-threaded code.

Node doesn’t necessarily make it easier to solve difficult problems. The onus is still on the developer to identify and abstract away common async patterns. There are no freebies. Clean, maintainable, readable code still requires thoughtfulness and diligence. Or at least it does for less-than-experts like myself who want to try their hands at fast systems.

Step One: Put Your Node in a Box

2011-02-21T00:00:00Z

In my last post, I extolled the virtues of isolating repeatable, versionable development environments in VirtualBox virtual machines with Vagrant. I find this to be particularly important in a rapidly changing software ecosystem like that of Node.js. Node has quickly advanced to the latest 0.4 branch, and the API (though now ‘stable’) has been something of a moving target. Combine this with the relatively poor state of documentation for third party Node libraries, and it is quite easy to get into version hell on a development machine.

Introducing NodeBox

NodeBox uses a set of Chef cookbooks and a custom Vagrant configuration to automatically create VMs for Node application development. The NodeBox repository is intended to be an independent submodule of an application, and I put together a basic “Hello World” example app to demonstrate.

Here’s all it takes to get started with NodeBox (This requires Vagrant and its dependencies. See here. NodeBox requires Vagrant 0.7 or higher and VirtualBox 0.4.x):

$ git clone git://github.com/repeatingbeats/nodebox-example.git --recursive
$ cd nodebox-example/nodebox
$ vagrant up

After Vagrant takes a few minutes to do its magic, point your browser at http://localhost:8000 to see the application in action. The Node app serving this response can be found in app.js in the toplevel nodebox-example directory:

:::javascript
var fs = require('fs'),
    http = require('http')
;

var config = JSON.parse(fs.readFileSync(__dirname + '/nodebox.json'));
var env = config.environment;
var port = config.app_port || (env == "production" ? 8080: 80);

http.createServer(function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
    res.end('Hello World, I Am A NodeBox\n');
}).listen(port);
console.log("Server listening on port: " + port);

This simple application reads configuration variables out of nodebox.json and responds to all requests with the Hello World string.

Isn’t It Easier to Just Build Node and Get On With It?

To get started? Yes. But my goal here isn’t to build the most complex Hello World contraption imaginable. I’m trying to make it easy to mix and match Node versions, Node modules, server configurations, and eventually databases. I’m trying to make it easy to do this in a way that isolates each environment so that when I find one that works, I have a setup that’s ready to roll into production. I don’t want or need eighteen kajillion competing libraries on my laptop. NodeBox is a tool that lets me easily spin up a new environment, and I can blow it up whenever I feel like it.

NodeBox Configuration

NodeBox’s configuration can be adjusted using a required nodebox.json in the application’s root directory. Here’s the configuration for nodebox-example:

{
  "app_name": "nodebox-example",
  "environment": "production",
  "app_port": 8080,
  "host_port": 8000,
  "modules": [ "express",
               "nodeunit",
               "mongoose" ],
  "node_version": "0.4.0",
  "npm_version": "0.3.0-9"
}

Most of the options are self-explanatory. app_port is the port on the VM that Node listens on. This should be set to 80 if the environment is “development”. host_port is the port on on the host machine that forwards to port 80 on the VM. modules is an array of node modules that will be installed by NPM on the VM.

Production-ish NodeBox

NodeBox has recipes for both Development and Production server configurations. The Production recipe comes with the caveat that it is a work in progress, but it contains some basic extra features that make more sense on a production server than on a development server. NodeBox defaults to Production, mostly because the Production box takes fewer steps to get from git clone to Hello World.

The Production recipe hides Node behind the Nginx web server, which proxies requests to Node on an upstream port. Additionally, upstart is used to daemonize the Node process and allow easy starting and stopping of the app. The recipe starts the app during the initial chef configuration, but if you need to restart you can do this:

$ cd /path/to/nodebox-example/nodebox
$ vagrant ssh
$ sudo stop nodebox-example
$ sudo start nodebox-example

The Production NodeBox also uses Monit to restart the application when it dies. There are certainly plenty of other options for monitoring Node processes. I borrowed mine from here, with some slight modifications and of course the Chef templating.

Development NodeBox

Getting started with the development NodeBox requires an extra couple steps:

$ git clone git://github.com/repeatingbeats/nodebox-example.git --recursive
$ cd nodebox-example

You’ll need to modify nodebox.json slightly:

{
  ...
  "environment": "development",
  "app_port": 80,
  ...
}

$ cd nodebox
$ vagrant up
$ vagrant ssh

Finally, now that you’re in a shell on the VM:

$ cd /var/www/nodebox-example
$ sudo supervisor -p app.js -w app.js

Sudo is required so that Node can listen on port 80. Do this in the safety of your own VM! Back on the host machine, point your browser at http://localhost:8000 once more. You should see the Hello World message. Now, change something in app.js. Node-supervisor will detect the change, crash the app, and restart it. The next time you go to http://localhost:8000, you’ll be able to see your new changes.

Yes, the Development NodeBox takes a few more steps to get started, but for good reason. Starting Node from a shell on the VM lets you see your errors!

What Now?

I’ve just walked you through how to get started with a basic Node application using NodeBox. Presumably, if you actually tried this, you might want to start building something. Feel free to poke through the nodebox-example and nodebox repositories to see what’s going on under the hood.

The next step for NodeBox is going to be integration of persistence mechanisms. I’m envisioning an option in nodebox.json that will tell Chef to do all the necessary setup to get one or more of Redis, MongoDB, MySQL, etc. up and running on the VM. Once there are persistence options, the production NodeBox will have everything it needs to start bootstrapping real deployments.

If you’d like to be the very first to hear about future NodeBox developments intermixed with other inconsequential 140-character outbursts, you’ll probably want to follow me on Twitter.

A Slightly Less Intolerable PHP Dev Environment with Vagrant and Chef

2011-02-03T00:00:00Z

Let’s get one thing out of the way: I am not a PHP guru. I have however, wrestled with PHP in attempts to put my own stamp on my WordPress theme. Given my general lack of day-to-day PHP engagement, the first challenge in redesigning this site’s theme was figuring out what on earth I was using for a development environment the first time around. Some digging on the old MacBook revealed that the answer was MAMP. I fired up the MAMP app, hit ‘Start Servers’, and … nothing happened.

In a previous life I might have resigned myself to a few hours of turning the knobs on Apache and MySQL. After 3+ years of port install whatever, curl http://whizbang.org/install.sh | sh, and vanilla building from source, my veteran MacBook is somewhat cluttered. Apparently I never get around to make uninstall.

Enter Vagrant

Lately at work, I’ve been using Vagrant to sandbox my per-project development environments onto their own VMs. Vagrant is a tool to help create and provision VirtualBox machines. The short list of reasons why this is a good idea:

The development environment can be isolated from all the other junk that accumulates on my primary computer.
The development environment can be tuned to match a production server environment as closely as possible.
Provisioning scripts define the machine configuration in code. This means the configuration is repeatable and versionable.

Getting started with Vagrant is easy. All you need is ruby, the gem command line tool, and Virtualbox. The latest-and-greatest Vagrant requires Virtualbox 4.0.x, which you can grab over here. If you need more information for Ruby and RubyGems, you may want to check out the links on Vagrant’s getting started page.

Once you have those dependencies, install the vagrant gem:

$ gem install vagrant

Next, download a ‘base box’. This command fetches the base box for Ubuntu 10.04 (32 bit):

$ vagrant box add ubuntu-lucid-32 http://files.vagrantup.com/lucid32.box

This downloads the base box and stores it at /~/.vagrant/boxes, and you can then reference it locally with vagrant as ubuntu-lucid-32 or whatever you decided to call it.

Project Setup

I have both my rb2.0 WordPress theme and my vagrant configuration in a repository laid out like this:

-repeatingbeats.com/
  -rb2.0/
    -comments.php
    -footer.php
    -functions.php
    -header.php
    and so forth ...
  -vagrant/
    -cookbooks/

The first step is to initialize the vagrant folder:

$ cd /path/to/repeatingbeats.com/vagrant
$ vagrant init

This creates a default Vagrantfile that is helpfully commented with various configuration options. The Vagrantfile uses Ruby code to configure the virtual machine. Here’s what my Vagrantfile looks like for the WordPress project:

:::ruby
Vagrant::Config.run do | config |

  # Use our base ubuntu box
  config.vm.box = "ubuntu-lucid-32"

  # Forward port 8080 on the host to port 80 on the VM
  config.vm.forward_port "http", 80, 8080

  # Map project source folder to VM
  config.vm.share_folder("repeatingbeats.com",
                         "/var/www/wordpress/wp-content/themes/rb2.0",
                         "../rb2.0")

  # Provision with chef solo
  config.vm.provision :chef_solo do | chef |
    chef.cookbooks_path = "cookbooks"
    chef.add_recipe("repeatingbeats.com")
  end

end

This configures the VM to use the ubuntu base box, sets up port forwarding so I’ll be able to view the WordPress site from a browser on the host machine, shares my WordPress theme’s source folder with the VM, and configures Chef Solo to manage the server provisioning.

Chef, Cookbooks, and the Metaphor Really Just Keeps Going

Chef is a server configuration management tool. Server configuration (e.g. packages, services, directories, users, etc.) is described in code. It would take a full blog post to describe the small subset of Chef that I have thus far uncovered, so just head over to the Chef Wiki if you want to learn more.

A closer look at the vagrant folder:

-repeatingbeats.com/
  -vagrant/
    -Vagrantfile
    -cookbooks/
      -apache2/
      -mysql/
      -openssl/
      -php/
      -repeatingbeats.com/
      -wordpress/

Uhoh. This is starting to look like a lot of work! Each cookbook directory contains instructions (ahem … “recipes”) for configuring that aspect of the server. Luckily, most of the work was already done for me and I merely had to fork the Opscode cookbook repository to get everything except the custom repeatingbeats.com cookbook.

The repeatingbeats.com cookbook ties it all together. It’s only a single file: cookbooks/repeatingbeats.com/recipes/default.rb

:::ruby
# Optional packages that I like to have installed
%w{vim man-db git-core}.each do | pkg |
  package pkg
end

# Run apt-get update before the chef convergence stage
r = execute "apt-get update" do
  user "root"
  command "apt-get update"
  action :nothing
end
r.run_action(:run)

include_recipe "wordpress"

The wordpress recipe does the heavy lifting. It installs apache2, mysql, and php, and it drops a WordPress install at a default location of /var/www/wordpress on the VM. Conveniently, my shared theme source folder lands in exactly the right spot: /var/www/wordpress/wp-content/themes/rb2.0.

Hello WordPress (Can I See My Site Already?)

Everything is now ready to go. Let’s start the VM:

$ cd /path/to/repeatingbeats.com/vagrant
$ vagrant up

Don’t hold your breath. The startup and provisioning can take some time. Once it’s done, you can ssh into the new VM to poke around with a simple:

$ vagrant ssh

If you’re playing along at home, you should be able to find the WordPress folder in /var/www/wordpress, and you can ps to your heart’s content to verify that Apache and MySQL are up and running.

I Meant In a Browser

Ok! Open up your favorite browser and navigate to the WordPress install page at http://localhost:8080/wordpress/wp-admin/install.php. You should be able to quickly finish the install, configure the admin user, and get logged in to the admin page. Click ‘Appearance’ in the left navigation column, and your fancy theme should be listed as one of the available themes. Click ‘Activate’, then go to the home page at http://localhost:8080/wordpress. Here’s what I see:

But Didn’t This Seem Like a Lot of Work?

Admittedly, this probably didn’t save much time in terms of initial environment setup. I probably could have cajoled the MacBook into running WordPress with a comparable level of effort.

However, what about the next time I want to tweak the site? All I need to do is clone my repository and run vagrant up. I’m right back into my development environment with no effort at all.

If I want to blow it up and start over, I just write a new Chef recipe. If I’m on another machine, I can get right into the same environment just as easily once I’ve obtained the minimal dependencies for Vagrant. No need to mess with Apache and MySQL just to test some changes to the footer.

As an added bonus, I got to do this on my MacBook:

$ port uninstall php5

Blog 2.0 - What I Did and How I Did It

2011-01-29T00:00:00Z

It is an indisputable fact [1] that there are 152 million blogs on the internet. Even if one assumes that 95% of these blogs are abandoned, that still leaves a healthy 7.6 million active bloggy destinations.

There is always someone with better ideas. There is always someone with more correct instructions on how to get MySQL to do what you want it to do. There is always someone more entertaining. In fact, there is probably even someone so good at footnotes that it makes you want to avoid using them out of sheer inadequacy [2].

So why blog? Why now, after intensely focusing on not blogging for about as long as this blog has existed?

I don’t do much writing anymore. I certainly do plenty of typing, some of which actually involves communicating by stringing together combinations of English words in a passably intelligible fashion. But the majority of this communication occurs in short IRC/AIM bursts, and actually writing demands practice that I haven’t been getting.

My loose goal for Blog 2.0 is to write frequently and to generally try to keep to the theme of “What I Did and How I Did It” [3]. While Stack Overflow is a great resource for targeted programming questions, I’ve found random blogs to be the most reliable source of “How To” manuals for various tasks and problems. Code documentation is almost invariably written by people too close to the actual code [4], and sometimes the thrilling blogged account of how someone actually got the darn thing to work is essential. Thus, there is a small possibility that accounts of how I got something to work will prove useful. But that is not the only motivation.

It can be easy to gloss over small things that aren’t completely understood when there are no obvious negative side effects. That line of code you don’t really grok might just remain ungrokked. I generally walk away with a much greater understanding of how something works after I’ve been forced to explain it to someone else. Writing about What I Did and How I Did It promotes that higher level of understanding. As an added benefit, the internet is crawling with people who are dying to tell you why you are wrong. I’m eager to know why I’m wrong [5].

Naturally, my desire to actually legitimately try to blog was blocked by my compelling need to update this bit-rotting site’s theme. So that’s What I Did.

(psst … you should follow me on Twitter) [6]

footnotes

[1] Where we define “fact” as “a statistic published within the last month that was in the first page of google search results” (see here)

[2] Luckily, that has never stopped me from doing anything.

[3] This carries an implicit goal of “DO THINGS”.

[4] Blogging also appears to be a good outlet for expressing correct opinions.

[5] To clarify, I’m generally only interested in knowing when I’m technically wrong. My opinions tend to be indisputably correct.

[6] I feel less self-conscious about self-promotion when I do things like provide footnotes detailing my general feelings of self-consciousness about self-promotion.