January 20-22, 2012, Hilton Los Angeles Airport Hotel

Automated EC2 AMI generation with Puppet

Carl Caum


Puppet has dramatically helped solve the issues of coordination and automation of configuration management while Puppet's cloud provisioning capabilities aid IT groups in areas of automated scaling. The most common approach for getting Puppet running in AWS, for example, is to have a base AMI that runs puppet when launched. Then, Puppet gets the instance to the role it's assigned. However, this is not always a fast process. Depending on many factors, it may take upwards of several minutes for Puppet to get all the configurations in place for the instance to fulfill its assigned roles. If your auto scaler (or you manually) fired up several hundred or several thousand instances, time is of the essence. Utilizing revision control system hooks, Puppet's cloud provisioning, and a custom Puppet report, your AMIs can be ensured to be always as up to date as possible. This dramatically will lower the time to scale ensuring you're ready for whatever may come. The entire premise is based on the notion that your Puppet code is what's authoritative in your network. Your build system may know the latest deployable version of your applications, but it can't know what's required for a system to actually deploy the application. If you update your Puppet code's requirements to deploy your application, however, it directly pertains to what a system should look like. Therefor, it's your Puppet code or meta data that should trigger AMI rebuilds. When code is committed to git/svn, a script will bring up instances of each AMI we want to manage. Puppet runs on the instances to their assigned roles. If changes occurred, and the were all successful, then the image needs to be updated.