I've spent four years working with Puppet. It's earliest incarnation was good, somewhat rough on the edges but simpler than cfengine and touted the Open Source badge. Proprietary solutions were never an option as companies today squeeze every last penny and are not willing to invest in infrastructure unless there's no way around it. As with any Open Source project, funding was an issue. Building a Shiny New Toy takes money, and PuppetLabs is no exception. The Enterprise version (licensed) of Puppet offers nice features and support, but with today's economy, it's a challenge for the engineers to convince management to cough up the dough. Then there's the learning curve. Today, Puppet has evolved into a mammoth. Training the infrastructure support team on the language (a ruby knock-off), besides the complexity of certificates, certificate authority, revocation lists, as well as the multitude of addon's that have been included, such as mcollective, passenger, unicorn, hiera, storedconfigs, has made the deployment of Puppet complex. And when we take a deeper look at Puppets lack of good high-availability and scale-ability, it's enough to make you doubt it's simplicity.
Rule 1 in systems administration; KISS (Keep It Simple Stupid). Early versions of Puppet were a challenge with their requirements on Ruby, the interpreter language Puppet was written in. Trying to find a package for the OS Distribution you were running was often difficult, and most of the time, if there was a package available, it was sorely out of date. This usually meant pulling in packages from other sources, just to run puppet, just so you could simplify your infrastructure administration. I remember the craze that occurred when Perl hit the mainstream. Perl, in its earliest incarnation was simple, and then fractured. How many times have you struggled trying to get some new Perl program to work, spending all your time on CPAN trying to get the latest-and-greatest support packages just to run the stupid thing?
Rule 2 in systems administration; Make It Fast. I have the highest regard for the Fathers of Unix. Ritchie, Hall, Kernnigan, and yes, even Stallman, (my appologies to the hundreds of others) built simple basic utilities in C that could be used, in concert, to administrate the system. You will not get any faster performance in writing a tool in anything other than C. It goes against my better judgement to build core tools in an interpreter. What I see as a significant advantage for using a compiled language is that any support libraries can be compiled statically with the tool. This is not always the case, just look at the reference to libc. But at least we all know that if libc is going to get updated, the impact of those changes is system wide. With an interpreted language, some obscure little package, that may only be used by the tool, may get updated, and you'll only find out about the breakage when the tool is run. Sure, I've seen the same happen with compiled tools, but not to the same frequency as when using an interpreter. Interpreters are fine, and I hold no bias on them, and it's a joy to write a tool in just 100 lines of interpreter code versus 1000 lines of C code. But when that interpreter code reaches 10000 lines, it becomes messy and slow.
Rule 3 in systems administration; Make it resilient. I've worn many hats during my career. That of administrator, developer, support, repair technician, hardware and software instructor. I find nothing more frustrating than when I'm actually trying to use a computer, it fails. It's the nature of the beast. Shit happens, components overheat and fry, static zaps, cables get cut, poorly written programs will crash, systems will get overloaded, people make mistakes. We can mitigate some of those issues. We can configure redundant hardware to alleviate component failures. We can set resource limits to restrict what poorly written programs will do. We can try to automate tasks and procedures to minimize human error (which is what we want to use Puppet for). And for overloaded systems, we can design balanced and scale-able solutions. This is one area where puppet performs poorly. The puppetmaster certificate authority becomes a single point of failure. It's possible to scale puppetmasters that are not CA's. Even PuppetLabs, after many years, has admitted they do not have a solution. There's been many hacks to try to work around the problem; only resulting in more complexity (See Rule #1).
I enjoy working with Puppet. It's early versions were simple with moderate complexity. But, as with many open source projects than run out of control, it's complexity has become a detriment. I remember a term used by a past manager; "Feature Creep". This has happened throughout Unix and Linux history. Early versions of sed had fewer options than the current rendition. But those additional options (or features) did not require the installation of multitudes of packages (bar any requirements on core system libraries, i.e. libc). On the flip-side, the complexity of a solution is proportional to the number of problems solved. In the early days of Unix, we'd write many little tools (scripts), each would address specific problems. We could then call these scripts from higher level programs or scripts. A very common tool used to remotely invoke these scripts was ssh, well - er, in the early days it was rsh. Why not go back to our simplistic roots? Simplicity is the key to reliability and maintainability.
A very promising Puppet alternative that gets us "back-to-the-farm" is called Ansible. I like the fact that it uses ssh for the method of remote management. Any Unix/Linux administrator worth his weight already knows how to work with ssh, as well do developers. I like the fact that the tasks are written in plain English. What impresses me most, we don't have to install extra baggage on the nodes to manage. We configure ssh keys between a control node and the managed nodes. Most shops already do that as part of their node provisioning (at least we do at my current gig). As for HA, we configure a pair of control nodes, with ssh public keys exchanged; just plain simple. With all the work and cost I see of people trying to build whiz-bang GUI tools to manage Puppet and it's shortcomings, missed deadlines and bug fixes, I believe that Ansible has risen to the top of the configuration management stack.
Some of the features that make Ansible so appealing:
Push architecture. Ansible works on the idea of pushing changes, rather than Puppets method of pull. Seeing the way our shop currently runs, developers expect the changes to be applied immediately. Older versions of Puppet supported "puppet kick", which is being deprecated, with the claim that Mcollective will be used to handle remote Puppet Agent invocation, which BTW, isn't complete yet. Don't like the idea of having to layer on more baggage just to trigger a Puppet run.
Parallel execution. Ansible, when executing a playbook, will by default fork 5 threads. This is configurable via the "ansible-playbook playbook.yml -f num-of-threads" option.
Grouping of hosts. Ansible will allow definition of hosts into groups. The Puppet extension Mcollective would allow this, but it's still extra baggage. For example (/etc/ansible/hosts):
[storage]
ss[1:2].tsand.org
[local]
ansible.tsand.org
[clients]
client1.tsand.org
client2.tsand.org
Playbooks. The Playbook, in Puppet terms is referred to as a module/class, is written in plain English, such as:
---
- hosts: clients
tasks:
- name: apt update
action: apt update_cache=yes
- name: apt upgrade
action: apt upgrade=dist
- name: ensure facter is latest
action: apt pkg=facter state=latest
- name: ensure python json is latest
action: apt pkg=python-simplejson state=latest
- name: ensure ruby-json is latest
action: apt pkg=ruby-json state=latest
- name: push root authorized_keys
action: copy src=/root/.ssh/authorized_keys dest=/root/.ssh/authorized_keys
- name: ensure ntpdate is latest
action: apt pkg=ntpdate state=latest
- name: ensure ntp is latest
action: apt pkg=ntp state=latest
This "playbook" (called /root/clients.yml) is invoked with:
ansible-playbook /root/clients.yml
Nice collection of Modules. These are bundled modules to provide basic tasks. There's even support for libvirt, EC2, AWS CloudFormation. See Ansible Modules
Imperative vs Declarative. Puppet is a declarative language, which does not define order. There are "tricks" in Puppet that can be used, such as dependancies and the stage command. Ansible is more like Chef in that the playbooks execute like a typical program defining an "order".
Agentless. Ansible does not require a remote agent on the target nodes. Just an sshd running on the hosts and the public key. And there are options to prompt for ssh password and sudo password if desired. (Yes, Ansible will also allow remote execution via a userID and sudo).
Just Plain Simple. Only took about 30 minutes to get up and running. Installed the ansible software on my "control" host. What took the most time was deciding what to put in my playbooks. ;)
Thanks for letting me rant on this.