Being a shop that is mostly hpc, our compute nodes are pretty disposable, so we rebuild them from time to time. We’re coming up on a push to normalize them a bit, and will be looking to rebuild a bunch in big batches. One of the headaches, that isn’t REALLY a headache, is cleaning up the puppet certs when a system is rebuilt. We autosign puppet certs, so the new ones will come in just fine, but you’ve got to remember to clean the old ones during/before the rebuild. Add storedconfigs to this, and salt minion keys, and there is a good bit of cleanup to get done during a rebuild.
So, first, I wrapped the 3 things we want to clean up, in a script:
1 2 3 4 5 6 7 8 9 10 11 12 13 14
So, pretty obviously, that cleans the puppet cert, the storedconfigs db entry, and the salt key (puppet master = salt master)
Okay, so, one stop shopping there, but I want this automatic. Wel, we use Cobbler to build systems/define kickstarts, and one of the last things in all of our kickstarts is:
Which lets cobbler know the build is done. This can optioanlly trigger scripts in
/var/lib/cobbler/triggers/install/post, so, I added one:
1 2 3 4 5 6 7 8 9 10 11 12
So, its passed 3 arguments: The object type (system), the system name, and the IP. I take the name, and query out forman api for the fqdn (we have a few domains so I can’t assume hostname.my.domain.com), and the call teh script above to clean out everything for that host!
So, when it comes to puppet/salt certs, we don’t care now. New system are automatically accepted, and if you rebuild, the old ones are removed and new ones accepted, just like that!