Coffee & Beer

Rantings and Ravings of the technical sort

Finding Compute Resources With Puppet Facts

One question we get asked at wrk a good bit centers around finding compute resources that match a specific set of specifications.Since the compute system we haev are of all sort of various ages and hardware types, jobs that need 8GB of memory might not work everywhere, but most places.Same goes for a specific number of cores (or, minimal #), per node. Some have 4, some have 12+.

Well, facter, and therefore puppet, knows about these things. And there is a REST api we can query. And I needed an excuse to do a little ruby:

(find_compute.rb) download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
#!/usr/bin/env ruby

require 'yaml'
require 'puppet'

puts "Welcome to the Compute Finder script!"
puts "This script aims to help you locate compute nodes based on simple requirements"
puts "Such as the minimum amount of RAM or # of processor cores"
puts "---"
puts "---"
puts "What minimum amount of ram would you like?"
puts "(in GB, leave blank for no minimum)"
mem=gets.chomp
puts "What minimum number of processor cores would you like?"
puts "(leave blank for no minimum)"
procs=gets.chomp

sleep 1

search ="facts.compute_node=true"
if mem != ""
  search << "&facts.memorysize.ge=#{mem}"
end

if procs !=""
  search << "&facts.processorcount.ge=#{procs}"
end

if (procs == "" and mem == "" )
  puts "You must specify some requirments!"
  exit
end
puppet_base = "https://puppet:8140"
path="/production/facts_search/search?"


puts "Finding matching nodes, please wait..."
cmd = "curl -s -k -H \"Accept: yaml\" \"#{puppet_base}#{path}#{search}\""
ans = %x[#{cmd}]
nodes = YAML::load(ans)
nodearr = []
nodes.sort.each do |node|
  nodearr.push(node)
end
puts nodearr
puts "Would you like more info on the nodes found? This may take bit longer..."
puts "(y/n)"
more=gets.chomp
if more == "y"

  node_array = []
  nodes.each do |node|
      node_url="https://puppet:8140/production/facts/#{node}"
      node_cmd="curl -s -k -H \"Accept: yaml\" #{node_url}"
      node_ans= %x[#{node_cmd}]
      node_array << YAML::load(node_ans)
  end

  results =[]
  node_array.each do |node|
      name = node.name
      facts = node.values
      ram=facts['memorysize']
      procs=facts['processorcount']
      results.push("Host: #{name} RAM: #{ram} ProcessorCores: #{procs}")
  end
  results.sort!
  puts results
end

The output of which looks like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
Welcome to the Compute Finder script!
This script aims to help you locate compute nodes based on simple requirements
Such as the minimum amount of RAM or # of processor cores
---
---
What minimum amount of ram would you like?
(in GB, leave blank for no minimum)
64
What minimum number of processor cores would you like?
(leave blank for no minimum)
12
Finding matching nodes, please wait...
node01.local
node02.local
node03.local
node04.local
Would you like more info on the nodes found? This may take bit longer...
(y/n)
y
Host: node01.local RAM: 94.28 GB ProcessorCores: 12
Host: node02.local RAM: 94.28 GB ProcessorCores: 12
Host: node03.local RAM: 86.39 GB ProcessorCores: 12
Host: node04.local RAM: 251.89 GB ProcessorCores: 32

Comments