MCC-320723/bootstrap without chef

.gitignore

@@ -20,3 +20,4 @@ tmp
 # editors
 [#]*[#]
 .\#*
+.idea

ERB_VARS.md

@@ -19,6 +19,14 @@ The Chef-based examples in the README illustrate the included Chef support.
 * `:chef_attributes` - a Hash of Chef attributes that is read as `node[chef_client][config]` by the [Opscode chef-client cookbook][1]. This is where you specify your `node_name` (if desired), `chef_server_url`, `validation_client_name`, etc. *Always use strings for chef attribute keys, not symbols!*
 
 
+Ansible settings
+----------------
+The Ansible-based examples in the README illustrate the included Ansible support.
+* `:tower_url` - When set to any non-nil value, enables the built-in Ansible support. The value is the URL at which Ansible Tower listens for requests to run playbooks.
+* `:tower_post_data_script` - The path to the script that will generate the body of the POST request made to the tower_url.
+* `:tower_post_data_file` - The path to the file containing the POST request made to the tower_url.
+
+
 RAMdisk settings
 ----------------
 * `:create_ramdisk` - Setting this to `true` generates a bootscript that creates a RAMdisk of a configurable size and at a configurable filesystem location. This happens even before the archive is unpacked, so you can extract files into the RAMdisk.

README.md

@@ -22,6 +22,7 @@ Why is it?
 ----------------
 * makes specification of complex, cross-platform boot data simple and portable.
 * simplifies initial Chef setup
+* enables an initial request to a Url for bootstrapping (as for Ansible)
 
 ----------------
 Where is it? (Installation)
@@ -34,7 +35,7 @@ Install the gem and its dependencies from RubyGems:
 ----------------
 How is it [done]? (Usage)
 ----------------
-Call the gem's main public method: `Bootscript.generate()`. It accepts a Hash of template variables as its first argument, which is passed directly to any ERB  template files as they render. All the data in the Hash is available to the templates, but some of the key-value pairs also control the gem's rendering behavior, as demonstrated in the following examples. (There's also a [list of such variables](ERB_VARS.md).)
+Call the gem's main public method: `Bootscript.generate()`. It accepts a Hash of template variables as its first argument, which is passed directly to any ERB template files as they render. All the data in the Hash is available to the templates, but some of the key-value pairs also control the gem's rendering behavior, as demonstrated in the following examples. (There's also a [list of such variables](ERB_VARS.md).)
 
 
 ### Simplest - make a RAMdisk
@@ -140,12 +141,47 @@ Finally, generate *without* an explicit node name, but filling in the other valu
     }, data_map)
 
 
+### Ansible support
+
+The software's Ansible support is triggered when you pass a `:tower_url` option to the `generate()` method.
+A request will be made to the specified url, sending data via POST. The data to send may require information that
+is only available on the bootstrapped machine. The data to be POSTed is created by a script executed on the
+bootstrapping node, immediately prior to making the POST request to `:tower_url`. The path to this script is
+specified in the call to `generate()`:
+
+    script = Bootscript.generate(
+      logger:                 Logger.new(STDOUT),           # Monitor progress
+      tower_url:              'https://tower-host.foo.com', # Obtain from Ansible Tower administrators
+      tower_post_data_script: '/path/to/your/script',
+      tower_post_data_file:   '/path/to/the/file',
+    )
+
+The script specified by `:tower_post_data_script` is expected to create the file specified by `:tower_post_data_file`.
+If the bootstrap call to `:tower_url` succeeds (i.e., the call to curl returns a zero exit status), both the data
+file `:tower_post_data_file` and script `:tower_post_data_script` will be deleted from the filesystem.
+
+### Failure (inconceivable!)
+
+The `generate()` method accepts an option, `:startup_failed_command`, which is a string. If the startup command
+exits with a non-zero status, the command configured with this option will be executed. One could, for example,
+provide a template for a script that marks an EC2 instance as unhealthy using the AWS CLI, and then invoke that
+script by supplying the path to the script as the value of `:startup_failed_command`.
+
+The default value for `:startup_failed_command` is an empty string, so the default is to take no action on failure.
+
+### Using Chef and Ansible together
+
+The code is written so that if `generate()` receives both `:chef_validation_pem` and `:tower_url` parameters,
+installers for both Chef and Ansible will be created and executed. The installer for Chef will be executed first. Either
+of these is complicated enough, so using just one seems wise. That said, one may need to switch from one to the other,
+and having both available may assist in that transition.
+
 ----------------
 *Known Limitations / Bugs*
 ----------------
-* bash and tar are required on Unix boot targets
+* bash, curl, and tar are required on Unix boot targets
 * Powershell is required on Windows boot targets
-* bash, tar and uudecode are required to run the tests
+* bash, tar, and uudecode are required to run the tests
 
 
 ----------------

lib/bootscript.rb

@@ -3,24 +3,30 @@
 require 'bootscript/script'
 require 'bootscript/uu_writer'
 require 'bootscript/chef'
+require 'bootscript/ansible'
 
 # provides the software's only public method, generate()
 module Bootscript
 
   # These values are interpolated into all templates, and can be overridden
   # in calls to {Bootscript#generate}
   DEFAULT_VARS = {
-    platform:       :unix,          # or :windows
-    create_ramdisk: false,
+    platform:        :unix,         # or :windows
+    create_ramdisk:  false,
     startup_command: '',  # customized by platform if chef used
-    ramdisk_mount:  '',   # customized by platform, see platform_defaults
-    ramdisk_size:   20,   # Megabytes
+    ramdisk_mount:   '',  # customized by platform, see platform_defaults
+    ramdisk_size:    20,  # Megabytes
     add_script_tags: false,
-    script_name:    'bootscript', # base name of the boot script
-    strip_comments: true,
-    imdisk_url:     'http://www.ltr-data.se/files/imdiskinst.exe',
-    update_os:      false,
-    inst_pkgs:     'bash openssl openssh-server' #list of packages you want upgraded
+    script_name:     'bootscript',  # base name of the boot script
+    strip_comments:  true,
+    imdisk_url:      'http://www.ltr-data.se/files/imdiskinst.exe',
+    update_os:       false,
+    inst_pkgs:       'bash openssl openssh-server', #list of packages you want upgraded
+    curl_options:    '',
+    bash_options:    '',
+    use_chef:        false,
+    use_ansible:     false,
+    package_update:  true
   }
 
   # Generates the full text of a boot script based on the supplied
@@ -53,28 +59,54 @@ def self.default_logger(output = nil, level = Logger::FATAL)
     logger
   end
 
+  def self.powershell_cmd(script_path, script_log)
+    %Q{PowerShell -Command "& {#{script_path}}" > #{script_log} 2>&1}
+  end
+
   # Returns the passed Hash of template vars, merged over a set of
   # computed, platform-specific default variables
   def self.merge_platform_defaults(vars)
     defaults = DEFAULT_VARS.merge(vars)
+    defaults[:use_chef] = Chef::included?(defaults)
+    defaults[:use_ansible] = Ansible::included?(defaults)
+    startup_cmd = []
     if defaults[:platform].to_s == 'windows'
       defaults[:ramdisk_mount]      = 'R:'
       defaults[:script_name]        = 'bootscript.ps1'
-      if Chef::included?(defaults)
-        defaults[:startup_command]  = 'PowerShell -Command "& '+
-          '{C:/chef/chef-install.ps1}" > c:/chef/bootscript_setup.log 2>&1'
+      if defaults[:use_ansible]
+        startup_cmd << powershell_cmd(
+            'C:/ansible/ansible-install.ps1',
+            'C:/ansible/bootscript_setup.log')
+        defaults[:tower_post_data_file] = 'C:/ansible/ansible-post-data.txt'
+      end
+      if defaults[:use_chef]
+        startup_cmd << powershell_cmd(
+            'C:/chef/chef-install.ps1',
+            'C:/chef/bootscript_setup.log')
       end
-    else
+      if startup_cmd.size > 0
+        defaults[:startup_command]  = startup_cmd.join(' ; ')
+      end
+    else # unix! the only way it should be...
       defaults[:ramdisk_mount]      = '/etc/secrets'
       defaults[:script_name]        = 'bootscript.sh'
-      if Chef::included?(defaults)
-        defaults[:startup_command]  = '/usr/local/sbin/chef-install.sh'
+      if defaults[:use_chef]
+        startup_cmd << 'chef-install.sh'
+      end
+      if defaults[:use_ansible]
+        startup_cmd << 'ansible-install.sh'
+        defaults[:tower_post_data_file] = '/tmp/ansible-post-data'
+      end
+      if startup_cmd.size > 0
+        defaults[:startup_command]  = startup_cmd.join(' && ')
       end
     end
+    defaults[:startup_failed_command] ||= ''
+    defaults[:startup_command] ||= 'echo "No bootstrap requested"'
     defaults.merge(vars)  # return user vars merged over platform defaults
   end
 
-  BUILTIN_TEMPLATE_DIR  = File.dirname(__FILE__)+"/templates"
+  BUILTIN_TEMPLATE_DIR  = "#{File.dirname(__FILE__)}/templates"
   UNIX_TEMPLATE         = "#{BUILTIN_TEMPLATE_DIR}/bootscript.sh.erb"
   WINDOWS_TEMPLATE      = "#{BUILTIN_TEMPLATE_DIR}/bootscript.ps1.erb"
 

lib/bootscript/ansible.rb

@@ -0,0 +1,51 @@
+module Bootscript
+  # provides built-in Ansible templates and attributes
+  module Ansible
+
+    # returns a map of the built-in Ansible templates, in the context of erb_vars
+    # The presence of :tower_url triggers the inclusion of Ansible
+    def self.files(erb_vars)
+      if Bootscript.windows?(erb_vars)
+        files_for_windows(erb_vars)
+      else
+        files_for_unix(erb_vars)
+      end
+    end
+
+    # defines whether or not Ansible support will be included in the boot script,
+    # based on the presence of a certain key or keys in erb_vars
+    # @param [Hash] erb_vars template vars to use for determining Ansible inclusion
+    # @return [Boolean] true if erb_vars has the key :tower_url
+    def self.included?(erb_vars = {})
+      erb_vars.has_key? :tower_url
+    end
+
+    private
+
+    # Callers will almost certainly replace the ansible-post-data-script.{sh,ps1}.erb
+    # template with their own. But they don't have to. We supply here a default
+    # script which simply creates an empty body for the POST request to the Ansible
+    # Tower url.
+
+    def self.files_for_unix(erb_vars)
+      template_dir = "#{Bootscript::BUILTIN_TEMPLATE_DIR}/ansible"
+      { # built-in files
+        '/usr/local/sbin/ansible-install.sh' =>
+            File.new("#{template_dir}/ansible-install.sh.erb"),
+        '/usr/local/sbin/ansible-post-data-script.sh' =>
+            File.new("#{template_dir}/ansible-post-data-script.sh.erb"),
+      }
+    end
+
+    def self.files_for_windows(erb_vars)
+      template_dir = "#{Bootscript::BUILTIN_TEMPLATE_DIR}/ansible"
+      { # built-in files
+        'ansible/ansible-install.ps1' =>
+            File.new("#{template_dir}/ansible-install.ps1.erb"),
+        'ansible/ansible-post-data-script.ps1' =>
+            File.new("#{template_dir}/ansible-post-data-script.ps1.erb"),
+      }
+    end
+
+  end
+end

lib/bootscript/script.rb

@@ -123,13 +123,13 @@ def write_windows_archive(destination)
     # Archive::Tar::Minitar::Writer (if unix), or a Zip::OutputStream (windows)
     def render_data_map_into(archive)
       full_data_map.each do |remote_path, item|
-        if item.is_a? String            # case 1: data item is a String
+        if item.is_a?(String)           # case 1: data item is a String
           @log.debug "Rendering ERB data (#{item[0..16]}...) into archive"
           data  = render_erb_text(item)
           input = StringIO.open(data, 'r')
           size  = data.bytes.count
         elsif item.is_a?(File)          # case 2: data item is an ERB file
-          if item.path.upcase.sub(/\A.*\./,'') == 'ERB'
+          if item.path.upcase.end_with?('.ERB')
             @log.debug "Rendering ERB file #{item.path} into archive"
             data  = render_erb_text(item.read)
             input = StringIO.open(data, 'r')
@@ -154,10 +154,11 @@ def render_data_map_into(archive)
       end
     end
 
-    # merges the @data_map with the Chef built-ins, as-needed
+    # merges the @data_map with the Chef and/or Ansible built-ins, as-needed
     def full_data_map
-      Bootscript::Chef.included?(@vars) ?
-        @data_map.merge(Bootscript::Chef.files(@vars)) : @data_map
+      ansible_vars = Ansible.included?(@vars) ? Ansible.files(@vars) : {}
+      chef_vars = Chef.included?(@vars) ? Chef.files(@vars) : {}
+      @data_map.merge(ansible_vars).merge(chef_vars) # Chef wins collisions.
     end
 
     # renders erb_text, using @vars
@@ -170,8 +171,8 @@ def render_erb_text(erb_text)
     def strip_shell_comments(text)
       lines = text.lines.to_a
       return text if lines.count < 2
-      lines.first + lines[1..lines.count].
-        reject{|l| (l =~ /^\s*#/) || (l =~ /^\s+$/)}.join('')
+      lines.first + lines.drop(1).
+        reject { |l| (l =~ /^\s*#/) || (l =~ /^\s+$/) }.join('')
     end
   end
 

lib/templates/ansible/ansible-install.ps1.erb

@@ -0,0 +1,32 @@
+echo "Starting Ansible installation..."
+
+$AnsiblePath = "C:\Ansible"
+
+function main()
+{
+  try
+  {
+    Create-Ansible-Directory-Structure
+    Call-Ansible-Tower
+  }
+  catch {
+    write-error $error[0]
+    exit 1
+  }
+}
+
+function Create-Ansible-Directory-Structure()
+{
+  New-Item -force -type directory -Path $AnsiblePath | out-null
+}
+
+function Call-Ansible-Tower()
+{
+  echo "Calling Ansible Tower..."
+  $wc = new-object System.Net.WebClient
+  $pathToFile = Resolve-Path <%= tower_post_data_file %>
+  $bootRequest = Get-Content $pathToFile -Raw
+  $wc.UploadString("<%= tower_url %>", $bootRequest)
+}
+
+main

lib/templates/ansible/ansible-install.sh.erb

@@ -0,0 +1,51 @@
+#!/usr/bin/env bash <%= bash_options %>
+# Calls out to Ansible Tower to request bootstrapping actions.
+set -e
+test $UID == 0 || (echo "Error: must run as root"; exit 1)
+echo "Calling Ansible Tower with ${0}..."
+
+######### STEP 0: CONFIG, rendered by bootscript gem
+TOWER_URL="<%= tower_url %>"
+
+######### STEP 1: CONFIGURE OPERATING SYSTEM AND INSTALL DEPENDENCIES
+<% if package_update %>
+echo "Performing package update..."
+if [ "$DistroBasedOn" == 'debian' ] ; then
+  export DEBIAN_FRONTEND=noninteractive
+  apt-get update -y
+  echo "Installing build-essential and curl..."
+  apt-get --force-yes -y install build-essential curl
+else
+  echo "Skipping..."
+fi
+<% else %>
+echo "Initial package update not configured; skipping..."
+<% end %>
+
+######### STEP 2: SIGNAL TO TOWER
+echo "Call out to Ansible Tower to bootstrap us..."
+if which curl > /dev/null; then
+
+  # Run the command to create the file curl will read
+  <%= tower_post_data_script %>
+
+  curl <%= curl_options %>                  \
+      -H "Content-Type: application/json"   \
+      -X POST                               \
+      $TOWER_URL                            \
+      --data-binary "@<%= tower_post_data_file %>"
+
+  exit_status="$?"
+  if [ "${exit_status}" != "0" ]; then
+    echo "The call to Ansible Tower (via curl) exited with status code: ${exit_status}"
+    exit $exit_status
+  else
+    echo "Cleaning up after successful curl request to ${TOWER_URL}"
+    rm -f "<%= tower_post_data_file %>" "<%= tower_post_data_script %>"
+  fi
+else
+  echo "The curl command is not available; unable to signal Tower for bootstrap"
+  exit 1
+fi
+
+echo "Done."

lib/templates/ansible/ansible-post-data-script.ps1.erb

@@ -0,0 +1,3 @@
+echo "Creating data file for POST to Ansible Tower..."
+
+New-Item <%= tower_post_data_file %> -type file -force -value "{}"

lib/templates/ansible/ansible-post-data-script.sh.erb

@@ -0,0 +1,4 @@
+#!/usr/bin/env bash <%= bash_options %>
+# Creates the body of the POST request to Ansible Tower.
+
+echo "{}" > "<%= tower_post_data_file %>"