Automating The Creation And Installation Of Execution Environments For The Ansible Automation Platform
First things first, here’s my blog post that gives you a step by step of creating, preparing, and using custom Execution Environments. This article picks up from there and tries to automate as much of the process as is possible using the Ansible Automation Platform(AAP). It’s a little meta; I’m using AAP to build the Execution Environments(EEs) that are going to be used by AAP.
Video Demo
Alas, I’ve made adjustments to the playbook already, so there are slight differences.
Playbook
The link to the newest updated version of the playbook is right here in my github.
This is the complete playbook at the time of this writing(again, check github for the newest version). Below will be the playbook broken into pieces and explained.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 | ---
- name: Build/Push/Install Custom EEs
hosts: aap.gregsowell.com
gather_facts: false
vars:
# aap username/pass
controller_user: "{{ gen2_user }}"
controller_pass: "{{ gen2_pword }}"
# # what is the base EE that is used for building my custom EE
# base_ee: registry.redhat.io/ansible-automation-platform-21/ee-supported-rhel8
# what is the name of the new EE being built
new_ee: azure_ee_supported
# what is the path where the EE build files are stored
path_ee: /root/ansible-execution-environments
# base folder for cloning EE
base_ee_path: /root
# private automation hub host
pah_host: pah.gregsowell.com
# ee repo
ee_repo: https://github.com/gregsowell/ansible-execution-environments
# private automation hub host
pah_host: pah.gregsowell.com
# credential name used in AAP
pah_cred: Automation Hub Container Registry
# # new version number for the pushed ee
# new_ee_ver: 1.0.2
# pah username/pass
pah_user: "{{ gen1_user }}"
pah_pass: "{{ gen1_pword }}" |
I setup a username and password for connecting via the AAP API; I’m passing in custom credentials at run-time(a benefit of using custom credentials is that if I specify a field as password, then no_log is set for that variable and will be obfuscated in output).
You may notice that I have base_ee commented out. This is the base container I’m using to build the EE with. I’ve got it commented out because I’m pulling that directly from the EE configuration files, so it’s not necessary here(I left it if you want to change things up.
There’s also setup for the build files repo; I’ve set it up so that all of my kit is stored in Source Control Management!
You’ll also notice that I’m setting up info for using Red Hat’s Private Automation Hub(PAH or pah). I’m using this as my container registry and will be pushing my custom EEs here.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | tasks:
- name: Git clone the EE repo to ensure I have the newest files
ansible.builtin.shell: "git clone {{ ee_repo }}"
args:
chdir: "{{ base_ee_path }}"
- name: Grab the execution-environment.yml file to use as variables
ansible.builtin.fetch:
src: "{{ path_ee }}/{{ new_ee }}/execution-environment.yml"
dest: execution-environment.yml
register: ee_yml
- name: Include the EE file as variables
include_vars:
file: "{{ ee_yml.dest }}"
name: inc_vars
- name: Setup some base variables
ansible.builtin.set_fact:
base_ee: "{{ inc_vars.build_arg_defaults.EE_BASE_IMAGE }}"
new_ee_ver: "{{ inc_vars.version }}" |
This beginning is all about reading the EE config files and grabbing some of the info. First things first, I connect to my git repo and clone down the newst build files. Next reach out to the remote server I’m using to build EEs at and pulling that file locally into my current EE for processing. Notice I’m registering the results to a variable; this is because the path that containers use are pretty bonkers, and it’s easier to just reference it this way. For this example, the path is “/runner/project/execution-environment.yml/aap.gregsowell.com/root/ee/azure_ee_supported/execution-environment.yml”.
I next include the execution-environment.yml file as variables(it is a simply YAML formatted file after all).
Last here I grab the interesting variables(base image and version) and assign them.
1 2 3 4 5 6 7 8 9 | - name: Grab the list of EEs to ensure the correct EE exists
ansible.builtin.shell: podman image list
changed_when: false
register: ee_list
- name: Install the required base EE if not already present
when: ee_list.stdout_lines is not search(base_ee)
ansible.builtin.debug:
msg: "Install the missing base EE here" |
Here I’m issuing a “podman image list” command to view what EEs exist on this server and registering it to a variable. Next I do a quick check to see if that base EE I’m building from exists on the server; if not it will pull this EE(I’ll be adding that in version 2 of the script).
1 2 3 4 5 6 7 8 9 | - name: Begin the build process on the EE
when: ee_list.stdout_lines is search(base_ee)
block:
- name: Run the build on the selected new EE
ansible.builtin.shell: "ansible-builder build --tag {{ new_ee }}"
args:
chdir: "{{ path_ee }}/{{new_ee}}"
register: ee_build_out
failed_when: ee_build_out.stdout_lines is not search('Complete!') |
On my AAP server I have a directory named “ee”. Inside of it I keep a directory for each custom EE I’m building. So for example I have /root/ee/azure_ee_supported/. This is where I’m building, you guessed it, my Azure execution environment. Inside of this folder is all of my ansible-builder files(and ultimately where the context folder will be placed by builder). I’m using the shell module here to fire off the build command, but I have to first add the args option to specify which folder to execute the command in. I’m also adding in a failed_when condition so that if the command doesn’t return “Complete!”, it will fail the task. This is the standard build.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 | - name: search containing folder to determine if there are any crt files(used for self-signed pah server)
ansible.builtin.find:
paths: "{{ path_ee }}/{{ new_ee }}"
patterns: '*.crt'
register: crt_files
- name: Begin the build process on the EE when certs do need to be installed
when: ee_list.stdout_lines is search(base_ee) and crt_files.matched > 0
# when: ee_list.stdout_lines is search(inc_vars.build_arg_defaults.EE_BASE_IMAGE)
block:
- name: Run the create on the selected new EE - **install certs process**
ansible.builtin.shell: "ansible-builder create"
args:
chdir: "{{ path_ee }}/{{new_ee}}"
register: ee_build_out
failed_when: ee_build_out.stdout_lines is not search('Complete!')
- name: Copy cert files to the build folder
ansible.builtin.shell: "cp {{ path_ee }}/{{new_ee}}/*.crt {{ path_ee }}/{{new_ee}}/context/_build"
args:
chdir: "{{ path_ee }}/{{new_ee}}"
- name: Add line to Containerfile to ensure the certificates are copied and applied
ansible.builtin.lineinfile:
path: "{{ path_ee }}/{{new_ee}}/context/Containerfile"
insertbefore: '^RUN ansible-galaxy role install.*'
line: 'RUN cp *.crt /etc/pki/ca-trust/source/anchors && update-ca-trust'
- name: Run the build on the selected new EE - with installed certs
ansible.builtin.shell: "podman build -f context/Containerfile -t {{ new_ee }} context"
args:
chdir: "{{ path_ee }}/{{new_ee}}"
register: ee_build_out |
^^The above section is run when certificates need to be installed to connect to my private automation hub.
The first task checks the EE’s folder to see if there are any certificate files.
I then have a block with a conditional checking to see if there were any cert files, and when it found some the block will run.
I first run the ansible-builder create command which will build out the Containerfile which will be used to create the EE.
I then copy my certs to the _build folder, and then modify the Container file to install the certs.
Last I run the podman build command to create my new EE.
I have a breakdown of what’s happening in the blog post on all of the steps being performed here.
1 2 3 4 5 6 | - name: Push EE to container registry
ansible.builtin.shell: "podman push --creds={{ pah_user }}:{{ pah_pass }} localhost/{{ new_ee }}:latest docker://{{ pah_host }}/{{ new_ee }}:{{ new_ee_ver }} --tls-verify=false"
args:
chdir: "{{ path_ee }}/{{new_ee}}"
register: pah_push_out
failed_when: pah_push_out.stderr_lines is not search('Storing signatures') |
I’m next using the podman command to push the newly created EE to my PAH server.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | - name: Collect credentials
uri:
url: "https://{{ inventory_hostname }}/api/v2/credentials/"
user: "{{ controller_user }}"
password: "{{ controller_pass }}"
method: GET
validate_certs: false
force_basic_auth: true
status_code:
- 200
- 201
- 204
register: aap_cred_out
- name: Set ansible credential name to ID
when: item.name == pah_cred
ansible.builtin.set_fact:
pah_cred_id: "{{ item.id }}"
loop: "{{ aap_cred_out.json.results }}" |
Alright, the first thing I need to do is map the credential name over to the credential ID. I do this by first pulling all of the existing creds and registering them to a variable(aap_cred_out).
I next loop through aap_cred_out looking for the credential name specified in the vars section(pah_cred), then once it finds the cred name in question, it will assign it’s ID to a variable(pah_cred_id).
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | - name: Collect EEs to check if it already exists
uri:
url: "https://{{ inventory_hostname }}/api/v2/execution_environments/"
user: "{{ controller_user }}"
password: "{{ controller_pass }}"
method: GET
validate_certs: false
force_basic_auth: true
status_code:
- 200
- 201
- 204
changed_when: false
register: aap_ee_out
- name: Set ansible ee id if name already exists
when: item.name == new_ee
ansible.builtin.set_fact:
new_ee_id: "{{ item.id }}"
loop: "{{ aap_ee_out.json.results }}" |
Here I’m now going to collect a list of EEs. I’m then looping through them to see if the EE I’m trying to add already exists or not(just in name, not in version).
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 | - name: Configure AAP for new EE
when: new_ee_id is undefined
vars:
hosts: controllers
uri:
url: "https://{{ inventory_hostname }}/api/v2/execution_environments/"
user: "{{ controller_user }}"
password: "{{ controller_pass }}"
method: POST
validate_certs: false
force_basic_auth: true
body_format: json
body: |
{
"name": "{{ new_ee }}",
"image": "{{ pah_host }}/{{ new_ee }}:{{ new_ee_ver }}",
"description": "added by automation",
"pull": "missing",
"credential": "{{ pah_cred_id }}"
}
status_code:
- 200
- 201
- 204
failed_when: aap_conf_out.status != 201 and aap_conf_out.json.name is not search('already exists')
register: aap_conf_out
# - name: Debug aap_conf_out
# ansible.builtin.debug:
# var: aap_conf_out
- name: If EE already exists on AAP, update version number
# when: aap_conf_out.status == 400 and aap_conf_out.json.name is search('already exists')
when: new_ee_id is defined
vars:
hosts: controllers
uri:
url: "https://{{ inventory_hostname }}/api/v2/execution_environments/{{ new_ee_id }}/"
user: "{{ controller_user }}"
password: "{{ controller_pass }}"
method: PUT
validate_certs: false
force_basic_auth: true
body_format: json
body: |
{
"name": "{{ new_ee }}",
"image": "{{ pah_host }}/{{ new_ee }}:{{ new_ee_ver }}",
"description": "added by automation",
"pull": "missing",
"credential": "{{ pah_cred_id }}"
}
status_code:
- 200
- 201
- 204
# failed_when: aap_conf_out.status != 201 and aap_conf_out.json.name is not search('already exists')
register: aap_conf_patch_out |
Now I a add the new EE into my controllers.
First I check if the EE exists, and if not, I simply add it.
If the EE does exist, then I do an update to the existing settings. This way if a new version of the EE has been created, the version number will be updated in the controller.
Just as a note I like to set my EEs to pull if missing. This way it will only pull the EE the one time it needs to(much more efficient). Also note that if I do an update on the EE version number in my PAH and also update the EE version number in my controller, on the next piece of automation that runs using that EE, it will check and see that the stored EE has an older version, so it will connect to the PAH and download the new version. So in short, updating the EE version number in controller will signal that it should pull the new EE from my container registry!
Conclusion
At this point the automation is done and I’m ready to start using the EE with my job templates. This cuts the majority of manual labor out of the process, and makes my life just that much easier.
Again, this is version 1, so I plan to make it more efficient, so watch for updates.
If you have any questions or comments, please fire them my way, and happy automating.