Skip to content
Jul 28 / Greg

Jinja2 HTML Templates For Reporting With Ansible Automation Platform The Easy Way


One of the really awesome uses for ansible is to perform an action and be able to develop a pretty report…for you or, let’s face it…management likes pretty graphs and spreadsheets.

I worked in tandem with the talented tiger Nick Arellano at a recent hackaton, and he cleaned up a tidy little jinja2 template to create pretty reports for both websites as well as emails. Let’s be honest, he did most of the work, so thank him 😉 This piece of automation will create a CSV file and then put it into a table for easy insertion into a web page or (as shown below) use in an email.

Video Demo

Playbook And Template

First, all of the materials can be found HERE IN THIS GIT REPOSITORY.

The playbook, as it stands at the time of this writing, is below:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
---
- name: Generate an HTML report from jinja template
  hosts: all
  gather_facts: true
  vars:
    #email settings
    email_subject: Patching Report
    email_host: 10.1.5.25
    email_from: [email protected]
    email_to: [email protected]
 
    #random settings
    csv_path: /tmp
    csv_filename: report.csv
    headers: Hostname,Distro Ver,Kernel Ver,Last Rebooted
 
  tasks:
  - name: Gather last reboot
    ansible.builtin.shell: last reboot | grep -m1 "" | awk '{ print $1, $6, $7, $8 }'
    register: rebooted
 
  - name: Save CSV headers
    ansible.builtin.lineinfile:
      dest: "{{ csv_path }}/{{ csv_filename }}"
      line: "{{ headers }}"
      create: true
      state: present
    delegate_to: localhost
    run_once: true
 
  - name: Build out CSV file
    ansible.builtin.lineinfile:
      dest: "{{ csv_path }}/{{ csv_filename }}"
      line: "{{ inventory_hostname }},{{ ansible_distribution_version }},{{ ansible_kernel }},{{ rebooted.stdout }}"
      create: true
      state: present
    delegate_to: localhost
 
  - name: Read in CSV to variable
    community.general.read_csv:
      path: "{{ csv_path }}/{{ csv_filename }}"
    register: csv_file
    delegate_to: localhost
    run_once: true
 
#  - name: debug csv_file
#    debug:
#      var: csv_file
#    run_once: true
 
  - name: Send Email
    community.general.mail:
      host: "{{ email_host }}"
      from: "{{ email_from }}"
      port: 25
      to: "{{ email_to }}"
      subject: "[Ansible] {{ email_subject }}"
      body: "{{ lookup('template', 'report.html.j2') }}"
      attach: "{{ csv_path }}/{{ csv_filename }}"
      subtype: html
    delegate_to: localhost
    run_once: true

Let me break down some of the pieces of the playbook.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
---
- name: Generate an HTML report from jinja template
  hosts: all
  gather_facts: true
  vars:
    #email settings
    email_subject: Patching Report
    email_host: 10.1.5.25
    email_from: [email protected]
    email_to: [email protected]
 
    #random settings
    csv_path: /tmp
    csv_filename: report.csv
    headers: Hostname,Distro Ver,Kernel Ver,Last Rebooted

The vars section isn’t very interesting save for the “headers” variable. This is the variable that will not only create the CSV header, but also used in the Jinja2 template for building out the HTML.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
  tasks:
  - name: Gather last reboot
    ansible.builtin.shell: last reboot | grep -m1 "" | awk '{ print $1, $6, $7, $8 }'
    register: rebooted
 
  - name: Save CSV headers
    ansible.builtin.lineinfile:
      dest: "{{ csv_path }}/{{ csv_filename }}"
      line: "{{ headers }}"
      create: true
      state: present
    delegate_to: localhost
    run_once: true
 
  - name: Build out CSV file
    ansible.builtin.lineinfile:
      dest: "{{ csv_path }}/{{ csv_filename }}"
      line: "{{ inventory_hostname }},{{ ansible_distribution_version }},{{ ansible_kernel }},{{ rebooted.stdout }}"
      create: true
      state: present
    delegate_to: localhost

This section starts by finding out when this host was last rebooted. You’ll notice the shell command does some grepping and awking. I’m a big fan of formatting as much of the data before it’s brought into Ansible as possible.
After that the CSV headers are inserted into the CSV, then the remainder of the CSV is built out for each host.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
  - name: Read in CSV to variable
    community.general.read_csv:
      path: "{{ csv_path }}/{{ csv_filename }}"
    register: csv_file
    delegate_to: localhost
    run_once: true
 
#  - name: debug csv_file
#    debug:
#      var: csv_file
#    run_once: true
 
  - name: Send Email
    community.general.mail:
      host: "{{ email_host }}"
      from: "{{ email_from }}"
      port: 25
      to: "{{ email_to }}"
      subject: "[Ansible] {{ email_subject }}"
      body: "{{ lookup('template', 'report.html.j2') }}"
      attach: "{{ csv_path }}/{{ csv_filename }}"
      subtype: html
    delegate_to: localhost
    run_once: true

Next is where the cool stuff happens. I use the read_csv module to read in the newly created CSV file and register it to a variable. This means complicated variables can be created quite simply.
Last is the email generation. In here I not only attach the newly created CSV, but I also use the template lookup plugin to call the Jinja2 template in the “body” section.

The way this playbook is written, virtually all of it is written flexibly in such a way that the only changes to generate a completely different report would be to change the CSV build tasks.

Now for the Jinja2 template(report.html.j2):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
<table style="border: 1px solid black; border-collapse: collapse;">
<tr>
    {% for header in headers.split(",") %}
    <th style="border: 1px solid black; padding: 8px 16px;">{{ header }}</th>
    {% endfor %}
</tr>
{% for host in csv_file.list %}
<tr>
    {% for header in headers.split(",") %}
    <td style="border: 1px solid black; padding: 8px 16px;">{{ host[header] }}</td>
    {% endfor %}
</tr>
{% endfor %}
</table>

This is a pretty small template, but it is mighty!
I’m not going to cover HTML tables; W3schools has a nice little doc on it here.
Let me try and break it down into smaller pieces so that it makes a little more sense.

1
2
3
    {% for header in headers.split(",") %}
    <th style="border: 1px solid black; padding: 8px 16px;">{{ header }}</th>
    {% endfor %}

The very first table row is going to setup the column headers. If you recall the “headers” variable was created to setup a method for initial creation, but ultimately so we could use it here in this first jinja loop. Whenever you see {% %} this is breaking into jinja saying that “there’s something to evaluate here.” In the for section I’m setting up a for loop. A for loop will iterate over a list of things one by one, until I’ve covered all of the contents. Here I’m taking the headers variable and “split”ing it into a list of values. This list is saved into a new variable named “header”. I then move down and create a row entry for each {{ header }} variable that exists. Now there is a lot more information here than just the result of the variable. This little bit of Nick Magic® does formatting on the cell itself so that it will have a nice border along with padding around the value(it makes it look pretty and easily readable). The formatting has to be done inline like this because different email clients interpret HTML differently and this seems to have the most support.

Next I fill out the actual data:

1
2
3
4
5
6
7
{% for host in csv_file.list %}
<tr>
    {% for header in headers.split(",") %}
    <td style="border: 1px solid black; padding: 8px 16px;">{{ host[header] }}</td>
    {% endfor %}
</tr>
{% endfor %}

Here I’m doing nested loops(super easy to do inside of Jinja). The outter loop is built off of the csv_file.list variable that was created by importing my CSV file. This populates the for variable of host.
On the inner loop I’m taking the headers section and splitting it in the exact same way…in fact it looks almost identical to the above safe for one change. The table data variable is different “{{ host[header] }}”. Notice that it is take the first host and plucking each header’s information from it and applying it to the cell one by one. So if the host entries are “Hostname,Distro Ver,Kernel Ver,Last Rebooted”, then it will loop through those one at a time and apply those entries to the cell.

Sample Output

Conclusion

I hope you can see how easy it would be to adapt this simple reporting technique to fit almost any scenario. Just a few quick tweaks and it covers routers, switches, security devices, AWS resources…whatever you can imagine.

If you would change it to adapt to your environment, what would that look like?

Good luck and happy reporting!

Jul 5 / Greg

Why Am I Taylor Bikowski

Hey everybody, I’m Greg Sowell and this is Why Am I, a podcast where I talk to interesting people and try to trace a path to where they find themselves today.  My guest this go around is Taylor Bikowski.  They are, and I quote: “a celestial starchild, learning from past lives, and changing the world with love and light.”  What I really see when I look at Taylor is what pure energy would look like in human form.  They are a self described neurodivergent who is putting in alllll of the work, and experimenting with some really interesting therapies.  I think the line that is going to stick with me was “my trauma was your Tuesday…”  woof.  At any rate, I hope you enjoy this enlightening conversation with Taylor.
Youtube version here:
If you want to support the podcast you can do so via https://www.patreon.com/whyamipod (this gives you access to bonus content including their Fantasy Restaurant!)
Jun 30 / Greg

Resize Windows Disk On VMWare Via Ansible Automation Platform

I really like having little bite sized demos, though some argue telling a big story works best…I suppose a mix of the two is likely the real solution.
This demo shows the Ansible Automation Platform(AAP) connecting to VMware Vcenter to expand a disk on a windows host. AAP then connects to the disk and expands the disk in windows itself.

Video Demo

Playbooks

You can find the playbook in question here.

I’ll breakdown some of its parts here:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
  tasks:
  - name: Connect to VMWare and shutdown guest
    community.vmware.vmware_guest:
      hostname: "{{ vcenter_hostname }}"
      username: "{{ vcenter_username }}"
      password: "{{ vcenter_password }}"
      datacenter: "{{ datacenter_name }}"
      validate_certs: no
      name: "{{ inventory_hostname }}"
      state: shutdownguest
    delegate_to: localhost
    failed_when: vm_shutdown.instance.hw_power_status != 'poweredOff' and vm_shutdown.changed == false
    register: vm_shutdown
 
  - name: Wait 15 seconds
    ansible.builtin.pause:
      seconds: 15
 
  - name: Connect to VMWare and resize disk on VM
    community.vmware.vmware_guest_disk:
      hostname: "{{ vcenter_hostname }}"
      username: "{{ vcenter_username }}"
      password: "{{ vcenter_password }}"
      datacenter: "{{ datacenter_name }}"
      validate_certs: no
      name: "{{ inventory_hostname }}"
      disk:
        - size_mb: 2000
          state: present
          unit_number: 1
          scsi_controller: 0
          scsi_type: 'lsilogicsas'
    delegate_to: localhost
    register: disk_facts
 
  - name: Connect to VMWare and boot up guest
    community.vmware.vmware_guest:
      hostname: "{{ vcenter_hostname }}"
      username: "{{ vcenter_username }}"
      password: "{{ vcenter_password }}"
      datacenter: "{{ datacenter_name }}"
      validate_certs: no
      name: "{{ inventory_hostname }}"
      state: poweredon
    delegate_to: localhost

Most of the work in the above is done via the vmware_guest modules from the vmware collection.
I first shutdown the guest.
I then do the resize on the disk in question. Something I learned is that, while optional, the scsi_type parameter needs to be specified. It will assume a default(likely other than is already set), and then your VM guest likely won’t boot!
After the resize is done, I boot the guest back up.

1
2
3
4
5
6
7
8
9
10
11
12
13
  - name: Wait 300 seconds, but only start checking after 15 seconds
    wait_for_connection:
      delay: 15
      timeout: 300
 
  - name: Connect to windows host and expand the drive
    ansible.windows.win_powershell:
      script: |
        # Variable specifying the drive you want to extend
        $drive_letter = "E"
        # Script to get the partition sizes and then resize the volume
        $size = (Get-PartitionSupportedSize -DriveLetter $drive_letter)
        Resize-Partition -DriveLetter $drive_letter -Size $size.SizeMax

The last little bit is pretty straight forward.
I first use the wait for connection module to repeatedly test until the guest is booted back up and ready to be connected to.
I next use the win_powershell module to tell the drive to expand using the newly adjusted vmware disk size. Note that I could have used an ansible module, but at the time of creation I was trying to demonstrate how to use powershell scripts directly.

Conclusion

While this isn’t revolutionary, it is very convenient. I can imagine monitoring my server infrastructure for disk utilization. When I violate a certain threshold my monitoring system will call the AAP API and pass over the name of the host, and the offending drive. AAP can then automatically connect to the server and resize the disk, then create a ticket or send an email. This feels pretty powerful to me!

As always, if you have any questions or comments I’d love to hear them.
Thanks and happy automating!

Jun 19 / Greg

Why Am I Saskia Wilson-Brown

Hey everybody, I’m Greg Sowell and this is Why Am I, a podcast where I talk to interesting people and try to trace a path to where they find themselves today.  My guest this go around is Saskia Wilson-Brown.  Saskia has been in and around TV production for some time, even being part of the Silver Lake Film Festival in LA.  She’s also a great podcaster, and BTW has founded and runs the Institute of Art and Olfaction.  More than that she’s a driven artist that tears walls down so that people can connect with each other and connect with arts of all kinds no matter who wants to keep the secrets locked away.  She’s kind of a badass.  At any rate, I hope you enjoy this conversation with Saskia.
Youtube version here:
If you want to support the podcast you can do so via https://www.patreon.com/whyamipod (this gives you access to bonus content including their Fantasy Restaurant!)
Jun 18 / Tommy Croghan

The Brother WISP 163 – Preseem Sponsor Highlight

 

This week we have Tommy, John Osman, Greg Lipschitz, Jeremy Austin and Dan Siemon to talk about Preseem as our new sponsor

**Sponsors**
Sonar.Software
Towercoverage.com
Preseem.com
**/Sponsors**

This week we talk about:

0:00:59 Introduction
0:05:30 Preseem elevator pitch
0:09:16 Tommy’s Story with Preseem
0:12:28 What is QOE?
0:15:35 Onboarding Time and what Preseem boxed look like (Hardware requirements)
0:19:50 Question “How many queues can you handle?”
0:22:40 CGNAT?
0:25:04 QOS is Bunk!
0:28:59 Is Preseem afraid of the availability of FQ_Codel/Cake in commodity (Mikrotik) hardware?
0:32:40 Is Preseem only for Wireless? What about other Tech? (DSL, Fiber, Cable, PTMP technologies)
0:34:29 The value of Preseem, our access networks need improvement. Why Preseem
0:50:30 Does not replace actually having sufficient bandwidth! But can help.
0:53:41 The classical solution of throwing bandwidth at all of the problems doesn’t help either.
0:57:40 Where does Preseem+ fit in?
0:59:48 What about when you have multiple ingress/egress paths? (diverse upstreams in your network)
1:04:30 What about when the Preseem box fails?
1:06:30 Is Docker an option?
1:08:50 What about integration with other platforms? (Billing software)
1:10:30 Can I just get Preseem+?
1:12:05 IPv6 support?
1:16:10 Multi-Queueing?
1:20:56 Transport Technologies beyond IPv4 and IPv6? (VLAN, VXLAN, MPLS)
1:20:20 Why the WISP Market and what’s the future look like for other technologies?
1:28:40 Goal is improving customer’s experience. (both you and your customer’s)
1:32:00 How to get ahold of Preseem.
1:36:45 How to get ahold of Greg
1:38:07 How to get ahold of John

https://www.youtube.com/c/Preseem
John’s site: https://www.miscreantsinaction.com/

Jun 12 / Greg

Why Am I Anna Ream

Hey everybody, I’m Greg Sowell and this is Why Am I, a podcast where I talk to interesting people and try to trace a path to where they find themselves today.  My guest this go around is Anna Ream.  She came with an impressive list of “who she is”, but the highlights are: practicing artist, photographer, mother, wife, purchaser of things, and person of faith.  We discuss how photography is a super power that opens avenues through which Anna is able to examine herself, her life, and her feelings.  At the end of all of it she has a tangible object that can then help others along their path.  I hope you enjoy this chat with Anna.
Youtube version here:
Please show them some love on their socials here:http://www.annaream.com/about/, https://www.instagram.com/anna_ream/.
If you want to support the podcast you can do so via https://www.patreon.com/whyamipod (this gives you access to bonus content including their Fantasy Restaurant!)
Jun 6 / Tommy Croghan

The Brother WISP 162 – Ubiquiti 60 GHz gear, Mikrotik V7, Flat network recommendations and more

 

This week we have Tommy and Jon

**Sponsors**
Sonar.Software
Towercoverage.com
**/Sponsors**

This week we talk about:
Ubiquiti 60 GHz gear announcements, various 60 GHz gear and a cool teardown video, AF60 and AF60LR are Class A Radios Use Caution when placing them!

1 Petabit per second per strand of Fiber

Ubiquiti Edgeswitch s16 Issues and Switch/Power Discussion.

Mikrotik ROS V7.3 RC1 & 2 and Comments on Cake Queues

Flat network woes and recommendations for avoiding annoying things 1:19:00 TCP Windowing issue?

Mimosa Announcements

Subscription Models and how unpopular they are.

Tom Smith BGP on BSD, X86 Routers

Relevant Links

https://www.youtube.com/watch?v=KxC5R…

https://www.youtube.com/watch?v=JI9fv…

https://newatlas.com/telecommunicatio…

Fiber Store’s Mux ponder https://www.fs.com/products/148739.html

https://www.businesswire.com/news/hom…

Tom Smith did a presentation at BSDCan about BGP on BSD https://www.bsdcan.org/events/bsdcan_…

Here’s the video:(if you don’t see it, hit refresh)