{"id":7605,"date":"2023-11-28T09:39:12","date_gmt":"2023-11-28T15:39:12","guid":{"rendered":"https:\/\/gregsowell.com\/?p=7605"},"modified":"2023-11-28T09:39:12","modified_gmt":"2023-11-28T15:39:12","slug":"migrate-red-hat-aap-or-awx-to-ascender-with-automation","status":"publish","type":"post","link":"https:\/\/gregsowell.com\/?p=7605","title":{"rendered":"Migrate Red Hat AAP or AWX to Ascender with Automation"},"content":{"rendered":"<p>AWX derivatives are great tools for putting enterprise controls around your automation.  Ascender brings the best of open source automation into one easy-to-install package, and now when you are ready to migrate from either AWX or Red Hat\u2019s Ansible Automation Platform (AAP), you can with a few simple steps. It doesn\u2019t stop there, though. You can migrate portions of your dev\/test Ascender installs over to production, or simply use them as templates for Configuration As Code (CAC). The materials here will be showing how to do a 1 to 1 migration, but in the near future, I\u2019ll be creating and demoing some interesting use cases for distribution across your enterprise.<\/p>\n<h2>Video Demo<\/h2>\n<p><iframe loading=\"lazy\" width=\"560\" height=\"315\" src=\"https:\/\/www.youtube.com\/embed\/MzDtxICoORA?si=XMtibu80W3-x0oel\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" allowfullscreen><\/iframe><\/p>\n<h2>Playbooks<\/h2>\n<p>All of my materials can be found <a href=\"https:\/\/github.com\/gregsowell\/ascender-configuration-as-code-migration\">here in my public repository<\/a>.<\/p>\n<p>I\u2019m also using a few collections and tools:<br \/>\n<a href=\"https:\/\/github.com\/ansible-collections\/community.general\">Community.general<\/a> &#8211; Used for some json_query filtering.<br \/>\n<a href=\"https:\/\/github.com\/redhat-cop\/controller_configuration\">Controller_configuration<\/a> &#8211; Awesome RH COP collection of roles used to interact with the awx.awx collection.<br \/>\n<a href=\"https:\/\/github.com\/ansible\/awx\/tree\/devel\/awx_collection\">Awx.awx<\/a> &#8211; Modules designed to easily interact with the API. This is already baked into the default Ascender EE.<br \/>\n<a href=\"https:\/\/github.com\/ansible\/awx\/tree\/devel\/awxkit\">Awxkit<\/a> &#8211; A Python CLI tool for interacting with the API. This too is already baked into the default Ascender EE.<\/p>\n<p>I\u2019m going to break them into two categories: ones used for exporting information and those used for importing information.<\/p>\n<p><strong>Exporting<\/strong><br \/>\nThe primary file is <em>export.yml<\/em>. I\u2019m not going to walk through every piece; rather, I\u2019ll cover the important bits\u2026 or maybe those where I think I\u2019m being particularly clever. \ud83d\ude09<\/p>\n<pre class=\"gs-code\"><code class=\"language-plaintext\">  hosts: save-files\r\n\r\n  vars:\r\n    # These are the objects you wish to export. Be sure to comment out any you don&#039;t want.\r\n    export_objects: \r\n      - applications\r\n      - credentials\r\n      - credential_types\r\n      - execution_environments\r\n      - inventory\r\n      - inventory_sources\r\n      - job_templates\r\n      - notification_templates\r\n      - organizations\r\n      - projects\r\n      - schedules\r\n      - teams\r\n      - users\r\n      - workflow_job_templates<\/code><\/pre>\n<p>First is the host section. Here I have a server that will act as a temporary file repository. Future versions will be saving all of these files to a git repository, but for now I wanted it to be as quick and simple as possible. So choose a server that can hold a few KB of files.<\/p>\n<p>Next is the <em>export_objects<\/em> variable in the <em>vars <\/em>section. Any items in this list that aren\u2019t commented out will be exported into their yaml formatted configuration file (then stored on the save-files host).<\/p>\n<pre class=\"gs-code\"><code class=\"language-plaintext\">  environment:\r\n    CONTROLLER_HOST: &quot;https:\/\/{{ export_host }}&quot;\r\n    CONTROLLER_USERNAME: &quot;{{ gen1_user }}&quot;\r\n    CONTROLLER_PASSWORD: &quot;{{ gen1_pword }}&quot;\r\n    CONTROLLER_VERIFY_SSL: False<\/code><\/pre>\n<p>The awx.awx collection uses these environment variables to authenticate and connect to the API of the server that will be exporting data. You can set these variables in the vars section, hard code them here, or like me, you can securely pass them in from Ascender!<\/p>\n<p><strong>Once the export is complete, I copy these files into my repository in a directory named \u201cconfigs.\u201d<\/strong><\/p>\n<p><strong>Importing<\/strong><br \/>\nThere was a LOT more work that had to be done on the import side. In fact, this is where I spent the majority of my dev time. There is a lot of formatting of variables, filtering of information, reconfiguring of variable structures, or outright creation of variables that had to be done\u2026 and lucky for you, it should just work now!<\/p>\n<p>The primary file for importing information is <em>import.yml<\/em>. Again, I\u2019ll only cover the important bits here, though all of my tasks have comments and descriptions.<\/p>\n<pre class=\"gs-code\"><code class=\"language-plaintext\">    # These are the objects you want to import into the server.\r\n    export_objects: \r\n      - testing_value\r\n      - applications \r\n      - credentials \r\n      - credential_types \r\n      - execution_environments\r\n      - projects \r\n      - inventory\r\n      - inventory_sources\r\n      - job_templates\r\n      - notification_templates \r\n      - organizations \r\n      - teams \r\n      - users \r\n      - workflow_job_templates\r\n      - schedules <\/code><\/pre>\n<p>This is the exact same variable from the export.yml file. It defines what sections should be imported.<\/p>\n<pre class=\"gs-code\"><code class=\"language-plaintext\">    # Add hosts and or groups when adding\/updating inventories\r\n    # By default inventories won&#039;t import hosts\/groups, so setting these variables will control that.\r\n    add_hosts: true\r\n    add_groups: true<\/code><\/pre>\n<p>When  you import an inventory, it only creates the inventory, which means none of the hosts or groups within it are imported. If you want those to be imported also, you modify these two variables. By default they are set to true, so if you import an inventory, then the hosts and groups will come along with it.<\/p>\n<pre class=\"gs-code\"><code class=\"language-plaintext\">    # These are the users that won&#039;t be imported. Recall these users will have their passwords and keys wiped.\r\n    # I&#039;m using the admin user for importing so I want to ignore it.\r\n    filter_user_import:\r\n      - admin<\/code><\/pre>\n<p><strong>When importing users, it\u2019s often desirable to exempt some users.<\/strong> When user accounts are imported, their passwords, keys, etc. are wiped. I\u2019m using the admin account for importing, so I exempt it.<\/p>\n<pre class=\"gs-code\"><code class=\"language-plaintext\">    # List of schedule names to remove when importing.\r\n    # By default I&#039;m removing all of the system defaults\r\n    schedules_to_remove: \r\n      - &#039;Cleanup Job Schedule&#039;\r\n      - &#039;Cleanup Activity Schedule&#039;\r\n      - &#039;Cleanup Expired Sessions&#039;\r\n      - &#039;Cleanup Expired OAuth 2 Tokens&#039;<\/code><\/pre>\n<p>When importing some schedules, you may want to ignore some. In my case, I\u2019m removing any of the system defaults from the import (they will already be in the other host).<\/p>\n<pre class=\"gs-code\"><code class=\"language-plaintext\">  environment:\r\n    CONTROLLER_HOST: &quot;https:\/\/{{ import_host }}&quot;\r\n    CONTROLLER_USERNAME: &quot;{{ gen2_user }}&quot;\r\n    CONTROLLER_PASSWORD: &quot;{{ gen2_pword }}&quot;\r\n    CONTROLLER_VERIFY_SSL: False<\/code><\/pre>\n<p>As in the export.yml file, you must set up the environment variable information here for the host you plan to import configurations to.<\/p>\n<p>Once you hit the task section, we are entering what I call the \u201cmodification\u201d section. This is where I do massive cleanup first to the variable files (just the copy in the Execution Environment (EE)). Some changes were easier to fix in the variable files before they were brought into memory.<\/p>\n<p>Then all of the variable files are imported into memory, and after that another big round of cleanup\/manipulation is done to the variables.<\/p>\n<pre class=\"gs-code\"><code class=\"language-plaintext\">####### start importing various information\r\n\r\n  - name: block and rescue for multiple attempts\r\n    block:\r\n    - name: Add credential types\r\n      when: item == &quot;credential_types&quot;\r\n      ansible.builtin.include_role:\r\n        name: credential_types\r\n      loop: &quot;{{ export_objects }}&quot;\r\n    rescue:\r\n    - name: Add credential types - second attempt\r\n      ansible.builtin.include_role:\r\n        name: credential_types<\/code><\/pre>\n<p>Last, we enter the importing section. This is where all of the data is actually pushed into the remote host.<\/p>\n<p>You can see I use a block and rescue for each one of the tasks.  Essentially, I just call the relevant role and it will push in the data using an asynchronous method, which means it fires multiple tasks in the background so that they will complete as quickly as possible. It then checks the status for each. This definitely speeds up the import process.<\/p>\n<p>I noticed that hammering the API in this fashion also leads to failures when importing a lot of information at once. It\u2019s for this reason I went with a block and rescue. If the initial attempt fails for some of the imports, it will then rerun the exact same role, and in my testing this usually does the trick.<\/p>\n<h2>Configure Ascender, AWX, or AAP<\/h2>\n<p>I won\u2019t be doing the step-by-step here\u2026 if you are migrating automation platforms, I assume you already know how to add a project and create a job template. I will show you my simple modifications.<br \/>\n<a href=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2023\/11\/Screenshot_1.jpg\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2023\/11\/Screenshot_1-300x274.jpg\" alt=\"\" width=\"300\" height=\"274\" class=\"aligncenter size-medium wp-image-7606\" srcset=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2023\/11\/Screenshot_1-300x274.jpg 300w, https:\/\/gregsowell.com\/wp-content\/uploads\/2023\/11\/Screenshot_1-768x700.jpg 768w, https:\/\/gregsowell.com\/wp-content\/uploads\/2023\/11\/Screenshot_1.jpg 806w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p>In the above example, I\u2019m configuring the export playbook.<br \/>\nI created a custom credential that supplies the gen1_user and gen1_pword values.<br \/>\nI also added the machine credential to connect to my save-files host that will be holding the configs as they are created.<br \/>\nIn the variables section, I\u2019m adding the IP or Fully Qualified Domain Name(FQDN) of the host I\u2019ll be connecting to and exporting info out of.<br \/>\n<a href=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2023\/11\/Screenshot_2.jpg\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2023\/11\/Screenshot_2-251x300.jpg\" alt=\"\" width=\"251\" height=\"300\" class=\"aligncenter size-medium wp-image-7607\" srcset=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2023\/11\/Screenshot_2-251x300.jpg 251w, https:\/\/gregsowell.com\/wp-content\/uploads\/2023\/11\/Screenshot_2.jpg 522w\" sizes=\"auto, (max-width: 251px) 100vw, 251px\" \/><\/a><\/p>\n<p>In this screenshot, I\u2019m configuring the import job template.<br \/>\nAgain, I have a custom credential that in this instance supplies gen2_user and gen2_pword.<br \/>\nLast in the variables section, I specify the import_host FQDN or IP address. This will be the host I\u2019m pushing all of the configurations to.<\/p>\n<h2>Caveats<\/h2>\n<p>Passwords aren\u2019t exported; therefore, they aren\u2019t imported. If you are using a secrets engine, then this is a moot point as all of the private information is looked up at automation execution time.<\/p>\n<p>This isn\u2019t exporting or configuring specific Ascender configurations like: logging, authentication, etc. Right now it\u2019s only (mostly) pulling\/pushing configurations specific to automation. I plan to continually expand this, though.<\/p>\n<p>Roles aren\u2019t imported at this time (the awx.awx collection isn\u2019t bringing that in).<\/p>\n<p>User accounts aren\u2019t currently optimally imported if you are also using an external authentication system. It will try and use the local account first before attempting external auth, so if it is imported manually, this can cause issues. You can either not import users or have all of your users login once to the new system first.<\/p>\n<h2>Conclusion<\/h2>\n<p>This is, perhaps, the most impactful automation I\u2019ve created to date. I hope you can use it to test drive Ascender, or maybe just improve your backups &#x1f642;.  <\/p>\n<p>This is V1.0 of this automation, so know that it isn\u2019t yet in its final evolution. I implore you to let me know how I can improve it (again, I have very interesting variations on this that are coming soon). If you find bugs (I can\u2019t test every scenario), please let me know, or contribute code if you can.<\/p>\n<p>Thanks and happy migrating!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>AWX derivatives are great tools for putting enterprise controls around your automation. Ascender brings the best of open source automation into one easy-to-install package,\u2026<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[49],"tags":[],"class_list":["post-7605","post","type-post","status-publish","format-standard","hentry","category-ansible"],"_links":{"self":[{"href":"https:\/\/gregsowell.com\/index.php?rest_route=\/wp\/v2\/posts\/7605","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/gregsowell.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/gregsowell.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/gregsowell.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/gregsowell.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=7605"}],"version-history":[{"count":1,"href":"https:\/\/gregsowell.com\/index.php?rest_route=\/wp\/v2\/posts\/7605\/revisions"}],"predecessor-version":[{"id":7608,"href":"https:\/\/gregsowell.com\/index.php?rest_route=\/wp\/v2\/posts\/7605\/revisions\/7608"}],"wp:attachment":[{"href":"https:\/\/gregsowell.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=7605"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/gregsowell.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=7605"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/gregsowell.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=7605"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}