{"id":7690,"date":"2024-03-21T13:04:41","date_gmt":"2024-03-21T19:04:41","guid":{"rendered":"https:\/\/gregsowell.com\/?p=7690"},"modified":"2024-03-21T13:04:41","modified_gmt":"2024-03-21T19:04:41","slug":"vmware-to-proxmox-ve-migration-via-ansible-and-ascender","status":"publish","type":"post","link":"https:\/\/gregsowell.com\/?p=7690","title":{"rendered":"VMware To Proxmox VE Migration Via Ansible And Ascender"},"content":{"rendered":"<p><a href=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/thumbnail-template.jpg\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/thumbnail-template-300x169.jpg\" alt=\"\" width=\"300\" height=\"169\" class=\"aligncenter size-medium wp-image-7692\" srcset=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/thumbnail-template-300x169.jpg 300w, https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/thumbnail-template-768x432.jpg 768w, https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/thumbnail-template-1024x576.jpg 1024w, https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/thumbnail-template.jpg 1280w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p>Once we choose a system, we often feel married to it.  The idea of migrating our infrastructure often seems more expensive than the new price the vendor just handed us for what we are using.  I\u2019ve felt\/seen this with software, cloud providers, and hypervisors.  While this can be the truth at first glance, automation can often level the playing field.  Case and point, I\u2019m about to demonstrate how to migrate migrate a Rocky VM from VMWare to Proxmox VE completely via automation &#x1f642;<\/p>\n<h2>Video Demo<\/h2>\n<p><iframe loading=\"lazy\" width=\"560\" height=\"315\" src=\"https:\/\/www.youtube.com\/embed\/_A5uz9DMWPA?si=q8JlmyMiATYbEA10\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<h2>Overview<\/h2>\n<p>I thought I might quickly begin by describing the process with graphics.<\/p>\n<p>First, we have a VMWare cluster and our Proxmox cluster.  As you can see I have a virtual machine(VM1) that I want to migrate from VMware over to Proxmox.<br \/>\n<a href=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/Screenshot_1.jpg\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/Screenshot_1-300x129.jpg\" alt=\"\" width=\"300\" height=\"129\" class=\"aligncenter size-medium wp-image-7693\" srcset=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/Screenshot_1-300x129.jpg 300w, https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/Screenshot_1.jpg 642w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p>My first step is to use automation to connect into the running VM and prep it for migration.  My Rocky host didn\u2019t have the Virtio drivers in the initramfs, so I need to add them.  The initramfs is a file system that boots into ram that allows the kernel to mount the disks and do the actual file system boot.  Your mileage may vary\u2026perhaps these are already in your boot system, so give it a test as is and see how your system responds.  Once the prep work is done, the VM is shutdown.<br \/>\n<a href=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/Screenshot_2.jpg\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/Screenshot_2-300x260.jpg\" alt=\"\" width=\"300\" height=\"260\" class=\"aligncenter size-medium wp-image-7694\" srcset=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/Screenshot_2-300x260.jpg 300w, https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/Screenshot_2.jpg 627w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p>Next we install VMware\u2019s OVFTool somewhere to extract the VMs as OVFs from the VMware system.  For my small demo I\u2019m going to install it on my Proxmox host and do the migrations directly from there.  You will do the migrations to a temporary storage location either directly on the Proxmox host or via something like an NFS share mapped to the host.  If you are doing this to an NFS share you could easily be running the OVFTool from a completely separate server(this would cause the least impact to your Proxmox host).<br \/>\n<a href=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/Screenshot_3.jpg\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/Screenshot_3-300x124.jpg\" alt=\"\" width=\"300\" height=\"124\" class=\"aligncenter size-medium wp-image-7695\" srcset=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/Screenshot_3-300x124.jpg 300w, https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/Screenshot_3.jpg 601w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p>Now on the Proxmox host I\u2019ll use the qm importovf command to bring the newly created VM1 OVF into the system as a VM.<br \/>\n<a href=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/Screenshot_4.jpg\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/Screenshot_4-300x264.jpg\" alt=\"\" width=\"300\" height=\"264\" class=\"aligncenter size-medium wp-image-7696\" srcset=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/Screenshot_4-300x264.jpg 300w, https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/Screenshot_4.jpg 600w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p>Before starting the VM we\u2019ll need to modify any hardware settings required for this specific VM.  Again, this is a \u201cyour mileage may vary\u201d scenario.  In my case, I had to update the bios, add an efi disk for the new bios, modify the CPU type, and reconfigure the networking.<br \/>\n<a href=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/Screenshot_5.jpg\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/Screenshot_5-300x266.jpg\" alt=\"\" width=\"300\" height=\"266\" class=\"aligncenter size-medium wp-image-7697\" srcset=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/Screenshot_5-300x266.jpg 300w, https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/Screenshot_5.jpg 618w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p>Now the last step is pretty simple; all that needs to be done is to start the new VM.<br \/>\n<a href=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/Screenshot_6.jpg\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/Screenshot_6-300x117.jpg\" alt=\"\" width=\"300\" height=\"117\" class=\"aligncenter size-medium wp-image-7698\" srcset=\"https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/Screenshot_6-300x117.jpg 300w, https:\/\/gregsowell.com\/wp-content\/uploads\/2024\/03\/Screenshot_6.jpg 595w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<h2>Playbooks<\/h2>\n<p>All of my playbooks can be found here in <a href=\"https:\/\/github.com\/gregsowell\/ansible-vmware-to-proxmox\">my git repository<\/a>.<\/p>\n<p>The whole process actually took a good bit of work to figure out(really down to driver\/bios issues), so hopefully I can save you some time and effort.<\/p>\n<p><strong>preconfigure-hosts.yml<\/strong><br \/>\nThis playbook\u2019s purpose is to do any preconfiguration work necessary.  In my case the prework is to update the initramfs with virtio drivers and then to shut the machine down(OVFTool needs the VM to be shutdown before it can migrate it).  Depending on your environment and OS, there may be different things required.  Here you could also do things like backup network configuration settings so that when the migration is complete you can apply them.<\/p>\n<pre class=\"gs-code\"><code class=\"language-plaintext\">---\r\n- name: Preconfigure the VMWare hosts for migration\r\n  hosts: all\r\n  gather_facts: false\r\n  vars:\r\n  tasks:\r\n    - name: Connect to hosts and add virtio drivers to boot(using virtio in proxmox)\r\n      ansible.builtin.shell: &#039;dracut --add-drivers &quot;virtio_scsi virtio_pci&quot; -f -v \/boot\/initramfs-`uname -r`.img `uname -r`&#039;\r\n      register: driver_install\r\n\r\n    - name: Shutdown the host\r\n      community.general.shutdown:<\/code><\/pre>\n<p>In this playbook I create the new boot file based on the name of the kernel currently running( the uname -r portion).<\/p>\n<p><strong>migrate-vms.yml<\/strong><br \/>\nNext the migrate playbooks is run.  This is what actually grabs the OVF, imports into Proxmox, and updates all settings.<\/p>\n<pre class=\"gs-code\"><code class=\"language-plaintext\">---\r\n- name: Use ovftool to pull vms from vmware and import into proxmox\r\n  hosts: all\r\n  gather_facts: false\r\n  vars:\r\n\r\n    proxmox_auth: &amp;proxmox_auth\r\n      api_host: proxmox.gregsowell.com\r\n      api_user: &quot;{{ gen2_user }}&quot;\r\n      # api_user: root@pam # username format example\r\n      # Use standard password\r\n      api_password: &quot;{{ gen2_pword }}&quot;\r\n      # Use api token and secret - example format\r\n      # api_token_id: gregisa5.0\r\n      # api_token_secret: 72a72987-ff68-44f1-9fee-c09adaaecf4d\r\n\r\n    #Full path to ovftool directory\r\n    ovftool_path: \/root\/ovftool\r\n\r\n    #Hostname or IP of vmware vcenter\r\n    vmware_host: 10.0.2.10\r\n\r\n    #Hostname or IP of proxmox\r\n    proxmox_host: proxmox.gregsowell.com\r\n\r\n    #Datacenter the VMs are in\r\n    vmware_datacenter: MNS\r\n\r\n    #Directory path to vms\r\n    vmware_directory: Greg\/ciq\r\n\r\n    #VMware username\r\n    # vmware_username: administrator\r\n    vmware_username: &quot;{{ gen1_user }}&quot;\r\n\r\n    #VMware password\r\n    # vmware_password: test\r\n    vmware_password: &quot;{{ gen1_pword }}&quot;\r\n\r\n    #Proxmox migration folder\r\n    proxmox_migration_dir: \/root\/migration\r\n    \r\n    #Storage proxmox will import new vms into with qm import command\r\n    proxmox_storage: local-lvm\r\n\r\n  tasks:\r\n    - name: Add proxmox host to inventory\r\n      ansible.builtin.add_host:\r\n        name: &quot;{{ proxmox_host }}&quot;\r\n        ansible_host: &quot;{{ proxmox_host }}&quot;\r\n      run_once: true\r\n\r\n    - name: Create the proxmox migration directory if it doesn&#039;t exist\r\n      ansible.builtin.file:\r\n        path: &quot;{{ proxmox_migration_dir }}&quot;\r\n        state: directory\r\n        mode: &#039;0755&#039;\r\n      delegate_to: &quot;{{ proxmox_host }}&quot;\r\n      run_once: true\r\n\r\n    - name: Check for each host and see if they already have an export folder created\r\n      stat:\r\n        path: &quot;{{ proxmox_migration_dir }}\/{{ hostvars[inventory_hostname].config.name }}&quot;\r\n      register: directory_status\r\n      delegate_to: &quot;{{ proxmox_host }}&quot;\r\n\r\n    - name: Block for exporting\/importing hosts\r\n      when: inventory_hostname != proxmox_host and ( not directory_status.stat.exists or not directory_status.stat.isdir )\r\n      block:\r\n        - name: Run ovftool to export vm from vmware to proxmox server if folder isn&#039;t already there\r\n          ansible.builtin.shell: &quot;{{ ovftool_path }}\/ovftool --noSSLVerify vi:\/\/{{ vmware_username }}:{{ vmware_password }}@{{ vmware_host }}:443\/{{ vmware_datacenter }}\/vm\/{{ vmware_directory }}\/{{ hostvars[inventory_hostname].config.name }} .&quot;\r\n          args:\r\n            chdir: &quot;{{ proxmox_migration_dir }}\/&quot;\r\n          delegate_to: &quot;{{ proxmox_host }}&quot;\r\n          no_log: true\r\n\r\n        - name: Call task file to provision new proxmox vm.  Loop over hosts in play\r\n          ansible.builtin.include_tasks:\r\n            file: import-ovf.yml\r\n          run_once: true\r\n          loop: &quot;{{ play_hosts }}&quot;\r\n\r\n        - name: Modify proxmox vms to have required settings\r\n          community.general.proxmox_kvm:\r\n            &lt;&lt;: *proxmox_auth\r\n            name: &quot;{{ hostvars[inventory_hostname].config.name }}&quot;\r\n            node: proxmox\r\n            scsihw: virtio-scsi-pci\r\n            cpu: x86-64-v2-AES\r\n            bios: ovmf\r\n            net:\r\n              net0: &#039;virtio,bridge=vmbr0,firewall=1&#039;\r\n            efidisk0:\r\n              storage: &quot;{{ proxmox_storage }}&quot;\r\n              format: raw\r\n              efitype: 4m\r\n              pre_enrolled_keys: false\t\r\n            update: true\r\n            update_unsafe: true\r\n          register: vm_update\r\n          delegate_to: localhost\r\n\r\n        - name: Start VM\r\n          community.general.proxmox_kvm:\r\n            &lt;&lt;: *proxmox_auth\r\n            name: &quot;{{ hostvars[inventory_hostname].config.name }}&quot;\r\n            node: proxmox\r\n            state: started\r\n          delegate_to: localhost\r\n          # ignore_errors: true<\/code><\/pre>\n<p>I\u2019ll attempt to cover the highlights instead of the entire playbook(which I do cover in the video).<\/p>\n<p>Task number 2(<em>Check for each host and see if they already have an export folder created<\/em>) checks to see if the directory for the current host has already been migrated.  This is used in the migration conditional later, so that if it already exists it won\u2019t be exported\/imported again.<\/p>\n<p>Once the OVF has been created the (<em>Call task file to provision new proxmox vm.  Loop over hosts in play<\/em>) task is run.  This task runs once as a loop over a variable that contains all of the hosts in the current play(<em>play_hosts<\/em> which is a magic variable).  For each host it will locate the next available Proxmox vid and then it will import the OVF.<\/p>\n<p><strong>import-ovf.yml<\/strong><\/p>\n<pre class=\"gs-code\"><code class=\"language-plaintext\">---\r\n- name: Find the next available vmid\r\n  ansible.builtin.shell: pvesh get \/cluster\/nextid\r\n  register: next_vmid\r\n  delegate_to: &quot;{{ proxmox_host }}&quot;\r\n\r\n- name: Run qm import with the new next_vmid\r\n  when: item != proxmox_host\r\n  ansible.builtin.shell: &quot;qm importovf {{ next_vmid.stdout }} {{ hostvars[item].config.name }}\/{{ hostvars[item].config.name }}.ovf {{ proxmox_storage }}&quot;\r\n  args:\r\n    chdir: &quot;{{ proxmox_migration_dir }}\/&quot;\r\n  delegate_to: &quot;{{ proxmox_host }}&quot;<\/code><\/pre>\n<p>Once all OVFs have been imported we need to modify their hardware settings via this task(<em>Modify proxmox vms to have required settings<\/em>).<br \/>\nYou can see with the proxmox_kvm module I\u2019m using the <em>update: true<\/em> option.  This works for most of the options I\u2019m trying to modify, but it DOESN\u2019T allow me to update my network settings or create an EFI disk, which is required for proper operation.  To fix this you have to use the <em>update_unsafe: true<\/em> option.  They have this because in the update I could break configs that result in data loss, but in this case I\u2019m only adding required components on my new install, so I\u2019m good to go.<\/p>\n<h2>Manual Testing<\/h2>\n<p>While the automation is awesome, you will likely still want\/need to do some manual testing.  I\u2019ve got the various steps here in brief so you can do just that.<\/p>\n<p><strong>Install OVFTool<\/strong><br \/>\nVisit <a href=\"https:\/\/developer.vmware.com\/web\/tool\/ovf\/\">this VMware page<\/a> and grab whichever version of OVFTool is appropriate for you.  I grabbed the 4.4.3 archive.  I extracted it with tar and made the ovftool and ovftool.bin files executable.  I put this on my Proxmox host as I was running all the commands from there.<\/p>\n<p><strong>VM Prep<\/strong><br \/>\nOn the VMs you intend to migrate, you can run the following command to see if you have the virtio drivers on your machine:<\/p>\n<pre class=\"gs-code\"><code class=\"language-plaintext\">find \/lib\/modules\/`uname -r` -type f -name virtio_*<\/code><\/pre>\n<p>If they are present(which has always been the case for me), you can run the manual steps to create the new initramfs file<\/p>\n<pre class=\"gs-code\"><code class=\"language-plaintext\">dracut --add-drivers &quot;virtio_scsi virtio_pci&quot; -f -v \/boot\/initramfs-`uname -r`.img `uname -r`<\/code><\/pre>\n<p><strong>Browse For VMs With OVFTool<\/strong><br \/>\nThere\u2019s a specific folder structure your VMs will follow.  To browse around and make sure things are where you expect them to be, you can use the following command:<\/p>\n<pre class=\"gs-code\"><code class=\"language-plaintext\">ovftool --noSSLVerify vi:\/\/username:password@host:443\/YourDatacenter\/vm<\/code><\/pre>\n<p>My environment doesn\u2019t have a valid cert on the vcenter, so I\u2019m using the <em>noSSLVerify<\/em> option.<br \/>\nYou can also leave off <em>:password<\/em> in the command and it will prompt you for your password.<br \/>\n<em>YourDatacenter <\/em>will be the case sensitive datacenter name and <em>\/vm<\/em> will always need to be there(it\u2019s where VMs show up).<br \/>\nThis will begin to start allowing you to browse to your VM paths.<\/p>\n<p><strong>Export OVFs<\/strong><br \/>\nThe command to actually export the OVFs should be executed from inside of the folder you want things saved to:<\/p>\n<pre class=\"gs-code\"><code class=\"language-plaintext\">ovftool --noSSLVerify vi:\/\/gsowell@10.0.2.1:443\/MNS\/vm\/Greg\/ciq\/Greg-rocky8 .<\/code><\/pre>\n<p>Notice in the command I followed the folder structure <em>Greg\/ciq<\/em>, then added the name of my VM.<br \/>\nAt the very end of the command is a space and a period.  If you leave off the period it won\u2019t do the export, so make sure \u201c<em> .<\/em>\u201d is on the end.<\/p>\n<p><strong>Import OVFs<\/strong><br \/>\nThe command to import an OVF is as follows:<\/p>\n<pre class=\"gs-code\"><code class=\"language-plaintext\">qm importovf &lt;vid&gt; &lt;contained folder&gt;\/&lt;name of vm&gt;.ovf &lt;proxmox storage location&gt;<\/code><\/pre>\n<p>An example would be:<\/p>\n<pre class=\"gs-code\"><code class=\"language-plaintext\">qm importovf 400 Greg-rocky8-to-9-conversion\/Greg-rocky8-to-9-conversion.ovf local-lvm<\/code><\/pre>\n<p>Notice you have to specify a unique vid for each imported VM.  You can look at the Proxmox interface to find the next one, or you can issue this CLI command and it will tell you what it is:<\/p>\n<pre class=\"gs-code\"><code class=\"language-plaintext\">pvesh get \/cluster\/nextid<\/code><\/pre>\n<h2>Conclusion<\/h2>\n<p>Migrating from one product to another can feel impossible at times, but automation is here to be your secret super power.  We are quickly becoming migration experts, and we\u2019d love to assist your team with complete migrations that have automation leave behinds\u2026or we can also just do validation; as much or as little as you require.<\/p>\n<p>As always, I appreciate you reading through, and would love your feedback.<\/p>\n<p>Thanks and happy migrating!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Once we choose a system, we often feel married to it. The idea of migrating our infrastructure often seems more expensive than the new\u2026<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[49,14,13,40],"tags":[],"class_list":["post-7690","post","type-post","status-publish","format-standard","hentry","category-ansible","category-linux","category-server","category-vmware"],"_links":{"self":[{"href":"https:\/\/gregsowell.com\/index.php?rest_route=\/wp\/v2\/posts\/7690","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/gregsowell.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/gregsowell.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/gregsowell.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/gregsowell.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=7690"}],"version-history":[{"count":5,"href":"https:\/\/gregsowell.com\/index.php?rest_route=\/wp\/v2\/posts\/7690\/revisions"}],"predecessor-version":[{"id":7702,"href":"https:\/\/gregsowell.com\/index.php?rest_route=\/wp\/v2\/posts\/7690\/revisions\/7702"}],"wp:attachment":[{"href":"https:\/\/gregsowell.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=7690"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/gregsowell.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=7690"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/gregsowell.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=7690"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}