Vagrant-ception or ViV (Vagrant in Vagrant)

elrey741 -

Why?

Jump down to the next section if you don't care why, and want to get into the details right away. 😁️

So, I run Linux as my main operating system and I enjoy using the native hypervisor stack of linux ( more info in a section below ), because I feel as though it is more performant and battle tested since a lot of major cloud venders still use it and it is very prominent in the linux community. The unfortunate thing is that I also like to use vagrant...a lot! ( I use it for pentesting at my work ) Which vagrant does support running vagrant libvirt boxes through an awesome plugin, but unfortunately you can't run libvirt boxes along side of VirtualBox ( vbox ) boxes ( similar to not running hyper-v and virtualbox at the same time on windows ).

So a little bit of info for why I knew how to solve this issue, back when I used to be a lab manager at the 49th Security Division ( 49SD ), and I was investigating how we could run virtual machines ( vms ) inside of other vms. Since our entire infrastructure was virtualized and I was curious about using vbox inside of vms, so that students can tinker around quickly on a vm we give them. I found out it was possibly to run a vm inside of another vm through Second Level Address Translation ( SLAT ), and so since I knew that was a thing I figured I would check out if consumer CPUs had SLAT ( normally refered to as EPT/NPT ). I found out that there were quite a lot of CPUs that supported SLAT and specifically almost all the hardware I had supported it ( see below for the hardware I was using ).

So, since I like using vagrant so much I though...

Why don't I mimic docker and do...Vagrant inside of Vagrant!

So, I can run my baremetal hypervisor, which is running a libvirt vagrant box, and then run all my vbox boxes inside of the libvirt vagrant box ( hence vagrant-ception 😁️ ). That way I don't have to stop all my libvirt boxes to run my vbox boxes and visa versa.

My personal laptop ( Intel CPU ) it worked perfectly, and I was super excited to try it on my desktop ( AMD Ryzen CPU ). Unfortunately for some reason I wasn't able to get it to work... 😢️ ( I am guessing it is something to do with my motherboard and the BIOS that it isn't working properly... ). So, I wasn't able to use this very much since I like to run a lot of vms at once for running infrastructure ( infra ) stuff all the time...but then my AMAZING work got me a new System76 laptop that actually has better specs than my desktop!!! So, running lots of vms on that machine is no problem, and using this as my default setup for work related things is a lot more viable than my previous laptop ( which would crash because of chrome filling up it's 16Gb of ram with all my other vms ).

Setup

My setup is using QEMU/KVM/libvirt on Linux as my baremetal hypervisor and then VirtualBox ( vbox ) inside the baremetal hypervisor as the second layer hypervisor.

Since I like vagrant so much you can find all this code inside of my GitHub ( GH ) repo, with the necessary Vagrantfile and ansible playbook to install all necessary software for you to do the same thing (on linux ) here: https://github.com/elreydetoda/vagrant-files/tree/master/ViV . A windows version ( with 2 layers of vbox ) is in the works, but not going to promise a deadline for it, but let me know in the comments below if you would be interested.

Here is a recording of me doing the vagrant in vagrant ( ViV ):

Conclusion

While this isn't practical for all setups, and honestly I don't do this consistently even with my new laptop yet, it is definitely something to consider when you are running multiple hypervisors at one time and don't want to stop one running one. Plus it is a pretty cool PoC in my opinion 😁️. I know I didn't have very much technical content in this post, besides the backstory for why I wanted to do it, but that is because 1) I am far from an expert in this technology and am pretty much just a consumer 2) my setup was all in Infrastructure as Code ( IaC ), so I don't have to give a lot of commands because it is all in ansible on my GH repo. I have thought about some more optimization on how to streamline this setup, as an example you could probably mapping your ~/.vagrant.d folder inside the baremetal ( 1st layer ) vm to your host OS ~/.vagrant.d so you don't have to re-download all the boxes you have already downloaded. So, you might see another post sooner or later ( probably later, because I have a lot of other things I want to blog about ), but if you want to keep up with this topic here is the tag where all future updates will be.

Also, at the bottom of the article is a some troubleshooting steps in case your vagrant box doesn't support a big enough disk size for you initially and how you can resize it.

I know that I am super excited to do this more frequently since I got the new System76 computer ( thank you System76 for making such an awesome laptop!! ), and thank you Secure Ideas for allowing me to get this awesome computer!

If you are interested in more vagrant stuff you can go here: https://blog.elreydetoda.site/tag/vagrant/

Hardware

Personal

laptop

Razer Blade Stealth: RZ09-02393E32

desktop

Work

laptop

System76: Serval WS

Troubleshooting

NOTE: still in progress to get exact commands, but this is the general idea

resizing disk w/parted

If the provided vagrant box doesn't have enough disk space by default, than you can follow along with these instructions to make it larger.

srcs:

  1. get info
    • free disk space
    • starting sector of root partition
  2. run disk operations
    1. delete root partition
    2. create new partition
      • make sure you start the disk at the same spot as before
    3. make sure to do 100% to do whole disk
    4. write partition table
  3. run partprobe to re-read the partition table
  4. run resize2fs /dev/sda3 to expand filesystem to match block device partition