It is common knowledge that running software on bare-metal is faster, yet virtualization is everywhere, and for good reasons. When you run software in the cloud, as we do in our Beamr Video Cloud service, the environment is virtualized, and the underlying VM technology is determined by the service provider (in our case – Amazon Web Services). So as platform users, the question of bare-metal vs. virtualization is not relevant. But what happens when you run Beamr Video on-premises, and manage your own servers? Does bare-metal significantly increase the efficiency of Beamr Video? In this post I will share some insights about the performance overhead of virtualization when running Beamr Video, and the ease of system setup.
Fair comparisons and benchmarks are hard to perform, first of all, because it is not easy to get both the bare-metal and virtualized systems running on the exact same HW and environment. Moreover, each benchmark measures performance of a specific set of tasks, thus results might not be typical for the Beamr Video workload. In order to get some insights into the VM overhead, I ran tests of Beamr Video on a few bare metal and virtualized configurations. The goal was to create comparable environments and test the overhead of VMs (e.g. Amazon’s Xen, or Virtualbox), and Containers (e.g. LXC or Docker).
Note this is absolutely not a complete report on VM performance, nor a guide to VM optimization — the idea was just to get a “feeling” about what to expect from virtualization when using Beamr Video.
Three systems were chosen:
- Dell PowerEdge server – Dual Xeon E5-2670
- Toshiba Satellite Laptop – Dual Core i7-4510U
- Amazon EC2 g2.8xlarge instance
I believe that the configuration of the g2.8xlarge server is very similar to the Dell PowerEdge server, both using the same E5-2670 CPUs, 64GB RAM, and SSD drives.
The first tests I ran suggested that on small workloads the g2.8xlarge instance was almost twice as fast as the Dell server. Larger workloads resulted in about 15% performance in favor of the Amazon server. This of course did not make any sense. Modifying the Dell BIOS settings to favor performance over efficiency, I was able to turn the tables, with 6-10% faster runtime on bare-metal (the longer workloads yielded smaller differences).
Lesson #1 – When hosting your own servers, the BIOS optimization settings make a real difference.
Lesson #2 – The power/performance optimization methods have a bigger effect on variable workloads, where the CPU has more opportunities to throttle down in order to save power.
Lesson #3 – Bare-metal is indeed faster. No surprise there.
I ran another series of smaller workloads on the dual core laptop, comparing a native Linux 3.19 kernel to the same Linux running inside a VirtualBox VM with an underlying Windows 10 operating system. The virtualized Linux was about 5-7% slower than the native Linux. This test used only one of the two cores available on the HW, as the second core was reserved for Windows and the VM. In other words, while the overhead is relatively small per core, utilization of the entire system is somewhat limited. You can read more on this here.
Following I examined containers. Linux containers use the underlying host kernel, and therefore introduce less overhead compared to virtualization. I tested LXC, the default Ubuntu container service, and Docker. Docker containers are appealing due to the ease of deployment, high popularity, and since they support running containerized applications on Windows or OSX inside a “Docker Machine” which itself runs inside VirtualBox. (Note that recently Microsoft and Docker announced native integration of Docker with Windows Hyper-V).
Beamr Video inside an LXC container resulted in around 6% overhead compared to the bare-metal configuration. While this may look similar to the VM running on Windows, this setup allows much better utilization of CPU cores and RAM, which may be almost fully allocated to the container.
Next I tested Beamr Video running inside Docker running on Windows and Linux. The results were inconsistent, with performance overhead varying from 7% up to 25%. I believe this is due to the storage driver I am using. Mounting an external volume should resolve this.
Lesson #4 – Bare-metal is still faster. Container configuration matters.
The next point I considered is the ease of configuration:
The bare-metal servers required BIOS configuration, OS installation, followed by the installation of Beamr Video. The software installation itself is managed with SaltStack (which is similar to Chef or Puppet). The same salt configuration was also used to install Beamr Video inside the various VMs and containers.
While the initial procedure is similar in both the virtualized and bare-metal cases, I believe the VMs and Docker containers have a great advantage here. Using VM snapshots allow easy deployment to multiple systems, moving between hosts, backups and on-demand usage.
If you host your own servers, and are running a production workload, bare-metal will indeed provide better performance. On the other-hand, with a performance penalty of less than 10%, come the benefits and convenience of firing-up pre-configured VMs on-demand, in the cloud or on premises. Such configuration could be appropriate for variable size workloads, handling peaks, testing, and evaluations.