virtualization-mem-balloon
Overview
Memory ballooning is a memory management feature used in most virtualization platforms which allows a host system to artificially enlarge its pool of memory by taking advantage or reclaiming unused memory previously allocated to various virtual machines.
This is achieved through a balloon driver which is installed on the guest operating system which the hypervisor communicates with when it needs to reclaim memory through ballooning.
Through memory ballooning, a host server can reclaim unused memory from other less busy virtual machines and reassign it to ones that require it more. Theoretically, a server with 32GB of memory might be able to support a combined virtual machine memory capacity allocation of 64GB simply because all of those virtual machines will not be using the maximum amount of memory they have been assigned at the same time.
Inside
How balloon driver works
VirtIO provides Memory Ballooning: the host system can reclaim memory from virtual machines (VM) by telling them to give back part of their memory to the host system. This is achieved by inflating the memory balloon(balloon driver) inside the VM, which reduced the memory available to other tasks inside the VM. Which memory pages are given back is the decision of the guest operating system (OS): It just tells the host OS which pages it does no longer need and will no longer access. The host OS then un-maps(kvm in kernel) those pages from the guests and marks them as unavailable for the guest VM. The host system can then use them for other tasks like starting even more VMs or other processes.
If later on the VM need more free memory itself, the host can later on return pages to the guest and shrink the holes. This allows to dynamically adjust the memory available to each VM even while the VMs keep running.
The balloon driver(virtio_balloon) in each guest operating system keeps track of the excess memory of each VM and when the hypervisor calls for a memory reclamation through ballooning, the balloon driver in the VM pins down a specific amount of memory so that the VM cannot consume it, and then the hypervisor reclaims that pinned memory for reallocation. If there is a scarcity of unused memory then a memory swap might be initiated in order to fulfill the balloon quota. If this happens too much, there would be a lot of I/O overhead between the various VMs that are doing memory swapping with the disk and might adversely affect overall performance of the virtual system
The ballooning driver inflation is a smart way to claim memory from the virtual machine to the host. It’s beneficial because it will only generally claim back memory which is free inside the virtual machine, thus is usually a non-disruptive memory reclaim technique.
Cost of memory balloon
Memory ballooning while an impressive solution can cause issues to virtual machines that have reoccurring spikes in memory demand. Also, ballooning will not work if your virtual machines are using all of their memory by applications within the virtual machine, this is commonly noticed in applications such as databases.
Ballooning can also become a problem if it’s relied upon too much by the hypervisor. In an ideal situation, there would be no ballooning taking place. This would indicate a healthy environment. Ballooning would only commence if there is too much demand for memory on the host, in other words when the ESXi host does not have enough free physical memory to allocate to virtual machines. If ballooning is happening all the time then a performance issue can start on the host.
Many ballooning operations could cause additional CPU cycles to be used to perform the ballooning operations. This, in turn, could reduce the amount of physical CPU available to virtual machines on the host.
Suggestion
- Set proper balloon size of each VM
- Disable memory balloon for vm used for database
libvirt
1 | # disable memballoon |