This is the blog of John Dulaney, a hacker of Fedora, SCAdian, player of Music, blacksmith, sailor, and consumer of Bacon.
Tuning Fedora as a hypervisor
February 18, 2014Posted by on
For those of you that use Fedora to host virtual machines, especially if you are using kvm and libvirt, here are some easy tunings to put into place that will improve virtual machine performance.
vm.dirty_background_ratio = 3
vm.dirty_ratio = 15
vm.dirty_expire_centisecs = 500
vm.dirty_writeback_centisecs = 100
vm.swappiness = 10
kernel.sched_migration_cost_ns = 5000000
These settings set the maximum percentages of dirty pages in active and total memory, increase the nimbleness of cleaning them out, set your swappiness to minimum, and reduce the chances of cpu migration. These settings go into /etc/sysctl.conf
For most, these tunings alone will really improve performance. However, if you have a system with plenty of memory and generally have multiple vms running all the time, you may want to consider putting hugepages into place. Hugepages improve the efficiency of memory utilization by breaking memory up into larger chunks, or pages, than is the default. On most x86_64 systems, hugepages are 2048kb in size.
Before you decide to enable hugepages, you will need to make sure that you have plenty of ram. A vm that is set to use 1024MB requires 576 hugepages, or 1152MB. Luckily, adding additional ram to vms or running multiple vms causes this utilization to be fairly linear. You’ll also want to be sure to keep enough free ram available for anything else you’ll be doing on your system; for most use cases, hugepages shouldn’t take up more than half of the system’s ram.
A good general rule of thumb for most x86_64 systems is to figure out the total ram, in kb, for all the vms you’ll want to use hugepages, then divide by the hugepage size (generally 2048 kb) then add thirty or so, rounding up to the nearest hundred (if the nearest hundred requires rounding down, round to the fifty). This is your starting hugepage number; experimentation may be required to adjust this for your use case.
To set the number of hugepages, once again, edit /etc/sysctl.conf and add the following line:
vm.nr_hugepages = <number calculated above>
Then, you’ll want to reboot your system to make sure that your memory gets cleared and hugepages get reserved.
Finally, you’ll want to edit each of the vms’ xmls with virsh edit <vm name> and include the following:
Then, when you start your virtual machine(s), check your hugepage utiliziation by running grep Huge /proc/meminfo
[jdulaney@volsung ~]$ grep Huge /proc/meminfo
AnonHugePages: 120832 kB
Hugepagesize: 2048 kB