Crap Nullpointer Exceptions

This is the blog of John Dulaney, a hacker of Fedora, SCAdian, player of Music, blacksmith, sailor, and consumer of Bacon.

Tuning Fedora as a hypervisor

For those of you that use Fedora to host virtual machines, especially if you are using kvm and libvirt, here are some easy tunings to put into place that will improve virtual machine performance.

vm.dirty_background_ratio = 3
vm.dirty_ratio = 15
vm.dirty_expire_centisecs = 500
vm.dirty_writeback_centisecs = 100
vm.swappiness = 10
kernel.sched_migration_cost_ns = 5000000

These settings set the maximum percentages of dirty pages in active and total memory, increase the nimbleness of cleaning them out, set your swappiness to minimum, and reduce the chances of cpu migration.  These settings go into /etc/sysctl.conf

For most, these tunings alone will really improve performance.  However, if you have a system with plenty of memory and generally have multiple vms running all the time, you may want to consider putting hugepages into place.  Hugepages improve the efficiency of memory utilization by breaking memory up into larger chunks, or pages, than is the default.  On most x86_64 systems, hugepages are 2048kb in size.

Before you decide to enable hugepages, you will need to make sure that you have plenty of ram.  A vm that is set to use 1024MB requires 576 hugepages, or 1152MB.  Luckily, adding additional ram to vms or running multiple vms causes this utilization to be fairly linear.  You’ll also want to be sure to keep enough free ram available for anything else you’ll be doing on your system; for most use cases, hugepages shouldn’t take up more than half of the system’s ram.

A good general rule of thumb for most x86_64 systems is to figure out the total ram, in kb, for all the vms you’ll want to use hugepages, then divide by the hugepage size (generally 2048 kb) then add thirty or so, rounding up to the nearest hundred (if the nearest hundred requires rounding down, round to the fifty).  This is your starting hugepage number; experimentation may be required to adjust this for your use case.

To set the number of hugepages, once again, edit /etc/sysctl.conf and add the following line:

vm.nr_hugepages = <number calculated above>

Then, you’ll want to reboot your system to make sure that your memory gets cleared and hugepages get reserved.

Finally, you’ll want to edit each of the vms’ xmls with virsh edit <vm name> and include the following:

<memoryBacking>
<hugepages/>
</memoryBacking>

Then, when you start your virtual machine(s), check your hugepage utiliziation by running grep Huge /proc/meminfo

[jdulaney@volsung ~]$ grep Huge /proc/meminfo
AnonHugePages:    120832 kB
HugePages_Total:    3000
HugePages_Free:     2424
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

About these ads

One response to “Tuning Fedora as a hypervisor

  1. Bob February 19, 2014 at 5:17 am

    Hi, vm.dirty* numbers depend on how much RAM do you have and are defferent from a desktop to a server with 64GB RAM. There are other experiments that show for example vm.dirty_ratio = 10 is the best for KVM virtualization. Maybe it is better to write how these numbers affect performance. But thank you.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 50 other followers

%d bloggers like this: