2010/01/26

Enabling KSM on gentoo

Update: linux-headers-2.6.32 is now available in gentoo, i have updated the post to use that package, instead of patched 2.6.30. The patched linux-headers-2.6.30 is still available in the ebuild bundle if somebody needs it.

Last week I finally created some virtual machines on my home server (virt-manager is a pain to work with compared to VirtualBox), and I wanted to make sure Kernel Samepage Merging is working. I have kernel 2.6.32, CONFIG_KSM enabled in kernel, and qemu-kvm has support for it. But KSM didn't work, because /sys/kernel/mm/ksm/pages_sharing always returned 0. It turns out, that to get it working, linux-headers and also glibc needs to support KSM. KSM support appeared in 2.6.32, so you need linux-headers-2.6.32, which is now available in portage, see gentoo bug #297755. So let's make KSM work on gentoo:

First download into your local overlay two ebuilds I made for you:
  • sys-libs/glibc-2.11-r1 with KSM patch from piavlo
  • app-emulation/ksm package has an initscript to enable ksm, and ksmtuned daemon, all from fedora qemu rpm

cd /var/tmp/
wget -O gentoo-ksm-ebuilds.tbz2 http://sites.google.com/site/buddsite/gentoo-ksm-ebuilds.tbz2
tar xjf gentoo-ksm-ebuilds.tbz2 -C /usr/local/portage/

(Make sure you have something like PORTDIR_OVERLAY="/usr/local/portage" in your /etc/make.conf)

Unmask the packages we want to emerge (binutils is a dependency for glibc):
echo '~sys-kernel/gentoo-sources-2.6.32' >> /etc/portage/package.keywords
echo '~sys-kernel/linux-headers-2.6.32' >> /etc/portage/package.keywords
echo '=sys-libs/glibc-2.11-r1' >> /etc/portage/package.keywords
echo '=sys-devel/binutils-2.20' >> /etc/portage/package.keywords
echo 'app-emulation/ksm' >> /etc/portage/package.keywords


Now emerge linux-headers and glibc with the KSM patch:
emerge -v1 linux-headers glibc


After this, you need to recompile qemu-kvm too. Plus emerge the ksm ebuild, because it has a nice initscript to enable ksm:
emerge -v1 qemu-kvm ksm


Now enable KSM:
/etc/init.d/ksm start


Start some virtual machines, and examine how much memory are you saving with KSM:
echo "KSM pages shared: $(( $(cat /sys/kernel/mm/ksm/pages_sharing) * $(getconf PAGESIZE) / 1024 / 1024 ))Mb"


You can also try out the ksmtuned daemon from fedora:
/etc/init.d/ksmtuned start

7 comments:

  1. Hi
    Thanks for the ksm/ksmtuned staff.

    Do you know by chance why the /sys/kernel/mm/ksm/max_kernel_pages
    is missing in recent 2.6.33 kernels?

    BTW linux-headers-2.6.32 is now in portage , so you need only the glibc patch.

    Regards
    Piavlo

    ReplyDelete
  2. Hi,

    I have updated the post to use linux-headers-2.6.32.

    I did not know max_kernel_pages disappeared in 2.6.33, thank you for bringing it to my attention.

    I managed to find the commit which removed max_kernel_pages. It is not needed anymore because in 2.6.33 the KSM pages are swappable.

    Also there is Red Hat bug #558281 which says:
    "Now that the pages are swappable in 2.6.33 kernels, max_kernel_pages is no
    longer necessary. The ksm init script has been modified to only change
    /sys/kernel/mm/ksm/max_kernel_pages if it already exists. This should work
    with both F12 and future rawhide kernels."

    I could not find the updated initscript, when I'll find it, I'll update the ebuild to use that.

    ReplyDelete
  3. Thanks for digging it up.
    ps. The ksmtuned turned to be a terribly ugly hackish piece of crap IMHO.

    ReplyDelete
  4. I have found the new qemu rpm which in which the initscripts are 2.6.33 aware. Updated the ebuild bundle to include the new version.

    ReplyDelete
  5. I have updated the ebuilds, because one of the rpm packages disappeared. No functional changes though.

    ReplyDelete
  6. It looks like the glibc patch is not needed anymore with qemu-kvm-0.12.3-r1: http://bugs.gentoo.org/305785
    I did not try it though.

    ReplyDelete
  7. Anonymous04:06

    please update ebuild.
    The RPM package is broken again.
    Thanks

    ReplyDelete