Last week I finally created some virtual machines on my home server (virt-manager is a pain to work with compared to VirtualBox), and I wanted to make sure Kernel Samepage Merging is working. I have kernel 2.6.32, CONFIG_KSM enabled in kernel, and qemu-kvm has support for it. But KSM didn't work, because /sys/kernel/mm/ksm/pages_sharing always returned 0. It turns out, that to get it working, linux-headers and also glibc needs to support KSM. KSM support appeared in 2.6.32, so you need linux-headers-2.6.32, which is now available in portage, see gentoo bug #297755. So let's make KSM work on gentoo:
First download into your local overlay two ebuilds I made for you:
- sys-libs/glibc-2.11-r1 with KSM patch from piavlo
- app-emulation/ksm package has an initscript to enable ksm, and ksmtuned daemon, all from fedora qemu rpm
cd /var/tmp/
wget -O gentoo-ksm-ebuilds.tbz2 http://sites.google.com/site/buddsite/gentoo-ksm-ebuilds.tbz2
tar xjf gentoo-ksm-ebuilds.tbz2 -C /usr/local/portage/
(Make sure you have something like PORTDIR_OVERLAY="/usr/local/portage" in your /etc/make.conf)
Unmask the packages we want to emerge (binutils is a dependency for glibc):
echo '~sys-kernel/gentoo-sources-2.6.32' >> /etc/portage/package.keywords
echo '~sys-kernel/linux-headers-2.6.32' >> /etc/portage/package.keywords
echo '=sys-libs/glibc-2.11-r1' >> /etc/portage/package.keywords
echo '=sys-devel/binutils-2.20' >> /etc/portage/package.keywords
echo 'app-emulation/ksm' >> /etc/portage/package.keywords
Now emerge linux-headers and glibc with the KSM patch:
emerge -v1 linux-headers glibc
After this, you need to recompile qemu-kvm too. Plus emerge the ksm ebuild, because it has a nice initscript to enable ksm:
emerge -v1 qemu-kvm ksm
Now enable KSM:
/etc/init.d/ksm start
Start some virtual machines, and examine how much memory are you saving with KSM:
echo "KSM pages shared: $(( $(cat /sys/kernel/mm/ksm/pages_sharing) * $(getconf PAGESIZE) / 1024 / 1024 ))Mb"
You can also try out the ksmtuned daemon from fedora:
/etc/init.d/ksmtuned start
Hi
ReplyDeleteThanks for the ksm/ksmtuned staff.
Do you know by chance why the /sys/kernel/mm/ksm/max_kernel_pages
is missing in recent 2.6.33 kernels?
BTW linux-headers-2.6.32 is now in portage , so you need only the glibc patch.
Regards
Piavlo
Hi,
ReplyDeleteI have updated the post to use linux-headers-2.6.32.
I did not know max_kernel_pages disappeared in 2.6.33, thank you for bringing it to my attention.
I managed to find the commit which removed max_kernel_pages. It is not needed anymore because in 2.6.33 the KSM pages are swappable.
Also there is Red Hat bug #558281 which says:
"Now that the pages are swappable in 2.6.33 kernels, max_kernel_pages is no
longer necessary. The ksm init script has been modified to only change
/sys/kernel/mm/ksm/max_kernel_pages if it already exists. This should work
with both F12 and future rawhide kernels."
I could not find the updated initscript, when I'll find it, I'll update the ebuild to use that.
Thanks for digging it up.
ReplyDeleteps. The ksmtuned turned to be a terribly ugly hackish piece of crap IMHO.
I have found the new qemu rpm which in which the initscripts are 2.6.33 aware. Updated the ebuild bundle to include the new version.
ReplyDeleteI have updated the ebuilds, because one of the rpm packages disappeared. No functional changes though.
ReplyDeleteIt looks like the glibc patch is not needed anymore with qemu-kvm-0.12.3-r1: http://bugs.gentoo.org/305785
ReplyDeleteI did not try it though.
please update ebuild.
ReplyDeleteThe RPM package is broken again.
Thanks