Virtualization research
Fast forward to
AMD's Operating System Research Center works with the operating-system vendors and the open-source community to enable hypervisor products for AMD-V™, AMD's virtualization technology. We identify bottlenecks and propose both software and hardware enhancements to address the problems we have found.
AMD-V's nested-paging technology simplifies hypervisor implementations and improves performance dramatically. We have added support for nested paging to the Xen and Linux KVM hypervisors. Please find information on the performance improvement we observed in the Publications section.
Lock-holder preemption refers to a problem where a hypervisor preempts a guest's virtual CPU that has acquired a spin lock and is currently executing a critical section. When another virtual CPU needs to acquire the same lock, its spin wait will take longer—until the preempted vCPU is rescheduled and completes its critical section.
We have observed that the Xen hypervisor is susceptible to lock-holder preemption. The problem is exacerbated when the guest OS is a version of Linux that uses ticket spin locks.
We have proposed a solution for lock-holder preemption that prevents overly long waits. Please find more information in the Publications section.
How to Deal with Lock-Holder Preemption Thomas Friebel. Presentation at the Xen Summit North America, July 2008. Boston, MA
Lock-holder preemption is the preemption of a virtual CPU (VCPU) holding a spinlock. Other VCPUs of the same guest that try to acquire the same lock will have to wait until the lock-holder is scheduled again and releases the lock. On a multi-core machine, lock-holder preemption can cause Xen guests to waste about 7% of their time waiting for spinlocks. In this presentation we will show the effects of lock-holder preemption, show two ways to counteract it, and analyze one approach in detail. We will give a short overview of our modifications to the Xen scheduler, and show how we regained the lost performance.
Extended abstract: PDF
Talk: PDF, PDF with comments
Nested paging hardware and software Benjamin Serebrin, Joerg Roedel. Presentation at the KVM Forum, June 2008. Napa, CA
This presentation covers the ASPLOS paper 'Accelerating two-dimensional page walks for virtualized systems', and implementation details and performance of nested paging support for KVM.
Talk: PDF
Accelerating two-dimensional page walks for virtualized systems Ravi Bhargava, Benjamin Serebrin, Francesco Spadini, Srilatha Manne. In the proceedings of the 13th international conference on Architectural support for programming languages and operating systems (ASPLOS), March 2008. Seattle, WA
Nested paging is a hardware solution for alleviating the software memory management overhead imposed by system virtualization. Nested paging complements existing page walk hardware to form a two-dimensional (2D) page walk, which reduces the need for hypervisor intervention in guest page table management. However, the extra dimension also increases the maximum number of architecturally-required page table references.
This paper presents an in-depth examination of the 2D page table walk overhead and options for decreasing it. These options include using the AMD Opteron processor's page walk cache to exploit the strong reuse of page entry references. For a mix of server and SPEC benchmarks, the presented results show a 15%-38% improvement in guest performance by extending the existing page walk cache to also store the nested dimension of the 2D page walk. Caching nested page table translations and skipping multiple page entry references produce an additional 3%-7% improvement.
Much of the remaining 2D page walk overhead is due to low-locality nested page entry references, which result in additional memory hierarchy misses. By using large pages, the hypervisor can eliminate many of these long-latency accesses and further improve the guest performance by 3%-22%.
Paper: PDF
Partitioning the physical TLB with SVM ASIDs Sebastian Biemueller. Presentation at Xen Summit, April 2007. Yorktown Heights, NY
Slide deck used at the 2007 Xen Summit.
Talk: PDF
Nested paging support in Xen Wei Huang. Presentation at Xen Summit, April 2007. Yorktown Heights, NY
Slide deck used at the 2007 Xen Summit including an introduction to the AMD Barcelona technology by Elsie Wahlig.
Talk: PDF
|
| Contact 