Bug 53761 - nVMX: MSR bitmap merging
Summary: nVMX: MSR bitmap merging
Status: NEW
Alias: None
Product: Virtualization
Classification: Unclassified
Component: kvm (show other bugs)
Hardware: All Linux
: P1 enhancement
Assignee: virtualization_kvm
URL:
Keywords:
Depends on:
Blocks: 94971 53601
  Show dependency tree
 
Reported: 2013-02-13 09:38 UTC by Nadav Har'El
Modified: 2015-03-17 03:53 UTC (History)
0 users

See Also:
Kernel Version:
Subsystem:
Regression: No
Bisected commit-id:


Attachments

Description Nadav Har'El 2013-02-13 09:38:45 UTC
The current code already exposes the MSR bitmap feature (CPU_BASED_USE_MSR_BITMAPS) to L1, which L1 can use to run an L2 without getting exits on particular MSRs.
However, what L0 actually does now is to run L2 without an MSR bitmap, so any MSR access in L2 does cause an exit to L0 - but at this point we uses the L1-specified MSR bitmap to decide whether a nested exit (from L2 to L1) is necessary, or we should proceed in L2.

This behavior saves us the very costly L2->L1 exit, but in some cases even the exit to L0 is avoidable - when both L0 and L1 specify in their MSR bitmap that they don't care about a particular MSR, there is no reason to exit at all. For example, KVM disables intercept of a bunch of MSRs, so if we run nested KVM, we could allow L2 to use these MSRs without exits.

To do this, L0 needs to run L2 with an MSR bitmap created by merging the MSR bitmap from vmcs01 (L0's desire) with the MSR bitmap from vmcs12 (L1's desire).

In more detail, we can do this:

1. In prepare_vmcs02, if vmcs12 has msr_bitmap, and nested_cpu_has_vmx_msr_bitmap, and if we haven't already done
so, allocate a new "shadow" bitmap.

2. We know vmcs01 (L0's desired) msr-bitmap: it is always (see setup_msrs()) either vmx_msr_bitmap_longmode or vmx_msr_bitmap_legacy, depending on longmode.

3. In prepare_vmcs02, we need to "or" the msr bitmap of vmcs01 (see above) and vmcs12. The former never changes (actually, can switch between two), but the latter might change every time prepare_vmcs02 is called, so:

4. To avoid the slow ORing of 4K on every L1->L2 entry (step 4), we need to
know if the L1's msr bitmap page's content (in guest memory) changed. In some cases (like original EPT) we can't use the dirty bit on that page to know if it changed since the last merge, so we need to use tracing (making the page read-only and catch when it is written to). We assume L1 won't write to that page much (KVM doesn't - it only fills the bitmaps in vmx_init()).

5. Remember to also do the merge if the MSR page pointer changed (and in this case, we also need to change the tracing).

6. In prepare_vmcs02, don't remove the msr bitmap feature - remove the line:
    exec_control &= ~CPU_BASED_USE_MSR_BITMAPS;

7. Remember to free the shadow msr bitmap when freeing vmcs02.
Comment 1 Nadav Har'El 2013-02-25 13:59:58 UTC
The same technique, and code, can be reused also for merging of IO bitmaps.
This will offer less of a performance impact than that on MSR bitmaps, because the KVM L0 anyway wants exit on all IO ports (except the esoteric port 0x80 used for delays). So I am guessing no performance improvement will be seen because of merging IO bitmaps.

Note You need to log in before you can comment on or make changes to this bug.