VMX Instruction Set and VMCS Structure Fully Explained
The VMX instruction set is the only channel between the VMM and hardware, and the VMCS holds all VM state. Mastering instruction semantics and field layout is the foundation for writing VT debugger core logic such as VM-exit dispatch and EPT control.
Core Concepts
VMXON / VMXOFF
VMXON requires CR0.PE=1, CR4.VMXE=1 and paging enabled; failure is reported via CF/ZF flags. VMXOFF exits VMX operation.
VMPTRLD / VMPTRST
VMPTRLD loads the current-VMCS pointer and VMPTRST saves it. Only one VMCS is current at any time.
VMLAUNCH / VMRESUME
VMLAUNCH launches a VMCS for the first time; subsequent entries must use VMRESUME or a #VMXERR/#GP occurs.
VMREAD / VMWRITE
VMREAD/VMWRITE access VMCS fields by encoding. The high 16 bits select guest/host/control areas; the low 16 bits select width and field.
VMX Instruction Failure
Most VMX instructions set CF=1 and write the VM-instruction error field on failure; ZF=1 indicates an invalid current VMCS.
VM-exit Reason Codes
The VM-exit reason field is 32-bit: low 16 bits are the reason (e.g., 0x0E=EPT violation, 0x00=exception), high bits hold nested flags.
Key Code
; 在 VM-exit 处理中读取退出原因（汇编示意）
vmx_handle_exit:
vmread rax, [VM_EXIT_REASON] ; 编码 0x4402
and eax, 0xFFFF ; 取低 16 位原因码
cmp eax, 14 ; 14 = EPT Violation
je ept_violation_handler
cmp eax, 0 ; 0 = Exception or NMI
je exception_handler
; ... 其他原因分发
vmxresume ; 回到客户机继续执行Structure Cheat Sheet
| Guest-State Area | Guest registers and system state (CR0/CR3/CR4, IDTR/GDTR, MSRs) |
|---|---|
| Host-State Area | Host state loaded by the CPU after VM-exit |
| VM-Execution Control | Controls which instructions/events cause VM-exit (pin/proc/exit/entry groups) |
| VM-Exit Control | VM-exit behavior and MSR save/load list configuration |
| VM-Entry Control | VM-entry behavior, event injection and MSR list configuration |
| VM-Exit Information | Reason code, qualifiers and interrupt info of the last VM-exit |
Related Reading
A systematic introduction to virtualization concepts, evolution and taxonomy, and the roles of Intel VT-x and AMD-V.
→A retrospective of three decades of x86 hardware virtualization: binary translation, Intel VT-x, AMD SVM, EPT/NPT, and the modern virtualization security ecosystem.
→In-depth explanation of Intel VT-x VMX root/non-root modes, the VM-exit/VM-entry mechanism, and the VMCS virtual machine control structure.
→Explains VMX root/non-root dual-mode switching, the VM-exit event flow, and how VT debuggers exploit dual modes for invisible monitoring.
→How nested virtualization works: nested VMCS/VMCB, shadow-VMCS optimization, and its use in WSL2 and cloud environments.
→A full breakdown of AMD SVM (Secure Virtual Machine): VMCB, #VMEXIT, guest/host modes and AMD's hardware virtualization design.
→Field-by-field analysis of the AMD VMCB memory layout: control area, save area, intercept bitmap and precise offsets.
→Explains AMD NPT two-level address translation, the nCR3 root pointer, TLB control, and its impact on performance and security monitoring.
→Explains VMX root/non-root dual-mode switching, the VM-exit event flow, and how VT debugge…
→A systematic comparison of AMD SVM vs Intel VT-x: mode design, state structures, interception, nested paging, and ecosystem support.
→In-depth explanation of Intel VT-x VMX root/non-root modes, the VM-exit/VM-entry mechanism…
→Explains AMD-Vi IOMMU DMA remapping, interrupt remapping and device passthrough, plus its security roles.
→A systematic introduction to virtualization concepts, evolution and taxonomy, and the role…
→Step-by-step guide to enabling SVM on AMD platforms: BIOS entry points, naming differences, verification and common issues.
→An overview of the VT Debugger: hypervisor-layer isolation, EPT-based residue-free breakpoints, VM-exit event handling and invisible memory access.
→A feature-by-feature breakdown of the VT Debugger: invisible breakpoints, invisible hooks, kernel-level memory access, process protection and anti-anti-debugging.
→A complete getting-started tutorial from download and install to your first breakpoint: requirements, driver loading, attaching and troubleshooting.
→Quickly confirm CPU virtualization support and enablement via Task Manager, systeminfo, CPU-Z and command-line methods.
→Deep dive into the two core VT Hook implementations: EPT page-remapping hooks and write-protect hooks, and how they defeat integrity checks.
→From the detection surface of traditional breakpoints to EPT page-level and virtualized hardware breakpoints: implementation and anti-detection power.
→The VT-layer process protection stack: EPT memory hiding, TerminateProcess interception, anti-injection and anti-debugging.
→The role of virtualization in anti-cheat: from kernel-level detection to hypervisor-grade monitoring, and the bypass/counter-bypass arms race.
→