Xen 4.21 regression

[tklengyel]

The following regression is observed with Xen 4.21 running Windows 10 guests with the viridian=1 flag set in the domain config:

(XEN) d1v1 Unexpected vmexit: reason 49
(XEN) domain_crash called from arch/x86/hvm/vmx/vmx.c:4849
(XEN) Domain 1 (vcpu#1) crashed on cpu#2:
(XEN) ----[ Xen-4.21.0  x86_64  debug=y  Not tainted ]----
(XEN) CPU:    2
(XEN) RIP:    0010:[<fffff806622b1967>]
(XEN) RFLAGS: 0000000000050282   CONTEXT: hvm guest (d1v1)
(XEN) rax: fffff67e27a809cc   rbx: 0000000000000000   rcx: fffff67e27a809d0
(XEN) rdx: 00000b1985be6000   rsi: 00000197ad6669cc   rdi: fffff67e27a809cc
(XEN) rbp: ffff87829aad8519   rsp: ffff87829aad83c8   r8:  0000000000000358
(XEN) r9:  0000000000000000   r10: ffff9183464a4910   r11: 00000197ad666d24
(XEN) r12: 0000000000000358   r13: ffff87829aad84c0   r14: 000000000000006e
(XEN) r15: ffff87829aad8498   cr0: 0000000080050031   cr4: 0000000000370678
(XEN) cr3: 000000003e6a7002   cr2: 00007ffaa8d3e510
(XEN) fsb: 0000000000000000   gsb: ffffe6810ffa8000   gss: 000000000029b000
(XEN) ds: 002b   es: 002b   fs: 0053   gs: 002b   ss: 0000   cs: 0010

[tklengyel]

With viridian=0 or viridian=['base'] a VCPU pause/unpause deadlock appears.

[lfreijo]

I used Claude Code to test this on my local machine and it identified this:

The DRAKVUF crash with the syscalls plugin is caused by a Xen bug, not a DRAKVUF bug.

Issue Chain:

1. DRAKVUF syscalls plugin creates many breakpoints (syscall entry + return hooks for each syscall)
2. Each breakpoint requires DRAKVUF to set up altp2m page table entries for shadow pages
3. p2m_alloc_ptp() (xen/arch/x86/mm/p2m.c:462) allocates pages from the domain's paging memory pool
4. When memory is exhausted, p2m_alloc_ptp() returns INVALID_MFN, causing p2m_set_entry() to return -ENOMEM (-12)
5. In altp2m.c:302-307, when p2m_set_entry() fails, Xen crashes the domain instead of returning an error:
   rc = p2m_set_entry(ap2m, gfn, amfn, cur_order, *p2mt, *p2ma);
   if ( rc )
   {
   gprintk(XENLOG_ERR, "failed to set entry... rc %d\n", ...);
   domain_crash(ap2m->domain); // <-- BUG: Should not crash!
   }

Evidence from Xen dmesg:
d19v0 failed to set entry for 249c8 -> 14d5c8 altp2m 0, rc -12
domain_crash called from arch/x86/mm/altp2m.c:307

This is a Xen 4.21 regression - the domain should not be crashed when altp2m runs out of p2m memory. The error should be propagated back
to the caller (DRAKVUF/libvmi) which can then handle it gracefully.

Adding shadow_memory = 256 to the vm config file seems to be a work around for now. I did not test it for a long period to see if it eventually ran out of memory again.

[tklengyel]

When it starts up it usually works fine, try to stop it with CTRL+C and restart it. That's when I start to see either the whole system freeze up or vmexit: reason 49