Blocking NtReadFile freezes all guest compilation once DEP is disabled

[AviKndr]

BTCpuNotifyReadFile holds ThreadCreationMutex and the process-wide
CodeInvalidationMutex — exclusively — for the entire duration of the read it
brackets. Every other pBTCpuNotify* hook wraps a memory-management syscall
that returns promptly; NtReadFile can block indefinitely. While it does, no
other guest thread in the process can compile code, so any thread that takes a
dispatcher miss deadlocks until the read completes.

Affects both front ends: Source/Windows/WOW64/Module.cpp and
Source/Windows/ARM64EC/Module.cpp.

Why it isn't hit constantly

The locked path is only taken when BeginUntrackedWriteLocked returns true,
i.e. the read destination overlaps an RWX interval. An ordinary stack buffer
doesn't, so normally both locks are dropped before the read.

That changes when DEP is switched off process-wide. Any 32-bit DLL without
IMAGE_DLLCHARACTERISTICS_NX_COMPAT causes Wine's loader to disable DEP, and
InvalidationTracker::HandleProcessExecuteFlagsChange then sweeps the whole
address space and promotes every committed readable+writable region — thread
stacks and heaps included — into RWXIntervals. From that point, a plain
blocking read into a stack buffer is a global compilation barrier.

Real-world impact

msiexec /i <package>.msi /q hangs on packages with 32-bit custom actions.
msiexec's custom-action host (programs/msiexec/msiexec.c, custom_action_server)
loops on a blocking ReadFile of the next action's GUID into a stack local. The
parent is simultaneously in WaitForSingleObject(thread, INFINITE) on the action
thread it just handed over, and that thread needs the invalidation mutex shared
for its first dispatcher miss. Nothing proceeds.

Characteristically, the first custom action succeeds and the second never does
— the first runs before the non-NX_COMPAT DLL loads and disables DEP.

This isn't MSI-specific: any 32-bit guest that loads such a DLL and then blocks
in a read while another thread runs new code will hang. Older installers and
games commonly do both.

Minimal reproduction

No MSI needed. A 32-bit guest that:

1. Disables DEP — NtSetInformationProcess(ProcessExecuteFlags) with
   MEM_EXECUTE_OPTION_ENABLE (0x02; 0x01 is _DISABLE).
2. Creates a pipe with nothing to read.
3. Blocks its main thread in ReadFile into a stack buffer.
4. Has a second thread execute never-before-run code (e.g. a few LoadLibraryA
   calls) while that read is outstanding.

With DEP left enabled the worker completes immediately and the read returns on
schedule. With DEP disabled the whole process wedges until the read is
interrupted.

Confirming it's this code path and not something else: FEX_SMCCHECKS=none
makes the hang disappear, because SMCDetectionDisabled short-circuits
ProtectRWXIntervalsInternal, so BeginUntrackedWriteLocked returns false and
the locks are released before the read.

Environment

FEX a6e74cdb, arm64 macOS host, Wine 11.16 WoW64. Reproduces on both the
WoW64 and ARM64EC front ends.

Suggested direction

Two independent points, either of which would fix it:

• Don't hold process-wide locks across NtReadFile. Untrap and invalidate
  before the syscall, then invalidate the written range afterwards — invalidating
  after the write is sound, since it discards anything compiled from a
  partially-written buffer. Something is needed to stop a concurrent trap
  re-protecting the pages mid-read.
• Don't treat DEP-promoted intervals as RWX for this hook. Those exist because
  DEP was disabled, not because the guest declared the memory executable, and
  a read destination in one is far more likely a stack buffer than code.