Basic Virtual Memory Implementation Fixes & Improvements #165

nemecad · 2025-09-04T10:53:57Z

This PR addresses some previous feedback, most notably:

Set-associative TLB (machine::TLB): Implements a set-associative Translation Lookaside Buffer (TLB) frontend over physical memory, handling virtual to physical translation, flush, and replacement policy.
Pluggable Replacement Policies (machine::TLBPolicy): Abstract TLB replacement policy interface & implementations (RAND, LRU, LFU, PLRU) for set-associative tables.
SV32 Page-Table Walker (machine::PageTableWalker): Performs multi-level page-table walks (SV32) in memory to resolve a virtual address to a physical one.
Sv32Pte Bitfield Helpers (sv32.h): SV32-specific definitions: page-table entry (PTE) bitfields, shifts/masks, and PTE to physical address helpers.
VirtualAddress (virtual_address.h): Lightweight VirtualAddress wrapper offering raw access, alignment checks, arithmetic, and comparisons.
Add supervisor CSRs and sstatus handling: supervisor CSRs (sstatus, stvec, sscratch, sepc, scause, stval, satp) and a write handler that presents sstatus as a masked view of mstatus so supervisor-visible bits stay in sync.
Store current privilege level in CoreState: tracking of the hart's current privilege level in CoreState so exception/return handling and visualization can read/update it from the central CoreState structure.

Tests:

Add SV32 page-table + TLB integration tests: a set of small assembly tests that exercise the SV32 page-table walker, SATP enablement and the new TLB code. The tests create a root page table and map a virtual page at 0xC4000000, then exercise several scenarios. The tests verify page-table walker behaviour, SATP switching and TLB caching/flush logic. Tests were written based on the consultation.

UI Components:

Show current privilege level in core state view:

Virtual memory configuration to NewDialog:

TLB visualization and statistics dock:

VM toggle and "As CPU" memory access view:

src/gui/CMakeLists.txt

src/gui/dialogs/new/newdialog.cpp

src/gui/windows/tlb/tlbview.cpp

src/gui/windows/tlb/tlbview.h

src/machine/CMakeLists.txt

src/machine/machine.cpp

src/machine/machine.h

src/machine/machineconfig.cpp

src/machine/memory/frontend_memory.cpp

src/machine/memory/virtual/sv32.h

jdupak · 2025-09-28T18:14:08Z

I am getting this weird zoom.

jdupak · 2025-09-28T18:14:47Z

Notice that address sanitizer is failing in CI.

jdupak · 2025-09-28T18:15:58Z

src/gui/windows/tlb/tlbview.h

+        void tlb_update(unsigned way, unsigned set, bool valid, unsigned asid, quint64 vpn, quint64 phys, bool write);
+
+private:
+    const machine::TLB *tlb;


Who owns this pointer?

jdupak · 2025-09-28T18:17:38Z

src/machine/memory/tlb/tlb.h

+#include <cstdint>
+
+namespace machine {
+enum TLBType { PROGRAM, DATA };


Why does LTB need to know this?

ppisa · 2025-09-29T08:51:35Z

@jdupak thanks for review of interfacing to the memory model architecture.

ppisa · 2025-09-29T08:51:45Z

From my side, the changes to the processor pipeline diagram has been applied directly to the SVG files (src/gui/windows/coreview/schemas), but current design uses DRAW.IO source (extras/core_graphics) as the authoritative source of the pipeline visualization and SVGs are generated from this file. So the commit with SVG change should include extras/core_graphics/diagram.drawio change as well or extras/core_graphics/diagram.drawio change should be commit before SVG files regeneration commit. In long term, I would lean to single SVG file with tags for conditional rendering, but we have not got to that state yet and current solution implemented by @jdupak is based on DRAW.IO and exports controlled by tagging (some documentation there docs/developer/coreview-graphics/using-drawio-diagram.md).

ppisa · 2025-09-29T08:55:46Z

For memory view, I would not complicate it with Show virtual checkbox. I would use only switching between As CPU (VMA), Cached and Raw.

nemecad · 2025-10-05T19:38:26Z

@ppisa @jdupak Thank you for your detailed feedback. I appreciate it and have made some changes based on your review. I would be grateful for any further feedback.

jdupak · 2025-10-19T14:49:24Z

tests/cli/virtual_memory/template/program.S

There is a typo in the dir name

jdupak · 2025-10-19T14:50:16Z

tests/cli/virtual_memory/template/program.S

There is no cmake logic to run these tests. I think we want to run them as cli tests.

Thank you for the comment. I’ve added the CMake logic to run these as CLI tests in the commit 7a204cf.

jdupak · 2025-10-19T16:16:39Z

I pushed some slight edits. Barring the issue with new tests not being run I am fine with merging this.

ppisa · 2025-10-29T15:14:09Z

I am going through the code. I have one overall remark, that there are lot of formatting changes included in functional changes. I am not reluctant to formatting changes even that I think that sometimes formatting left by human to align for example some case lines into columns etc. has some value. But formatting changes unrelated to the functional changes make review harder. So I would keep with patches as they are but I would suggest to separate formatting, even over all later modified files in series, separate from functional changes.

ppisa · 2025-10-29T15:51:19Z

I am do not like is_mmio_region() and bypass_mmio() concept. The peripherals accesses should go through regular address translation. It is responsibility of the OS to map regions related to I/O into virtual address space of kernel and or even user application, i.e. for mmap() like accesses.

As for enabling cache for accesses there is a hack in the QtRvSim which enforces next uncached region

Cache::Cache
    uncached_start(0xf0000000_addr)
    uncached_last(0xfffffffe_addr)

In the longer term, cacheability should be controlled from page tables. But PBMT (Page-Based Memory Types) are supported only for Sv39 and bigger translation configurations, see Chapter 14. "Svpbmt" Extension for Page-Based Memory Types

Mode	Value	Requested Memory Attributes
PMA	0	None
NC	1	Non-cacheable, idempotent, weakly-ordered (RVWMO), main memory
IO	2	Non-cacheable, non-idempotent, strongly-ordered (I/O ordering), I/O
-	3	Reserved for future standard use

But the physical region marked to skip caching in cache implementation (current state) should be enough for now.

ppisa · 2025-10-29T16:17:04Z

Not so critical for now, but should be solved in the longer time perspective. SRET can be executed even in M mode. So the type of the return should be propagated to the control_state->exception_return in the Core::memory(const ExecuteInterstage &dt). It is question if to add signal which goes through all stages (more readable) or to use bit from instruction for local decode of the type. MRET should not be allowed in system mode. In general, I think that current version does not mark system level instructions and access to the system and machine mode CSRs as invalid in U mode. So some masking would be required on the decode level in future. Some more flags needs to be added into enum InstructionFlags to allow that checking and flags_to_check and it should then be updated on mode transition.

The behavior of xRET instructions is described in 3.1.6.1. Privilege and Global Interrupt-Enable Stack in mstatus register. When SRET is executed in M mode then it executes the same as in the S mode but it
should clear MPRV=0. This is to allow emulate some system level operations in machine level code.

ppisa · 2025-10-29T16:56:29Z

It seems that TLBs are updated from the start of the system. The TLB and its updates should be enable only when root register is set. And they should not be updated in M mode at all.

jdupak · 2025-10-29T18:49:40Z

There is one actual issue from CI: you cannot use ftruncate. It fails compilation on Win.

jdupak · 2025-10-30T10:41:10Z

There is one actual issue from CI: you cannot use ftruncate. It fails compilation on Win.

Never mind, this is broken on master. I will fix that. It does not block this PR.

jdupak · 2025-11-10T19:27:44Z

@nemecad notice that I force pushed your branch - there is zero diff at the end but all spurious format changes should be gone now. My apologies for introducing them.

CC @ppisa should be now easier to review

ppisa

Thanks to @nemecad for the virtual memory implementation. The code is clean and readable in general. Thanks to @jdupak for review and formatting.

There are some minor issues to resolve or discuss. Same some suggestion to history cleanup but I think that we can merge code soon.

To speedup discussion, I would like to meet or call with @nemecad.

But congratulation to good job generally. As the next step I would like to discuss possibility to work on Sv39 which would allow to test some more real operating system scenarios.

ppisa · 2025-11-11T07:49:16Z

src/machine/memory/tlb/tlb.cpp

+
+namespace machine {
+
+static bool is_mmio_region(uint64_t virt) {


This function should be removed from the history. The logic is corrected by the commit 667bd4f
But it would be much better, if it does not appear in the history at all.

ppisa · 2025-11-11T07:49:27Z

src/machine/memory/tlb/tlb.cpp

+    return false;
+}
+
+static Address bypass_mmio(Address vaddr) {


ppisa · 2025-11-11T07:56:28Z

src/gui/windows/memory/memorymodel.cpp

+                mem = machine->cache_data();
+            }
+        } else {
+            if (access_through_cache == 2) {


The modes should be changed to proper enum to make code readable.

The field name access_through_cache should be adjusted. Something like mem_access_kind, mem_access_level or some better name name.

The proposed enum and when I think about use there could be interesting to to have option to look to memory at virtual level even when CPU is in machine mode, because the you can observe what hypervisor or SBI does with some or system memory

enum MemoryAccessAtLevel { MEM_ACC_AS_CPU = 0, MEM_ACC_VIRT_ADDR = 1, MEM_ACC_PHYS_ADDR = 2, MEM_ACC_PHYS_ADDR_SKIP_CACHES = 3, MEM_ACC_AS_MACHINE = 4, };

ppisa · 2025-11-11T08:00:09Z

tests/cli/virtual_memory/itlb/program.S

This commit and previous one should be squashed or kept but location of the test files as they are introduced in the previous commit should be already the final one. The move is abundant in new component history.

ppisa · 2025-11-11T08:03:00Z

src/machine/instruction.cpp

    flags = (enum InstructionFlags)im.flags;
    alu_op = im.alu;
    mem_ctl = im.mem_ctl;
+    if (flags & IMF_CSR) {


I agree with this additional logic to map CSR to required privilege level. But see proposed testing remark.

ppisa · 2025-11-11T08:23:57Z

src/machine/core.cpp

    ExceptionCause excause = dt.excause;

    dt.inst.flags_alu_op_mem_ctl(flags, alu_op, mem_ctl);
+    auto current_priv = state.current_privilege();


I would suggest to use some masking there. Ideally included directly by updates of check_inst_flags_val and check_inst_flags_mask when which would be updated when the privilege level changes. This would allow speedup and even check on individual instruction level when mret, sret etc, can be augmented by required privilege level directly in the instruction table. This common illegal instruction processing is possible because privilege violation should lead to regular illegal instruction exception. See

2.1. CSR Address Mapping Conventions

Instructions that access a non-existent CSR are reserved. Attempts to access a CSR without appropriate privilege level raise illegal-instruction exceptions or, as described in Section 21.6.1, virtual-instruction exceptions. Attempts to write a read-only register raise illegal-instruction exceptions. read/write register might also contain some bits that are read-only, in which case writes to the read-
only bits are ignored.

Add tracking of the hart's current privilege level to the core state so code handling exceptions/returns and visualization can read/update it from the central CoreState structure.

The next supervisor CSRs has been added: sstatus, stvec, sscratch, sepc, scause, stval, satp Write handler has been added as well. It presents sstatus as a masked view of mstatus so supervisor-visible bits stay in sync.

ppisa

I have added comments and documented some which has been already expressed in discussion.

I have noticed some problems in rv32ui-p-fence_i official RISC-V tests. It is in cached variant regardless of pipeline/single-cycle and 32/64/bits variants. @jdupak it is strange that the failure of given/single official test does not propagate to failure of whole test series.

Problem seems to appear in some change after Machine: add supervisor CSRs and status handling commit or it could be introduced by my rearrangement of the changes.

rv32ui-p-fence_i: ERROR
[INFO]  machine.ProgramLoader:	Loaded executable: 32bit
[INFO]  machine.TLB:	TLB[I] constructed; sets=16 way=1
[INFO]  machine.TLB:	TLB[D] constructed; sets=16 way=1
[INFO]  machine.TLB:	TLB: SATP changed → flushed all; new SATP=0x00000000
[INFO]  machine.TLB:	TLB: SATP changed → flushed all; new SATP=0x00000000
[INFO]  machine.TLB:	TLB: SATP changed → flushed all; new SATP=0x00000000
[INFO]  machine.TLB:	TLB: SATP changed → flushed all; new SATP=0x00000000
[INFO]  machine.BranchPredictor:	Initialized branch predictor: None
[INFO]  machine.TLB:	TLB[D]: flushed all entries
[INFO]  machine.TLB:	TLB[I]: flushed all entries
[DEBUG] machine.core:	Exception cause 11 instruction PC 0x80000180 next PC 0x80000184 jump branch PC 0x8000017cregisters PC 0x80000184 mem ref 0x00000000

Machine stopped on ECALL_M exception.

ppisa · 2025-11-15T10:28:56Z

src/gui/windows/memory/memorymodel.cpp

+                mem = machine->cache_data();
+            }
+        } else {
+            if (access_through_cache == 2) {


The proposed enum and when I think about use there could be interesting to to have option to look to memory at virtual level even when CPU is in machine mode, because the you can observe what hypervisor or SBI does with some or system memory

enum MemoryAccessAtLevel { MEM_ACC_AS_CPU = 0, MEM_ACC_VIRT_ADDR = 1, MEM_ACC_PHYS_ADDR = 2, MEM_ACC_PHYS_ADDR_SKIP_CACHES = 3, MEM_ACC_AS_MACHINE = 4, };

ppisa · 2025-11-15T10:59:02Z

src/machine/core.cpp

+            }
+            if (auto data_tlb = dynamic_cast<TLB *>(mem_data)) {
+                data_tlb->on_privilege_changed(restored);
+            }


The dynamic cast are the last resort and the core should know (ideally) nothing about TLB except some control instructions to commands propagation.

One option is to add standard (synchronous) signal emit at set_current_privilege in the Core (it is QObject) and interconnect this signal to TLBs.

But when I think about it, then the right solution is to modify memory access FrontendMemory::write_ctl and FrontendMemory::read_ctl to propagate some control signals. Probably by pointer which can be null or may be with default parameters when passed by value (some struct which fits into 32 bits or uint in such case). These additional signals should propagate the privilege level and asid. This is how it is done o real CPUs. I.e., when the processor chips exposed bus to external MMU (68020) or when the buses are routed into FPGA fabric today. The control signals should be privilege level and current ASID. ASID should be held in core state and synchronized by some signal from CSR writes...

The TLB::on_privilege_changed should not be needed and for sure it should not flush TLB entries. It would cause extreme overhead for system calls and machine exceptions. The TLB flushes are maintained by operation system when page tables are modified or there is change of mapping of memory contexts to ASIDs. Seven switch to other memory context does not need the flush when ASIDs are unique.

ppisa · 2025-11-15T11:00:15Z

src/machine/core.cpp


 #include "common/logging.h"
 #include "execute/alu.h"
+#include "memory/tlb/tlb.h"


The TLB integration has to be solved such way that internal core logic does not need know how it works and how it is implemented. Same for tests etc.

ppisa · 2025-11-15T11:02:48Z

src/machine/memory/tlb/tlb.h

    TLBType type;
    const TLBConfig tlb_config;
    uint32_t current_satp_raw = 0;
+    CSR::PrivilegeLevel current_priv_ = CSR::PrivilegeLevel::MACHINE;


The logic should be solved such way, that this field is not needed.

There can be use for keeping some last access privilege level and ASID or something similar for visualization purposes. But for sure not for real work.

ppisa · 2025-11-15T11:09:03Z

src/machine/memory/tlb/tlb.cpp

 namespace machine {

+inline bool is_mode_enabled_in_satp(uint32_t satp_raw) {
+        return (satp_raw & (1u << 31)) != 0;


I do not like this inline there. It should go probably into TLB header.

ppisa · 2025-11-15T11:27:06Z

src/machine/core.cpp

    return InstructionFlags(flags_to_check);
 }

+static CSR::PrivilegeLevel decode_xret_type_from_inst(const Instruction &inst) {


This should be solved some other way. I would suggest to not solve this at decode level at all and left decision on ControlState::read and write or at least to the memory stage where illegal-instruction exception exception would be raised. It cannot be through standard exception signal from CSR, it has to be too late. It has to be by return value or some other way, optional pointer to status return. The illegal-instruction exception should be raised even if write to read only register is attempted and even when non-existent registers is addressed. All these information cannot be gathered at decode state. It would cost too much.

There is related discussion about RISC-V standard, which allows some situations where accesses to non existent/unspecified CSRs are reserved, but conclusion is that it should result in illegal-instruction as well except for some exotic arrangements

riscv/riscv-isa-manual#1116

ppisa · 2025-11-15T11:31:56Z

src/machine/core.cpp

+        // Mark illegal if current privilege is lower than encoded xRET type (e.g. MRET executed in S-mode)
+        if (state.current_privilege() < inst_xret_priv) {
+            excause = EXCAUSE_INSN_ILLEGAL;
+        }


This change should not be needed. Should be solved by

const InstructionFlags check_inst_flags_val; const InstructionFlags check_inst_flags_mask;

manipulation in set_current_privilege and snntation of the instructions by required mode IMF_PRIV_S, IMF_PRIV_H, IMF_PRIV_M in decoding tables.

Thank you for your detailed feedback. I appreciate it and have made some changes based on your review. I would be grateful for any further feedback.

Implements a set-associative Translation Lookaside Buffer (TLB) with replacement policies, Page-Table Walker, and adds SV32-specific definitions.

Add privilege level mapping to the GUI so the current hart privilege (UNPRIV, SUPERV, HYPERV, MACHINE) is displayed in core state visualization.

Extend NewDialog with controls for virtual memory setup, including TLB number of sets, associativity, and replacement policy.

Introduce new components for displaying and tracking TLB state similar to cache. TLBViewBlock and TLBAddressBlock render per-set and per-way TLB contents, updated on tlb_update signals. TLBViewScene assembles these views based on associativity. TLBDock integrates into the GUI, showing hit/miss counts, memory accesses, stall cycles, hit rate, and speed improvement, with live updates from the TLB.

Introduce an "As CPU (VMA)" access option in the cached access selector to render memory contents as observed by the CPU through the frontend interface.

Add a set of small assembly tests that exercise the SV32 page-table walker, SATP enablement and the new TLB code. The tests create a root page table and map a virtual page at 0xC4000000, then exercise several scenarios. The tests verify page-table walker behaviour, SATP switching and TLB caching/flush logic. Tests were written based on the consultation.

Ensure that TLBs are only updated when the root register is set, and disable TLB updates while running in Machine mode.

…ion checks Decode MRET/SRET/URET in the decode stage, carry the return type through the interstage registers, and pass it to ControlState::exception_return in the memory stage. Extend instruction metadata with privilege flags (IMF_PRIV_M/H/S) for privileged operations and use them for masking.

jdupak self-requested a review September 28, 2025 17:17

jdupak reviewed Sep 28, 2025

View reviewed changes

nemecad force-pushed the feature/sv32-vm-tlb-ptw-cleanup branch 5 times, most recently from 0bb04d1 to ca4300b Compare October 5, 2025 19:16

ppisa mentioned this pull request Oct 10, 2025

CLI: bugfix and improvements #166

Closed

jdupak force-pushed the feature/sv32-vm-tlb-ptw-cleanup branch from ca4300b to a6cbf71 Compare October 19, 2025 14:48

jdupak reviewed Oct 19, 2025

View reviewed changes

tests/cli/virtual_memory/template/program.S

Copy link

Collaborator

jdupak Oct 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a typo in the dir name

jdupak reviewed Oct 19, 2025

View reviewed changes

jdupak force-pushed the feature/sv32-vm-tlb-ptw-cleanup branch from a6cbf71 to bc32933 Compare October 19, 2025 16:13

nemecad force-pushed the feature/sv32-vm-tlb-ptw-cleanup branch 3 times, most recently from c846a9e to 442a091 Compare November 2, 2025 16:54

nemecad force-pushed the feature/sv32-vm-tlb-ptw-cleanup branch 2 times, most recently from 3238c4f to 62777b5 Compare November 9, 2025 16:02

jdupak force-pushed the feature/sv32-vm-tlb-ptw-cleanup branch from 62777b5 to 6c37dd9 Compare November 10, 2025 19:23

jdupak force-pushed the feature/sv32-vm-tlb-ptw-cleanup branch from 6c37dd9 to 861f836 Compare November 11, 2025 07:16

ppisa reviewed Nov 11, 2025

View reviewed changes

jdupak and others added 4 commits November 13, 2025 10:26

Fix deprecation warnings for custom literals

bbbcd98

GUI: fix zoom scaling issue This resolves the incorrect zoom behavior.

028f6aa

Machine: store current privilege level in CoreState

a963bc3

Add tracking of the hart's current privilege level to the core state so code handling exceptions/returns and visualization can read/update it from the central CoreState structure.

Machine: add supervisor CSRs and sstatus handling

97df4e4

The next supervisor CSRs has been added: sstatus, stvec, sscratch, sepc, scause, stval, satp Write handler has been added as well. It presents sstatus as a masked view of mstatus so supervisor-visible bits stay in sync.

nemecad force-pushed the feature/sv32-vm-tlb-ptw-cleanup branch from 861f836 to b5940e8 Compare November 14, 2025 12:11

ppisa requested changes Nov 15, 2025

View reviewed changes

jdupak and others added 8 commits November 19, 2025 20:23

Machine: add TLB with policies and add SV32 page-table walker

ff9542f

Implements a set-associative Translation Lookaside Buffer (TLB) with replacement policies, Page-Table Walker, and adds SV32-specific definitions.

GUI: show current privilege level in core state view

d193030

Add privilege level mapping to the GUI so the current hart privilege (UNPRIV, SUPERV, HYPERV, MACHINE) is displayed in core state visualization.

GUI: add virtual memory configuration to NewDialog

5629a7e

Extend NewDialog with controls for virtual memory setup, including TLB number of sets, associativity, and replacement policy.

GUI: add "As CPU (VMA)" memory access view

3c24225

Introduce an "As CPU (VMA)" access option in the cached access selector to render memory contents as observed by the CPU through the frontend interface.

Machine: gate TLB updates by root register and privilege mode

ee811fa

Ensure that TLBs are only updated when the root register is set, and disable TLB updates while running in Machine mode.

nemecad force-pushed the feature/sv32-vm-tlb-ptw-cleanup branch from b5940e8 to 0fc627f Compare November 19, 2025 19:40


		namespace machine {

		static bool is_mmio_region(uint64_t virt) {

Basic Virtual Memory Implementation Fixes & Improvements #165

Are you sure you want to change the base?

Basic Virtual Memory Implementation Fixes & Improvements #165

Uh oh!

Conversation

nemecad commented Sep 4, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jdupak commented Sep 28, 2025

Uh oh!

jdupak commented Sep 28, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ppisa commented Sep 29, 2025

Uh oh!

ppisa commented Sep 29, 2025

Uh oh!

ppisa commented Sep 29, 2025

Uh oh!

nemecad commented Oct 5, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jdupak commented Oct 19, 2025

Uh oh!

ppisa commented Oct 29, 2025

Uh oh!

ppisa commented Oct 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ppisa commented Oct 29, 2025

Uh oh!

ppisa commented Oct 29, 2025

Uh oh!

jdupak commented Oct 29, 2025

Uh oh!

jdupak commented Oct 30, 2025

Uh oh!

jdupak commented Nov 10, 2025

Uh oh!

ppisa left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ppisa left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ppisa commented Oct 29, 2025 •

edited

Loading