RISC-V Test Binary Not Executing Correctly on Sargantana Core (U280 FPGA)

[EmmanuelScience]

Problem Description
Simple RISC-V test binaries load successfully to the Sargantana core on the U280 FPGA but don't appear to execute correctly. Memory reads return all zeros instead of the expected test values.

Environment

• FPGA: Xilinx Alveo U280
• Core: Sargantana RISC-V core
• Toolchain: /opt/Xilinx/Vitis/2023.2/gnu/riscv/lin/riscv64-unknown-elf/bin/riscv64-unknown-elf-gcc
• FPGA Shell: BSC FPGA shell
• Driver: QDMA driver

Steps to Reproduce

1. Generate Bitstream
   git clone https://github.com/bsc-loca/fpga-shell
   cd fpga-shell
   make u280
   make initialize LOAD_EA=sargantana
   make project
   make bitstream

2. Load Bitstream
   ./load-bitstream-onic.sh qdma -i /home/eirabor/fpga-shell/bitstream/system.bit -f ux -b 1
   Result: Bitstream loads successfully, PCIe device enumerated correctly

3. Create Test Binary
   Simple RISC-V assembly program:
   .section .text
   .global _start

_start:
# Load test value 0x12345678 into register t0
lui t0, 0x12345
addi t0, t0, 0x678

# Write to memory address 0x100  
li t1, 0x100  
sw t0, 0(t1)  

# Infinite loop  

loop:
j loop

4. Build Binary
   riscv64-unknown-elf-gcc -march=rv64gc -mabi=lp64d -nostdlib -nostartfiles -T linker.ld -o test_binary.elf test.S
   riscv64-unknown-elf-objcopy -O binary test_binary.elf test_binary.bin
   Result: Binary builds successfully (48 bytes)

5. Load and Execute Binary
   ./load_binary.sh test_binary.bin
   Result: Binary loads successfully via DMA

6. Read Memory
   dma-from-device -d /dev/qdma08000-MM-1 -s 4 -a 0x100 -f output.bin
   hexdump -C output.bin
   Problem: Returns 00 00 00 00 instead of expected 78 56 34 12

Expected Behavior
Memory at address 0x100 should contain the value 0x12345678 (little-endian: 78 56 34 12)

Actual Behavior
Memory reads return all zeros, suggesting either:

1. The core is not executing the program
2. There are memory addressing or mapping issues
3. The core is stuck in reset or not running

Boot Script Details
The boot script performs:

1. Reset core: dma-ctl qdma08000 reg write bar 2 0x0 0x0
2. Load binary via DMA: dma-to-device -d /dev/qdma08000-MM-1 -s 48 -a 0x0 -f test_binary.bin
3. Release reset: dma-ctl qdma08000 reg write bar 2 0x0 0x1

Boot Script Output
*** Sargantana Boot Script ***
Booting using test_binary.bin image file which is 48 bytes
*** Resetting core ***
qdma08000, 08:00.00, bar#2, reg 0x0, read back 0x0.
*** Uploading binary into main memory ***
dev /dev/qdma08000-MM-1, address 0x0, size 0x30, offset 0x0, count 1
size=48 Average BW = 1.340857 MB/sec
*** Releasing core's reset ***
qdma08000, 08:00.00, bar#2, reg 0x0, read back 0x1.

Binary Content
hexdump -C test_binary.bin
00000000 b7 42 34 12 93 82 82 67 37 03 00 00 93 82 03 10
00000010 23 20 53 00 6f 00 00 00 00 00 00 00 00 00 00 00
00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Memory Map Notes

• Boot script comment: "Bootrom makes the core jump to address 0x8000_0000, which is translated to HBM address 0x0000_0000"
• Binary is loaded at HBM address 0x0
• Test writes to address 0x100 (should be accessible as HBM address 0x100)

1. Is the core actually executing after reset is released?
2. Are there any status registers to check the core execution state?
3. Is the memory addressing correct (e.g., is 0x100 relative to the right base)?
4. Are there specific requirements for the binary format or entry point?

Request for Help
Looking for guidance on:

1. How to verify if the Sargantana core is actually running
2. Correct memory addressing scheme
3. Any missing steps in the boot/execution process

[ArnauBigas]

Thank you for the detailed report. From your description, the toolchain setup and bitstream loading process appear to be working correctly, and your test binary should be building and transferring without issue.

Root Cause

The likely issue here is incorrect memory addressing in the test binary.

As noted in your “Memory Map Notes” section, the Sargantana core’s bootrom redirects execution to address 0x8000_0000, from the view point of the core, which is mapped by the memory system to HBM address 0x0000_0000. This translation is critical for both code and data accesses.

Your test binary probably gets correctly loaded to HBM physical address 0x0000_0000 — corresponding to the address 0x8000_0000 as seen by the core. However, in your assembly code, you write to address 0x0000_0100, which is outside the expected execution/memory region. Moreover, it's unmapped in the RTL.

Suggested Fix

In your test assembly, change the memory write to use address 0x80000100 instead of 0x00000100. That will ensure the write lands at physical address 0x00000100 in HBM, where you're correctly reading it back via DMA.

So the line:

li t1, 0x100  

Should become:

li t1, 0x80000100

This way, the core will write to address 0x80000100, which is mapped to HBM physical address 0x00000100, and matches the DMA address you're reading from.

Additional Suggestions

You can verify what's going on by using the sim-meep verilator simulator, which simulates the whole FPGA infrastructure (and works the same as sim, and is compiled in the same way). By looking at those waves you will see that the write doesn't actually go anywhere near the main memory.

[EmmanuelScience]

---

Thanks for the suggestion on the memory addressing. I've updated my test case based on your advice but am still running into issues.

Here is a complete summary of the code I'm running and the output I'm seeing.

Code Used

• Linker Script (linker.ld):

  ENTRY(_start)
  MEMORY {
      RAM : ORIGIN = 0x80000000, LENGTH = 64K
  }
  SECTIONS {
      .text : {
          *(.text)
          *(.text.*)
      } > RAM
  
      .rodata : { *(.rodata .rodata.*) } > RAM
      .data   : { *(.data .data.*) } > RAM
      .bss    : { *(.bss .bss.* COMMON) } > RAM
  
      /DISCARD/ : {
          *(.note.gnu.build-id)
          *(.comment)
          *(.eh_frame)
      }
  }

• Assembly (test.S):

  .section .text
  .global _start
  .option norvc
  
  _start:
      # Load value 0x12345678 into t0
      lui t0, 0x12345
      addi t0, t0, 0x678
  
      # Load address 0x80001000 into t1
      lui t1, 0x80001
  
      # Store value to memory
      sw t0, 0(t1)
  
  loop:
      j loop

---

Execution & Output

The binary is loaded using the following script, which resets the core, uses dma-to-device to load the binary to HBM address 0x0, and releases the reset.

$ ./load_binary.sh test_binary.bin
+ FILENAME=test_binary.bin
+ FILESIZE=20
...
*** Resetting core ***
+ dma-ctl qdma08000 reg write bar 2 0x0 0x0
qdma08000, 08:00.00, bar#2, reg 0x0, read back 0x0.
...
*** Uploading binary into main memory ***
+ dma-to-device -v -d /dev/qdma08000-MM-1 -s 20 -a 0x0000000 -f test_binary.bin
...
*** Releasing core's reset ***
+ dma-ctl qdma08000 reg write bar 2 0x0 0x1
qdma08000, 08:00.00, bar#2, reg 0x0, read back 0x1.

Even with these changes, a read from the target HBM address 0x1000 still returns all zeros.

$dma-from-device -d /dev/qdma08000-MM-1 -s 4 -a 0x1000 -f output.bin$ hexdump -C output.bin
00000000  00 00 00 00                                       |....|
00000004

Could you possibly provide a simple, known-working example of an assembly file and linker script for this target? It would be a great help to verify my setup.

[EmmanuelScience]

Hi, Can I please get some help with this?

Best

[ArnauBigas]

Hi, sorry for the delay, I've had some downtime and other issues to work on and I couldn't look at your issue.

Nevertheless, after a second look, it seemed to me like your test code wasn't quite right. The lui instruction performs sign extension, meaning that the address could be malformed. Thus, using the sim-meep strategy I discussed in my previous comment, I analyzed at the execution in simulation, from which I generated this commit log (via sim-meep +commit_log +load=test_binary.elf:

core   0: 0x0000000000000100 (0x0010041b) addiw   s0, zero, 1
core    0:  3 0x0000000000000100 (0x0010041b) x8  0x0000000000000001
core   0: 0x0000000000000104 (0x01f41413) slli    s0, s0, 31
core    0:  3 0x0000000000000104 (0x01f41413) x8  0x0000000080000000
core   0: 0x0000000000000108 (0xf1402573) csrr    a0, mhartid
core    0:  3 0x0000000000000108 (0xf1402573) x10 0x0000000000000000
core   0: 0x000000000000010c (0x18000593) li      a1, 384
core    0:  3 0x000000000000010c (0x18000593) x11 0x0000000000000180
core   0: 0x0000000000000110 (0x00040067) jr      s0
core    0:  3 0x0000000000000110 (0x00040067)
core    0:  >>>>  _start
core   0: 0x0000000080000000 (0x123452b7) lui     t0, 0x12345
core    0:  3 0x0000000080000000 (0x123452b7) x5  0x0000000012345000
core   0: 0x0000000080000004 (0x67828293) addi    t0, t0, 1656
core    0:  3 0x0000000080000004 (0x67828293) x5  0x0000000012345678
core   0: 0x0000000080000008 (0x80001337) lui     t1, 0x80001
core    0:  3 0x0000000080000008 (0x80001337) x6  0xffffffff80001000
core   0: 0x000000008000000c (0x00532023) sw      t0, 0(t1)
core   0: exception trap_fault_store, epc 0x000000008000000c
core   0:           tval 0xffffffff80001000
core   0: 0x0000000000000140 (0xf1402573) csrr    a0, mhartid
core    0:  3 0x0000000000000140 (0xf1402573) x10 0x0000000000000000
core   0: 0x0000000000000144 (0x18000593) li      a1, 384
core    0:  3 0x0000000000000144 (0x18000593) x11 0x0000000000000180
core   0: 0x0000000000000148 (0x10500073) wfi
core    0:  3 0x0000000000000148 (0x10500073)

And, indeed, it seems that your code is malforming the address 0xffffffff80001000, causing the subsequent store fault and hanging the core. The simplest way to solve this is to use the li pseudo-instruction, which the compiler translates into the adequate instructions for loading an immediate into a register:

.section .text
.global _start
.option norvc

_start:
    # Load value 0x12345678 into t0
    li t0, 0x12345678

    # Load address 0x80001000 into t1
    li t1, 0x80001000

    # Store value to memory
    sw t0, 0(t1)

loop:
    j loop

This generates the following commit log:

core   0: 0x0000000000000100 (0x0010041b) addiw   s0, zero, 1
core    0:  3 0x0000000000000100 (0x0010041b) x8  0x0000000000000001
core   0: 0x0000000000000104 (0x01f41413) slli    s0, s0, 31
core    0:  3 0x0000000000000104 (0x01f41413) x8  0x0000000080000000
core   0: 0x0000000000000108 (0xf1402573) csrr    a0, mhartid
core    0:  3 0x0000000000000108 (0xf1402573) x10 0x0000000000000000
core   0: 0x000000000000010c (0x18000593) li      a1, 384
core    0:  3 0x000000000000010c (0x18000593) x11 0x0000000000000180
core   0: 0x0000000000000110 (0x00040067) jr      s0
core    0:  3 0x0000000000000110 (0x00040067)
core    0:  >>>>  _start
core   0: 0x0000000080000000 (0x123452b7) lui     t0, 0x12345
core    0:  3 0x0000000080000000 (0x123452b7) x5  0x0000000012345000
core   0: 0x0000000080000004 (0x6782829b) addiw   t0, t0, 1656
core    0:  3 0x0000000080000004 (0x6782829b) x5  0x0000000012345678
core   0: 0x0000000080000008 (0x00080337) lui     t1, 0x80
core    0:  3 0x0000000080000008 (0x00080337) x6  0x0000000000080000
core   0: 0x000000008000000c (0x0013031b) addiw   t1, t1, 1
core    0:  3 0x000000008000000c (0x0013031b) x6  0x0000000000080001
core   0: 0x0000000080000010 (0x00c31313) slli    t1, t1, 12
core    0:  3 0x0000000080000010 (0x00c31313) x6  0x0000000080001000
core   0: 0x0000000080000014 (0x00532023) sw      t0, 0(t1)
core    0:  3 0x0000000080000014 (0x00532023) mem 0x0000000080001000 0x12345678
core    0:  >>>>  loop
core   0: 0x0000000080000018 (0x0000006f) j       pc + 0x0
core    0:  3 0x0000000080000018 (0x0000006f)
core    0:  >>>>  loop
core   0: 0x0000000080000018 (0x0000006f) j       pc + 0x0
core    0:  3 0x0000000080000018 (0x0000006f)
...

As you can see, now the store is actually being performed, and the core enters the loop you've written. I haven't tested this on FPGA, but I'm pretty sure it should work.