west: runner: add support for ecpprog #81482

josuah · 2024-11-16T21:33:36Z

This adds the ecpprog utility to flash FPGA boards on Zephyr.

There is currently no board upstream configured to use it, but 3rd-party module soon will, and later be submitted to Zephyr.

Example board.cmake:

include(${ZEPHYR_BASE}/boards/common/ecpprog.board.cmake)

Optional Kconfig.defconfig for CONFIG_FLASH_LOAD_OFFSET support:

# Workaround for not being able to have commas in macro arguments
DT_CHOSEN_Z_CODE_PARTITION := zephyr,code-partition

config FLASH_LOAD_OFFSET
        default $(dt_chosen_reg_addr_hex,$(DT_CHOSEN_Z_CODE_PARTITION))
        # or directly some number

Sample output:

$ west flash
-- west flash: rebuilding
ninja: no work to do.
-- west flash: using runner ecpprog
ecpprog -o 0x100000 /home/tinyvision/zephyrproject/tinyclunx33_zephyr_example/build/zephyr/zephyr.bin
init..
IDCODE: 0x010fb043 (LIFCL-33U)
NX Status Register: 0x0000110100000000
reset..
flash ID: 0xEF 0x80 0x18
file size: 157096
erase 64kB sector at 0x100000..
erase 64kB sector at 0x110000..
erase 64kB sector at 0x120000..
programming..  157096/157096
verify..       157096/157096  VERIFY OK
Bye.
$

pdgendt

You need to add an entry in scripts/west_commands/tests/test_imports.py too.

scripts/west_commands/runners/ecpprog.py

josuah · 2024-11-17T13:41:28Z

Force-push:

Added the entry into test includes:

-- west flash: using runner ecpprog
init..
IDCODE: 0x010fb043 (LIFCL-33U)
NX Status Register: 0x0000110100000100
reset..
flash ID: 0xEF 0x80 0x18
file size: 156472
erase 64kB sector at 0x100000..
erase 64kB sector at 0x110000..
erase 64kB sector at 0x120000..
programming..  156472/156472
verify..       156472/156472  VERIFY OK
Bye.

Good to see there is a complete test infrastructure for twister runners.

kartben · 2024-11-22T20:58:17Z

@pdgendt should this be rebased to be sure it passes Ruff checks? I think we want to be extra careful with all PRs adding Python files that predate the introduction of Ruff, correct?

josuah · 2024-11-22T21:45:50Z

Ruff has issues, and I'm happy to use this PR as guinea pig for it.

I can reproduce by running check_compliance locally:

(py) tinyvision@vm1:~/zephyrproject/zephyr$ python scripts/ci/check_compliance.py ^G
Running BinaryFiles      tests in /home/tinyvision/zephyrproject/zephyr ...
Running Kconfig          tests in /home/tinyvision/zephyrproject/zephyr ...
Running YAMLLint         tests in /home/tinyvision/zephyrproject/zephyr ...
Running Identity         tests in /home/tinyvision/zephyrproject/zephyr ...
Running ModulesMaintainers tests in /home/tinyvision/zephyrproject/zephyr ...
Running Pylint           tests in /home/tinyvision/zephyrproject/zephyr ...
Running MaintainersFormat tests in /home/tinyvision/zephyrproject/zephyr ...
Running DevicetreeBindings tests in /home/tinyvision/zephyrproject/zephyr ...
Running Gitlint          tests in /home/tinyvision/zephyrproject/zephyr ...
Running BoardYml         tests in /home/tinyvision/zephyrproject/zephyr ...
Running KconfigBasicNoModules tests in /home/tinyvision/zephyrproject/zephyr ...
Running TextEncoding     tests in /home/tinyvision/zephyrproject/zephyr ...
Running GitDiffCheck     tests in /home/tinyvision/zephyrproject/zephyr ...
Running Ruff             tests in /home/tinyvision/zephyrproject/zephyr ...
warning: The following rules have been removed and ignoring them has no effect:
    - UP027

Python exception in `/home/tinyvision/zephyrproject/zephyr/scripts/ci/check_compliance.py`:

` ` `
Traceback (most recent call last):
  File "/home/tinyvision/zephyrproject/zephyr/scripts/ci/check_compliance.py", line 1936, in main
    n_fails = _main(args)
              ^^^^^^^^^^^
  File "/home/tinyvision/zephyrproject/zephyr/scripts/ci/check_compliance.py", line 1871, in _main
    test.run()
  File "/home/tinyvision/zephyrproject/zephyr/scripts/ci/check_compliance.py", line 1666, in run
    messages = json.loads(output)
               ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

` ` `
Traceback (most recent call last):
  File "/home/tinyvision/zephyrproject/zephyr/scripts/ci/check_compliance.py", line 1963, in <module>
    main(sys.argv[1:])
  File "/home/tinyvision/zephyrproject/zephyr/scripts/ci/check_compliance.py", line 1936, in main
    n_fails = _main(args)
              ^^^^^^^^^^^
  File "/home/tinyvision/zephyrproject/zephyr/scripts/ci/check_compliance.py", line 1871, in _main
    test.run()
  File "/home/tinyvision/zephyrproject/zephyr/scripts/ci/check_compliance.py", line 1666, in run
    messages = json.loads(output)
               ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

(py) tinyvision@vm1:~/zephyrproject/zephyr$ uname -a
Linux vm1 6.8.0-49-generic #49-Ubuntu SMP PREEMPT_DYNAMIC Mon Nov  4 02:06:24 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

(py) tinyvision@vm1:~/zephyrproject/zephyr$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 24.04 LTS
Release:        24.04
Codename:       noble

(py) tinyvision@vm1:~/zephyrproject/zephyr$ python --version
Python 3.12.3

josuah · 2024-11-22T21:53:25Z

I think stderr and stdout got merged together somehow...

                output = ex.output.decode("utf-8")
                print('<<<' + output + '>>>')

<<<warning: The following rules have been removed and ignoring them has no effect:
    - UP027

[
  {
    "cell": null,
    "code": "I001",
    "end_location": {
      "column": 1,
      "row": 11
    },
    "filename": "/home/tinyvision/zephyrproject/zephyr/scripts/west_commands/runners/ecpprog.py",
    "fix": {
      "applicability": "safe",
      "edits": [
        {
          "content": "from runners.core import BuildConfiguration, RunnerCaps, ZephyrBinaryRunner\n\n\n",
          "end_location": {
            "column": 1,
            "row": 11
          },
          "location": {
            "column": 1,
            "row": 8
          }
        }
      ],
      "message": "Organize imports"
    },
    "location": {
      "column": 1,
      "row": 8
    },
    "message": "Import block is un-sorted or un-formatted",
    "noqa_row": 8,
    "url": "https://docs.astral.sh/ruff/rules/unsorted-imports"
  }
]>>>

josuah · 2024-11-23T01:07:59Z

Glad this PR could be useful to someone else than me!
I also added a small fix for a recent refactoring affecting a rare case I apparently use.

[EDIT: it looks like the linting works now! I will fix it then... nice feature!]

scripts/west_commands/runners/__init__.py

The ecpprog command is an utility written by Greg Davill for flashing FPGAs such as ECP5 or CrossLink-NX series. Devkits typically have an FTDI interface chip to access the external flash. FPGA image is typically at flash offset 0x00000000 flash offset, and the Zephyr image offset can be set via CONFIG_FLASH_LOAD_OFFSET. Signed-off-by: Josuah Demangeon <[email protected]>

kartben · 2024-12-19T05:20:19Z

Ping @pdgendt

pdgendt

I'm not against adding the runner, the downside is it doesn't have an in-tree board that needs it.

I've created #83190 to showcase supporting external runners.

josuah · 2024-12-19T12:33:20Z

<@pdgendt> it doesn't have an in-tree board that needs it

Good point, I will need to add it to litex_vexriscv and the upcoming tinyclunx33.

josuah added area: West West utility area: Scripts Scripts area: FPGA Field-Programmable Gate Array (FPGA) labels Nov 16, 2024

zephyrbot requested review from carlescufi, mbolivar-ampere, pdgendt and swinslow November 16, 2024 21:38

pdgendt reviewed Nov 17, 2024

View reviewed changes

scripts/west_commands/runners/ecpprog.py Outdated Show resolved Hide resolved

josuah force-pushed the pr-runner-ecpprog branch from c1b7db1 to 6041dc5 Compare November 17, 2024 13:39

josuah force-pushed the pr-runner-ecpprog branch from 6041dc5 to 9f38098 Compare November 17, 2024 14:18

pdgendt previously approved these changes Nov 18, 2024

View reviewed changes

kartben assigned pdgendt Nov 22, 2024

josuah force-pushed the pr-runner-ecpprog branch from 9f38098 to 841d0f2 Compare November 22, 2024 21:38

josuah closed this Nov 22, 2024

josuah reopened this Nov 22, 2024

josuah dismissed pdgendt’s stale review via 732dfaf November 23, 2024 00:03

josuah force-pushed the pr-runner-ecpprog branch from 841d0f2 to 732dfaf Compare November 23, 2024 00:03

josuah force-pushed the pr-runner-ecpprog branch from 732dfaf to c1334a9 Compare November 23, 2024 01:10

josuah requested a review from pdgendt November 23, 2024 01:13

josuah commented Nov 23, 2024

View reviewed changes

scripts/west_commands/runners/__init__.py Outdated Show resolved Hide resolved

josuah force-pushed the pr-runner-ecpprog branch 3 times, most recently from 6cbed62 to 6eed115 Compare November 23, 2024 14:20

josuah force-pushed the pr-runner-ecpprog branch from 6eed115 to e0dc7f8 Compare November 23, 2024 16:27

pdgendt mentioned this pull request Dec 18, 2024

Support out-of-tree runners #83145

Closed

kartben approved these changes Dec 19, 2024

View reviewed changes

pdgendt approved these changes Dec 19, 2024

View reviewed changes

kartben merged commit 5847e04 into zephyrproject-rtos:main Dec 19, 2024
36 checks passed

josuah mentioned this pull request Dec 19, 2024

ecpprog: Now with Zephyr support gregdavill/ecpprog#23

Closed

josuah deleted the pr-runner-ecpprog branch January 10, 2025 21:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

west: runner: add support for ecpprog #81482

west: runner: add support for ecpprog #81482

Uh oh!

josuah commented Nov 16, 2024

Uh oh!

pdgendt left a comment

Uh oh!

Uh oh!

josuah commented Nov 17, 2024

Uh oh!

kartben commented Nov 22, 2024

Uh oh!

josuah commented Nov 22, 2024 •

edited

Loading

Uh oh!

josuah commented Nov 22, 2024 •

edited

Loading

Uh oh!

josuah commented Nov 23, 2024 •

edited

Loading

Uh oh!

Uh oh!

kartben commented Dec 19, 2024

Uh oh!

pdgendt left a comment

Uh oh!

Uh oh!

josuah commented Dec 19, 2024

Uh oh!

Uh oh!

west: runner: add support for ecpprog #81482

west: runner: add support for ecpprog #81482

Uh oh!

Conversation

josuah commented Nov 16, 2024

Uh oh!

pdgendt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

josuah commented Nov 17, 2024

Uh oh!

kartben commented Nov 22, 2024

Uh oh!

josuah commented Nov 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

josuah commented Nov 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

josuah commented Nov 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

kartben commented Dec 19, 2024

Uh oh!

pdgendt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

josuah commented Dec 19, 2024

Uh oh!

Uh oh!

josuah commented Nov 22, 2024 •

edited

Loading

josuah commented Nov 22, 2024 •

edited

Loading

josuah commented Nov 23, 2024 •

edited

Loading