Skip to content

tests: benchdnn: print csv friendly perf header#4866

Open
jondea wants to merge 1 commit intomainfrom
add-benchdnn-perf-csv-header
Open

tests: benchdnn: print csv friendly perf header#4866
jondea wants to merge 1 commit intomainfrom
add-benchdnn-perf-csv-header

Conversation

@jondea
Copy link
Contributor

@jondea jondea commented Mar 20, 2026

Description

Most of the benchdnn perf-templates already emit CSV-shaped data, but there are no headers, and the template doesn't match the columns. Specifically, %DESC% can expand to multiple columns for some drivers, which makes the output hard to parse. This change adds a CSV header row that matches each driver's concrete %DESC% expansion while preserving the existing Output template: ... banner. This makes the CSV output easy to consume with standard tools (e.g. xsv).

E.g. after this change,

./tests/benchdnn/benchdnn --conv --mode=P --perf-template=csv  mb1_ic1oc1_id1ih5iw5_od1oh3ow3_kd1kh3kw3_sd1sh1sw1_pd0ph0pw0_dd0dh0dw0 \
    | grep '^perf,' \
    | xsv table

perf  engine  impl         name  dir    sdt          stag  wtag  dtag  alg     attr  g   mb  ic  id  ih  iw  oc  od  oh  ow  kd  kh  kw  sd  sh  sw  pd  ph  pw  dd  dh  dw  Gops      max_ctime  min_time  min_Gflops  avg_time  avg_Gflops
perf  cpu     jit:sve_128        FWD_B  f32:f32:f32  any   any   any   direct        1   1   1   1   5   5   1   1   3   3   1   3   3   1   1   1   0   0   0   0   0   0   1.62e-07  0.994385   0.137695  0.00117651  0.32537   0.000497895

Not sure how to test this using ctest, I'm open to ideas...

Checklist

General

  • Do all unit and benchdnn tests (make test and make test_benchdnn_*) pass locally for each commit?
  • Have you formatted the code using clang-format?

Print a csv header in perf mode (alongside the existing perf template),
which removes the % and expands DESC into the driver specific entries.
This make the output much easier to process with other tools.
@jondea jondea requested a review from a team as a code owner March 20, 2026 15:52
@github-actions github-actions bot added the component:tests Codeowner: @oneapi-src/onednn-arch label Mar 20, 2026
@vpirogov
Copy link
Contributor

I wonder if we actually need information included into problem descriptor (%DESC%) split into multiple columns. If the only goal is getting 1-1 correspondence between column headers and data I would argue for printing %DESC% in the same format as used in benchdnn input. Something like this in your example:

Output template: perf,%engine%,%impl%,%name%,%dir%,%sdt%,%stag%,%wtag%,%dtag%,%alg%,%attr%,%DESC%,%Gops%,%+ctime%,%-time%,%-Gflops%,%0time%,%0Gflops%
perf,cpu,jit:avx512_core,,FWD_B,f32:f32:f32,any,any,any,direct,,mb1ic1id1ih5iw5oc1od1oh3ow3kd1kh3kw3pd0ph0pw0,1.62e-07,0.689209,0.00146484,0.110592,0.00220769,0.07338

This approach has a benefit of keeping the same number of columns between different primitives in CSV format.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component:tests Codeowner: @oneapi-src/onednn-arch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants