Skip to content

feat(hwmon): add IIO power meter fallback for Jetson Nano#2438

Open
noFlowWater wants to merge 2 commits intosustainable-computing-io:mainfrom
noFlowWater:feat/iio-power-meter-fallback
Open

feat(hwmon): add IIO power meter fallback for Jetson Nano#2438
noFlowWater wants to merge 2 commits intosustainable-computing-io:mainfrom
noFlowWater:feat/iio-power-meter-fallback

Conversation

@noFlowWater
Copy link

Summary

Add Linux IIO subsystem support as a fallback power data source when hwmon sensors are unavailable. This enables Kepler to monitor power on NVIDIA Jetson Nano (L4T R32) where the INA3221 chip uses the ina3221x IIO driver instead of the ina3221 hwmon driver.

Problem

Kepler's hwmon power meter scans /sys/class/hwmon/ for power sensors. On Jetson Nano (L4T R32, kernel 4.9), the INA3221 power monitor is bound to NVIDIA's custom IIO driver (ina3221x) rather than the upstream hwmon driver (ina3221). This means Kepler finds no power zones and cannot attribute energy to containers.

Jetson Device Kernel INA3221 Driver Kepler Status
AGX Orin (L4T R35) 5.10 ina3221 (hwmon) ✅ Works
Xavier NX (L4T R35) 5.10 ina3221 (hwmon) ✅ Works
Nano (L4T R32) 4.9 ina3221x (IIO) ❌ No zones found

Solution

  • sysfsIIOReader: Implements hwmonReader interface, scans /sys/bus/iio/devices/ for in_power{N}_input sensors
  • iioPowerZone: Implements EnergyZone interface with mW→µW conversion (IIO reports milliwatts, hwmon uses microwatts)
  • compositeReader: Wraps both hwmon and IIO readers; tries hwmon first, falls back to IIO, combines zones from all available sources

No API changes — NewHwmonPowerMeter() signature and behavior are unchanged for existing hwmon-only systems.

IIO Sensor Layout (Jetson Nano)

/sys/bus/iio/devices/iio:device0/
├── name              → "ina3221x"
├── rail_name_0       → "POM_5V_IN"    (total input)
├── rail_name_1       → "POM_5V_GPU"
├── rail_name_2       → "POM_5V_CPU"
├── in_power{0,1,2}_input   (mW, integer)
├── in_voltage{0,1,2}_input (mV)
└── in_current{0,1,2}_input (mA)

Downstream Compatibility

IIO zones only support Power() (not Energy()). This works because internal/monitor/node.go already handles power-only zones:

  1. Detects zone type via MaxEnergy() > 0 || energyReading > 0
  2. For power-only zones: integrates E = P × Δt over collection intervals
  3. Container attribution and Prometheus export use pre-computed snapshot data

No code path requires Energy() to succeed.

Hardware Testing

Verified on 3 Jetson devices with cross-compiled arm64 binary:

Device Subsystem Zones Discovered Power Reading Result
Jetson Nano (L4T R32) IIO fallback pom_5v_in, pom_5v_gpu, pom_5v_cpu ~2.3W total (idle) ✅ PASS
Jetson AGX Orin (L4T R35) hwmon (unchanged) vin_sys_5v0, vdd_gpu_soc, vdd_cpu_cv, vddq ~2.6W total ✅ PASS
Jetson Xavier NX (L4T R35) hwmon (unchanged) vdd_in, vdd_cpu_gpu_cv, vdd_soc ~7.0W total ✅ PASS

Changes

File Change
internal/device/iio_power_meter.go New: sysfsIIOReader + iioPowerZone (~205 LOC)
internal/device/iio_power_meter_test.go New: 13 tests covering discovery, conversion, fallback, errors (~243 LOC)
internal/device/hwmon_power_meter.go Add compositeReader, update NewHwmonPowerMeter to use it (~80 LOC changed)
internal/device/hwmon_power_meter_test.go Add iioPowerZone case to TestHwmonPowerMeter_ZoneDetails
internal/device/testdata/sys/bus/iio/ IIO testdata from real Jetson Nano values

Test Plan

  • All existing internal/device/ tests pass with -race
  • 13 new IIO-specific tests pass
  • Full project go build ./... succeeds
  • Cross-compiled arm64 binary verified on real Jetson hardware (Nano, Orin, Xavier)
  • CI pipeline (triggered by this PR)

Add support for reading power data from the Linux IIO subsystem as a
fallback when hwmon power sensors are not available. This enables Kepler
to monitor power consumption on NVIDIA Jetson Nano devices where the
INA3221 power monitor chip is bound to the ina3221x IIO driver (L4T R32,
kernel 4.9) instead of the upstream ina3221 hwmon driver (L4T R35+).

The implementation introduces:
- sysfsIIOReader: implements hwmonReader interface, scans
  /sys/bus/iio/devices/ for IIO power sensors (in_power{N}_input)
- iioPowerZone: implements EnergyZone interface with mW to µW conversion
- compositeReader: tries hwmon first, falls back to IIO, combining zones
  from all available readers

IIO power sensors report milliwatts while hwmon uses microwatts. The
iioPowerZone.Power() method handles this conversion transparently.

Tested on real hardware:
- Jetson Nano (L4T R32): IIO fallback discovers 3 zones (POM_5V_IN,
  POM_5V_GPU, POM_5V_CPU), reads power correctly
- Jetson AGX Orin (L4T R35): hwmon path unchanged, no regression
- Jetson Xavier NX (L4T R35): hwmon path unchanged, no regression

Signed-off-by: noFlowWater <noyusu98@gmail.com>
@github-actions github-actions bot added the feat A new feature or enhancement label Mar 12, 2026
@KaiyiLiu1234 KaiyiLiu1234 self-requested a review March 13, 2026 04:27
@codecov
Copy link

codecov bot commented Mar 13, 2026

Codecov Report

❌ Patch coverage is 76.11940% with 32 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.51%. Comparing base (be70b52) to head (2faaa53).

Files with missing lines Patch % Lines
internal/device/iio_power_meter.go 74.46% 14 Missing and 10 partials ⚠️
internal/device/hwmon_power_meter.go 80.00% 8 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2438      +/-   ##
==========================================
- Coverage   91.83%   91.51%   -0.33%     
==========================================
  Files          55       56       +1     
  Lines        5857     5984     +127     
==========================================
+ Hits         5379     5476      +97     
- Misses        343      363      +20     
- Partials      135      145      +10     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions
Copy link
Contributor

📊 Profiling reports are ready to be viewed

⚠️ Variability in pprof CPU and Memory profiles
When comparing pprof profiles of Kepler versions, expect variability in CPU and memory. Focus only on significant, consistent differences.

💻 CPU Comparison with base Kepler
File: kepler
Build ID: f27a5a848510f57329ed95ee28835b0ef7f4086f
Type: cpu
Time: 2026-03-13 06:10:38 UTC
Duration: 120s, Total samples = 3.80s ( 3.17%)
Active filters:
   show=github.com/sustainable-computing-io
Showing nodes accounting for 0.10s, 2.63% of 3.80s total
Dropped 1 node (cum <= 0.02s)
      flat  flat%   sum%        cum   cum%
         0     0%     0%      0.16s  4.21%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*PowerCollector).Collect
         0     0%     0%     -0.16s  4.21%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*gpuInfoCollector).Collect
         0     0%     0%      0.08s  2.11%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).scheduleNextCollection.func1
         0     0%     0%      0.06s  1.58%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).refreshSnapshot
         0     0%     0%      0.06s  1.58%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).synchronizedPowerRefresh
         0     0%     0%      0.06s  1.58%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).synchronizedPowerRefresh.func1
     0.06s  1.58%  1.58%      0.06s  1.58%  github.com/sustainable-computing-io/kepler/internal/resource.(*procWrapper).CPUTime
         0     0%  1.58%      0.05s  1.32%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).calculatePower
         0     0%  1.58%      0.05s  1.32%  github.com/sustainable-computing-io/kepler/internal/resource.(*resourceInformer).Refresh
         0     0%  1.58%      0.05s  1.32%  github.com/sustainable-computing-io/kepler/internal/resource.(*resourceInformer).refreshProcesses
     0.01s  0.26%  1.84%      0.05s  1.32%  github.com/sustainable-computing-io/kepler/internal/resource.populateProcessFields
    -0.01s  0.26%  1.58%      0.04s  1.05%  github.com/sustainable-computing-io/kepler/internal/resource.(*resourceInformer).updateProcessCache
     0.03s  0.79%  2.37%      0.03s  0.79%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*realProcFS).CPUInfo
     0.02s  0.53%  2.89%      0.02s  0.53%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*PowerCollector).collectProcessMetrics
         0     0%  2.89%     -0.02s  0.53%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).Snapshot
    -0.02s  0.53%  2.37%     -0.02s  0.53%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).calculateNodePower
         0     0%  2.37%     -0.02s  0.53%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).ensureFreshData
     0.02s  0.53%  2.89%      0.02s  0.53%  github.com/sustainable-computing-io/kepler/internal/resource.(*procFSReader).CPUUsageRatio
         0     0%  2.89%      0.02s  0.53%  github.com/sustainable-computing-io/kepler/internal/resource.(*resourceInformer).Refresh.func3
         0     0%  2.89%      0.02s  0.53%  github.com/sustainable-computing-io/kepler/internal/resource.(*resourceInformer).refreshNode
    -0.02s  0.53%  2.37%      0.01s  0.26%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*cpuInfoCollector).Collect
     0.01s  0.26%  2.63%      0.01s  0.26%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).calculateContainerPower
     0.01s  0.26%  2.89%      0.01s  0.26%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).calculateProcessPower
         0     0%  2.89%     -0.01s  0.26%  github.com/sustainable-computing-io/kepler/internal/monitor.(*Process).Clone (inline)
     0.01s  0.26%  3.16%      0.01s  0.26%  github.com/sustainable-computing-io/kepler/internal/monitor.(*Process).StringID (inline)
     0.01s  0.26%  3.42%      0.01s  0.26%  github.com/sustainable-computing-io/kepler/internal/monitor.NewSnapshot (inline)
    -0.01s  0.26%  3.16%     -0.01s  0.26%  github.com/sustainable-computing-io/kepler/internal/monitor.newProcess (inline)
     0.01s  0.26%  3.42%      0.01s  0.26%  github.com/sustainable-computing-io/kepler/internal/resource.(*procFSReader).AllProcs
    -0.01s  0.26%  3.16%     -0.01s  0.26%  github.com/sustainable-computing-io/kepler/internal/resource.(*procWrapper).Comm
    -0.01s  0.26%  2.89%     -0.01s  0.26%  github.com/sustainable-computing-io/kepler/internal/resource.(*procWrapper).Executable
    -0.01s  0.26%  2.63%     -0.01s  0.26%  maps.Copy[go.shape.map[github.com/sustainable-computing-io/kepler/internal/device.EnergyZone]github.com/sustainable-computing-io/kepler/internal/monitor.Usage,go.shape.map[github.com/sustainable-computing-io/kepler/internal/device.EnergyZone]github.com/sustainable-computing-io/kepler/internal/monitor.Usage,go.shape.interface { Energy ; Index int; MaxEnergy github.com/sustainable-computing-io/kepler/internal/device.Energy; Name string; Path string; Power  },go.shape.struct { EnergyTotal github.com/sustainable-computing-io/kepler/internal/device.Energy; Power github.com/sustainable-computing-io/kepler/internal/device.Power }] (inline)
💾 Memory Comparison with base Kepler (Inuse)
File: kepler
Build ID: f27a5a848510f57329ed95ee28835b0ef7f4086f
Type: inuse_space
Time: 2026-03-13 06:12:39 UTC
Duration: 120.02s, Total samples = 13225.36kB 
Active filters:
   show=github.com/sustainable-computing-io
Showing nodes accounting for 3067.95kB, 23.20% of 13225.36kB total
Dropped 2 nodes (cum <= 66.13kB)
      flat  flat%   sum%        cum   cum%
         0     0%     0%  3584.52kB 27.10%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*PowerCollector).Collect
 2048.10kB 15.49% 15.49%  2048.10kB 15.49%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*PowerCollector).collectProcessMetrics
         0     0% 15.49%  1024.40kB  7.75%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).Snapshot
         0     0% 15.49%  1024.28kB  7.74%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).calculateProcessPower
         0     0% 15.49%  1024.28kB  7.74%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).ensureFreshData
 1024.28kB  7.74% 23.23%  1024.28kB  7.74%  github.com/sustainable-computing-io/kepler/internal/monitor.newProcess (inline)
         0     0% 23.23%  -516.64kB  3.91%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).scheduleNextCollection.func1
 -516.64kB  3.91% 19.32%  -516.64kB  3.91%  github.com/sustainable-computing-io/kepler/internal/resource.(*resourceInformer).Refresh
         0     0% 19.32%   512.14kB  3.87%  github.com/sustainable-computing-io/kepler/internal/monitor.(*Process).Clone (inline)
  512.14kB  3.87% 23.20%   512.14kB  3.87%  maps.Copy[go.shape.map[github.com/sustainable-computing-io/kepler/internal/device.EnergyZone]github.com/sustainable-computing-io/kepler/internal/monitor.Usage,go.shape.map[github.com/sustainable-computing-io/kepler/internal/device.EnergyZone]github.com/sustainable-computing-io/kepler/internal/monitor.Usage,go.shape.interface { Energy ; Index int; MaxEnergy github.com/sustainable-computing-io/kepler/internal/device.Energy; Name string; Path string; Power  },go.shape.struct { EnergyTotal github.com/sustainable-computing-io/kepler/internal/device.Energy; Power github.com/sustainable-computing-io/kepler/internal/device.Power }] (inline)
         0     0% 23.20%   512.09kB  3.87%  github.com/sustainable-computing-io/kepler/internal/resource.computeTypeInfoFromProc.func1
  512.09kB  3.87% 27.07%   512.09kB  3.87%  github.com/sustainable-computing-io/kepler/internal/resource.containerInfoFromCgroupPaths
         0     0% 27.07%   512.09kB  3.87%  github.com/sustainable-computing-io/kepler/internal/resource.containerInfoFromProc
         0     0% 27.07%  -512.02kB  3.87%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*gpuInfoCollector).Collect
 -512.02kB  3.87% 23.20%  -512.02kB  3.87%  github.com/sustainable-computing-io/kepler/internal/monitor.(*Node).Clone (inline)
         0     0% 23.20%   507.64kB  3.84%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).calculatePower
         0     0% 23.20%   507.64kB  3.84%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).refreshSnapshot
         0     0% 23.20%   507.64kB  3.84%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).synchronizedPowerRefresh
         0     0% 23.20%   507.64kB  3.84%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).synchronizedPowerRefresh.func1
💾 Memory Comparison with base Kepler (Alloc)
File: kepler
Build ID: f27a5a848510f57329ed95ee28835b0ef7f4086f
Type: alloc_space
Time: 2026-03-13 06:12:39 UTC
Duration: 120.02s, Total samples = 182969.98kB 
Active filters:
   show=github.com/sustainable-computing-io
Showing nodes accounting for 13643.20kB, 7.46% of 182969.98kB total
Dropped 4 nodes (cum <= 914.85kB)
      flat  flat%   sum%        cum   cum%
         0     0%     0% 13326.46kB  7.28%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*PowerCollector).Collect
         0     0%     0%  8673.80kB  4.74%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).calculatePower
         0     0%     0%  8673.80kB  4.74%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).refreshSnapshot
         0     0%     0%  8673.80kB  4.74%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).synchronizedPowerRefresh
         0     0%     0%  8673.80kB  4.74%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).synchronizedPowerRefresh.func1
         0     0%     0%  8192.16kB  4.48%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).Snapshot
         0     0%     0%  8172.34kB  4.47%  github.com/sustainable-computing-io/kepler/internal/resource.(*resourceInformer).Refresh
 -521.05kB  0.28%  0.28%  8172.34kB  4.47%  github.com/sustainable-computing-io/kepler/internal/resource.(*resourceInformer).refreshProcesses
 7691.70kB  4.20%  3.92%  7691.70kB  4.20%  github.com/sustainable-computing-io/kepler/internal/resource.(*procWrapper).CPUTime
         0     0%  3.92%  7691.70kB  4.20%  github.com/sustainable-computing-io/kepler/internal/resource.(*resourceInformer).updateProcessCache
         0     0%  3.92%  7691.70kB  4.20%  github.com/sustainable-computing-io/kepler/internal/resource.populateProcessFields
         0     0%  3.92%  6669.10kB  3.64%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).ensureFreshData
 6139.81kB  3.36%  7.27%  6139.81kB  3.36%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*PowerCollector).collectProcessMetrics
   -0.02kB 1.3e-05%  7.27% -5335.15kB  2.92%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*cpuInfoCollector).Collect
-5335.13kB  2.92%  4.36% -5335.13kB  2.92%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*realProcFS).CPUInfo
 1024.09kB  0.56%  4.92%  2560.51kB  1.40%  github.com/sustainable-computing-io/kepler/internal/monitor.(*Process).Clone (inline)
 2128.82kB  1.16%  6.08%  2128.82kB  1.16%  github.com/sustainable-computing-io/kepler/internal/resource.(*procFSReader).CPUUsageRatio
         0     0%  6.08%  2128.82kB  1.16%  github.com/sustainable-computing-io/kepler/internal/resource.(*resourceInformer).Refresh.func3
         0     0%  6.08%  2128.82kB  1.16%  github.com/sustainable-computing-io/kepler/internal/resource.(*resourceInformer).refreshNode
         0     0%  6.08%  2004.70kB  1.10%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).scheduleNextCollection.func1
         0     0%  6.08%  1540.88kB  0.84%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*PlatformCollector).Collect
         0     0%  6.08%  1540.88kB  0.84%  github.com/sustainable-computing-io/kepler/internal/platform/redfish.(*PowerReader).ReadAll
 1540.88kB  0.84%  6.92%  1540.88kB  0.84%  github.com/sustainable-computing-io/kepler/internal/platform/redfish.(*PowerReader).readPowerSubsystem
         0     0%  6.92%  1540.88kB  0.84%  github.com/sustainable-computing-io/kepler/internal/platform/redfish.(*Service).Power
 1536.42kB  0.84%  7.76%  1536.42kB  0.84%  maps.Copy[go.shape.map[github.com/sustainable-computing-io/kepler/internal/device.EnergyZone]github.com/sustainable-computing-io/kepler/internal/monitor.Usage,go.shape.map[github.com/sustainable-computing-io/kepler/internal/device.EnergyZone]github.com/sustainable-computing-io/kepler/internal/monitor.Usage,go.shape.interface { Energy ; Index int; MaxEnergy github.com/sustainable-computing-io/kepler/internal/device.Energy; Name string; Path string; Power  },go.shape.struct { EnergyTotal github.com/sustainable-computing-io/kepler/internal/device.Energy; Power github.com/sustainable-computing-io/kepler/internal/device.Power }] (inline)
 -525.43kB  0.29%  7.48%  1523.06kB  0.83%  github.com/sustainable-computing-io/kepler/internal/monitor.(*Snapshot).Clone
         0     0%  7.48% -1028.01kB  0.56%  github.com/sustainable-computing-io/kepler/internal/resource.computeTypeInfoFromProc.func1
         0     0%  7.48% -1028.01kB  0.56%  github.com/sustainable-computing-io/kepler/internal/resource.containerInfoFromProc
-1028.01kB  0.56%  6.91% -1028.01kB  0.56%  github.com/sustainable-computing-io/kepler/internal/resource.(*procWrapper).Cgroups
 1024.44kB  0.56%  7.47%  1024.44kB  0.56%  github.com/sustainable-computing-io/kepler/internal/monitor.newProcess (inline)
         0     0%  7.47%  1005.51kB  0.55%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*gpuInfoCollector).Collect
  489.69kB  0.27%  7.74%  1001.70kB  0.55%  github.com/sustainable-computing-io/kepler/internal/resource.(*procFSReader).AllProcs
         0     0%  7.74%   514.38kB  0.28%  github.com/sustainable-computing-io/kepler/internal/device.(*AggregatedZone).Energy
  514.38kB  0.28%  8.02%   514.38kB  0.28%  github.com/sustainable-computing-io/kepler/internal/device.sysfsRaplZone.Energy
         0     0%  8.02%   514.38kB  0.28%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).calculateNodePower
 -512.02kB  0.28%  7.74%  -512.02kB  0.28%  github.com/sustainable-computing-io/kepler/internal/monitor.(*Node).Clone (inline)
 -512.02kB  0.28%  7.46%  -512.02kB  0.28%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).calculateContainerPower
  512.02kB  0.28%  7.74%   512.02kB  0.28%  github.com/sustainable-computing-io/kepler/internal/resource.WrapProc (inline)
 -525.33kB  0.29%  7.46%   499.10kB  0.27%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).calculateProcessPower

⬇️ Download the Profiling artifacts from the Actions Summary page

📦 Artifact name: profile-artifacts-2438

🔧 Or use GitHub CLI to download artifacts:

gh run download 23001898484 -n profile-artifacts-2438

Pre-commit end-of-file-fixer requires all files to end with a newline.

Signed-off-by: noFlowWater <noyusu98@gmail.com>
@github-actions github-actions bot added the fix A bug fix label Mar 13, 2026
@github-actions
Copy link
Contributor

📊 Profiling reports are ready to be viewed

⚠️ Variability in pprof CPU and Memory profiles
When comparing pprof profiles of Kepler versions, expect variability in CPU and memory. Focus only on significant, consistent differences.

💻 CPU Comparison with base Kepler
File: kepler
Build ID: 9fcde22e1e86b229a8cf62ff795110fb54583c54
Type: cpu
Time: 2026-03-15 16:36:21 UTC
Duration: 120s, Total samples = 3.84s ( 3.20%)
Active filters:
   show=github.com/sustainable-computing-io
Showing nodes accounting for -0.02s, 0.52% of 3.84s total
      flat  flat%   sum%        cum   cum%
         0     0%     0%     -0.18s  4.69%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*PowerCollector).Collect
         0     0%     0%      0.14s  3.65%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*gpuInfoCollector).Collect
         0     0%     0%     -0.05s  1.30%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).Snapshot
         0     0%     0%     -0.05s  1.30%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).calculatePower
         0     0%     0%     -0.05s  1.30%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).refreshSnapshot
         0     0%     0%     -0.05s  1.30%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).synchronizedPowerRefresh
         0     0%     0%     -0.05s  1.30%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).synchronizedPowerRefresh.func1
         0     0%     0%     -0.04s  1.04%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).ensureFreshData
     0.04s  1.04%  1.04%      0.04s  1.04%  github.com/sustainable-computing-io/kepler/internal/resource.(*procFSReader).CPUUsageRatio
         0     0%  1.04%      0.04s  1.04%  github.com/sustainable-computing-io/kepler/internal/resource.(*resourceInformer).Refresh.func3
         0     0%  1.04%      0.04s  1.04%  github.com/sustainable-computing-io/kepler/internal/resource.(*resourceInformer).refreshNode
    -0.03s  0.78%  0.26%     -0.03s  0.78%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*realProcFS).CPUInfo
    -0.03s  0.78%  0.52%     -0.03s  0.78%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).calculateProcessPower
    -0.03s  0.78%  1.30%     -0.03s  0.78%  github.com/sustainable-computing-io/kepler/internal/resource.(*procWrapper).CPUTime
         0     0%  1.30%      0.02s  0.52%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*PlatformCollector).Collect
     0.01s  0.26%  1.04%     -0.02s  0.52%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*cpuInfoCollector).Collect
         0     0%  1.04%     -0.02s  0.52%  github.com/sustainable-computing-io/kepler/internal/monitor.(*Process).Clone (inline)
         0     0%  1.04%      0.02s  0.52%  github.com/sustainable-computing-io/kepler/internal/platform/redfish.(*PowerReader).ReadAll
     0.02s  0.52%  0.52%      0.02s  0.52%  github.com/sustainable-computing-io/kepler/internal/platform/redfish.(*PowerReader).readPowerSubsystem
         0     0%  0.52%      0.02s  0.52%  github.com/sustainable-computing-io/kepler/internal/platform/redfish.(*Service).Power
         0     0%  0.52%     -0.02s  0.52%  github.com/sustainable-computing-io/kepler/internal/resource.(*resourceInformer).updateProcessCache
         0     0%  0.52%     -0.02s  0.52%  github.com/sustainable-computing-io/kepler/internal/resource.populateProcessFields
    -0.02s  0.52%  1.04%     -0.02s  0.52%  maps.Copy[go.shape.map[github.com/sustainable-computing-io/kepler/internal/device.EnergyZone]github.com/sustainable-computing-io/kepler/internal/monitor.Usage,go.shape.map[github.com/sustainable-computing-io/kepler/internal/device.EnergyZone]github.com/sustainable-computing-io/kepler/internal/monitor.Usage,go.shape.interface { Energy ; Index int; MaxEnergy github.com/sustainable-computing-io/kepler/internal/device.Energy; Name string; Path string; Power  },go.shape.struct { EnergyTotal github.com/sustainable-computing-io/kepler/internal/device.Energy; Power github.com/sustainable-computing-io/kepler/internal/device.Power }] (inline)
     0.01s  0.26%  0.78%      0.01s  0.26%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*PowerCollector).collectVMMetrics
    -0.01s  0.26%  1.04%     -0.01s  0.26%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).calculateNodePower
         0     0%  1.04%     -0.01s  0.26%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).scheduleNextCollection.func1
     0.01s  0.26%  0.78%     -0.01s  0.26%  github.com/sustainable-computing-io/kepler/internal/monitor.(*Snapshot).Clone
     0.02s  0.52%  0.26%      0.01s  0.26%  github.com/sustainable-computing-io/kepler/internal/resource.(*procFSReader).AllProcs
    -0.01s  0.26%  0.52%     -0.01s  0.26%  github.com/sustainable-computing-io/kepler/internal/resource.(*procWrapper).CmdLine
     0.01s  0.26%  0.26%      0.01s  0.26%  github.com/sustainable-computing-io/kepler/internal/resource.(*procWrapper).Comm
         0     0%  0.26%     -0.01s  0.26%  github.com/sustainable-computing-io/kepler/internal/resource.(*resourceInformer).Refresh
         0     0%  0.26%     -0.01s  0.26%  github.com/sustainable-computing-io/kepler/internal/resource.(*resourceInformer).refreshProcesses
    -0.01s  0.26%  0.52%     -0.01s  0.26%  github.com/sustainable-computing-io/kepler/internal/resource.WrapProc (inline)
         0     0%  0.52%     -0.01s  0.26%  github.com/sustainable-computing-io/kepler/internal/resource.computeTypeInfoFromProc.func2
         0     0%  0.52%      0.01s  0.26%  github.com/sustainable-computing-io/kepler/internal/resource.newProcess
         0     0%  0.52%     -0.01s  0.26%  github.com/sustainable-computing-io/kepler/internal/resource.vmInfoFromProc
💾 Memory Comparison with base Kepler (Inuse)
File: kepler
Build ID: 9fcde22e1e86b229a8cf62ff795110fb54583c54
Type: inuse_space
Time: 2026-03-15 16:38:22 UTC
Duration: 120.02s, Total samples = 11085.17kB 
Active filters:
   show=github.com/sustainable-computing-io
Showing nodes accounting for -1023.95kB, 9.24% of 11085.17kB total
Dropped 2 nodes (cum <= 55.43kB)
      flat  flat%   sum%        cum   cum%
-1536.09kB 13.86% 13.86% -1536.09kB 13.86%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*PowerCollector).collectProcessMetrics
         0     0% 13.86% -1023.99kB  9.24%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*PowerCollector).Collect
         0     0% 13.86%   512.14kB  4.62%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).calculatePower
         0     0% 13.86%   512.14kB  4.62%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).calculateProcessPower
         0     0% 13.86%   512.14kB  4.62%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).ensureFreshData
         0     0% 13.86%   512.14kB  4.62%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).refreshSnapshot
         0     0% 13.86%   512.14kB  4.62%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).synchronizedPowerRefresh
         0     0% 13.86%   512.14kB  4.62%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).synchronizedPowerRefresh.func1
  512.14kB  4.62%  9.24%   512.14kB  4.62%  github.com/sustainable-computing-io/kepler/internal/monitor.newProcess (inline)
         0     0%  9.24%   512.10kB  4.62%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).Snapshot
💾 Memory Comparison with base Kepler (Alloc)
File: kepler
Build ID: 9fcde22e1e86b229a8cf62ff795110fb54583c54
Type: alloc_space
Time: 2026-03-15 16:38:22 UTC
Duration: 120.02s, Total samples = 203262.63kB 
Active filters:
   show=github.com/sustainable-computing-io
Showing nodes accounting for -10573.87kB, 5.20% of 203262.63kB total
Dropped 1 node (cum <= 1016.31kB)
      flat  flat%   sum%        cum   cum%
-11772.15kB  5.79%  5.79% -11772.15kB  5.79%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*PowerCollector).collectProcessMetrics
         0     0%  5.79% -10740.99kB  5.28%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*PowerCollector).Collect
         0     0%  5.79% -4622.89kB  2.27%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*gpuInfoCollector).Collect
         0     0%  5.79% -4103.79kB  2.02%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).Snapshot
  521.05kB  0.26%  5.54%  3617.67kB  1.78%  github.com/sustainable-computing-io/kepler/internal/resource.(*resourceInformer).refreshProcesses
   -0.42kB 0.00021%  5.54%  3617.25kB  1.78%  github.com/sustainable-computing-io/kepler/internal/resource.(*resourceInformer).Refresh
-1567.49kB  0.77%  6.31% -3103.72kB  1.53%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).calculateProcessPower
 1024.05kB   0.5%  5.80%  2659.87kB  1.31%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*cpuInfoCollector).Collect
-2566.50kB  1.26%  7.07% -2566.50kB  1.26%  github.com/sustainable-computing-io/kepler/internal/resource.(*procFSReader).CPUUsageRatio
         0     0%  7.07% -2566.50kB  1.26%  github.com/sustainable-computing-io/kepler/internal/resource.(*resourceInformer).Refresh.func3
         0     0%  7.07% -2566.50kB  1.26%  github.com/sustainable-computing-io/kepler/internal/resource.(*resourceInformer).refreshNode
         0     0%  7.07% -2560.70kB  1.26%  github.com/sustainable-computing-io/kepler/internal/monitor.(*Process).Clone (inline)
         0     0%  7.07% -2560.70kB  1.26%  github.com/sustainable-computing-io/kepler/internal/monitor.(*Snapshot).Clone
-2560.70kB  1.26%  8.33% -2560.70kB  1.26%  maps.Copy[go.shape.map[github.com/sustainable-computing-io/kepler/internal/device.EnergyZone]github.com/sustainable-computing-io/kepler/internal/monitor.Usage,go.shape.map[github.com/sustainable-computing-io/kepler/internal/device.EnergyZone]github.com/sustainable-computing-io/kepler/internal/monitor.Usage,go.shape.interface { Energy ; Index int; MaxEnergy github.com/sustainable-computing-io/kepler/internal/device.Energy; Name string; Path string; Power  },go.shape.struct { EnergyTotal github.com/sustainable-computing-io/kepler/internal/device.Energy; Power github.com/sustainable-computing-io/kepler/internal/device.Power }] (inline)
         0     0%  8.33%  2101.14kB  1.03%  github.com/sustainable-computing-io/kepler/internal/resource.computeTypeInfoFromProc.func2
 2101.14kB  1.03%  7.29%  2101.14kB  1.03%  github.com/sustainable-computing-io/kepler/internal/resource.vmInfoFromCmdLine
         0     0%  7.29%  2101.14kB  1.03%  github.com/sustainable-computing-io/kepler/internal/resource.vmInfoFromProc
 2042.92kB  1.01%  6.29%  2042.92kB  1.01%  github.com/sustainable-computing-io/kepler/internal/resource.(*procWrapper).CPUTime
         0     0%  6.29%  2042.92kB  1.01%  github.com/sustainable-computing-io/kepler/internal/resource.(*resourceInformer).updateProcessCache
         0     0%  6.29%  2042.92kB  1.01%  github.com/sustainable-computing-io/kepler/internal/resource.populateProcessFields
 1635.81kB   0.8%  5.48%  1635.81kB   0.8%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*realProcFS).CPUInfo
         0     0%  5.48% -1543.13kB  0.76%  github.com/sustainable-computing-io/kepler/internal/device.(*AggregatedZone).Energy
-1543.13kB  0.76%  6.24% -1543.13kB  0.76%  github.com/sustainable-computing-io/kepler/internal/device.sysfsRaplZone.Energy
         0     0%  6.24% -1543.13kB  0.76%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).calculateNodePower
         0     0%  6.24% -1543.08kB  0.76%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).ensureFreshData
         0     0%  6.24% -1541.65kB  0.76%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).calculatePower
         0     0%  6.24% -1541.65kB  0.76%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).refreshSnapshot
         0     0%  6.24% -1541.65kB  0.76%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).synchronizedPowerRefresh
         0     0%  6.24% -1541.65kB  0.76%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).synchronizedPowerRefresh.func1
         0     0%  6.24%  1537.75kB  0.76%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*PlatformCollector).Collect
         0     0%  6.24%  1537.75kB  0.76%  github.com/sustainable-computing-io/kepler/internal/platform/redfish.(*PowerReader).ReadAll
 1537.75kB  0.76%  5.48%  1537.75kB  0.76%  github.com/sustainable-computing-io/kepler/internal/platform/redfish.(*PowerReader).readPowerSubsystem
         0     0%  5.48%  1537.75kB  0.76%  github.com/sustainable-computing-io/kepler/internal/platform/redfish.(*Service).Power
-1536.23kB  0.76%  6.24% -1536.23kB  0.76%  github.com/sustainable-computing-io/kepler/internal/monitor.newProcess (inline)
         0     0%  6.24%  1056.33kB  0.52%  github.com/sustainable-computing-io/kepler/internal/resource.computeTypeInfoFromProc.func1
 1056.33kB  0.52%  5.72%  1056.33kB  0.52%  github.com/sustainable-computing-io/kepler/internal/resource.containerInfoFromCgroupPaths
         0     0%  5.72%  1056.33kB  0.52%  github.com/sustainable-computing-io/kepler/internal/resource.containerInfoFromProc
 1565.71kB  0.77%  4.95%  1053.69kB  0.52%  github.com/sustainable-computing-io/kepler/internal/resource.(*procFSReader).AllProcs
         0     0%  4.95%   512.05kB  0.25%  github.com/sustainable-computing-io/kepler/internal/exporter/prometheus/collector.(*PowerCollector).Collect.func1
  512.05kB  0.25%  4.70%   512.05kB  0.25%  github.com/sustainable-computing-io/kepler/internal/logger.handlerForFormat.func1
         0     0%  4.70%  -512.05kB  0.25%  github.com/sustainable-computing-io/kepler/internal/monitor.(*PowerMonitor).calculateContainerPower
 -512.05kB  0.25%  4.95%  -512.05kB  0.25%  github.com/sustainable-computing-io/kepler/internal/monitor.newContainer (inline)
 -512.02kB  0.25%  5.20%  -512.02kB  0.25%  github.com/sustainable-computing-io/kepler/internal/resource.WrapProc (inline)

⬇️ Download the Profiling artifacts from the Actions Summary page

📦 Artifact name: profile-artifacts-2438

🔧 Or use GitHub CLI to download artifacts:

gh run download 23058951663 -n profile-artifacts-2438

@KaiyiLiu1234 KaiyiLiu1234 requested a review from vimalk78 March 16, 2026 16:02
@KaiyiLiu1234
Copy link
Collaborator

KaiyiLiu1234 commented Mar 16, 2026

@vimalk78 are you able to see if this works on your nvidia jetson machine? - specifically this: Jetson Nano (L4T R32)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feat A new feature or enhancement fix A bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants