Skip to content

Conversation

@promalert
Copy link
Contributor

These changes use b.Loop() to simplify the code and improve performance
Supported by Go Team, more info: https://go.dev/blog/testing-b-loop

Before:

➜  alertmanager git:(main)  go test -run=^$ -bench=. ./matcher/parse   
goos: darwin
goarch: arm64
pkg: github.com/prometheus/alertmanager/matcher/parse
cpu: Apple M4
BenchmarkParseSimple-10     	 2337806	       500.5 ns/op
BenchmarkParseComplex-10    	  680635	      1950 ns/op
PASS
ok  	github.com/prometheus/alertmanager/matcher/parse	3.287s
➜  alertmanager git:(main) go test -run=^$ -bench=. ./notify    
goos: darwin
goarch: arm64
pkg: github.com/prometheus/alertmanager/notify
cpu: Apple M4
BenchmarkHashAlert-10    	 8728893	       126.3 ns/op
PASS
ok  	github.com/prometheus/alertmanager/notify	1.768s

After:

➜  alertmanager git:(main)  go test -run=^$ -bench=. ./matcher/parse         
goos: darwin
goarch: arm64
pkg: github.com/prometheus/alertmanager/matcher/parse
cpu: Apple M4
BenchmarkParseSimple-10     	 2392150	       493.4 ns/op
BenchmarkParseComplex-10    	  690627	      1762 ns/op
PASS
ok  	github.com/prometheus/alertmanager/matcher/parse	3.117s
➜  alertmanager git:(main) 
➜  alertmanager git:(main)  go test -run=^$ -bench=. ./notify                  
goos: darwin
goarch: arm64
pkg: github.com/prometheus/alertmanager/notify
cpu: Apple M4
BenchmarkHashAlert-10    	 8409073	       129.0 ns/op
PASS
ok  	github.com/prometheus/alertmanager/notify	1.343s

@Spaceman1701
Copy link
Contributor

Just stumbled on this one and gave it a look. This looks totally fine to me, and the go.dev blog says that

The testing.B.Loop method is now the preferred way to write benchmarks:

One thing though: It looks like you missed inhibit_bench_test.go, silence_bench_test.go, and tls_transport_test.go. If we're going to make this change, I think we should do it everywhere to be consistent. Would you mind updating those as well?

@promalert
Copy link
Contributor Author

Just stumbled on this one and gave it a look. This looks totally fine to me, and the go.dev blog says that

The testing.B.Loop method is now the preferred way to write benchmarks:

One thing though: It looks like you missed inhibit_bench_test.go, silence_bench_test.go, and tls_transport_test.go. If we're going to make this change, I think we should do it everywhere to be consistent. Would you mind updating those as well?

Thank you for your suggestions. However, I found that tls_transport_test.go doesn't have any modifications available.
The other two places have been updated. Please review again.

➜  alertmanager git:(main) ✗ go test -run=^$ -bench=. ./inhibit         
goos: darwin
goarch: arm64
pkg: github.com/prometheus/alertmanager/inhibit
cpu: Apple M4
BenchmarkMutes/1_inhibition_rule,_1_inhibiting_alert-10         	 1170290	       967.3 ns/op
BenchmarkMutes/10_inhibition_rules,_1_inhibiting_alert-10       	 1163775	       969.8 ns/op
BenchmarkMutes/100_inhibition_rules,_1_inhibiting_alert-10      	 1195722	       992.3 ns/op
BenchmarkMutes/1000_inhibition_rules,_1_inhibiting_alert-10     	  985664	      1176 ns/op
BenchmarkMutes/10000_inhibition_rules,_1_inhibiting_alert-10    	  997599	      1197 ns/op
BenchmarkMutes/1_inhibition_rule,_10_inhibiting_alerts-10       	 1057458	      1086 ns/op
BenchmarkMutes/1_inhibition_rule,_100_inhibiting_alerts-10      	 1057344	      1086 ns/op
BenchmarkMutes/1_inhibition_rule,_1000_inhibiting_alerts-10     	 1053831	      1085 ns/op
BenchmarkMutes/1_inhibition_rule,_10000_inhibiting_alerts-10    	 1068854	      1073 ns/op
BenchmarkMutes/100_inhibition_rules,_1000_inhibiting_alerts-10  	 1144471	      1010 ns/op
BenchmarkMutes/10_inhibition_rules,_last_rule_matches-10        	  227220	      5213 ns/op
BenchmarkMutes/100_inhibition_rules,_last_rule_matches-10       	   24458	     46616 ns/op
BenchmarkMutes/1000_inhibition_rules,_last_rule_matches-10      	    2451	    484289 ns/op
BenchmarkMutes/10000_inhibition_rules,_last_rule_matches-10     	     206	   5388896 ns/op
PASS
ok  	github.com/prometheus/alertmanager/inhibit	30.887s
➜  alertmanager git:(main) ✗ 
➜  alertmanager git:(main) ✗ go test -run=^$ -bench=. ./silence       
goos: darwin
goarch: arm64
pkg: github.com/prometheus/alertmanager/silence
cpu: Apple M4
BenchmarkMutes/1_silence_mutes_alert-10         	  368256	      3233 ns/op
--- BENCH: BenchmarkMutes/1_silence_mutes_alert-10
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
	... [output truncated]
BenchmarkMutes/10_silences_mute_alert-10        	  293503	      3794 ns/op
--- BENCH: BenchmarkMutes/10_silences_mute_alert-10
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
	... [output truncated]
BenchmarkMutes/100_silences_mute_alert-10       	  129474	      9183 ns/op
--- BENCH: BenchmarkMutes/100_silences_mute_alert-10
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
	... [output truncated]
BenchmarkMutes/1000_silences_mute_alert-10      	   14574	     79275 ns/op
--- BENCH: BenchmarkMutes/1000_silences_mute_alert-10
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
	... [output truncated]
BenchmarkMutes/10000_silences_mute_alert-10     	    1287	    891458 ns/op
--- BENCH: BenchmarkMutes/10000_silences_mute_alert-10
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
	... [output truncated]
BenchmarkQuery/100_silences-10                  	   85952	     13694 ns/op
--- BENCH: BenchmarkQuery/100_silences-10
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
	... [output truncated]
BenchmarkQuery/1000_silences-10                 	    8983	    130814 ns/op
--- BENCH: BenchmarkQuery/1000_silences-10
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
	... [output truncated]
BenchmarkQuery/10000_silences-10                	     715	   1729676 ns/op
--- BENCH: BenchmarkQuery/10000_silences-10
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
    mock.go:202: Mock Clock - Now([]) call, matched 0 traps
	... [output truncated]
PASS
ok  	github.com/prometheus/alertmanager/silence	12.547s
➜  alertmanager git:(main) ✗ 
➜  alertmanager git:(main) ✗ 

Copy link
Contributor

@Spaceman1701 Spaceman1701 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! This LGTM

We'll need to wait for a real maintainer to merge this.

@SuperQ SuperQ merged commit 7d6cebe into prometheus:main Nov 3, 2025
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants