Some error code that signifies that retry is in order. Unavailable maybe.
{"level":"warn","ts":"2026-04-21T22:04:25.607018+0200","caller":"etcdserver/raft.go:387","msg":"leader failed to send out heartbeat on time; took too long, leader is overloaded likely from slow disk","to":"95012eee2c61378b","heartbeat-interval":"250ms","expected-duration":"500ms","exceeded-duration":"1.484060419s"}
{"level":"warn","ts":"2026-04-21T22:04:25.607340+0200","caller":"etcdserver/raft.go:387","msg":"leader failed to send out heartbeat on time; took too long, leader is overloaded likely from slow disk","to":"836bb8986617b0d6","heartbeat-interval":"250ms","expected-duration":"500ms","exceeded-duration":"1.484390506s"}
{"level":"warn","ts":"2026-04-21T22:04:25.742549+0200","caller":"etcdserver/v3_server.go:911","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":10377592809537642783,"retry-timeout":"500ms"}
{"level":"info","ts":"2026-04-21T22:04:26.006035+0200","logger":"raft","caller":"v3@v3.6.0/raft.go:1101","msg":"a77ddc750103e129 [logterm: 216, index: 26712135, vote: a77ddc750103e129] ignored MsgPreVote from 95012eee2c61378b [logterm: 216, index: 26712130] at term 216: lease is not expired (remaining ticks: 1)"}
{"level":"info","ts":"2026-04-21T22:04:26.006102+0200","logger":"raft","caller":"v3@v3.6.0/raft.go:1101","msg":"a77ddc750103e129 [logterm: 216, index: 26712135, vote: a77ddc750103e129] ignored MsgVote from 95012eee2c61378b [logterm: 216, index: 26712130] at term 216: lease is not expired (remaining ticks: 1)"}
{"level":"info","ts":"2026-04-21T22:04:26.006137+0200","logger":"raft","caller":"v3@v3.6.0/raft.go:1116","msg":"a77ddc750103e129 [term: 216] received a MsgHeartbeat message with higher term from 95012eee2c61378b [term: 217]"}
{"level":"info","ts":"2026-04-21T22:04:26.006207+0200","logger":"raft","caller":"v3@v3.6.0/raft.go:897","msg":"a77ddc750103e129 became follower at term 217"}
{"level":"info","ts":"2026-04-21T22:04:26.006238+0200","logger":"raft","caller":"v3@v3.6.0/node.go:372","msg":"raft.node: a77ddc750103e129 changed leader from a77ddc750103e129 to 95012eee2c61378b at term 217"}
{"level":"warn","ts":"2026-04-21T22:04:26.006316+0200","caller":"etcdserver/server.go:939","msg":"Failed to check current member's leadership","error":"etcdserver: leader changed"}
{"level":"info","ts":"2026-04-21T22:04:26.008988+0200","logger":"raft","caller":"v3@v3.6.0/log.go:158","msg":"found conflict at index 26712131 [existing term: 216, conflicting term: 217]"}
{"level":"info","ts":"2026-04-21T22:04:26.009030+0200","logger":"raft","caller":"v3@v3.6.0/log_unstable.go:203","msg":"replace the unstable entries from index 26712131"}
{"level":"info","ts":"2026-04-21T22:04:26.019329+0200","caller":"etcdserver/v3_server.go:903","msg":"first commit in current term: resending ReadIndex request"}
{"level":"warn","ts":"2026-04-21T22:04:26.039277+0200","caller":"etcdserver/server.go:946","msg":"Current member isn't a leader","local-member-id":12069044971509113129,"new-lead":10736914587109635979}
{"level":"warn","ts":"2026-04-21T22:04:26.039397+0200","caller":"etcdserver/server.go:946","msg":"Current member isn't a leader","local-member-id":12069044971509113129,"new-lead":10736914587109635979}
...
Bug report criteria
What happened?
Due to a performance hiccup there was a leader switch, which apparently caused
/v3/lease/keepaliverequests return an error:{"code": 2, "message": "not a primary lessor"}. Everything is as expected, except the Unknown error code.What did you expect to happen?
Some error code that signifies that retry is in order. Unavailable maybe.
How can we reproduce it (as minimally and precisely as possible)?
Have
s.ensureLeadership()check fail inEtcdServer.LeaseRenew().Anything else we need to know?
No response
Etcd version (please run commands below)
3.6.7
Etcd configuration (command line flags or environment variables)
No response
Etcd debug information (please run commands below, feel free to obfuscate the IP address or FQDN in the output)
No response
Relevant log output
{"level":"warn","ts":"2026-04-21T22:04:25.607018+0200","caller":"etcdserver/raft.go:387","msg":"leader failed to send out heartbeat on time; took too long, leader is overloaded likely from slow disk","to":"95012eee2c61378b","heartbeat-interval":"250ms","expected-duration":"500ms","exceeded-duration":"1.484060419s"} {"level":"warn","ts":"2026-04-21T22:04:25.607340+0200","caller":"etcdserver/raft.go:387","msg":"leader failed to send out heartbeat on time; took too long, leader is overloaded likely from slow disk","to":"836bb8986617b0d6","heartbeat-interval":"250ms","expected-duration":"500ms","exceeded-duration":"1.484390506s"} {"level":"warn","ts":"2026-04-21T22:04:25.742549+0200","caller":"etcdserver/v3_server.go:911","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":10377592809537642783,"retry-timeout":"500ms"} {"level":"info","ts":"2026-04-21T22:04:26.006035+0200","logger":"raft","caller":"v3@v3.6.0/raft.go:1101","msg":"a77ddc750103e129 [logterm: 216, index: 26712135, vote: a77ddc750103e129] ignored MsgPreVote from 95012eee2c61378b [logterm: 216, index: 26712130] at term 216: lease is not expired (remaining ticks: 1)"} {"level":"info","ts":"2026-04-21T22:04:26.006102+0200","logger":"raft","caller":"v3@v3.6.0/raft.go:1101","msg":"a77ddc750103e129 [logterm: 216, index: 26712135, vote: a77ddc750103e129] ignored MsgVote from 95012eee2c61378b [logterm: 216, index: 26712130] at term 216: lease is not expired (remaining ticks: 1)"} {"level":"info","ts":"2026-04-21T22:04:26.006137+0200","logger":"raft","caller":"v3@v3.6.0/raft.go:1116","msg":"a77ddc750103e129 [term: 216] received a MsgHeartbeat message with higher term from 95012eee2c61378b [term: 217]"} {"level":"info","ts":"2026-04-21T22:04:26.006207+0200","logger":"raft","caller":"v3@v3.6.0/raft.go:897","msg":"a77ddc750103e129 became follower at term 217"} {"level":"info","ts":"2026-04-21T22:04:26.006238+0200","logger":"raft","caller":"v3@v3.6.0/node.go:372","msg":"raft.node: a77ddc750103e129 changed leader from a77ddc750103e129 to 95012eee2c61378b at term 217"} {"level":"warn","ts":"2026-04-21T22:04:26.006316+0200","caller":"etcdserver/server.go:939","msg":"Failed to check current member's leadership","error":"etcdserver: leader changed"} {"level":"info","ts":"2026-04-21T22:04:26.008988+0200","logger":"raft","caller":"v3@v3.6.0/log.go:158","msg":"found conflict at index 26712131 [existing term: 216, conflicting term: 217]"} {"level":"info","ts":"2026-04-21T22:04:26.009030+0200","logger":"raft","caller":"v3@v3.6.0/log_unstable.go:203","msg":"replace the unstable entries from index 26712131"} {"level":"info","ts":"2026-04-21T22:04:26.019329+0200","caller":"etcdserver/v3_server.go:903","msg":"first commit in current term: resending ReadIndex request"} {"level":"warn","ts":"2026-04-21T22:04:26.039277+0200","caller":"etcdserver/server.go:946","msg":"Current member isn't a leader","local-member-id":12069044971509113129,"new-lead":10736914587109635979} {"level":"warn","ts":"2026-04-21T22:04:26.039397+0200","caller":"etcdserver/server.go:946","msg":"Current member isn't a leader","local-member-id":12069044971509113129,"new-lead":10736914587109635979} ...