By Wenceslas Sanchez
A very special event occurred on Ethereum between epochs 200754 and 200760 (timestamped at 2023-05-12 17:45:59 and 2023-05-12 18:24:23): there was an inactivity leak, and the first one encountered by Ethereum since the beginning of the Beacon chain.
An inactivity leak is a state of emergency on the Ethereum blockchain triggered by the beacon chain’s failure to finalize a checkpoint for more than four epochs. More precisely, this happens when more than one-third of active validators (expressed in terms of total active stake) are absent (meaning they are not attesting).
Why is finality important?
Finality refers to the validation by two-thirds of validators at the checkpoint (the first block of the epoch). It guarantees that a block cannot be changed/removed from the blockchain without burning at least one-third of the total active stake following the Ethereum Foundation blog (that’s because two-thirds of all validators committed their stake to bet that a state is finalized).
On Beaconcha.in, we can see that at epoch 200759 the participation rate was around 98% (98% of the active stake were committed) while at epoch 200751, this rate was around 42% (i.e. less than two-third of the staked ETH). This means that many validators were absent.
If a lot of active validators are absent, then finality can’t be targeted and the blockchain is weakened. The network’s defense mechanism results in the inactivity leak.
How does the inactivity leak work?
When the network enters this state of emergency, the stakes of inactive validators are reduced until they come back online or until the active validators control two-thirds of the active stake again.
This is done with the help of an inactivity score assigned to each validator, which is increased if a validator is inactive during the inactivity leak period (this score is used to compute the penalty; the higher the score, the higher the penalty).
When the inactivity leak period is over (meaning that there are less than four unfinalized epochs), the inactivity score tends to zero for all active validators.
What caused this loss of finality and in turn the inactivity leak?
The most likely cause of this situation is a bug in some consensus client implementation. Performance issues have been reported by Prysm (used by almost 38% of all validators) and Teku (17%), following the inactivity leak. Both released emergency fixes (Prysm v4.0.3 and Teku v23.5.0) a few hours after the issue was reported. In the meantime, validators that used these clients were inactive. It seems that other clients like Lighthouse or Nimbus were not affected.
How did the network recover?
One of the Ethereum blockchain’s main strenghts is its strong Consensus client diversity.
Since there are different implementations of the consensus layer but no strong majority in client usage, the blockchain recovered and went back to its normal state after six epochs (the end of the inactivity leak).
What can we learn from this issue?
Consensus client diversity is indispensable when we are talking about decentralized ledgers; it’s necessary but not sufficient— something similar could have happened with the execution client which, evidently, doesn’t have such strong diversity. Some blockchains are even worse with only one client implementation (Polygon or Solana for instance). For more information about Ethereum client diversity, read this article from Ethereum Foundation Blog.
Fun fact (maybe not that fun?):
Some validators exited during the inactivity leak, which means that even some epoch after the end of the inactivity leak, they kept inactivity scores other than 0. This is the case for validator 619397, who continues to keep an inactivity score of 20 until they come back online, or until their index is reused for another validator (EIP6914).
Sources:
– [Ethereum website](https://ethereum.org/en/)
– [Ethereum Foundation blog](https://blog.ethereum.org/)
– [Ethereum Consensus specs](https://github.com/ethereum/consensus-specs)
– [beaconcha.in](https://beaconcha.in/)
– [clientdiversity.org](https://clientdiversity.org/)
– [eth2book.info](https://eth2book.info/capella/)
– [Casper paper](https://arxiv.org/abs/1710.09437)
