May 14, 2019
The finality gadget
Putting eth2.0 to work…
As we get closer to the launch of Ethereum 2.0, you may be wondering about the fate of the existing Ethereum blockchain. A number of ideas aimed at increasing scalability of the existing network in the short-term have been proposed under the umbrella term “Ethereum 1.x.” The goal of these efforts is to ensure continued, smooth operation of the current blockchain to promote the sustainability of the ecosystem at large.
This post introduces one of these initiatives known as the “finality gadget.” The finality gadget leverages the consensus of Phase 0 of Ethereum 2.0 to enhance the security of the existing 1.0 chain; this initiative represents a way the Ethereum community can get value from the 2.0 deployment from day one. After introducing the concept at a high-level, we discuss how it works, how the community benefits and what deployment looks like. We conclude by reflecting on some outstanding risks and open questions.
Introduction
The idea behind the finality gadget is to leverage a property, called finality, of the Casper proof-of-stake consensus protocol to be deployed in Ethereum 2.0. Phase 0 of Ethereum 2.0 is the deployment of the beacon chain, the main system-level chain in the sharded blockchain. Validators participating in the beacon chain consensus generate finality on this data after some conditions have been met according to the Casper algorithm. We call this property finality because once a certain block is finalized, that block is in the canonical blockchain for all time; a reversion can only occur with the provable burning of more than 1/3 of the validating ETH at stake. Assuming a healthy level of 10M ETH participating in the system, a successful attack would result in the burning of around 3.3M ETH which at the time of writing is a cost of over $600M. I’ve written a bit more about finality here if you are curious for more detail: https://medium.com/@ralexstokes/how-secure-is-ethereum-2-0-consensus-41523a59f270.
The aim of the finality gadget is to use this process in the beacon chain to finalize Ethereum 1.0 blocks. If we can feed Ethereum 1.0 block data into Casper’s “finality engine”, then we can utilize the enhanced security of the proof-of-stake protocol on the existing proof-of-work network. Greater security opens up several avenues for improvement like reducing the mining subsidy (therefore decreasing supply issuance) and exposing the finalized block data to the EVM leading to more robust user-level applications. Leveraging Phase 0 for the finality gadget means we immediately get tangible benefits for the existing network from the launch of Ethereum 2.0, even though the full, phased deployment of 2.0 is expected to take more time.
The idea to use proof-of-stake consensus to finalize a proof-of-work blockchain has been floating around the ecosystem for some time (e.g. refer to EIP-1011 https://eips.ethereum.org/EIPS/eip-1011 or the Casper FFG publication https://arxiv.org/abs/1710.09437) and there is a nice presentation by Alexey Akhunov (beginning of the slides here: https://drive.google.com/file/d/16KLZKAutK79NxMh8L7B6hpNKuoOaAPZT/view) that goes into some of the recent history of the finality gadget in the context of Ethereum 2.0.
How it works
Let’s now look at how the first version of the finality gadget works, given the current status of the Ethereum 2.0 project. Note that while we are getting close to a final specification of the beacon chain, some details (particularly around parameters in the system) are still being discussed and are subject to change.
To build the finality gadget, we need two things: (i) a finality engine and (ii) a way to feed Ethereum 1.0 block data into this engine. It turns out that both tasks are achieved via the normal operation of the beacon chain. The beacon chain uses the Casper protocol for its consensus so any data included on-chain will be finalized in the course of normal activity. The second task is a little trickier as it implies that the beacon chain needs to be a light client of the Ethereum 1.0 chain. Luckily, Ethereum 2.0 will be bootstrapped from the Ethereum mainnet and by this process of bootstrapping we end up with Ethereum 1.0 block hashes included in the Ethereum 2.0 beacon chain.
A light client tracking deposits
To become a validator in the new Ethereum 2.0 system you send your collateral along with some registration data to a smart contract on the existing mainnet which then emits a log. This is a one-way deposit that can only be recovered by submitting a proof of the registration on the beacon chain. Once this deposit event has been processed, you enter into an activation queue which ultimately results in your inclusion in the set of active validators on the Ethereum 2.0 network.
In order to determine the validity of these deposit proofs, the beacon chain has to keep a record of the current state of the deposit smart contract on the Ethereum 1.0 chain. The relevant state at the time of writing is (i) the root of a Merkle tree of all deposits to date and (ii) the total number of deposits made to date (refer to the Eth1Datastructure in the current spec https://github.com/ethereum/eth2.0-specs/blob/dev/specs/core/0_beacon-chain.md). Inclusion proofs are made against this Merkle root in the usual way and the count of total deposits is used to ensure any deposits included in this Merkle tree are in fact processed (blocks are invalid if they lack the maximum number of possible outstanding deposits). Importantly for the finality gadget, the block hash of the Ethereum 1.0 block corresponding to the state of this contract is also included.
Validators are required to submit their view of the state of this contract in each beacon block produced. If a majority of all submissions match in a given period of time, the protocol deems consensus on this Ethereum 1.0 data and it is recorded in the beacon chain’s state as such. As the beacon chain blocks are finalized, the corresponding state for each block becomes finalized; once a given Ethereum 1.0 data has been included in a finalized state, we can say that the corresponding Ethereum 1.0 block has been finalized. This is the basic construction of the finality gadget.
Parameters to the finality gadget
There are two important parameters relating to the timing and frequency of this process to update the beacon chain with new Ethereum 1.0 data. They are integral to the quality of the finality gadget.
The first parameter is the number of SLOTS_PER_ETH1_VOTING_PERIOD which indicates how many slots (a slot is just a duration of time, e.g. 6 seconds) occur before pending votes to update the Ethereum 1.0 data are removed from the state. This length of time is the period during which a running tally of each unique 1.0 vote is kept in the state. The rolling set of pending votes is cleared in every slot that is a multiple of SLOTS_PER_ETH1_VOTING_PERIOD. The Ethereum 1.0 data used for deposit proofs can only be updated by having a majority of validators indicating the same data in this period of time. If a majority is not reached, then the pending votes are cleared and the process starts over leaving the existing 1.0 data unchanged.
The second important parameter is the ETH1_FOLLOW_DISTANCE. This number indicates how far behind the tip of the 1.0 chain a validator should look for candidate blocks to update the beacon chain’s state. Validators are directed to watch the chain for new 1.0 data candidates coming in and pick the majority candidate if one exists. If there is no such candidate (for example, because we just went through another aligned cycle of SLOTS_PER_ETH1_VOTING_PERIOD slots), then the validator should go to the tip of their 1.0 chain and find the block that is ETH1_FOLLOW_DISTANCE behind. For example, if the follow distance is 1024, and the tip of the chain is at block 8M, then we should find the state of the deposit contract at block 7,998,976 and include it along with that block’s hash in our beacon chain block.
This parameter is not enshrined in the beacon chain consensus but is provided as a guideline for an honest validator to follow. This constant helps validators coordinate around increasingly higher Ethereum 1.0 blocks (making progress on the deposit mechanism) and has the side effect of fixing a maximum height (at a given point in time) at which the Ethereum 1.0 chain could become finalized.
Aren’t we skipping a lot of blocks?
Realize that given the “chain” nature of the 1.0 blockchain, finalizing a block at height N implicitly finalizes every block in that chain below height N all the way back to the genesis block. This means that the set of finalized blocks on the 1.0 chain will suddenly jump some number of blocks according to a periodic cycle determined by the previous two parameters and the time to finality on the beacon chain. Given the existing constants for the beacon chain, this period of time works out to about every 2 hours in the optimal case. As we will see below, progress can be slowed or even stopped under adversarial conditions.
Coming full circle
As described so far, the existing 1.0 chain doesn’t have to know anything about the new system. To fully implement the finality gadget, we have to go one step further. This last step involves a fork in which the Ethereum 1.0 fork choice is altered to respect finality. Concretely, Ethereum 1.0 would still use GHOST to find the canonical chain, but it would start this search from the most recent finalized block (as determined by the beacon chain validators), not the genesis block as happens today. The fork to enable the finality gadget is where most of the work (from the 1.0 perspective) comes into play, as 1.0 clients now have to be aware of the 2.0 consensus. However, modification of the fork choice rule lets the security of finality extend to the proof-of-work chain, allowing the realization of all of the benefits allotted by this extra security.
Improving the finality gadget with the finality ratchet
Clients can get a big increase in quality from the finality gadget if the device sketched above is successfully deployed. This gain in quality comes from shrinking the 2.0 ETH1_FOLLOW_DISTANCE to as small a number as possible. The closer finality can trail behind the tip of the 1.0 chain, the greater the additional security margin and the higher the quality of finalized data exposed to users.
The ideal scenario is having a follow distance of just a few blocks (to account for baseline latency in block propagation). Ethereum 2.0 starts out with a much larger follow distance (currently1024blocks) in part because the finality gadget doesn’t exist (yay, bootstrapping!). The way this works has to do with the deposit mechanism. Deposits are required to be processed in the order of their creation and are required to be submitted in the next available block once the chain learns of their existence. In the event that a reorg of the proof-of-work chain occurs about a block that the beacon chain has considered canonical, the stream of deposits from the 1.0 chain would become invalidated (invalidating the 2.0 validator set) and would likely break the potential for any future deposits (by breaking the machinery that ensure deposits are processed sequentially) and could even halt the 2.0 chain (as no validators could propose valid blocks). A large follow distance makes this outcome unlikely as the locus of finality will effectively have 1024 proof-of-work confirmations on top of it. If 2.0 clients can count on 1.0 clients to respect finality (avoiding reorgs past a certain depth), then the follow distance can be shortened. At a high level, you can imagine the “highest possible” candidate for finality to “ratchet” towards the tip of the Ethereum 1.0 chain in a progressive manner as the follow distance is lowered, until a safe minimum is reached. How short the follow distance can get is a subject for future research/discussion.
Why do we want it?
Reducing issuance
Finalizing the 1.0 chain has a number of benefits. A big one is that the mining subsidy could be reduced assuming a well-functioning finality gadget. By sharing security from the beacon chain to the proof-of-work chain, the proof-of-work chain can retain the same level of security while reducing dependency on the miner’s hashrate. Because we wouldn’t need as much hashpower to secure the chain, we are safe to lower the mining subsidy (therefore offering a less attractive incentive for miners and reducing aggregate mining activity). An immediate side-effect here is lowering the issuance rate of ETH and the corresponding total supply, a goal that generally seems to be supported by the community. After successfully deploying the finality gadget on mainnet with a modified fork choice, this benefit could be achieved via another fork that gradually lowers the block reward like described in EIP-1011 here: http://eips.ethereum.org/EIPS/eip-1011#block-reward.
While the exact plans are still up for discussion, reducing issuance via the finality gadget is a concrete first step to migrate the 1.0 chain to the 2.0 system. A smooth transition will happen gradually over time and the process can begin with the gradual issuance reduction here. We as a community can support the larger migration into the future by supporting this initiative today.
Finality for applications
Once 1.0 clients are aware of finality, this data can be exposed to smart contracts and related applications utilizing the EVM. Finality is a useful property to have as the current state-of-the-art involves waiting a “good enough” number of “confirmations” where it becomes unlikely (but still possible) for a proof-of-work reorg to occur. Picture an exchange that currently waits for a large number of confirmations before releasing user funds in a withdrawal operation. If this exchange instead waits for finality, it can shorten this length of time giving a better user experience.
Smart contracts can take advantage of finality, too. Prediction markets are an example where waiting for finality at some height is useful before determining the outcome of an event. The deployment of this benefit looks like a fork (possibly the same one reducing issuance) adding a set of opcodes so that the EVM can determine if a particular block hash or block height is finalized. A similar EIP to this feature would be the addition of new opcodes like the bitwise shifting instructions in EIP-145 (https://eips.ethereum.org/EIPS/eip-145).
Putting eth2.0 to work…
The finality gadget represents a benefit we can get as soon as Phase 0 of Ethereum 2.0 is launched. Ethereum 2.0 is being deployed in a series of phases: Phase 0, Phase 1, and Phase 2. Each phase introduces another layer of complexity to the sharded system and while the 2.0 teams are working to launch Phase 0 by the end of 2019, the remaining phases are expected to take more time. Given that user-level accounts resembling those found in Ethereum 1.0 today are not available until Phase 2, it is worth pointing out that the community stands to realize substantial benefits at each phase of the 2.0 deployment.
With the launch of Phase 0, we can introduce the finality gadget to 1.0 and get all of the benefits laid out in this article. With Phase 1, Ethereum 2.0 gains shard chains that hold arbitrary data. Implementing the finality gadget means that 1.0 clients are now light clients of Ethereum 2.0. With light client functionality, proofs can be made in 1.0 smart contracts against this data in the shard chains. This functionality unlocks a number of interesting use cases for layer 2 scaling techniques like high-performance Plasma chains or privacy-enabling zk-SNARKs applications. I’ll refer you to Vitalik’s recent presentation on “Scalable blockchains as data layers” for more on this: https://www.youtube.com/watch?v=mOm47gBMfg8.
How do we get it?
The path to deployment of the finality gadget involves several moving pieces and may take more than a few upgrades to enhance its effect; in short, it is a complex initiative that will take a lot of work :)
The first step is to refine the idea of the finality gadget as given here in the form of proof-of-concept prototypes and simulations to better understand the ramifications of coupling the two systems in this way. A good outcome here would be the implementation of the finality gadget in one of the popular mainnet clients. This way we can start collecting live data on the operation of the finality gadget to gain confidence it follows the consensus of the existing clients.
In parallel to the work on the finality gadget, we need to develop and launch the beacon chain; needless to say, we need the beacon chain finalizing blocks before we can deploy the finality gadget.
Once we have a clear understanding of exactly what we want to deploy, we can author an EIP to specify the process and move through the steps to get the change included in an upcoming fork. I think it is sensible to deploy the finality gadget itself before discussing a reduction to the mining subsidy or the introduction of new EVM semantics. This suggestion is in the spirit of “make one change at a time” and makes it clear at each stage which parts of the finality gadget have which effects. We wouldn’t want to reduce the block reward when it turns out that the finality gadget doesn’t work quite how we thought leading to a loss of security on mainnet.
Risks to the finality gadget
In broad strokes, the finality gadget represents a bidirectional coupling between the 1.0 chain and the 2.0 beacon chain. The more coupled the systems are, the greater the attack surface and therefore greater cause for concern. We will want to do a thorough analysis of the coupling in each direction before suggesting a specific design for the finality gadget.
The main liability of the finality gadget is the process of getting new 1.0 data into the beacon chain. Validators on the beacon chain are required to be light clients of Ethereum 1.0; any beacon chain blocks that form consensus around a 1.0 block hash that is not in a particular validator’s view of the 1.0 chain is considered invalid. This validity condition ensures that as long as a majority of the 2.0 validators are honest, no invalid data with respect to the 1.0 chain will become finalized. In the case where more than 2/3 of the validators are malicious, it will be possible for them to collude towards finalizing a 1.0 block that may or may not follow the underlying proof-of-work consensus. This concern is mitigated by having a large initial validator set and taking a conservative approach to the deployment of the finality gadget. If the beacon chain launches and it becomes clear that the validator set does not have an honest majority then we have an opportunity to rethink our approach. The design of 2.0 and the finalty gadget ensure that if this happens, no changes or impact are made to the Ethereum 1.0 blockchain.
There is another concern given the way validators are expected to coordinate around the next block for a voting period that would not affect the correctness of the finality gadget, but affects the performance. The current specification states that validators choose the majority block suggested by previous block proposers if that block is in their view of the chain. In practice this means that honest validators should pick the very first valid suggestion with each round of voting and all coalesce on that one value. The block in this suggestion would be the next candidate for finalization so a failure to coordinate around the highest possible block (given the allotted SLOTS_PER_ETH1_VOTING_PERIOD) suggests the finality gadget is not finalizing 1.0 blocks as quickly as is possible. This concern can be mitigated by further research into better voting strategies to encourage expedient finalization of the 1.0 chain.
A final category of risks concerns the behavior of the 1.0 fork choice when the follow distance is very short and the 1.0 chain is under attack resulting in (possibly many) short-range forks. There is a trade-off between the quality of finality (determined by the distance between the finalized head and the GHOST head of the chain) and risk of an unstable fork choice on the proof-of-work chain. Future research can try to get a better understanding of this trade-off via simulations and can inform the ultimate finality gadget proposal.
Attacks like these have the effect of reducing or even halting the finalization of the 1.0 chain, degrading the finality gadget’s performance and being particularly risky in the case where we rely on finality for the chain’s security.
Open questions
- Assuming we have a functioning finality gadget, how do we want to change the mining subsidy (if at all)? Should it be a flat reduction in one hard fork? Should the issuance be a function of the follow distance? Can we map the economic security of finality (denominated in ETH) to the security derived from hashrate on mainnet today?
- How small can the follow distance be? How should it be adjusted? Slowly over time or is it safe to reduce it near the minimum after deploying the finality gadget?
- Is the voting process for Ethereum 1.0 block data on the beacon chain sufficient for a performant functioning of the finality gadget?
Conclusion
Hopefully you better understand what the finality gadget is, why we want it and a path forward to get it. There are a number of open questions we need to work through and any help on that front would be great!
If you are interested in helping, Alexey and I are starting a working group to organize efforts towards making the finality gadget a reality. You can get involved here:
I would like to say thanks to Danny Ryan for many helpful discussions in better understanding the beacon chain and how we can employ its finality in Ethereum 1.0 and to Alexey Akhunov for getting things started on the working group. Thanks to Terence Tsao for feedback on this article.
Update July 2019: translation to Chinese at https://ethfans.org/posts/the-finality-gadget-by-alex-stokes
