MakerDAO

Highlight to Annotate. Click to Learn.

The Dai Stablecoin System

Overview of the Dai Stablecoin System

Popular digital assets such as Bitcoin (BTC) and Ether (ETH) are too volatile to be used as everyday currency. The value of a bitcoin often experiences large fluctuations, rising or falling by as much as 25% in a single day and occasionally rising over 300% in a month 1.

The Dai Stablecoin is a collateral-backed cryptocurrency whose value is stable relative to the US Dollar. We believe that stable digital assets like Dai Stablecoin are essential to realizing the full potential of blockchain technology.

Maker is a smart contract platform on Ethereum that backs and stabilizes the value of Dai through a dynamic system of Collateralized Debt Positions (CDPs), autonomous feedback mechanisms, and appropriately incentivized external actors.

Maker enables anyone to leverage their Ethereum assets to generate Dai on the Maker Platform. Once generated, Dai can be used in the same manner as any other cryptocurrency: it can be freely sent to others, used as payments for goods and services, or held as long-term savings. Importantly, the generation of Dai also creates the components needed for a robust decentralized margin trading platform.

Collateralized Debt Position Smart Contracts

Anyone who has collateral assets can leverage them to generate Dai on the Maker Platform through Maker’s unique smart contracts known as Collateralized Debt Positions. 2

CDPs hold collateral assets deposited by a user and permit this user to generate Dai, but generating also accrues debt. This debt effectively locks the deposited collateral assets inside the CDP until it is later covered by paying back an equivalent amount of Dai, at which point the owner can again withdraw their collateral . Active CDPs are always collateralized in excess, meaning that the value of the collateral is higher than the value of the debt.

The CDP interaction process

  • Step 1: Creating the CDP and depositing collateralThe CDP user first sends a transaction to Maker to create the CDP, and then sends another transaction to fund it with the amount and type of collateral that will be used to generate Dai. At this point the CDP is considered collateralized.
  • Step 2: Generating Dai from the collateralized CDPThe CDP user then sends a transaction to retrieve the amount of Dai they want from the CDP, and in return the CDP accrues an equivalent amount of debt, locking them out of access to the collateral until the outstanding debt is paid.
  • Step 3: Paying down the debt and Stability FeeWhen the user wants to retrieve their collateral, they have to pay down the debt in the CDP, plus the Stability fee that continuously accrue on the debt over time. The Stability Fee can only be paid in MKR. Once the user sends the requisite Dai and MKR to the CDP, paying down the debt and Stability Fee, the CDP becomes debt free.
  • Step 4: Withdrawing collateral and closing the CDPWith the Debt and Stability Fee paid down, the CDP user can freely retrieve all or some of their collateral back to their wallet by sending a transaction to Maker. Single-Collateral Dai vs Multi-Collateral Dai Dai will initially launch with support for only one type of collateral, Pooled Ether. In the next 6-12 months we plan to upgrade Single-Collateral Dai to Multi-Collateral Dai. The primary difference is that it will support any number of CDP types. 3

Pooled Ether (Temporary mechanism for Single-Collateral Dai)

At first, Pooled Ether (PETH) will be the only collateral type accepted on Maker. Users who wish to open a CDP and generate Dai during the first phase of the Maker Platform need to first obtain PETH. This is done instantly and easily on the blockchain by depositing ETH into a special smart contract that pools the ETH from all users, and gives them corresponding PETH in return.

If there is a sudden market crash in ETH, and a CDP ends up containing more debt than the value of its collateral, the Maker Platform automatically dilutes the PETH to recapitalize the system. This means that the proportional claim of each PETH goes down. After the Maker Platform is upgraded to support multiple collateral types, PETH will be removed and replaced by ETH alongside the other new collateral types.

Price Stability Mechanisms

Target Price

The Dai Target Price has two primary functions on the Maker Platform: 1) It is used to calculate the collateral-to-debt ratio of a CDP, and 2) It is used to determine the value of collateral assets Dai holders receive in the case of a global settlement. The Target Price is initially denominated in USD and starts at 1, translating to a 1:1 USD soft peg.

Target Rate Feedback Mechanism

In the event of severe market instability, the Target Rate Feedback Mechanism (TRFM) can be engaged. Engaging the TRFM breaks the fixed peg of Dai, but maintains the same denomination.

The TRFM is the automatic mechanism by which the Dai Stablecoin System adjusts the Target Rate in order to cause market forces to maintain stability of the Dai market price around the Target Price. The Target Rate determines the change of the Target Price over time, so it can act either as an incentive to hold Dai (if the Target Rate is positive) or an incentive to borrow Dai (If the Target Rate is negative). When the TRFM is not engaged the target rate is fixed at 0%, so the target price doesn’t change over time and Dai is pegged. When the TRFM is engaged, both the Target Rate and the Target Price change dynamically to balance the supply and demand of Dai by automatically adjusting user incentives for generating and holding Dai. The feedback mechanism pushes the market price of Dai towards the variable Target Price, dampening its volatility and providing real-time liquidity during demand shocks.

With the TRFM engaged, when the market price of Dai is below the Target Price, the Target Rate increases. This causes the Target Price to increase at a higher rate, causing generation of Dai with CDPs to become more expensive. At the same time, the increased Target Rate causes the capital gains from holding Dai to increase, leading to a corresponding increase in demand for Dai. This combination of reduced supply and increased demand causes the Dai market price to increase, pushing it back up towards the Target Price.

The same mechanism works in reverse if the Dai market price is higher than the Target Price: the Target Rate decreases, leading to an increased demand for generating Dai and a decreased demand for holding it. This causes the Dai market price to decrease, pushing it down towards the Target Price.

This mechanism is a negative feedback loop: Deviation away from the Target Price in one direction increases the force in the opposite direction.

Sensitivity Parameter

The TRFM’s Sensitivity Parameter is a parameter that determines the magnitude of Target Rate change in response to Dai target/market price deviation. This tunes the rate of feedback to the scale of the system. MKR voters can set the Sensitivity Parameter but when the TRFM is engaged the Target Price and the Target Rate are determined by market dynamics, and not directly controlled by MKR voters.

The Sensitivity Parameter is also what is used to engage or disengage the TRFM. If the Sensitivity Parameter and the Target Rate are both zero, Dai is pegged to the current Target Price.

Global Settlement

Global settlement is a process that can be used as a last resort to cryptographically guarantee the Target Price to holders of Dai. It shuts down and gracefully unwinds the Maker Platform while ensuring that all users, both Dai holders and CDP users, receive the net value of assets they are entitled to. The process is fully decentralized, and MKR voters govern access to it to ensure that it is only used in case of serious emergencies. Examples of serious emergencies are long term market irrationality, hacking or security breaches, and system upgrades.

Global Settlement: Step by Step

  • Step 1: Global Settlement is activatedIf enough actors who have been designated as global settlers by Maker Governance believe that the system is subject to a serious attack, or if a global settlement is scheduled as part of a technical upgrade, they can active the Global Settlement function. This stops CDP creation and manipulation, and freezes the Price Feed at a fixed value that is then used to process proportional claims for all users.
  • Step 2: Global Settlement claims are processedAfter Global Settlement has been activated, a period of time is needed to allow keepers to process the proportional claims of all Dai and CDP holders based on the fixed feed value. After this processing is done, all Dai holders and CDP holders will be able to claim a fixed amount of ETH with their Dai and CDPs.
  • Step 3: Dai and CDP holders claim the collateral with their Dai and CDPsEach Dai and CDP holder can call a claim function on the Maker Platform to exchange their Dai and CDPs directly for a fixed amount of ETH that corresponds to the calculated value of their assets, based on the target price of Dai. E.g. If the Dai Target Price is 1 U.S. Dollar, The ETH/USD Price is 200 and a user holds 1000 Dai when Global Settlement is activated, after the processing period they will be able to claim exactly 5 ETH from the Maker Platform. There is no time limit for when the final claim can be made.

Risk Management of The Maker Platform

The MKR token allows holders to vote to perform the following Risk Management actions:

  • Add new CDP type: Create a new CDP type with a unique set of Risk Parameters. A CDP type can either be a new type of collateral, or a new set of Risk Parameters for an existing collateral type.
  • Modify existing CDP types: Change the Risk Parameters of one or more existing CDP types that were already added
  • Modify Sensitivity Parameter: Change the sensitivity of the Target Rate Feedback Mechanism
  • Modify Target Rate: Governance can change the Target Rate. In practice modifying the Target Rate will only be done in one specific circumstance: When MKR voters want to peg the price of Dai to its current Target Price. It will always be done in conjunction with modifying the Sensitivity Parameter. By setting both Sensitivity Parameter and Target Rate to 0%, the TRFM becomes disabled and the Target Price of Dai becomes pegged to its current value.
  • Choose the set of trusted oracles: The Maker Platform derives its internal prices for collateral and the market price of Dai from a decentralized oracle infrastructure, consisting of a wide set of individual oracle nodes. MKR voters control how many nodes are in the set of trusted oracles, and who those nodes are. Up to half of the oracles can be compromised or malfunction without causing a disruption to the continued safe operation of the system
  • Modify Price Feed Sensitivity: Change the rules that determine the largest change that the price feeds can affect on the internal price values in the system.
  • Choose the set of global settlers: Global settlement is a crucial mechanic that allows the Maker Platform to survive attacks against the oracles or the governance process. The governance process chooses a set of global settlers and determines how many settlers are needed to activate global settlement.

Risk Parameters

Collateralized Debt Positions have multiple Risk Parameters that enforce how they can be used. Each CDP type has its own unique set of Risk Parameters, and these parameters are determined based on the risk profile of the collateral used by the CDP type. These parameters are directly controlled by MKR holders through voting, with one MKR giving its holder one vote.

The key Risk Parameters for CDPs are:

  • Debt Ceiling: The Debt Ceiling is the maximum amount of debt that can be created by a single type of CDP. Once enough debt has been created by a CDP of any given type, it becomes impossible to create more unless existing CDPs are closed. The debt ceiling is used to ensure sufficient diversification of the collateral portfolio.
  • Liquidation Ratio: The Liquidation Ratio is the collateral-to-debt ratio at which a CDP becomes vulnerable to Liquidation. A low Liquidation Ratio means MKR voters expect low price volatility of the collateral, while a high Liquidation Ratio means high volatility is expected.
  • Stability Fee: The Stability Fee is a fee paid by every CDP. It is an annual percentage yield that is calculated on top of the existing debt of the CDP and has to be paid by the CDP user. The Stability Fee is denominated in Dai, but can only be paid using the MKR token. The amount of MKR that has to be paid is calculated based on a Price Feed of the MKR market price. When paid, the MKR is burned, permanently removing it from the supply.
  • Penalty Ratio: The Penalty Ratio is used to determined the maximum amount of Dai raised from a Liquidation Auction that is used to buy up and remove MKR from the supply, with excess collateral getting returned to the CDP user who owned the CDP prior to its liquidation. The Penalty Ratio is used to cover the inefficiency of the liquidation mechanism. During the phase of Single-Collateral Dai, the Liquidation Penalty goes to buy and burn of PETH, benefitting the PETH to ETH ratio.

MKR Token Governance

In addition to payment of the Stability Fee on active CDPs, the MKR token plays an important role in the governance of the Maker Platform.

Governance is done at the system level through election of an Active Proposal by MKR voters. The Active Proposal is the smart contract that has been empowered by MKR voting to gain root access to modify the internal governance variables of the Maker Platform. Proposals can be in two forms: Single Action Proposal Contracts [SAPC], and Delegating Proposal Contracts [DPC].

Single Action Proposal Contracts are proposals that can only be executed once after gaining root access, and after execution immediately applies its changes to the internal governance variables of the Maker Platform. After the one-time execution, the SAPC deletes itself and cannot be re-used. This type of proposal is what will be used during the first phases of the system, as it is not very complicated to use, but is less flexible.

Delegating Proposal Contracts are proposals that continuously utilize their root access through second layer governance logic that is codified inside the DPC. The second layer governance logic can be relatively simple, such as defining a protocol for holding a weekly vote on updated risk parameters. It can also implement more advanced logic, such as restrictions on the magnitude of governance actions within defined time periods, or even delegating some or all of its permissions further to one or more third layer DPCs with or without restrictions.

Any Ethereum account can deploy valid proposal smart contracts. MKR voters can then use their MKR tokens to cast approval votes for one or more proposals that they want to elect as the Active Proposal. The smart contract that has the highest total number of approval votes from MKR voters is elected as the Active Proposal.

MKR and Multi-Collateral Dai

After the upgrade to Multi-Collateral Dai, MKR will take on a more significant role in the Dai Stablecoin System by replacing PETH as the the recapitalization resource. When CDPs become undercollateralized due to market crashes, the MKR supply is automatically diluted and sold off in order to raise enough funds to recapitalize the system.

Automatic Liquidations of risky CDPs

To ensure there is always enough collateral in the system to cover the value of all outstanding Debt (according to the Target Price), a CDP can be liquidated if it is deemed to be too risky. The Maker Platform determines when to liquidate a CDP by comparing the Liquidation Ratio with the current collateral-to-debt ratio of the CDP.

Each CDP type has its own unique Liquidation Ratio that is controlled by MKR voters and established based on the risk profile of the particular collateral asset of that CDP type.

Liquidation occurs when a CDP hits its Liquidation Ratio. The Maker Platform will automatically buy the collateral of the CDP and subsequently sell it off. There is a temporary mechanism in place for Single-Collateral Dai called a Liquidity Providing Contract. For Multi-Collateral Dai an auction mechanism will be used.

Liquidity Providing Contract (Temporary mechanism for Single-Collateral Dai)

During Single-Collateral Dai, the mechanism for liquidation is a Liquidity Providing Contract: a smart contract that trades directly with ethereum users and keepers according to the price feed of the system.

When a CDP is liquidated, it is immediately acquired by the system. The CDP owner receives the value of the leftover collateral minus the debt, Stability Fee and Liquidation Penalty.

The PETH collateral is set for sale in the Liquidity Providing Contract, and keepers can atomically purchase the PETH by paying Dai. All Dai paid this way are immediately removed from the Dai supply, until an amount equal to the CDP debt has been removed. If any Dai is paid in excess of the debt shortfall, the excess Dai is used to purchase PETH from the market and burn it, which positively changes the ETH to PETH ratio. This results in a net value gain for PETH holders.

If the PETH selloff initially does not raise enough Dai to cover the entire debt shortfall, more PETH is continuously created and sold off. New PETH created this way negatively changes the ETH to PETH ratio, causing PETH holders to lose value.

Debt and Collateral Auctions (Multi-Collateral Dai)

During a liquidation, the Maker platform buys the collateral of a CDP and subsequently sells it in an automatic auction. This auction mechanism enables the system to settle CDPs even when price information is unavailable.

In order to take over the collateral of the CDP so that it can be sold, the system first needs to raise enough Dai to cover the CDP’s debt. This is called a Debt Auction, and works by diluting the supply of the MKR token and selling it to bidders in an auction format.

In parallel, the collateral of the CDP is sold in a Collateral Auction where all proceeds (also denominated in Dai) up to the CDP debt amount plus a Liquidation Penalty (A Risk Parameter determined by MKR voting) is used to buy MKR and remove it from the supply.

This directly counteracts the MKR dilution that happened during the Debt Auction. If enough Dai is bid to fully cover the CDP debt plus the Liquidation Penalty, the Collateral Auction switches to a reverse auction mechanism and tries to sell as little collateral as possible–any leftover collateral is returned to the original owner of the CDP.

Key External Actors

In addition to its smart contract infrastructure, the Maker Platform relies on certain external actors to maintain operations. Keepers are external actors who take advantage of the economic incentives presented by the Maker platform. Oracles and Global Settlers are external actors with special permissions in the system assigned to them by MKR voters.

Keepers

A keeper is an independent (usually automated) actor that is incentivized by profit opportunities to contribute to decentralized systems. In the context of the Dai Stablecoin System, keepers participate in the Debt Auctions and Collateral Auctions when CDPs are liquidated.

Keepers also trade Dai around the Target Price. Keepers sell Dai when the market price is higher than the Target Price and buy Dai when the market price is below the Target Price to profit from the expected long-term convergence towards the Target Price.

Oracles

The Maker Platform requires real time information about the market price of the assets used as collateral in CDPs in order to know when to trigger liquidations. The Maker Platform also needs information about the market price of Dai and its deviation from the Target Price in order to adjust the Target Rate when the TRFM is engaged. MKR voters choose a set of trusted oracles to feed this information to the Maker Platform through Ethereum transactions. To protect the system from an attacker who gains control of a majority of the oracles, and from other forms of collusion, there is a global variable that determines the maximum change to the value of the price feed permitted by the system. This variable is known as the Price Feed Sensitivity Parameter.

As an example of how the Price Feed Sensitivity Parameter works, if the Price Feed Sensitivity Parameter is defined as “5% in 15 minutes”, the price feeds cannot change more than 5% within one 15 minute period, and changing ~15% would take 45 minutes. This restriction ensures there is enough time to trigger a global settlement in the event that an attacker gains control over a majority of the oracles.

Global Settlers

Global Settlers are external actors similar to price feed oracles and are the last line of defense for the Dai Stablecoin System in the event of an attack. The set of global settlers, selected by MKR voters, have the authority to trigger global settlement. Aside from this authority, these actors do not have any additional special access or control within the system.

Examples

The Dai Stablecoin System can be used by anyone without any restrictions or sign-up process.

  • Example 1: Bob needs a loan, so he decides to generate 100 Dai. He locks an amount of ETH worth significantly more than 100 Dai into a CDP and uses it to generate 100 Dai. The 100 Dai is instantly sent directly to his Ethereum account. Assuming that the Stability Fee is 1% per year, Bob will need 101 Dai to cover the CDP if he decides to retrieve his ETH one year later.

One of the primary use cases of CDPs is margin trading by CDP users.

  • Example 2: Bob wishes to go margin long on the ETH/Dai pair, so he generates 100 USD worth of Dai by posting 150 USD worth of ETH to a CDP. He then buys another 100 USD worth of ETH with his newly generated Dai, giving him a net 1.66x ETH/USD exposure. He’s free to do whatever he wants with the 100 USD worth of ETH he obtained by selling the Dai. The original ETH collateral (150 USD worth) remains locked in the CDP until the debt plus the Stability Fee is covered.

Although CDPs are not fungible with each other, the ownership of a CDP is transferable. This allows CDPs to be used in smart contracts that perform more complex methods of Dai generation (for example, involving more than one actor).

  • Example 3: Alice and Bob collaborate using an Ethereum OTC contract to issue 100 USD worth of Dai backed by ETH. Alice contributes 50 USD worth of ETH, while Bob contributes 100 USD worth. The OTC contract takes the funds and creates a CDP, thus generating 100 USD worth of Dai. The newly generated Dai are automatically sent to Bob. From Bob’s point of view, he is buying 100 USD worth of Dai by paying the equivalent value in ETH. The contract then transfers ownership of the CDP to Alice. She ends up with 100 USD worth of debt (denominated in Dai) and 150 USD worth of collateral (denominated in ETH). Since she started with only 50 USD worth of ETH, she is now 3x leveraged long ETH/USD.

Liquidations ensure that in the event of a price crash of the collateral backing a CDP type, the system will automatically be able to close CDPs that become too risky. This ensures that the outstanding Dai supply remains fully collateralized.

  • Example 4: Let’s assume that there is an Ether CDP type with a Liquidation Ratio of 145%, a Penalty Ratio of 105%, and we have an Ether CDP with a collateral-to-debt ratio of 150% . The Ether price now crashes 10% against the Target Price, causing the collateral-to-debt ratio of the CDP to fall to ~135%. As it falls below the Liquidation Ratio, traders can trigger its Liquidation and begin bidding with Dai for buying MKR in the debt auction. Simultaneously, traders can begin bidding with Dai for buying the ~135 Dai worth of collateral in the collateral auction. Once there is at least 105 Dai being bid on the Ether collateral, traders reverse bid to take the least amount of collateral for 105 Dai. Any remaining collateral is returned to the CDP owner.

Addressable Market

As mentioned in the introduction, a cryptocurrency with price stability is a basic requirement for the majority of decentralized applications. As such, the potential market for Dai is at least as large as that of the entire blockchain industry. The following is a short, non-exhaustive list of some of the immediate markets (in both the blockchain and the wider industry) for the Dai Stablecoin System in its capacity as a cryptocurrency with price stability and its use case as a decentralized margin trading platform:

  • Prediction Markets & Gambling Applications: When making an unrelated prediction, it is obvious not to want to increase one’s risk by placing the bet using a volatile cryptocurrency. Long term bets become especially infeasible if the user has to also gamble on the future price of the volatile asset used to place the bet. Instead, a cryptocurrency with price stability like Dai will be the natural choice for prediction market and gambling users.
  • Financial Markets; Hedging, Derivatives, Leverage: CDPs will allow for permissionless leveraged trading. Dai will also be useful as stable and reliable collateral in custom derivative smart contracts, such as options or CFD’s.
  • Merchant receipts, Cross-border transactions and remittances: F oreign exchange volatility mitigation and a lack of intermediaries means the transaction costs of international trade can be significantly reduced by using Dai.
  • Transparent accounting systems: Charities, NGO’s and Governments will all see increases in efficiency and lower levels of corruption by utilizing Dai.

Risks and their Mitigation

There are many potential risks facing the successful development, deployment, and operation of the Maker Platform. It is vital that the Maker community takes all necessary steps to mitigate these risks. The following is a list spells out some of the risks identified and the accompanying plan for risk mitigation:

Malicious hacking attack against the smart contract infrastructure

The greatest risk to the system during its early stages is the risk of a malicious programmer finding an exploit in the deployed smart contracts, and using it to break or steal from the system before the vulnerability can be fixed. In a worst case scenario, all decentralized digital assets that are held as collateral in The Maker Platform, such as Ether (ETH) or Augur Reputation (REP), could be stolen without any chance of recovery. The part of the collateral portfolio that is not decentralized, such as Digix Gold IOU’s, would not be stolen in such an event as they can be frozen and controlled through a centralized backdoor.

Mitigation: Smart contract security and best security practices have been the absolute highest priority of the Dai development effort since its inception. The codebase has already undergone three independent security audits by some of the best security researchers in the blockchain industry.

In the very long term, the risk of getting hacked can theoretically be almost completely mitigated through formal verification of the code. This means mathematically proving that the code does exactly what it is intended to do. While complete formal verification is a very long term goal, significant work towards it has already been completed, including a full reference implementation of the Dai Stablecoin System in the functional programming language Haskell, which serves as a stepping stone towards more sophisticated formalizations that are currently under active research and development.

Black swan event in one or more collateral assets

Another high impact risk is a potential Black Swan event on collateral used for the Dai. This could either happen in the early stages of Dai Stablecoin System, before MKR is robust enough to support inflationary dilutions, or after the Dai Stablecoin System supports a diverse portfolio of collateral.

Mitigation:  CDP collateral will be limited to ETH in the early stages, with the debt ceiling initially limited and growing gradually over time.

Competition and the importance of ease-of-use

As mentioned previously, there is a large amount of money and brainpower working on cryptocurrency with price stability. By virtue of having “true decentralization”, the Dai Stablecoin System is by far the most complex model being contemplated in the blockchain industry. A perceived risk is a movement among cryptocurrency users where the ideals of decentralization are exchanged for the simplicity and marketing of centralized digital assets. Mitigation: We expect that Dai will be very easy to use for a regular cryptocurrency user. Dai will be a standard Ethereum token adhering to the ERC-20 standard and will be readily available with high liquidity across the ecosystem. Dai has been designed in such a way that the average user need not understand the underlying mechanics of the system in order to use it.

The complexities of the Dai Stablecoin System will need to be understood primarily by Keepers and capital investment companies that use the Dai Stablecoin System for margin trading. These types of users have enough resources to onboard themselves as long as there is abundant and clear documentation of every aspect of the system’s mechanics. The Maker community will ensure that this is the case.

Pricing errors, irrationality and unforeseen events

A number of unforeseen events could potentially occur, such as a problem with the price feed from the Oracles, or irrational market dynamics that cause variation in the value of Dai for an extended period of time. If confidence is lost in the system, the TRFM adjustments or even MKR dilution could reach extreme levels while still not bringing enough liquidity and stability to the market.

Mitigation:  The Maker community will need to incentivize a sufficiently large capital pool to act as Keepers of the market in order to maximize rationality and market efficiency and allow the Dai supply to grow at a steady pace without major market shocks.

Failure of centralized infrastructure

The Maker Team plays a major role in the development and governance of the Maker Platform in its early days: budgeting for expenses, hiring new developers, seeking partnerships and institutional users, and interfacing with regulators and other key external stakeholders. Should the Maker Team fail in some capacity — for legal reasons, or due to internal problems with management — the future of Maker could be at risk without a proper backup plan.

Mitigation:  The Maker community exists partly to act as the decentralized counterparty to the Maker Team. It is a loose collective of independent actors who are all aligned by holding the MKR token, giving them a strong incentive to see the Maker Platform succeed. During the early phases of MKR distribution, great care was taken to ensure that the most important core developers received a significant MKR stake. In the event that the Maker Team is no longer effectively able to lead the development of the Maker Platform, individual MKR holders will be incentivized to fund developers (or simply carry out development themselves) in an effort to protect their investment.

Conclusion

The Dai Stablecoin System was designed to solve the crucial problem of stable exchange of value in the Ethereum ecosystem and the wider blockchain economy. We believe that the mechanism through which Dai is created, transacted, and retired, along with the direct Risk Management role of MKR holders, will allow for self-interested Keepers to maintain the price stability of Dai over time in an efficient manner. The founders of the Maker community have established a prudent governance roadmap that is appropriate for the needs of agile development in the short term, but also coherent with the ideals of decentralization over time. The development roadmap is aggressive and focused on widespread adoption of Dai in a responsible fashion.

Glossary of Terms

  • Collateralized Debt Position (CDP): A smart contract whose users receive an asset (Dai), which effectively operates as a debt instrument with an interest rate. The CDP user has posted collateral in excess of the value of the loan in order to guarantee their debt position.
  • Dai: The cryptocurrency with price stability that is the asset of exchange in the Dai Stablecoin System. It is a standard Ethereum token adhering to the ERC20 standard.
  • Debt Auction: The reverse auction selling MKR for Dai to cover Emergency Debt when a CDP becomes undercollateralized.
  • Collateral Auction: The auction selling collateral from a CDP undergoing liquidation. It is designed to prioritize covering the debt owed by the CDP, and secondarily to give the CDP owner the best possible price for their excess collateral refund.
  • The Dai Foundation: A decentralized team of smart contract developers committed to the development and successful launch of the Maker Platform.
  • Keepers: Independent economic actors that trade Dai, CDPs and/or MKR; create Dai or close CDPs; and seek arbitrage on The Dai Stablecoin System. As a result, Keepers help maintain Dai market rationality and price stability.
  • MKR: The ERC20 token used by MKR voters for voting. It also serves as a backstop in the case of insolvent CDPs.
  • MKR Voters: MKR holders who actively manage the risk of the Dai Stablecoin System by voting on Risk Parameters.
  • Maker: The name of the Decentralized Autonomous Organization that is made up of the Maker Platform technical infrastructure, and the community of MKR voters.
  • Oracles: Ethereum accounts (either contracts or users) selected to provide price feeds into various components of Maker Platform.
  • Risk Parameters: The variables that determine (among other things) when the Maker Platform automatically judges a CDP to be Risky, allowing Keepers to liquidate it.
  • Sensitivity Parameter: The variable that determines how aggressively the Dai Stablecoin System automatically changes the Target Rate in response to Dai market price deviations.
  • Target Rate Feedback Mechanism (TRFM): The automatic mechanism by which the Dai Stablecoin System adjusts the Target Rate in order to cause market forces to maintain stability of the Dai market price around the Target Price.

Footnotes


  1. David Ernst Hard Problems in Cryptocurrency.
  2. https://github.com/makerdao
  3. Mechanics that are temporarily in place in the system during the Single-Collateral phase are marked in this white paper

 

TrueBit – White Paper

Highlight to Annotate. Click to Learn.

 

A scalable verification solution for blockchains

Jason Teutsch TrueBit Establishment jt@truebit.io

Christian Reitwießner Ethereum Foundation chris@ethereum.org

November 16, 2017

 

 

Abstract

Bitcoin and Ethereum, whose miners arguably collectively comprise the most powerful computational resource in the history of mankind, offer no more power for processing and verifying transactions than a typical smart phone. The system described herein bypasses this bottle- neck and brings scalable computation to Ethereum. Our new system consists of a financial incentive layer atop a dispute resolution layer where the latter takes form of a versatile “verification game.” In addi- tion to secure outsourced computation, immediate applications include decentralized mining pools whose operator is an Ethereum smart con- tract, a cryptocurrency with scalable transaction throughput, and a trustless means for transferring currency between disjoint cryptocur- rency systems.

Contents

Securing computations with economics2

  1. How TrueBit works5
    1. System properties 7
    2. Assumptions 8
    3. Attacker model 9
  2. Dispute resolution layer10
    1. Bottleneck: The Verifier’s Dilemma 11
    2. Solution: The verification game 12
    3. Detailed protocol 13
    4. Runtime and security analysis 16

Incentive layer17

Defenses26

Implementation36

Applications37

Addendum40

Securing computations with economics

Every Nakamoto consensus-based cryptocurrency (e.g. Bitcoin or Ethereum) offers something that everyone in the world agrees on: a definitive, public ledger of financial transactions, or blockchain. This consensus technology en- ables basic Bitcoin transactions, which transfer currency from one party to

another, while Ethereum transactions perform financial and database oper- ations contingent on the evaluation of more complex computational scripts. Anonymous miners, who freely join and leave cryptocurrency networks, de- termine the validity of these transactions and thereby establish who owns which coins. Remarkably, this verification scheme requires no central au- thority to operate securely.

In practice, miners can successfully maintain blockchain integrity so long as the computational burden of verification remains minimal. Indeed, Bit- coin and Ethereum, whose miners arguably collectively comprise the most powerful computational resource in the history of mankind, offer no more power for processing and verifying transactions than a typical smart phone. One cannot simply increase the volume or complexity of transactions flow- ing into the blockchain without risking inclusion of invalid transactions due to the so-called Verifier’s Dilemma [50] (see Section3.1). In 2015, the Ver- ifier’s Dilemma manifested itself in the form of the July 4 Bitcoin fork [35] which temporarily derailed the entire network. The Ethereum community also witnessed a related exploit in its 2016 denial-of-service attacks [16].

A consensus computer [50] permits users to outsource computations to the Ethereum network and receive correct answers in exchange for payment so long as the effort required to verify solutions does not exceed the threshold induced by the Verifier’s Dilemma. Thus the trustless consensus computer offers a small but reliable kernel of semantic truth.

Our contribution. We present here a system, called TrueBit, which am- plifies the consensus computer’s capabilities. TrueBit enables trustless smart contracts, in theory, to securely perform any computation task. Moreover, TrueBit vastly reduces the number of redundant network node computations used in traditional Ethereum smart contracts. Presently every Ethereum miner has to independently replicate each smart contract action in its en- tirety, whereas TrueBit outsources most computation work to a handful of entities. In this way, TrueBit makes secure computing affordable.

Outsourced computation

Let us ponder for a moment why cryptocurrencies might offer an especially convenient framework for secure outsourced computation, the core appli- cation of TrueBit. In the traditional cloud models, users must trust that the machine hardware, software, and cloud administrator all perform as ex- pected. A wide range of things can go wrong, particularly when one wishes to tie the results of such computations to monetized entities such as smart

contracts. Proper economic incentives, the cornerstone of any cryptocur- rency, can deter many types of errors from occurring in ways that simple task repetition cannot. Furthermore, in contrast to a cloud whose configu- ration details may not be visible to users, any systemic network errors that might occur on a blockchain-based system like TrueBit would appear in plain sight to the entire community. Cryptocurrencies indeed provide an excellent starting point as they already achieve several desirable properties.

      1. As witnessed by the Ethereum consensus computer, Nakamoto con- sensus grants networks the ability to trustlessly perform small compu- tations correctly.
      2. Blockchain public ledgers provide perfect transparency and immutabil- ity. Ethereum smart contracts inherit these characteristics.
      3. Currency incentives, which in cryptocurrencies are inextricably tied to computational processes, can be used to recruit and reward partic- ipants.

In general, economic forces greatly reduce the scope of possible network errors. We can safely assume that errors which incur economic penalties are much less likely to occur than those which do not.

Practical impact

A market for work related to computationally-intensive smart contracts al- ready exists. The Golem Project, which crowdfunded 820,000 ETH on the morning of November 11, 2016 (∼$8.6 million on that date), already cites TrueBit as a verification mechanism for their forthcoming outsourced com- putation network [14]. TrueBit has the potential to support many practical new applications. Here are some examples.

  • Outsourced computation. TrueBit functions as a worldwide computa- tion market. Anyone in the world can post a computational task, and anyone else can receive a reward for completing it. The system’s incentive structure guarantees correctness of returned solutions.
  • Truly decentralized mining. Centralized mining pools pose security threats. For any Nakamoto consensus-based cryptocurrency, one can use TrueBit to build a practical, efficient, and trustless mining pool managed by an Ethereum smart contract (see also [51]).
  • Trustless cryptocurrency exchange. Currently users must find trusted exchange partners to transfer currency (or messages) between block- chains. TrueBit can facilitates secure transfer, for example, of doge- coins to the Ethereum blockchain and back (modulo some new opcodes for dogecoin) [8].
  • Scalable blockchain. By decoupling verification for miners into a sep- arate protocol, we can achieve high transaction throughput without facing a Verifier’s Dilemma.
  • Scalable “on-chain” storage. Swarm [28] is developing a platform for incentivized, peer-to-peer storage. TrueBit can make Swarm data ac- cessible to Ethereum smart contracts.

We discuss these ideas further in Section7.

Smart contracts

Smart contracts, introduced in 1994 by Szabo [61] and first realized in the Ethereum [11] cryptocurrency in 2015, are uncensorable programs that live on Ethereum’s blockchain and have their own executable code and internal states, including storage for variable values and ether currency balance. Pro- posed applications for smart contracts include outsourced computation and storage [13,28,50], prediction markets [2], decentralized venture funds [29], and Bitcoin mining pools [27,51]. Ethereum’s variable gasLimit restricts the number of computation steps that smart contracts can perform.

TrueBit itself is an Ethereum smart contract which allows users to call TrueBit contracts for trusted, computationally-intensive applications. Tradi- tional Ethereum smart contracts can call a TrueBit contract as a subroutine, thereby effectively bypassing Ethereum’s gasLimit. In other words, TrueBit increases the per block work that the Ethereum network can process cor- rectly.

How TrueBit works

TrueBit’s primary purpose is to realize correct, trustless computations de- spite miners’ limited computation bandwidth. Intuitively, we wish to reward participants who correctly perform computational tasks, but who decides whether these tasks were done correctly? In absence of a dispute, the party who performs a computational task on behalf of a TrueBit contract simply receives a reward. On the other hand, if a dispute does occur, we must

rely on the only trusted resource, the limited network of miners, to resolve it. Dispute resolution occurs as a “verification game” subroutine in TrueBit, and we will discuss its details in Section3.

Who initiates a dispute in the case of an incorrect computation, and how can we ensure that such a dispute will actually occur? One option is to assume that each party with some stake in the contract brings their own trusted (although not necessarily mutually trusted) verifier, and flags for a challenge if needed. This trusted verifier approach suffices for some applications, but in general parties may need to rely on the results of a previously executed TrueBit contract ex post facto even if they themselves did not exist at the time of its creation. Like Ethereum smart contracts, TrueBit contracts must have universal validity.

Since there exist no trusted parties on Ethereum’s network, by symmetry we must allow any party to be hired to solve any computational task, and similarly anyone should be able to challenge a Solver’s outcome of a com- putational task. The latter requirement ensures that TrueBit operates by unanimous consensus. Assuming that an honest and motivated Challenger exists who will challenge any wrong answer given by the Solver, the system’s incentives for the Solver are straightforward: reward correct solutions. But how can we make sure that such a motivated Challenger exists?

Idea 1. Offer rewards for checking computations.

While this strategy may encourage participation, it provides no incentive for correct verification. We have no a priori reason to trust that the Verifier will substantively inspect the computation task, and we gain no additional assurance by increasing her reward for checking.

Idea 2. Offer rewards for finding bugs.

While a bug bounty may incentivize a Verifier to correctly perform a check, it only does so when the Verifier has some reason to believe that a bug might actually exist. Unless a potential Verifier believes that she has some real chance to find a bug, in practice we cannot expect her to participate in the system. In summary, we need both an incentive to partake in verification and an incentive to perform checks correctly. This leads us to the following proposition.

Idea 3. Offer a bug bounty and provide expectation that bugs will exist.

TrueBit takes this third approach by occasionally forcing Solvers to sub- mit incorrect solutions. During “forced errors” TrueBit reverses the normal

system incentives: the Solver gets paid for submitting an incorrect solu- tion and penalized for submitting a correct one. We discuss the details of TrueBit’s forced error protocol in Section4.

Let us return to our dispute resolution subroutine. At the heart of TrueBit’s protocol lies an interactive “verification game” which decides whe- ther or not a contested computational task was performed correctly (see Section3). The verification game proceeds through a series of rounds, where each round recursively checks a smaller and smaller subset of the compu- tation. A trusted network, in our case the Ethereum platform [11], merely enforces the rules of the game and therefore does not bear the bulk of the verification burden. Anyone on the network can generate tasks, compute, or verify in exchange for rewards. TrueBit does not require “honest” or “altru- istic” nodes for correct operation but rather runs under the assumption that each node wishes to maximize its own profit. We shall discuss the details of TrueBit’s incentives for participation in Section4.

System properties

Any outsourced computation system should be fair in the sense that parties who perform computational tasks indeed receive compensation for their work and reliable in the sense that paid computations are performed correctly. In addition to these properties, TrueBit also ensures accessibility through easy joining or exiting of the verification ecosystem. Any anonymous party can play the role of Task Giver, Solver, or Verifier, where a Verifier is a party who checks solutions and becomes a Challenger whenever she reports an error. In particular, TrueBit does not trust or rely on the reputation of its participants. Everyone who puts a deposit into the system has a fair chance to be hired for a given computational task.

TrueBit offers several novel advantages over traditional cloud computing

and verifiable computable models. Verifiable computing ensures a correct answer for an outsourced computation by forcing the cloud to provide a short proof which witnesses the correctness of its computation. The idea is that this “short proof” should be much easier to check than performing the computation oneself. Researchers have achieved much progress on this method in recent years, however the cryptographic setup costs and computa- tional overhead for the cloud in state-of-the-art systems make these methods unsuitable for most real-world applications. Moreover, many of the proof- based systems to-date, including Zaatar, Pinocchio, Ginger, and TinyRAM, require one to run thousands of instances of a single function before breaking even on the overall verification time for a 128 × 128 matrix multiplication

puzzle [65]. The new cryptocurrency Zcash [34] successfully makes use of verifiable computing machinery, albeit only for a very restricted class of computation tasks. Below we contrast TrueBit with verifiable computing and traditional cloud computing.

      1. Incentives. Unlike traditional cloud computing, where the user simply trusts the cloud to provide correct answers, TrueBit provides financial incentives to ensure correctness.
      2. Transparency. The entire inner workings of TrueBit’s interpreter sit on the blockchain and are open for inspection (see Section6). Further- more, the user community can democratically update the interpreter as needed.
      3. Efficiency. Solvers in TrueBit have low computational overhead and minimal initial setup costs. The verification game (Section3) does introduce some extra work, but in practice, due to high penalties for wrong answers and bogus challenges, we expect participants to appeal to the verification game only rarely, if at all.
      4. Simplicity. TrueBit’s operation is relatively straightforward. Unlike traditional verifiable computing, TrueBit avoids deep probabilistically checkable proofs (PCPs), succinct non-interactive arguments of knowl- edge (SNARKs) [37], and exotic cryptographic assumptions (e.g. those used in zkSNARKs [38]). The standard cryptography used in TrueBit, namely hash functions and digital signatures, sits in the underlying blockchain network and does not surface in the TrueBit protocol itself.
      5. Adaptability. TrueBit runs on top of Ethereum’s current protocol with- out impacting functionality.
      6. Keyless entry. Participants do not need to manage cryptographic keys beyond those used in Ethereum. TrueBit establishes identities through financial deposits alone, and therefore the system avoids risks from cryptographic trapdoors.

Assumptions

Traditional distributed systems models focus on tolerance against arbitrary, Byzantine errors. In Ethereum and other Nakamoto consensus-based cryp- tocurrencies, however, we have no reason to expect arbitrary errors—nodes generally deviate from protocol due to financial incentives. Our system model makes two basic assumptions about the underlying network.

  1. There exists a trusted network (i.e. Ethereum) that correctly performs very small computation tasks.
  2. Participants are rational in the sense that they act to maximize their individual profits. In particular, we expect that CPUs will show up to do computational tasks if and only if they expect fair compensation for their work.

The consensus computer [50] shows how one can use the incentives in Ether- eum to establish assumption (i) above both in theory and in practice. The task of our new system will be to amplify this small amount of correct com- putation in order to handle larger tasks using assumption (ii). Even though solo Ethereum miners rarely earn mining rewards, expected long-term re- turns suffice to merit their participation in the system (via mining pools). As we shall discuss in Section3.1, anonymous parties may not necessarily perform correct computations when economic incentives, including conser- vation of CPU cycles, pressure them to do otherwise. This observation justifies assumption (ii). We emphasize that we do not find the assumption that there exists a single honest (and non-lazy) participant [46] to be real- istic. A person who loyally contributes CPU cycles without incentive to do so most likely does not exist in a large, general purpose, public system.

This paper takes a somewhat simplified view of Nakamoto consensus. In some cryptocurrencies, such as Bitcoin, miners can selectively censor a class of transactions by deciding not to include its members in their blocks. In the case of TrueBit, censorship of challenges (see Section4) or transactions in the verification game (Section3.2) could result in catastrophic, bogus solutions being accepted on the blockchain. In Ethereum, however, censor- ing individual transactions is difficult as miners cannot easily see where an Ethereum transaction might call without executing it. Indeed Ethereum users can obfuscate the function of their transactions, making it computa- tionally costly for miners to hand-pick transactions for their blocks. Hence we shall assume that censorship of individual transactions does not happen. In Section5.1, we analyze an attack on TrueBit in which miners withhold entire blocks and show that it is not profitable.

Attacker model

TrueBit’s security relies on two basic assumptions about the behavior and capabilities of potential attackers.

  1. Attackers cannot subvert the underlying consensus computer. The

Judges in the verification game (Section3) always adjudicate correctly, as do the Referees in the incentive layer (Section4).

  1. Attackers are rational. We assume that attackers have limited financial resources and will not deviate from protocol unless it is profitable for them to do so.

While an adversary with sufficient computational power can derail a Naka- moto consensus-based cryptocurrency [56], assumption (i) above neatly sweeps this issue outside our domain. TrueBit itself does not utilize proof- of-work. Although we have chosen to implement TrueBit in the Ethereum platform for convenience, we could easily replace the “Judges” from (i) with any other universally trusted computing mechanism without affecting the system’s functionality.

Due to anonymity of all participants, Sybil attacks, in which a single party deviantly assumes multiple identities on the network, pose special threats to TrueBit (see Section5.1and Section5.2). We will assume that an attacker may create as many identities as she likes, including simultaneously playing the roles of Task Giver, Solver, and Verifier, appearing as two dis- tinct Verifiers, or managing multiple identities via pooled financial resources (see Section5.3). Using assumption (ii) above, we shall argue that TrueBit resists such attacks in Section5.

The “forced errors” described in the beginning of this section (Section2) pose a special challenge to TrueBit due to opportunistic attacks. As the prize for discovering forced errors is necessarily enormous, attackers may seek ways to obtain these large prizes prizes without performing the intended verification tasks (Section5.4). For this reason, the occurrence of forced errors must appear truly random to all prospective Verifiers. Moreover, Solvers must be incentivized from prematurely sharing the fact that their error was forced or claiming that an error was forced when it really wasn’t. In general, we want consistent, not sporadic, verification. At least one Verifier should be present for every task. The attraction of forced errors represents the “Achilles’ heel” of TrueBit, and we devote the incentive structure in Section4to careful defending of this potential attack surface.

Dispute resolution layer

In this section we present and analyze TrueBit’s dispute resolution mecha- nism. TrueBit relies on network “Judges” with limited computational power

who adjudicate all disputes, and we show how such Judges can correctly resolve disputes over complex computations.

Bottleneck: The Verifier’s Dilemma

Ethereum’s incentive structure severely restricts the amount of computation work that smart contracts can accurately enforce. Users interact with smart contracts by broadcasting transactions to the network. Such transactions contain program scripts which require computational work to verify their validity prior to miners’ acceptance on the blockchain. Each miner must not only locally verify but also remember the state of every single smart contract. On the surface, this redundant work limits smart contracts’ computational scope due to inefficiency, but the true bottleneck lies deeper at the incentive level.

Let us recall miners’ incentives for participating in Ethereum. In Naka- moto consensus-based cryptocurrencies [56], new transactions enter the block- chain through a process called mining. Participating miners maintain the integrity of the blockchain data by racing to solve computation-intensive, proof-of-work puzzles in exchange for rewards. The first miner who success- fully broadcasts a solution to the current proof-of-work puzzle proves that she has spent the necessary computation power to merit appending her new set of transactions to the blockchain, and this step awards the miner a set of newly minted coins. Then the race begins again on top of this new block.

When an Ethereum user broadcasts a transaction to the network, the miner who appends her valid transaction to the blockchain receives a trans- action fee for the service. On the other hand, Ethereum’s mining protocol dictates that other miners should verify this transaction gratis, for the “com- mon good” so to speak. More specifically, the mining protocol dictates that the longest chain consisting of valid transactions is the correct one, and miners should mine on top of it. If the time to verify new blocks is negli- gible, miners may indeed agree to check them, but if the verification work becomes substantial, then miners risk falling behind in the mining race by following protocol and actually verifying. In order to save time, rational miners might simply skip the verification step, leaving the blockchain open to invalid transactions. On the other hand, by skipping verification, a miner might also waste her CPU cycles by mining on top of a stale chain which the rest of the community deems invalid. Either way, the miner risks forfeiting rewards. A rational miner doesn’t know what to do!

Any substantial verification burden placed on miners thus results in a

Verifier’s Dilemma [50]. Because of the Verifier’s Dilemma, smart contracts

whose verifications require non-trivial computational efforts will fail to ex- ecute correctly as rational miners, who may choose to deviate from the suggested protocol, stop donating their limited resources to thankless ver- ification tasks. In short, Ethereum lacks a scalable verification mechanism and remains vulnerable to serious attacks [50] when miners have to verify more than a tiny amount.

TrueBit contracts securely execute computationally-intensive tasks for use in Ethereum smart contracts. The system’s core consists of a smart contract interface where a user can request a solution to a computational task or puzzle, and anyone can solve it in exchange for a reward. Our inter- active verification game, described below, empowers users to both outsource arbitrary computations in a decentralized manner and receive back correct solutions. To achieve this goal, we will not only reduce the redundancy of verification work but also refine the incentive structure responsible for the Verifier’s Dilemma.

Solution: The verification game

The goal of TrueBit’s interactive verification game is to resolve a given dis- pute between a Solver, who has provided a solution to some computational task, and a Challenger, who disagrees with the solution and therefore calls the verification game to be played. The outer layer of TrueBit (Section4) uses the verification game as a subroutine. The roles in the verification game include a Solver, who offers a solution to a given task, and a Chal- lenger who disagrees with the Solver’s solution. The final party, the Judges, always perform computations correctly but possess extremely limited com- putational bandwidth. The Judges in TrueBit are the entire community of Ethereum miners who reach verdicts through Nakamoto consensus.

The verification game proceeds in a series of rounds, each of which nar- rows down the portion of the computation in dispute. In the first round, the Challenger forces the Solver to commit to a pair of computation steps delimiting some time interval. In the next round, the Challenger iteratively challenges a subset of the Solver’s computation over this time interval, and next she challengers over a subset of that subset, etc. until in the final round the final challenge is sufficiently trivial that the Judges can make a final rul- ing on whether the challenge was justified. The Judges also enforce that the Solver and Challenger follow the rules of the game. At the end of the verification game, either the cheating Solver will be discovered and punished in the outer layer of TrueBit (Section4), or the Challenger will pay for the resources consumed by the false alarm.

Example (matrix multiplication [46]). We give an illustrative example of a verification game. Suppose that the given task is to compute the product of two matrices A and B. The Solver claims that A × B = C. In the first round, the Challenger must claim an error in one of the entries in C, and outputs coordinates i, j such that

n

Σ

ci,j ƒ= ai,k · bk,j

k=1

with corresponding evidence consisting of partial sums d0, d1, . . . , dn where

m

Σ

dm = ai,k · bk,j .

k=1

The Judges only verify that i, j are coordinates and that dn ƒ= ci,j and that d0 = 0. The Challenger loses if this fails to hold. In the second round, the Solver tries to defend himself by providing a k such that dk ƒ= dk−1 +ai,k ·bk,j . If the Judges can verify this claim then the Solver wins the game, and otherwise the Challenger wins.

Detailed protocol

We can use a variation of the Example above to check arbitrary computa- tions by creating a binary search to pinpoint the erroneous (or not erro- neous) step in a computation. The idea for the following verification game is essentially due to Canetti, Riva, and Rothblum [39,40] and was later in- dependently discovered by the authors of this paper [46,58]. Canetti, Riva, and Rothblum did not consider this game in the context of blockchains but rather a simpler scenario in which a user outsources a computation to k dif- ferent clouds under the assumption that at least 1 out of k of the clouds will perform the computation correctly. The assumption of at least one trusted Solver/Challenger cloud is too strong for our purposes since in a purely ra- tional network, such a Solver/Challenger may not exist. Section4describes our workaround for the smart contract setting.

For purposes of reducing the number of rounds of interaction and space demands on the blockchain, our protocol roughly follows [39,40] which com- bines the (parametrized) binary search optimization from [46, Proposition 9] with the probabilistic Merkle tree construction from [58]. Recall that a Merkle tree is a binary tree in which each node is the hash of the concate- nation of its children nodes. In general, the leaves of a Merkle tree will collectively contain some data of interest, and the root is a single hash value

which acts as a certificate commitment for the leaf values in the following sense. If one knows only the root of a Merkle tree and wants to confirm that some data x sits at one of the leaves, the holder of the original data can provide a path from the root to the leaf containing x together with the children of each node traversed in the Merkle tree. Such a path is difficult to fake because one needs to know the child preimages for each hash in the path, so with high likelihood the data holder will supply a correct path if and only if x actually sits at the designated leaf.

For clarity, we will present the verification game in the familiar and stan- dard language of Turing machines, but in actuality TrueBit uses the Google Lanai architecture to simulate steps in the computation (see Section6). The verification game consists of three parties:

  • a Solver who has allegedly performed some computational task and claims some solution,
  • a Challenger who disputes the correctness of the Solver’s solution, and
  • Judges with bounded computational power, who will rule on whether the output is correct.

TrueBit fills the roles of Solver and Challenger according to the incentive layer discussed in Section4. The Judges are the set of miners in Ethereum who collectively record their decisions on the blockchain. The incentive layer (Section4) determines the financial consequences of the Judges’ verdict.

To start, the Solver and Challenger each privately compile a tableau of Turing configurations mapping each time step of the task to its full internal state. Assume that the task runs in time t steps and space s, where s bits suffice to include the complete state of the Turing machine at any given time, including tape contents, head position, and machine state. The game protocol includes a fixed, integer parameter c > 1 which says how many configurations the Solver broadcasts to the blockchain in each round of the verification game. More configurations per round mean fewer total rounds needed, however it also means that the total number of configurations sent to the blockchain increases. In particular, we reach diminishing returns as the data sent per round reaches Ethereum’s per block capacity.

The verification game protocol proceeds as follows.

Main loop. The following steps are done iteratively to pinpoint the source of disagreement. The protocol determines timeout periods within which Solvers and Challengers must respond. Failure to respond within time bounds results in immediate loss for the non-responding party.

      1. The Solver selects c configurations equally spaced apart in time across the current range of the dispute. In the initial iteration, for example, the Solver selects configurations across the entire tableau at the following time steps:

t 2t

, ,

c c

3t ct

, . . . , .

c c

He then produces c Merkle trees, each with s leaves, where the leaves constitute the complete machine state at each of these times, and broadcasts each of the roots of these Merkle trees to the blockchain.

      1. The Challenger responds with a number i c, indicating the first time step in this list that differs from her own, and broadcasts this number i to the blockchain.
      2. The Judges check that the Solver indeed posted c Merkle roots to the blockchain and that the Challenger’s value i satisfies 1 ≤ i c. If either of these checks fails, the Solver, or respectively the Challenger, loses immediately.
      3. The next iteration of the game continues similarly, but restricted to configurations between the i − 1-st and i-th indexed configu- rations. Here we interpret a 0 as the computation’s initial con- figuration.

Final stage. After log t/ log c rounds, the loop above converges to the first, disputed computational step, and the Judges explicitly check this step of the computation as follows. Suppose that the first disagreement occurs at time e. The Solver provides paths from the Merkle root for time e to its leaves containing:

  • the location of the machine head at time e,
  • the content of the tape square under the machine head, the tape contents of its left and right neighbors, and
  • the machine state at time e.

The Solver also provides the analogous paths for time e−1. The Solver loses the dispute if the Judges find that any of these paths fail to be valid. Finally, using this information and the original task code (which existed on the blockchain prior to the verification game), the Judges check the computational step at time e and rule whether the challenge was justified.

Runtime and security analysis

We now show that the work done by the Judges is small compared to the work required to perform the task, and that the number of interaction rounds is modest. Note that the majority of effort from the Judges simply involves recording data onto the blockchain. The Judges’ work is split over many transactions and therefore avoids any chance of a Verifier’s Dilemma.

We fix σ as the size of each hash in the Merkle trees, and p as the space required to store a single machine state for the computation. For computational tasks that run in time t and space s with c configuration roots posted per round, the protocol requires log t/ log c rounds of interaction on the main loop, writes + log c bits to the blockchain during each iteration of the main loop, and writes 2 · ((2σ + 1) log s + 5 + p) bits in the final round (assuming that tape squares can contain either 0, 1, or be blank, so we need 3 · log 3/ log 2 ≈ 5 bits to describe 3 of them). Summing these parts, the total time required for community-wide verification from the Judges is then

Σ Σ· ·

O log t (+ log c) + 2 ((2σ + 1) log s + 5 + p) log c

where the hidden constant depends on the implementation of this protocol on the Judges’ local machines. This estimate does not include the space and time required to store the instructions for the initial computation task. We assume that this data already exists on the blockchain before the verification game begins.

In comparison with other proposed blockchain-based computation sys- tems, or even PCP-based verifiable computing systems [65], the work im- posed on the Judges by the verification game is small [12], as is the compu- tational overhead required to join the system [63]. Moreover, the economic costs to the Task Giver, who pays only a Solver and a Challenger and not all Judges to process the full task, is modest, as are the computational over- heads for the Solver and Challenger.

Note that larger values for σ reduce the chances of hash collisions. Hash collisions might allow a dishonest Solver to substitute configurations in his computation tableau with fake ones sharing the same Merkle roots. Two consecutive configurations in the tableau whose Merkle roots happen to agree with the Merkle roots of two consecutive configurations that the dis- honest Solver wishes to apply as substitute suffice to disrupt the integrity of the verification game protocol. Indeed, the Challenger will agree with both Merkle roots, and the Judges will confirm that the transition between them is valid and therefore incorrectly reject the challenge. For a fixed size

σ, the chance that such collisions occurs by accident even becomes likely for sufficiently enormous tasks.

One could entirely eliminate the security risk discussed in the previous paragraph by posting complete machine states on the blockchain rather than just Merkle roots, but this makes the protocol more expensive. Alternatively, one could either increase the parameter σ or the number of states checked by the Judges in the final step. In this sense, the choice of σ bounds the maximum complexity of secure computations in TrueBit. We will also see in Section4.1that the capital held by the TrueBit contract as a “jackpot” for finding a forced error poses an additional constraint on computational ca- pacity, but this value can scale as well. Finally, the effectiveness of TrueBit’s verification game may degrade for extremely complex tasks due to the com- putational limitations of the Challenger. If the expected execution time of a verification game exceeds a human life expectancy, for example, then potential Challengers may lack motivation to participate despite financial incentives. Note that even when a task itself is feasible, its correspond- ing verification game may not be as the verification game carries significant overhead. We anticipate that future versions of TrueBit may optimize the verification game for certain kinds of tasks so as to reduce this discrepancy (see Section7.4).

Remark. The verification game provides a significant privacy-protecting ef- fect in that only disputed parts of a given computation must touch the public eye [44].

Incentive layer

We now discuss the financial incentives which encourage anonymous par- ties to both contribute CPU cycles and perform requested computational tasks correctly. We use the dispute resolution layer from Section3as a subroutine and connect its role to the present construction. A Verifier is a potential Challenger, that is, someone who checks submitted solutions and calls for a challenge (see Section3) if and when she detects an error. By assumption (ii) in Section2.2, Verifiers must receive adequate payment for verification tasks. As argued in the beginning of Section2, however, Veri- fiers can only receive rewards when they actually find bugs. It follows that the reward for discovering errors must encompass amortized payment for all verification tasks checked, including those in which no bugs were found.

Any Ethereum user who offers a reward through TrueBit for performing a computational task is called a Task Giver. A party which offers a solu-

tion for performing these tasks in exchange for a reward is called a Solver, and, as indicated in the previous paragraph, Verifiers check that Solvers’ solutions are correct. Solvers and Verifiers will play the roles of “Solvers” and “Challengers” respectively from the dispute resolution layer (Section3) whenever a Verifier initiates a verification game. Referees enforce the rules of the TrueBit protocol in the incentive layer. While Ethereum miners enforce protocol rules both through the roles of “Judges” in Section3and the role of “Referees” in the present section, we use distinct labels for them because the layers and functions differ for these roles. The primary role of Judges is to interactively resolve a dispute, whereas Referees primarily enforce that Solvers and Verifiers timely submit appropriate data in the incentive layer.

In a nutshell. The main steps of a TrueBit contract are as follows. Items in quotes will be throughly explained in due course.

1.A Task Giver announces a task and offers a reward for its solution.

    1. A Solver is elected by lottery, and he prepares both a “correct” and an “incorrect” solution to the task.
      1. If a “forced error” is in effect, the Solver reveals the incorrect solution on the blockchain.
      2. Otherwise the Solver reveals the correct one.
    2. Verifiers check the Solver’s solution. They win a large “jackpot” pay- out if both:

(a)they correctly identify the solution as erroneous, and (b)a forced error was in effect.

    1. If no Verifier signals an error, then the system accepts the solution. Otherwise acceptance depends on the outcome of a verification game.

In the rest of this section, we discuss the security measures in TrueBit which ensure that participants comply with the skeletal procedure outlined above. A more complete description of the protocol appears in Section4.6.

Jackpots

TrueBit periodically imposes forced errors in which a Solver must offer a wrong solution to a task (see Section2). This ensures that Verifiers who diligently search for errors will eventually find them. Accordingly, Verifiers

who correctly report forced errors receive substantial jackpot payouts. By design, Verifiers cannot predict when forced errors will occur and therefore have incentive to check all tasks carefully. Forced errors occur only rarely, and we expect Solvers to make only few, if any, other errors. Rewards for identifying unforced errors may be modest compared to jackpot payouts, and we consider any such payments of these type incidental, rather than fundamental, to TrueBit’s secure incentive structure.

The jackpot payout effectively bounds the complexity of computation tasks that TrueBit can perform securely. The Verifier must, on average, re- ceive a payment which fairly compensates her for the task at hand, which means that the jackpot payout should at least consist of fair compensation for the current task times the forced error rate. In this way, the jackpot cap bounds the Verifier’s expected per task compensation which, by assump- tion (ii) in Section2.2, restricts Verifiers’ available CPU cycles per task.

We fix a rate for forced errors among tasks. This fraction should not be so low so as to discourage Verifier participation through excessively infrequent rewards, but neither should it be so high so as to run the risk of Referees bias (see Section5.1). We set forced errors to occur, on average, once every thousand tasks.

Taxes

Given that a jackpot repository must exist, we now describe the mechanism for funding it. We assume that a generous philanthropist deposits some initial funds into the repository, but thereafter TrueBit will be self-sustaining through taxes. Any Task Giver that calls a TrueBit contract must pay not only the cost of computational work done by the Solver but also for the work done by the Verifier(s) (excluding unforced errors and bogus challenges), as well as the work done by Referees and Judges. We refer to the latter two costs as the verification tax. We ensure that the jackpot repository never disappears entirely by placing a cap on the jackpot size. To this end, we set the maximum jackpot payout for a forced error to be one third of the total repository size.

While a single, attentive Verifier suffices to ensure correctness and ach- ieves ideal tax efficiency, in practice the verification tax requires a substantial cushion. We estimate the necessary verification tax to be 500% – 5000% of the cost of performing the given task. As we shall see in Section5.2, there is a quantitative tradeoff between tax efficiency, Solver deposits, and security of computations, so Solvers could potentially absorb some of this tax burden by contributing higher deposits. Our tax rate estimate incorporates Veri-

fiers’ incentives for participation and the fact that both the Task Giver and the Solver for a given task each have incentive to perform verification. Par- ticipation from these parties may necessitate higher taxes because the total jackpot payoff decreases exponentially as the number of challenges increases (see Section5.3). Expected jackpot payoffs must be sufficiently high to con- sistently attract at least one Verifier per task. The required tax amounts may also depend on peculiarities of human behavior. Indeed, we may have to pay Verifiers more than Solvers per CPU cycle because Verifier rewards have the human-undesirable property of being sporadic whereas Solvers al- ways receive immediate rewards. Thus the optimal tax rate must, at least in part, be determined experimentally.

Deposits

TrueBit requires deposits from Solvers and Verifiers in order to thwart Sybil attacks (see Section5.1) and incentivize correct computations. We set these deposits to be more than the expected jackpot payout for a given task plus the cost of playing a verification game. In particular, the deposits must be large enough to:

      1. pay for the (expensive) cost of a verification game, including all re- wards and penalties for Solver and Challengers and work done by Judges,
      2. discourage Solvers and Verifiers from sacrificing deposits in order to obtain jackpots without performing verification (see Section5.1and Section5.3),
      3. discourage Task Givers who collude with Solvers in effort to get bogus solutions onto the blockchain (see Section5.2),
      4. refund taxes to the Task Giver in case the Solver causes an unforced error,
      5. deter Solvers from randomly guessing solutions to obtain task rewards instead actually performing computations (as might be profitable for binary decision tasks with very high task rewards), and
      6. deter external, temporal pathologies.

Note that the currency used to pay Solvers and Verifiers need not be the same as the currency used to pay Judges and Referees, but for simplicity of presentation, we fix ether (ETH) as the unique underlying currency.

As an example of the second type, consider a situation where the Solver deposit is small (say 10 ETH) but the expected jackpot payout per task is high (say 1000 ETH). An individual playing both the role of the Solver and Verifier could offer a bogus solution and then challenge his own answer, hypothetically netting, on average, 1000 − 10 = 990 ETH without providing any useful service. Such an action would degrade other Verifiers’ incentive to participate.

As an example of the last case, if the outcome of a time-sensitive TrueBit contract controlled a 1000 ETH payout but only required a 10 ETH Solver deposit, then the party who stands to lose 1000 ETH from the TrueBit contract could attempt to cause a delay by posing as a Solver to the TrueBit contract and giving a bogus answer. We leave it to the Task Giver to determine appropriate minimum deposit values for such specific situations, as such contextual determinations lie outside of the scope of TrueBit itself.

Generating forced errors

In order to motivate verification of all tasks and to guard the jackpot repos- itory against swindle, forced errors must appear unpredictably. TrueBit uses strings of random bits to determine whether or not a forced error occurs for a given task. The system derives its unpredictability via the following properties.

      1. The Task Giver does not know the random bits at the time that she announces a task.
      2. The Solver does not know the random bits until after he has committed his solution.
      3. Verifiers do not know the random bits until after they decide whether to challenge.

The first property makes it difficult for a Task Giver to create a task designed to swindle the jackpot repository, and the second discourages Solvers from being lazy and only volunteering to solve tasks which have forced errors. In order to satisfy these three properties, TrueBit combines random bits from the following two sources:

(a)“private” random bits from the Solver, and

(b)the hash of the block mined immediately after the block containing the Solver’s solution.

By hash of the block, or block hash, we more precisely mean the hash of the block’s header, a roughly 200-byte piece of data that contains the times- tamp, nonce, previous block hash, and Merkle root hash of the transactions occurring in the block [32]. Source (b) achieves properties 1. and 2. above, and source (a) achieves property 3.

TrueBit generates Solvers’ “private” random bits using a method reminis- cent of RANDAO’s random generator mechanism [25]. Before participating in a TrueBit contract, a Solver must privately generate a string of random bits r and publish its hash on the blockchain. This action commits the Solver to using r in the protocol without revealing the value r to others.

The Solver establishes property 2. above by committing both a hash of a “correct” solution and a hash of an “incorrect” solution prior to the broadcast of the block hash from item (b). At the time that this block hash is revealed, only the Solver, who has access to the value r, knows whether or not a forced error is in effect. He publicly designates one of his two hashed solutions for evaluation, however potential Verifiers do not know a prioiri whether the Solver intended his solution to be “correct” or “incorrect.” Only after the timeout for challenges has elapsed do the Verifiers witness the Solver’s private random bits in the clear and learn whether a forced error was in effect. In case the Solver’s solution is challenged, the Solver must reveal r as well as his second “correct” solution in order to prove that the error was forced and avoid a penalty (see Section5.4). Thus, in the case of a forced error, the Task Giver still obtains a correct solution. In case of an unforced error, i.e. when the Solver evidently fails to provide a correct solution, the Task Giver receives a full refund of her task reward and taxes (see Section4.3and Section4.6).

Although in theory one could securely run this part of the protocol with- out hashing the Solver’s solutions, hashing makes it easier for the Solver to provide a convincing “incorrect” solution which appears, upon casual in- spection, indistinguishable from a “correct” one. The Solver can effectively use any randomly selected “incorrect” solution during a forced error because he never has to reveal its preimage.

Solver and Verifier election

The system effectively chooses Solvers by lottery. When a task is announced, Solvers broadcast their interest in solving it to the Referees in the form of Ethereum transactions. Referees, or more specifically miners, choose one of these transactions to include the next block, thereby electing the Solver for that given task. In case the miner who chooses the transactions for the

current block happens to also hold a lottery ticket, he may bias his chances of winning the lottery. This bias does not affect the security of TrueBit, however, since the Solver must still provide a correct solution.

In contrast to the Solver selection, TrueBit does not impose a limit on the number of Verifiers, and we describe in Section5.3how multiple Verifiers who participate in a task must split their rewards after a successful challenge. Due to the computational work involved in verification, rational Verifiers will only verify tasks in case they expect to receive adequate compensation for their work (assumption (ii), Section2.2). Thus the number of Verifiers verifying each task will remain low due to the balance between computation costs and incentives (see Section5.2and Section5.3).

Protocol overview

In this section we present an overview of the TrueBit protocol. We will discuss how our chosen parameters enable system security in Section5. Throughout the description below, Solvers and Verifiers who deviate from protocol by broadcasting malformed data or failing to respond within time- out bounds forfeit their security deposits to the jackpot repository.

Preprocessing steps. The following must be done prior to commencing

TrueBit operation:

1.A substantial jackpot repository must be established for the TrueBit contract prior to commencement of any interactions (See Section4.1for details).

  1. Solvers and Verifiers that wish to participate must commit de- posits to the TrueBit smart contract (See Section4.3). The de- posits may be placed at any time during the protocol so long as the underlying contract has sufficient time to confirm receipt of the funds.
  2. Solvers must also generate private random bits and commit their respective hashes to the blockchain. We denote the private ran- dom bits of the unique Solver selected to perform the task below by r (see Section4.4for more details).
  3. A universal tax rate T must be established (see Section4.2).

Main algorithm. The protocol steps are as follows.

1.A Task Giver provides the following: (a)a computational task,

(b)the timeOut for accepting bids, performing the computation, and waiting for a challenge. In the protocol below, we do not distinguish between these various timeouts with distinct no- tation, and we colloquially use “timeOut” to refer to both events and lengths of time. In all cases, timeOut must be long enough to avoid microforks during which Referees tem- porarily disagree on current parameters.

(c)a reward for a correct output, which must be at least the cash equivalent of the task difficulty d based on timeOut (see Section5.5), plus a total tax of T · d. The reward is held in escrow by the TrueBit contract while the taxes are immediately deposited into the jackpot repository.

(d)the minimum deposit, minDeposit, needed to participate as a Solver or Verifier (see Section4.3, Section5.1, Section5.2, and Section5.3).

  1. Solvers who have the requisite minDeposit and random bits can bid to take on the task until the bidding timeOut. At most one Solver is selected (Section4.5), and if no Solver takes on the task in the allotted time, the task request is canceled and the Task Giver receives a full refund.
  2. The Solver privately computes task. In case of a timeOut, Solver forfeits his deposit to the jackpot repository and the protocol terminates.
    1. Solver commits two distinct hashes to the blockchain, thereby committing both a “correct” and an “incorrect” solution.
    2. The hash of the next mined block is revealed, and then Solver

knows whether or not there is a forced error (see Section4.4).

    1. Solver designates one of the two hashes as the hash of his

solution.

  1. Verifiers who have posted minDeposit can challenge (the hash of) solution until timeOut. Prior to timeOut, the Verifier must broadcast the hash of an even integer to the blockchain in order to commit to a challenge. Hashing an odd number in case of no challenge is optional and may be used to camouflage real chal- lenges from other Verifiers (see Section5.3). After timeOut, the Verifier broadcasts to the blockchain this hashed number in the clear to reveal her action.
    1. If no Verifier challenges solution, then
      1. Solver reveals r to prove that there was no forced error (i.e. the criteria in Step 4(b)i below fails),
      2. Solver reveals solution in the clear on the blockchain,
      3. Solver receives the task reward, and then iv.the protocol terminates.
    2. Otherwise, some Verifier challenges solution.
      1. Solver reveals his private random string r on the block- chain, and Referees check it against his commitment from preprocessing step 3. If the hash of the concatenation of r and the block hash following the solution announce- ment from 3(b) is small (as determined by the forced er- ror rate, see Section4.1), then a forced error is in effect (see Section4.4).
      2. If the value r reveals that Solver’s error was forced, then Solver must reveal his secondary solution in the clear (see Section5.4).
        1. If no Verifer challenges Solver’s secondary solution so- lution before timeOut, then Verifier wins a fraction the maximum jackpot amount J , scaled for task dif- ficulty. In case of challenges from multiple Verifiers, the jackpot is split among them. In more detail, if there are k distinct challenges, then each participating Verifier receives J/2k−1 times the ratio of the task’s difficulty to the system’s maximum task difficulty (See Sections4.1,5.3, and5.4for further discussion).

B.Otherwise the Solver must play the verification game with the challenging Verifier(s). Verifier penalties, Ver- ifier rewards, Solver penalties, and refunds to the Task Giver here are the same as described in the next step. In case the Solver successfully defends himself against all challenges, however, then the jackpot payouts pro- ceed as in Step 4(b)i above.

      1. Otherwise the error was not forced. Solver reveals solution in the clear, and then Solver and Verifier must play a verification game (Section3.2). In case of chal- lenges from multiple Verifiers, the steps below are re- peated until either some Verifier wins a challenge or Solver defeats all Verifier challenges. The Solver collects reward only once he wins all of the challenges, and if this does

not happen, then the Task Giver receives a refund on his reward, and tax payments are reimbursed through Solver’s deposit as described below.

        1. If Solver wins, then Verifier forfeits half of her deposit to the jackpot repository and the other half to the Solver, where the latter suffices to compensate the Solver for playing the verification game (see Section4.3).

B.Otherwise Verifier wins. Solver pays at most half of his deposit to the Verifier(s), according to the distribution scheme in Step 4(b)ii above, pays back the Task Giver’s taxes out of this amount, and forfeits the remaining funds to the jackpot repository (see Section5.1and Section5.2).

End of protocol.

Note that Solver does not receive reward when a forced error occurs in Step 4(b)ii. In such situations, we expect the Solver to challenge his own “mistake” and receive a jackpot payout much greater than reward. TrueBit makes this payout automatic since the Solver would lose the verification game anyway if he and Verifier were to play it out.

Sanity check

The table on the next page recaps the parameters used in the TrueBit pro- tocol and hints at how to estimate their exact values. Each parameter is either a fixed constant, an input from a participant or something already on the blockchain, or can be computed from the elements which appear above it in the table. By inspecting the list below from top to bottom, one can confirm that all notions are well-defined.

Defenses

We now analyze the security of the incentive layer from Section4. TrueBit’s security relies on the presence of at least one Verifier to check each task performed. In this section we show that available incentives suffice to guar- antee the existence of such Verifier(s) according to the network assumptions in Section2.2and the attacker model from Section2.3. TrueBit defends against shortcuts which could divert verification incentives to non-verifying parties. In particular, we show that TrueBit resists Sybil attacks, collusion

parameter dependency
dispute layer parameters

p, c, and σ (Section3) tax rate

(Section4.2, Section5.2) forced error rate

(Section4.1)

maximum jackpot payout (Section4.2)

cash equivalent of CPU cycles (Section5.5)

maximum task difficulty

(Section4.1)

  • fixed constants.
  • fixed constant (500% – 5000%).
  • fixed constant (1/1000).
  • 1/3 of current jackpot repository.
  • based on external markets.
  • maximum jackpot payout divided by cash equivalent of a CPU cycle.
task

timeouts (Section4.6) task difficulty

Solver reward (Section5.5)

expected jackpot payout (Section4.1, Section5.2,

Section5.4)

Solver & Verifier deposits (Section4.3, Section5.1,

Section5.2, Section5.3)

Parameters in this box are chosen by the Task Giver with minimums as de- scribed below:

  • long enough to avoid microforks.
  • determined by timeouts.
  • no less than the cash equivalent of the task difficulty.
  • cash equivalent of task difficulty, and number of active Verifier deposits.
  • more than the cost of verification game plus the expected jackpot pay- out. Also depends on tax rate.
actual number of Verifiers (Section4.5)

jackpot payout for challenging a forced error (Section5.3)

Payout for detecting an unforced error (Section5.1, Section5.2)

  • as many join (incentives limit over- participation).
  • based on maximum jackpot payout, actual number of verifiers, and ratio of task difficulty to maximum task diffi- culty.
  • at most half of Solver deposit is split

among all Verifiers (rest goes to jackpot repository).

Table: Relations between parameters used in TrueBit.

pools, opportunistic attacks related to jackpot payoffs, and certain external threats.

Pairwise Sybil attacks

In a Sybil attack, an adversary assumes multiple identities on the network in order to execute an exploit. Identities on TrueBit include Task Givers, Solvers, Verifiers, Judges, and Referees. By our assumption (Section2.3), Judges and Referees always function as intended, and we provide additional justification for this axiom here. In this subsection, we consider all sets of pairs among the remaining identity types and show that pairwise coopera- tion does not harm operation of the system. While parties can freely join and leave TrueBit, each identity must make a deposit in order to participate. This deposit alone is a general deterrent against Sybil attacks, however as we shall see, multiple identities do not provide much cheating advantage.

Judges and Referees. Recall that Ethereum miners play the roles of Judges and Referees in TrueBit. Our analyses in Section3.4and Section4.6 show that the work done by these parties is small, and hence they are not vulnerable to a Verifier’s Dilemma (Section3.1). Nakamoto consensus [56] therefore ensures that miners will not post bogus transactions, i.e. enforce rules incorrectly, lest other miners reject their block and cost them a block reward. Therefore the only threat from miners is block withholding, specif- ically with regard to random number generator bias (see Section4.4).

While in theory miners could discard blocks, along with their mining reward and associated random bits for TrueBit, in practice this may never happen. A miner who drops a block must expect in exchange some income greater than the usual mining reward in order to make this action worth- while. If such income were to come as a bribe from a TrueBit participant, it would have to come from a Solver since only the Solver, who has unique access to his own private random bits, knows how to bias miners’ random bits towards or away from a forced error. In short, miners cannot disturb randomness in TrueBit without a tip-off from a Solver. The Solver, however, has no reason to bias against a forced error because this would prevent him from challenging his own answer and winning the jackpot, or at the very least would invite others to share it with him, thereby decreasing the total jackpot payout (see Section5.3). Moreover, the Solver is unlikely to suc- ceed in biasing towards a forced error since miners have little control over their own block hashes. This “one-sided” effect of block withholding, which can lock away jackpot funds but (almost) never release them, makes block

hashes a safe source of randomness in TrueBit. Hence the Solver’s potential reward for challenging his own solution under a forced error is not merely an artifact of TrueBit’s incentive structure — it guarantees unbiased Referees.

Task Giver and Solver. A Task Giver who also acts as a Solver does not benefit from solving her own task. One idea for an attack vector would be to create a task such that the Solver’s private random bits force an error for the given task. Since the Task Giver cannot predict the random bits from the block hash at the time the task is created (Section4.4, part (b)), the Task Giver cannot create such a task. Moreover, the Task Giver cannot afford to flood TrueBit with useless tasks and solve them herself in the hopes of eventually, by chance, encountering a forced error. The Task Giver would pay more taxes to the jackpot repository than she would expect to win from the jackpot payout (Section4.2), even taking into consideration any rewards she might win by correctly solving her own tasks. Her losses are amplified through payments to other Solvers, Verifiers, and Judges who choose to participate in the protocol.

Solver and Verifter. The Solver’s burned deposit always exceeds the Ver- ifier’s income for successfully challenging an unforced error (see Section4.3). Hence the Solver has no incentive to challenge himself as a Verifier in such cases. Similarly, a Verifier can only lose her deposit by posing bogus chal- lenges. In certain situations it is conceivable that a Solver–Verifier pair could benefit from submitting a false solution and then challenging it due to temporal constraints external to TrueBit itself (as mentioned in Section4.3), and in such cases the Task Giver must determine the necessary deposits to deter such actions.

In the case of an forced error, we expect the Solver will challenge himself in order to win the jackpot. Nevertheless, TrueBit’s tax rate (Section4.2), and hence its jackpot payout (Section4.1), suffices to incentivize an inde- pendent Verifier to also check the solution. As we shall see in Section5.3, the Solver lacks incentive to claim the same jackpot more than once, hence the Solver’s self-verification does not spoil other Verifiers’ motivation to par- ticipate.

Task Giver and Verifter. A Task giver can certainly verify the solution she receives. This checking can only improve the reliability of the system! We cannot, however, assume that the Task Giver will always perform such checks due to the Task Giver’s possible computational and financial resource

constraints or lack of motivation to involve herself in the details of a TrueBit

contract.

The trifecta

Up until this point, we have implicitly assumed that the attacker’s goal was to extract money from the jackpot without performing verification. How- ever there is another possibility: the attacker wishes to get a bogus solution accepted onto the blockchain. No penalty scheme can deter all such attacks since TrueBit, as a closed system, has no way to estimate the true economic impact of a bogus solution on Ethereum’s blockchain. Hence we now con- sider scenarios in which a party is willing to sacrifice a jackpot payout (or more) in order to skew computational results.

Scaring off Verifters. Suppose that a Task Giver wishes to obtain an incorrect solution for a task. Posing as a Solver, she gives a wrong solution to her own task. Now the only thing that could stand in the way of her success would be a pesky Verifier. In order to dodge this obstacle, the attacker poses as a regular Verifier prior to posting her malicious task. She checks every task that comes along, until eventually he encounters a forced error. At this point, the attacker challenges not once but a hundred times, which according to Step 4(b)ii in Section4.6essentially wipes out the jackpot payout for herself and any other Verifiers that also challenged this solution. The upshot is that legitimate Verifiers no longer wish to participate because TrueBit failed to deliver their well-deserved jackpot payout, and more to the point, they have no way to estimate their income from future jackpots. So when the attacker finally broadcasts her malicious task, legitimate Verifiers simply ignore it.

As Verifier participation, and hence security, depends entirely on the expected value of jackpot payouts (assumption (ii) in Section2.2), TrueBit must ensure sufficient expected value for these payouts. We therefore offer the following default strategy for all Verifiers. New tasks are submitted to TrueBit at various and overlapping times. A prospective Verifier who wishes to earn rewards for verification should count the number of active Verifier deposits and compare this to the average number of simultaneous tasks oc- curring on the network. If the tax rate and task flow profitably support an additional Verifier, then the prospective Verifier submits a deposit, ran- domly chooses an active task, and begins verifying. As soon as she finishes verifying this task, she randomly selects another active task and begins ver- ifying that next one. Thus a Verifier with either more powerful hardware or

better algorithms can gain a market advantage in TrueBit. Such a Verifier can check more solutions per unit time and therefore increase her expected jackpot rewards per unit time. Verifiers are free to leave the network at any time, for any reason. If the current number of active Verifiers exceeds the number supported by the tax rate and task flow, then the probabilis- tic default strategy tells each Verifier to exit the system with the minimal probability that one would expect to restore equilibrium if all Verifiers were following the default strategy.

Verifiers who choose tasks to verify at random according to the default strategy, as opposed to changing their behavior based on guesses as to who- solved-what before, can reasonably estimate their compensation for partici- pation. When a significant number of Verifiers (not necessarily a majority) follow the default strategy, the adversary wishing to establish a bogus so- lution can neither predict which tasks will be verified, nor can he influence which tasks default strategy Verifiers will verify. According to assump- tion (ii) in Section2.2, there exists some bound on the adversary’s financial means. If not, he could simply flood the Verifier pool with deposits and be- gin verifying, which in turn would incentivize legitimate Verifiers to leave the system. Therefore the adversary can only post a bogus solution and hope that, by chance, it gets ignored. In this way, the default Verifier strategy spoils the scare-off strategy above.

Let us now estimate the chances that the adversary succeeds to post a bogus solution which all legitimate Verifiers fail to notice. We make the worst-case assumption that the adversary Solver always gets elected to solve his own task. Suppose that the tax rate (see Section4.2) supports at least 6 Verifiers per task, and that the adversary holds at most 1/6 of all Verifier deposits. Then effectively there are an average of 5 verifiers per task. Sup- pose n is the average number of simultaneous tasks on the network. Then the number of Verifiers is approximately 5n, and so the chance that none of them check a particular solution is the chance that each of them inde- pendently and randomly choose other tasks, or [(n − 1)/n]5n. Therefore the probability that the adversary gets caught, regardless of n, is

1 − .1 −

1 5n

n

Σ

> 1 − e

5 > 99%.

The leftmost inequality follows from the standard limit definition for the Calculus constant e. We conclude that the adversary will most likely lose his Solver deposit by attempting such an attack.

By raising taxes further, we can make such an attack even more costly. In this sense the tax rate, together with the minimums for Solver deposits,

bounds the mandatory “honesty” of the network. If bogus solutions never occur, then there would be no need for taxes. On the other hand, higher taxes make it more expensive to cheat. There is a tradeoff between overhead tax expenses for Task Givers, minimum deposits for Solvers, and security of computations.

High-stake tasks. For some high-stakes applications it is possible to en- tirely fall back on the dispute resolution layer of Section3. In a high-stakes situation, it is reasonable to expect that other parties who have a significant financial stake in the outcome of a given task will make an effort to verify the solution regardless of expected jackpot payouts. This motivation acts as a secondary deterrent against the “Scaring off Verifiers” attack above, as does the fact that the Verifier receives a fraction of the Solver’s deposit in case of positive error detection (TrueBit burns part of the Solver’s deposit in order to avoid collusion between Solvers and Verifiers).

Collusion pools

In this section, we analyze the potential effects of pooling computational, informational, and financial resources among Verifiers and Solvers. Par- ticipants in TrueBit may voluntarily form a Cartel which may in turn use external smart contracts to enforce cooperation among its (mutually dis- trusting) members, however, Solver and Verifier deposits in TrueBit alone prevent many potentially harmful financial collusions designed to extract jackpot funds without exerting computational effort. Secondly, carefully chosen jackpot payout amounts, together with Solver and Verifier incentives and private commitment channels, prevent financial gain from unintended signaling of private random bits or challenges.

Rich but powerless. First, consider a Cartel of Solvers with limited CPU bandwidth but deep pockets which volunteers to solve each and every task that comes along. The Cartel makes no attempt to provide correct solutions to any of these tasks, but instead intends to absorb lost Solver deposits un- til a forced error comes along, at which point it splits the jackpot among its members. By construction (see Section4.3), Solver deposits exceed the expected jackpot payout per task. Therefore, in the long run, such a Cartel strategy loses money. If members of the Cartel instead decide to start pro- ducing correct solutions, this does not harm security because that’s what they are supposed to do anyway. We remark that, also by construction (Section4.4), a Solver cannot simply decide to accept only tasks which have

forced errors because the Solver does not know whether or not a forced error is in effect until after he has committed his solution to the blockchain.

Similarly, a Cartel which challenges every single task in the hopes of eventually obtaining a jackpot sustains losses in the long run due to lost Verifier deposits.

A flood of Challengers. Timely information about forced errors has eco- nomic value. While Solvers can earn jackpots by challenging their own forced errors, they risk sharing jackpot payouts by divulging information about such errors. It follows that Solvers have incentive to keep their knowledge of forced errors private and their fake solutions convincing. Information about active challenges also has economic value because a challenge posted to the blockchain could be a tip-off about a forced error. For this reason, Verifiers have incentive to announce “fake” challenges rather than remaining silent on tasks in which they don’t detect errors (See Step 5. in Section4.6). “Fake” challenges serve as noise to mask real challenges and protect Verifiers’ jack- pot revenue from potential copycats who might challenge just because they see someone else doing it.

If the jackpot for a given task were a fixed amount and equally divided among challenging participants, then one could potentially flood the set of participants with aliases to obtain more than one’s share of the jackpot. A Cartel in which members inform each other about forced errors could therefore have the economic means to flood the network with challenges and monopolize jackpot payouts. This in turn would dilute jackpots for legitimate Verifiers which would, in turn, degrade Verifier incentives for participation. While such a Cartel might successfully detect forced errors, it might also cause tasks with unforced errors to go unverified.

In order to dissuade a Cartel from flooding the network with harmful challenges, we must reduce the total jackpot payout based on the number of challenges that occur. The total jackpot paid out on a forced error with k challengers must not exceed

J

2k−1 . (5.1)

The reason is as follows. We want a Challenger’s net reward to decrease if she superfluously clones herself via the Cartel. Assume that there are n legitimate challenges plus a Cartel which initially contributes k challenges to the pool and is considering adding another alias. We want the per challenge jackpot distribution among n + k challenges to be less than it would be among n + k + 1 challenges regardless of k, and n. Let Ji denote the total

reward paid out when there are exactly i challenges. Then we want

k k + 1

n · Jn+k > n + 1 · Jn+k+1,

or

n + 1 k

Jn+k+1 < Jn+k · ·

n

>1 sfor˛a¸llxn≥0

.

1/2sfo˛r¸alxl k≥1

k + 1

Thus it suffices to set Jn+k+1 = Jn+k/2, or by induction Jk J1/2k−1. In fact we cannot do better than this because the case k = 1 asymptotically forces a matching bound.

The upper bound in (5.1) does not take into consideration the Verifier’s cost of playing the verification game. In the case where the Solver’s solution has been found to have an error through one of the challenges, there is no need to repeat the verification game with the other challenges. The Verifier who actually plays the verification game with the Solver receives a bonus from the Solver’s deposit to compensate her for the CPU cycles spent during the challenge.

On low-hanging fruit

We analyze the security implications of Verifiers and Solvers who oppor- tunistically disappear for hard tasks and reappear for easy ones, or who take advantage of solution discrepancies.

Easy winners. If the jackpot for all tasks were equal, rational Verifiers might choose to verify only simple tasks and ignore complex ones. For this reason, the jackpot for each task scales proportionally with the task’s complexity (Step 5. in Section4.6). Scaling ensures that a Verifier’s expected jackpot payout per CPU cycle remains constant across all tasks, and it equally incentivizes Verifiers to inspect simple and complex tasks. Solvers always appear to perform tasks since the minimum task reward suffices to compensate them for their work (see Section4.6and assumption (ii) in Section2.2, ).

Multiple solvers. For security reasons, TrueBit explicitly does not allow Task Givers to hire redundant Solvers for a single task (Main Algorithm, Step 2., Section4.6). Suppose that two Solvers provided solutions to a single task and exactly one of them receives a forced error. Any Observer who notices a difference in the two solutions on the blockchain could, without

verifying anything, challenge both solutions. By playing two verifications and sacrificing one deposit, such an Observer could potentially win a jackpot at negligible cost, thereby degrading the overall incentives for verification.

Forced error in disguise. What happens if a Solver has a forced error but gives a correct solution anyway? The Solver could then challenge his own solution while other Verifiers ignore it, resulting in both the Solver receiving a bigger share of the jackpot as well as damage to the system’s verification mechanism. Therefore, when the Solver reveals that a forced error was in effect, the protocol also forces him to reveal his committed “correct” solution which must be distinct from the “incorrect” solution that he showed at first. If the second solution revealed by the Solver is correct and distinct from his first solution, then by uniqueness of deterministic processes, the first solution must have been incorrect (as desired). Verifiers have an opportunity to challenge this second solution. If an error is detected in it via a verification game, the Verifier(s) split the Solver’s deposit rather than the jackpot payout according to Step 4(b)ii in Section4.6. Since the Verifier(s) presumably already performed the task themselves when checking the first solution, no additional CPU cycles are required to do this second check. As the Solver loses both a jackpot payout and a deposit by following this course, by assumption (ii) in Section2.3, the situation described in this paragraph should never arise.

A cash equivalence problem

Consider the following scenario. A Task Giver wishes to get a bogus solution onto the blockchain. He offers a minimal reward for a difficult task so as to ensure that no legitimate Solvers or Verifiers volunteer to inspect it. Acting as a Solver, she then provides a bogus solution, and the system accepts her solution because no one bothers to check it. It follows that for security purposes, the system must require that Task Givers compensate fairly based on the difficulty of the task. But how can TrueBit estimate the true cost of executing a given task in its native currency?

While one could potentially establish a long-term lower bound on the cost of a CPU cycle relative to a stable currency like the US dollar or Euro, calculating a lower bound relative to the value of a cryptocurrency token is another matter. Cryptocurrencies are extremely volatile. Moreover, TrueBit lives on the blockchain and does not have access to a newspaper with current exchange rates.

In the first iteration of TrueBit, we will manually update the internal cash equivalent of a CPU cycle based on a live feed (e.g. [4,23,26]). Ultimately, however, we would like to input these prices in a decentralized way without relying on a third-party. Later versions of the protocol may make use of Augur [2], a decentralized prediction market which ports outside information sources onto the Ethereum blockchain. As of this writing, Augur is currently in beta testing. Alternatively, we may build an independent blockchain for reporting prices whose “transactions” consist of exchange rate updates.

Implementation

Formally, TrueBit is a smart contract in Ethereum which uses Ethereum’s ex- isting smart contract framework to bootstrap the creation of computationally- intensive TrueBit contracts. Tasks in TrueBit take the form of C, C++, or Rust code, but the user must pass this code as input to the Google Lanai architecture [15] (discussed below) prior to submission to TrueBit. This lat- ter step guarantees consistency of simulated architecture and allows Judges to adjudicate fairly.

Google’s Lanai interpreter. The theoretical framework for the verifi- cation game requires a fixed computer architecture to be used for all veri- fication tasks. In [40], the authors used the regular Intel X86 architecture for this purpose, but we believe that this architecture is far too complicated to be used in TrueBit. All participants of the verification game, including the Judges, have to simulate the behavior of the entire architecture. Even a slight difference in these implementations could result in catastrophic loss of funds for one of the parties. Moreover, simpler architecture costs less to sim- ulate on the blockchain, and because of its simplicity and the fact that there now exists an efficient and trusted LLVM compiler writer, we have chosen to use Google’s Lanai architecture [15]. The full Google Lanai interpreter will be available as a smart contract in Ethereum so that it can be used by Judges. In all TrueBit contracts, the Lanai bytecode will be authoritative.

TrueBit’s on-chain interpreter runs in Solidity. For efficiency reasons, tasks will not be run using the interpreters except in case of dispute. In general, users will run tasks written in native programming languages and running on regular hardware. In the majority of cases where tasks do not require the Lanai interpreter, Solvers and Verifiers can privately optimize implementation of task executions and potentially gain a market advantage over other participants.

Applications

TrueBit is more than just an outsourced computation system. It is designed for use in trustless smart contracts. We present some examples of possible use cases.

Practical decentralized pooled mining

Mining rewards are extremely competitive. A typical desktop computer might only mine on average one Bitcoin block every thousand years. To reduce income variance, miners often join mining pools which share both computing resources and mining rewards. Each pool has an operator who distributes computing tasks and rewards to pool members. This central- ization degrades the security of the system by giving the operator undue influence to censor transactions [51]. In extreme cases where an operator controls more than half of the network’s hash rate, as has been the case with DwarfPool [9] in Ethereum, GHash.io [30] in Bitcoin, and could hap- pen again with Bitmain’s proposed gigantic mining center [3], the operator can even withdraw cleared transactions and double-spend money by way of a 51% attack [48].

SmartPool [27] introduces mining pools whose operators are Ethereum smart contracts. As decentralized pool operators, smart contracts have many virtues. In addition to counteracting the censorship issues described in the previous paragraph, they can operate at low cost relative to centralized pools and do not rely on a social contract to ensure fairness. Unlike other decentralized mining pools, which either have higher payout variance [24] that negate incentives for joining the pool or require a change in the proof- of-work protocol [54], SmartPool offers low variance payouts and retrofits existing cryptocurrencies, and it can handle a huge number of participants with a wide range of computing power.

SmartPool’s proposed Ethereum mining pool minimizes verification work by precomputing the 1 GB data sets needed to check Ethereum’s proof-of- work [10,51]. While this shortcut may help in checking Ethereum’s proof-of- work, not all cryptocurrencies have this lucky feature. Fortunately TrueBit contracts can check any proof-of-work, which means that with TrueBit, we can build a smart contract-based mining pool for any Nakamoto consensus- based cryptocurrency. At the time of this writing, for example, the effort required to check a Zcash proof-of-work [33] appears to exceed Ethereum’s gasLimit capacity by a factor of 50. TrueBit is an option for bringing this task within reach of Ethereum smart contracts. Project Alchemy [17],

which aims to bring smart contract functionality to Zcash in the spirit of the Dogecoin–Ethereum bridge below, may also benefit from TrueBit’s ability to check Zcash’s proof-of-work.

Dogecoin–Ethereum bridge

We can use TrueBit to build a two-way peg between Dogecoin [6] and Ether- eum, enabling Dogecoin users to move dogecoins between Dogecoin’s block- chain and Ethereum’s blockchain without transmitting currency through a third-party exchange and effectively adding smart contract functionality to Dogecoin. The Dogecoin community maintains an active interest in such a bridge [8], and current offers a prize of more than 6000 ETH for its con- struction [5,7].

TrueBit contracts can check Dogecoin’s Scrypt-based proof-of-work whereas traditional Ethereum smart contracts cannot. If Dogecoin were to enhance its scripting language with an operation indicating currency transfer to Ethereum addresses, a TrueBit contract could then confirm such transfers. The newly created dogecoin token could then be passed around Ethereum, and a final signed transaction in Ethereum could finally send the dogecoin back onto Dogecoin’s blockchain, assuming that Dogecoin miners are willing to treat the Ethereum blockchain as authoritative for such transfers.

Scalable transaction throughput

Building a system that can securely meet even the modest transaction vol- ume demands of current Bitcoin users remains a formidable challenge [41]. Proposed Byzantine “sharding” [36,42,47,49,53,55,57], and miner-based “serializing” [43,60] protocols exist which aim to distribute verification work, but here we take a simple and fundamentally different approach to decouple the two tasks which miners perform, namely

      1. selecting which transactions to include in the blockchain, and 2.verifying that blockchain transactions are valid.

Using TrueBit, one can construct a verification protocol whose incentives guarantee that task 2 is correctly performed by off-chain Solvers and Verifiers (with some help from Judges and Referees), while miners continue to perform task 1. In this way, complex transactions can securely reach the blockchain without overburdening miners.

Towards a big data system

In order to perform as a truly scalable cloud computer, TrueBit must have access to a scalable data storage system. Ethereum’s blockchain alone does not suffice as storing even moderate amounts of data directly on Ethereum’s blockchain is prohibitively expensive. TrueBit can securely access and use portions of massive data sets so long as the data is stored somewhere pub- licly and permanently, for example in Swarm [28] or on another blockchain. Parties who wish to rely on such data in a TrueBit contract must be sure that Verifiers have access to the full data set.

To use TrueBit on external data, one need only store a Merkle root of the massive data set on the blockchain and add non-deterministic steps in the verification game in which the Solver can “guess” the original data set rep- resented by the Merkle root. While Solvers and Verifiers must have access to the full data, Judges and Referees do not. Indeed, if we modify the ver- ification game so as to permit tasks for nondeterministic Turing machines, then the Solver can nondeterministically guess the certificate data as a step in the TrueBit contract. Only in cases of disputes would the Solver have to reveal external certificate data to the Judges via the blockchain. In some applications, the Solver might even even be able to reveal to the Judges a privacy-preserving zkSNARK rather than the data itself. zkSNARKs have the potential to enable privacy for many different kinds of systems on Ether- eum [59].

While in theory TrueBit’s scalable protocol can process arbitrarily com- plex tasks, in practice the verification game is inefficient and therefore secu- rity of TrueBit computations degrades as tasks reach the level of big data. For big data applications, TrueBit may not be able to rely on a one-size-fits- all verification game. Therefore we anticipate optimizing the verification game for certain classes of tasks. For example, Section3.2gives an example of an efficient, specialized verification game for matrix multiplication. In future versions of TrueBit, Task Givers might broadcast not only tasks but also an indication of the corresponding verification game to be played in case of disputes.

Remark. We conclude with a caveat: TrueBit may expose latent security vulnerabilities in the underlying Ethereum network as a result of new kinds of interactions between smart contracts and miners. By allowing smart contracts to check proof-of-works, for example, TrueBit may facilitate 38.2% attacks [62].

Acknowledgments. We thank Vitalik Buterin and Vlad Zamfir for sug- gesting the use of forced errors in the TrueBit protocol and Eli Bendersky for introducing us to Google Lanai. We also thank Loi Luu and Julia Koch for useful discussions.

Addendum

In the months following the initial release of this whitepaper, new work and feedback informed TrueBit’s development roadmap and helped refine the protocol itself. We discuss a few recent developments.

Security patches

A TrueBit adversary has either one of two goals: 1.get bogus computations onto the blockchain, or

2.extract jackpot funds without performing verification.

We investigate three attacks of the first type followed by two of the second.

Premature disclosure of random bits (Zack Lawrence). A Solver can dissuade Verifier participation by publicly broadcasting his private random bits prior to the designated reveal time. Verifiers then know immediately whether or not a forced error is in effect. Since Verifiers expect to gain little from checking solutions without forced errors, they may decide not to verify, thereby offering opportunity to get bogus solutions onto the blockchain.

1protocol’s [1] random number generator protocol, Arbit, solves this problem by instituting penalties for Solvers who prematurely reveal private random bits and rewarding users who report them. When a user correctly reports a premature reveal to TrueBit, the following occurs.

  1. The Solver’s solutions are discarded, and a new Solver lottery takes place. This re-incentivizes Verifiers to participate in the task while voiding the Solver’s incentive to reveal private information.
  2. Half of the Solver’s deposit gets burned. This makes the above attack expensive for the Solver.
  3. The other half of the Solver’s deposit goes to the party who reported the Solver’s random bits. This incentivizes Observers to report the Solver’s prematurely revealed random bits.

Incorrect secondary solution (Sina Habibian and Harley Swick [22]). Suppose that a forced error is in effect and that the Solver submits two incorrect solutions. When the Solver reveals his “correct” secondary solution in Step 4(b)ii of the protocol (Section4.6), Verifiers ignore it because there’s no chance of a jackpot payout. Indeed, the only “reward” for correctly challenging this secondary solution is to play a verification game. Hence one of the Solver’s bogus solutions ends up on the blockchain.

We eliminate the incorrect secondary solution vulnerability as follows. Denote the Solver’s two solutions by A and B. In the beginning of Step 4, rather than signaling for a challenge with the hash of an even integer, the Verifier hashes an integer whose value mod 3 the protocol interprets as follows:

  1. mod 3: challenge solution A,
  2. mod 3: challenge solution B,
  3. mod 3: challenge both A and B.

The Verifiers indicate their choice without knowing which of the two solu- tions the Solver puts forth as an answer. The protocol hides this information from Verifiers via the following process. The Solver commits to either solu- tion A or solution B by hashing either A or B paired with his private random bits, where the Solver’s private random bits serve as “noise” which prevent Verifiers from guessing which option the Solver chose. The Solver has incen- tive not to share his private random bits due to the “Premature disclosure of random bits” patch above as well as the fact that the Solver risks reducing his jackpot share by exposing this private information. Finally, once the timeout for challenges has passed, the Solver reveals his random bits in the clear, thereby indicating his commitment to either solution A or solution

        1. Challenges and forced errors then proceed as usual. In case the protocol forces the Solver to reveal his second “correct” solution, Verifiers who earlier committed to a challenge against this second solution are obligated to play a verification game. In this way, Verifiers catch errant secondary solutions just as they catch errant primary ones.

In case a forced error is not in effect, broadcasting a pair of incorrect solutions poses a cost to the Solver in the form of a lost deposit. Indeed Verifiers have proper incentives to check the Solver’s primary answer. Since forced errors occur rarely and unpredictably, the Solver expects to sacrifice several deposits in order to mount an “incorrect secondary solution” attack. This loss offsets the Solver’s eventual jackpot gain from challenging his own forced error solution. We implicitly assume that the chance to win a jackpot sufficiently motivates a Verifier to challenge whenever a Solver submits a pair

of incorrect solutions; any Verifier who challenges both submitted solutions must play a verification game.

The fix above has a potentially useful side effect of publicly indicating how many Verifiers are monitoring a given task. Indeed, a Verifier broad- casts one of the three commitment forms above if and only if she is paying attention. The option to signal “no challenge” is no longer needed for cam- ouflage because broadcasting challenges no longer indicates presence of a forced error. Moreover, if the Solver were to submit two correct solutions, the smart contract could immediately recognize them as identical and pe- nalize the Solver accordingly.

An adversary could potentially exploit the monitoring feature in the pre- vious paragraph by broadcasting multiple challenge commitments from Sybil identities, thereby reducing the total payout in case of a forced error and discouraging other rational Verifiers from participating. For this reason, the protocol must prevent Verifiers from revealing which task they are challeng- ing until the final phase of the protocol. Since each Verifier performs the given computational task without distraction from others’ commitments, an adversary cannot deter Verifier participation via Sybil attack.

Program abort (Yaron Velner). A task must explicitly specify an upper bound on the number of steps for which the verification game can run, and the Solver’s solution should return an “error” if and only if the task computation exceeds this bound. Indeed, the computation steps for a given task must form a deterministic sequence in order for Judges to determine their correctness. Ideally, one would like to run tasks directly in a native language like C, C++, or Rust, however this approach requires accurate metering of the number of computation steps. We can reduce the metering overhead by processing steps with a compiler rather than an interpreter.

Jackpot balloon attack (Cl´ement Lesaege). In the “flood of challengers” attack (Section5.3), a Cartel could artificially inflate the jackpot repository by repeatedly challenging a single forced error solution and thereby reduce jackpot payout. Eventually, the Cartel could offset the cost of this attack by cashing in on an extra large jackpot. This action could recruit membership for the Cartel at low cost by removing incentives for Verifiers who do not belong to the Cartel.

We mitigate against the benefit of this attack by placing a fixed, absolute bound on the jackpot payout, regardless of the actual jackpot repository balance. Extra revenue accumulated in the jackpot repository only becomes

available for payout after a protocol upgrade. Note that the maximum jackpot amount does not necessarily pay out at each forced error; the actual payout depends on the difficulty of the task.

Incentivizing block withholding (Cl´ement Lesaege). The attacker, who is a Solver, deploys a smart contract which pays money to miners who with- hold (or intentionally uncle) blocks whose random seed fails to yield a forced error. See [64] for possible implementations. This attack is profitable for tasks in which the expected jackpot payout times the probability of a forced error exceeds a block reward. Through repetitions, this attack could drain the jackpot repository.

Arbit’s security mechanism, discussed in “Premature disclosure of ran- dom bits” above, defends against this attack. In order to execute the present “incentivizing block withholding” attack, the Solver either has to reveal his bits publicly or has to provide them to miners through private channels. If the Solver’s penalty for a premature reveal exceeds the expected jackpot payout times the probability of a forced error, then this attack results in an expected net loss.

The TrueBit Virtual Machine

Section3.3describes a verification game which operates with respect to a Turing machine. In practice, however, no compiler exists which can trans- form C++ code into something as simple as Turing machine language. In Section6, we proposed Google Lanai as a practical compromise in the ab- sence of Turing machine architecture. Due to Lanai’s complexity, the fact that Google controls its codebase, and that progress on its development ap- pears to have slowed, we have since migrated development away from Google Lanai.

In addition to executing exactly the same computation steps regardless of hardware configuration, the complier architecture underlying the verifica- tion game, or TrueBit Virtual Machine (TVM), must satisfy the following simplicity properties.

1.A single computation step on the TVM runs comfortably within Ether- eum’s gas limit, and

2.the space required to describe a TVM state change fits inside a single Ethereum transaction.

WebAssembly architecture [31] comes sufficiently close to matching these properties so as to make TVM execution practical today. WebAssembly has

become increasingly ready-to-use due to contributions and testing by Apple, Google, Microsoft, and Mozilla [45]. Several cryptocurrency projects have begun to develop on WebAssembly, including Dfinity, eWASM, and Parity, due to the platform’s machine independence and relative simplicity.

The TVM consists of two parts:

  1. an off-chain interpreter which enumerates a list of states for a given computation, and
  2. an on-chain stepper which, given a state, can compute the next state.

Solvers and Challengers use the interpreter to create a Merklized list of states for a computation. Once the Solver and Challenger have determined the first step at which they disagree, the Judges use the stepper to run this step and rule on the outcome of the verification game.

Since Native WebAssembly does not entirely satisfy the simplicity prop- erties above, the interpreter must either further compile input code into a simpler architecture, or it must divide each WebAssembly instruction into substeps. In order to reduce the chances of inducing compiler error, the TVM follows the latter strategy. The TVM’s off-chain interpreter parses WebAssembly instructions into OCaml execution which in turn creates reg- isters describing WebAssembly suboperations. Each Ocaml-based sub-step only accesses one dynamic data structure at a time.

Additional applications

Finally, we mention a few more illustrative applications.

Video broadcasting. Livepeer [18] offers a new platform for decentral- ized, live streaming video. Users can broadcast, watch, or get paid for performing the computationally intensive process of transcoding video into different codecs and formats. TrueBit ensures that transcoders perform this work correctly, while Swarm [28] guarantees that video data remains avail- able to TrueBit during the necessary verification period.

Autonomous machine learning. McConaghy’s ArtDAO [52] generates art, sells it, and then uses its revenue to improve its own code. A TrueBit- based ArtDAO would allow a financed piece of code on the blockchain to access computational resources in such a way that no one can “pull its plug.” We may eventually see other blockchain-based machine learning ap- plications, like computer vision, as well.

Data marketplace. Hedge fund Numerai [19] crowdsources modeling problems to data scientists and then executes trades based upon their work. Numerai rewards successful models, however contributing data scientists must trust Numerai to both test their work and compensate them fairly. TrueBit enables autonomous data markets. Open Mined [21], paired with TrueBit, opens the possibility of trustless renumeration based on stream- ing flows of models and data. The Ocean Protocol [20], which facilitates contribution and sharing of data, also requires a verification mechanism.

Staking and random numbers. 1protocol [1] allows users who have ei- ther computing resources or capital, but not necessarily both, to participate as TrueBit Solvers and Verifiers by decoupling security deposits from work done. In addition, 1protocol’s Arbit protocol uses interactive verification to generate random numbers in a decentralized way.

Other applications. Please check the TrueBit website for other current ideas in progress! https://truebit.io.

References

[1]1protocol. http://1protocol.com. [2]Augur. https://www.augur.net/.

[16]I thikn the attacker is this miner—today he made over $50k. https://www.reddit.com/r/ethereum/comments/55xh2w/i_thikn_ the_attacker_is_this_miner_today_he_made/.

  1. Introducing Project Alchemy. https://z.cash/blog/ project-alchemy.html.
  2. Livepeer. https://livepeer.org/. [19]Numerai. https://numer.ai.

[20]Ocean Protocol. https://oceanprotocol.com/. [21]Open Mined. https://openmined.org/.

  1. Open problems. https://github.com/TrueBitFoundation/ Developer-Resources/wiki/Open-Problems.
  2. Oraclize. http://www.oraclize.it/. [24]P2Pool. http://p2pool.org/.

[25]RANDAO. https://github.com/randao/randao. [26]Reality Keys. https://www.realitykeys.com/. [27]SmartPool. http://smartpool.io.

[28]Swarm. http://swarm-gateways.net/. [29]The DAO. http://daohub.org/.

  1. Warning: Ghash.io is nearing 51% – leave the pool. http://www.cryptocoinsnews.com/ warning-ghash-io-nearing-51-leave-pool/.
  2. WebAssembly. http://webassembly.org/.
  3. White paper. https://github.com/ethereum/wiki/wiki/ White-Paper.
  4. Why Equihash? https://z.cash/blog/why-equihash.html. [34]Zcash. https://z.cash/.
  5. Some miners generating invalid blocks. https://bitcoin.org/en/ alert/2015-07-04-spv-mining, July 2015.
  6. Ittai Abraham, Dahlia Malkhi, Kartik Nayak, Ling Ren, and Alexander Spiegelman. Solidus: An incentive-compatible cryptocurrency based on permissionless Byzantine consensus. https://arxiv.org/abs/1612. 02916, 2016.
  7. Eli Ben-Sasson, Alessandro Chiesa, Daniel Genkin, Eran Tromer, and Madars Virza. SNARKs for C: Verifying program executions succinctly and in zero knowledge. In Ran Canetti and Juan A. Garay, editors, Advances in Cryptology – CRYPTO 2013: 33rd Annual Cryptology Conference, Santa Barbara, CA, USA, August 18-22, 2013. Proceed- ings, Part II, pages 90–108, Berlin, Heidelberg, 2013. Springer Berlin Heidelberg.
  8. Nir Bitansky, Ran Canetti, Omer Paneth, and Alon Rosen. Indistin- guishability obfuscation vs. auxiliary-input extractable functions: One must fall. https://eprint.iacr.org/2013/468.pdf.
  9. Ran Canetti, Ben Riva, and Guy N. Rothblum. Practical delegation of computation using multiple servers. In Proceedings of the 18th ACM Conference on Computer and Communications Security, CCS ’11, pages 445–454, New York, NY, USA, 2011. ACM.
  10. Ran Canetti, Ben Riva, and Guy N. Rothblum. Refereed delegation of computation. Information and Computation, 226:16 – 36, 2013. Special Issue: Information Security as a Resource.
  11. Kyle Croman, Christian Decker, Ittay Eyal, Adem Efe Gencer, Ari Juels, Ahmed Kosba, Andrew Miller, Prateek Saxena, Elaine Shi, Emin Gu¨n Sirer, Dawn Song, and Roger Wattenhofer. On scaling decen- tralized blockchains (a position paper). In Financial Cryptography and Data Security 2016 BITCOIN Workshop, volume 9604 of Lecture Notes in Computer Science, pages 106–125. Springer Berlin Heidelberg, February 2016.
  12. Christian Decker, Jochen Seidel, and Roger Wattenhofer. Bitcoin meets strong consistency. In Proceedings of the 17th International Conference on Distributed Computing and Networking, ICDCN ’16, pages 13:1– 13:10, New York, NY, USA, 2016. ACM.
  13. Ittay Eyal, Adem Efe Gencer, Emin Gun Sirer, and Robbert Van Re- nesse. Bitcoin-NG: A scalable blockchain protocol. In 13th USENIX Symposium on Networked Systems Design and Implementation (NSDI 16), pages 45–59, Santa Clara, CA, March 2016. USENIX Association.
  14. Tim Goddard. AdversariallyVerifiableMachine. https:

//www.reddit.com/r/ethereum/comments/51qjz6/interactive_ verification_of_c_programs/d7ey41n/.

  1. Andreas Haas, Andreas Rossberg, Derek L. Schuff, Ben L. Titzer, Michael Holman, Dan Gohman, Luke Wagner, Alon Zakai, and JF Bastien. Bringing the web up to speed with WebAssembly. In Proceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2017, pages 185–200, New York, NY, USA, 2017. ACM.
  2. Sanjay Jain, Prateek Saxena, Frank Stephan, and Jason Teutsch. How to verify computation with a rational network. https://arxiv.org/ abs/1606.05917, June 2016.
  3. Eleftherios Kokoris Kogias, Philipp Jovanovic, Nicolas Gailly, Ismail Khoffi, Linus Gasser, and Bryan Ford. Enhancing Bitcoin security and performance with strong consistency via collective signing. In 25th USENIX Security Symposium (USENIX Security 16), pages 279–296, Austin, TX, 2016. USENIX Association.
  4. Joshua A. Kroll, Ian C. Davey, and Edward W. Felten. The economics of Bitcoin mining, or Bitcoin in the presence of adver- saries. http://www.econinfosec.org/archive/weis2013/papers/ KrollDaveyFeltenWEIS2013.pdf, June 2013.
  5. Loi Luu, Viswesh Narayanan, Chaodong Zheng, Kunal Baweja, Seth Gilbert, and Prateek Saxena. A secure sharding protocol for open blockchains. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, CCS ’16, pages 17–30, New York, NY, USA, 2016. ACM.
  6. Loi Luu, Jason Teutsch, Raghav Kulkarni, and Prateek Saxena. Demys- tifying incentives in the consensus computer. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security (CCS 2015), pages 706–719, New York, NY, USA, 2015. ACM.
  7. Loi Luu, Yaron Welner, Jason Teutsch, and Prateek Saxena. Smart- Pool: Practical decentralized pooled mining. http://smartpool.io/ docs/smartpool.pdf.
  8. Trent McConaghy. Wild, wooly AI DAOs. https://blog.bigchaindb. com/wild-wooly-ai-daos-d1719e040956.
  9. Silvio Micali. ALGORAND: the efficient and democratic ledger. http:

//arxiv.org/abs/1607.01341, 2016.

  1. Andrew Miller, Ahmed Kosba, Jonathan Katz, and Elaine Shi. Nonout- sourceable scratch-off puzzles to discourage Bitcoin mining coalitions. In Proceedings of the 22Nd ACM SIGSAC Conference on Computer and Communications Security, CCS ’15, pages 680–691, New York, NY, USA, 2015. ACM.
  2. Andrew Miller, Yu Xia, Kyle Croman, Elaine Shi, and Dawn Song. The honey badger of BFT protocols. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, CCS ’16, pages 31–42, New York, NY, USA, 2016. ACM.
  3. Satoshi Nakamoto. Bitcoin P2P e-cash paper. http://www. mail-archive.com/cryptography@metzdowd.com/msg09959.html, November 2008.
  4. Rafael Pass and Elaine Shi. Hybrid consensus: Efficient consensus in the permissionless model. https://eprint.iacr.org/2016/917.pdf, 2016.
  5. Christian Reitwiessner. From smart contracts to courts with not so smart judges. https://blog.ethereum.org/2016/02/17/ smart-contracts-courts-not-smart-judges/.
  6. Christian Reitwiessner. zkSNARKs in a nutshell. https://blog. ethereum.org/2016/12/05/zksnarks-in-a-nutshell/, Dececmber 2016.
  7. Yonatan Sompolinsky, Yoad Lewenberg, and Aviv Zohar. SPECTRE: A fast and scalable cryptocurrency protocol. https://eprint.iacr. org/2016/1159.pdf, 2016.
  8. Nick Szabo. The idea of smart contracts. http://szabo.best.vwh. net/smart_contracts_idea.html, 1997.
  9. Jason Teutsch, Sanjay Jain, and Prateek Saxena. When cryptocur- rencies mine their own business. In Financial Cryptography and Data Security: 20th International Conference (FC 2016) Christ Church, Bar- bados, pages 499–514. Springer Berlin / Heidelberg, 2017.
  10. Jelle van den Hooff, M. Frans Kaashoek, and Nickolai Zeldovich. Ver- sum: Verifiable computations over large public logs. In Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security, CCS ’14, pages 1304–1316, New York, NY, USA, 2014. ACM.
  11. Yaron Velner, Jason Teutsch, and Loi Luu. Smart contracts make Bit- coin mining pools vulnerable. To appear in 4th Workshop on Bitcoin and Blockchain Research (BITCOIN 2017).
  12. Michael Walfish and Andrew J. Blumberg. Verifying computations without reexecuting them. Communications of the ACM, 58(2):74–84, January 2015.

Polkadot

Highlight to Annotate. Click to Learn.

 

POLKADOT: VISION FOR A HETEROGENEOUS MULTI-CHAIN FRAMEWORK

DR. GAVIN WOOD FOUNDER, ETHEREUM & PARITY | GAVIN@PARITY.IO

Abstract. Present-day blockchain architectures all suffer from a number of issues not least practical means of extensibility and scalability. We believe this stems from tying two very important parts of the consensus architecture, namely canonicality and validity, too close together. This paper introduces an architecture, the heterogeneous multi-chain, which fundamentally sets the two apart.

In compartmentalizing these two parts, and by keeping the overall functionality provided to an absolute minimum of security and transport, we introduce practical means of core extensibility in situ. Scalability is addressed through a divide-and-conquer approach to these two functions, scaling out of its bonded core through the incentivization of untrusted public nodes.

The heterogeneous nature of this architecture enables many highly divergent types of consensus systems interoperating in a trustless, fully decentralized “federation”, allowing open and closed networks to have trust-free access to each other.

We put forward a means of providing backward compatibility with one or more pre-existing networks such as Ethereum. We believe that such a system provides a useful base-level component in the overall search for a practically implementable system capable of achieving global-commerce levels of scalability and privacy.

Preface

This is intended to be a technical “vision” summary of one possible direction that may be taken in further developing the blockchain paradigm together with some rationale as to why this direction is sensible. It lays out in as much detail as is possible at this stage of development a system which may give a concrete improvement on a number of aspects of blockchain technology.

It is not intended to be a specification, formal or otherwise. It is not intended to be comprehensive nor to be a final design. It is not intended to cover non-core aspects of the framework such as APIs, bindings, languages, and usage. This is notably experimental; where parameters are specified, they are likely to change. Mechanisms will be added, refined and removed in response to community ideas and critiques. Large portions of this paper will likely be revised as experimental evidence and prototyping gives us information about what will work and what not.

This document includes a core description of the pro- tocol together with ideas for directions that may be taken to improve various aspects. It is envisioned that the core description will be used as the starting point for an initial series of proofs-of-concept. A final “version 1.0” would be based around this refined protocol together with the ad- ditional ideas that become proven and are determined to be required for the project to reach its goals.

 

Introduction

Blockchains have demonstrated great promise of utility over several fields including “Internet of Things” (IoT), finance, governance, identity management, web- decentralization and asset-tracking. However, despite the technological promise and grand talk, we have yet to see significant real-world deployment of present technology. We believe that this is down to five key failures of present technology stacks:

Scalability: How much resources are spent globally on processing, bandwidth and storage for the system to process a single transaction and how many transactions can be reasonably processed under peak conditions?

Isolatability: Can the divergent needs of multiple parties and applications be addressed to a near-optimal degree under the same framework?

Developability: How well do the tools work? Do the APIs address the developers’ needs? Are educational materials available? Are the right integrations there?

Governance: Can the network remain flexible to evolve and adapt over time? Can decisions be made with sufficient inclusivity, legitimacy, and transparency to provide effective leadership of a decentralized system?

Applicability: Does the technology actually address a burning need on its own? Is other “middleware” required in order to bridge the gap to actual applications?

In the present work, we aim to address the first two issues: scalability and isolatability. That said, we believe the Polkadot framework can provide meaningful improvements in each of these classes of problems.

Modern, efficient blockchain implementations such as the Parity Ethereum client [16] can process in excess of 3,000 transactions per second when running on performant consumer hardware. However, current real-world blockchain networks are practically limited to around 30 transactions per second. This limitation mainly originates from the fact that the current synchronous consensus mechanisms require wide timing margins of safety on the expected processing time, which is exacerbated by the desire to support slower implementations. This is due to the underlying consensus architecture: the state transition mechanism, or the means by which parties collate and execute transactions, has its logic fundamentally tied into the consensus “canonicalization” mechanism, or the means by which parties agree upon one of a number of possible, valid, histories.

This applies equally to both proof-of-work (PoW) systems such as Bitcoin [14] and Ethereum [5, 22] and proof- of-stake (PoS) systems such as NXT [8] and Bitshares [12]: all ultimately suffer from the same handicap. It is a simple strategy that helped make blockchains a success. However, by tightly coupling these two mechanisms into a single unit of the protocol, we also bundle together multiple different actors and applications with different risk profiles, different scalability requirements and different privacy needs. One size does not fit all. Too often it is the case that in a desire for broad appeal, a network adopts a degree of conservatism which results in a lowest-common-denominator optimally serving few and ultimately leading to a failure in the ability to innovate, perform and adapt, sometimes dramatically so.

Some systems such as e.g. Factom [20] drop the state- transition mechanism altogether. However, much of the utility that we desire requires the ability to transition state according to a shared state-machine. Dropping it solves an alternative problem; it does not provide an alternative solution.

It seems clear, therefore, that one reasonable direction to explore as a route to a scalable decentralized compute platform is to decouple the consensus architecture from the state-transition mechanism. And, perhaps unsurprisingly, this is the strategy that Polkadot adopts as a solution to scalability.

Protocol, Implementation, and Network. Like Bitcoin and Ethereum, Polkadot refers at once to a network protocol and the (hitherto presupposed) primary public network that runs this protocol. Polkadot is intended to be a free and open project, the protocol specification being under a Creative Commons license and the code being placed under a FLOSS license. The project is developed in an open manner and accepts contributions where ever they are useful. A system of RFCs, not unlike the Python Improvement Process, will allow a means of publicly collaborating over protocol changes and upgrades. Our initial implementation of the Polkadot protocol will be known as the Parity Polkadot Platform and will include a full protocol implementation together with API bindings. Like other Parity blockchain implementations, PPP is designed to be a general-purpose blockchain tech- nology stack, neither uniquely for a public network nor for private/consortium operation. The development of it thus far has been funded by several parties including through a grant from the British government.

This paper nonetheless describes Polkadot under the context of a public network. The functionality we envision in a public network is a superset of that required in alternative (e.g. private and/or consortium) settings. Furthermore, in this context, the full scope of Polkadot can be more clearly described and discussed. This does mean the reader should be aware that certain mechanisms may be described (for example interoperation with other public networks) which are not directly relevant to Polkadot when deployed under non-public (“permissioned”) situations.

Previous work. Decoupling the underlying consen- sus from the state-transition has been informally proposed in private for at least two years—Max Kaye was a pro- ponent of such a strategy during the very early days of Ethereum.

A more complex scalable solution known as Chain fibers, dating back to June 2014 and first published later that year1, made the case for a single relay-chain and mul- tiple homogeneous chains providing a transparent interchain execution mechanism. Decoherence was paid for through transaction latency—transactions requiring the coordination of disparate portions of the system would take longer to process. Polkadot takes much of its architecture from that and the follow-up conversations with various people, though it differs greatly in much of its design and provisions.

While there are no systems comparable to Polkadot actually in production, several systems of some relevance have been proposed, though few in any substantial level of detail. These proposals can be broken down into systems which drop or reduce the notion of a globally coherent state machine, those which attempt to provide a globally coherent singleton machine through homogeneous shards and those which target only heterogeneity.

Systems without Global State. Factom [20] is a system that demonstrates canonicality without the according validity, effectively allowing the chronicling of data. Be- cause of the avoidance of global state and the difficulties with scaling which this brings, it can be considered a scal- able solution. However, as mentioned previously, the set of problems it solves is strictly and substantially smaller. Tangle [17] is a novel approach to consensus systems. Rather than arranging transactions into blocks and form- ing consensus over a strictly linked list to give a globally canonical ordering of state-changes, it largely aban- dons the idea of a heavily structured ordering and instead pushes for a directed acyclic graph of dependent transactions with later items helping canonicalise earlier items through explicit referencing. For arbitrary state-changes, this dependency graph would quickly become intractable, however for the much simpler UTXO model2 this becomes quite reasonable. Because the system is only loosely coherent and transactions are generally independent of each other, a large amount of global parallelism becomes quite natural. Using the UTXO model does have the effect of limiting Tangle to a purely value-transfer “currency” system rather than anything more general or extensible. Furthermore without the hard global coherency, interaction with other systems—which tend to need an absolute degree knowledge over the system state—becomes impractical.

 

Heterogeneous Chain Systems. Side-chains [3] is a proposed addition to the Bitcoin protocol which would al- low trustless interaction between the main Bitcoin chain and additional side-chains. There is no provision for any degree of ‘rich’ interaction between side-chains: the in- teraction would be limited to allowing side-chains to be custodians of each other’s assets, effecting—in the local jargon—a two-way peg 3. The end vision is for a frame- work where the Bitcoin currency could be provided with additional, if peripheral, functionality through pegging it onto some other chains with more exotic state transition systems than the Bitcoin protocol allows. In this sense, side-chains addresses extensibility rather than scalability. Indeed, there is fundamentally no provision for the va- lidity of side-chains; tokens from one chain (e.g. Bitcoin) held on behalf of a side-chain are secured only by the side-chain’s ability to incentivise miners to canonicalise valid transitions. The security of the Bitcoin network cannot easily be transitioned to work on behalf of other blockchains. Furthermore, a protocol for ensuring Bitcoin miners merge-mine (that is duplicate their canonicalisa- tion power onto that of the side-chain) and, more impor- tantly, validate the side-chain’s transitions is outside the

scope of this proposal.

Cosmos [10] is a proposed multi-chain system in the same vein as side-chains, swapping the Nakamoto PoW consensus method for Jae Kwon’s Tendermint algorithm. Essentially, it describes multiple chains (operating in zones) each using individual instances of Tendermint, to- gether with a means for trust-free communication via a master hub chain. This interchain communication is lim- ited to the transfer of digital assets (“specifically about to- kens”) rather than arbitrary information, however such in- terchain communication does have a return path for data,

e.g. to report to the sender on the status of the transfer. Validator sets for the zoned chains, and in particular the means of incentivising them, are, like side-chains, left as an unsolved problem. The general assumption is that each zoned chain will itself hold a token of value whose in- flation is used to pay for validators. Still in the early stages of design, at present the proposal lacks comprehensive de- tails over the economic means of achieving the scalable certainty over global validity. However, the loose coher- ence required between the zones and the hub will allow for additional flexibility over the parameters of the zoned chains compared to that of a system enforcing stronger

coherence.

Casper. As yet no comprehensive review or side- by-side comparison between Casper [6] and Polkadot have been made, though one can make a fairly sweeping (and accordingly inaccurate) characterisation of the two. Casper is a reimagining of how a PoS consensus algorithm could be based around participants betting on which fork would ultimately become canonical. Substantial consider- ation was given to ensuring that it be robust to network forks, even when prolonged, and have some additional de- gree of scalability on top of the basic Ethereum model. As such, Casper to date has tended to be a substantially more complex protocol than Polkadot and its forebears, and a substantial deviation from the basic blockchain format. It

remains unseen as to how Casper will iterate in the future and what it will look like should it finally be deployed.

While Casper and Polkadot both represent interest- ing new protocols and, in some sense, augmentations of Ethereum, there are substantial differences between their ultimate goals and paths to deployment. Casper is an Ethereum Foundation-centered project originally designed to be a PoS alteration to the protocol with no desire to create a fundamentally scalable blockchain. Crucially, it is designed to be a hard-fork, rather than anything more ex- pansive and thus all Ethereum clients and users would be required to upgrade or remain on a fork of uncertain adop- tion. As such, deployment is made substantially more dif- ficult as is inherent in a decentralised project where tight coordination is necessary.

Polkadot differs in several ways; first and foremost, Polkadot is designed to be a fully extensible and scalable blockchain development, deployment and interaction test bed. It is built to be a largely future-proof harness able to assimilate new blockchain technology as it becomes avail- able without over-complicated decentralised coordination or hard forks. We already envision several use cases such as encrypted consortium chains and high-frequency chains with very low block times that are unrealistic to do in any future version of Ethereum currently envisioned. Fi- nally, the coupling between it and Ethereum is extremely loose; no action on the part of Ethereum is necessary to enable trustless transaction forwarding between the two networks.

In short, while Casper/Ethereum 2.0 and Polkadot share some fleeting similarities we believe their end goal is substantially different and that rather than competing, the two protocols are likely to ultimately co-exist under a mutually beneficial relationship for the foreseeable future.

Summary

Polkadot is a scalable heterogeneous multi-chain. This means that unlike previous blockchain implementations which have focused on providing a single chain of varying degrees of generality over potential applications, Polka- dot itself is designed to provide no inherent applica- tion functionality at all. Rather, Polkadot provides the bedrock “relay-chain” upon which a large number of val- idatable, globally-coherent dynamic data-structures may be hosted. We call these data-structures “parallelised” chains or parachains, though there is no specific need for them to be blockchain in nature.

In other words, Polkadot may be considered equiva- lent to a set of independent chains (e.g. the set containing Ethereum, Ethereum Classic, Namecoin and Bitcoin) ex- cept for two very important points:

Pooled security;

trust-free interchain transactability.

These points are why we consider Polkadot to be “scal- able”. In principle, a problem to be deployed on Polka- dot may be substantially parallelised—scaled out—over a large number of parachains. Since all aspects of each parachain may be conducted in parallel by a different seg- ment of the network, the system has some ability to scale.

3as opposed to a one-way peg which is essentially the action of destroying tokens in one chain to create tokens in another without the mechanism to do the converse in order to recover the original tokens

image

Polkadot provides a rather bare-bones piece of infrastruc- ture leaving much of the complexity to be addressed at the middleware level. This is a conscious decision intended to reduce development risk, enabling the requisite software to be developed within a short time span and with a good level of confidence over its security and robustness.

3.1. The Philosophy of Polkadot. Polkadot should provide an absolute rock-solid foundation on which to build the next wave of consensus systems, right through the risk spectrum from production-capable mature designs to nascent ideas. By providing strong guarantees over se- curity, isolation and communication, Polkadot can allow parachains to select from a range of properties themselves. Indeed, we foresee various experimental blockchains push- ing the properties of what could be considered sensible today.

We see conservative, high-value chains similar to Bitcoin or Z-cash [19] co-existing alongside lower-value “theme-chains” (such marketing, so fun) and test-nets with zero or near-zero fees. We see fully-encrypted,

offloaded into middleware, placed through a parachain or introduced in a later optimisation.

General: no unnecessary requirement, constraint or limitation should be placed on parachains; Polkadot should be a test bed for consensus sys- tem development which can be optimised through making the model into which extensions fit as ab- stract as possible.

Robust: Polkadot should provide a fundamentally stable base-layer. In addition to economic sound- ness, this also means decentralising to minimise the vectors for high-reward attacks.

Participation in Polkadot

There are four basic roles in the upkeep of an Polkadot network: collator, fisherman, nominator and validator. In one possible implementation of Polkadot, the latter role may actually be broken down into two roles: basic valida- tor and availability guarantor; this is discussed in section 6.5.3.

“dark”, consortium chains operating alongside—and even providing services to—highly functional and open chains such as those like Ethereum. We see experimental new

Collator Fisherman

image

provides block

Nominator

VM-based chains such as a subjective time-charged Wasm chain being used as a means of outsourcing difficult com- pute problems from a more mature Ethereum-like chain or a more restricted Bitcoin-like chain.

To manage chain upgrades, Polkadot will inherently

candidates

for

monitors

reports bad

behaviour to

approves

support some sort of governance structure, likely based on existing stable political systems and having a bicam- eral aspect similar to the Yellow Paper Council [23]. As

Validators

(this group)

becomes

Validators

(other groups)

the ultimate authority, the underlying stakable token hold- ers would have “referendum” control. To reflect the users’ need for development but the developers’ need for legiti- macy, we expect a reasonable direction would be to form the two chambers from a “user” committee (made up of bonded validators) and a “technical” committee made up of major client developers and ecosystem players. The body of token holders would maintain the ultimate legit- imacy and form a supermajority to augment, reparam- eterise, replace or dissolve this structure, something we don’t doubt the eventual need for: in the words of Twain “Governments and diapers must be changed often, and for the same reason”.

Whereas reparameterisation is typically trivial to ar- range within a larger consensus mechanism, more qualita- tive changes such as replacement and augmentation would likely need to be either non-automated “soft-decrees” (e.g. through the canonicalisation of a block number and the hash of a document formally specifying the new protocol) or necessitate the core consensus mechanism to contain a sufficiently rich language to describe any aspect of itself which may need to change. The latter is an eventual aim, however, the former more likely to be chosen in order to facilitate a reasonable development timeline.

Polkadot’s primary tenets and the rules within which we evaluate all design decisions are:

Minimal: Polkadot should have as little functional- ity as possible.

Simple: no additional complexity should be present in the base protocol than can reasonably be

Figure 1. The interaction between the

four roles of Polkadot.

Validators. A validator is the highest charge and helps seal new blocks on the Polkadot network. The val- idator’s role is contingent upon a sufficiently high bond being deposited, though we allow other bonded parties to nominate one or more validators to act for them and as such some portion of the validator’s bond may not neces- sarily be owned by the validator itself but rather by these nominators.

A validator must run a relay-chain client implementa- tion with high availability and bandwidth. At each block the node must be ready to accept the role of ratifying a new block on a nominated parachain. This process involves receiving, validating and republishing candidate blocks. The nomination is deterministic but virtually un- predictable much in advance. Since the validator cannot reasonably be expected to maintain a fully-synchronised database of all parachains, it is expected that the valida- tor will nominate the task of devising a suggested new parachain block to a third-party, known as a collator.

Once all new parachain blocks have been properly rat- ified by their appointed validator subgroups, validators must then ratify the relay-chain block itself. This involves updating the state of the transaction queues (essentially moving data from a parachain’s output queue to another parachain’s input queue), processing the transactions of the ratified relay-chain transaction set and ratifying the final block, including the final parachain changes.

image

A validator not fulfilling their duty to find consensus under the rules of our chosen consensus algorithm is pun- ished. For initial, unintentional failures, this is through withholding the validator’s reward. Repeated failures re- sult in the reduction of their security bond (through burn- ing). Provably malicious actions such as double-signing or conspiring to provide an invalid block result in the loss of the entire bond (which is partially burnt but mostly given to the informant and the honest actors).

In some sense, validators are similar to the mining pools of current PoW blockchains.

Nominators. A nominator is a stake-holding party who contributes to the security bond of a validator. They have no additional role except to place risk capital and as such to signal that they trust a particular validator (or set thereof) to act responsibly in their maintenance of the network. They receive a pro-rata increase or reduction in their deposit according to the bond’s growth to which they contribute.

Together with collators, next, nominators are in some sense similar to the miners of the present-day PoW net- works.

Collators. Transaction collators (collators for short) are parties who assist validators in producing valid parachain blocks. They maintain a “full-node” for a par- ticular parachain; meaning that they retain all necessary information to be able to author new blocks and execute transactions in much the same way as miners do on cur- rent PoW blockchains. Under normal circumstances, they will collate and execute transactions to create an unsealed block, and provide it, together with a zero-knowledge proof, to one or more validators presently responsible for proposing a parachain block.

The precise nature of the relationship between colla- tors, nominators and validators will likely change over time. Initially, we expect collators to work very closely with validators, since there will be only a few (perhaps only one) parachain(s) with little transaction volume. The initial client implementation will include RPCs to allow a parachain collator node to unconditionally supply a (relay- chain) validator node with a provably valid parachain block. As the cost of maintaining a synced version of all such parachains increases, we expect to see additional infrastructure in place which will help separate out the duties to independent, economically-motivated, parties.

Eventually, we expect to see collator pools who vie to collect the most transaction fees. Such collators may be- come contracted to serve particular validators over a pe- riod of time for an on-going share in the reward proceeds. Alternatively, “freelance” collators may simply create a market offering valid parachain blocks in return for a com- petitive share of the reward payable immediately. Simi- larly, decentralised nominator pools would allow multiple bonded participants to coordinate and share the duty of a validator. This ability to pool ensures open participation leading to a more decentralised system.

Fishermen. Unlike the other two active parties, fishermen are not directly related to the block-authoring process. Rather they are independent “bounty hunters” motivated by a large one-off reward. Precisely due to

the existence of fishermen, we expect events of misbe- haviour to happen seldom, and when they do only due to the bonded party being careless with secret key security, rather than through malicious intent. The name comes from the expected frequency of reward, the minimal re- quirements to take part and the eventual reward size.

Fishermen get their reward through a timely proof that at least one bonded party acted illegally. Illegal actions include signing two blocks each with the same ratified par- ent or, in the case of parachains, helping ratify an invalid block. To prevent over-rewarding of the compromise and illicit use of a session’s secret key, the base reward for providing a single validator’s illegally signed message is minimal. This reward increases asymptotically as more corroborating illegal signatures from other validators are provided implying a genuine attack. The asymptote is set at 66% following our base security assertion that at least two-thirds of the validators act benevolently.

Fishermen are somewhat similar to “full nodes” in present-day blockchain systems that the resources needed are relatively small and the commitment of stable uptime and bandwidth is not necessary. Fishermen differ in so much as they must post a small bond. This bond prevents sybil attacks from wasting validators’ time and compute resources. It is immediately withdrawable, probably no more than the equivalent of a few dollars and may lead to reaping a hefty reward from spotting a misbehaving validator.

Design Overview

This section is intended to give a brief overview of the system as a whole. A more thorough exploration of the system is given in the section following it.

Consensus. On the relay-chain, Polkadot achieves low-level consensus over a set of mutually agreed valid blocks through a modern asynchronous Byzantine fault- tolerant (BFT) algorithm. The algorithm will be inspired by the simple Tendermint [11] and the substantially more involved HoneyBadgerBFT [13]. The latter provides an efficient and fault-tolerant consensus over an arbitrarily defective network infrastructure, given a set of mostly be- nign authorities or validators.

For a proof-of-authority (PoA) style network, this alone would be sufficient, however Polkadot is imagined to be also deployable as a network in a fully open and public situation without any particular organisation or trusted authority required to maintain it. As such we need a means of determining a set of validators and incentivising them to be honest. For this we utilise PoS based selection criteria.

Proving the Stake. We assume that the network will have some means of measuring how much “stake” any particular account has. For ease of comparison to pre-existing systems, we will call the unit of measurement “tokens”. Unfortunately the term is less than ideal for a number of reasons, not least that being simply a scalar value associated with an account, there is no notion of individuality.

We imagine validators be elected, infrequently (at most once per day but perhaps as seldom as once per quarter), through a Nominated Proof-of-Stake (NPoS ) scheme. In- centivisation can happen through a pro-rata allocation of

image

Collator Propagated block

Transaction

image

(submitted by external actor)

Propagated transactions Block candidate submission

Fisherman Parachain community

Account

Inbound transaction

Validator swarm

(each coloured by its designated parachain)

2nd order Relay-chain

Interchain transactions

(managed by validators)

Outbound transaction

Relay chain

Parachain Parachain

Parachain bridge

queues and I/O

Virtual parachain (e.g. Ethereum)

Figure 2. A summary schematic of the Polkadot system. This shows collators collecting and propa- gating user-transactions, as well as propagating block candidates to fishermen and validators. It also shows how an account can post a transaction which is carried out of its parachain, via the relay-chain and on into another parachain where it can be interpreted as a transaction to an account there.

funds coming from a token base expansion (up to 100% per year, though more likely around 10%) together with any transaction fees collected. While monetary base ex- pansion typically leads to inflation, since all token owners would have a fair opportunity at participation, no token- holder would need to suffer a reduction in value of their holdings over time provided they were happy to take a role in the consensus mechanism. A particular proportion of tokens would be targeted for the staking process; the effective token base expansion would be adjusted through a market-based mechanism to reach this target.

Validators are bonded heavily by their stakes; exiting validators’ bonds remain in place long after the valida- tors’ duties cease (perhaps around 3 months). This long bond-liquidation period allows future misbehaviour to be punished up until the periodic checkpointing of the chain. Misbehaviour results in punishment, such as reduction of reward or, in cases which intentionally compromise the network’s integrity, the validator losing some or all of its stake to other validators, informants or the stakeholders as a whole (through burning). For example, a validator who attempts to ratify both branches of a fork (sometimes known as a “short-range” attack) may be identified and punished in the latter way.

Long-range “nothing-at-stake” attacks4 are circum- vented through a simple “checkpoint” latch which pre- vents a dangerous chain-reorganisation of more than a particular chain-depth. To ensure newly-syncing clients are not able to be fooled onto the wrong chain, regular “hard forks” will occur (of at most the same period of the validators’ bond liquidation) that hard-code recent check- point block hashes into clients. This plays well with a fur- ther footprint-reducing measure of “finite chain length” or periodic reseting of the genesis-block.

Parachains and Collators. Each parachain gets similar security affordances to the relay-chain: the parachains’ headers are sealed within the relay-chain block ensuring no reorganisation, or “double-spending”, is possi- ble following confirmation. This is a similar security guar- antee to that offered by Bitcoin’s side-chains and merge- mining. Polkadot, however, also provides strong guaran- tees that the parachains’ state transitions are valid. This happens through the set of validators being cryptograph- ically randomly segmented into subsets; one subset per parachain, the subsets potentially differing per block. This setup generally implies that parachains’ block times will be at least as long as that of the relay-chain. The specific means of determining the partitioning is outside the scope

image

4Such an attack is where the adversary forges an entirely new chain of history from the genesis block onwards. Through controlling a relatively insignificant portion of stake at the offset, they are able to incrementally increase their portion of the stake relative to all other stakeholders as they are the only active participants in their alternative history. Since no intrinsic physical limitation exists on the creation of blocks (unlike PoW where quite real computational energy must be spent), they are able to craft a chain longer than the real chain in a relatively short timespan and potentially make it the longest and best, taking over the canonical state of the network.

image

of this document but would likely be based either around a commit-reveal framework similar to the RanDAO [18] or use data combined from previous blocks of each parachain under a cryptographically secure hash.

Such subsets of validators are required to provide a parachain block candidate which is guaranteed valid (on pain of bond confiscation). Validity revolves around two

Account sends post: entry placed in egress Merkle tree

for destination parachain

Source: shares egress data with next block’s

Account receives post: entry removed from ingress Merkle tree

Destination: gets data from prior

important points; firstly that it is intrinsically valid—that all state transitions were executed faithfully and that all external data referenced (i.e. transactions) is valid for in-

validators

ingress

block’s validators.

clusion. Secondly, that any data which is extrinsic to its candidate, such as those external transactions, has suffi- ciently high availability so that participants are able to download it and execute the block manually.5 Valida-

proof-of-post stored in parachain egress Merkle tree

routed reference placed in destination parachain’s ingress Merkle tree

tors may provide only a “null” block containing no ex- ternal “transactions” data, but may run the risk of get- ting a reduced reward if they do. They work alongside a parachain gossip protocol with collators—individuals who collate transactions into blocks and provide a non- interactive, zero-knowledge proof that the block consti- tutes a valid child of its parent (and taking any transaction fees for their trouble).

It is left to parachain protocols to specify their own means of spam-prevention: there is no fundamental no- tion of “compute-resource metering” or “transaction fee” imposed by the relay-chain. There is also no direct en- forcement on this by the relay-chain protocol (though it is unlikely that the stakeholders would choose to adopt a parachain which didn’t provide a decent mechanism). This is an explicit nod to the possibility of chains unlike Ethereum, e.g. a Bitcoin-like chain which has a much sim- pler fee model or some other, yet-to-be-proposed spam- prevention model.

Polkadot’s relay-chain itself will probably exist as an Ethereum-like accounts and state chain, possibly an EVM- derivative. Since the relay-chain nodes will be required to do substantial other processing, transaction throughput will be minimised partly through large transaction fees and, should our research models require, a block size limit.

Interchain Communication. The critical final in- gredient of Polkadot is interchain communication. Since parachains can have some sort of information channel be- tween them, we allow ourselves to consider Polkadot a scalable multi-chain. In the case of Polkadot, the commu- nication is as simple as can be: transactions executing in a parachain are (according to the logic of that chain) able to effect the dispatch of a transaction into a second parachain or, potentially, the relay-chain. Like external transactions on production blockchains, they are fully asynchronous and there is no intrinsic ability for them to return any kind of information back to its origin.

Figure 3. A basic schematic showing the main parts of routing for posted transactions (”posts”).

To ensure minimal implementation complexity, min- imal risk and minimal straight-jacketing of future parachain architectures, these interchain transactions are effectively indistinguishable from standard externally- signed transactions. The transaction has an origin seg- ment, providing the ability to identify a parachain, and an address which may be of arbitrary size. Unlike com- mon current systems such as Bitcoin and Ethereum, in- terchain transactions do not come with any kind of “pay- ment” of fee associated; any such payment must be man- aged through negotiation logic on the source and desti- nation parachains. A system such as that proposed for Ethereum’s Serenity release [7] would be a simple means of managing such a cross-chain resource payment, though we assume others may come to the fore in due course.

Interchain transactions are resolved using a simple queuing mechanism based around a Merkle tree to ensure fidelity. It is the task of the relay-chain maintainers to move transactions on the output queue of one parachain into the input queue of the destination parachain. The passed transactions get referenced on the relay-chain, how- ever are not relay-chain transactions themselves. To pre- vent a parachain from spamming another parachain with transactions, for a transaction to be sent, it is required that the destination’s input queue be not too large at the time of the end of the previous block. If the input queue is too large after block processing, then it is con- sidered “saturated” and no transactions may be routed to it within subsequent blocks until reduced back below the limit. These queues are administered on the relay-chain allowing parachains to determine each other’s saturation status; this way a failed attempt to post a transaction to a stalled destination may be reported synchronously. (Though since no return path exists, if a secondary trans- action failed for that reason, it could not be reported back to the original caller and some other means of recovery would have to take place.)

Polkadot and Ethereum. Due to Ethereum’s Tur- ing completeness, we expect there is ample opportu- nity for Polkadot and Ethereum to be interoperable with each other, at least within some easily deducible secu- rity bounds. In short, we envision that transactions from Polkadot can be signed by validators and then fed into

5Such a task might be shared between validators or could become the designate task of a set of heavily bonded validators known as

availability guarantors.

image

Ethereum where they can be interpreted and enacted by a transaction-forwarding contract. In the other direction, we foresee the usage of specially formatted logs coming from a “break-out contract” to allow a swift verification that a particular message should be forwarded.

Polkadot to Ethereum. Through the choice of a BFT consensus mechanism with validators formed from a set of stakeholders determined through an approval voting mechanism, we are able to get a secure consensus with an infrequently changing and modest number of validators. In a system with a total of 144 validators, a block time of 4 seconds and a 900-block finality (allowing for malicious behaviour such as double-votes to be reported, punished and repaired), the validity of a block can reasonably be considered proven through as little as 97 signatures (two- thirds of 144 plus one) and a following 60-minute verifica- tion period where no challenges are deposited.

Ethereum is able to host a “break-in contract” which can maintain the 144 signatories and be controlled by them. Since elliptic curve digital signature (ECDSA) re- covery takes only 3,000 gas under the EVM, and since we would likely only want the validation to happen on a super-majority of validators (rather than full unanimity), the base cost of Ethereum confirming that an instruction was properly validated as coming from the Polkadot net- work would be no more than 300,000 gas—a mere 6% of the total block gas limit at 5.5M. Increasing the num- ber of validators (as would be necessary for dealing with dozens of chains) inevitably increases this cost, however it is broadly expected for Ethereum’s transaction band- width to grow over time as the technology matures and infrastructure improves. Together with the fact that not all validators need to be involved (e.g. only the highest staked validators may be called upon for such a task) the limits of this mechanism extend reasonably well.

Assuming a daily rotation of such validators (which is fairly conservative—weekly or even monthly may be ac- ceptable), then the cost to the network of maintaining this Ethereum-forwarding bridge would be around 540,000 gas per day or, at present gas prices, $45 per year. A ba- sic transaction forwarded alone over the bridge would cost around $0.11; additional contract computation would cost more, of course. By buffering and bundling transactions together, the break-in authorisation costs can easily be shared, reducing the cost per transaction substantially; if 20 transactions were required before forwarding, then the cost for forwarding a basic transaction would fall to around $0.01.

One interesting, and cheaper, alternative to this multi- signature contract model would be to use threshold sig- natures in order to achieve the multi-lateral ownership se- mantics. While threshold signature schemes for ECDSA are computationally expensive, those for other schemes such as Schnorr signatures are very reasonable. Ethereum plans to introduce primitives which would make such schemes cheap to use in the upcoming Metropolis hard- fork. If such a means were able to be utilised, the gas costs for forwarding a Polkadot transaction into the Ethereum network would be dramatically reduced to a near zero overhead over and above the basic costs for validating the signature and executing the underlying transaction.

In this model, Polkadot’s validator nodes would have to do little other than sign messages. To get the trans- actions actually routed onto the Ethereum network, we assume either validators themselves would also reside on the Ethereum network or, more likely, that small bounties be offered to the first actor who forwards the message on to the network (the bounty could trivially be paid to the transaction originator).

Ethereum to Polkadot. Getting transactions to be forwarded from Ethereum to Polkadot uses the simple no- tion of logs. When an Ethereum contract wishes to dis- patch a transaction to a particular parachain of Polkadot, it need simply call into a special “break-out contract”. The break-out contract would take any payment that may be required and issue a logging instruction so that its ex- istence may be proven through a Merkle proof and an as- sertion that the corresponding block’s header is valid and canonical.

Of the latter two conditions, validity is perhaps the most straightforward to prove. In principle, the only re- quirement is for each Polkadot node needing the proof (i.e. appointed validator nodes) to be running a fully syn- chronised instance of a standard Ethereum node. Unfor- tunately, this is itself a rather heavy dependency. A more lightweight method would be to use a simple proof that the header was evaluated correctly through supplying only the part of Ethereum’s state trie needed to properly execute the transactions in the block and check that the logs (con- tained in the block receipt) are valid. Such “SPV-like”6 proofs may yet require a substantial amount of informa- tion; conveniently, they would typically not be needed at all: a bond system inside Polkadot would allow bonded third-parties to submit headers at the risk of losing their bond should some other third-parity (sometimes referred to as a “fisherman”) provide a proof that the header is in- valid (specifically that the state root or receipt roots were impostors).

On a non-finalising PoW network like Ethereum, the canonicality is impossible to proof conclusively. To ad- dress this, applications that attempt to rely on any kind of chain-dependent cause-effect wait for a number of “con- firmations”, or until the dependent transaction is at some particular depth within the chain. On Ethereum, this depth varies from 1 block for the least valuable transac- tions with no known network issues to 1200 blocks as was the case during the initial Frontier release for exchanges. On the stable “Homestead” network, this figure sits at 120 blocks for most exchanges, and we would likely take a similar parameter.

So we can imagine our Polkadot-side Ethereum- interface to have some simple functions: to be able to accept a new header from the Ethereum network and val- idate the PoW, to be able to accept some proof that a particular log was emitted by the Ethereum-side break- out contract for a header of sufficient depth (and forward the corresponding message within Polkadot) and finally to be able to accept proofs that a previously accepted but not-yet-enacted header contains an invalid receipt root.

To actually get the Ethereum header data itself (and any SPV proofs or validity/canonicality refutations) into the Polkadot network, an incentivisation for forwarding

6SPV refers to Simplified Payment Verification in Bitcoin and describes a method for clients to verify transactions while keeping only a copy of all blocks headers of the longest PoW chain.

image

data is needed. This could be as simple as a payment (funded from fees collected on the Ethereum side) paid to anyone able to forward a useful block whose header is valid. Validators would be called upon to retain informa- tion relating to the last few thousand blocks in order to be able to manage forks, either through some protocol- intrinsic means or through a contract maintained on the relay chain.

5.6. Polkadot and Bitcoin. Bitcoin interoperation presents an interesting challenge for Polkadot: a so-called “two-way peg” would be a useful piece of infrastructure to have on the side of both networks. However, due to the limitations of Bitcoin, providing such a peg securely is a non-trivial undertaking. Delivering a transaction from Bitcoin to Polkadot can in principle be done with a pro- cess similar to that for Ethereum; a “break-out address” controlled in some way by the Polkadot validators could receive transferred tokens (and data sent alongside them). SPV proofs could be provided by incentivised oracles and, together with a confirmation period, a bounty given for identifying non-canonical blocks implying the transaction has been “double-spent”. Any tokens then owned in the “break-out address” would then, in principle, be con- trolled by those same validators for later dispersal.

The problem however is how the deposits can be se- curely controlled from a rotating validator set. Unlike Ethereum which is able to make arbitrary decisions based upon combinations of signatures, Bitcoin is substantially more limited, with most clients accepting only multi- signature transactions with a maximum of 3 parties. Ex- tending this to 36, or indeed thousands as might ulti- mately be desired, is impossible under the current proto- col. One option is to alter the Bitcoin protocol to enable such functionality, however so-called “hard forks” in the Bitcoin world are difficult to arrange to say the least. One possibility is the use of threshold signatures, cryptographic schemes to allow a singly identifiable public key to be effec- tively controlled by multiple secret “parts”, some or all of which must be utilised to create a valid signature. Unfor- tunately, threshold signatures compatible with Bitcoin’s ECDSA are computationally expensive to create and of polynomial complexity. Other schemes such a Schnorr sig- natures provide far lower costs, however the timeline on which they may be introduced into the Bitcoin protocol is uncertain.

Since the ultimate security of the deposits rests with a number of bonded validators, one other option is to reduce the multi-signature key-holders to only a heavily bonded subset of the total validators such that threshold signatures become feasible (or, at worst, Bitcoin’s native multi-signature is possible). This of course reduces the total amount of bonds that could be deducted in repara- tions should the validators behave illegally, however this is a graceful degradation, simply setting an upper limit of the amount of funds that can securely run between the two networks (or indeed, on the % losses should an attack from the validators succeed).

As such we believe it not unrealistic to place a reason- ably secure Bitcoin interoperability “virtual parachain”

between the two networks, though nonetheless a substan- tial effort with an uncertain timeline and quite possibly requiring the cooperation of the stakeholders within that network.

Protocol in Depth

The protocol can be roughly broken down into three parts: the consensus mechanism, the parachain interface and interchain transaction routing.

Relay-chain Operation. The relay-chain will likely be a chain broadly similar to Ethereum in that it is state-based with the state mapping address to account information, mainly balances and (to prevent replays) a transaction counter. Placing accounts here fulfils one pur- pose: to provide accounting for which identity possesses what amount of stake in the system.7 There will be no- table differences, though:

Contracts cannot be deployed through transac- tions; following from the desire to avoid applica- tion functionality on the relay-chain, it will not support public deployment of contracts.

Compute resource usage (“gas”) is not accounted; since the only functions available for public usage will be fixed, the rationale behind gas accounting no longer holds. As such, a flat fee will apply in all cases, allowing for more performance from any dynamic code execution that may need to be done and a simpler transaction format.

Special functionality is supported for listed con- tracts that allows for auto-execution and network- message outputs.

In the event that the relay-chain has a VM and it be based around the EVM, it would have a number of mod- ifications to ensure maximal simplicity. It would likely have a number of built-in contracts (similar to those at addresses 1-4 in Ethereum) to allow for platform-specific duties to be managed including a consensus contract, a validator contract and a parachain contract.

If not the EVM, then a Web-Assembly (Wasm) back- end is the most likely alternative; in this case the overall structure would be similar, but there would be no need for the built-in contracts with Wasm being a viable target for general purpose languages rather than the immature and limited languages for the EVM.

Other likely deviations from the present Ethereum pro- tocol are quite possible, for example a simplification of the transaction-receipt format allowing for the parallel execu- tion of non-conflicting transactions within the same block, as proposed for the Serenity series of changes.

It is possible, though unlikely, that a Serenity-like “pure” chain be deployed as the relay-chain, allowing for a particular contract to manage things like the staking token balances rather than making that a fundamental part of the chain’s protocol. At present, we feel it is unlikely this will offer a sufficiently great protocol simplification to be worth the additional complexity and uncertainty involved in developing it.

7As a means of representing the amount a given holder is responsible for the overall security of the system, these stake accounts will inevitably encode some economic value. However, it should be understood that since there is no intention that such values be used in any way for the purpose of exchanging for real-world goods and services, it should be accordingly noted that the tokens not be likened to currency and as such the relay-chain retain its nihilistic philosophy regarding applications.

image

There are a number of small pieces of functionality re- quired for administrating the consensus mechanism, val- idator set, validation mechanism and parachains. These could be implemented together under a monolithic proto- col. However, for reasons of auguring modularity, we de- scribe these as “contracts” of the relay-chain. This should be taken to mean that they are objects (in the sense of object-orientated programming) managed by the relay- chain’s consensus mechanism, but not necessarily that they are defined as programs in EVM-like opcodes, nor even that they be individually addressable through the account-system.

Staking Contract. This contract maintains the val- idator set. It manages:

which accounts are currently validators;

which are available to become validators at short notice;

which accounts have placed stake nominating to a validator;

properties of each including staking volume, ac- ceptable payout-rates and addresses and short- term (session) identities.

It allows an account to register a desire to become a bonded validator (along with its requirements), to nom- inate to some identity, and for preexisting bonded val- idators to register their desire to exit this status. It also includes the machinery itself for the validation and canon- icalisation mechanism.

Stake-token Liquidity. It is generally desirable to have as much of the total staking tokens as possible to be staked within the network maintenance operations since this directly ties the network security to the overall “mar- ket capitalisation” of the staking token. This can easily be incentivised through inflating the currency and hand- ing out the proceeds to those who participate as valida- tors. However, to do so presents a problem: if the token is locked in the Staking Contract under punishment of re- duction, how can a substantial portion remain sufficiently liquid in order to allow price discovery?

One answer to this is allowing a straight-forward de- rivative contract, securing fungible tokens on an underly- ing staked token. This is difficult to arrange in a trust- free manner. Furthermore, these derivative tokens can- not be treated equally for the same reason that differ- ent Eurozone government’s bonds are not fungible: there is a chance of the underlying asset failing and becoming worthless. With Eurozone governments, there could be a default. With validator-staked tokens, the validator may act maliciously and be punished.

Keeping with our tenets, we elect for the simplest so- lution: not all tokens be staked. This would mean that some proportion (perhaps 20%) of tokens will forcibly re- main liquid. Though this is imperfect from a security per- spective, it is unlikely to make a fundamental difference in the security of the network; 80% of the reparations possi- ble from bond-confiscations would still be able to be made compared to the “perfect case” of 100% staking.

The ratio between staked and liquid tokens can be tar- geted fairly simply through a reverse auction mechanism. Essentially, token holders interested in being a validator would each post an offer to the staking contract stating the minimum payout-rate that they would require to take

part. At the beginning of each session (sessions would happen regularly, perhaps as often as once per hour) the validator slots would be filled according to each would-be validator’s stake and payout rate. One possible algorithm for this would be to take those with the lowest offers who represent a stake no higher than the total stake targeted divided by the number of slots and no lower than a lower- bound of half that amount. If the slots cannot be filled, the lower bound could be repeatedly reduced by some fac- tor in order to satisfy.

Nominating. It is possible to trustlessly nominate ones staking tokens to an active validator, giving them the responsibility of validators duties. Nominating works through an approval-voting system. Each would-be nomi- nator is able to post an instruction to the staking contract expressing one or more validator identities under whose responsibility they are prepared to entrust their bond.

Each session, nominators’ bonds are dispersed to be represented by one or more validators. The dispersal al- gorithm optimises for a set of validators of equivalent total bonds. Nominators’ bonds become under the effective re- sponsibility of the validator and gain interest or suffer a punishment-reduction accordingly.

Bond Confiscation/Burning. Certain validator be- haviour results in a punitive reduction of their bond. If the bond is reduced below the allowable minimum, the session is prematurely ended and another started. A non- exhaustive list of punishable validator misbehaviour in- cludes:

  • Being part of a parachain group unable to provide consensus over the validity of a parachain block;
  • actively signing for the validity of an invalid parachain block;
  • inability to supply egress payloads previously voted as available;
  • inactivity during the consensus process;
  • validating relay-chain blocks on competing forks. Some cases of misbehaviour threaten the network’s in- tegrity (such as signing invalid parachain blocks and val-

idating multiple sides of a fork) and as such result in ef-

fective exile through the total reduction of the bond. In other, less serious cases (e.g. inactivity in the consensus process) or cases where blame cannot be precisely allot- ted (being part of an ineffective group), a small portion of the bond may instead be fined. In the latter case, this works well with sub-group churn to ensure that malicious nodes suffer substantially more loss than the collaterally- damaged benevolent nodes.

In some cases (e.g. multi-fork validation and invalid sub-block signing) validators cannot themselves easily de- tect each others’ misbehaviour since constant verification of each parachain block would be too arduous a task. Here it is necessary to enlist the support of parties external to the validation process to verify and report such misbe- haviour. The parties get a reward for reporting such ac- tivity; their term, “fishermen” stems from the unlikeliness of such a reward.

Since these cases are typically very serious, we envi- sion that any rewards can easily be paid from the con- fiscated bond. In general we prefer to balance burning (i.e. reduction to nothing) with reallocation, rather than attempting wholesale reallocation. This has the effect of

image

increasing the overall value of the token, compensating the network in general to some degree rather than the specific party involved in discovery. This is mainly as a safety mechanism: the large amounts involved could lead to ex- treme and acute behaviour incentivisation were they all bestowed on a single target.

In general, it is important that the reward is suffi- ciently large to make verification worthwhile for the net- work, yet not so large as to offset the costs of fronting a well-financed, well-orchestrated ”industrial-level” criminal hacking attack on some unlucky validator to force misbe- haviour.

In this way, the amount claimed should generally be no greater than the direct bond of the errant validator, lest a perverse incentive arise of misbehaving and reporting one- self for the bounty. This can be combated either explicitly through a minimum direct bond requirement for being a validator or implicitly by educating nominators that val- idators with little bonds deposited have no great incentive to behave well.

  1. Parachain Registry. This is the component of the system which records what each of the parachains are. It is a relatively simple database-like construct and holds both static and dynamic information on each chain.Static information includes the chain index (a simple integer), along with the validation protocol identity, a means of distinguishing between the different classes of parachain so that the correct validation algorithm can be run by validators consigned to putting forward a valid can- didate. An initial proof-of-concept would focus on placing the new validation algorithms into clients themselves, ef- fectively requiring a hard fork of the protocol each time an additional class of chain were added. Ultimately, though, it may be possible to specify the validation algorithm in a way both rigorous and efficient enough that clients are able to effectively work with new parachains without a hard-fork. One possible avenue to this would be to specify the parachain validation algorithm in a well-established, natively-compiled, platform-neutral language such as We- bAssembly [2]. Additional research is necessary to deter- mine whether this is truly feasible, however if so, it could bring with it the tremendous advantage of banishing hard- forks for good.Dynamic information includes aspects of the transac- tion routing system that must have global agreement such as the parachain’s ingress queue (described in a later sec- tion).The registry is able to have parachains added only through full referendum voting; this could be managed internally but would more likely be placed in an external referendum contract in order to facilitate re-usage under more general governance components. The parameters to voting requirements (e.g. any quorum required, majority required) for registration of additional chains and other, less formal system upgrades will be set out in a “master constitution” but are likely to follow a fairly traditional path, at least initially. The precise formulation is out of scope for the present work, but e.g. a two thirds super- majority to pass with more than one third of total system stake voting positively may be a sensible starting point.Additional operations include the suspension and re- moval of parachains. Suspension would hopefully never happen, however it is designed to be a safeguard least there be some intractable problem in a parachain’s vali- dation system. The most obvious instance where it might be needed is a consensus-critical difference between im- plementations leading validators to be unable to agree on validity or blocks. Validators would be encouraged to use multiple client implementations in order that they are able to spot such a problem prior to bond confiscation.Since suspension is an emergency measure, it would be under the auspices of the dynamic validator-voting rather than a referendum. Re-instating would be possible both from the validators or a referendum.The removal of parachains altogether would come only after a referendum and with which would be required a substantial grace period to allow an orderly transition to either a standalone chain or to become part of some other consensus-system. The grace period would likely be of the order of months and is likely to be set out on a per- chain basis in the parachain registry in order that different parachains can enjoy different grace periods according to their need.
  2. Sealing Relay Blocks. Sealing refers, in essence, to the process of canonicalisation; that is, a basic data transform which maps the original into something funda- mentally singular and meaningful. Under a PoW chain, sealing is effectively a synonym for mining. In our case, it involves the collection of signed statements from val- idators over the validity, availability and canonicality of a particular relay-chain block and the parachain blocks that it represents.The mechanics of the underlying BFT consensus al- gorithm is out of scope for the present work. We will instead describe it using a primitive which assumes a consensus-creating state-machine. Ultimately we expect to be inspired by a number of promising BFT consensus algorithms in the core; Tangaora [9] (a BFT variant of Raft [15]), Tendermint [11] and HoneyBadgerBFT [13]. The algorithm will have to reach an agreement on mul- tiple parachains in parallel, thus differing from the usual blockchain consensus mechanisms. We assume that once consensus is reached, we are able to record the consensus in an irrefutable proof which can be provided by any of the participants to it. We also assume that misbehaviour within the protocol can be generally reduced to a small group containing misbehaving participants to minimise the collateral damage when dealing out punishment.8The proof, which takes the form of our signed state- ments, is placed in the relay-chain block’s header together with certain other fields not least the relay-chain’s state- trie root and transaction-trie root.The sealing process takes place under a single consensus-generating mechanism addressing both the relay-chain’s block and the parachains’ blocks which make up part of the relay’s content: parachains are not sepa- rately “committed” by their sub-groups and then collated later. This results in a more complex process for the relay- chain, but allows us to complete the entire system’s con- sensus in a single stage, minimising latency and allowing8Existing PoS-based BFT consensus schemes such as Tendermint BFT and the original Slasher fulfill these assertions.

    image

    for quite complex data-availability requirements which are helpful for the routing process below.The state of each participant’s consensus machine may be modelled as a simple (2-dimensional) table. Each par- ticipant (validator) has a set of information, in the form of signed-statements (“votes”) from other participants, re- garding each parachain block candidate as well the relay- chain block candidate. The set of information is two pieces of data:Availability: does this validator have egress transaction-post information from this block so they are able to properly validate parachain can- didates on the following block? They may vote either 1(known) or 0 (not yet known). Once they vote 1, they are committed to voting similarly for the rest of this process. Later votes that do not respect this are grounds for punishment.Validity: is the parachain block valid and is all externally-referenced data (e.g. transactions) available? This is only relevant for validators as- signed to the parachain on which they are voting. They may vote either 1 (valid), -1 (invalid) or 0 (not yet known). Once they vote non-zero, they are committed to voting this way for the rest of the process. Later votes that do not respect this are grounds for punishment.All validators must submit votes; votes may be resub- mitted, qualified by the rules above. The progression of consensus may be modelled as multiple standard BFT con- sensus algorithms over each parachain happening in par- allel. Since these are potentially thwarted by a relatively small minority of malicious actors being concentrated in a single parachain group, the overall consensus exists to establish a backstop, limiting the worst-case scenario from deadlock to merely one or more void parachain blocks (and a round of punishment for those responsible).The basic rules for validity of the individual blocks (that allow the total set of validators as a whole to come to consensus on it becoming the unique parachain candidate to be referenced from the canonical relay):

    • must have at least two thirds of its validators vot- ing positively and none voting negatively;

    • must have over one third validators voting posi- tively to the availability of egress queue informa- tion.

If there is at least one positive and at least one nega- tive vote on validity, an exceptional condition is created and the whole set of validators must vote to determine if there are malicious parties or if there is an accidental fork. Aside from valid and invalid, a third kind of votes are allowed, equivalent to voting for both, meaning that the node has conflicting opinions. This could be due to the node’s owner running multiple implementations which do not agree, indicating a possible ambiguity in the protocol. After all votes are counted from the full validator set, if the losing opinion has at least some small proportion (to be parameterised; at most half, perhaps significantly less) of the votes of the winning opinion, then it is assumed to be an accidental parachain fork and the parachain is au- tomatically suspended from the consensus process. Oth- erwise, we assume it is a malicious act and punish the minority who were voting for the dissenting opinion.

The conclusion is a set of signatures demonstrating canonicality. The relay-chain block may then be sealed and the process of sealing the next block begun.

  1. Improvements for Sealing Relay Blocks. While this sealing method gives strong guarantees over the sys- tem’s operation, it does not scale out particularly well since every parachain’s key information must have its availability guaranteed by over one-third of all validators. This means that every validator’s responsibility footprint grows as more chains are added.

While data availability within open consensus networks is essentially an unsolved problem, there are ways of miti- gating the overhead placed on validator nodes. One simple solution is to realise that while validators must shoulder the responsibility for data availability, they need not actu- ally store, communicate or replicate the data themselves. Secondary data silos, possibly related to (or even the very same) collators who compile this data, may manage the task of guaranteeing availability with the validators pro- viding a portion of their interest/income in payment.

However, while this might buy some intermediate scal- ability, it still doesn’t help the underlying problem; since adding more chains will in general require additional val- idators, the ongoing network resource consumption (par- ticularly in terms of bandwidth) grows with the square of the chains, an untenable property in the long-term.

Ultimately, we are likely to keep bashing our heads against the fundamental limitation which states that for a consensus network to be considered available safe, the ongoing bandwidth requirements are of the order of total validators times total input information. This is due to the inability of an untrusted network to properly distrib- ute the task of data storage across many nodes, which sits apart from the eminently distributable task of processing.

  1. Introducing Latency. One means of softening this rule is to relax the notion of immediacy. By requir- ing 33%+1 validators voting for availability only eventu- ally, and not immediately, we can better utilise exponen- tial data propagation and help even out peaks in data- interchange. A reasonable equality (though unproven) may be
    1. latency = participants × chains.

      Under the current model, the size of the system scales with the number of chains to ensure that processing is distributed; since each chain will require at least one val- idator and we fix the availability attestation to a constant proportion of validators, then participants similarly grows with the number of chains. We end up with:

    2. latency = size

      1. Meaning that as the system grows, the bandwidth re- quired and latency until availability is known across the network, which might also be characterised as the number of blocks before finality, increases with its square. This is a substantial growth factor and may turn out to be a no- table road blocker and force us into “non-flat” paradigms such as composing several “Polkadotes” into a hierarchy for multi-level routing of posts through a tree of relay- chains. image
      2. Public Participation. One more possible direction is to enlist public participation in the process through a micro-complaints system. Similar to the fishermen, there could be external parties to police the validators who claim availability. Their task is to find one who appears un- able to demonstrate such availability. In doing so they can lodge a micro-complaint to other validators. PoW or a staked bond may be used to mitigate the sybil attack which would render the system largely useless.

      3. Availability Guarantors. A final route would be to nominate a second set of bonded validators as “availability guarantors”. These would be bonded just as with the nor- mal validators, and may even be taken from the same set (though if so, they would be chosen over a long-term pe- riod, at least per session). Unlike normal validators, they would not switch between parachains but rather would form a single group to attest to the availability of all im- portant interchain data. 

        This has the advantage of relaxing the equivalence be- tween participants and chains. Essentially, chains can grow (along with the original chain validator set), whereas the participants, and specifically those taking part in data- availability testament, can remain at the least sub-linear and quite possibly constant.

      4. Collator Preferences. One important aspect of this system is to ensure that there is a healthy selection of collators creating the blocks in any given parachain. If a single collator dominated a parachain then some attacks become more feasible since the likelihood of the lack of availability of external data would be less obvious.

        One option is to artificially weight parachain blocks in a pseudo-random mechanism in order to favour a wide va- riety of collators. In the first instance, we would require as part of the consensus mechanism that validators favour parachain block candidates determined to be “heavier”. Similarly, we must incentivise validators to attempt to suggest the weightiest block they can find—this could be done through making a portion of their reward propor- tional to the weight of their candidate.To ensure that collators are given a reasonable fair chance of their candidate being chosen as the winning candidate in consensus, we make the specific weight of a parachain block candidate determinate on a random func- tion connected with each collator. For example, taking the XOR distance measure between the collator’s address and some cryptographically-secure pseudorandom number determined close to the point of the block being created (a notional “winning ticket”). This effectively gives each collator (or, more specifically, each collator’s address) a random chance of their candidate block “winning” over all others.To mitigate the sybil attack of a single collator “min- ing” an address close to the winning ticket and thus being a favourite each block, we would add some inertia to a col- lator’s address. This may be as simple as requiring them to have a baseline amount of funds in the address. A more elegant approach would be to weight the proximity to the winning ticket with the amount of funds parked at the address in question. While modelling has yet to be done, it is quite possible that this mechanism enables even very small stakeholders to contribute as a collator.

 

  1. Overweight Blocks. If a validator set is compro- mised, they may create and propose a block which though valid, takes an inordinate amount of time to execute and validate. This is a problem since a validator group could reasonably form a block which takes a very long time to execute unless some particular piece of information is al- ready known allowing a short cut, e.g. factoring a large prime. If a single collator knew that information, then they would have a clear advantage in getting their own candidates accepted as long as the others were busy pro- cessing the old block. We call these blocks overweight.Protection against validators submitting and validat- ing these blocks largely falls under the same guise as for invalid blocks, though with an additional caveat: Since the time taken to execute a block (and thus its status as overweight) is subjective, the final outcome of a vote on misbehaviour will fall into essentially three camps. One possibility is that the block is definitely not overweight— in this case more than two-thirds declare that they could execute the block within some limit (e.g. 50% of the to- tal time allowed between blocks). Another is that the block is definitely overweight—this would be if more than two-thirds declare that they could not execute the block within said limit. One final possibility is a fairly equal split of opinion between validators. In this case, we may choose to do some proportionate punishment. To ensure validators can predict when they may be proposing an overweight block, it may be sensible to re- quire them to publish information on their own perfor- mance for each block. Over a sufficient period of time, this should allow them to profile their processing speed relative to the peers that would be judging them.
  1. Collator Insurance. One issue remains for valida- tors: unlike with PoW networks, to check a collator’s block for validity, they must actually execute the trans- actions in it. Malicious collators can feed invalid or over- weight blocks to validators causing them grief (wasting their resources) and exacting a potentially substantial op- portunity cost.

To mitigate this, we propose a simple strategy on the part of validators. Firstly, parachain block candidates sent to validators must be signed from a relay chain account with funds; if they are not, then the validator should drop it immediately. Secondly, such candidates should be or- dered in priority by a combination (e.g. multiplication) of the amount of funds in the account up to some cap, the number of previous blocks that the collator has success- fully proposed in the past (not to mention any previous punishments), and the proximity factor to the winning ticket as discussed previously. The cap should be the same as the punitive damages paid to the validator in the case of them sending an invalid block.

To disincentivise collators from sending invalid or over- weight block candidates to validators, any validator may place in the next block a transaction including the offend- ing block alleging misbehaviour with the effect of transfer- ring some or all of the funds in the misbehaving collator’s account to the aggrieved validator. This type of trans- action front-runs any others to ensure the collator cannot remove the funds prior to the punishment. The amount of funds transferred as damages is a dynamic parameter yet

image

to be modelled but will likely be a proportion of the val- idator block reward to reflect the level of grief caused. To prevent malicious validators arbitrarily confiscating colla- tors’ funds, the collator may appeal the validator’s deci- sion with a jury of randomly chosen validators in return for placing a small deposit. If they find in the valida- tor’s favour, the deposit is consumed by them. If not, the deposit is returned and the validator is fined (since the validator is in a much more vaulted position, the fine will likely be rather hefty).

    1. Interchain Transaction Routing. Interchain transaction routing is one of the essential maintenance tasks of the relay-chain and its validators. This is the logic which governs how a posted transaction (often short- ened to simply “post ”) gets from being a desired output from one source parachain to being a non-negotiable in- put of another destination parachain without any trust requirements.

      We choose the wording above carefully; notably we don’t require there to have been a transaction in the source parachain to have explicitly sanctioned this post. The only constraints we place upon our model is that parachains must provide, packaged as a part of their overall block processing output, the posts which are the result of the block’s execution.

      These posts are structured as several FIFO queues; the number of lists is known as the routing base and may be around 16. Notably, this number represents the quantity of parachains we can support without having to resort to multi-phase routing. Initially, Polkadot will support this kind of direct routing, however we will outline one possible multi-phase routing process (“hyper-routing”) as a means of scaling out well past the initial set of parachains.

      We assume that all participants know the sub- groupings for next two blocks n, n + 1. In summary, the routing system follows these stages:

      • CollatorS : Contact members of V alidators[n][S]

      • CollatorS : FOR EACH subgroup s: ensure at least 1 member of V alidators[n][s] in contact

      • CollatorS : FOR EACH subgroup s: assume egress[n 1][s][S] is available (all incoming post data to ‘S‘ from last block)

      • CollatorS : Compose block candidate b for S: (b.header, b.ext, b.proof, b.receipt, b.egress)

      • CollatorS : Send proof information proof [S] = (b.header, b.ext, b.proof, b.receipt) to V alidators[n][S]

      • CollatorS : Ensure external transaction data b.ext

        is made available to other collators and validators

      • CollatorS : FOR EACH subgroup s: Send egress information egress[n][S][s] = (b.header, b.receipt, b.egress[s]) to the re- ceiving sub-group’s members of next block V alidators[n + 1][s]

      • V alidatorV : Pre-connect all same-set members for next block: let N = Chain[n + 1][V ]; connect all validators v such that Chain[n + 1][v] = N

      • V alidatorV : Collate all data ingress for this block: FOR EACH subgroup s: Retrieve egress[n 1][s][Chain[n][V ]], get from other val- idators v such that Chain[n][v] = Chain[n][V ].

        Possibly going via randomly selected other val- idators for proof of attempt.

      • V alidatorV : Accept candidate proofs for this block proof [Chain[n][V ]]. Vote block validity

      • V alidatorV : Accept candidate egress data for next block: FOR EACH subgroup s, accept egress[n][s][N ]. Vote block egress availability; re- publish among interested validators v such that Chain[n + 1][v] = Chain[n + 1][V ].

      • V alidatorV : UNTIL CONSENSUS

Where: egress[n][f rom][to] is the current egress queue information for posts going from parachain ‘from‘, to parachain ‘to‘ in block number ‘n‘. CollatorS is a col- lator for parachain S. V alidators[n][s] is the set of val- idators for parachain s at block number n. Conversely, Chain[n][v] is the parachain to which validator v is as- signed on block number n. block.egress[to] is the egress queue of posts from some parachain block block whose destination parachain is to.

Since collators collect (transaction) fees based upon their blocks becoming canonical they are incentivised to ensure that each next-block destination subgroup’s mem- bers are informed of the egress from the present block. Validators are incentivised only to form a consensus on a (parachain) block, as such they care little about which col- lator’s block ultimately becomes canonical. In principle, a validator could form an allegiance with a collator and con- spire to reduce the chances of other collators’ blocks be- coming canonical, however this is both difficult to arrange due to the random selection of validators for parachains and could be defended against with a reduction in fees payable for parachain blocks which hold up the consensus process.

      1. External Data Availability. Ensuring a parachain’s external data is actually available is a perennial issue with decentralised systems aiming to distribute workload across the network. At the heart of the issue is the availability problem which states that since it is neither possible to make a non-interactive proof of availability nor any sort of proof of non-availability, for a BFT system to properly validate any transition whose correctness relies upon the availability of some external data, the maximum number of acceptably Byzantine nodes, plus one, of the system must attest to the data being available.

        For a system to scale out properly, like Polkadot, this invites a problem: if a constant proportion of validators must attest to the availability of the data, and assuming that validators will want to actually store the data be- fore asserting it is available, then how do we avoid the problem of the bandwidth/storage requirements increas- ing with the system size (and therefore number of valida- tors)? One possible answer would be to have a separate set of validators (availability guarantors), whose order grows sublinearly with the size of Polkadot as a whole. This is described in 6.5.3.

        We also have a secondary trick. As a group, colla- tors have an intrinsic incentive to ensure that all data is available for their chosen parachain since without it they are unable to author further blocks from which they can collect transaction fees. Collators also form a group, mem- bership of which is varied (due to the random nature of parachain validator groups) non-trivial to enter and easy

        image

        to prove. Recent collators (perhaps of the last few thou- sand blocks) are therefore allowed to issue challenges to the availability of external data for a particular parachain block to validators for a small bond.

        Validators must contact those from the apparently of- fending sub-group who testified and either acquire and re- turn the data to the collator or escalate the matter by tes- tifying to the lack of availability (direct refusal to provide the data counts as a bond-confiscating offence, therefore the misbehaving validator will likely just drop the connec- tion) and contacting additional validators to run the same test. In the latter case, the collator’s bond is returned.

        Once a quorum of validators who can make such non- availability testimonials is reached, they are released, the misbehaving sub-group is punished, and the block re- verted.

      2. Posts Routing. Each parachain header includes an egress-trie-root; this is the root of a trie containing the routing-base bins, each bin being a concatenated list of egress posts. Merkle proofs may be provided across parachain validators to prove that a particular parachain’s block had a particular egress queue for a particular desti- nation parachain.

        At the beginning of processing a parachain block, each other parachain’s egress queue bound for said block is merged into our block’s ingress queue. We assume strong, probably CSPR9, sub-block ordering to achieve a deter- ministic operation that offers no favouritism between any parachain block pairing. Collators calculate the new queue and drain the egress queues according to the parachain’s logic.

        The contents of the ingress queue is written explicitly into the parachain block. This has two main purposes: firstly, it means that the parachain can be trustlessly syn- chronised in isolation from the other parachains. Secondly, it simplifies the data logistics should the entire ingress queue not be able to be processed in a single block; val- idators and collators are able to process following blocks without having to source the queue’s data specially.

        If the parachain’s ingress queue is above a threshold amount at the end of block processing, then it is marked saturated on the relay-chain and no further messages may be delivered to it until it is cleared. Merkle proofs are used to demonstrate fidelity of the collator’s operation in the parachain block’s proof.

      3. Critique. One minor flaw relating to this basic mechanism is the post-bomb attack. This is where all parachains send the maximum amount of posts possible to a particular parachain. While this ties up the target’s ingress queue at once, no damage is done over and above a standard transaction DoS attack.

        Operating normally, with a set of well-synchronised and non-malicious collators and validators, for N parachains, N × M total validators and L collators per parachain, we can break down the total data pathways per block to:

        Validator: M 1+L+L: M 1 for the other validators in the parachain set, L for each collator providing a can- didate parachain block and a second L for each collator of the next block requiring the egress payloads of the pre- vious block. (The latter is actually more like worst-case

        9cryptographically secure pseudo-random

        operation since it is likely that collators will share such data.)

        Collator: M + kN : M for a connection to each relevant parachain block validator, kN for seeding the egress pay- loads to some subset of each parachain validator group for the next block (and possibly some favoured collator(s)).

        As such, the data path ways per node grow linearly with the overall complexity of the system. While this is reasonable, as the system scales into hundreds or thou- sands of parachains, some communication latency may be absorbed in exchange for a lower complexity growth rate. In this case, a multi-phase routing algorithm may be used in order to reduce the number of instantaneous pathways at a cost of introducing storage buffers and latency.

        Hyper-cube Routing. Hyper-cube routing is a mechanism which can mostly be build as an extension to the basic routing mechanism described above. Essentially, rather than growing the node connectivity with the number of parachains and sub-group nodes, we grow only with the logarithm of parachains. Posts may transit between sev- eral parachains’ queues on their way to final delivery.

        Routing itself is deterministic and simple. We begin by limiting the number of bins in the ingress/egress queues; rather than being the total number of parachains, they are the routing-base (b) . This will be fixed as the number of parachains changes, with the routing-exponent (e) in- stead being raised. Under this model, our message volume grows with O(be), with the pathways remaining constant and the latency (or number of blocks required for delivery) with O(e).

        Our model of routing is a hypercube of e dimensions, with each side of the cube having b possible locations. Each block, we route messages along a single axis. We alternate the axis in a round-robin fashion, thus guaran- teeing worst-case delivery time of e blocks.

        As part of the parachain processing, foreign-bound messages found in the ingress queue are routed imme- diately to the appropriate egress queue’s bin, given the current block number (and thus routing dimension). This process necessitates additional data transfer for each hop on the delivery route, however this is a problem itself which may be mitigated by using some alternative means of data payload delivery and including only a reference, rather than the full payload of the post in the post-trie.

        An example of such a hyper-cube routing for a system with 4 parachains, b = 2 and e = 2 might be:

        Phase 0, on each message M :

        • sub0: if Mdest ∈ {2, 3} then sendTo(2) else keep

        • sub1: if Mdest ∈ {2, 3} then sendTo(3) else keep

        • sub2: if Mdest ∈ {0, 1} then sendTo(0) else keep

        • sub3: if Mdest ∈ {0, 1} then sendTo(1) else keep Phase 1, on each message M :

        • sub0: if Mdest ∈ {1, 3} then sendTo(1) else keep

        • sub1: if Mdest ∈ {0, 2} then sendTo(0) else keep

        • sub2: if Mdest ∈ {1, 3} then sendTo(3) else keep

        • sub3: if Mdest ∈ {0, 2} then sendTo(2) else keep The two dimensions here are easy to see as the first

two bits of the destination index; for the first block, the

higher-order bit alone is used. The second block deals with the low-order bit. Once both happen (in arbitrary order) then the post will be routed.

Maximising Serendipity. One alteration of the basic pro- posal would see a fixed total of c2 c validators, with c 1 validators in each sub-group. Each block, rather than there being an unstructured repartitioning of validators among parachains, instead for each parachain sub-group, each validator would be assigned to a unique and different parachain sub-group on the following block. This would lead to the invariant that between any two blocks, for any two pairings of parachain, there exists two validators who have swapped parachain responsibilities. While this can- not be used to gain absolute guarantees on availability (a single validator will occasionally drop offline, even if benevolent), it can nonetheless optimise the general case. This approach is not without complications. The addi- tion of a parachain would also necessitate a reorganisation of the validator set. Furthermore the number of valida- tors, being tied to the square of the number of parachains, would start initially very small and eventually grow far too fast, becoming untenable after around 50 parachains. None of these are fundamental problems. In the first case, reorganisation of validator sets is something that must be done regularly anyway. Regarding the size of the validator set, when too small, multiple validators may be assigned to the same parachain, applying an integer factor to the overall total of validators. A multi-phase routing mecha- nism such as Hypercube Routing, discussed in 6.6.3 would alleviate the requirement for large number of validators

when there is a large number of chains.

    1. Parachain Validation. A validator’s main purpose is to testify, as a well-bonded actor, that a parachain’s block is valid, including but not limited to any state tran- sition, any external transactions included, the execution of any waiting posts in the ingress queue and the final state of the egress queue. The process itself is fairly simple. Once the validator sealed the previous block they are free to begin working to provide a candidate parachain block candidate for the next round of consensus.

      Initially, the validator finds a parachain block candi- date through a parachain collator (described next) or one of its co-validators. The parachain block candidate data includes the block’s header, the previous block’s header, any external input data included (for Ethereum and Bit- coin, such data would be referred to as transactions, how- ever in principle they may include arbitrary data struc- tures for arbitrary purposes), egress queue data and inter- nal data to prove state-transition validity (for Ethereum this would be the various state/storage trie nodes re- quired to execute each transaction). Experimental evi- dence shows this full dataset for a recent Ethereum block to be at the most a few hundred KiB.

      Simultaneously, if not yet done, the validator will be attempting to retrieve information pertaining to the pre- vious block’s transition, initially from the previous block’s validators and later from all validators signing for the availability of the data.

      Once the validator has received such a candidate block, they then validate it locally. The validation process is contained within the parachain class’s validator module, a consensus-sensitive software module that must be written for any implementation of Polkadot (though in principle a library with a C ABI could enable a single library to be shared between implementations with the appropriate

      reduction in safety coming from having only a single “ref- erence” implementation).

      The process takes the previous block’s header and ver- ifies its identity through the recently agreed relay-chain block in which its hash should be recorded. Once the par- ent header’s validity is ascertained, the specific parachain class’s validation function may be called. This is a sin- gle function accepting a number of data fields (roughly those given previously) and returning a simple Boolean proclaiming the validity of the block.

      Most such validation functions will first check the header-fields which are able to be derived directly from the parent block (e.g. parent hash, number). Following this, they will populate any internal data structures as necessary in order to process transactions and/or posts. For an Ethereum-like chain this amounts to populating a trie database with the nodes that will be needed for the full execution of transactions. Other chain types may have other preparatory mechanisms.

      Once done, the ingress posts and external transac- tions (or whatever the external data represents) will be enacted, balanced according to chain’s specification. (A sensible default might be to require all ingress posts be processed before external transactions be serviced, how- ever this should be for the parachain’s logic to decide.) Through this enactment, a series of egress posts will be created and it will be verified that these do indeed match the collator’s candidate. Finally, the properly populated header will be checked against the candidate’s header.

      With a fully validated candidate block, the validator can then vote for the hash of its header and send all requi- site validation information to the co-validators in its sub- group.

      1. Parachain Collators. Parachain collators are un- bonded operators who fulfill much of the task of miners on the present-day blockchain networks. They are specific to a particular parachain. In order to operate they must maintain both the relay-chain and the fully synchronised parachain.

The precise meaning of “fully synchronised” will de- pend on the class of parachain, though will always in- clude the present state of the parachain’s ingress queue. In Ethereum’s case it also involves at least maintaining a Merkle-tree database of the last few blocks, but might also include various other data structures including Bloom filters for account existence, familial information, logging outputs and reverse lookup tables for block number.

In addition to keeping the two chains synchronised, it must also “fish” for transactions by maintaining a transac- tion queue and accepting properly validated transactions from the public network. With the queue and chain, it is able to create new candidate blocks for the validators cho- sen at each block (whose identity is known since the relay- chain is synchronised) and submit them, together with the various ancillary information such as proof-of-validity, via the peer network.

For its trouble, it collects all fees relating to the trans- actions it includes. Various economics float around this arrangement. In a heavily competitive market where there is a surplus of collators, it is possible that the transaction fees be shared with the parachain validators to incentivise the inclusion of a particular collator’s block. Similarly,

some collators may even raise the required fees that need to be paid in order to make the block more attractive to validators. In this case, a natural market should form with transactions paying higher fees skipping the queue and having faster inclusion in the chain.

    1. Networking. Networking on traditional blockchains like Ethereum and Bitcoin has rather simple requirements. All transactions and blocks are broadcast in a simple undi- rected gossip. Synchronisation is more involved, especially with Ethereum but in reality this logic was contained in the peer strategy rather than the protocol itself which re- solved around a few request and answer message types.

      While Ethereum made progress on current protocol of- ferings with the devp2p protocol, which allowed for many subprotocols to be multiplexed over a single peer connec- tion and thus have the same peer overlay support many p2p protocols simultaneously, the Ethereum portion of the protocol still remained relatively simple and the p2p protocol as a while remains unfinished with important functionality missing such as QoS support. Sadly, a de- sire to create a more ubiquitous “web 3” protocol largely failed, with the only projects using it being those explicitly funded from the Ethereum crowd-sale.

      The requirements for Polkadot are rather more sub- stantial. Rather then a wholly uniform network, Polkadot has several types of participants each with different re- quirements over their peer makeup and several network “avenues” whose participants will tend to converse about particular data. This means a substantially more struc- tured network overlay—and a protocol supporting that— will likely be necessary. Furthermore, extensibility to fa- cilitate future additions such as new kinds of “chain” may themselves require a novel overlay structure.

      While an in-depth discussion of how the networking protocol may look is outside of the scope of this docu- ment, some requirements analysis is reasonable. We can roughly break down our network participants into two sets (relay-chain, parachains) each of three subsets. We can also state that each of the parachain participants are only interested in conversing between themselves as opposed to participants in other parachains:

      • Relay-chain participants:

      • Validators: P, split into subsets P[s] for each parachain

      • Availability Guarantors: A (this may be repre- sented by Validators in the basic form of the pro- tocol)

      • Relay-chain clients: M (note members of each parachain set will also tend to be members of M)

      • Parachain participants:

      • Parachain Collators: C[0], C[1], . . .

      • Parachain Fishermen: F[0], F[1], . . .

      • Parachain clients: S[0], S[1], . . .

      • Parachain light-clients: L[0], L[1], . . .

        In general we name particular classes of communication will tend to take place between members of these sets:

      • P | A <> P | A: The full set of valida- tors/guarantors must be well-connected to achieve consensus.

      • P[s] <> C[s] | P[s]: Each validator as a mem- ber of a given parachain group will tend to gossip with other such members as well as the collators

        of that parachain to discover and share block can- didates.

      • A <> P[s] | C | A: Each availability guarantor will need to collect consensus-sensitive cross-chain data from the validators assigned to it; collators may also optimise the chance of consensus on their block by advertising it to availability guarantors. Once they have it, the data will be disbursed to other such guarantor to facilitate consensus.

      • P[s] <> A | P[s]: Parachain validators will

        need to collect additional input data from the pre- vious set of validators or the availability guaran- tors.

      • F[s] <> P: When reporting, fishermen may place a claim with any participant.

      • M <> M | P | A: General relay-chain clients dis- burse data from validators and guarantors.

      • S[s] <> S[s] | P[s] | A: Parachain clients dis- burse data from the validator/guarantors.

      • L[s] <> L[s] | S[s]: Parachain light clients disburse data from the full clients.

To ensure an efficient transport mechanism, a “flat” overlay network—like Ethereum’s devp2p—where each node does not (non-arbitrarily) differentiate fitness of its peers is unlikely to be suitable. A reasonably extensible peer selection and discovery mechanism will likely need to be included within the protocol as well as aggressive planning a lookahead to ensure the right sort of peers are “serendipitously” connected at the right time.

The precise strategy of peer make-up will be differ- ent for each class of participant: for a properly scaled-out multi-chain, collators will either need to be continuously reconnecting to the accordingly elected validators, or will need on-going agreements with a subset of the validators to ensure they are not disconnected during the vast ma- jority of the time that they are useless for that valida- tor. Collators will also naturally attempt to maintain one or more stable connections into the availability guarantor set to ensure swift propagation of their consensus-sensitive data.

Availability guarantors will mostly aim to maintain a stable connection to each other and to validators (for con- sensus and the consensus-critical parachain data to which they attest), as well as to some collators (for the parachain data) and some fishermen and full clients (for dispersing information). Validators will tend to look for other val- idators, especially those in the same sub-group and any collators that can supply them with parachain block can- didates.

Fishermen, as well as general relay-chain and parachain clients will generally aim to keep a connection open to a validator or guarantor, but plenty of other nodes similar to themselves otherwise. Parachain light clients will simi- larly aim to be connected to a full client of the parachain, if not just other parachain light-clients.

      1. The Problem of Peer Churn. In the basic proto- col proposal, each of these subsets constantly alter ran- domly with each block as the validators assigned to verify the parachain transitions are randomly elected. This can be a problem should disparate (non-peer) nodes need to pass data between each other. One must either rely on a fairly-distributed and well-connected peer network to

        image

        ensure that the hop-distance (and therefore worst-case la- tency) only grows with the logarithm of the network size (a Kademlia-like protocol may help here), or one must introduce longer block times to allow the necessary con- nection negotiation to take place to keep a peer-set that reflects the node’s current communication needs.

        Neither of these are great solutions: long block times being forced upon the network may render it useless for particular applications and chains. Even a perfectly fair and connected network will result in substantial wastage of bandwidth as it scales due to uninterested nodes having to forward data useless to them.

        While both directions may form part of the solution, a reasonable optimisation to help minimise latency would be to restrict the volatility of these parachain validator sets, either reassigning the membership only between se- ries of blocks (e.g. in groups of 15, which at a 4 second block time would mean altering connections only once per minute) or by rotating membership in an incremental fash- ion, e.g. changing by one member at a time (e.g. if there are 15 validators assigned to each parachain, then on av- erage it would be a full minute between completely unique sets). By limiting the amount of peer churn, and ensur- ing that advantageous peer connections are made well in advance through the partial predictability of parachain sets, we can help ensure each node keep a permanently serendipitous selecti

      2. Path to an Effective Network Protocol. Likely the most effective and reasonable development effort will fo- cus on utilising a pre-existing protocol rather than rolling our own. Several peer-to-peer base protocols exist that we may use or augment including Ethereum’s own devp2p [21], IPFS’s libp2p [1] and GNU’s GNUnet [4]. A full re- view of these protocols and their relevance for building a modular peer network supporting certain structural guar- antees, dynamic peer steering and extensible sub-protocols is well beyond the scope of this document but will be an important step in the implementation of Polkadot.

  1. Practicalities of the Protocol

      1. Interchain Transaction Payment. While a great amount of freedom and simplicity is gained through drop- ping the need for a holistic computation resource account- ing framework like Ethereum’s gas, this does raise an im- portant question: without gas, how does one parachain avoid another parachain from forcing it to do computa- tion? While we can rely on transaction-post ingress queue buffers to prevent one chain from spamming another with transaction data, there is no equivalent mechanism pro- vided by the protocol to prevent the spamming of trans- action processing.

        This is a problem left to the higher level. Since chains are free to attach arbitrary semantics on to the incoming transaction-post data, we can ensure that computation must be paid-for before started. In a similar vein to the model espoused by Ethereum Serenity, we can imagine a “break-in” contract within a parachain which allows a validator to be guaranteed payment in exchange for the provision of a particular volume of processing resources. These resources may be measured in something like gas, but could also be some entirely novel model such as sub- jective time-to-execute or a Bitcoin-like flat-fee model.

        On its own this isn’t so useful since we cannot read- ily assume that the off-chain caller has available to them whatever value mechanism is recognised by the break-in contract. However, we can imagine a secondary “break- out” contract in the source chain. The two contracts to- gether would form a bridge, recognising each other and providing value-equivalence. (Staking-tokens, available to each, could be used to settle up the balance-of-payments.) Calling into another such chain would mean proxying through this bridge, which would provide the means of negotiating the value transfer between chains in order to pay for the computation resources required on the desti- nation parachain.

      2. Additional Chains. While the addition of a parachain is a relatively cheap operation, it is not free. More parachains means fewer validators per parachain and, eventually, a larger number of validators each with a reduced average bond. While the issue of a smaller coer- cion cost for attacking a parachain is mitigated through fishermen, the growing validator set essentially forces a higher degree of latency due to the mechanics of the un- derlying consensus method. Furthermore each parachain brings with it the potential to grief validators with an over-burdensome validation algorithm.

        As such, there will be some “price” that validators and/or the stake-holding community will extract for the addition of a new parachain. This market for chains will possibly see the addition of either:

        • Chains that likely have zero net contribution pay- ing (in terms of locking up or burning staking to- kens) to be made a part (e.g. consortium chains, Doge-chains, app-specific chains);

        • chains that deliver intrinsic value to the network through adding particular functionality difficult to get elsewhere (e.g. confidentiality, internal scal- ability, service tie-ins).

    Essentially, the community of stakeholders will need to be incentivized to add child chains—either financially or through the desire to add featureful chains to the relay.

    It is envisioned that new chains added will have a very short notice period for removal, allowing for new chains to be experimented with without any risk of compromising the medium or long-term value proposition.

  2. Conclusion

We have outlined a direction one may take to author a scalable, heterogeneous multi-chain protocol with the po- tential to be backwards compatible to certain, pre-existing blockchain networks. Under such a protocol, participants work in enlightened self-interest to create an overall sys- tem which can be extended in an exceptionally free man- ner and without the typical cost for existing users that comes from a standard blockchain design. We have given a rough outline of the architecture it would take including the nature of the participants, their economic incentives and the processes under which they must engage. We have identified a basic design and discussed its strengths and limitations; accordingly we have further directions which may ease those limitations and yield further ground to- wards a fully scalable blockchain solution.

image

    1. Missing Material and Open Questions. Net- work forking is always a possibility from divergent im- plementations of the protocol. The recovery from such an exceptional condition was not discussed. Given the net- work will necessarily have a non-zero period of finalisation, it should not be a large issue to recover from the relay- chain forking, however will require careful integration into the consensus protocol.

      Bond-confiscation and conversely reward provision has not been deeply explored. At present we assume rewards are provided under a winner-takes-all basis: this may not give the best incentivisation model for fishermen. A short- period commit-reveal process would allow many fishermen to claim the prize giving a fairer distribution of rewards, however the process could lead to additional latency in the discovery of misbehaviour.

    2. Acknowledgments. Many thanks to all of the proof-readers who have helped get this in to a vaguely presentable shape. In particular, Peter Czaban, Bj¨orn Wagner, Ken Kappler, Robert Habermeier, Vitalik Bu- terin, Reto Trinkler and Jack Petersson. Thanks to all the people who have contributed ideas or the beginnings thereof, Marek Kotewicz and Aeron Buchanan deserve es- pecial mention. And thanks to everyone else for their help along the way. All errors are my own.

Portions of this work, including initial research into consensus algorithms, was funded in part by the British Government under the Innovate UK programme.

References

  1. The specs for libp2p and associated submodules. https:// github.com/libp2p/specs.

  2. Webassembly. http://webassembly.org/, 2016.

  3. Adam Back, Matt Corallo, Luke Dashjr, Mark Friedenbach, Gregory Maxwell, Andrew Miller, Andrew Poelstra, Jorge Ti- mon, and Pieter Wuille. Enabling blockchain innovations with pegged sidechains. 2014.

  4. Krista Bennett, Christian Grothoff, Tzvetan Horozov, Ioana Patrascu, and Tiberiu Stef. Gnunet-a truly anonymous net- working infrastructure. In In: Proc. Privacy Enhancing Tech- nologies Workshop (PET. Citeseer, 2002.

  5. Vitalik Buterin. Ethereum: A next-generation smart contract and decentralized application platform. https://github.com/ ethereum/wiki/wiki/White-Paper, 2013.

  6. Vitalik Buterin. Ethereum 2.0 mauve paper. 2016.

  7. Vitalik Buterin. Serenity poc2. 2016.

  8. Nxt community. Whitepaper: Nxt. http://wiki.nxtcrypto. org/wiki/Whitepaper:Nxt, 2013.

  9. Christopher Copeland and Hongxia Zhong. Tangaroa: a

    byzantine fault tolerant raft. http://www.scs.stanford.edu/ 14au-cs244b/labs/projects/copeland_zhong.pdf, 2016.

  10. Ethan Buchman Jae Kwon. Cosmos: A network of

    distributed ledgers. https://github.com/cosmos/cosmos/blob/ master/WHITEPAPER.md, 2016.

  11. Jae Kwon. Tendermint: Consensus without mining. http:

    //tendermint.com/docs/tendermint.pdf, 2014.

  12. Daniel Larimer. Bitshares. http://docs.bitshares.org/ bitshares/history.html, 2013.

  13. Andrew Miller, Yu Xia, Kyle Croman, Elaine Shi, and Dawn

    Song. The honey badger of bft protocols. Technical report, Cryptology ePrint Archive 2016/199, 2016.

  14. Satoshi Nakamoto. Bitcoin: A peer-to-peer electronic cash sys- tem. https://bitcoin.org/bitcoin.pdf, 2008.

  15. Diego Ongaro and John Ousterhout. In search of an un-

    derstandable consensus algorithm. In 2014 USENIX Annual Technical Conference (USENIX ATC 14), pages 305–319, 2014.

  16. Parity. Parity ethereum client. https://parity.io, 2016.

  17. Serguei Popov. The tangle. https://www.iotatoken.com/IOTA_ Whitepaper.pdf, 2016.

  18. Youcai Qian. Randao. https://github.com/randao/randao, 2015.

  19. Eli Ben Sasson, Alessandro Chiesa, Christina Garman, Matthew Green, Ian Miers, Eran Tromer, and Madars Virza. Zerocash: Decentralized anonymous payments from bitcoin. In 2014 IEEE Symposium on Security and Privacy, pages 459–474. IEEE, 2014.

  20. Paul Snow, Brian Deery, Jack Lu, David Johnston, and Peter Kirb. Factom: Business processes secured by immutable audit trails on the blockchain. https://raw.githubusercontent. com/FactomProject/FactomDocs/master/Factom_Whitepaper.pdf, 2014.

  21. Gavin Wood. Devp2p wire protocol. https://github.com/ ethereum/wiki/wiki/libp2p-Whitepaper, 2014.

  22. Gavin Wood. Ethereum: a secure decentralised generalised

    transaction ledger. http://gavwood.com/paper.pdf, 2014.

  23. Gavin Wood. Yellow paper committee. https://github.com/ gavofyork/curly-engine, 2016.

Appendix A. Functional Components

Seen from a high-level, the Parity Polkadot Platform stack has a number of functional components and any develop- ment of it will require a substantial amount of research and development prior to it being completed.

Some components are dependent on others existing, others are independent. Some are very important for the platform to function properly, others are nice-to-haves. Some are of indeterminate complexity and have not yet been deemed feasible, others are relatively straight-forward.

Here we’ll try to list as many such components as possible along with where they would fit into our development roadmap.

Networking subsystem: This is the means by which a peer network is formed and maintained. Simple alterations of existing peer-to-peer networking libraries (devp2p most likely) will be sufficient for an initial system. However, additional research and development will be necessary to ensure that as the network grows, the network topology becomes increasingly structured allowing for optimal data logistics. For the final deployment, adaptations of libp2p, devp2p and GNUnet should be first considered. If requirements are not likely to be met then a new protocol should be considered.

Consensus mechanism: Proof-of-authority consensus mechanism supporting rich validator statements and al- lowing multiple independent items to be agreed upon under a single series based upon subjective reception of the partial set of validator statements. The mechanism should allow the proof of misbehaviour for the dismissal of malicious validators but need not involve any staking mechanism. A substantial amount of research and prototyping will precede the development of this component.

Proof-of-stake chain: Extending the consensus mechanism into proof-of-stake territory; this module includes staking tokens, managing entry and exit from the validator pool, a market mechanism for determining validator rewards, finalising the approval-voting nomination mechanism and managing bond-confiscation and dismissal. It includes a substantial amount of research and prototyping prior to final development.

image

Parachain implementation: A first parachain implementation, likely to be based heavily on an existing blockchain protocol such as Bitcoin or (more likely, since it provides for rich transactions) Ethereum. This will include an integration with the proof-of-stake chain, allowing the parachain to gain consensus without its own internal consensus mechanism.

Transaction processing subsystem: An evolution of the parachain and relay-chain, this will allow for trans- actions to be sent, received and propagated. It includes the designs of transaction queuing and optimised transaction routing on the network layer.

Transaction-routing subsystem: This introduces more specifics into the relay-chain’s behaviour. Initially, adding transactability into parachains will be needed. Following that, with two parachains hard-coded into the relay-chain, it will include the management of the ingress/egress queues. Eventually this will develop along with the network protocol into a means of directed transaction propagation, ensuring independent parachain collators are not overly exposed to transactions that are not of interest.

Relay chain: This is the final stage of the relay-chain, allowing the dynamic addition, removal and emergency pausing of parachains, the reporting of bad behaviour and includes implementation of the “fisherman” function- ality.

Independent collators: This is the delivery of an alternative chain-specific collator functionality. It includes proof creation (for collators), parachain misbehaviour detection (for fishermen) and the validation function (for validators). It also includes any additional networking required to allow the two to discover and communicate. Network dynamics modelling and research: The overall dynamics of this protocol should be researched thor- oughly. This can happen both through offline network modelling and through empirical evidence through simu- lated nodes. The latter is dependent on the relay-chain and will include the development of a structured logging

protocol allowing nodes to submit detailed reports on their actions to a central hub for collation.

Network intelligence: As a complex decentralised multi-party network, a network intelligence hub similar to http://ethstats.net is needed to monitor the life-signs of the network as a whole and flag potentially disruptive behaviour. Structured logging should be used to generate and visualise this data in real-time for maximum efficiency. It is dependent on the relay-chain being in a reasonably complete state.

Information publication platform: This is a mechanism for publishing data relating to the blockchain, and effectively means a decentralised content-discovery network. Initially it can be handled by basic peer-to-peer lookups but for deployment will likely require a more structured and robust solution since availability will become a critical piece of information. IPFS integration may be the most sensible means of achieving these goals.

Javascript interaction bindings: The main means of interacting with the network will likely follow the example of Ethereum and as such high-quality Javascript bindings are critical to have. Our bindings will cover interactions with the relay-chain and the initial parachain and as such depends on them.

Governance: Initially, this will be a meta-protocol on the relay-chain for managing exceptional events such as hard-forks, soft-forks and protocol reparameterisation. It will include a modern structure to help manage conflict and prevent live-lock. Ultimately, this may become a full meta-protocol layer able to enact changes normally reserved for hard-forks. Requires the relay chain.

Interaction platform: A platform by which “typical” users are able to interact with the system, along with minimal functionality to facilitate common tasks such as entering the bond process, voting, nomination, becoming a validator, fisherman or collator and staking token transfer. Depends upon having a functional relay-chain.

Light clients: Light-client technology for both the relay-chain and any parachains developed. This will allow clients to be able to gain information regarding activity on the chains with a high degree of trust-freedom and without any substantial requirement of storage or bandwidth. Depends upon the relay-chain.

Parachain UI: A multi-chain, multi-token wallet and identity management system. It requires light-client tech- nology and the interaction platform. This is not needed for any initial network deployment.

On-chain Dapp services: There may be various services that will need to be deployed on any initial parachains such as registration hubs for APIs, names, natural-language specifications and code. This depends on the parachain but is not strictly needed for any initial network deployment.

Application development tools: This includes various bits of software required to help developers. Examples include compilers, key management tools, data archivers and VM simulators. Many others exist. These will need to be developed as required. Technologies will be chosen partly to minimise the need for such tools. Many will not be strictly required.

Ethereum-as-a-parachain: Trust-free bridge to Ethereum and back again, allowing posted transactions to be routed from parachains into Ethereum (and treated like any other transaction of external origin) and allow Ethereum contracts to be able to post transactions to parachains and the accounts therein. Initially, this will be modelled to ascertain feasibility and get any structural limitations based around the number of validators required by the consensus process, a component on which it is dependent. A proof-of-concept can be built following this and final development will be dependent on the relay-chain itself.

Bitcoin-RPC compatibility layer: A simple RPC compatibility layer for the relay-chain allowing infrastructure already built using that protocol to be interoperable with Polkadot and as such minimise effort for support. A stretch goal requiring the relay-chain.

image

Web 2.0 bindings: Bindings into common Web 2.0 technology stacks to facilitate the usage of Polkadot infras- tructure with legacy systems. A stretch goal dependent on the initial parachains and any on-chain infrastructure to be exposed.

zk-SNARK parachain example: A parachain utilising the properties of zk-SNARKs in order to ensure identities of transactors on it are kept private. A stretch goal dependent on the relay-chain.

Encrypted parachain example: A parachain that keeps each item of state encrypted and signed. These ensure the enforcement of access controls over inspection and modification of the data therein allowing commercial regulated operations to conform as necessary. Would include proof-of-authority mechanisms to allow Polkadot validators to guarantee some degree of correctness of state transitions without gaining exposure to the underlying information. A stretch goal depending on the relay-chain.

Trust-free Bitcoin bridge: Trust-free Bitcoin “two-way-peg” functionality. This would possibly use threshold signatures or alternatively an n of m multi-signature Bitcoin account together with SPV proofs & specialised fishermen. Development is predicated on an initial feasibility analysis giving a favourable result. Some additional functionality may need to be added to (or unlocked from) the Bitcoin protocol to support this functionality.

Abstract/low-level decentralised applications: Trust-free token exchange, asset-tracking infrastructure, crowd- sale infrastructure.

Contract language: Certainly not an integral part of the project, but a nice stretch-goal nonetheless. Would include a safe contract language together with tutorials, guidelines and educational tools. It may include means of making formal proofs as per the original Solidity language vision or could be based around some other programming paradigm which helps minimise critical process errors such as functional programming or condition- orientated programming.

IDE: Based on the contract language, this would facilitate editing, collaboration, publication and debugging of contracts on the parachains. A far stretch goal.

Appendix B. Frequently Asked Questions

Is Polkadot designed to replace (insert blockchain here): No. The goal of Polkadot is to provide a frame- work under which new blockchains may be created and to which existing blockchains can, if their communities desire, be transitioned.

Is Polkadot designed to replace (insert crypto-currency here ): No. Polkadot tokens are neither intended nor designed to be used as a currency. They would make a bad currency: most will remain illiquid in the staking system and those that are liquid will face substantial fees for transfer of ownership. Rather, the purpose of Polkadot tokens is to be a direct representation of stake in the Polkadot network.

What is the inflation rate for Polkadot staking tokens: The Polkadot staking token base expansion is un- limited. It rises and lowers according to market effects in order to target a particular proportion of tokens held under long-term bond in the validation process.

Why does staking token ownership reflect stakeholding: This is a mechanism realised by the fact that they underpin the network’s security. As such their value is tied to the overall economic value that Polkadot provides. Any actors who gain overall value from Polkadot operating correctly are incentivised to ensure it continues to do so. The best means of doing so is to take part in the validation process. This generally implies ownership of staking tokens.

 

Monero

Highlight to Annotate. Click to Learn.

CryptoNote v 2.0

Nicolas van Saberhagen
October 17, 2013

Introduction

“Bitcoin” [1] has been a successful implementation of the concept of p2p electronic cash. Both professionals and the general public have come to appreciate the convenient combination of public transactions and proof-of-work as a trust model. Today, the user base of electronic cash is growing at a steady pace; customers are attracted to low fees and the anonymity provided by electronic cash and merchants value its predicted and decentralized emission. Bitcoin has effectively proved that electronic cash can be as simple as paper money and as convenient as credit cards.

Unfortunately, Bitcoin suffers from several deficiencies. For example, the system’s distributed nature is inflexible, preventing the implementation of new features until almost all of the net- work users update their clients. Some critical flaws that cannot be fixed rapidly deter Bitcoin’s widespread propagation. In such inflexible models, it is more efficient to roll-out a new project rather than perpetually fix the original project.

In this paper, we study and propose solutions to the main deficiencies of Bitcoin. We believe that a system taking into account the solutions we propose will lead to a healthy competition among different electronic cash systems. We also propose our own electronic cash, “CryptoNote”, a name emphasizing the next breakthrough in electronic cash.

 

Bitcoin drawbacks and some possible solutions

Traceability of transactions

Privacy and anonymity are the most important aspects of electronic cash. Peer-to-peer payments seek to be concealed from third party’s view, a distinct difference when compared with traditional banking. In particular, T. Okamoto and K. Ohta described six criteria of ideal electronic cash, which included “privacy: relationship between the user and his purchases must be untraceable by anyone” [30]. From their description, we derived two properties which a fully anonymous electronic cash model must satisfy in order to comply with the requirements outlined by Okamoto and Ohta:

Untraceability: for each incoming transaction all possible senders are equiprobable.

Unlinkability: for any two outgoing transactions it is impossible to prove they were sent to the same person.

Unfortunately, Bitcoin does not satisfy the untraceability requirement. Since all the trans- actions that take place between the network’s participants are public, any transaction can be

unambiguously traced to a unique origin and final recipient. Even if two participants exchange funds in an indirect way, a properly engineered path-finding method will reveal the origin and final recipient.

It is also suspected that Bitcoin does not satisfy the second property. Some researchers stated ([33, 35, 29, 31]) that a careful blockchain analysis may reveal a connection between the users of the Bitcoin network and their transactions. Although a number of methods are disputed [25], it is suspected that a lot of hidden personal information can be extracted from the public database.

Bitcoin’s failure to satisfy the two properties outlined above leads us to conclude that it is not an anonymous but a pseudo-anonymous electronic cash system. Users were quick to develop solutions to circumvent this shortcoming. Two direct solutions were “laundering services” [2] and the development of distributed methods [3, 4]. Both solutions are based on the idea of mixing several public transactions and sending them through some intermediary address; which in turn suffers the drawback of requiring a trusted third party.

Recently, a more creative scheme was proposed by I. Miers et al. [28]: “Zerocoin”. Zerocoin utilizes a cryptographic one-way accumulators and zero-knoweldge proofs which permit users to “convert” bitcoins to zerocoins and spend them using anonymous proof of ownership instead of explicit public-key based digital signatures. However, such knowledge proofs have a constant but inconvenient size – about 30kb (based on today’s Bitcoin limits), which makes the proposal impractical. Authors admit that the protocol is unlikely to ever be accepted by the majority of Bitcoin users [5].

 

The proof-of-work function
Bitcoin creator Satoshi Nakamoto described the majority decision making algorithm as “one- CPU-one-vote” and used a CPU-bound pricing function (double SHA-256) for his proof-of-work scheme. Since users vote for the single history of transactions order [1], the reasonableness and consistency of this process are critical conditions for the whole system.
The security of this model suffers from two drawbacks. First, it requires 51% of the network’s mining power to be under the control of honest users. Secondly, the system’s progress (bug fixes, security fixes, etc…) require the overwhelming majority of users to support and agree to the changes (this occurs when the users update their wallet software) [6].Finally this same voting mechanism is also used for collective polls about implementation of some features [7].
This permits us to conjecture the properties that must be satisfied by the proof-of-work pricing function. Such function must not enable a network participant to have a significant advantage over another participant; it requires a parity between common hardware and high cost of custom devices. From recent examples [8], we can see that the SHA-256 function used in the Bitcoin architecture does not posses this property as mining becomes more efficient on GPUs and ASIC devices when compared to high-end CPUs.
Therefore, Bitcoin creates favourable conditions for a large gap between the voting power of participants as it violates the “one-CPU-one-vote” principle since GPU and ASIC owners posses a much larger voting power when compared with CPU owners. It is a classical example of the Pareto principle where 20% of a system’s participants control more than 80% of the votes.
One could argue that such inequality is not relevant to the network’s security since it is not the small number of participants controlling the majority of the votes but the honesty of these participants that matters. However, such argument is somewhat flawed since it is rather the possibility of cheap specialized hardware appearing rather than the participants’ honesty which poses a threat. To demonstrate this, let us take the following example. Suppose a malevolent individual gains significant mining power by creating his own mining farm through the cheap

hardware described previously. Suppose that the global hashrate decreases significantly, even for a moment, he can now use his mining power to fork the chain and double-spend. As we shall see later in this article, it is not unlikely for the previously described event to take place.

 

Irregular emission

Bitcoin has a predetermined emission rate: each solved block produces a fixed amount of coins. Approximately every four years this reward is halved. The original intention was to create a limited smooth emission with exponential decay, but in fact we have a piecewise linear emission function whose breakpoints may cause problems to the Bitcoin infrastructure.

When the breakpoint occurs, miners start to receive only half of the value of their previous reward. The absolute difference between 12.5 and 6.25 BTC (projected for the year 2020) may seem tolerable. However, when examining the 50 to 25 BTC drop that took place on November 28 2012, felt inappropriate for a significant number of members of the mining community. Figure 1 shows a dramatic decrease in the network’s hashrate in the end of November, exactly when the halving took place. This event could have been the perfect moment for the malevolent individual described in the proof-of-work function section to carry-out a double spending attack [36].

image

Fig. 1. Bitcoin hashrate chart (source: http://bitcoin.sipa.be)

 

Hardcoded constants

Bitcoin has many hard-coded limits, where some are natural elements of the original design (e.g. block frequency, maximum amount of money supply, number of confirmations) whereas other seem to be artificial constraints. It is not so much the limits, as the inability of quickly changing

them if necessary that causes the main drawbacks. Unfortunately, it is hard to predict when the constants may need to be changed and replacing them may lead to terrible consequences.

A good example of a hardcoded limit change leading to disastrous consequences is the block size limit set to 250kb1. This limit was sufficient to hold about 10000 standard transactions. In early 2013, this limit had almost been reached and an agreement was reached to increase the limit. The change was implemented in wallet version 0.8 and ended with a 24-blocks chain split and a successful double-spend attack [9]. While the bug was not in the Bitcoin protocol, but rather in the database engine it could have been easily caught by a simple stress test if there was no artificially introduced block size limit.

Constants also act as a form of centralization point. Despite the peer-to-peer nature of Bitcoin, an overwhelming majority of nodes use the official reference client [10] developed by a small group of people. This group makes the decision to implement changes to the protocol and most people accept these changes irrespective of their “correctness”. Some decisions caused heated discussions and even calls for boycott [11], which indicates that the community and the developers may disagree on some important points. It therefore seems logical to have a protocol with user-configurable and self-adjusting variables as a possible way to avoid these problems.

 

Bulky scripts

The scripting system in Bitcoin is a heavy and complex feature. It potentially allows one to create sophisticated transactions [12], but some of its features are disabled due to security concerns and some have never even been used [13]. The script (including both senders’ and receivers’ parts) for the most popular transaction in Bitcoin looks like this:

image

image

image

image

<sig> <pubKey> OP DUP OP HASH160 <pubKeyHash> OP EQUALVERIFY OP CHECKSIG.

The script is 164 bytes long whereas its only purpose is to check if the receiver possess the secret key required to verify his signature.

 

The CryptoNote Technology

Now that we have covered the limitations of the Bitcoin technology, we will concentrate on presenting the features of CryptoNote.

 

Untraceable Transactions

In this section we propose a scheme of fully anonymous transactions satisfying both untraceability and unlinkability conditions. An important feature of our solution is its autonomy: the sender is not required to cooperate with other users or a trusted third party to make his transactions; hence each participant produces a cover traffic independently.

 

Literature review

Our scheme relies on the cryptographic primitive called a group signature. First presented by

D. Chaum and E. van Heyst [19], it allows a user to sign his message on behalf of the group. After signing the message the user provides (for verification purposes) not his own single public

image

1This is so-called “soft limit” — the reference client restriction for creating new blocks. Hard maximum of

possible blocksize was 1 MB

key, but the keys of all the users of his group. A verifier is convinced that the real signer is a member of the group, but cannot exclusively identify the signer.

The original protocol required a trusted third party (called the Group Manager), and he was the only one who could trace the signer. The next version called a ring signature, introduced by Rivest et al. in [34], was an autonomous scheme without Group Manager and anonymity revocation. Various modifications of this scheme appeared later: linkable ring signature [26, 27, 17] allowed to determine if two signatures were produced by the same group member, traceable ring signature [24, 23] limited excessive anonymity by providing possibility to trace the signer of two messages with respect to the same metainformation (or “tag” in terms of [24]).

A similar cryptographic construction is also known as a ad-hoc group signature [16, 38]. It emphasizes the arbitrary group formation, whereas group/ring signature schemes rather imply a fixed set of members.

For the most part, our solution is based on the work “Traceable ring signature” by E. Fujisaki and K. Suzuki [24]. In order to distinguish the original algorithm and our modification we will call the latter a one-time ring signature, stressing the user’s capability to produce only one valid signature under his private key. We weakened the traceability property and kept the linkability only to provide one-timeness: the public key may appear in many foreign verifying sets and the private key can be used for generating a unique anonymous signature. In case of a double spend attempt these two signatures will be linked together, but revealing the signer is not necessary for our purposes.

 

Definitions

 

Elliptic curve parameters

As our base signature algorithm we chose to use the fast scheme EdDSA, which is developed and implemented by D.J. Bernstein et al. [18]. Like Bitcoin’s ECDSA it is based on the elliptic curve discrete logarithm problem, so our scheme could also be applied to Bitcoin in future.

Common parameters are:

q: a prime number; q = 2255 19;

d: an element of Fq; d = 121665/121666;

E: an elliptic curve equation; x2 + y2 = 1 + dx2y2; G: a base point; G = (x, 4/5);

l: a prime order of the base point; l = 2252 + 27742317777372353535851937790883648493;

Hs: a cryptographic hash function {0, 1}Fq;

Hp: a deterministic hash function E(Fq) E(Fq).

 

Terminology

Enhanced privacy requires a new terminology which should not be confused with Bitcoin entities.

private ec-key is a standard elliptic curve private key: a number a [1, l 1];

public ec-key is a standard elliptic curve public key: a point A = aG;

one-time keypair is a pair of private and public ec-keys;

private user key is a pair (a, b) of two different private ec-keys;

tracking key is a pair (a, B) of private and public ec-key (where B = bG and a /= b);

public user key is a pair (A, B) of two public ec-keys derived from (a, b);

standard address is a representation of a public user key given into human friendly string with error correction;

truncated address is a representation of the second half (point B) of a public user key given into human friendly string with error correction.

The transaction structure remains similar to the structure in Bitcoin: every user can choose several independent incoming payments (transactions outputs), sign them with the corresponding private keys and send them to different destinations.

Contrary to Bitcoin’s model, where a user possesses unique private and public key, in the proposed model a sender generates a one-time public key based on the recipient’s address and some random data. In this sense, an incoming transaction for the same recipient is sent to a one-time public key (not directly to a unique address) and only the recipient can recover the corresponding private part to redeem his funds (using his unique private key). The recipient can spend the funds using a ring signature, keeping his ownership and actual spending anonymous. The details of the protocol are explained in the next subsections.

 

Unlinkable payments

Classic Bitcoin addresses, once being published, become unambiguous identifier for incoming payments, linking them together and tying to the recipient’s pseudonyms. If someone wants to receive an “untied” transaction, he should convey his address to the sender by a private channel. If he wants to receive different transactions which cannot be proven to belong to the same owner he should generate all the different addresses and never publish them in his own pseudonym.

image

image

image

image

Public Private

Alice

Bob’s addr 1

Bob’s key 1

Bob

image image

Carol

Bob’s addr 2

Bob’s key 2

Fig. 2. Traditional Bitcoin keys/transactions model.

We propose a solution allowing a user to publish a single address and receive unconditional unlinkable payments. The destination of each CryptoNote output (by default) is a public key, derived from recipient’s address and sender’s random data. The main advantage against Bitcoin is that every destination key is unique by default (unless the sender uses the same data for each of his transactions to the same recipient). Hence, there is no such issue as “address reuse” by design and no observer can determine if any transactions were sent to a specific address or link two addresses together.

image

image

image

One-time key

Public Private

Alice

One-time key

Bob

image

Bob’s Address

Bob’s Key

One-time key

Carol

Fig. 3. CryptoNote keys/transactions model.

First, the sender performs a Diffie-Hellman exchange to get a shared secret from his data and half of the recipient’s address. Then he computes a one-time destination key, using the shared secret and the second half of the address. Two different ec-keys are required from the recipient for these two steps, so a standard CryptoNote address is nearly twice as large as a Bitcoin wallet address. The receiver also performs a Diffie-Hellman exchange to recover the corresponding secret key.

A standard transaction sequence goes as follows:

 

Alice wants to send a payment to Bob, who has published his standard address. She unpacks the address and gets Bob’s public key (A, B).

 

Alice generates a random r [1, l 1] and computes a one-time public key P = Hs(rA)G +

B.

image

 

Alice uses P as a destination key for the output and also packs value R = rG (as a part of the Diffie-Hellman exchange) somewhere into the transaction. Note that she can create other outputs with unique public keys: different recipients’ keys (Ai, Bi) imply different Pi even with the same r.

Transaction

Amount

Tx public key Tx output

R = rG

Sender’s ran- dom data

r

Destination key

P = Hs(rA)G + B

(A, B)

Receiver’s public key

Fig. 4. Standard transaction structure.

 

Alice sends the transaction.

 

Bob checks every passing transaction with his private key (a, b), and computes P 1 =

H

s(aR)G + B. If Alice’s transaction for with Bob as the recipient was among them, then aR = arG = rA and P 1 = P .

Transaction

H

 

Bob can recover the corresponding one-time private key: x = s(aR) + b, so as P = xG. He can spend this output at any time by signing a transaction with x.

Receiver’s private key

(a, b)

one-time private key

image

x = Hs(aR) + b

one-time public key

R

P I = Hs(aR)G + bG

P I =? P

Tx public key

Tx output

Destination key

Amount

Fig. 5. Incoming transaction check.

As a result Bob gets incoming payments, associated with one-time public keys which are

unlinkable for a spectator. Some additional notes:

When Bob “recognizes” his transactions (see step 5) he practically uses only half of his private information: (a, B). This pair, also known as the tracking key, can be passed to a third party (Carol). Bob can delegate her the processing of new transactions. Bob doesn’t need to explicitly trust Carol, because she can’t recover the one-time secret key p without Bob’s full private key (a, b). This approach is useful when Bob lacks bandwidth or computation power (smartphones, hardware wallets etc.).

In case Alice wants to prove she sent a transaction to Bob’s address she can either disclose r or use any kind of zero-knowledge protocol to prove she knows r (for example by signing the transaction with r).

H H

If Bob wants to have an audit compatible address where all incoming transaction are linkable, he can either publish his tracking key or use a truncated address. That address represent only one public ec-key B, and the remaining part required by the protocol is derived from it as follows: a = s(B) and A = s(B)G. In both cases every person is able to “recognize” all of Bob’s incoming transaction, but, of course, none can spend the funds enclosed within them without the secret key b.

 

One-time ring signatures

A protocol based on one-time ring signatures allows users to achieve unconditional unlinkability. Unfortunately, ordinary types of cryptographic signatures permit to trace transactions to their respective senders and receivers. Our solution to this deficiency lies in using a different signature type than those currently used in electronic cash systems.

We will first provide a general description of our algorithm with no explicit reference to electronic cash.

A one-time ring signature contains four algorithms: (GEN, SIG, VER, LNK): GEN: takes public parameters and outputs an ec-pair (P, x) and a public key I.

SIG: takes a message m, a set S1 of public keys {Pi}i

and a set S = S1 ∪ {Ps}.

s, a pair (Ps, xs) and outputs a signature σ

VER: takes a message m, a set S, a signature σ and outputs “true” or “false”.

LNK: takes a set I = {Ii}, a signature σ and outputs “linked” or “indep”.

The idea behind the protocol is fairly simple: a user produces a signature which can be checked by a set of public keys rather than a unique public key. The identity of the signer is indistinguishable from the other users whose public keys are in the set until the owner produces a second signature using the same keypair.

image

Private keys

x0

· · ·

xn

xi

· · ·

Ring Signature

sign

Public keys

P0

· · ·

Pi

· · ·

Pn

Fig. 6. Ring signature anonymity.

verify

H

GEN: The signer picks a random secret key x [1, l 1] and computes the corresponding public key P = xG. Additionally he computes another public key I = x p(P ) which we will call the “key image”.

SIG: The signer generates a one-time ring signature with a non-interactive zero-knowledge proof using the techniques from [21]. He selects a random subset S1 of n from the other users’ public keys Pi, his own keypair (x, P ) and key image I. Let 0 s n be signer’s secret index in S (so that his public key is Ps).

{ | } { |

He picks a random qi i = 0 . . . n and wi i = 0 . . . n, i

following transformations:

s} from (1 . . . l) and applies the

i

L = qiG, if i = s qiG + wiPi, if i /= s

i

R = qiHp(Pi), if i = s qiHp(Pi) + wiI, if i =/ s

The next step is getting the non-interactive challenge:

c = Hs(m, L1, . . . , Ln, R1, . . . , Rn)

Finally the signer computes the response:

),

ci =

wi, if i /= s

c

n

i=0

ci mod l, if i = s

c

n

i=0

ci mod l, if i = s

i

r = qi, if i /= s

qs csx mod l, if i = s

The resulting signature is σ = (I, c1, . . . , cn, r1, . . . , rn).

VER: The verifier checks the signature by applying the inverse transformations:

L1i = riG + ciPi

Ri1 = riHp(Pi) + ciI

Finally, the verifier checks if

ci = Hs(m, L0, . . . , Ln, R0, . . . , Rn) mod l

),n ?

i=0

1 1 1 1

i=0

If this equality is correct, the verifier runs the algorithm LNK. Otherwise the verifier rejects

the signature.

I

LNK: The verifier checks if I has been used in past signatures (these values are stored in the set ). Multiple uses imply that two signatures were produced under the same secret key.

The meaning of the protocol: by applying L-transformations the signer proves that he knows such x that at least one Pi = xG. To make this proof non-repeatable we introduce the key image as I = xHp(P ). The signer uses the same coefficients (ri, ci) to prove almost the same statement: he knows such x that at least one Hp(Pi) = I · x1.

If the mapping x I is an injection:

 

Nobody can recover the public key from the key image and identify the signer;

 

The signer cannot make two signatures with different I’s and the same x. A full security analysis is provided in Appendix A.

 

Standard CryptoNote transaction

By combining both methods (unlinkable public keys and untraceable ring signature) Bob achieves new level of privacy in comparison with the original Bitcoin scheme. It requires him to store only one private key (a, b) and publish (A, B) to start receiving and sending anonymous transactions. While validating each transaction Bob additionally performs only two elliptic curve multi- plications and one addition per output to check if a transaction belongs to him. For his every output Bob recovers a one-time keypair (pi, Pi) and stores it in his wallet. Any inputs can be circumstantially proved to have the same owner only if they appear in a single transaction. In

fact this relationship is much harder to establish due to the one-time ring signature.

With a ring signature Bob can effectively hide every input among somebody else’s; all possible spenders will be equiprobable, even the previous owner (Alice) has no more information than any observer.

When signing his transaction Bob specifies n foreign outputs with the same amount as his output, mixing all of them without the participation of other users. Bob himself (as well as anybody else) does not know if any of these payments have been spent: an output can be used in thousands of signatures as an ambiguity factor and never as a target of hiding. The double spend check occurs in the LNK phase when checking against the used key images set.

Bob can choose the ambiguity degree on his own: n = 1 means that the probability he has spent the output is 50% probability, n = 99 gives 1%. The size of the resulting signature increases linearly as O(n + 1), so the improved anonymity costs to Bob extra transaction fees. He also can set n = 0 and make his ring signature to consist of only one element, however this will instantly reveal him as a spender.

Transaction

Sender’s output

image

Output1

Tx input

Destination key

Output0

One-time keypair

Destination key

Outputi

. . .

Outputn

One-time private key

. . .

Destination key

Outputn

Foreign transactions

P, x

I = xHp(P )

Key image

Signatures

Ring Signature

Fig. 7. Ring signature generation in a standard transaction.

 

Egalitarian Proof-of-work

In this section we propose and ground the new proof-of-work algorithm. Our primary goal is to close the gap between CPU (majority) and GPU/FPGA/ASIC (minority) miners. It is appropriate that some users can have a certain advantage over others, but their investments should grow at least linearly with the power. More generally, producing special-purpose devices has to be as less profitable as possible.

 

Related works

The original Bitcoin proof-of-work protocol uses the CPU-intensive pricing function SHA-256. It mainly consists of basic logical operators and relies solely on the computational speed of processor, therefore is perfectly suitable for multicore/conveyer implementation.

However, modern computers are not limited by the number of operations per second alone, but also by memory size. While some processors can be substantially faster than others [8], memory sizes are less likely to vary between machines.

Memory-bound price functions were first introduced by Abadi et al and were defined as “functions whose computation time is dominated by the time spent accessing memory” [15]. The main idea is to construct an algorithm allocating a large block of data (“scratchpad”) within memory that can be accessed relatively slowly (for example, RAM) and “accessing an unpredictable sequence of locations” within it. A block should be large enough to make preserving the data more advantageous than recomputing it for each access. The algorithm also should prevent internal parallelism, hence N simultaneous threads should require N times more memory at once.

Dwork et al [22] investigated and formalized this approach leading them to suggest another variant of the pricing function: “Mbound”. One more work belongs to F. Coelho [20], who

proposed the most effective solution: “Hokkaido”.

To our knowledge the last work based on the idea of pseudo-random searches in a big array is the algorithm known as “scrypt” by C. Percival [32]. Unlike the previous functions it focuses on key derivation, and not proof-of-work systems. Despite this fact scrypt can serve our purpose: it works well as a pricing function in the partial hash conversion problem such as SHA-256 in Bitcoin.

By now scrypt has already been applied in Litecoin [14] and some other Bitcoin forks. How- ever, its implementation is not really memory-bound: the ratio “memory access time / overall time” is not large enough because each instance uses only 128 KB. This permits GPU miners to be roughly 10 times more effective and continues to leave the possibility of creating relatively cheap but highly-efficient mining devices.

image

2

·

Moreover, the scrypt construction itself allows a linear trade-off between memory size and CPU speed due to the fact that every block in the scratchpad is derived only from the previous. For example, you can store every second block and recalculate the others in a lazy way, i.e. only when it becomes necessary. The pseudo-random indexes are assumed to be uniformly distributed, hence the expected value of the additional blocks’ recalculations is 1 N , where N is the number of iterations. The overall computation time increases less than by half because there are also time independent (constant time) operations such as preparing the scratchpad and hashing on

every iteration. Saving 2/3 of the memory costs 1 · N + 1 · 2 · N = N additional recalculations;

image image

image

3 3

· · ·

9/ 1 1 1

10 results in 10 N + . . . + 10 9 N = 4.5N . It is easy to show that storing only s of all blocks

2

increases the time less than by a factor of s1 . This in turn implies that a machine with a CPU

200 times faster than the modern chips can store only 320 bytes of the scratchpad.

 

The proposed algorithm

We propose a new memory-bound algorithm for the proof-of-work pricing function. It relies on random access to a slow memory and emphasizes latency dependence. As opposed to scrypt every new block (64 bytes in length) depends on all the previous blocks. As a result a hypothetical “memory-saver” should increase his calculation speed exponentially.

Our algorithm requires about 2 Mb per instance for the following reasons:

 

It fits in the L3 cache (per core) of modern processors, which should become mainstream in a few years;

 

A megabyte of internal memory is an almost unacceptable size for a modern ASIC pipeline;

 

GPUs may run hundreds of concurrent instances, but they are limited in other ways: GDDR5 memory is slower than the CPU L3 cache and remarkable for its bandwidth, not random access speed.

 

Significant expansion of the scratchpad would require an increase in iterations, which in turn implies an overall time increase. “Heavy” calls in a trust-less p2p network may lead to serious vulnerabilities, because nodes are obliged to check every new block’s proof-of-work. If a node spends a considerable amount of time on each hash evaluation, it can be easily DDoSed by a flood of fake objects with arbitrary work data (nonce values).

 

Further advantages

 

Smooth emission

The upper bound for the overall amount of CryptoNote digital coins is: MSupply = 264 1

atomic units. This is a natural restriction based only on implementation limits, not on intuition such as “N coins ought to be enough for anybody”.

To ensure the smoothness of the emission process we use the following formula for block rewards:

BaseReward = (MSupply A) » 18,

where A is amount of previously generated coins.

 

Adjustable parameters

 

Difficulty

CryptoNote contains a targeting algorithm which changes the difficulty of every block. This decreases the system’s reaction time when the network hashrate is intensely growing or shrinking, preserving a constant block rate. The original Bitcoin method calculates the relation of actual and target time-span between the last 2016 blocks and uses it as the multiplier for the current difficulty. Obviously this is unsuitable for rapid recalculations (because of large inertia) and results in oscillations.

The general idea behind our algorithm is to sum all the work completed by the nodes and divide it by the time they have spent. The measure of work is the corresponding difficulty values in each block. But due to inaccurate and untrusted timestamps we cannot determine the exact time interval between blocks. A user can shift his timestamp into the future and the next time intervals might be improbably small or even negative. Presumably there will be few incidents of this kind, so we can just sort the timestamps and cut-off the outliers (i.e. 20%). The range of the rest values is the time which was spent for 80% of the corresponding blocks.

 

Size limits

Users pay for storing the blockchain and shall be entitled to vote for its size. Every miner deals with the trade-off between balancing the costs and profit from the fees and sets his own “soft-limit” for creating blocks. Also the core rule for the maximum block size is necessary for preventing the blockchain from being flooded with bogus transaction, however this value should not be hard-coded.

·

Let MN be the median value of the last N blocks sizes. Then the “hard-limit” for the size of accepting blocks is 2 MN . It averts the blockchain from bloating but still allows the limit to slowly grow with time if necessary.

Transaction size does not need to be limited explicitly. It is bounded by the size of a block; and if somebody wants to create a huge transaction with hundreds of inputs/outputs (or with the high ambiguity degree in ring signatures), he can do so by paying sufficient fee.

 

Excess size penalty

A miner still has the ability to stuff a block full of his own zero-fee transactions up to its maximum size 2 · Mb. Even though only the majority of miners can shift the median value, there is still a

possibility to bloat the blockchain and produce an additional load on the nodes. To discourage malevolent participants from creating large blocks we introduce a penalty function:

NewReward = BaseReward ·

BlkSize 2

(

 

\

1

MN

·

This rule is applied only when BlkSize is greater than minimal free block size which should be close to max(10kb, MN 110%). Miners are permitted to create blocks of “usual size” and even exceed it with profit when the overall fees surpass the penalty. But fees are unlikely to grow quadratically unlike the penalty value so there will be an equilibrium.

 

 

Transaction scripts

CryptoNote has a very minimalistic scripting subsystem. A sender specifies an expression Φ =

i=1

f (x1, x2, . . . , xn), where n is the number of destination public keys {Pi}n . Only five binary

operators are supported: min, max, sum, mul and cmp. When the receiver spends this payment, he produces 0 k n signatures and passes them to transaction input. The verification process simply evaluates Φ with xi = 1 to check for a valid signature for the public key Pi, and xi = 0. A verifier accepts the proof iff Φ > 0.

Despite its simplicity this approach covers every possible case:

 

≤ ≤ ≥

 

Multi-/Threshold signature. For the Bitcoin-style “M-out-of-N” multi-signature (i.e. the receiver should provide at least 0 M N valid signatures) Φ = x1+x2+. . .+xN M (for clarity we are using common algebraic notation). The weighted threshold signature (some keys can be more important than other) could be expressed as Φ = w1 · x1 + w2 · x2 + . . . + wN · xN wM . And scenario where the master-key corresponds to Φ = max(M x, x1 + x2 + . . . + xN ) M . It is easy to show that any sophisticated case can be expressed with these operators, i.e. they form basis.

Password protection. Possession of a secret password s is equivalent to the knowledge of a private key, deterministically derived from the password: k = KDF(s). Hence, a receiver can prove that he knows the password by providing another signature under the key k. The sender simply adds the corresponding public key to his own output. Note that this method is much more secure than the “transaction puzzle” used in Bitcoin [13], where the password is explicitly passed in the inputs.

Degenerate cases. Φ = 1 means that anybody can spend the money; Φ = 0 marks the output as not spendable forever.

In the case when the output script combined with public keys is too large for a sender, he can use special output type, which indicates that the recipient will put this data in his input while the sender provides only a hash of it. This approach is similar to Bitcoin’s “pay-to-hash” feature, but instead of adding new script commands we handle this case at the data structure level.

 

Conclusion

We have investigated the major flaws in Bitcoin and proposed some possible solutions. These ad- vantageous features and our ongoing development make new electronic cash system CryptoNote a serious rival to Bitcoin, outclassing all its forks.

Nobel prize laureate Friedrich Hayek in his famous work proves that the existence of con- current independent currencies has a huge positive effect. Each currency issuer (or developer in our case) is trying to attract users by improving his product. Currency is like a commodity: it can have unique benefits and shortcomings and the most convenient and trusted currency has the greatest demand. Suppose we had a currency excelling Bitcoin: it means that Bitcoin would develop faster and become better. The biggest support as an open source project would come from its own users, who are interested in it.

We do not consider CryptoNote as a full replacement to Bitcoin. On the contrary, having two (or more) strong and convenient currencies is better than having just one. Running two and more different projects in parallel is the natural flow of electronic cash economics.

Nicolas van

Digitally signed by Nicolas van Saberhagen

image

Saberhagen

DN: cn=Nicolas van Saberhagen, email=nvsaberhagen@gmail.co

m

Date: 2013.10.17 14:44:50

+02’00’

A Security

We shall give a proof for our one-time ring signature scheme. At some point it coincides with the parts of the proof in [24], but we decided to rewrite them with a reference rather than to force a reader to rush about from one paper to another.

These are the properties to be established:

i=1

 

Linkability. Given all the secret keys {xi}n for a set S it is impossible to produce n + 1

valid signatures σ1, σ2, . . . , σn+1, such that all of them pass the LNK phase (i.e. with n + 1 different key images Ii). This property implies the double spending protection in the context of CryptoNote.

 

Exculpability. Given set S, at most n 1 corresponding private keys xi (excluding i = j) and the image Ij of the keys xj it is impossible to produce a valid signature σ with Ij. This property implies theft protection in the context of CryptoNote.

 

S

Unforgeability. Given only a public keys set it is impossible to produce a valid signature

σ.

n

 

S

Anonymity. Given a signature σ and the corresponding set it is impossible to determine the secret index j of the signer with a probability p > 1 .

Linkability

Theorem 1. Our one-time ring signature scheme is linkable under the random oracle model.

Proof. Suppose an adversary can produce n + 1 valid signatures σi with key images Ii /= Ij for any i, j [1 . . . n]. Since #S = n, at least one Ii /= xiHp(Pi) for every i. Consider the corresponding signature σ = (I, c1, . . . , cn, r1, . . . , rn). VER(σ) = “true”, this means that

L1i = riG + ciPi

Ri1 = riHp(Pi) + ciI

ci = Hs(m, L1, . . . , Ln, R1, . . . , Rn) mod l

),n

1 1 1 1

i=1

The first two equalities imply

logG L1i = ri + cixi

logHp(Pi) Ri1 = ri + ci logHp(Pi) I

where logA B informally denotes the discrete logarithm of B to the base A.

image

H

As in [24] we note that i : xi = logHp(Pi) I implies that all ci’s are uniquely determined. The third equality forces the adversary to find a pre-image of s to succeed in the attack, an event whose probability is considered to be negligible.

Exculpability

Theorem 2. Our one-time ring signature scheme is exculpable under the discrete logarithm assumption in the random oracle model.

Proof. Suppose an adversary can produce a valid signature σ = (I, c1, . . . , cn, r1, . . . , rn) with I = xjHP (Pj) with given {xi | i = 1, . . . , j 1, j +1, . . . , n}. Then, we can construct an algorithm A which solves the discrete logarithm problem in E(Fq).

S

Suppose inst = (G, P ) E(Fq) is a given instance of the DLP and the goal is to get s, such that P = sG. Using the standard technique (as in [24]), A simulates the random and signing oracles and makes the adversary produce two valid signatures with Pj = P in the set : σ = (I, c1, . . . , cn, r1, . . . , rn) and σ1 = (I, c11, . . . , c1n, r11 , . . . , rn1 ).

Since I = x H (P ) in both signatures we compute x

= log

I = rj rjt

image

image

mod l

j p j

j Hp(Pj )

ctj cj

A outputs xj because Lj = rjG + cjPj = rj1 G + c1j Pj and Pj = P .

Unforgeability

It has been shown in [24] that unforgeability is just an implication of both linkability and excul- pability.

Theorem 3. If a one-time ring signature scheme is linkable and exculpable, then it is unforgeable.

Proof. Suppose an adversary can forge a signature for a given set S: σ0 = (I0, . . .). Consider all valid signatures (produced by the honest signers) for the same message m and the set S: σ1, σ2, . . . , σn. There are two possible cases:

i=1

1. I0 ∈ {Ii}n

i=1

2. I0 /∈ {Ii}n

. Which contradicts exculpability.

image

. Which contradicts linkability.

Anonymity

Theorem 4. Our one-time ring signature scheme is anonymous under the decisional Diffie- Hellman assumption in the random oracle model.

Proof. Suppose an adversary can determine the secret index j of the Signer with a probability

p = 1 + E. Then, we can construct algorithm A which solves the decisional Diffie-Hellman

image image

n 1 E

problem in E(Fq) with the probability 2 + 2 .

Let inst = (G1, G2, Q1, Q2) E(Fq) be the instance of DDH and the goal to determine if

logG1 Q1 = logG2 Q2. A feeds the adversary with valid signature σ0 = (I, . . .), where Pj =

xjG1 = Q1 and I = Q2 and simulates oracle Hp, returning G2 for query Hp(Pj).

image

image

2

2

The adversary returns k as his guess for the index i : I = xiHP (Pi). If k = j, then A returns 1 (for “yes”) otherwise a random r ∈ {1, 0}. The probability of the right choice is com- puted as in [24]: 1 +Pr (1 | inst DDH)Pr (1 | inst / DDH) = 1 +Pr (k = j | inst DDH)+

Pr (k /= j | inst DDH)·Pr (r = 1)Pr (k = j | inst / DDH)Pr (k /= j | inst / DDH)·Pr (r = 0) =

image

image

image

image

image

2

n

n

2

n

n

2

2

2

1 + 1 + E + ( n1 E) · 1 1n1 · 1 = 1 + E

image

H

In fact, the result should be reduced by the probability of collision in s, but this value is considered to be negligible.

Notes on the hash function Hp

H → H

We defined p as deterministic hash function E(Fq) E(Fq). None of the proofs demands p to be an ideal cryptographic hash function. It’s main purpose is to get a pseudo-random base for image key I = xHp(xG) in some determined way.

With fixed base (I = xG2) the following scenario is possible:

 

Alice sends two standard transactions to Bob, generating one-time tx-keys: P2 = Hs(r1A)G+

B and P1 = Hs(r2A)G + B.

 

Bob recovers corresponding one-time private tx-keys x1 and x2 and spends the outputs with valid signatures and images keys I1 = x1G2 and I2 = x2G2.

 

Now Alice can link these signatures, checking the equality I1I2 =? (Hs(r1A)−Hs(r2A))G2.

The problem is that Alice knows the linear correlation between public keys P1 and P2 and in case of fixed base G2 she also gets the same correlation between key images I1 and I2. Replacing G2 with Hp(xG2), which does not preserve linearity, fixes that flaw.

For constructing deterministic Hp we use algorithm presented in [37].

References

 

http://bitcoin.org.

image

 

https://en.bitcoin.it/wiki/Category:Mixing Services.

 

http://blog.ezyang.com/2012/07/secure-multiparty-bitcoin-anonymization.

 

https://bitcointalk.org/index.php?topic=279249.0.

 

http://msrvideo.vo.msecnd.net/rmcvideos/192058/dl/192058.pdf.

 

https://github.com/bitcoin/bips/blob/master/bip-0034.mediawiki#Specification.

image

 

https://github.com/bitcoin/bips/blob/master/bip-0016.mediawiki#Backwards Compatibility.

image

image

 

https://en.bitcoin.it/wiki/Mining hardware comparison.

 

https://github.com/bitcoin/bips/blob/master/bip-0050.mediawiki.

 

http://luke.dashjr.org/programs/bitcoin/files/charts/branches.html.

 

https://bitcointalk.org/index.php?topic=196259.0.

 

https://en.bitcoin.it/wiki/Contracts.

 

https://en.bitcoin.it/wiki/Script.

 

http://litecoin.org.

 

Mart´ın Abadi, Michael Burrows, and Ted Wobber. Moderately hard, memory-bound func- tions. In NDSS, 2003.

 

Ben Adida, Susan Hohenberger, and Ronald L. Rivest. Ad-hoc-group signatures from hi- jacked keypairs. In in DIMACS Workshop on Theft in E-Commerce, 2005.

 

Man Ho Au, Sherman S. M. Chow, Willy Susilo, and Patrick P. Tsang. Short linkable ring signatures revisited. In EuroPKI, pages 101–115, 2006.

 

Daniel J. Bernstein, Niels Duif, Tanja Lange, Peter Schwabe, and Bo-Yin Yang. High-speed high-security signatures. J. Cryptographic Engineering, 2(2):77–89, 2012.

 

David Chaum and Eug`ene van Heyst. Group signatures. In EUROCRYPT, pages 257–265, 1991.

 

Fabien Coelho. Exponential memory-bound functions for proof of work protocols. IACR Cryptology ePrint Archive, 2005:356, 2005.

 

Ronald Cramer, Ivan Damg˚ard, and Berry Schoenmakers. Proofs of partial knowledge and simplified design of witness hiding protocols. In CRYPTO, pages 174–187, 1994.

 

Cynthia Dwork, Andrew Goldberg, and Moni Naor. On memory-bound functions for fighting spam. In CRYPTO, pages 426–444, 2003.

 

Eiichiro Fujisaki. Sub-linear size traceable ring signatures without random oracles. In CT- RSA, pages 393–415, 2011.

 

Eiichiro Fujisaki and Koutarou Suzuki. Traceable ring signature. In Public Key Cryptogra- phy, pages 181–200, 2007.

 

Jezz Garzik. Peer review of “quantitative analysis of the full bitcoin transaction graph”. https://gist.github.com/3901921, 2012.

 

Joseph K. Liu, Victor K. Wei, and Duncan S. Wong. Linkable spontaneous anonymous group signature for ad hoc groups (extended abstract). In ACISP, pages 325–335, 2004.

 

Joseph K. Liu and Duncan S. Wong. Linkable ring signatures: Security models and new schemes. In ICCSA (2), pages 614–623, 2005.

 

Ian Miers, Christina Garman, Matthew Green, and Aviel D. Rubin. Zerocoin: Anonymous distributed e-cash from bitcoin. In IEEE Symposium on Security and Privacy, pages 397– 411, 2013.

 

Micha Ober, Stefan Katzenbeisser, and Kay Hamacher. Structure and anonymity of the bitcoin transaction graph. Future internet, 5(2):237–250, 2013.

 

Tatsuaki Okamoto and Kazuo Ohta. Universal electronic cash. In CRYPTO, pages 324–337, 1991.

 

Marc Santamaria Ortega. The bitcoin transaction graph — anonymity. Master’s thesis, Universitat Oberta de Catalunya, June 2013.

 

Colin Percival. Stronger key derivation via sequential memory-hard functions. Presented at BSDCan’09, May 2009.

 

Fergal Reid and Martin Harrigan. An analysis of anonymity in the bitcoin system. CoRR, abs/1107.4524, 2011.

 

Ronald L. Rivest, Adi Shamir, and Yael Tauman. How to leak a secret. In ASIACRYPT, pages 552–565, 2001.

 

Dorit Ron and Adi Shamir. Quantitative analysis of the full bitcoin transaction graph.

IACR Cryptology ePrint Archive, 2012:584, 2012.

 

Meni Rosenfeld. Analysis of hashrate-based double-spending. 2012.

 

Maciej Ulas. Rational points on certain hyperelliptic curves over finite fields. Bulletin of the Polish Academy of Sciences. Mathematics, 55(2):97–104, 2007.

 

Qianhong Wu, Willy Susilo, Yi Mu, and Fangguo Zhang. Ad hoc group signatures. In

IWSEC, pages 120–135, 2006.

 

 

Steem

Highlight to Annotate. Click to Learn.

An incentivized, blockchain-based, public content platform.

August 2017

 

Abstract

Steem is a blockchain database that supports community building and social interaction with cryptocurrency rewards. Steem combines concepts from social media with lessons learned from building cryptocurrencies and their communities. An important key to inspiring participation in any community, currency or free market economy is a fair accounting system that consistently reflects each person’s contribution. Steem is the first cryptocurrency that attempts to accurately and transparently reward an unbounded number of individuals who make subjective contributions to its community.

Table of Contents

Introduction

Collectively, user-generated content has created billions of dollars worth of value for the shareholders of social media companies, such as Reddit, Facebook, and Twitter. In 2014, Reddit hypothesized that its platform would be improved if everyone who contributed to reddit.com by posting stories, adding comments or voting were rewarded with a fair share in Reddit, Inc.[^1]. Steem aims to support social media and online communities by returning much of its value to the people who provide valuable contributions by rewarding them with cryptocurrency, and through this process create a currency that is able to reach a broad market, including people who have yet to participate in any cryptocurrency economy.

There are some key principles that have been used to guide the design of Steem. The most important principle is that everyone who contributes to a venture should receive pro-rata ownership, payment or debt from the venture. This principle is the same principle that is applied to all startups as they allocate shares at founding and during subsequent funding rounds.

The second principle is that all forms of capital are equally valuable. This means that those who contribute their scarce time and attention toward producing and curating content for others are just as valuable as those who contribute their scarce cash. This is the sweat equity principle[^2] and is a concept that prior cryptocurrencies have often had trouble providing to more than a few dozen individuals.

The third principle is that the community produces products to serve its members. This principle is exemplified by credit unions, food co-ops, and health sharing plans, which serve the members of their community rather than sell products or services to people outside the community.

The Steem community provides the following services to its members:

  1. A source of curated news and commentary.
  2. A means to get high quality answers to personalized questions.
  3. A stable cryptocurrency pegged to the U.S. dollar.
  4. Free payments.
  5. Jobs providing above services to other members.

Steem’s purposeful realignment of economic incentives has the potential to produce fairer and more inclusive results for everyone involved than the social media and cryptocurrency platforms that have gone before it. This paper will explore the existing economic incentives and demonstrate how Steem’s incentives may result in better outcomes for most participants.

Recognizing Contribution

Steem is designed from the ground up to address the major barriers to adoption and monetization of a social media based economy. Our thesis is that the same techniques used to grow major social media platforms can be used to bootstrap a successful cryptocurrency. Economic incentives enabled by cryptocurrency can dramatically facilitate the growth of a new social media platform. It is the synergy between cryptocurrency and social media that we believe may give Steem a powerful advantage in the market.

The challenge faced by Steem is deriving an algorithm for scoring individual contributions that most community members consider to be a fair assessment of the subjective value of each contribution. In a perfect world, community members would cooperate to rate each other’s contribution and derive a fair compensation. In the real world, algorithms must be designed in such a manner that they are resistant to intentional manipulation for profit. Any widespread abuse of the scoring system could cause community members to lose faith in the perceived fairness of the economic system.

Existing platforms operate on a one-user, one-vote principle. This creates an environment where rankings can be manipulated by sybil attacks and the service providers must pro-actively identify and block abusers. People already attempt to manipulate the Reddit, Facebook, and Twitter scoring algorithms when the only reward is web traffic or censorship.

The fundamental unit of account on the Steem platform is STEEM, a crypto currency token. Steem operates on the basis of one-STEEM, one-vote. Under this model, individuals who have contributed the most to the platform, as measured by their account balance, have the most influence over how contributions are scored. Furthermore, Steem only allows members to vote with STEEM when it is committed to a vesting schedule. Under this model, members have a financial incentive to vote in a way that maximises the long term value of their STEEM.

Steem is designed around a relatively simple concept: everyone’s meaningful contribution to the community should be recognized for the value it adds. When people are recognized for their meaningful contributions, they continue contributing and the community grows. Any imbalance in the give and take within a community is unsustainable. Eventually the givers grow tired of supporting the takers and disengage from the community.

The challenge is creating a system capable of identifying what contributions are needed and their relative worth in a way that can scale to an unbounded number of people.

A proven system for evaluating and rewarding contributions is the free market. The free market can be viewed as a single community where everyone trades with one another and rewards are allocated by profit and loss. The market system rewards those who provide value to others and punishes those who consume more value than they produce. The free market supports many different currencies and money is simply a commodity that everyone finds easy to exchange.

Since the free market is a proven system, it is tempting to try to create a free-market system where content consumers directly pay content producers. However, direct payment is inefficient and not really viable for content creation and curation. The value of most content is so low relative to the cognitive, financial, and opportunity costs associated with making a payment that few readers choose to tip. The abundance of free alternatives means that enforcing a ‘paywall’ will drive readers elsewhere. There have been several attempts to implement per-article micropayments from readers to authors, but none have become widespread.

Steem is designed to enable effective micropayments for all kinds of contribution by changing the economic equation. Readers no longer have to decide whether or not they want to pay someone from their own pocket, instead they can vote content up or down and Steem will use their votes to determine individual rewards. This means that people are given a familiar and widely used interface and no longer face the cognitive, financial, and opportunity costs associated traditional micropayment and tipping platforms.

Voting input from community members is critical for Steem to accurately allocate payments to contributors. Voting can therefore be viewed as a crucial contribution and worthy of rewards on its own. Some platforms, such as Slashdot, use meta-moderation[^3] as a way to rank and reward honest moderators. Steem chooses to reward those who contribute the most to the total promotion of a piece of content and rewards the voters proportional to the ultimate reward paid to the content creator.

Ways to Contribute

This section outlines the ideas behind Steem and its rewards for people who provide meaningful and measurable contributions to the Steem community.

Capital Contributions

There are two items a community can offer to attract capital: debt and ownership. Those who buy ownership profit when the community grows but lose if the community shrinks. Those who buy debt are guaranteed a certain amount of interest but do not get to participate in any profits realized by the growth of the community. Both types of capital contributions are valuable to the growth of the community and value of its currency. Additionally there are two ways ownership can be held: liquid and vesting. Vesting ownership makes a long-term commitment and cannot be sold for a minimum period of time.

The Steem network calls these different asset classes Steem (STEEM), Steem Power (SP), and Steem Dollars (SBD).

Steem (STEEM)

Steem is the fundamental unit of account on the Steem blockchain. All other tokens derive their value from the value of STEEM. STEEM is a liquid currency, and therefore can be bought or sold on exchanges, as well as transferred to other users as a form of payment.

Steem Power (SP)

Start up companies require long-term capital commitment. Those who invest their money in a startup expect to wait years before they can sell their shares and realize their profits. Without long-term commitment, a startup seeking to raise additional capital through the sale of additional shares would be competing with existing shareholders looking to exit. Savvy investors want their capital contributions to grow the company, but growth cannot happen if the new capital is given away to those looking to exit.

There is significant value to having long-term commitment because it enables communities to make long-term plans. Long term commitment of stakeholders also causes them to vote for long-term growth rather than short-term pumps.

In the cryptocurrency space, speculators jump from cryptocurrency to cryptocurrency based mostly on which one is expected to have short-term growth. Steem wants to build a community that is mostly owned and entirely controlled by those with a long-term perspective.

Users are able to commit their STEEM to a thirteen week vesting schedule, providing them with additional benefits within the platform. STEEM that has been committed to a thirteen week vesting schedule is called Steem Power (SP). SP balances are non-transferrable and non-divisible except via the automatically recurring conversion requests. This means that SP cannot be easily traded on cryptocurrency exchanges.

When users vote on content, their influence over the distribution of the rewards pool is directly proportional to the amount of SP that they have. Users with more SP have more influence on the distribution of rewards. This means that SP is an access token that grants its holders exclusive powers within the Steem platform.

SP holders are also paid interest on the balance of SP that remains vested. 15% of the yearly inflation is paid to SP holders as interest. The amount of the interest that they receive is directly proportional to the amount of SP they hold relative to the total amount of vested SP across all users.

Transferring from STEEM to SP is referred to as “powering up”, while transferring from SP to STEEM is referred to as “powering down.” SP that is powered down is returned to the user over a period of thirteen weeks, via 13 equal weekly payments, starting one week after the power down is initiated.

Steem Dollars (SBD)

Stability is an important feature of successful global economies. Without stability, individuals across the world could not have low cognitive costs while engaging in commerce and savings. Because stability is an important feature of successful economies, Steem Dollars were designed as an attempt to bring stability to the world of cryptocurrency and to the individuals who use the Steem network.

Steem Dollars are created by a mechanism similar to convertible notes, which are often used to fund startups. In the startup world, convertible notes are short-term debt instruments that can be converted to ownership at a rate determined in the future, typically during a future funding round. A blockchain based token can be viewed as ownership in the community whereas a convertible note can be viewed as a debt denominated in any other commodity or currency. The terms of the convertible note allow the holder to convert to the backing token with a minimum notice at the fair market price of the token. Creating token-convertible-dollars enables blockchains to grow their network effect while maximizing the return for token holders.

Steem Dollars are referred to with the symbol SBD, an acronym for Steem Blockchain Dollars. Creating SBD requires a combination of a reliable price feed, and rules to prevent abuse. Providing a reliable price feed involves three factors: minimizing the impact of an incorrect feed, maximizing the cost of producing an incorrect feed, and minimizing the importance of timing.

Minimizing Fraudulent Feeds

SP holders elect individuals, called witnesses, to publish price feeds. The elected witnesses are presumably trusted by those who have a vested interest in the quality of the feed. By paying those who are elected, Steem creates market competition to earn the right to produce feeds. The more the feed producers are paid the more they have to lose by publishing false information.

Given a set of trusted and elected feed producers, the actual price used for conversions can be derived as the median of the feeds. In this way if any minority of individual feed producers produce outliers they have minimal impact on the actual median while still having the ability impact their reputation.

Even if all feed producers are honest, it is possible for the majority of feed producers to be impacted by events beyond their control. The Steem network is designed to tolerate short-term corruption of the median price feed while the community actively works to correct the issue. One example of an issue that may take some time to correct is short-term market manipulation. Market manipulation is difficult and expensive to maintain for long periods of time. Another example would be the failure of a centralized exchange or the corruption of the data published by the exchange.

Steem factors out short-term price fluctuations by using the median price over a period of three and a half days. The median published feed is sampled every hour on the hour.

As long as the price feed corruption lasts for less than half the moving median time window it will have minimal impact on the conversion price. In the event the feed does get corrupted, network participants will have an opportunity to vote-out corrupt feed producers before the corrupted feed can impact the actual conversion price. Perhaps more importantly, it gives feed producers an opportunity to detect and correct issues before their feeds start impacting the price.

With a three and a half day window, community members have approximately one and a half days to respond to any issues that come up.

Mitigating Timing Attacks

Market participants have access to information faster than the blockchain’s three and a half day moving median conversion price can react. This information could be used to benefit of traders at the expense of the community. If there is a sudden increase in the value of STEEM traders could request conversion of their SBD at the old, lower price, and then sell the STEEM they receive a the new higher price with minimal risk.

Steem levels the playing field by requiring all conversion requests to be delayed for three and a half days. This means that neither the traders nor the blockchain has any information advantage regarding the price at the time the conversion is executed.

Minimizing Abuse of Conversions

If people could freely convert in both directions then traders could take advantage of the blockchains conversion rates by trading large volumes without changing the price. Traders who see a massive run up in price would convert to SBD at the high price (when it is most risky) and then convert back after the correction. The Steem protocol protects the community from this kind of abuse by only allowing people to convert from SBD to STEEM and not the other way around.

The blockchain decides how and when to create SBD and who should get it. This keeps the rate of SBD creation stable and removes most avenues of abuse.

Sustainable Debt to Ownership Ratios

If a token is viewed as ownership in the whole supply of tokens, then a token-convertible-dollar can be viewed as debt. If the debt to ownership ratio gets too high the entire currency can become unstable. Debt conversions can dramatically increase the token supply, which in turn is sold on the market suppressing the price. Subsequent conversions require the issuance of even more tokens. Left unchecked the system can collapse leaving worthless ownership backing a mountain of debt. The higher the debt to ownership ratio becomes the less willing new investors are to bring capital to the table.

A rapid change in the value of STEEM can dramatically change the debt-to-ownership ratio. The blockchain prevents the debt-to-ownership ratio from getting too high, by reducing the amount of STEEM awarded through SBD conversions if the debt level were to exceed 10%. If the amount of SBD debt ever exceeds 10% of the total STEEM market cap, the blockchain will automatically reduce the amount of STEEM generated through conversions to a maximum of 10% of the market cap. This ensures that the blockchain will never have higher than a 10% debt-to-ownership ratio.

The percentage floors used to compute STEEM creation are based on the supply including the STEEM value of all outstanding SBD and SP (as determined by the current rate / feed).

Interest

SBD pays holders interest. The interest rate is set by the same people who publish the price feed so that it can adapt to changing market conditions. All debt carries risk to the lender. Someone who holds SBD without redeeming it is effectively lending the community the value of a dollar. They are trusting that at some point in the future someone will be willing to buy the SBD from them for a dollar or that there will be speculators and investors willing to buy the STEEM they convert it into.

STEEM and SP holders gain leverage when members of the community are willing to hold SBD. This leverage amplifies the gains from growth while also contributing to growth. STEEM holders do suffer from increased dilution if the price falls. Cryptocurrency projects have shown that the gains from increasing the user base willing to trust the network with capital ultimately add more value to the network than any dilution that may occur during a downturn.

Setting Price Feeds

Astute readers will recognize that an interest bearing asset of limited supply may trade higher or lower than the underlying asset depending upon other opportunities to earn interest on the same asset. With a high interest rate paid on an asset pegged to the US dollar many people will bid up the limited supply of Steem Dollars until they are no longer valued at $1. In economics there is a principle known as the Impossible Trinity[^4] which states that it is impossible to have all three of the following at the same time:

  1. A stable exchange rate
  2. Free capital movement
  3. An independent monetary policy

If Steem feed producers aim to have an independent monetary policy allowing it to create and destroy Steem Dollars while simultaneously having full control over the interest rate then they will encounter problems. The Impossible Trinity says that Steem Dollars either need to restrict capital movement, have an unstable exchange rate with the dollar, or have limited control over the interest rate.

The primary concern of Steem feed producers is to maintain a stable one-to-one conversion between SBD and the U.S. Dollar (USD). Any time SBD is consistently trading above $1.00 USD interest payments must be stopped. In a market where 0% interest on debt still demands a premium, it is safe to say the market is willing to extend more credit than the debt the community is willing to take on. If this happens a SBD will be valued at more than $1.00 and there is little the community can do without charging negative interest rates.

If the debt-to-ownership ratio is low and SBD is trading for less than $1.00, then the interest rate should be increased. This will encourage more people to hold their SBD and support the price.

If SBD trades for less than $1.00 USD and the debt-to-ownership ratio is high, then the feeds should be adjusted upward give more STEEM per SBD. This will increase demand for SBD while also reducing the debt-to-ownership ratio and returning SBD to parity with USD.

Assuming the value of STEEM is growing faster than Steem is creating new SBD, the debt-to-ownership ratio should remain under the target ratio and the interest offered benefits everyone. If the value of the network is flat or falling, then any interest offered will only make the debt-to-ownership ratio worse.

In effect, feed producers are entrusted with the responsibility of setting monetary policy for the purpose of maintaining a stable peg to the USD. Abuse of this power can harm the value of STEEM so SP holders are wise to vote for witnesses that can be counted on to adjust the price feed and interest rates according to the rules outlined above.

If the debt-to-ownership ratio gets dangerously high and market participants choose to avoid conversion requests, then the feed should be adjusted to increase the rate at which STEEM paid for converting SBD.

Changes to the interest rate policy and/or any premiums/discounts on the STEEM/SBD conversion rate should be a slow and measured response to long-term average deviations rather than attempting to respond to short-term market conditions.

It is our belief that these rules will give market participants confidence that they are unlikely lose money by holding SBD purchased at a price of $1.00. We fully expect there to be a narrow trading range between $0.95 and $1.05 for SBD under normal market conditions.

Subjective Contributions

Subjective Proof of Work presents an alternative approach to distributing a currency that improves upon fully objective Proof of Work systems such as mining. The applications of a currency implementing subjective proof of work are far wider than any objective proof of work system because they can be applied to build a community around any concept that has a sufficiently defined purpose. When individuals join a community they buy into a particular set of beliefs and can vote to reinforce the community values or purpose.

In effect, the criteria by which work is evaluated is completely subjective and its definition lives outside the source code itself. One community may wish to reward artists, another poets, and another comedians. Other communities may choose to reward charitable causes or help advance political agendas.

The value each currency achieves depends upon the demand for influence within a particular community and how large the market believes each community can get. Unlike prior systems, subjective proof of work enables a community to collectively fund the development of whatever it finds valuable and enables the monetization of previously non monetizable time.

Distributing Currency

There are two ways people can get involved with a crypto-currency community: they can buy in, or they can work in. In both cases users are adding value to the currency, however, the vast majority of people have more free time than they do spare cash. Imagine the goal of bootstrapping a currency in a poor community with no actual cash but plenty of time. If people can earn money by working for one another then they will bootstrap value through mutual exchange facilitated by a fair accounting/currency system.

Distributing a currency to as many people as possible in a manner that is generally perceived as fair is a challenging task. The tasks that can be entirely evaluated by an objective computer algorithm are limited in nature and generally speaking have limited positive external benefits. In the case of Bitcoin-style mining, it can result in the production of specialized hardware and cause people to invest time developing more efficient algorithms. It may even help find prime numbers, but none of these things provide meaningful value to society or the currency holding community at large. More importantly, economies of scale and market forces will end up excluding everyone but experts from participating in this kind of distribution. Ultimately, computation-based mining is just another way of buying in because it requires money to pay the electric bill or the development of hardware necessary to do the work.

In order to give everyone an equal opportunity to get involved and earn the currency people must be given an opportunity to work. The challenge is how to judge the relative quality and quantity of work that individuals provide and to do so in a way that efficiently allocates rewards to millions of users. This requires the introduction of a scalable voting process. In particular it requires that authority to allocate funds must be as distributed and decentralized as possible.

The first step in rewarding millions of users is to commit to distributing a fixed amount of currency regardless of how much work is actually done or how users vote. This changes the question from being “Should we pay?” to “Whom should we pay?” and signals to the market that money is being distributed and is being auctioned off to whoever “bids” the most work. This is similar to Bitcoin committing to award 50 BTC to whoever finds the most difficult hashes. Like Bitcoin, all work must be done prior-to payout and nothing should be paid speculatively on the promise to do work in the future.

The next step is to reward everyone who does anything even remotely positive with something. This is accomplished by ranking all work done and distributing proportionally to its value. The more competitive the market becomes, the more difficult (higher quality or quantity) it becomes to earn the same payout.

Voting on Distribution of Currency

Assume there is a fixed amount of money to distribute, and that those who have a long-term vested interest in the future value and utility of the currency are the ones who must decide how to allocate it. Every vesting user casts their votes on who did the best work and at the end of the day the available money for that day is divided proportional to the votes such that everyone with even one net positive vote gets something.

The naive voting process creates a N-Person Prisoner’s Dilemma[^5] whereby each individual voter has incentive to vote for themselves at the expense of the larger community goal. If every voter defects by voting for themselves then no currency will end up distributed and the currency as a whole will fail to gain network effect. On the other hand, if only one voter defects then that voter would win undeserved profits while having minimal effect on the overall value of the currency.

Voting Abuse

Regardless of how much money any one individual has, there are always many other individuals with similar wealth. Even the wealthiest individual rarely has much more than the next couple wealthiest combined. Furthermore, those who have a large investment in a community also have the most to lose by attempting to game the voting system for themselves. It would be like the CEO of a company deciding to stop paying salaries so he could pocket all of the profits. Everyone would leave to work for other companies and the company would become worthless, leaving the CEO bankrupt rather than wealthy.

Fortunately, any work that is getting a large concentration of votes is also gaining the most scrutiny (publicity). Through the addition of negative-voting it is possible for many smaller stakeholders to nullify the voting power of collusive groups or defecting large stakeholders. Furthermore, large-stakeholders have more to lose if the currency falls in value due to abuse than they might gain by voting for themselves. In fact, honest large stakeholders are likely to be more effective by policing abuse and using negative voting than they would be by voting for smaller contributions.

The use of negative-voting to keep people from abusing the system leverages the crab mentality that many people have when it is perceived that one individual is profiting at the expense of everyone else. While crab mentality normally refers to short-sighted people keeping good people down, it is also what allows good people to keep bad people down. The only “problem” with crab mentality is when people wrongly believe someone is profiting at everyone else’s expense.

The Story of the Crab Bucket[^6]

A man was walking along the beach and saw another man fishing in the surf with a bait bucket beside him. As he drew closer, he saw that the bait bucket had no lid and had live crabs inside.

“Why don’t you cover your bait bucket so the crabs won’t escape?”, he said.

“You don’t understand.”, the man replied, “If there is one crab in the bucket it would surely crawl out very quickly. However, when there are many crabs in the bucket, if one tries to crawl up the side, the others grab hold of it and pull it back down so that it will share the same fate as the rest of them.”

So it is with people. If one tries to do something different, get better grades, improve herself, escape her environment, or dream big dreams, other people will try to drag her back down to share their fate.

Eliminating “abuse” is not possible and shouldn’t be the goal. Even those who are attempting to “abuse” the system are still doing work. Any compensation they get for their successful attempts at abuse or collusion is at least as valuable for the purpose of distributing the currency as the make-work system employed by traditional Bitcoin mining or the collusive mining done via mining pools. All that is necessary is to ensure that abuse isn’t so rampant that it undermines the incentive to do real work in support of the community and its currency.

The goal of building a community currency is to get more “crabs in the bucket”. Going to extreme measures to eliminate all abuse is like attempting to put a lid on the bucket to prevent a few crabs from escaping and comes at the expense of making it harder to add new crabs to the bucket. It is sufficient to make the walls slippery and give the other crabs sufficient power to prevent others from escaping.

Rate Limited Voting

A major part of minimizing abuse is the rate-limiting of voting. Individual users can only read and evaluate so many work items per day. Any attempt to vote more frequently than this is a sign of automation and potential abuse. Through rate limiting, stakeholders who vote more frequently have each vote count for less than stakeholders who vote less frequently. Attempts to divide tokens among multiple accounts also divides influence and therefore does not result in a net increase in influence nor bypass the rate-limit imposed on voting.

Users are allotted a fixed amount of voting power. Voting power is multiplied by a user’s vesting tokens to determine how much share in the reward pool should be allocated to a given work item. Every vote that is cast uses a percentage of remaining voting power. Users can vote for more posts, but each vote will be worth less, and it will take longer to reach full voting power again. Voting power recharges at a fixed linear rate of 20% per day.

Payout Distribution

One of the primary goals of Steem’s reward system is to produce the best discussions on the internet. Each and every year 75% of the yearly inflation is distributed to users submitting, voting on, and discussing content. At the size of Bitcoin this could be several million dollars per day being given to the top contributors.

The actual distribution will depend upon the voting patterns of users, but we suspect that the vast majority of the rewards will be distributed to the most popular content.

Zipf’s Law[^7] is one of those empirical rules that characterize a surprising range of real-world phenomena remarkably well. It says that if we order some large collection by size or popularity, the second element in the collection will be about half the measure of the first one, the third one will be about one-third the measure of the first one, and so on. In general, the k th-ranked item will measure about 1/k of the first one.

Taking popularity as a rough measure of value, then the value of each individual item is given by Zipf’s Law. That is, if we have a million items, then the most popular 100 will contribute a third of the total value, the next 10,000 another third, and the remaining 989,900 the final third. The value of the collection of n items is proportional to log(n).

 

The impact of this voting and payout distribution is to offer large bounties for good content while still rewarding smaller players for their long-tail contribution.

The economic effect of this is similar to a lottery where people overestimate their probability of getting votes and thus do more work than the expected value of their reward and thereby maximize the total amount of work performed in service of the community. The fact that everyone “wins something” plays on the same psychology that casinos use to keep people gambling. In other words, small rewards help reinforce the idea that it is possible to earn bigger rewards.

Payouts

When a post receives a payout it takes the form of 50% SBD and 50% SP. The Steem Power give the user increased voting and transaction power while the SBD gives the user an immediate benefit in a stable currency. As we’ve already discussed at length, SP is designed to encourage long-term holding rather than short-term selling. This encourages more users to have a vested interest in the long-term success of the platform.

Users also have the option to be paid in 100% SP, as well as decline payout on posts. When a user declines payout on a post, the money that would have been paid to them remains in the rewards pool, to be distributed to other users.

Consensus Algorithm

Consensus is the process by which a community comes to a universally recognized, unambiguous agreement on piece of information. There are many algorithms society has developed for reaching consensus about who owns what. Every government on earth is a primitive consensus algorithm whereby the population agrees to abide by a certain set of rules enshrined in a constitution. Governments establish courts, judges, and juries to interpret the subjective facts and render a final decision. Most of the time people abide by the decision even if it was wrong.

The algorithms used by cryptocurrencies provide a better way to reach consensus. Cryptographically signed testimony from individuals is recorded in a public ledger that establishes the absolute global order of events. A deterministic computer algorithm can then process this ledger to derive a universally accepted conclusion. So long as the members of a community agree on the processing algorithm, the result of the algorithm is authoritative.

The primary consideration is determining what testimony is allowed to enter the public record. Systems should be designed to minimize the potential for censorship. Censorship on the public ledger is similar to preventing someone from voting in an election. In both cases an individual is prevented from impacting the global consensus.

Consensus in Steem

Conceptually, the consensus algorithm adopted by Steem is similar to the consensus algorithm adopted by companies throughout the world. People with a vested interest in the future value of Steem vote to select individuals responsible for including testimony in the public record. Voting is weighted proportional to each individual’s vested interest.

In the world of cryptocurrencies, the public record is commonly referred to as a blockchain. A block is a group of signed transactions.

With Steem, block production is done in rounds. Each round 21 witnesses are selected to create and sign blocks of transactions. Twenty (20) of these witnesses are selected by approval voting and one is timeshared by every witness that didn’t make it into the top 20 proportional to their total votes. The 21 active witnesses are shuffled every round to prevent any one witness from constantly ignoring blocks produced by the same witness placed before. Any witness who misses a block and hasn’t produced in the last 24 hours will be disabled until they update their block signing key.

This process is designed to provide the best reliability while ensuring that everyone has the potential to participate in block production regardless of whether they are popular enough to get voted to the top. People have three options to overcome censorship by the top 20 elected witnesses: patiently wait in line with everyone else not in the top 20, or purchase more SP to improve voting power. Generally speaking, applying censorship is a good way for elected witnesses to lose their job and therefore, it is unlikely to be a real problem on the Steem network.

Because the active witnesses are known in advance, Steem is able to schedule witnesses to produce blocks every 3 seconds. Witnesses synchronize their block production via the NTP protocol. A variation of this algorithm has been in use by the BitShares network for over a year where it has been proven to be reliable.

Eliminating Transaction Fees

Steem goes to great lengths to reward people for contributing to the network. It would be counterproductive to turn around and charge people every time they attempt to interact with the community.

Blockchain technology currently depends upon transaction fees to prevent spam. These fees suffer all of the known problems with microtransactions and prevent blockchains from being used for low-value transactions. Truly decentralized applications must offer users the appearance of free transactions if they wish to compete with their centralized alternatives. This paper outlines the approach used by Steem to eliminate the need for fees and thereby enable a wide range of previously untenable decentralized applications.

The Problem With Fees

Blockchains are decentralized networks where all transactions are broadcast to all peers. Every so often a block is produced that includes some or all of the pending transactions. All blockchains must find a solution to prevent malicious users from consuming all of the available network capacity with worthless transactions. These worthless transactions can prevent other valuable transactions from being processed and ultimately destroy the network.

The solution adopted by most blockchains thus far is to charge a minimum transaction fee. A fee worth just a few cents is enough to make attacking the network expensive and unprofitable. While this approach solves the spam problem, it introduces new problems. Imagine solving the email spam problem by introducing a small fee on every email; people wouldn’t use email.

Micropayments Don’t Work

The fundamental problem with charging transaction fees is that micropayments don’t work, especially for low-value user actions. When a fee is charged on every transaction, it limits the types of transactions that a decentralized network can process. Regardless of how rational the argument for the necessity of fees, users still hate the experience of being nickeled and dimed for everything that they do.

Imagine if the websites we use every day charged us a fee every time we modify our accounts by changing the password. Users expect certain things to be free. Requiring users to make a decision on whether or not an action is worth a small fee creates anxiety that causes users to leave. A transaction can’t be worth so much as to require a decision but worth so little that that decision is automatic. There is a certain amount of anxiety involved in any decision to buy, no matter how small, and it derives not from the interface used or the time required, but from the very act of deciding.

Micropayments, like all payments, require a comparison: “Is this much of X worth that much of Y?” There is a minimum mental transaction cost created by this fact that cannot be optimized away, because the only transaction a user will be willing to approve with no thought will be one that costs them nothing, which is no transaction at all.

– Clay Shirky[^8]

In the world of financial payments, small fees are acceptable because the value of the transaction is extremely high relative to the fee charged, and the buyer has already made a decision to buy. The world of potential blockchain applications is far greater than just financial payments and includes many necessary transactions for which fees are simply unacceptable to users.

Systems like BitShares, Nxt, Ripple, Counter Party and Stellar all allow users to place limit orders on the blockchain and all of them charge users a small fee to perform this action. Later if the user wishes to cancel their order, another fee is charged. Systems like Ethereum take micropayments to a whole new level: charging per calculation. All of these systems struggle to attract new mainstream users for the same reasons that a decentralized search engine would struggle to attract users from Google if it charged a small fee for every search. It doesn’t matter how good the service is, people expect certain things to be free. This is true even if a user ends up paying more overall under a different fee structure.

Fees are a Barrier to Entry

Any fee creates a barrier to entry for new users. Before someone can experiment with Ethereum they must acquire some ETH tokens. Anyone wanting to build a decentralized application on Ethereum must pass on the cost to their customers. Buying a crypto currency is not an easy task and rarely makes sense for amounts less than $10. This means that new users wanting to try out a new decentralized application must first be convinced to part with $10.

Changing Fees

Over time a network must adjust fees. This can happen either due to an increase in the value of the token or due to a surge in capacity. Users like predictable fees and guaranteed service. While it is possible to dynamically adjust fees during times of heavy use, the result is a poor user experience.

Sybil Attacks

Centralized websites prevent spam through rate limiting and some form of ID verification. Even something as simple as reCAPTCHA[^9] is sufficient to limit the creation of fake accounts. If someone abuses their account then centralized websites are free to block the account.

In a decentralized system there is no direct way to ban users nor centralized provider able to host a reCAPTCHA and enforce rate limiting of accounts. In fact, the inability to censor users is one of the main selling points of blockchain technology.

Full Reserve vs Fractional Reserve

Let’s view a blockchain like an Internet Service Provider (ISP) co-op which owns all of the cables in the town and has a maximum amount of bandwidth that it can provide at any time. People living in the town can buy shares in the ISP and in exchange they are entitled to utilize a portion of the available bandwidth.

The ISP has two choices, run a “full reserve” or “fractional reserve” system. Under a full reserve system each user is only allowed a fraction of the maximum bandwidth proportional to her shares. Because not everyone uses the Internet at the same time, the town’s network would be significantly underutilized.

Under a fractional reserve system the individual users could utilize more bandwidth than they are entitled to at any given point in time so long as not everyone uses the Internet at the same time. The problem with operating a fractional reserve is that congestion occurs anytime too many people wish to use the network at the same time. The ISP needs a way to prioritize bandwidth during congested periods. In the most extreme case, a fully congested network must revert to a full reserve system. The challenge is setting the proper fractional reserve ratio.

Bandwidth Instead of Micropayment Channels

The solution to the problems with micropayments is in implementing dynamic fractional reserves. Under this model the blockchain will automatically adjust the reserve ratio for the network during times of congestion. The blockchain will set a target utilization that leaves enough headroom for short term surges in demand. Any time the surges are sustained the blockchain reduces the maximum bandwidth-per-share. When a surge is over and there is surplus capacity the blockchain can slowly increase the bandwidth-per-share.

Bandwidth used by an individual user should be measured over a suitably long period of time to allow that user to time-shift their usage. Users tend to login, do many things at once, then logout. This means that their bandwidth over a short period of time may appear much higher than if viewed over a longer period of time. If the time window is stretched too far then the reserve ratio will not adjust fast enough to respond to short-term surges, if the window is too short then clustering usage will have too big of an impact on normal users.

In our estimate it should be sufficient to measure the average weekly bandwidth usage of users. Every time a user signs a transaction, that transaction is factored into their own individual moving average. Any time a user’s moving average exceeds the current network limit their transaction is delayed until their average falls below the limit.

Impact of Capacity

Blockchain capacity isn’t necessarily capped. It is well within the technological capability of internet infrastructure to increase the Bitcoin block size to 10MB which in turn will reduce the minimum required balance by a factor of 10. While Bitcoin currently supports about 3 transactions per second, alternative implementations are capable of over 1000 transactions per second.

Comparison to Fees

If we assume a user with $25 dollars worth of BTC transacts once per week and pays a $0.04 cent fee each time then they would pay over $2.00 in fees per year. A user would have to earn a 8% rate of return on their $25 dollars just to break even with paying fees. Chances are that users were going to hold their money on the blockchain anyway, so this user with $25 worth of BTC just saved $2 over the course of a year by adopting a rate-limiting approach rather than a fee-based approach. With just $175 they could transact every single day and save $14 per year.

Account Creation

Steem’s account-based system with publicly known balances simplifies the implementation of the bandwidth-based rate limiting algorithm. Any account with a balance below the minimum required to transact once per week would be unable to transact. This implies that all new accounts should be funded with at least this minimum balance. It also implies that users wishing to transact in smaller amounts can, so long as they hold a larger balance and reuse the account.

It is possible for a low-balance account created during a time of low usage to become inaccessible if the network usage picks up. The funds could be recovered at any time by temporarily delegating a larger balance to the account.

In order to maintain a reasonable user experience with a minimum number of hung accounts, all new accounts should start out with a balance 10 times the minimum required to transact weekly. This way even if demand increases by a factor of 10 the account will remain viable.

Any initial account balance would have to come from the user creating the account and not from token creation due to the potential for sybil attacks.

Justifying Minimum Balances

The concept of forcing users to maintain a minimum balance flows naturally from the value of a user[^10]. Anyone running a business knows that every single user has significant value. Businesses spend anywhere from $30 to $200 to acquire a user. Sometimes they pay users directly, other times they pay for advertizing, and still other times entire companies are bought just for their user base. After a company acquires a user they often given them many free services just to keep them around long enough to monetize them through some other means.

Ripple uses a minimum balance[^11] that scales with account resource use and requires that new accounts get funded with at least this minimum balance. Currently this minimum balance is about $0.15 which is greater than the $0.10 we estimated would allow someone to transact freely at least once per week.

A blockchain can enforce a minimum value per user through the simple process of requiring a minimum balance. Any business that wishes to bring a new customer to the blockchain can pre-fund that user’s account with the minimum balance that would allow them to transact. Requiring a relatively large fee ($1.00) to sign up new users will naturally force anyone offering free accounts to vet the quality and uniqueness of each account before registering them with the blockchain.

Maintaining a minimum balance is effectively the same as making users pay transaction fees with the interest they could have earned on their balance. The minimum balance is simply the balance required to earn enough interest to pay a fee in a relatively short period of time.

Fortunately, the minimum balance required can be as low as a dollar and this is something users can understand and appreciate. The opportunity cost of lost interest doesn’t incur the cognitive cost of a micro-fee and is far more acceptable to users.

The STEEM used to pre-fund an account is Powered Up in the new account (i.e., converted to Steem Power). A portion of the SP used to fund a new account may be delegated from the creator of the account. When a user is delegated SP, they may use the SP for voting and bandwidth purposes as if it were their own, but the ownership of the SP remains with the user who delegated it. A user may remove the delegation at any time. After a cool-down period, the SP is returned to their account.

Effectiveness Relative to Fees

To compare the effectiveness of rate limiting to fees we must consider how the two systems react to intentional network flooding by an attacker. Under Bitcoin an attacker with $10,000 dollars could disrupt service for an entire day by filling every single block. The same attacker would be unable to disrupt service for even a single block under the dynamic fractional reserve rate limiting approach.

If we go to a more extreme case and assume the attacker holds 1% of all coins then we presume an attacker with $60 million dollars. Such an attacker could deny the Bitcoin blockchain service for 16 years unless the miners increased fees or capacity. Even if fees were raised to $15 per transaction, the attacker could still keep the network flooded for 16 days.

Under the rate limiting approach, someone who holds 1% of all coins with an intent to flood the network would achieve their goal for less than 30 seconds.

Renting vs. Buying vs. Time Sharing

When someone owns a house they expect the right to use the house for free. If a group of people buy a house together then each can expect the right to use the house proportional to their percentage ownership in the house. A fee based blockchain is like renting the house from its owners, whereas rate limiting is like a timeshare among owners.

If a house is owned by multiple people then those individuals must decide how they wish to timeshare the house. Someone who owns 50% of the house but only uses it one weekend per year might expect to be paid by the individuals who take their unused time. This is the mindset of a fee based system.

On the other hand, someone who owns 50% of the house is speculating that demand for the house will increase in the future and they will be able to sell their stake for more. Any owner who owns more of a house than they use becomes a real estate speculator. With this mindset rather than collecting rent, they collect appreciation.

The value of a share is derived from how much time it can potentially grant its owner. Owning 1% of a house and getting it 1 weekend per year is the lowest value of a share. However, if half of the shareholders never use their weekend, then the value per timeshare rises to 2 weekends per year. If those inactive users instead opt to rent their unused time, then it falls back to 1 weekend per year. If those unused timeshares were sold to people who would use them then the value of a timeshare would fall by 50%. Unless the rent collected is greater than the fall in share value the timeshare owners are making an economic miscalculation.

Using this rationale we can assume that a system based on fees will either be more expensive for its users or be less profitable for its collective owners. An individual small owner may profit by renting out his small time slice, but only at the expense of all other timeshare owners. In effect, the cost of the falling timeshare value is shared among all owners whereas the profits are centralized in the single owner who decided to rent his share.

We can conclude from this that a blockchain is best served by not using usage fees at all. If a usage fee were to be charged as an alternative to rate limiting, then it should be the equivalent of buying enough timeshares and committing to hold them long enough to gain the right use it once.

Stated another way, a transaction fee should be equal to the minimum account balance necessary to transact once per week and it should be refunded at the end of the week. Assume the minimum account balance is $1 and allows someone to transact once per week. If someone with a $1 balance that wishes to perform 5 transactions at once they will have to increase their balance to $5 for a week either before or after their transactions.

In theory a market could form where users can borrow the stake required. In practice it is more efficient for users to simply buy and sell the timeshares necessary to meet their desired usage rate. In other words, the cost of negotiating micro-loans is greater than the cost of maintaining a balance suitable for your maximum weekly usage.

Decentralized rate limiting of transactions can enable new types of decentralized applications that were not viable when every use of the application required a micropayment. This new model gives application developers the ability to decide if and when to charge their users for transactions.

Performance and Scalability

The Steem network is built upon Graphene, the same technology that powers BitShares. Graphene has been publicly demonstrated sustaining over 1000 transactions per second on a distributed test network. Graphene can easily scale to 10,000 or more transactions per second with relatively straightforward improvements to server capacity and communication protocols.

Reddit Scale

Steem is capable of handling a larger userbase than Reddit. In 2015 Reddit’s 8.7 million users generated an average of 23 comments per second[^12], with an average of 83 comments per year per user. There were 73 million top-level posts, for an average of 2 new posts per second. There were about 7 billion up votes creating an average voting rate of 220 votes per second. All told, if Reddit were operating on a blockchain it would require an average of 250 transactions per second.

To achieve this industry-leading performance, Steem has borrowed lessons learned from the LMAX Exchange[^13], which is able to process 6 million transactions per second. Among these lessons are the following key points:

  1. Keep everything in memory.
  2. Keep the core business logic in a single thread.
  3. Keep cryptographic operations (hashes and signatures) out of the core business logic.
  4. Divide validation into state-dependent and state-independent checks.
  5. Use an object oriented data model.

By following these simple rules, Steem is able to process 10,000 transactions per second without any significant effort devoted to optimization.

Keeping everything in memory is increasingly viable given the recent introduction of Optane™ technology from Intel[^14]. It should be possible for commodity hardware to handle all of the business logic associated with Steem in a single thread with all posts kept in memory for rapid indexing. Even Google keeps their index of the entire internet in RAM. The use of blockchain technology makes it trivial to replicate the database to many machines to prevent loss of data. As Optane™ technology takes over, RAM will become even faster while gaining persistence. In other words, Steem is designed for the architectures of the future and is designed to scale.

Allocation & Supply

Initial Allocation & Supply

The Steem network started with a currency supply of 0 and allocated STEEM via proof of work at a rate of approximately 40 STEEM per minute to miners, with an additional 40 STEEM per minute being created to seed the content and curation reward pools (for a total of 80 STEEM per minute). Then the network started rewarding users who converted to SP. At this point, STEEM grew at a rate of approximately 800 STEEM per minute due to the combined effects of the various Contribution Rewards summarized below:

Contribution Rewards:

  • Curation rewards: 1 STEEM per block or 3.875% per year, whichever is greater
  • Content Creation rewards: 1 STEEM per block or 3.875% per year, whichever is greater
  • Block production rewards: 1 STEEM per block or 0.750% per year, whichever is greater
  • POW inclusion rewards before block 864,000: 1 STEEM per block (awarded as 21 STEEM per round)
  • POW inclusion rewards after block 864,000: 0.0476 STEEM per block (awarded as 1 STEEM per round) or 0.750% per year, whichever is greater.
  • Liquidity rewards: 1 STEEM per block (awarded as 1200 STEEM per hour) or 0.750% per year, whichever is greater

Power Rewards:

  • Steem Power rewards: For each STEEM created by the above rewards, 9 STEEM are divided among all Steem Power holders.

SBD operations:

  • SBD rewards: A percentage of SBD value is created at an APR set by the witnesses and paid to SBD holders as SBD

The overall supply picture is complicated by the effect of SBD operations, which may result in large-scale creation or destruction of STEEM through feed rate following and SBD rewards, as discussed in the SBD section. Other, smaller-scale complicating effects also exist, including unclaimed incentives (e.g. block rewards for missed blocks), and abandoned accounts.

Current Allocation & Supply

Starting with the network’s 16th hard fork in December 2016, Steem began creating new tokens at a yearly inflation rate of 9.5%. The inflation rate decreases at a rate of 0.01% every 250,000 blocks, or about 0.5% per year. The inflation will continue decreasing at this pace until the overall inflation rate reaches 0.95%. This will take about 20.5 years from the time hard fork 16 went into effect.

75% of the new tokens that are generated go to fund the reward pool, which is split between authors and curators. 15% of the new tokens are awarded to holders of SP. The remaining 10% pays for the witnesses to power the blockchain.

Impact of Token Creation Rate

It is often said that a coin with an inflationary model is not sustainable, but we know from countless real-world examples that the quantity of money does not have a direct and immediate impact on its value, though it certainly plays a role.

From August 2008 through January 2009 the U.S. money supply[^15] grew from $871B to $1,737B, a rate of over 100% per year and then continued to grow at about 20% per year for the next 6 years. All told the money supply in the U.S. has grown by 4.59x over less than 7 years. During that same time, the value of the dollar relative to goods and services has fallen less than 10% according to the government’s price index[^16]. This real-world example demonstrates that supply is only one component of price.

For the first 2 years of Bitcoin’s life the network sustained an annual inflation rate[^17] of over 100%. For the first 5 years it was over 30%, and for the first 8 years it was over 10%. All told the total “spending” Steem does to fund content, curation, and block production amounts to less than 10% APR.

The price of a digital commodity, like STEEM, is driven by both supply and demand. When a long-term holder decides to exit, the supply of STEEM on the market will increase and push the price down. This downward pressure is countered when a new long-term holder decides to buy up the STEEM and convert it back into SP. Additional supply and demand may be be added due to market speculators buying and selling liquid STEEM based on their predictions of the future market price.

The Power of Steem

Steem recognizes that the value of all user contributions (posts and votes) is greater than the sum of the parts. A single comment is worth next to nothing, but millions of curated posts is worth many millions (or possibly even billions) of dollars. A single vote provides little curation value, but billions of votes is very effective curation. Content without curation is of limited value. Given all the content of the Internet minus the links between it, Google would struggle to produce useful search results. It is the links between information that give it significant value.

Because everyone benefits, everyone should pay. In other words, no individual user should be expected to pay for anything, but instead should be paid for everything they do that brings value to Steem. All we need to do is ascertain which user contributions bring a social network value and which ones don’t.

Collectively Reddit users vote 220 times per second and make 23 posts per second. Reddit is valued between $500 million[^18] and $4 billion[^19] which means that each and every upvote and post is worth between $0.06 and $0.50 assuming the value of Reddit is mostly within the past year’s worth of activity. One could argue that most of the value of Reddit is the near-real-time discussions that have occurred within the past week which would dramatically increase the value of new activity. People go where people are today, not where people were last year.

No Micropayments, Tips Optional

Existing attempts at integrating a cryptocurrency into a social media platform have focused on enabling users to pay one another. Many services have attempted to introduce tipping. The theory is that if we make tipping simple enough then more people will do it. Other services attempt to get people to pay to promote or boost their content’s ranking. Still others attempt to build small prediction markets on how many tips an article will receive.

All of these approaches boil down to micropayments. They differ only in who is making the payment. They all suffer from insufficient engagement of people making the micropayments. In the search for incentivised content production entrepreneurs have been so focused on who should pay that they missed the obvious reality: everyone benefits from everyone’s actions so everyone should pay or no one should pay, depending on how you look at it.

Steem bypasses micropayments completely because when a user upvotes a post it is the community that pays the bill. The same amount of money will be spent whether the user upvotes a post or not and the funds will not come from the voter.

The mental energy associated with making an economic decision becomes a barrier to participation for most people.

We already face a multitude of choices everyday with regards to what to access online in this digital era of the information explosion, and every additional decision that we must make simply adds on to the uncertainty and anxiety we face. Micropayment supporters believe that a simplified implementation can minimize the intrusiveness of micropayments and improve user experience, but their argument only creates double standards for the decision making process [2]. A transaction cannot simultaneously be worth enough to warrant a decision and worth so little that the decision is automatic. The only transactions that users can approve without thought are ones that cost them nothing, thus any micro-transaction of positive value will incur mental costs through its requiring a decision. Furthermore, mental transaction costs actually rise below a certain threshold value, a phenomenon that places micropayments at an even greater disadvantage. For instance, it is easy to think that a copy of today’s newspapers costs $1, but readers face much more difficulty and anxiety in deciding on the value of each article or word. Such a dilemma will only be replicated and exacerbated if all online content were to be broken down into their components and individually valued within a micropayment system.

-Micropayments: A Viable Business Model[^20]

Under Steem, micropayments are paid to content producer, but those who vote for the content do not pay. Instead, the cost of the reward is paid for via new tokens. Someone can join the system, vote to pay someone, and then exit the system with more money than they started with (assuming the market valuation of the Steem system remained constant). In other words, the micropayment solution provided by Steem provides a user-experience similar to many widely used websites that have user-moderated content.

Furthemore, Steem pays people to figure out who should be paid! This kind of thinking is revolutionary.

Value is in the Links

The Internet would lose the vast majority of its value if all links among content were removed. It is the relationship among web pages that allows Google to identify the best apple pie recipe among the 16 million results. Without the links the only information Google would have is word frequency.

Links can take many forms and have adapted over time. Every time a user votes on content in a social network they add a connection between themselves and the content. This in turn links the consumer to the producer through the content. The more connections a network has the more valuable the information becomes. It is the relative and intentional connectedness of information that gives it value.

A social network can maximize the value extracted from a set of content by maximizing the quantity and quality of connections. Curating content is expensive and time consuming while being near impossible for computers to perform in the absence of links. Steem rewards users who are among the first to find and identify new content.

By incentivising curation the Steem network is able to use automated algorithms to extract the most valuable information from a massive amount of content.

Solving the Cryptocurrency Onboarding Problem

It isn’t easy to get into cryptocurrency[^21]. Someone who discovers Bitcoin and wants to try it out quickly learns that they will need to sign up with an exchange and fund their account with a credit card or wire transfer. What would Facebook’s adoption rate have been like if you had to fork over money and a two forms of ID?

Steem solves this problem by giving everyone a way to get paid for doing simple, but valuable, tasks. This will help to widely distribute STEEM tokens. This is helpful because cryptocurrencies have a network effect (i.e. more users make it more useful; for an extreme example, consider that if Satoshi had kept 100% of Bitcoin for himself, Bitcoin would be worthless.)

Solving the Cryptocurrency Liquidation Problem

A currency that is difficult to use or impossible to sell has little value. Someone who comes across $1.00 worth of Bitcoin will discover that it costs more than $1.00 to sell that Bitcoin. They have to create an account with an exchange, perform KYC validation, and pay fees. Small amounts of cryptocurrency are like small change that people are unwilling to bend over to pick up.

Merchants give users a way to quickly convert their cryptocurrency into tangible goods and services. Merchants need a currency pegged to their unit of account, normally dollars. Accepting a volatile currency introduces significant accounting overhead.

Merchants will accept any currency if it increases their sales. Having a large user base with a stable currency such as SBD lowers the barrier to entry for merchants. The presence of merchants improves the system by creating an off-ramp for users to exit the system without going to the trouble of using an exchange.

Another way that people can liquidate the small amounts of cryptocurrency they receive from participating on the Steem platform is through tipping others. This is like leaving the small change as a tip for your waiter. When enough people leave small tips it adds up to a meaningful amount. You and the waiter each gain a benefit from the tip.

Censorship

Steem is a decentralized network that is operated by witnesses in jurisdictions around the world. All user actions are publicly recorded on the blockchain, and can be publicly verified. This means that there is no single entity that can censor content that is valued by STEEM holders.

Individual websites such as steemit.com may censor content on their particular site, but content published on the blockchain is inherently broadcast traffic and mirrors all around the world may continue to make it available.

Freedom of speech is the foundation of all other liberties and any infringement upon freedom of speech undermines the only peaceful means of reaching consensus: discussion. Without free discussion voters cannot be fully informed, and uninformed voters are a greater threat to society than losing the right to vote. Censorship is a means of stealing votes through limiting public discourse. Steem is committed to enabling free speech and building a free society.

Solving Organic Discovery via Search Engine Optimization

Most cryptocurrencies generate little value for those who are not actively using the network. Steem, by contrast, generates content and encourages users to share it. This content gets indexed by search engines and ultimately will bring value to a large number of passive users. This search traffic creates organic advertising for the Steem network and grows the network effect.

Shifting Toward Blockchain-based Attribution

The internet represents the easiest medium for distributing information in the world. With that said, it can be a frightening place for content creators who would like to own their content and have it shared with proper attribution. On current social media platforms, attribution is something that can be lost overnight – a posted video or image can be replicated and re-shared without consent or regard for the creator.

Under blockchain-based social media, a creator or author would always be able to point to a public record and timestamp showing proof of their content origination. In a circumstance where a creator would like to address those who have re-shared without permission or attribution, blockchain-based records provide public proof that the content was posted by a particular user at a particular time. In the future, blockchain-based attribution could come to be recognized by governments for its authenticity and could hold weight in court, which would give content creators greater powers to control their work.

While a timestamping service can be built on almost any blockchain, and several efforts exist to build this kind of service on the Bitcoin network, Steem has a useful advantage in this realm because content publishers are “first class citizens” — the Steem blockchain is built from the ground up around the use case of content publication, which allows content creators to have the blockchain to validate their content at a certain point in time simply by writing their post using the same authoring tools used by other Steem users.

Replacing Advertising with Blockchain-based Content Rewards

Under most content monetization models, content creators leverage advertising in one form or another. Many creators recognize how advertising may diminish their work’s value to the consumer, yet creators very often must seek returns on their time by monetizing. Advertising represents a double-edged sword: With ads, a creator can make money most easily. Without ads, monetization is difficult but the content is richer.

Creators posting to social media outlets that are connected to Steem may monetize merely by having their work recognized (or ”liked”) by the Steem community. Blockchain-based payouts are completely digital and have no middle-man. Therefore monetization by blockchain-based content rewards should be faster and much lower barrier to use than monetization by advertisements.

Conclusion

Steem is an experiment designed to address challenges in the cryptocurrency and social media industries by combining the best aspects from both. Steem presents earning opportunities to content creators and internet readers in ways that have not existed within the social media industry. Within Steem, individuals earn real rewards online that are directly correlated to their contributions. Those rewards may have dollar value due to the market price discovery and liquidity of Steem, and the people who hold Steem may have more exclusive earning powers than those who do not.

[^1]: Reddit’s Cryptocurrency, Forbes, Erika Morphy, October 2014 http://www.forbes.com/sites/erikamorphy/2014/10/01/reddits-cryptocurrency-could-have-many-uses/\#4e07b05332b9

[^2]: Sweat Equity, Investopedia http://www.investopedia.com/terms/s/sweatequity.asp

[^3]: Meta-moderation is a second level of comment moderation. Users are invited to rate a moderator’s decision in order to improve moderation. https://en.wikipedia.org/wiki/Meta-moderation\_system

[^4]: The Impossible Trinity, economic theory https://en.wikipedia.org/wiki/Impossible\_trinity

[^5]: N-Person Prisoner’s Dilemma https://cs.stanford.edu/people/eroberts/courses/soco/projects/1998-99/game-theory/npd.html

[^6]: The Story of the Crab Bucket http://guidezone.e-guiding.com/jmstory\_crabs.htm

[^7]: Zipf’s Law https://en.wikipedia.org/wiki/Zipf%27s\_law

[^8]: Clay Shirky, The Case Against Micropayments http://www.openp2p.com/pub/a/p2p/2000/12/19/micropayments.html

[^9]: reCAPTCHA, Easy on Humans, Hard on Bots https://www.google.com/recaptcha/intro/index.html

[^10]: Forbes, Tristan Louis, “How Much is a User Worth?” http://www.forbes.com/sites/tristanlouis/2013/08/31/how-much-is-a-us

[^11]: Ripple, Account Reserves https://ripple.com/build/reserves/

[^12]: Reddit Statistics, Number of Users and Comments per Second http://expandedramblings.com/index.php/reddit-stats/2/

[^13]: Martin Fowler, The LMAX Architecture http://martinfowler.com/articles/lmax.html

[^14]: Introducing Intel Optane Technology – Bringing 3D XPoint Memory to Storage and Memory Products https://newsroom.intel.com/press-kits/introducing-intel-optane-technology-bringing-3d-xpoint-memory-to-storage-and-memory-products/

[^15]: United States Money Supply, 2009 https://research.stlouisfed.org/fred2/graph/?s%5B1%5D%5Bid%5D=AMBNS

[^16]: CPI Inflation Index, United States Dollar 2008-2016 http://data.bls.gov/cgi-bin/cpicalc.pl?cost1=1&year1=2008&year2=2016

[^17]: Bitcoin Annual Inflation Rate, Bitcoin Talk Forum https://bitcointalk.org/index.php?topic=130619.0

[^18]: Reddit Valuaton, Newsweek, 2014 http://www.newsweek.com/investors-think-reddit-worth-500-million-26

[^19]: Worth of Web, March 2016 http://www.worthofweb.com/website-value/reddit.com/

[^20]: Micropayments: A Viable Business Model http://www.openp2p.com/pub/a/p2p/2000/12/19/micropayments.html

[^21]: Dailydot, Jon Southurt, April 2015 http://www.dailydot.com/opinion/bitcoin-cryptocurrency-adoption-hard

Dash

Highlight to Annotate. Click to Learn.

Dash: A Privacy-Centric Crypto-Currency

Evan Duffield – evan@dash.org
Daniel Diaz – daniel@dash.org

Abstract. A crypto-currency based on Bitcoin, the work of Satoshi Nakamoto, with various improvements such as a two-tier incentivized network, known as the Masternode network. Included are other improvements such as PrivateSend, for increasing fungibility and InstantSend which allows instant transaction confirmation without a centralized authority.

1 Introduction

Bitcoin [1] is a cryptocurrency that has emerged as a popular medium of exchange and is the first digital currency that has attracted a substantial number of users [2]. Since its inception in 2009, Bitcoin has been rapidly growing in mainstream adoption and merchant usage [3]. A main issue with the acceptance of Bitcoin in point-of-sale (POS) situations is the time required to wait for the network to confirm the transaction made is valid, alternatively payment companies have created methods to allow vendors to take zero-confirmation transactions, but these solutions utilize a trusted counterparty to mediate the transaction outside of the protocol.

Bitcoin provides pseudonymous transactions in a public ledger, with a one-to-one relationship between sender and receiver. This provides a permanent record of all transactions that have ever taken place on the network [5]. Bitcoin is widely known in academic circles to provide a low level of privacy, although with this limitation many people still entrust their financial history to it’s blockchain.

Dash is the first privacy-centric cryptographic currency based on the work of Satoshi Nakamoto. In this paper we propose a series of improvements to Bitcoin resulting in a decentralized, strongly anonymous crypto-currency, with tamper-proof instant transactions and a secondary peer-to-peer (P2P) network incentivized to provide services to the Dash Network.

2 Masternode Network

Full nodes are servers running on a P2P network, that allow peers to use them to receive updates about the events on the network. These nodes require significant amounts of traffic and other resources that carry substantial cost. As a result, on the Bitcoin network a steady decrease in the amount of these nodes has been observed for some time [7] and as a result block propagation have been upwards of 40 seconds[14]. Many solutions have been proposed such as a new reward scheme by Microsoft Research [4] and the Bitnodes incentive program [6].

Figure 1: Bitcoin Full nodes in the spring of 2014

Figure 1: Bitcoin Full nodes in the spring of 2014

These nodes are very important to the health of the network. They provide clients with the ability to synchronize and quick propagation of messages throughout the network. We propose adding a secondary network, known as the Dash Masternode network. These nodes will have high availability and provide a required level of service to the network in order to take part in the Masternode Reward Program.

2.1 Masternode Reward Program – Cost and Payments

Much of the reason for the decrease of full nodes on the Bitcoin network, is the lack of incentive to run one. Over time the cost of running a full node increases as the network gets used more, creating more bandwidth and costing the operator more money. As the cost rises, operators consolidate their services to be cheaper to run or run a light client, which does not help the network at all.

Masternodes are full nodes, just like in the Bitcoin network, except they must provide a level of service to the network and have a bond of collateral to participate. Collateral is never forfeit and is safe while the Masternode is operating. This allows investors to provide a service to the network, earn interest on their investment and reduce the volatility of the currency.

To run a Masternode, the node must store 1,000DASH. When active, nodes provide services to clients on the network and in return are paid in the form of a dividend. This allows the users to pay for the services and earn a return on investment. Masternodes are all paid from the same pool of money, approximately 45% [footnote] of the total block reward is dedicated to this program.

Due to the fact that the Masternode rewards program is a fixed percentage and the Masternode network nodes are fluctuating, expected Masternode rewards will vary according to the current total count of active Masternodes. Payments for a standard day for running a Masternode can be calculated by using the following formula:

(n/t)* rba

Where:

n is the number of Masternodes an operator controls

t is the total number of Masternodes

r is the current block reward (presently averaging about 5 DASH)

b is blocks in an average day. For the Dash network this usually is 576.

a is the average Masternode payment (45% of the average block amount)

Return on investment for running a Masternode can be calculated as

((n/t)r * ba*365) / 1000

Where variables are the same as above.

The cost associated with running a Masternode creates a hard and soft limit of active nodes on the network. Currently with 5.3 million DASH in circulation, only 5,300 nodes could possibly be running on the network. The soft limit is imposed by the price it costs to acquire a node and the limited liquidity on exchanges due to usage of Dash as a currency and not merely an investment.

2.2 Deterministic Ordering

A special deterministic algorithm is used to create a pseudo-random ordering of the Masternodes. By using the hash from the proof-of-work for each block, security of this functionality will be provided by the mining network.

Pseudo Code, for selecting a Masternode:

For(mastenode in masternodes){
    current_score = masternode.CalculateScore();

    if(current_score > best_score){
         best_score = current_score;
         winning_node = masternode;
    }
}

CMasterNode::CalculateScore(){
   pow_hash = GetProofOfWorkHash(nBlockHeight); // get the hash of this block
   pow_hash_hash = Hash(pow_hash); //hash the POW hash to increase the entropy
   difference = abs(pow_hash_hash - masternode_vin);
   return difference;
}

The example code can be extended further to provide rankings of Masternodes also, a “second”, “third”, “fourth” Masternode in the list to be selected.

2.3 Trustless Quorums

Currently the Dash network has ~2,400 active Masternodes[8]. By requiring 1,000DASH collateral to become an active Masternode, we create a system in which no one can control the entire network of Masternodes. For example, if someone wants to control 50% of the Masternode network, they would have to buy 2,300,000 DASH from the open market. This would raise the price substantially and it would become impossible to acquire the needed DASH.

With the addition of the Masternode network and the collateral requirements, we can use this secondary network to do highly sensitive tasks in a trustless way, where no single entity can control the outcome. By selecting N pseudo random Masternodes from the total pool to perform the same task, these nodes can act as an oracle, without having the whole network do the task.

For an example, implementation of a trustless quorum (see InstantSend[9]), which uses quorums to approve transactions and lock the inputs or the proof-of-service implementation[10].

Another example use for trustless quorums can include utilizing the Masternode network as a decentralized oracle for financial markets, making secure decentralized contracts a possibility. As an example contract, if Apple Stock (AAPL) is over $300 on Dec 31, 2016 pay public key A, otherwise pay public key B.

2.4 Roles and Proof-Of-Service

Masternodes can provide any number of extra services to the network. As a proof-of-concept, our first implementation included PrivateSend and InstantSend. By utilizing what we call proof-of-service, we can require that these nodes are online, responding and even at the correct block height.

Bad actors could also run Masternodes, but not provide any of the quality service that is required of the rest of the network. To reduce the possibility of people using the system to their advantage nodes must ping the rest of the network to ensure they remain active. This work is done by the Masternode network by selecting 2 quorums per block. Quorum A checks the service of Quorum B each block. Quorum A are the closest nodes to the current block hash, while Quorum B are the furthest nodes from said hash.

Masternode A (1) checks Masternode B (rank 2300)

Masternode A (2) checks Masternode B (rank 2299)

Masternode A (3) checks Masternode B (rank 2298)

All work done to check the network to prove that nodes are active is done by the Masternode network itself. Approximately 1% of the network will be checked each block. This results in the entire network being checked about six times per day. In order to keep this system trustless, we select nodes randomly via the Quorum system, then we also require a minimum of six violations in order to deactivate a node.

In order to trick this system, an attacker will need to be selected six times in a row. Otherwise, violations will be cancelled out by the system as other nodes are selected by the quorum system.

Attacker Controlled Masternodes / Total Masternodes Required Picked Times In A Row Probability of success(n/t)r DASH Required
1/2300 6 6.75e-21 1,000DASH
10/2300 6 6.75e-15 10,000DASH
100/2300 6 6.75e-09 100,000DASH
500/2300 6 0.01055% 500,000DASH
1000/2300 6 0.6755% 1,000,000DASH

Table 1. The probability of tricking the system representing one individual Masternode as failing proof-of-service

Where:

n is the total number of nodes controlled by the attacker

t is the total number of Masternodes in the network

r is the depth of the chain

The selection of Masternodes is pseudo random based on the Quorum system

2.5 Masternode Protocol

The Masternodes are propagated around the network using a series of protocol extensions including a Masternode announce message and Masternode ping message. These two messages are all that is needed to make a node active on the network, beyond these there are other messages for executing a proof-of-service request, PrivateSend and InstantSend.

Masternodes are originally formed by sending 1,000DASH to a specific address in a wallet that will “activate” the node making it capable of being propagated across the network. A secondary private key is created that is used for signing all further messages. The latter key allows the wallet to be completely locked when running in a standalone mode.

A cold mode is made possible by utilizing the secondary private key on two separate machines. The primary “hot” client signs the 1,000DASH input including the secondary signing private key in the message. Soon after the “cold” client sees a message including its secondary key and activates as a Masternode. This allows the “hot” client to be deactivated (client turned off) and leaves no possibility of an attacker gaining access to the 1,000DASH by gaining access to the Masternode after activation.

Upon starting, a Masternode sends a “Masternode Announce” message to the network, containing:

Message: (1K DASH Input, Reachable IP Address, Signature, Signature Time, 1K Dash Public Key, Secondary Public Key, Donation Public Key, Donation Percentage)

Every 15 minutes thereafter, a ping message is sent proving the node is still alive.

Message: (1K DASH Input, Signature (using secondary key), Signature Time, Stop)

After a time-to-live has expired the network will remove an inactive node from the network, causing the node to not be used by clients or paid. Nodes can also ping the network constantly, but if they do not have their ports open, they will eventually be flagged as inactive and not be paid.

2.6 Propagation of the Masternode List

New clients entering the Dash network must be made aware of the currently active Masternodes on the network to be able to utilize their services. As soon as they join the mesh network, a command is sent to their peers asking for the known list of Masternodes. A cache object is used for clients to record Masternodes and their current status, so when clients restart they will simply load this file rather than asking for the full list of Masternodes.

2.7 Payments via Mining and Enforcement

To ensure that each Masternode is paid its fair share of the block reward, the network must enforce that blocks pay the correct Masternode. If a miner is non-compliant their blocks must be rejected by the network, otherwise cheating will be incentivized.

We propose a strategy where Masternodes form quorums, select a winning Masternode and broadcast their message. After N messages have been broadcast to select the same target payee, a consensus will be formed and that block in question will be required to pay that Masternode.

When mining on the network, pool software (websites that merge the efforts of individual miners) use the RPC API interface to get information about how to make a block. To pay the Masternodes, this interface must be extended by adding a secondary payee to GetBlockTemplate. Pools then propagate their successfully mined blocks, with a split payment between themselves and a Masternode.

3 PrivateSend

We believe it is important to have a standard trust-less implementation for improving the privacy of it’s users in the reference client that provides a high degree of privacy. Other clients such as electrum, Android and iPhone will also have the same anonymity layer implemented directly and utilize the protocol extensions. This allows users a common experience anonymizing funds using a well understood system.

PrivateSend is an improved and extended version of the CoinJoin. In addition to the core concept of CoinJoin, we employ a series of improvements such as decentralization, strong anonymity by using a chaining approach, denominations and passive ahead-of-time mixing.

The greatest challenge when improving privacy and fungibility of a crypto-currency is doing it in a way that does not obscure the entire blockchain. In Bitcoin based crypto currencies, one can tell which outputs are unspent and which are not, commonly called UTXO, which stands for unspent transaction output. This results in a public ledger that allows any user to act as guarantor of the integrity of transactions. The Bitcoin protocol is designed to function without the participation of trusted counterparties, in their absence, it is critical that auditing capabilities remain readily accessible to the users through the public blockchain. Our goal is to improve privacy and fungibility without losing these key elements that we believe make a successful currency.

By having a decentralized mixing service within the currency we gain the ability to keep the currency itself perfectly fungible. Fungibility is an attribute of money, that dictates that all units of a currency should remain equal. When you receive money within a currency, it should not come with any history from the previous users of the currency or the users should have an easy way to disassociate themselves from that history, thus keeping all coins equal. At the same time, any user should be able to act as an auditor to guarantee the financial integrity of the public ledger without compromising others privacy.

To improve the fungibility and keep the integrity of the public blockchain, we propose using an ahead-of-time decentralized trustless mixing strategy. To be effective at keeping the currency fungible, this service is directly built into the currency, easy to use and safe for the average user.

3.1 Tracing CoinJoin By Amounts

A common strategy in existing Bitcoin implementations of CoinJoin is simply merging transactions together. This exposes the users to various methods of following the the users coins through these joined transaction.Figure 2: An example CoinJoin transaction with 2 users [11][12]

Figure 2: An example CoinJoin transaction with 2 users [11][12]

In this transaction, 0.05BTC was sent through the mixer. To identify the source of the money, one simply has to add up the values on the right until they match one of the values on the left.

Breaking apart the transaction:

  • 0.05 + 0.0499 + 0.0001(fee) = 0.10BTC.
  • 0.0499 + 0.05940182 + 0.0001(fee) = 0.10940182BTC.

This gets exponentially more difficult as more users are added to the mixer. However, these sessions can be retroactively de-anonymized at any point in the future.

3.2 Through linking and Forward Linking

In other proposed implementations of CoinJoin, it is possible that a user anonymizes money then eventually sends change from that transaction to an exchange or other entity that knows the users identity. This breaks the anonymity and allows the entity to walk backwards through that users transactions. We call this type of attack “Forward Linking”:

Figure 3: Forward Change Linking

Figure 3: Forward Change Linking

In this example, Alice anonymizes 1.2BTC, which goes to two outputs, 1BTC and 0.2BTC. She then spends 0.7BTC from the 1BTC output, receiving change of 0.3BTC. That 0.3BTC then goes to an identifiable source, confirming Alice also spent the 0.7BTC in the prior transaction.

To identify the sender of the anonymous transaction, start at the “exchange” transaction and go backwards in the blockchain till you get to the “Alice sends 0.7BTC anonymously”. As the exchange, you know it was your user who just recently bought something anonymously, thus breaking the anonymity completely. We call this type of attack “Through Change Linking”.

Figure 4: Through Change Linking

Figure 4: Through Change Linking

In the second example, Alice buys 1.2 BTC from coinbase, then anonymizes this amount into a 1BTC output. She then spends the 1BTC, receives change in the amount of 0.3BTC and then combines that with her 0.2BTC earlier change.

By combining the change from the anonymous transaction (0.3BTC) and the change she received from the CoinJoin transaction, you can link the entire history before and after, completely breaking the anonymity.

3.3 Improved Privacy and Denial-of-service (DOS) resistance

PrivateSend uses the fact that a transaction can be formed by multiple parties and made out to multiple parties to merge funds together in a way where they cannot be uncoupled thereafter. Given that all PrivateSend transactions are setup for users to pay themselves, the system is highly secure against theft and users coins always remain safe. Currently to mix usi
ng PrivateSend requires at least three participants.

Figure 5: Three users submit denominated funds into a common transaction. Users pay themselves back in the form of new outputs, which are randomly ordered.

Figure 5: Three users submit denominated funds into a common transaction. Users pay themselves back in the form of new outputs, which are randomly ordered.

To improve the privacy of the system as a whole we propose using common denominations of 0.1DASH, 1DASH, 10DASH AND 100DASH. In each mixing session, all users should submit the same denominations as inputs and outputs. In addition to denominations, fees should be removed from the transactions and charged in bulk in separate, sporadic unlinkable transactions.

To address the possible DOS attacks, we propose all users submit a transaction as collateral to the pool when joining. This transaction will be made out to themselves and will pay a high fee to miners. In the case when a user submits a request to the mixing pool, they must provide collateral at the beginning of this exchange. If at any point any user fails to cooperate, by refusing to sign for example, the collateral transaction will automatically be broadcasted. This will make it expensive to do a sustained attack on the privacy network.

3.4 Passive Anonymization of funds and chaining

PrivateSend is limited to 1,000DASH per session and requires multiple sessions to thoroughly anonymize significant amounts of money. To make the user experience easy and make timing attacks very difficult, PrivateSend runs in a passive mode. At set intervals, a user’s client will request to join with other clients via a Masternode. Upon entry into the Masternode, a queue object is propagated throughout the network detailing the denominations the user is looking to anonymize, but no information that can be used to identify the user.

Each PrivateSend session can be thought of as an independent event increasing the anonymity of user’s funds. However each session is limited to three clients, so an observer has a one in three chance of being able to follow a transaction. To increase the quality of anonymity provided, a chaining approach is employed, which funds are sent through multiple Masternodes, one after another.

Depth Of The Chain Possible Users (n)r
2 9
4 81
8 6561

Table 2. How many users could possibly be involved in N mixing sessions.

3.5 Security Considerations

As transactions are merged, Masternodes can possibly “snoop” on users funds as they pass through. This is not considered a serious limitation due to the requirement for Masternode’s to hold 1,000DASH and the fact that users utilize random Masternodes that they select to host their joins. The probability of following a transaction throughout a chaining event can be calculated as follows:

Attacker Controlled Masternodes / Total Masternodes Depth Of The Chain Probability of success (n/t)r DASH Required
10/1010 2 9.80e-05 10,000DASH
10/1010 4 9.60e-09 10,000DASH
10/1010 8 9.51e-11 10,000DASH
100/1100 2 8.26e-03 100,000DASH
100/1100 4 6.83e-05 100,000DASH
100/1100 8 4.66e-09 100,000DASH
1000/2000 2 25% 1,000,000DASH
1000/2000 4 6.25% 1,000,000DASH
1000/2000 8 0.39% 1,000,000DASH
2000/3000 2 44.4% 2,000,000DASH
2000/3000 4 19.75% 2,000,000DASH
2000/3000 8 3.90% 2,000,000DASH

Table 3. The probability of follow a PrivateSend transaction on the network given the attacker controls N Nodes.

Where:

n is the total number of nodes controlled by the attacker

t is the total number of Masternodes in the network

r is the depth of the chain

The selection of Masternodes is random.

Considering the limited supply of DASH (5.3 million at the time of writing, April 2015) and the low liquidity available on the market, it becomes an impossibility to attain a large enough number of Masternodes to succeed at such an attack.

Extending the system by blinding Masternodes to the transactions taking place on their node will also greatly enhance the security of the system.

3.6 Masternode Blinding via Relay System

In section 3.4 we describe the probabilities of following a single transaction through multiple sessions of PrivateSend mixing. This can further be addressed by blinding Masternodes, so they cannot see which inputs/outputs belong to which users. To do this we propose a simple relay system that users can use to protect their identity.

Instead of a user submitting the inputs and outputs directly into the pool, they will pick a random Masternode from the network and request that it relays the inputs/outputs/signatures to the target Masternode. This means that the Masternode will receive N sets of inputs/outputs and N sets of signatures. Each set belongs to one of the users, but the Masternode can’t know which belongs to which.

4 Instant Transactions via InstantSend

By utilizing Masternode quorums, users are able to send and receive instant irreversible transactions. Once a quorum has been formed, the inputs of the transaction are locked to only be spendable in a specific transaction, a transaction lock takes about four seconds to be set currently on the network. If consensus is reached on a lock by the Masternode network, all conflicting transactions or conflicting blocks would be rejected thereafter, unless they matched the exact transaction ID of the lock in place.

This will allow vendors to use mobile devices in place of traditional POS systems for real world commerce and users to quickly settle face-to-face non commercial transactions as with traditional cash. This is done without a central authority. An extensive overview of this feature can be found in the InstantSend white paper[9].

5 Additional Improvements

5.1 x11 hashing algorithm

x11 is a widely used hashing algorithm, which takes a different approach, known as algorithm chaining. x11 consists of all 11 SHA3 contestants[13], each hash is calculated then submitted to the next algorithm in the chain. By utilizing multiple algorithms, the likelihood that an ASIC is created for the currency is minimal until a later part of it’s life cycle.

In the life cycle of Bitcoin, mining began with hobbyists which used Central Processing Units (CPUs) to mine the currency, then shortly after Graphics Processing Units (GPUs) software was created, which quickly replaced the CPUs. Years after the GPUs cycle, ASICs or Application Specific Integrated Circuits were created, which quickly replaced the GPUs.

Due to the complexity and die size required to create an ASIC for mining x11, we expect that it will take considerably longer than it did in Bitcoin, allowing for hobbyists to take part in the mining for a longer period of time. We believe this is highly important for good distribution and growth of a cryptocurrency.

Another benefit of the chaining hashing approach is high end CPUs give an average return similar to that of GPUs. Also GPUs have been reported to run 30-50% cooler, with less wattage than the Scrypt algorithm used by most current crypto-currencies.

5.2 Mining Supply

A different approach to restricting the inflation of mining is taken in Dash, using a 7% reduction of the supply per year. This is done as opposed to halving implemented by other currencies. In addition supply each block is directly tied to the amount of miners on the network; more miners result in lower mining rewards.

Production of Dash is scheduled to carry on throughout this century and onto the next, slowly grinding down until finally near the year 2150, production will cease.

Figure 6: Mining Reward Schedule

Figure 6: Mining Reward Schedule

6 Conclusion

This paper introduces various concepts to improve the design of bitcoin resulting in improved privacy and fungibility for the average user, less price volatility and quicker message propagation throughout the network. This is all accomplished by utilizing an incentivized two-tier model, rather than the existing single-tier model in other crypto-currencies such as Bitcoin. By utilizing this alternative network design it becomes possible to add many types of services such as decentralized mixing of coins, instant transactions and decentralized oracles using masternode quorums.

References

  1. A peer-to-peer electronic cash system (2008)
  2. http://eprints.qut.edu.au/69169/1/Boyen_accepted_draft.pdf
  3. https://www.cryptocoinsnews.com/3-solutions-instant-bitcoin-confirmations/
  4. http://research.microsoft.com/pubs/156072/bitcoin.pdf
  5. http://www0.cs.ucl.ac.uk/staff/s.meiklejohn/files/imc13.pdf
  6. https://getaddr.bitnodes.io/nodes/incentive/
  7. https://medium.com/zapchain-magazine/why-don-t-people-run-bitcoin-nodes-anymore-d4da0b45aae5
  8. https://dashninja.pl/
  9. https://www.dash.org/wp-content/uploads/2014/09/InstantTX.pdf
  10. https://github.com/dashpay/dash/blob/master/src/Masternode-pos.cpp
  11. https://blockchain.info/tx/4eb3b2f9fe597d0aef6e43b58bbaa7b8fb727e645fa89f922952f3e57ee6d603
  12. https://blockchain.info/tx/1694122b34c8543d01ad422ce600d59f8d8fde495ac9ddd894edc7139aed7617
  13. http://en.wikipedia.org/wiki/NIST_hash_function_competition#Finalists
  14. http://www.tik.ee.ethz.ch/file/49318d3f56c1d525aabf7fda78b23fc0/P2P2013_041.pdf

 

NEO

Highlight to Annotate. Click to Learn.

NEO White Paper

A distributed network for the Smart Economy

NEO design goals: Smart Economy

NEO is the use of blockchain technology and digital identity to digitize assets, the use of smart contracts for digital assets to be self-managed, to achieve “smart economy” with a distributed network.

Digital Assets

Digital assets are programmable assets that exist in the form of electronic data. With blockchain technology, the digitization of assets can be decentralized, trustful, traceable, highly transparent, and free of intermediaries. On the NEO blockchain, users are able to register, trade, and circulate multiple types of assets. Proving the connection between digital and physical assets is possible through digital identity. Assets registered through a validated digital identity are protected by law.

NEO has two forms of digital assets: global assets and contract assets. Global assets can be recorded in the system space and can be identified by all smart contracts and clients. Contract assets are recorded in the private storage area of the smart contract and require a compatible client to recognize them. Contract assets can adhere to certain standards in order to achieve compatibility with most clients.

Digital Identity

Digital identity refers to the identity information of individuals, organizations, and other entities that exist in electronic form. The more mature digital identity system is based on the PKI (Public Key Infrastructure) X.509 standard. In NEO, we will implement a set of X.509 compatible digital identity standards. This set of digital identity standards, in addition to compatible X.509 level certificate issuance model, will also support Web Of Trust point-to-point certificate issuance model. Our verification of identity when issuing or using digital identities includes the use of facial features, fingerprint, voice, SMS and other multi-factor authentication methods. At the same time, we will also use the blockchain to replace the Online Certificate Status Protocol (OCSP) to manage and record the X.509 Certificate Revocation List (CRL).

Smart Contract

The smart contract was first proposed by the cryptographer Nick Szabo in 1994, only five years after the creation of the World Wide Web. According to Szabo’s definition: When a pre-programmed condition is triggered, the smart contract will execute the corresponding contract terms. Blockchain technology provides us with a decentralized, tamper-resistant, highly reliable system in which smart contracts are very useful. NEO has an independent smart contract system: NeoContract.

The NeoContract smart contract system is the biggest feature of the seamless integration of the existing developer ecosystem. Developers do not need to learn a new programming language but use C#, Java and other mainstream programming languages in their familiar IDE environments (Visual Studio, Eclipse, etc.) for smart contract development, debugging and compilation. NEO’s Universal Lightweight Virtual Machine, NeoVM, has the advantages of high certainty, high concurrency, and high scalability. The NeoContract smart contract system will allow millions of developers around the world to quickly carry out the development of smart contracts. NeoContract will have a separate white paper describing the implementation details.

Application and Ecosystem

Ecosystem is the vitality of the open source community. In order to achieve the goal of an intelligent economic network, NEO will be committed to the development of its ecosystem, providing mature development tools, improving development of documents, organizing education and training activities, and providing financial support. We plan to support the following NEO-based applications and ecology and to reward improvements to the design of the experience:

🔹 Node Program

  • A fully functioning Full node PC program
  • A light node PC program with a better user experience
  • Web / Android / iOS clients that do not need to synchronize with the blockchain
  • Hardware wallet

🔹 Blockchain Explorer

🔹 SDK Development Kit

  • Support Java / Kotlin, .NET C # / VB, JavaScript / Typescript, Python, Go

🔹 Smart Contract Compiler and IDE Plugin

  • C# / VB.Net / F#, Visual Studio
  • Java / Kotlin, Eclipse
  • C / C++ / GO
  • JavaScript / TypeScript
  • Python / Ruby

🔹 Decentralized Applications

  • Smart fund
  • AI-assisted legal smart contract
  • Social networking
  • Automated tokens liquidity providers
  • Decentralized exchange
  • Secure communication protocol
  • Data exchange market
  • Intellectual property trading market
  • Prediction market
  • Advertising market
  • Hashpower market
  • NeoGas market

NEO Management Model

Economic Model

NEO has two native tokens, NEO (abbreviated symbol NEO) and NeoGas (abbreviated symbol GAS).

NEO, with a total of 100 million tokens, represents the right to manage the network. Management rights include voting for bookkeeping, NEO network parameter changes, and so on. The minimum unit of NEO is 1 and tokens cannot be subdivided.

GAS is the fuel token for the realization of NEO network resource control, with a maximum total limit of 100 million. The NEO network charges for the operation and storage of tokens and smart contracts, thereby creating economic incentives for bookkeepers and preventing the abuse of resources. The minimum unit of GAS is 0.00000001.

In the genesis block of the NEO network, 100 million NEOs are generated, GAS has not yet been generated. 100 million GAS, corresponding to the 100 million NEO, will be generated through a decay algorithm in about 22 years time to address holding NEO. If NEO is transferred to a new address, the subsequent GAS generated will be credited to the new address.

The NEO network will set a threshold by voting to exempt GAS from a certain amount of transfer transactions and smart contract operations to enhance the user experience. When a large amount of spam transactions occur, NeoID can be used to prioritize transactions and smart contracts with qualified identities. Transactions and smart contracts with no qualifying digital identities can get priority by paying GAS.

Distribution Mechanism

NEO distribution:

NEO’s 100 million tokens is divided into two portions. The first portion is 50 million tokens distributed proportionally to supporters of NEO during the crowdfunding. This portion has been distributed.

The second portion is 50 million NEO managed by the NEO Council to support NEO’s long-term development, operation and maintenance and ecosystem. The NEO in this portion has a lockout period of 1 year and is unlocked only after October 16, 2017. This portion will not enter the exchanges and is only for long-term support of NEO projects. The plans for it are as below:

🔹 10 million tokens (10% total) will be used to motivate NEO developers and members of the NEO Council

🔹 10 million tokens (10% total) will be used to motivate developers in the NEO ecosystem

🔹 15 million tokens (15% total) will be used to cross-invest in other block-chain projects, which are owned by the NEO Council and are used only for NEO projects

🔹 15 million (15% total) will be retained as contingency

🔹 The annual use of NEO in principle shall not exceed 15 million tokens

GAS distribution:

GAS is generated with each new block. The initial total amount of GAS is zero. With the increasing rate of new block generation, the total limit of 100 million GAS will be achieved in about 22 years. The interval between each block is about 15-20 seconds, and 2 million blocks are generated in about one year.

Each year around 2 million blocks will be generated and the initial generation will be 8 GAS per block. There will be an annual reduction of 1 GAS per block, per year, to coincide with the passing of every 2 million blocks. The reduction will continue down to just 1 GAS per block and will be held at that rate for around 22 years. After the 44 millionth block the total GAS generated will have reached 100 million and from this point there will be no further generation of GAS from new blocks.

According to this release curve, 16% of the GAS will be created in the first year, 52% of the GAS will be created in the first four years, and 80% of the GAS will be created in the first 12 years. These GAS will be distributed proportionally in accordance with the NEO holding ratio, recorded in the corresponding addresses. NEO holders can initiate a claim transaction at any time and claim these GAS tokens at their holding addresses.

Governance mechanism

Chain governance: NEO token holders are the network owners and managers, managing the network through voting in the network, using the GAS generated from NEO to utilize the functions in the network. NEO tokens can be transferred.

Off-chain governance: NEO Council consists of the founding members of the NEO project, under which the management committee, technical committee and the secretariat, respectively, are responsible for strategic decision-making, technical decision-making and specific implementation. The NEO Council is responsible to the NEO community for the promotion and development of NEO ecosystem as its primary objective.

NEO technology implementation

Consensus mechanism: dBFT

The dBFT is called the Delegated Byzantine Fault Tolerant, a Byzantine fault-tolerant consensus mechanism that enables large-scale participation in consensus through proxy voting. The holder of the NEO token can, by voting, pick the bookkeeper it supports. The selected group of bookkeepers, through BFT algorithm, reach a consensus and generate new blocks. Voting in the NEO network continues in real time, rather than in accordance with a fixed term.

The dBFT provides fault tolerance of f = ⌊ (n-1) / 3 ⌋ for a consensus system consisting of n consensus nodes. This fault tolerance also includes both security and availability, resistant to general and Byzantine failures, and is suitable for any network environment. dBFT has good finality, meaning that once confirmations are final, the block can not be bifurcated, and the transaction will not be revoked or rolled back.

In the NEO dBFT consensus mechanism, taking about 15 to 20 seconds to generate a block, the transaction throughput is measured up to about 1,000TPS, which is excellent performance among the public chains. Through appropriate optimization, there is potential to reach 10,000TPS, allowing it to support large-scale commercial applications.

The dBFT combines digital identity technology, meaning the bookkeepers can be a real name of the individual or institution. Thus, it is possible to freeze, revoke, inherit, retrieve, and ownership transfer due to judicial decisons on them. This facilitates the registration of compliant financial assets in the NEO network. The NEO network plans to support such operations when necessary.

Smart contract system: NeoContract

NEO’s smart contract system consists of three parts:

NeoVM – Universal Block Chain Virtual Machine:

NeoVM is a lightweight, general-purpose virtual machine whose architecture is very close to the JVM and .NET Runtime, similar to a virtual CPU that reads and executes instructions in the contract in sequence, performs process control based on the functionality of the instruction operations, logic operations and so on. It has a good start-up speed and versatility, is very suitable for small programs such as smart contracts, can also be ported to non-blockchain systems, or integrated with the IDE to provide an optimal development experience. NeoVM’s functionality can be extended, like introducing a JIT (real-time compiler) mechanism, thereby enhancing the efficiency of the implementation.

InteropService – Interoperable Services:

Used to load the blockchain ledger, digital assets, digital identity, persistent storage area, NeoFS, and other underlying services. They are like virtual machines that are provided for virtual machines, enabling smart contracts to access these services at run time to achieve some advanced functionality. Through this low-coupling design, NeoVM can be ported to any blockchain or even non-blockchain system used, increasing the utility of the smart contracts.

DevPack – Compiler and IDE plugin:

DevPack includes the high-level language compiler and the IDE plug-in. Because NeoVM’s architecture is very similar to JVM and .NET Runtime, the compilers in DevPack can compile Java byte code and .NET MSIL into NeoVM’s instruction set. Java / Kotlin, C# developers do not need to learn new languages and will be able to immediately start developing smart contracts in VS, Eclipse and other familiar IDE environments. This greatly reduces the learning curve for developing smart contracts, allowing us to easily build a vibrant community around NeoContract.

NeoContract can create a smart contract call tree through static analysis before running a smart contract. Through the deterministic call tree, the NEO node can dynamically fragment the smart contract to achieve theoretically unlimited expansion, which overcomes the “jamming effect” caused by the static fragmentation of other block chain systems.

Cross-chain interoperability agreement: NeoX

NeoX is a protocol that implements cross-chain interoperability. NeoX is divided into two parts: “cross-chain assets exchange protocol” and “cross-chain distributed transaction protocol.”

Cross-chain assets exchange agreement:

NeoX has been extended on existing double-stranded atomic assets exchange protocols to allow multiple participants to exchange assets across different chains and to ensure that all steps in the entire transaction process succeed or fail together. In order to achieve this function, we need to use NeoContract function to create a contract account for each participant. If other blockchains are not compatible with NeoContract, they can be compatible with NeoX as long as they can provide simple smart contract functionality.

Cross-chain distributed transaction protocol:

Cross-chain distributed transactions mean that multiple steps of a transaction are scattered across different blockchains and that the consistency of the entire transaction is ensured. This is an extension of cross-chain assets exchange, extending the behavior of assets exchange into arbitrary behavior. In layman’s terms, NeoX makes it possible for cross-chain smart contracts where a smart contract can perform different parts on multiple chains, either succeeding or reverting as a whole. This gives excellent possibilities for cross-chain collaborations and we are exploring cross-chain smart contract application scenarios.

Distributed Storage Protocol: NeoFS

NeoFS is a distributed storage protocol that utilizes Distributed Hash Table (DHT) technology. NeoFS indexes the data through file content (Hash) rather than file path (URI). Large files will be divided into fixed-size data blocks that are distributed and stored in many different nodes.

The main problem with this type of system is the need to find a balance between redundancy and reliability. NeoFS plans to solve this contradiction by means of token incentives and the establishment of backbone nodes. Users can choose the reliability requirements of the file. Files with low reliability requirements can be stored and accessed for free or almost free. Stable and reliable services for files with high reliability requirement will be provided by backbone nodes.

NeoFS will serve as one of the InteropService interoperability services under the NeoContract system, enabling smart contracts to store large files on the blockchain and set access for those files. In addition, NeoFS can be combined with digital identity so that digital certificates used by digital identities can be assigned, sent, and revoked without a central server to manage them. In the future, the old block data can be stored in NeoFS, so that most of the full nodes can release the old data for better scalability and at the same time, ensure the integrity of historical data.

Anti-quantum cryptography mechanism: NeoQS

The emergence of quantum computers poses a major challenge to RSA and ECC-based cryptographic mechanisms. Quantum computers can solve the large number of decomposition problems (which RSA relies on) and the elliptic curve discrete logarithm (which ECC relies on) in a very short time. NeoQS (Quantum Safe) is a lattice-based cryptographic mechanism. At present, quantum computers do not have the ability to quickly solve the Shortest Vector Problem (SVP) and the Closest Vector Problem (CVP), which is considered to be the most reliable algorithm for resisting quantum computers.

Summary

NEO is a distributed network that combines digital assets, digital identities and smart contracts. The NEO system will use DBFT, NeoX, NeoFS, NeoQS and many other original technologies, as the infrastructure for the intelligent economy of the future.

 

EOS

Highlight to Annotate. Click to Learn.

 

EOS.IO Technical White Paper v2

March 16, 2018

Abstract: The EOS.IO software introduces a new blockchain architecture designed to enable vertical and horizontal scaling of decentralized applications. This is achieved by creating an operating system-like construct upon which applications can be built. The software provides accounts, authentication, databases, asynchronous communication, and the scheduling of applications across many of CPU cores or clusters. The resulting technology is a blockchain architecture that may ultimately scale to millions of transactions per second, eliminates user fees, and allows for quick and easy deployment and maintenance of decentralized applications, in the context of a governed blockchain.

Background

Blockchain technology was introduced in 2008 with the launch of the Bitcoin currency, and since then entrepreneurs and developers have attempted to generalize the technology to support a wider range of applications on a single blockchain platform.

While a number of blockchain platforms have struggled to support functional decentralized applications, application specific blockchains such as the BitShares decentralized exchange (2014) and Steem social media platform (2016) have become heavily used blockchains with tens of thousands of daily active users. They have achieved this by increasing performance to thousands of transactions per second, reducing latency to 1.5 seconds, eliminating per-transaction fees, and providing a user experience similar to those currently provided by existing centralized services.

Existing blockchain platforms are burdened by large fees and limited computational capacity that prevent widespread blockchain adoption.

Requirements for Blockchain Applications

In order to gain widespread use, applications on the blockchain require a platform that is flexible enough to meet the following requirements:

Support Millions of Users

Competing with businesses such as eBay, Uber, AirBnB, and Facebook, require blockchain technology capable of handling tens of millions of active daily users. In certain cases, an application may not work unless a critical mass of users is reached and therefore a platform that can handle very large numbers of users is paramount.

Free Usage

Application developers need the flexibility to offer users free services; users should not have to pay in order to use the platform or benefit from its services. A blockchain platform that is free to use for users will likely gain more widespread adoption. Developers and businesses can then create effective monetization strategies.

Easy Upgrades and Bug Recovery

Businesses building blockchain based applications need the flexibility to enhance their applications with new features. The platform must support software and smart contract upgrades.

All non-trivial software is subject to bugs, even with the most rigorous of formal verification. The platform must be robust enough to fix bugs when they inevitably occur.

Low Latency

A good user experience demands reliable feedback with a delay of no more than a few seconds. Longer delays frustrate users and make applications built on a blockchain less competitive with existing non-blockchain alternatives. The platform should support low latency of transactions.

Sequential Performance

There are some applications that just cannot be implemented with parallel algorithms due to sequentially dependent steps. Applications such as exchanges need enough sequential performance to handle high volumes. Therefore, the platform should support fast sequential performance.

Parallel Performance

Large scale applications need to divide the workload across multiple CPUs and computers.

Consensus Algorithm (BFT-DPOS)

EOS.IO software utilizes the only known decentralized consensus algorithm proven capable of meeting the performance requirements of applications on the blockchain, Delegated Proof of Stake (DPOS). Under this algorithm, those who hold tokens on a blockchain adopting the EOS.IO software may select block producers through a continuous approval voting system. Anyone may choose to participate in block production and will be given an opportunity to produce blocks, provided they can persuade token holders to vote for them.

The EOS.IO software enables blocks to be produced exactly every 0.5 second and exactly one producer is authorized to produce a block at any given point in time. If the block is not produced at the scheduled time, then the block for that time slot is skipped. When one or more blocks are skipped, there is a 0.5 or more second gap in the blockchain.

Using the EOS.IO software, blocks are produced in rounds of 126 (6 blocks each, times 21 producers). At the start of each round 21 unique block producers are chosen by preference of votes cast by token holders. The selected producers are scheduled in an order agreed upon by 15 or more producers.

If a producer misses a block and has not produced any block within the last 24 hours they are removed from consideration until they notify the blockchain of their intention to start producing blocks again. This ensures the network operates smoothly by minimizing the number of blocks missed by not scheduling producers who are proven to be unreliable.

Under normal conditions a DPOS blockchain does not experience any forks because, rather than compete, the block producers cooperate to produce blocks. In the event there is a fork, consensus will automatically switch to the longest chain. This method works because the rate at which blocks are added to a blockchain fork is directly correlated to the percentage of block producers that share the same consensus. In other words, a blockchain fork with more producers on it will grow in length faster than one with fewer producers, because the fork with more producers will experience fewer missed blocks.

Furthermore, no block producer should be producing blocks on two forks at the same time. A block producer caught doing this will likely be voted out. Cryptographic evidence of such double-production may also be used to automatically remove abusers.

Byzantine Fault Tolerance is added to traditional DPOS by allowing all producers to sign all blocks so long as no producer signs two blocks with the same timestamp or the same block height. Once 15 producers have signed a block the block is deemed irreversible. Any byzantine producer would have to generate cryptographic evidence of their treason by signing two blocks with the same timestamp or blockheight. Under this model a irreversible consensus should be reachable within 1 second.

Transaction Confirmation

Typical DPOS blockchains have 100% block producer participation. A transaction can be considered confirmed with 99.9% certainty after an average of 0.25 seconds from time of broadcast.

In addition to DPOS, EOS.IO adds asynchronous Byzantine Fault Tolerance (aBFT) for faster achievement of irreversibility. The aBFT algorithm provides 100% confirmation of irreversibility within 1 second.

Transaction as Proof of Stake (TaPoS)

The EOS.IO software requires every transaction to include part of the hash of a recent block header. This hash serves two purposes:

  1. prevents a replay of a transaction on forks that do not include the referenced block; and
  2. signals the network that a particular user and their stake are on a specific fork.

Over time all users end up directly confirming the blockchain which makes it difficult to forge counterfeit chains as the counterfeit would not be able to migrate transactions from the legitimate chain.

Accounts

The EOS.IO software permits all accounts to be referenced by a unique human readable name of up to 12 characters in length. The name is chosen by the creator of the account. The account creator must reserve the RAM required to store the new account until the new account stakes tokens to reserve its own RAM.

In a decentralized context, application developers will pay the nominal cost of account creation to sign up a new user. Traditional businesses already spend significant sums of money per customer they acquire in the form of advertising, free services, etc. The cost of funding a new blockchain account should be insignificant in comparison. Fortunately, there is no need to create accounts for users already signed up by another application.

Actions & Handlers

Each account can send structured Actions to other accounts and may define scripts to handle Actions when they are received. The EOS.IO software gives each account its own private database which can only be accessed by its own action handlers. Action handling scripts can also send Actions to other accounts. The combination of Actions and automated action handlers is how EOS.IO defines smart contracts.

To support parallel execution, each account can also define any number of scopes within their database. The block producers will schedule transaction in such a way that there is no conflict over memory access to scopes and therefore they can be executed in parallel.

Role Based Permission Management

Permission management involves determining whether or not an Action is properly authorized. The simplest form of permission management is checking that a transaction has the required signatures, but this implies that required signatures are already known. Generally, authority is bound to individuals or groups of individuals and is often compartmentalized. The EOS.IO software provides a declarative permission management system that gives accounts fine grained and high-level control over who can do what and when.

It is critical that authentication and permission management be standardized and separated from the business logic of the application. This enables tools to be developed to manage permissions in a general-purpose manner and also provide significant opportunities for performance optimization.

Every account may be controlled by any weighted combination of other accounts and private keys. This creates a hierarchical authority structure that reflects how permissions are organized in reality and makes multi-user control over accounts easier than ever. Multi-user control is the single biggest contributor to security, and, when used properly, it can greatly reduce the risk of theft due to hacking.

EOS.IO software allows accounts to define what combination of keys and/or accounts can send a particular Action type to another account. For example, it is possible to have one key for a user’s social media account and another for access to the exchange. It is even possible to give other accounts permission to act on behalf of a user’s account without assigning them keys.

Named Permission Levels

Using the EOS.IO software, accounts can define named permission levels each of which can be derived from higher level named permissions. Each named permission level defines an authority; an authority is a threshold multi-signature check consisting of keys and/or named permission levels of other accounts. For example, an account’s “Friend” permission level can be set for an Action on the account to be controlled equally by any of the account’s friends.

Another example is the Steem blockchain which has three hard-coded named permission levels: owner, active, and posting. The posting permission can only perform social actions such as voting and posting, while the active permission can do everything except change the owner. The owner permission is meant for cold storage and is able to do everything. The EOS.IO software generalizes this concept by allowing each account holder to define their own hierarchy as well as the grouping of actions.

Permission Mapping

EOS.IO software allows each account to define a mapping between a contract/action or contract of any other account and their own Named Permission Level. For example, an account holder could map the account holder’s social media application to the account holder’s “Friend” permission group. With this mapping, any friend could post as the account holder on the account holder’s social media. Even though they would post as the account holder, they would still use their own keys to sign the Action. This means it is always possible to identify which friends used the account and in what way.

Evaluating Permissions

When delivering an Action of type “Action”, from @alice to @bob the EOS.IO software will first check to see if @alice has defined a permission mapping for @bob.groupa.subgroup.Action. If nothing is found then a mapping for @bob.groupa.subgroup then @bob.groupa, and lastly @bob will be checked. If no further match is found, then the assumed mapping will be to the named permission group @alice.active.

Once a mapping is identified then signing authority is validated using the threshold multi-signature process and the authority associated with the named permission. If that fails, then it traverses up to the parent permission and ultimately to the owner permission, @alice.owner.

Default Permission Groups

The EOS.IO technology also allows all accounts to have an “owner” group which can do everything, and an “active” group which can do everything except change the owner group. All other permission groups are derived from “active”.

Parallel Evaluation of Permissions

The permission evaluation process is “read-only” and changes to permissions made by transactions do not take effect until the end of a block. This means that all keys and permission evaluation for all transactions can be executed in parallel. Furthermore, this means that a rapid validation of permission is possible without starting costly application logic that would have to be rolled back. Lastly, it means that transaction permissions can be evaluated as pending transactions are received and do not need to be re-evaluated as they are applied.

All things considered, permission verification represents a significant percentage of the computation required to validate transactions. Making this a read-only and trivially parallelizable process enables a dramatic increase in performance.

When replaying the blockchain to regenerate the deterministic state from the log of Actions there is no need to evaluate the permissions again. The fact that a transaction is included in a known good block is sufficient to skip this step. This dramatically reduces the computational load associated with replaying an ever growing blockchain.

Actions with Mandatory Delay

Time is a critical component of security. In most cases, it is not possible to know if a private key has been stolen until it has been used. Time based security is even more critical when people have applications that require keys be kept on computers connected to the internet for daily use. The EOS.IO software enables application developers to indicate that certain Actions must wait a minimum period of time after being included in a block before they can be applied. During this time, they can be cancelled.

Users can then receive notice via email or text message when one of these Actions is broadcast. If they did not authorize it, then they can use the account recovery process to recover their account and retract the Action.

The required delay depends upon how sensitive an operation is. Paying for a coffee might have no delay and be irreversible in seconds, while buying a house may require a 72 hour clearing period. Transferring an entire account to new control may take up to 30 days. The exact delays are chosen by application developers and users.

Recovery from Stolen Keys

The EOS.IO software provides users a way to restore control of their account when keys are stolen. An account owner can use any owner key that was active in the last 30 days along with approval from their designated account recovery partner to reset the owner key on their account. The account recovery partner cannot reset control of the account without the help of the owner.

There is nothing for the hacker to gain by attempting to go through the recovery process because they already “control” the account. Furthermore, if they did go through the process, the recovery partner would likely demand identification and multi-factor authentication (phone and email). This would likely compromise the hacker or gain the hacker nothing in the process.

This process is also very different from a simple multi-signature arrangement. With a multi-signature transaction, another entity is made a party to every transaction that is executed. By contrast, with the recovery process the recovery partner is only a party to the recovery process and has no power over the day-to-day transactions. This dramatically reduces costs and legal liabilities for everyone involved.

Deterministic Parallel Execution of Applications

Blockchain consensus depends upon deterministic (reproducible) behavior. This means all parallel execution must be free from the use of mutexes or other locking primitives. Without locks there must be some way to guarantee that transactions that may be executed in parallel do not create non-deterministic results.

The June 2018 release of EOS.IO software will run single threaded, yet it contains the data structures necessary for future multithreaded, parallel execution.

In an EOS.IO software-based blockchain, once parallel operation is enabled, it will be the job of the block producer to organize Action delivery into independent shards so that they can be evaluated in parallel. The schedule is the output of a block producer and will be deterministically executed, but the process for generating the schedule need not be deterministic. This means that block producers can utilize parallel algorithms to schedule transactions.

Part of parallel execution means that when a script generates a new Action it does not get delivered immediately, instead it is scheduled to be delivered in the next cycle. The reason it cannot be delivered immediately is because the receiver may be actively modifying its own state in another shard.

Minimizing Communication Latency

Latency is the time it takes for one account to send an Action to another account and then receive a response. The goal is to enable two accounts to exchange Actions back and forth within a single block without having to wait 0.5 seconds between each Action. To enable this, the EOS.IO software divides each block into cycles. Each cycle is divided into shards and each shard contains a list of transactions. Each transaction contains a set of Actions to be delivered. This structure can be visualized as a tree where alternating layers are processed sequentially and in parallel.

  Block

    Region

      Cycles (sequential)

        Shards (parallel)

          Transactions (sequential)

            Actions (sequential)

              Receiver and Notified Accounts (parallel)

Transactions generated in one cycle can be delivered in any subsequent cycle or block. Block producers will keep adding cycles to a block until the maximum wall clock time has passed or there are no new generated transactions to deliver.

It is possible to use static analysis of a block to verify that within a given cycle no two shards contain transactions that modify the same account. So long as that invariant is maintained a block can be processed by running all shards in parallel.

Read-Only Action Handlers

Some accounts may be able to process an Action on a pass/fail basis without modifying their internal state. If this is the case, then these handlers can be executed in parallel so long as only read-only Action handlers for a particular account are included in one or more shards within a particular cycle.

Atomic Transactions with Multiple Accounts

Sometimes it is desirable to ensure that Actions are delivered to and accepted by multiple accounts atomically. In this case both Actions are placed in one transaction and both accounts will be assigned the same shard and the Actions applied sequentially.

Partial Evaluation of Blockchain State

Scaling blockchain technology necessitates that components are modular. Everyone should not have to run everything, especially if they only need to use a small subset of the applications.

An exchange application developer runs full nodes for the purpose of displaying the exchange state to its users. This exchange application has no need for the state associated with social media applications. EOS.IO software allows any full node to pick any subset of applications to run. Actions delivered to other applications are safely ignored if your application never depends upon the state of another contract.

Subjective Best Effort Scheduling

The EOS.IO software cannot obligate block producers to deliver any Action to any other account. Each block producer makes their own subjective measurement of the computational complexity and time required to process a transaction. This applies whether a transaction is generated by a user or automatically by a smart contract.

On a launched blockchain adopting the EOS.IO software, at a network level all transactions are billed a computational bandwidth cost based on the number of WASM instructions executed. However, each individual block producer using the software may calculate resource usage using their own algorithm and measurements. When a block producer concludes that a transaction or account has consumed a disproportionate amount of the computational capacity they simply reject the transaction when producing their own block; however, they will still process the transaction if other block producers consider it valid.

In general, so long as even 1 block producer considers a transaction as valid and under the resource usage limits then all other block producers will also accept it, but it may take up to 1 minute for the transaction to find that producer.

In some cases, a producer may create a block that includes transactions that are an order of magnitude outside of acceptable ranges. In this case the next block producer may opt to reject the block and the tie will be broken by the third producer. This is no different than what would happen if a large block caused network propagation delays. The community would notice a pattern of abuse and eventually remove votes from the rogue producer.

This subjective evaluation of computational cost frees the blockchain from having to precisely and deterministically measure how long something takes to run. With this design there is no need to precisely count instructions which dramatically increases opportunities for optimization without breaking consensus.

Deferred Transactions

EOS.IO Software supports deferred transactions that are scheduled to execute in the future. This enables computation to move to different shards and/or the creation of long-running processes that continuously schedule a continuance transaction.

Context Free Actions

A Context Free Action involves computations that depend only on transaction data, but not upon the blockchain state. Signature verification, for example, is a computation that requires only the transaction data and a signature to determine the public key that signed the transaction. This is one of the most expensive individual computations a blockchain must perform, but because this computation is context free it can be performed in parallel.

Context Free Actions are like other user Actions, except they lack access to the blockchain state to perform validation. Not only does this enable EOS.IO to process all Context Free Actions such as signature verification in parallel, but more importantly, this enables generalized signature verification.

With support for Context Free Actions, scalability techniques such as Sharding, Raiden, Plasma, State Channels, and others become much more parallelizable and practical. This development enables efficient inter-blockchain communication and potentially unlimited scalability.

Token Model and Resource Usage

PLEASE NOTE: CRYPTOGRAPHIC TOKENS REFERRED TO IN THIS WHITE PAPER REFER TO CRYPTOGRAPHIC TOKENS ON A LAUNCHED BLOCKCHAIN THAT ADOPTS THE EOS.IO SOFTWARE. THEY DO NOT REFER TO THE ERC-20 COMPATIBLE TOKENS BEING DISTRIBUTED ON THE ETHEREUM BLOCKCHAIN IN CONNECTION WITH THE EOS TOKEN DISTRIBUTION.

All blockchains are resource constrained and require a system to prevent abuse. With a blockchain that uses EOS.IO software, there are three broad classes of resources that are consumed by applications:

  1. Bandwidth and Log Storage (Disk);
  2. Computation and Computational Backlog (CPU); and
  3. State Storage (RAM).

Bandwidth and computation have two components, instantaneous usage and long-term usage. A blockchain maintains a log of all Actions and this log is ultimately stored and downloaded by all full nodes. With the log of Actions, it is possible to reconstruct the state of all applications.

The computational debt is calculations that must be performed to regenerate state from the Action log. If the computational debt grows too large then, it becomes necessary to take snapshots of the blockchain’s state and discard the blockchain’s history. If computational debt grows too quickly then it may take 6 months to replay 1 year worth of transactions. It is critical, therefore, that the computational debt be carefully managed.

Blockchain state storage is information that is accessible from application logic. It includes information such as order books and account balances. If the state is never read by the application, then it should not be stored. For example, blog post content and comments are not read by application logic, so they should not be stored in the blockchain’s state. Meanwhile the existence of a post/comment, the number of votes, and other properties do get stored as part of the blockchain’s state.

Block producers publish their available capacity for bandwidth, computation, and state. The EOS.IO software allows each account to consume a percentage of the available capacity proportional to the amount of tokens held in a 3-day staking contract. For example, if a blockchain based on the EOS.IO software is launched and if an account holds 1% of the total tokens distributable pursuant to that blockchain, then that account has the potential to utilize 1% of the state storage capacity.

Adopting the EOS.IO software on a launched blockchain means bandwidth and computational capacity are allocated on a fractional reserve basis because they are transient (unused capacity cannot be saved for future use). The algorithm used by EOS.IO software is similar to the algorithm used by Steem to rate-limit bandwidth usage.

Objective and Subjective Measurements

As discussed earlier, instrumenting computational usage has a significant impact on performance and optimization; therefore, all resource usage constraints are ultimately subjective, and enforcement is done by block producers according to their individual algorithms and estimates. These would typically be implemented by a block producer via the writing of a custom plugin.

That said, there are certain things that are trivial to measure objectively. The number of Actions delivered, and the size of the data stored in the internal database are cheap to measure objectively. The EOS.IO software enables block producers to apply the same algorithm over these objective measures but may choose to apply stricter subjective algorithms over subjective measurements.

Receiver Pays

Traditionally, it is the business that pays for office space, computational power, and other costs required to run the business. The customer buys specific products from the business and the revenue from those product sales is used to cover the business costs of operation. Similarly, no website obligates its visitors to make micropayments for visiting its website to cover hosting costs. Therefore, decentralized applications should not force its customers to pay the blockchain directly for the use of the blockchain.

A launched blockchain that uses the EOS.IO software does not require its users to pay the blockchain directly for its use and therefore does not constrain or prevent a business from determining its own monetization strategy for its products.

While it is true that the receiver can pay, EOS.IO enables the sender to pay for bandwidth, computation, and storage. This empowers application developers to pick the method that is best for their application. In many cases sender-pays significantly reduces complexity for application developers who do not want to implement their own rationing system. Application developers can delegate bandwidth and computation to their users and then let the “sender pays” model enforce the usage. From the perspective of the end user it is free, but from the perspective of the blockchain it is sender-pays.

Delegating Capacity

A holder of tokens on a blockchain launched adopting the EOS.IO software who may not have an immediate need to consume all or part of the available bandwidth, can delegate or rent such unconsumed bandwidth to others; the block producers running EOS.IO software on such blockchain will recognize this delegation of capacity and allocate bandwidth accordingly.

Separating Transaction costs from Token Value

One of the major benefits of the EOS.IO software is that the amount of bandwidth available to an application is entirely independent of any token price. If an application owner holds a relevant number of tokens on a blockchain adopting EOS.IO software, then the application can run indefinitely within a fixed state and bandwidth usage. In such case, developers and users are unaffected from any price volatility in the token market and therefore not reliant on a price feed. In other words, a blockchain that adopts the EOS.IO software enables block producers to naturally increase bandwidth, computation, and storage available per token independent of the token’s value.

A blockchain using EOS.IO software also awards block producers tokens every time they produce a block. The value of the tokens will impact the amount of bandwidth, storage, and computation a producer can afford to purchase; this model naturally leverages rising token values to increase network performance.

State Storage Costs

While bandwidth and computation can be delegated, storage of application state will require an application developer to hold tokens until that state is deleted. If state is never deleted, then the tokens are effectively removed from circulation.

Block Rewards

A blockchain that adopts the EOS.IO software will award new tokens to a block producer every time a block is produced. In these circumstances, the number of tokens created is determined by the median of the desired pay published by all block producers. The EOS.IO software may be configured to enforce a cap on producer awards such that the total annual increase in token supply does not exceed 5%.

Worker Proposal System

In addition to electing block producers, pursuant to a blockchain based on the EOS.IO software, token holders can elect a number of Worker Proposals designed to benefit the community. The winning proposals will receive tokens of up to a configured percent of the token inflation minus those tokens that have been paid to block producers. These proposals will receive tokens proportional to the votes each application has received from token holders, up to the amount they request for performing their work. The elected proposals can be replaced by newly elected proposals by token holders.

The system contracts that implement Worker Proposals may not be in place at initial launch in June 2018, but the funding mechanism will. It will begin to accumulate funds at the same time block producer awards start. Since the Worker Proposal System will be implemented in WASM it can be added at a later date without a fork.

Governance

Governance is the process by which people in a community:

  1. Reach consensus on subjective matters of collective action that cannot be captured entirely by software algorithms;
  2. Carry out the decisions they reach; and
  3. Alter the governance rules themselves via Constitutional amendments.

An EOS.IO software-based blockchain implements a governance process that efficiently directs the existing influence of block producers. Absent a defined governance process, prior blockchains relied ad hoc, informal, and often controversial governance processes that result in unpredictable outcomes.

A blockchain based on the EOS.IO software recognizes that power originates with the token holders who delegate that power to the block producers. The block producers are given limited and checked authority to freeze accounts, update defective applications, and propose hard forking changes to the underlying protocol.

Embedded into the EOS.IO software is the election of block producers. Before any change can be made to the blockchain these block producers must approve it. If the block producers refuse to make changes desired by the token holders then they can be voted out. If the block producers make changes without permission of the token holders then all other non-producing full-node validators (exchanges, etc) will reject the change.

Freezing Accounts

Sometimes a smart contact behaves in an aberrant or unpredictable manner and fails to perform as intended; other times an application or account may discover an exploit that enables it to consume an unreasonable amount of resources. When such issues inevitably occur, the block producers have the power to rectify such situations.

The block producers on all blockchains have the power to select which transactions are included in blocks which gives them the ability to freeze accounts. A blockchain using EOS.IO software formalizes this authority by subjecting the process of freezing an account to a 15/21 vote of active producers. If the producers abuse the power they can be voted out and an account will be unfrozen.

Changing Account Code

When all else fails and an “unstoppable application” acts in an unpredictable manner, a blockchain using EOS.IO software allows the block producers to replace the account’s code without hard forking the entire blockchain. Similar to the process of freezing an account, this replacement of the code requires a 15/21 vote of elected block producers.

Constitution

The EOS.IO software enables blockchains to establish a peer-to-peer terms of service agreement or a binding contract among those users who sign it, referred to as a “constitution”. The content of this constitution defines obligations among the users which cannot be entirely enforced by code and facilitates dispute resolution by establishing jurisdiction and choice of law along with other mutually accepted rules. Every transaction broadcast on the network must incorporate the hash of the constitution as part of the signature and thereby explicitly binds the signer to the contract.

The constitution also defines the human-readable intent of the source code protocol. This intent is used to identify the difference between a bug and a feature when errors occur and guides the community on what fixes are proper or improper.

Upgrading the Protocol & Constitution

The EOS.IO software defines the following process by which the protocol, as defined by the canonical source code and its constitution, can be updated:

  1. Block producers propose a change to the constitution and obtains 15/21 approval.
  2. Block producers maintain 15/21 approval of the new constitution for 30 consecutive days.
  3. All users are required to indicate acceptance of the new constitution as a condition of future transactions being processed.
  4. Block producers adopt changes to the source code to reflect the change in the constitution and propose it to the blockchain using the hash of the new constitution.
  5. Block producers maintain 15/21 approval of the new code for 30 consecutive days.
  6. Changes to the code take effect 7 days later, giving all non-producing full nodes 1 week to upgrade after ratification of the source code.
  7. All nodes that do not upgrade to the new code shut down automatically.

By default, configuration of the EOS.IO software, the process of updating the blockchain to add new features takes 2 to 3 months, while updates to fix non-critical bugs that do not require changes to the constitution can take 1 to 2 months.

Emergency Changes

The block producers may accelerate the process if a software change is required to fix a harmful bug or security exploit that is actively harming users. Generally speaking it could be against the constitution for accelerated updates to introduce new features or fix harmless bugs.

Scripts & Virtual Machines

The EOS.IO software will be first and foremost a platform for coordinating the delivery of authenticated messages (called Actions) to accounts. The details of scripting language and virtual machine are implementation specific details that are mostly independent from the design of the EOS.IO technology. Any language or virtual machine that is deterministic and properly sandboxed with sufficient performance can be integrated with the EOS.IO software API.

Schema Defined Actions

All Actions sent between accounts are defined by a schema which is part of the blockchain consensus state. This schema allows seamless conversion between binary and JSON representation of the Actions.

Schema Defined Database

Database state is also defined using a similar schema. This ensures that all data stored by all applications is in a format that can be interpreted as human readable JSON but stored and manipulated with the efficiency of binary.

Generic Multi Index Database API

Developing smart contracts requires a defined database schema to track, store, and find data. Developers commonly need the same data sorted or indexed by multiple fields and to maintain consistency among all the indices.

Separating Authentication from Application

To maximize parallelization opportunities and minimize the computational debt associated with regenerating application state from the transaction log, EOS.IO software separates validation logic into three sections:

  1. Validating that an Action is internally consistent;
  2. Validating that all preconditions are valid; and
  3. Modifying the application state.

Validating the internal consistency of a Action is read-only and requires no access to blockchain state. This means that it can be performed with maximum parallelism. Validating preconditions, such as required balance, is read-only and therefore can also benefit from parallelism. Only modification of application state requires write access and must be processed sequentially for each application.

Authentication is the read-only process of verifying that an Action can be applied. Application is actually doing the work. In real time both calculations are required to be performed, however once a transaction is included in the blockchain it is no longer necessary to perform the authentication operations.

Inter Blockchain Communication

EOS.IO software is designed to facilitate inter-blockchain communication. This is achieved by making it easy to generate proof of Action existence and proof of Action sequence. These proofs combined with an application architecture designed around Action passing enables the details of inter-blockchain communication and proof validation to be hidden from application developers, enabling high level abstractions to be presented to developers.

Merkle Proofs for Light Client Validation (LCV)

Integrating with other blockchains is much easier if clients do not need to process all transactions. After all, an exchange only cares about transfers in and out of the exchange and nothing more. It would also be ideal if the exchange chain could utilize lightweight merkle proofs of deposit rather than having to trust its own block producers entirely. At the very least a chain’s block producers would like to maintain the smallest possible overhead when synchronizing with another blockchain.

The goal of LCV is to enable the generation of relatively light-weight proof of existence that can be validated by anyone tracking a relatively light-weight data set. In this case the objective is to prove that a particular transaction was included in a particular block and that the block is included in the verified history of a particular blockchain.

Bitcoin supports validation of transactions assuming all nodes have access to the full history of block headers which amounts to 4MB of block headers per year. At 10 transactions per second, a valid proof requires about 512 bytes. This works well for a blockchain with a 10 minute block interval, but is no longer “light” for blockchains with a 0.5 second block interval.

The EOS.IO software enables lightweight proofs for anyone who has any irreversible block header after the point in which the transaction was included. Using the hash-linked structure shown it is possible to prove the existence of any transaction with a proof less than 1024 bytes in size.

Given any block id for a block in the blockchain, and the headers a trusted irreversible block. It is possible to prove that the block is included in the blockchain. This proof takes ceil(log2(N)) digests for its path, where N is the number of blocks in the chain. Given a digest method of SHA256, you can prove the existence of any block in a chain which contains 100 million blocks in 864 bytes.

There is little incremental overhead associated with producing blocks with the proper hash-linking to enable these proofs which means there is no reason not to generate blocks this way.

When it comes time to validate proofs on other chains there are a wide variety of time/ space/ bandwidth optimizations that can be made. Tracking all block headers (420 MB/year) will keep proof sizes small. Tracking only recent headers can offer a trade off between minimal long-term storage and proof size. Alternatively, a blockchain can use a lazy evaluation approach where it remembers intermediate hashes of past proofs. New proofs only have to include links to the known sparse tree. The exact approach used will necessarily depend upon the percentage of foreign blocks that include transactions referenced by merkle proof.

After a certain density of interconnectedness, it becomes more efficient to simply have one chain contain the entire block history of another chain and eliminate the need for proofs all together. For performance reasons, it is ideal to minimize the frequency of inter-chain proofs.

Latency of Interchain Communication

When communicating with another outside blockchain, block producers must wait until there is 100% certainty that a transaction has been irreversibly confirmed by the other blockchain before accepting it as a valid input. Using an EOS.IO software-based blockchain and DPOS with 0.5 second blocks and the addition of Byzantine Fault Tolerant irreversibility, this takes approximately 0.5 second. If any chain’s block producers do not wait for irreversibility it would be like an exchange crediting a deposit that was later reversed and could impact the validity of the blockchain’s consensus. The EOS.IO Software uses both DPOS and aBFT to provide rapid irreversibility.

Proof of Completeness

When using merkle proofs from outside blockchains, there is a significant difference between knowing that all transactions processed are valid and knowing that no transactions have been skipped or omitted. While it is impossible to prove that all of the most recent transactions are known, it is possible to prove that there have been no gaps in the transaction history. The EOS.IO software facilitates this by assigning a sequence number to every Action delivered to every account. A user can use these sequence numbers to prove that all Actions intended for a particular account have been processed and that they were processed in order.

Segregated Witness

The concept of Segregated Witness (SegWit) is that transaction signatures are not relevant after a transaction is immutably included in the blockchain. Once it is immutable the signature data can be pruned and everyone else can still derive the current state. Since signatures represent a large percentage of most transactions, SegWit represents a significant savings in disk usage and syncing time.

This same concept can apply to merkle proofs used for inter-blockchain communication. Once a proof is accepted and irreversibly logged into the blockchain, the 2KB of sha256 hashes used by the proof are no longer necessary to derive the proper blockchain state. In the case of inter-blockchain communication, this savings is 32x greater than the savings on normal signatures.

Another example of SegWit would be for Steem blog posts. Under this model a post would contain only the sha256(blog content) and the blog content would be in the segregated witness data. The block producer would verify that the content exists and has the given hash, but the blog content would not need to be stored in order to recover the current state from the blockchain log. This enables proof that the content was once known without having to store said content forever.

Conclusion

The EOS.IO software is designed from experience with proven concepts and best practices, and represents fundamental advancements in blockchain technology. The software is part of a holistic blueprint for a globally scalable blockchain society in which decentralized applications can be easily deployed and governed.

Bitcoin

Highlight to Annotate. Click to Learn.

A Peer-to-Peer Electronic Cash System

Satoshi Nakamoto
October 31, 2008

Abstract

A purely peer-to-peer version of electronic cash would allow online payments to be sent directly from one party to another without going through a financial institution. Digital signatures provide part of the solution, but the main benefits are lost if a trusted third party is still required to prevent double-spending. We propose a solution to the double-spending problem using a peer-to-peer network. The network timestamps transactions by hashing them into an ongoing chain of hash-based proof-of-work, forming a record that cannot be changed without redoing the proof-of-work. The longest chain not only serves as proof of the sequence of events witnessed, but proof that it came from the largest pool of CPU power. As long as a majority of CPU power is controlled by nodes that are not cooperating to attack the network, they’ll generate the longest chain and outpace attackers. The network itself requires minimal structure. Messages are broadcast on a best effort basis, and nodes can leave and rejoin the network at will, accepting the longest proof-of-work chain as proof of what happened while they were gone.

1. Introduction

Commerce on the Internet has come to rely almost exclusively on financial institutions serving as trusted third parties to process electronic payments. While the system works well enough for most transactions, it still suffers from the inherent weaknesses of the trust based model. Completely non-reversible transactions are not really possible, since financial institutions cannot avoid mediating disputes. The cost of mediation increases transaction costs, limiting the minimum practical transaction size and cutting off the possibility for small casual transactions, and there is a broader cost in the loss of ability to make non-reversible payments for non-reversible services. With the possibility of reversal, the need for trust spreads. Merchants must be wary of their customers, hassling them for more information than they would otherwise need. A certain percentage of fraud is accepted as unavoidable. These costs and payment uncertainties can be avoided in person by using physical currency, but no mechanism exists to make payments over a communications channel without a trusted party.

What is needed is an electronic payment system based on cryptographic proof instead of trust, allowing any two willing parties to transact directly with each other without the need for a trusted third party. Transactions that are computationally impractical to reverse would protect sellers from fraud, and routine escrow mechanisms could easily be implemented to protect buyers. In this paper, we propose a solution to the double-spending problem using a peer-to-peer distributed timestamp server to generate computational proof of the chronological order of transactions. The system is secure as long as honest nodes collectively control more CPU power than any cooperating group of attacker nodes.

2. Transactions

We define an electronic coin as a chain of digital signatures. Each owner transfers the coin to the next by digitally signing a hash of the previous transaction and the public key of the next owner and adding these to the end of the coin. A payee can verify the signatures to verify the chain of ownership.

The problem of course is the payee can’t verify that one of the owners did not double-spend the coin. A common solution is to introduce a trusted central authority, or mint, that checks every transaction for double spending. After each transaction, the coin must be returned to the mint to issue a new coin, and only coins issued directly from the mint are trusted not to be double-spent. The problem with this solution is that the fate of the entire money system depends on the company running the mint, with every transaction having to go through them, just like a bank.

We need a way for the payee to know that the previous owners did not sign any earlier transactions. For our purposes, the earliest transaction is the one that counts, so we don’t care about later attempts to double-spend. The only way to confirm the absence of a transaction is to be aware of all transactions. In the mint based model, the mint was aware of all transactions and decided which arrived first. To accomplish this without a trusted party, transactions must be publicly announced[1], and we need a system for participants to agree on a single history of the order in which they were received. The payee needs proof that at the time of each transaction, the majority of nodes agreed it was the first received.

3. Timestamp Server

The solution we propose begins with a timestamp server. A timestamp server works by taking a hash of a block of items to be timestamped and widely publishing the hash, such as in a newspaper or Usenet post[2-5]. The timestamp proves that the data must have existed at the time, obviously, in order to get into the hash. Each timestamp includes the previous timestamp in its hash, forming a chain, with each additional timestamp reinforcing the ones before it.

4. Proof-of-Work

To implement a distributed timestamp server on a peer-to-peer basis, we will need to use a proof-of-work system similar to Adam Back’s Hashcash[6], rather than newspaper or Usenet posts. The proof-of-work involves scanning for a value that when hashed, such as with SHA-256, the hash begins with a number of zero bits. The average work required is exponential in the number of zero bits required and can be verified by executing a single hash.

For our timestamp network, we implement the proof-of-work by incrementing a nonce in the block until a value is found that gives the block’s hash the required zero bits. Once the CPU effort has been expended to make it satisfy the proof-of-work, the block cannot be changed without redoing the work. As later blocks are chained after it, the work to change the block would include redoing all the blocks after it.

The proof-of-work also solves the problem of determining representation in majority decision making. If the majority were based on one-IP-address-one-vote, it could be subverted by anyone able to allocate many IPs. Proof-of-work is essentially one-CPU-one-vote. The majority decision is represented by the longest chain, which has the greatest proof-of-work effort invested in it. If a majority of CPU power is controlled by honest nodes, the honest chain will grow the fastest and outpace any competing chains. To modify a past block, an attacker would have to redo the proof-of-work of the block and all blocks after it and then catch up with and surpass the work of the honest nodes. We will show later that the probability of a slower attacker catching up diminishes exponentially as subsequent blocks are added.

To compensate for increasing hardware speed and varying interest in running nodes over time, the proof-of-work difficulty is determined by a moving average targeting an average number of blocks per hour. If they’re generated too fast, the difficulty increases.

5. Network

The steps to run the network are as follows:

  1. New transactions are broadcast to all nodes.
  2. Each node collects new transactions into a block.
  3. Each node works on finding a difficult proof-of-work for its block.
  4. When a node finds a proof-of-work, it broadcasts the block to all nodes.
  5. Nodes accept the block only if all transactions in it are valid and not already spent.
  6. Nodes express their acceptance of the block by working on creating the next block in the chain, using the hash of the accepted block as the previous hash.

Nodes always consider the longest chain to be the correct one and will keep working on extending it. If two nodes broadcast different versions of the next block simultaneously, some nodes may receive one or the other first. In that case, they work on the first one they received, but save the other branch in case it becomes longer. The tie will be broken when the next proof-of-work is found and one branch becomes longer; the nodes that were working on the other branch will then switch to the longer one.

New transaction broadcasts do not necessarily need to reach all nodes. As long as they reach many nodes, they will get into a block before long. Block broadcasts are also tolerant of dropped messages. If a node does not receive a block, it will request it when it receives the next block and realizes it missed one.

6. Incentive

By convention, the first transaction in a block is a special transaction that starts a new coin owned by the creator of the block. This adds an incentive for nodes to support the network, and provides a way to initially distribute coins into circulation, since there is no central authority to issue them. The steady addition of a constant of amount of new coins is analogous to gold miners expending resources to add gold to circulation. In our case, it is CPU time and electricity that is expended.

The incentive can also be funded with transaction fees. If the output value of a transaction is less than its input value, the difference is a transaction fee that is added to the incentive value of the block containing the transaction. Once a predetermined number of coins have entered circulation, the incentive can transition entirely to transaction fees and be completely inflation free.

The incentive may help encourage nodes to stay honest. If a greedy attacker is able to assemble more CPU power than all the honest nodes, he would have to choose between using it to defraud people by stealing back his payments, or using it to generate new coins. He ought to find it more profitable to play by the rules, such rules that favour him with more new coins than everyone else combined, than to undermine the system and the validity of his own wealth.

7. Reclaiming Disk Space

Once the latest transaction in a coin is buried under enough blocks, the spent transactions before it can be discarded to save disk space. To facilitate this without breaking the block’s hash, transactions are hashed in a Merkle Tree [7][2][5], with only the root included in the block’s hash. Old blocks can then be compacted by stubbing off branches of the tree. The interior hashes do not need to be stored.

A block header with no transactions would be about 80 bytes. If we suppose blocks are generated every 10 minutes, 80 bytes * 6 * 24 * 365 = 4.2MB per year. With computer systems typically selling with 2GB of RAM as of 2008, and Moore’s Law predicting current growth of 1.2GB per year, storage should not be a problem even if the block headers must be kept in memory.

8. Simplified Payment Verification

It is possible to verify payments without running a full network node. A user only needs to keep a copy of the block headers of the longest proof-of-work chain, which he can get by querying network nodes until he’s convinced he has the longest chain, and obtain the Merkle branch linking the transaction to the block it’s timestamped in. He can’t check the transaction for himself, but by linking it to a place in the chain, he can see that a network node has accepted it, and blocks added after it further confirm the network has accepted it.

As such, the verification is reliable as long as honest nodes control the network, but is more vulnerable if the network is overpowered by an attacker. While network nodes can verify transactions for themselves, the simplified method can be fooled by an attacker’s fabricated transactions for as long as the attacker can continue to overpower the network. One strategy to protect against this would be to accept alerts from network nodes when they detect an invalid block, prompting the user’s software to download the full block and alerted transactions to confirm the inconsistency. Businesses that receive frequent payments will probably still want to run their own nodes for more independent security and quicker verification.

9. Combining and Splitting Value

Although it would be possible to handle coins individually, it would be unwieldy to make a separate transaction for every cent in a transfer. To allow value to be split and combined, transactions contain multiple inputs and outputs. Normally there will be either a single input from a larger previous transaction or multiple inputs combining smaller amounts, and at most two outputs: one for the payment, and one returning the change, if any, back to the sender.

It should be noted that fan-out, where a transaction depends on several transactions, and those transactions depend on many more, is not a problem here. There is never the need to extract a complete standalone copy of a transaction’s history.

10. Privacy

The traditional banking model achieves a level of privacy by limiting access to information to the parties involved and the trusted third party. The necessity to announce all transactions publicly precludes this method, but privacy can still be maintained by breaking the flow of information in another place: by keeping public keys anonymous. The public can see that someone is sending an amount to someone else, but without information linking the transaction to anyone. This is similar to the level of information released by stock exchanges, where the time and size of individual trades, the “tape”, is made public, but without telling who the parties were.

As an additional firewall, a new key pair should be used for each transaction to keep them from being linked to a common owner. Some linking is still unavoidable with multi-input transactions, which necessarily reveal that their inputs were owned by the same owner. The risk is that if the owner of a key is revealed, linking could reveal other transactions that belonged to the same owner.

11. Calculations

We consider the scenario of an attacker trying to generate an alternate chain faster than the honest chain. Even if this is accomplished, it does not throw the system open to arbitrary changes, such as creating value out of thin air or taking money that never belonged to the attacker. Nodes are not going to accept an invalid transaction as payment, and honest nodes will never accept a block containing them. An attacker can only try to change one of his own transactions to take back money he recently spent.

The race between the honest chain and an attacker chain can be characterized as a Binomial Random Walk. The success event is the honest chain being extended by one block, increasing its lead by +1, and the failure event is the attacker’s chain being extended by one block, reducing the gap by -1.

The probability of an attacker catching up from a given deficit is analogous to a Gambler’s Ruin problem. Suppose a gambler with unlimited credit starts at a deficit and plays potentially an infinite number of trials to try to reach breakeven. We can calculate the probability he ever reaches breakeven, or that an attacker ever catches up with the honest chain, as follows[8]:

p= probability an honest node finds the next blockq= probability the attacker finds the next blockqz= probability the attacker will ever catch up from z blocks behind”>
p= probability an honest node finds the next blockq= probability the attacker finds the next blockqz= probability the attacker will ever catch up from z blocks behind”>pqqz=== probability an honest node finds the next block probability the attacker finds the next block probability the attacker will ever catch up from z blocks behind
p= probability an honest node finds the next blockq= probability the attacker finds the next blockqz= probability the attacker will ever catch up from z blocks behind
Screen Shot 2018-04-19 at 7.04.46 PM

Given our assumption that p>q, the probability drops exponentially as the number of blocks the attacker has to catch up with increases. With the odds against him, if he doesn’t make a lucky lunge forward early on, his chances become vanishingly small as he falls further behind.

We now consider how long the recipient of a new transaction needs to wait before being sufficiently certain the sender can’t change the transaction. We assume the sender is an attacker who wants to make the recipient believe he paid him for a while, then switch it to pay back to himself after some time has passed. The receiver will be alerted when that happens, but the sender hopes it will be too late.

The receiver generates a new key pair and gives the public key to the sender shortly before signing. This prevents the sender from preparing a chain of blocks ahead of time by working on it continuously until he is lucky enough to get far enough ahead, then executing the transaction at that moment. Once the transaction is sent, the dishonest sender starts working in secret on a parallel chain containing an alternate version of his transaction.

The recipient waits until the transaction has been added to a block and blocks have been linked after it. He doesn’t know the exact amount of progress the attacker has made, but assuming the honest blocks took the average expected time per block, the attacker’s potential progress will be a Poisson distribution with expected value:

Screen Shot 2018-04-19 at 7.05.43 PM

To get the probability the attacker could still catch up now, we multiply the Poisson density for each amount of progress he could have made by the probability he could catch up from that point:

Screen Shot 2018-04-19 at 7.07.14 PM.png

Rearranging to avoid summing the infinite tail of the distribution…

Screen Shot 2018-04-19 at 7.07.20 PM.png

Converting to C code…

#include 
double AttackerSuccessProbability(double q, int z)
{
	double p = 1.0 - q;
	double lambda = z * (q / p);
	double sum = 1.0;
	int i, k;
	for (k = 0; k <= z; k++)
	{
		double poisson = exp(-lambda);
		for (i = 1; i <= k; i++)
			poisson *= lambda / i;
		sum -= poisson * (1 - pow(q / p, z - k));
	}
	return sum;
}

Running some results, we can see the probability drop off exponentially with z.

q=0.1
z=0    P=1.0000000
z=1    P=0.2045873
z=2    P=0.0509779
z=3    P=0.0131722
z=4    P=0.0034552  
z=5    P=0.0009137
z=6    P=0.0002428
z=7    P=0.0000647
z=8    P=0.0000173
z=9    P=0.0000046
z=10   P=0.0000012

q=0.3
z=0    P=1.0000000
z=5    P=0.1773523
z=10   P=0.0416605
z=15   P=0.0101008
z=20   P=0.0024804
z=25   P=0.0006132
z=30   P=0.0001522
z=35   P=0.0000379
z=40   P=0.0000095
z=45   P=0.0000024
z=50   P=0.0000006

Solving for P less than 0.1%…

P < 0.001
q=0.10   z=5
q=0.15   z=8
q=0.20   z=11
q=0.25   z=15
q=0.30   z=24
q=0.35   z=41
q=0.40   z=89
q=0.45   z=340	

12. Conclusion

We have proposed a system for electronic transactions without relying on trust. We started with the usual framework of coins made from digital signatures, which provides strong control of ownership, but is incomplete without a way to prevent double-spending. To solve this, we proposed a peer-to-peer network using proof-of-work to record a public history of transactions that quickly becomes computationally impractical for an attacker to change if honest nodes control a majority of CPU power. The network is robust in its unstructured simplicity. Nodes work all at once with little coordination. They do not need to be identified, since messages are not routed to any particular place and only need to be delivered on a best effort basis. Nodes can leave and rejoin the network at will, accepting the proof-of-work chain as proof of what happened while they were gone. They vote with their CPU power, expressing their acceptance of valid blocks by working on extending them and rejecting invalid blocks by refusing to work on them. Any needed rules and incentives can be enforced with this consensus mechanism.

References

  1. W. Dai, “b-money,” http://www.weidai.com/bmoney.txt, 1998. 
  2. H. Massias, X.S. Avila, and J.-J. Quisquater, “Design of a secure timestamping service with minimal trust requirements,” In 20th Symposium on Information Theory in the Benelux, May 1999.  
  3. S. Haber, W.S. Stornetta, “How to time-stamp a digital document,” In Journal of Cryptology, vol 3, no 2, pages 99-111, 1991. 
  4. D. Bayer, S. Haber, W.S. Stornetta, “Improving the efficiency and reliability of digital time-stamping,” In Sequences II: Methods in Communication, Security and Computer Science, pages 329-334, 1993. 
  5. S. Haber, W.S. Stornetta, “Secure names for bit-strings,” In Proceedings of the 4th ACM Conference on Computer and Communications Security, pages 28-35, April 1997.  
  6. A. Back, “Hashcash – a denial of service counter-measure,” http://www.hashcash.org/papers/hashcash.pdf, 2002. 
  7. R.C. Merkle, “Protocols for public key cryptosystems,” In Proc. 1980 Symposium on Security and Privacy, IEEE Computer Society, pages 122-133, April 1980. 
  8. W. Feller, “An introduction to probability theory and its applications,” 1957. 

 

Ethereum

Highlight to Annotate. Click to Learn.

A Next-Generation Smart Contract and Decentralized Application Platform

Satoshi Nakamoto’s development of Bitcoin in 2009 has often been hailed as a radical development in money and currency, being the first example of a digital asset which simultaneously has no backing or intrinsic value and no centralized issuer or controller. However, another – arguably more important – part of the Bitcoin experiment is the underlying blockchain technology as a tool of distributed consensus, and attention is rapidly starting to shift to this other aspect of Bitcoin. Commonly cited alternative applications of blockchain technology include using on-blockchain digital assets to represent custom currencies and financial instruments (colored coins), the ownership of an underlying physical device (smart property), non-fungible assets such as domain names (Namecoin), as well as more complex applications involving having digital assets being directly controlled by a piece of code implementing arbitrary rules (smart contracts) or even blockchain-based decentralized autonomous organizations (DAOs). What Ethereum intends to provide is a blockchain with a built-in fully fledged Turing-complete programming language that can be used to create “contracts” that can be used to encode arbitrary state transition functions, allowing users to create any of the systems described above, as well as many others that we have not yet imagined, simply by writing up the logic in a few lines of code.

Introduction to Bitcoin and Existing Concepts

History

The concept of decentralized digital currency, as well as alternative applications like property registries, has been around for decades. The anonymous e-cash protocols of the 1980s and the 1990s, mostly reliant on a cryptographic primitive known as Chaumian blinding, provided a currency with a high degree of privacy, but the protocols largely failed to gain traction because of their reliance on a centralized intermediary. In 1998, Wei Dai’s b-money became the first proposal to introduce the idea of creating money through solving computational puzzles as well as decentralized consensus, but the proposal was scant on details as to how decentralized consensus could actually be implemented. In 2005, Hal Finney introduced a concept of reusable proofs of work, a system which uses ideas from b-money together with Adam Back’s computationally difficult Hashcash puzzles to create a concept for a cryptocurrency, but once again fell short of the ideal by relying on trusted computing as a backend. In 2009, a decentralized currency was for the first time implemented in practice by Satoshi Nakamoto, combining established primitives for managing ownership through public key cryptography with a consensus algorithm for keeping track of who owns coins, known as “proof of work”.

The mechanism behind proof of work was a breakthrough in the space because it simultaneously solved two problems. First, it provided a simple and moderately effective consensus algorithm, allowing nodes in the network to collectively agree on a set of canonical updates to the state of the Bitcoin ledger. Second, it provided a mechanism for allowing free entry into the consensus process, solving the political problem of deciding who gets to influence the consensus, while simultaneously preventing sybil attacks. It does this by substituting a formal barrier to participation, such as the requirement to be registered as a unique entity on a particular list, with an economic barrier – the weight of a single node in the consensus voting process is directly proportional to the computing power that the node brings. Since then, an alternative approach has been proposed called proof of stake, calculating the weight of a node as being proportional to its currency holdings and not computational resources; the discussion of the relative merits of the two approaches is beyond the scope of this paper but it should be noted that both approaches can be used to serve as the backbone of a cryptocurrency.

Bitcoin As A State Transition System

statetransition.png

From a technical standpoint, the ledger of a cryptocurrency such as Bitcoin can be thought of as a state transition system, where there is a “state” consisting of the ownership status of all existing bitcoins and a “state transition function” that takes a state and a transaction and outputs a new state which is the result. In a standard banking system, for example, the state is a balance sheet, a transaction is a request to move $X from A to B, and the state transition function reduces the value in A’s account by $X and increases the value in B’s account by $X. If A’s account has less than $X in the first place, the state transition function returns an error. Hence, one can formally define:

APPLY(S,TX) -> S' or ERROR

In the banking system defined above:

APPLY({ Alice: $50, Bob: $50 },"send $20 from Alice to Bob") = { Alice: $30, Bob: $70 }

But:

APPLY({ Alice: $50, Bob: $50 },"send $70 from Alice to Bob") = ERROR

The “state” in Bitcoin is the collection of all coins (technically, “unspent transaction outputs” or UTXO) that have been mined and not yet spent, with each UTXO having a denomination and an owner (defined by a 20-byte address which is essentially a cryptographic public keyfn. 1). A transaction contains one or more inputs, with each input containing a reference to an existing UTXO and a cryptographic signature produced by the private key associated with the owner’s address, and one or more outputs, with each output containing a new UTXO to be added to the state.

The state transition function APPLY(S,TX) -> S' can be defined roughly as follows:

  1. For each input in TX:
    • If the referenced UTXO is not in S, return an error.
    • If the provided signature does not match the owner of the UTXO, return an error.
  2. If the sum of the denominations of all input UTXO is less than the sum of the denominations of all output UTXO, return an error.
  3. Return S' with all input UTXO removed and all output UTXO added.

The first half of the first step prevents transaction senders from spending coins that do not exist, the second half of the first step prevents transaction senders from spending other people’s coins, and the second step enforces conservation of value. In order to use this for payment, the protocol is as follows. Suppose Alice wants to send 11.7 BTC to Bob. First, Alice will look for a set of available UTXO that she owns that totals up to at least 11.7 BTC. Realistically, Alice will not be able to get exactly 11.7 BTC; say that the smallest she can get is 6+4+2=12. She then creates a transaction with those three inputs and two outputs. The first output will be 11.7 BTC with Bob’s address as its owner, and the second output will be the remaining 0.3 BTC “change”, with the owner being Alice herself.

Mining

block_picture.jpg

If we had access to a trustworthy centralized service, this system would be trivial to implement; it could simply be coded exactly as described, using a centralized server’s hard drive to keep track of the state. However, with Bitcoin we are trying to build a decentralized currency system, so we will need to combine the state transaction system with a consensus system in order to ensure that everyone agrees on the order of transactions. Bitcoin’s decentralized consensus process requires nodes in the network to continuously attempt to produce packages of transactions called “blocks”. The network is intended to produce roughly one block every ten minutes, with each block containing a timestamp, a nonce, a reference to (ie. hash of) the previous block and a list of all of the transactions that have taken place since the previous block. Over time, this creates a persistent, ever-growing, “blockchain” that constantly updates to represent the latest state of the Bitcoin ledger.

The algorithm for checking if a block is valid, expressed in this paradigm, is as follows:

  1. Check if the previous block referenced by the block exists and is valid.
  2. Check that the timestamp of the block is greater than that of the previous blockfn. 2 and less than 2 hours into the future
  3. Check that the proof of work on the block is valid.
  4. Let S[0] be the state at the end of the previous block.
  5. Suppose TX is the block’s transaction list with n transactions. For all i in 0...n-1, set S[i+1] = APPLY(S[i],TX[i]) If any application returns an error, exit and return false.
  6. Return true, and register S[n] as the state at the end of this block.

Essentially, each transaction in the block must provide a valid state transition from what was the canonical state before the transaction was executed to some new state. Note that the state is not encoded in the block in any way; it is purely an abstraction to be remembered by the validating node and can only be (securely) computed for any block by starting from the genesis state and sequentially applying every transaction in every block. Additionally, note that the order in which the miner includes transactions into the block matters; if there are two transactions A and B in a block such that B spends a UTXO created by A, then the block will be valid if A comes before B but not otherwise.

The one validity condition present in the above list that is not found in other systems is the requirement for “proof of work”. The precise condition is that the double-SHA256 hash of every block, treated as a 256-bit number, must be less than a dynamically adjusted target, which as of the time of this writing is approximately 2187. The purpose of this is to make block creation computationally “hard”, thereby preventing sybil attackers from remaking the entire blockchain in their favor. Because SHA256 is designed to be a completely unpredictable pseudorandom function, the only way to create a valid block is simply trial and error, repeatedly incrementing the nonce and seeing if the new hash matches.

At the current target of ~2187, the network must make an average of ~269 tries before a valid block is found; in general, the target is recalibrated by the network every 2016 blocks so that on average a new block is produced by some node in the network every ten minutes. In order to compensate miners for this computational work, the miner of every block is entitled to include a transaction giving themselves 12.5 BTC out of nowhere. Additionally, if any transaction has a higher total denomination in its inputs than in its outputs, the difference also goes to the miner as a “transaction fee”. Incidentally, this is also the only mechanism by which BTC are issued; the genesis state contained no coins at all.

In order to better understand the purpose of mining, let us examine what happens in the event of a malicious attacker. Since Bitcoin’s underlying cryptography is known to be secure, the attacker will target the one part of the Bitcoin system that is not protected by cryptography directly: the order of transactions. The attacker’s strategy is simple:

  1. Send 100 BTC to a merchant in exchange for some product (preferably a rapid-delivery digital good)
  2. Wait for the delivery of the product
  3. Produce another transaction sending the same 100 BTC to himself
  4. Try to convince the network that his transaction to himself was the one that came first.

Once step (1) has taken place, after a few minutes some miner will include the transaction in a block, say block number 270. After about one hour, five more blocks will have been added to the chain after that block, with each of those blocks indirectly pointing to the transaction and thus “confirming” it. At this point, the merchant will accept the payment as finalized and deliver the product; since we are assuming this is a digital good, delivery is instant. Now, the attacker creates another transaction sending the 100 BTC to himself. If the attacker simply releases it into the wild, the transaction will not be processed; miners will attempt to run APPLY(S,TX) and notice that TX consumes a UTXO which is no longer in the state. So instead, the attacker creates a “fork” of the blockchain, starting by mining another version of block 270 pointing to the same block 269 as a parent but with the new transaction in place of the old one. Because the block data is different, this requires redoing the proof of work. Furthermore, the attacker’s new version of block 270 has a different hash, so the original blocks 271 to 275 do not “point” to it; thus, the original chain and the attacker’s new chain are completely separate. The rule is that in a fork the longest blockchain is taken to be the truth, and so legitimate miners will work on the 275 chain while the attacker alone is working on the 270 chain. In order for the attacker to make his blockchain the longest, he would need to have more computational power than the rest of the network combined in order to catch up (hence, “51% attack”).

Merkle Trees

SPV in bitcoin

Left: it suffices to present only a small number of nodes in a Merkle tree to give a proof of the validity of a branch.

Right: any attempt to change any part of the Merkle tree will eventually lead to an inconsistency somewhere up the chain.

An important scalability feature of Bitcoin is that the block is stored in a multi-level data structure. The “hash” of a block is actually only the hash of the block header, a roughly 200-byte piece of data that contains the timestamp, nonce, previous block hash and the root hash of a data structure called the Merkle tree storing all transactions in the block. A Merkle tree is a type of binary tree, composed of a set of nodes with a large number of leaf nodes at the bottom of the tree containing the underlying data, a set of intermediate nodes where each node is the hash of its two children, and finally a single root node, also formed from the hash of its two children, representing the “top” of the tree. The purpose of the Merkle tree is to allow the data in a block to be delivered piecemeal: a node can download only the header of a block from one source, the small part of the tree relevant to them from another source, and still be assured that all of the data is correct. The reason why this works is that hashes propagate upward: if a malicious user attempts to swap in a fake transaction into the bottom of a Merkle tree, this change will cause a change in the node above, and then a change in the node above that, finally changing the root of the tree and therefore the hash of the block, causing the protocol to register it as a completely different block (almost certainly with an invalid proof of work).

The Merkle tree protocol is arguably essential to long-term sustainability. A “full node” in the Bitcoin network, one that stores and processes the entirety of every block, takes up about 15 GB of disk space in the Bitcoin network as of April 2014, and is growing by over a gigabyte per month. Currently, this is viable for some desktop computers and not phones, and later on in the future only businesses and hobbyists will be able to participate. A protocol known as “simplified payment verification” (SPV) allows for another class of nodes to exist, called “light nodes”, which download the block headers, verify the proof of work on the block headers, and then download only the “branches” associated with transactions that are relevant to them. This allows light nodes to determine with a strong guarantee of security what the status of any Bitcoin transaction, and their current balance, is while downloading only a very small portion of the entire blockchain.

Alternative Blockchain Applications

The idea of taking the underlying blockchain idea and applying it to other concepts also has a long history. In 1998, Nick Szabo came out with the concept of secure property titles with owner authority, a document describing how “new advances in replicated database technology” will allow for a blockchain-based system for storing a registry of who owns what land, creating an elaborate framework including concepts such as homesteading, adverse possession and Georgian land tax. However, there was unfortunately no effective replicated database system available at the time, and so the protocol was never implemented in practice. After 2009, however, once Bitcoin’s decentralized consensus was developed a number of alternative applications rapidly began to emerge.

  • Namecoin – created in 2010, Namecoin is best described as a decentralized name registration database. In decentralized protocols like Tor, Bitcoin and BitMessage, there needs to be some way of identifying accounts so that other people can interact with them, but in all existing solutions the only kind of identifier available is a pseudorandom hash like 1LW79wp5ZBqaHW1jL5TCiBCrhQYtHagUWy. Ideally, one would like to be able to have an account with a name like “george”. However, the problem is that if one person can create an account named “george” then someone else can use the same process to register “george” for themselves as well and impersonate them. The only solution is a first-to-file paradigm, where the first registerer succeeds and the second fails – a problem perfectly suited for the Bitcoin consensus protocol. Namecoin is the oldest, and most successful, implementation of a name registration system using such an idea.
  • Colored coins – the purpose of colored coins is to serve as a protocol to allow people to create their own digital currencies – or, in the important trivial case of a currency with one unit, digital tokens, on the Bitcoin blockchain. In the colored coins protocol, one “issues” a new currency by publicly assigning a color to a specific Bitcoin UTXO, and the protocol recursively defines the color of other UTXO to be the same as the color of the inputs that the transaction creating them spent (some special rules apply in the case of mixed-color inputs). This allows users to maintain wallets containing only UTXO of a specific color and send them around much like regular bitcoins, backtracking through the blockchain to determine the color of any UTXO that they receive.
  • Metacoins – the idea behind a metacoin is to have a protocol that lives on top of Bitcoin, using Bitcoin transactions to store metacoin transactions but having a different state transition function, APPLY'. Because the metacoin protocol cannot prevent invalid metacoin transactions from appearing in the Bitcoin blockchain, a rule is added that if APPLY'(S,TX) returns an error, the protocol defaults to APPLY'(S,TX) = S. This provides an easy mechanism for creating an arbitrary cryptocurrency protocol, potentially with advanced features that cannot be implemented inside of Bitcoin itself, but with a very low development cost since the complexities of mining and networking are already handled by the Bitcoin protocol. Metacoins have been used to implement some classes of financial contracts, name registration and decentralized exchange.

Thus, in general, there are two approaches toward building a consensus protocol: building an independent network, and building a protocol on top of Bitcoin. The former approach, while reasonably successful in the case of applications like Namecoin, is difficult to implement; each individual implementation needs to bootstrap an independent blockchain, as well as building and testing all of the necessary state transition and networking code. Additionally, we predict that the set of applications for decentralized consensus technology will follow a power law distribution where the vast majority of applications would be too small to warrant their own blockchain, and we note that there exist large classes of decentralized applications, particularly decentralized autonomous organizations, that need to interact with each other.

The Bitcoin-based approach, on the other hand, has the flaw that it does not inherit the simplified payment verification features of Bitcoin. SPV works for Bitcoin because it can use blockchain depth as a proxy for validity; at some point, once the ancestors of a transaction go far enough back, it is safe to say that they were legitimately part of the state. Blockchain-based meta-protocols, on the other hand, cannot force the blockchain not to include transactions that are not valid within the context of their own protocols. Hence, a fully secure SPV meta-protocol implementation would need to backward scan all the way to the beginning of the Bitcoin blockchain to determine whether or not certain transactions are valid. Currently, all “light” implementations of Bitcoin-based meta-protocols rely on a trusted server to provide the data, arguably a highly suboptimal result especially when one of the primary purposes of a cryptocurrency is to eliminate the need for trust.

Scripting

Even without any extensions, the Bitcoin protocol actually does facilitate a weak version of a concept of “smart contracts”. UTXO in Bitcoin can be owned not just by a public key, but also by a more complicated script expressed in a simple stack-based programming language. In this paradigm, a transaction spending that UTXO must provide data that satisfies the script. Indeed, even the basic public key ownership mechanism is implemented via a script: the script takes an elliptic curve signature as input, verifies it against the transaction and the address that owns the UTXO, and returns 1 if the verification is successful and 0 otherwise. Other, more complicated, scripts exist for various additional use cases. For example, one can construct a script that requires signatures from two out of a given three private keys to validate (“multisig”), a setup useful for corporate accounts, secure savings accounts and some merchant escrow situations. Scripts can also be used to pay bounties for solutions to computational problems, and one can even construct a script that says something like “this Bitcoin UTXO is yours if you can provide an SPV proof that you sent a Dogecoin transaction of this denomination to me”, essentially allowing decentralized cross-cryptocurrency exchange.

However, the scripting language as implemented in Bitcoin has several important limitations:

  • Lack of Turing-completeness – that is to say, while there is a large subset of computation that the Bitcoin scripting language supports, it does not nearly support everything. The main category that is missing is loops. This is done to avoid infinite loops during transaction verification; theoretically it is a surmountable obstacle for script programmers, since any loop can be simulated by simply repeating the underlying code many times with an if statement, but it does lead to scripts that are very space-inefficient. For example, implementing an alternative elliptic curve signature algorithm would likely require 256 repeated multiplication rounds all individually included in the code.
  • Value-blindness – there is no way for a UTXO script to provide fine-grained control over the amount that can be withdrawn. For example, one powerful use case of an oracle contract would be a hedging contract, where A and B put in $1000 worth of BTC and after 30 days the script sends $1000 worth of BTC to A and the rest to B. This would require an oracle to determine the value of 1 BTC in USD, but even then it is a massive improvement in terms of trust and infrastructure requirement over the fully centralized solutions that are available now. However, because UTXO are all-or-nothing, the only way to achieve this is through the very inefficient hack of having many UTXO of varying denominations (eg. one UTXO of 2k for every k up to 30) and having O pick which UTXO to send to A and which to B.
  • Lack of state – UTXO can either be spent or unspent; there is no opportunity for multi-stage contracts or scripts which keep any other internal state beyond that. This makes it hard to make multi-stage options contracts, decentralized exchange offers or two-stage cryptographic commitment protocols (necessary for secure computational bounties). It also means that UTXO can only be used to build simple, one-off contracts and not more complex “stateful” contracts such as decentralized organizations, and makes meta-protocols difficult to implement. Binary state combined with value-blindness also mean that another important application, withdrawal limits, is impossible.
  • Blockchain-blindness – UTXO are blind to blockchain data such as the nonce, the timestamp and previous block hash. This severely limits applications in gambling, and several other categories, by depriving the scripting language of a potentially valuable source of randomness.

Thus, we see three approaches to building advanced applications on top of cryptocurrency: building a new blockchain, using scripting on top of Bitcoin, and building a meta-protocol on top of Bitcoin. Building a new blockchain allows for unlimited freedom in building a feature set, but at the cost of development time, bootstrapping effort and security. Using scripting is easy to implement and standardize, but is very limited in its capabilities, and meta-protocols, while easy, suffer from faults in scalability. With Ethereum, we intend to build an alternative framework that provides even larger gains in ease of development as well as even stronger light client properties, while at the same time allowing applications to share an economic environment and blockchain security.

Ethereum

The intent of Ethereum is to create an alternative protocol for building decentralized applications, providing a different set of tradeoffs that we believe will be very useful for a large class of decentralized applications, with particular emphasis on situations where rapid development time, security for small and rarely used applications, and the ability of different applications to very efficiently interact, are important. Ethereum does this by building what is essentially the ultimate abstract foundational layer: a blockchain with a built-in Turing-complete programming language, allowing anyone to write smart contracts and decentralized applications where they can create their own arbitrary rules for ownership, transaction formats and state transition functions. A bare-bones version of Namecoin can be written in two lines of code, and other protocols like currencies and reputation systems can be built in under twenty. Smart contracts, cryptographic “boxes” that contain value and only unlock it if certain conditions are met, can also be built on top of the platform, with vastly more power than that offered by Bitcoin scripting because of the added powers of Turing-completeness, value-awareness, blockchain-awareness and state.

Philosophy

The design behind Ethereum is intended to follow the following principles:

  1. Simplicity: the Ethereum protocol should be as simple as possible, even at the cost of some data storage or time inefficiency.fn. 3 An average programmer should ideally be able to follow and implement the entire specification,fn. 4 so as to fully realize the unprecedented democratizing potential that cryptocurrency brings and further the vision of Ethereum as a protocol that is open to all. Any optimization which adds complexity should not be included unless that optimization provides very substantial benefit.
  2. Universality: a fundamental part of Ethereum’s design philosophy is that Ethereum does not have “features”.fn. 5 Instead, Ethereum provides an internal Turing-complete scripting language, which a programmer can use to construct any smart contract or transaction type that can be mathematically defined. Want to invent your own financial derivative? With Ethereum, you can. Want to make your own currency? Set it up as an Ethereum contract. Want to set up a full-scale Daemon or Skynet? You may need to have a few thousand interlocking contracts, and be sure to feed them generously, to do that, but nothing is stopping you with Ethereum at your fingertips.
  3. Modularity: the parts of the Ethereum protocol should be designed to be as modular and separable as possible. Over the course of development, our goal is to create a program where if one was to make a small protocol modification in one place, the application stack would continue to function without any further modification. Innovations such as Ethash (see the Yellow Paper Appendix or wiki article), modified Patricia trees (Yellow Paper, wiki) and RLP (YP, wiki) should be, and are, implemented as separate, feature-complete libraries. This is so that even though they are used in Ethereum, even if Ethereum does not require certain features, such features are still usable in other protocols as well. Ethereum development should be maximally done so as to benefit the entire cryptocurrency ecosystem, not just itself.
  4. Agility: details of the Ethereum protocol are not set in stone. Although we will be extremely judicious about making modifications to high-level constructs, for instance with the sharding roadmap, abstracting execution, with only data availability enshrined in consensus. Computational tests later on in the development process may lead us to discover that certain modifications, e.g. to the protocol architecture or to the Ethereum Virtual Machine (EVM), will substantially improve scalability or security. If any such opportunities are found, we will exploit them.
  5. Non-discrimination and non-censorship: the protocol should not attempt to actively restrict or prevent specific categories of usage. All regulatory mechanisms in the protocol should be designed to directly regulate the harm and not attempt to oppose specific undesirable applications. A programmer can even run an infinite loop script on top of Ethereum for as long as they are willing to keep paying the per-computational-step transaction fee.

Ethereum Accounts

In Ethereum, the state is made up of objects called “accounts”, with each account having a 20-byte address and state transitions being direct transfers of value and information between accounts. An Ethereum account contains four fields:

  • The nonce, a counter used to make sure each transaction can only be processed once
  • The account’s current ether balance
  • The account’s contract code, if present
  • The account’s storage (empty by default)

“Ether” is the main internal crypto-fuel of Ethereum, and is used to pay transaction fees. In general, there are two types of accounts: externally owned accounts, controlled by private keys, and contract accounts, controlled by their contract code. An externally owned account has no code, and one can send messages from an externally owned account by creating and signing a transaction; in a contract account, every time the contract account receives a message its code activates, allowing it to read and write to internal storage and send other messages or create contracts in turn.

Note that “contracts” in Ethereum should not be seen as something that should be “fulfilled” or “complied with”; rather, they are more like “autonomous agents” that live inside of the Ethereum execution environment, always executing a specific piece of code when “poked” by a message or transaction, and having direct control over their own ether balance and their own key/value store to keep track of persistent variables.

Messages and Transactions

The term “transaction” is used in Ethereum to refer to the signed data package that stores a message to be sent from an externally owned account. Transactions contain:

  • The recipient of the message
  • A signature identifying the sender
  • The amount of ether to transfer from the sender to the recipient
  • An optional data field
  • A STARTGAS value, representing the maximum number of computational steps the transaction execution is allowed to take
  • A GASPRICE value, representing the fee the sender pays per computational step

The first three are standard fields expected in any cryptocurrency. The data field has no function by default, but the virtual machine has an opcode which a contract can use to access the data; as an example use case, if a contract is functioning as an on-blockchain domain registration service, then it may wish to interpret the data being passed to it as containing two “fields”, the first field being a domain to register and the second field being the IP address to register it to. The contract would read these values from the message data and appropriately place them in storage.

The STARTGAS and GASPRICE fields are crucial for Ethereum’s anti-denial of service model. In order to prevent accidental or hostile infinite loops or other computational wastage in code, each transaction is required to set a limit to how many computational steps of code execution it can use. The fundamental unit of computation is “gas”; usually, a computational step costs 1 gas, but some operations cost higher amounts of gas because they are more computationally expensive, or increase the amount of data that must be stored as part of the state. There is also a fee of 5 gas for every byte in the transaction data. The intent of the fee system is to require an attacker to pay proportionately for every resource that they consume, including computation, bandwidth and storage; hence, any transaction that leads to the network consuming a greater amount of any of these resources must have a gas fee roughly proportional to the increment.

Messages

Contracts have the ability to send “messages” to other contracts. Messages are virtual objects that are never serialized and exist only in the Ethereum execution environment. A message contains:

  • The sender of the message (implicit)
  • The recipient of the message
  • The amount of ether to transfer alongside the message
  • An optional data field
  • A STARTGAS value

Essentially, a message is like a transaction, except it is produced by a contract and not an external actor. A message is produced when a contract currently executing code executes the CALL opcode, which produces and executes a message. Like a transaction, a message leads to the recipient account running its code. Thus, contracts can have relationships with other contracts in exactly the same way that external actors can.

Note that the gas allowance assigned by a transaction or contract applies to the total gas consumed by that transaction and all sub-executions. For example, if an external actor A sends a transaction to B with 1000 gas, and B consumes 600 gas before sending a message to C, and the internal execution of C consumes 300 gas before returning, then B can spend another 100 gas before running out of gas.

Ethereum State Transition Function

ethertransition.png

The Ethereum state transition function, APPLY(S,TX) -> S' can be defined as follows:

  1. Check if the transaction is well-formed (ie. has the right number of values), the signature is valid, and the nonce matches the nonce in the sender’s account. If not, return an error.
  2. Calculate the transaction fee as STARTGAS * GASPRICE, and determine the sending address from the signature. Subtract the fee from the sender’s account balance and increment the sender’s nonce. If there is not enough balance to spend, return an error.
  3. Initialize GAS = STARTGAS, and take off a certain quantity of gas per byte to pay for the bytes in the transaction.
  4. Transfer the transaction value from the sender’s account to the receiving account. If the receiving account does not yet exist, create it. If the receiving account is a contract, run the contract’s code either to completion or until the execution runs out of gas.
  5. If the value transfer failed because the sender did not have enough money, or the code execution ran out of gas, revert all state changes except the payment of the fees, and add the fees to the miner’s account.
  6. Otherwise, refund the fees for all remaining gas to the sender, and send the fees paid for gas consumed to the miner.

For example, suppose that the contract’s code is:

if !self.storage[calldataload(0)]:
    self.storage[calldataload(0)] = calldataload(32)

Note that in reality the contract code is written in the low-level EVM code; this example is written in Serpent, one of our high-level languages, for clarity, and can be compiled down to EVM code. Suppose that the contract’s storage starts off empty, and a transaction is sent with 10 ether value, 2000 gas, 0.001 ether gasprice, and 64 bytes of data, with bytes 0-31 representing the number 2 and bytes 32-63 representing the string CHARLIE.fn. 6 The process for the state transition function in this case is as follows:

  1. Check that the transaction is valid and well formed.
  2. Check that the transaction sender has at least 2000 * 0.001 = 2 ether. If it is, then subtract 2 ether from the sender’s account.
  3. Initialize gas = 2000; assuming the transaction is 170 bytes long and the byte-fee is 5, subtract 850 so that there is 1150 gas left.
  4. Subtract 10 more ether from the sender’s account, and add it to the contract’s account.
  5. Run the code. In this case, this is simple: it checks if the contract’s storage at index 2is used, notices that it is not, and so it sets the storage at index 2 to the value CHARLIE. Suppose this takes 187 gas, so the remaining amount of gas is 1150 – 187 = 963
  6. Add 963 * 0.001 = 0.963 ether back to the sender’s account, and return the resulting state.

If there was no contract at the receiving end of the transaction, then the total transaction fee would simply be equal to the provided GASPRICE multiplied by the length of the transaction in bytes, and the data sent alongside the transaction would be irrelevant.

Note that messages work equivalently to transactions in terms of reverts: if a message execution runs out of gas, then that message’s execution, and all other executions triggered by that execution, revert, but parent executions do not need to revert. This means that it is “safe” for a contract to call another contract, as if A calls B with G gas then A’s execution is guaranteed to lose at most G gas. Finally, note that there is an opcode, CREATE, that creates a contract; its execution mechanics are generally similar to CALL, with the exception that the output of the execution determines the code of a newly created contract.

Code Execution

The code in Ethereum contracts is written in a low-level, stack-based bytecode language, referred to as “Ethereum virtual machine code” or “EVM code”. The code consists of a series of bytes, where each byte represents an operation. In general, code execution is an infinite loop that consists of repeatedly carrying out the operation at the current program counter (which begins at zero) and then incrementing the program counter by one, until the end of the code is reached or an error or STOP or RETURN instruction is detected. The operations have access to three types of space in which to store data:

  • The stack, a last-in-first-out container to which values can be pushed and popped
  • Memory, an infinitely expandable byte array
  • The contract’s long-term storage, a key/value store. Unlike stack and memory, which reset after computation ends, storage persists for the long term.

The code can also access the value, sender and data of the incoming message, as well as block header data, and the code can also return a byte array of data as an output.

The formal execution model of EVM code is surprisingly simple. While the Ethereum virtual machine is running, its full computational state can be defined by the tuple (block_state, transaction, message, code, memory, stack, pc, gas), where block_state is the global state containing all accounts and includes balances and storage. At the start of every round of execution, the current instruction is found by taking the pcthe byte of code (or 0 if pc >= len(code)), and each instruction has its own definition in terms of how it affects the tuple. For example, ADD pops two items off the stack and pushes their sum, reduces gasby 1 and increments pc by 1, and SSTORE pops the top two items off the stack and inserts the second item into the contract’s storage at the index specified by the first item. Although there are many ways to optimize Ethereum virtual machine execution via just-in-time compilation, a basic implementation of Ethereum can be done in a few hundred lines of code.

Blockchain and Mining

apply_block_diagram.png

The Ethereum blockchain is in many ways similar to the Bitcoin blockchain, although it does have some differences. The main difference between Ethereum and Bitcoin with regard to the blockchain architecture is that, unlike Bitcoin, Ethereum blocks contain a copy of both the transaction list and the most recent state. Aside from that, two other values, the block number and the difficulty, are also stored in the block. The basic block validation algorithm in Ethereum is as follows:

  1. Check if the previous block referenced exists and is valid.
  2. Check that the timestamp of the block is greater than that of the referenced previous block and less than 15 minutes into the future
  3. Check that the block number, difficulty, transaction root, uncle root and gas limit (various low-level Ethereum-specific concepts) are valid.
  4. Check that the proof of work on the block is valid.
  5. Let S[0] be the state at the end of the previous block.
  6. Let TX be the block’s transaction list, with n transactions. For all i in 0...n-1, set S[i+1] = APPLY(S[i],TX[i]). If any applications returns an error, or if the total gas consumed in the block up until this point exceeds the GASLIMIT, return an error.
  7. Let S_FINAL be S[n], but adding the block reward paid to the miner.
  8. Check if the Merkle tree root of the state S_FINAL is equal to the final state root provided in the block header. If it is, the block is valid; otherwise, it is not valid.

The approach may seem highly inefficient at first glance, because it needs to store the entire state with each block, but in reality efficiency should be comparable to that of Bitcoin. The reason is that the state is stored in the tree structure, and after every block only a small part of the tree needs to be changed. Thus, in general, between two adjacent blocks the vast majority of the tree should be the same, and therefore the data can be stored once and referenced twice using pointers (ie. hashes of subtrees). A special kind of tree known as a “Patricia tree” is used to accomplish this, including a modification to the Merkle tree concept that allows for nodes to be inserted and deleted, and not just changed, efficiently. Additionally, because all of the state information is part of the last block, there is no need to store the entire blockchain history – a strategy which, if it could be applied to Bitcoin, can be calculated to provide 5-20x savings in space.

A commonly asked question is “where” contract code is executed, in terms of physical hardware. This has a simple answer: the process of executing contract code is part of the definition of the state transition function, which is part of the block validation algorithm, so if a transaction is added into block B the code execution spawned by that transaction will be executed by all nodes, now and in the future, that download and validate block B.

Applications

In general, there are three types of applications on top of Ethereum. The first category is financial applications, providing users with more powerful ways of managing and entering into contracts using their money. This includes sub-currencies, financial derivatives, hedging contracts, savings wallets, wills, and ultimately even some classes of full-scale employment contracts. The second category is semi-financial applications, where money is involved but there is also a heavy non-monetary side to what is being done; a perfect example is self-enforcing bounties for solutions to computational problems. Finally, there are applications such as online voting and decentralized governance that are not financial at all.

Token Systems

On-blockchain token systems have many applications ranging from sub-currencies representing assets such as USD or gold to company stocks, individual tokens representing smart property, secure unforgeable coupons, and even token systems with no ties to conventional value at all, used as point systems for incentivization. Token systems are surprisingly easy to implement in Ethereum. The key point to understand is that all a currency, or token system, fundamentally is a database with one operation: subtract X units from A and give X units to B, with the proviso that (1) A had at least X units before the transaction and (2) the transaction is approved by A. All that it takes to implement a token system is to implement this logic into a contract.

The basic code for implementing a token system in Serpent looks as follows:

def send(to, value):
    if self.storage[msg.sender] >= value:
        self.storage[msg.sender] = self.storage[msg.sender] - value
        self.storage[to] = self.storage[to] + value

This is essentially a literal implementation of the “banking system” state transition function described further above in this document. A few extra lines of code need to be added to provide for the initial step of distributing the currency units in the first place and a few other edge cases, and ideally a function would be added to let other contracts query for the balance of an address. But that’s all there is to it. Theoretically, Ethereum-based token systems acting as sub-currencies can potentially include another important feature that on-chain Bitcoin-based meta-currencies lack: the ability to pay transaction fees directly in that currency. The way this would be implemented is that the contract would maintain an ether balance with which it would refund ether used to pay fees to the sender, and it would refill this balance by collecting the internal currency units that it takes in fees and reselling them in a constant running auction. Users would thus need to “activate” their accounts with ether, but once the ether is there it would be reusable because the contract would refund it each time.

Financial derivatives and Stable-Value Currencies

Financial derivatives are the most common application of a “smart contract”, and one of the simplest to implement in code. The main challenge in implementing financial contracts is that the majority of them require reference to an external price ticker; for example, a very desirable application is a smart contract that hedges against the volatility of ether (or another cryptocurrency) with respect to the US dollar, but doing this requires the contract to know what the value of ETH/USD is. The simplest way to do this is through a “data feed” contract maintained by a specific party (eg. NASDAQ) designed so that that party has the ability to update the contract as needed, and providing an interface that allows other contracts to send a message to that contract and get back a response that provides the price.

Given that critical ingredient, the hedging contract would look as follows:

  1. Wait for party A to input 1000 ether.
  2. Wait for party B to input 1000 ether.
  3. Record the USD value of 1000 ether, calculated by querying the data feed contract, in storage, say this is $x.
  4. After 30 days, allow A or B to “reactivate” the contract in order to send $x worth of ether (calculated by querying the data feed contract again to get the new price) to A and the rest to B.

Such a contract would have significant potential in crypto-commerce. One of the main problems cited about cryptocurrency is the fact that it’s volatile; although many users and merchants may want the security and convenience of dealing with cryptographic assets, they may not wish to face that prospect of losing 23% of the value of their funds in a single day. Up until now, the most commonly proposed solution has been issuer-backed assets; the idea is that an issuer creates a sub-currency in which they have the right to issue and revoke units, and provide one unit of the currency to anyone who provides them (offline) with one unit of a specified underlying asset (eg. gold, USD). The issuer then promises to provide one unit of the underlying asset to anyone who sends back one unit of the crypto-asset. This mechanism allows any non-cryptographic asset to be “uplifted” into a cryptographic asset, provided that the issuer can be trusted.

In practice, however, issuers are not always trustworthy, and in some cases the banking infrastructure is too weak, or too hostile, for such services to exist. Financial derivatives provide an alternative. Here, instead of a single issuer providing the funds to back up an asset, a decentralized market of speculators, betting that the price of a cryptographic reference asset (eg. ETH) will go up, plays that role. Unlike issuers, speculators have no option to default on their side of the bargain because the hedging contract holds their funds in escrow. Note that this approach is not fully decentralized, because a trusted source is still needed to provide the price ticker, although arguably even still this is a massive improvement in terms of reducing infrastructure requirements (unlike being an issuer, issuing a price feed requires no licenses and can likely be categorized as free speech) and reducing the potential for fraud.

Identity and Reputation Systems

The earliest alternative cryptocurrency of all, Namecoin, attempted to use a Bitcoin-like blockchain to provide a name registration system, where users can register their names in a public database alongside other data. The major cited use case is for a DNS system, mapping domain names like “bitcoin.org” (or, in Namecoin’s case, “bitcoin.bit”) to an IP address. Other use cases include email authentication and potentially more advanced reputation systems. Here is the basic contract to provide a Namecoin-like name registration system on Ethereum:

def register(name, value):
    if !self.storage[name]:
        self.storage[name] = value

The contract is very simple; all it is is a database inside the Ethereum network that can be added to, but not modified or removed from. Anyone can register a name with some value, and that registration then sticks forever. A more sophisticated name registration contract will also have a “function clause” allowing other contracts to query it, as well as a mechanism for the “owner” (ie. the first registerer) of a name to change the data or transfer ownership. One can even add reputation and web-of-trust functionality on top.

Decentralized File Storage

Over the past few years, there have emerged a number of popular online file storage startups, the most prominent being Dropbox, seeking to allow users to upload a backup of their hard drive and have the service store the backup and allow the user to access it in exchange for a monthly fee. However, at this point the file storage market is at times relatively inefficient; a cursory look at various existing solutions shows that, particularly at the “uncanny valley” 20-200 GB level at which neither free quotas nor enterprise-level discounts kick in, monthly prices for mainstream file storage costs are such that you are paying for more than the cost of the entire hard drive in a single month. Ethereum contracts can allow for the development of a decentralized file storage ecosystem, where individual users can earn small quantities of money by renting out their own hard drives and unused space can be used to further drive down the costs of file storage.

The key underpinning piece of such a device would be what we have termed the “decentralized Dropbox contract”. This contract works as follows. First, one splits the desired data up into blocks, encrypting each block for privacy, and builds a Merkle tree out of it. One then makes a contract with the rule that, every N blocks, the contract would pick a random index in the Merkle tree (using the previous block hash, accessible from contract code, as a source of randomness), and give X ether to the first entity to supply a transaction with a simplified payment verification-like proof of ownership of the block at that particular index in the tree. When a user wants to re-download their file, they can use a micropayment channel protocol (eg. pay 1 szabo per 32 kilobytes) to recover the file; the most fee-efficient approach is for the payer not to publish the transaction until the end, instead replacing the transaction with a slightly more lucrative one with the same nonce after every 32 kilobytes.

An important feature of the protocol is that, although it may seem like one is trusting many random nodes not to decide to forget the file, one can reduce that risk down to near-zero by splitting the file into many pieces via secret sharing, and watching the contracts to see each piece is still in some node’s possession. If a contract is still paying out money, that provides a cryptographic proof that someone out there is still storing the file.

Decentralized Autonomous Organizations

The general concept of a “decentralized autonomous organization” is that of a virtual entity that has a certain set of members or shareholders which, perhaps with a 67% majority, have the right to spend the entity’s funds and modify its code. The members would collectively decide on how the organization should allocate its funds. Methods for allocating a DAO’s funds could range from bounties, salaries to even more exotic mechanisms such as an internal currency to reward work. This essentially replicates the legal trappings of a traditional company or nonprofit but using only cryptographic blockchain technology for enforcement. So far much of the talk around DAOs has been around the “capitalist” model of a “decentralized autonomous corporation” (DAC) with dividend-receiving shareholders and tradable shares; an alternative, perhaps described as a “decentralized autonomous community”, would have all members have an equal share in the decision making and require 67% of existing members to agree to add or remove a member. The requirement that one person can only have one membership would then need to be enforced collectively by the group.

A general outline for how to code a DAO is as follows. The simplest design is simply a piece of self-modifying code that changes if two thirds of members agree on a change. Although code is theoretically immutable, one can easily get around this and have de-facto mutability by having chunks of the code in separate contracts, and having the address of which contracts to call stored in the modifiable storage. In a simple implementation of such a DAO contract, there would be three transaction types, distinguished by the data provided in the transaction:

  • [0,i,K,V] to register a proposal with index i to change the address at storage index K to value V
  • [0,i] to register a vote in favor of proposal i
  • [2,i] to finalize proposal i if enough votes have been made

The contract would then have clauses for each of these. It would maintain a record of all open storage changes, along with a list of who voted for them. It would also have a list of all members. When any storage change gets to two thirds of members voting for it, a finalizing transaction could execute the change. A more sophisticated skeleton would also have built-in voting ability for features like sending a transaction, adding members and removing members, and may even provide for Liquid Democracy-style vote delegation (ie. anyone can assign someone to vote for them, and assignment is transitive so if A assigns B and B assigns C then C determines A’s vote). This design would allow the DAO to grow organically as a decentralized community, allowing people to eventually delegate the task of filtering out who is a member to specialists, although unlike in the “current system” specialists can easily pop in and out of existence over time as individual community members change their alignments.

An alternative model is for a decentralized corporation, where any account can have zero or more shares, and two thirds of the shares are required to make a decision. A complete skeleton would involve asset management functionality, the ability to make an offer to buy or sell shares, and the ability to accept offers (preferably with an order-matching mechanism inside the contract). Delegation would also exist Liquid Democracy-style, generalizing the concept of a “board of directors”.

Further Applications

1. Savings wallets. Suppose that Alice wants to keep her funds safe, but is worried that she will lose or someone will hack her private key. She puts ether into a contract with Bob, a bank, as follows:

  • Alice alone can withdraw a maximum of 1% of the funds per day.
  • Bob alone can withdraw a maximum of 1% of the funds per day, but Alice has the ability to make a transaction with her key shutting off this ability.
  • Alice and Bob together can withdraw anything.

Normally, 1% per day is enough for Alice, and if Alice wants to withdraw more she can contact Bob for help. If Alice’s key gets hacked, she runs to Bob to move the funds to a new contract. If she loses her key, Bob will get the funds out eventually. If Bob turns out to be malicious, then she can turn off his ability to withdraw.

2. Crop insurance. One can easily make a financial derivatives contract but using a data feed of the weather instead of any price index. If a farmer in Iowa purchases a derivative that pays out inversely based on the precipitation in Iowa, then if there is a drought, the farmer will automatically receive money and if there is enough rain the farmer will be happy because their crops would do well. This can be expanded to natural disaster insurance generally.

3. A decentralized data feed. For financial contracts for difference, it may actually be possible to decentralize the data feed via a protocol called SchellingCoin. SchellingCoin basically works as follows: N parties all put into the system the value of a given datum (eg. the ETH/USD price), the values are sorted, and everyone between the 25th and 75th percentile gets one token as a reward. Everyone has the incentive to provide the answer that everyone else will provide, and the only value that a large number of players can realistically agree on is the obvious default: the truth. This creates a decentralized protocol that can theoretically provide any number of values, including the ETH/USD price, the temperature in Berlin or even the result of a particular hard computation.

4. Smart multisignature escrow. Bitcoin allows multisignature transaction contracts where, for example, three out of a given five keys can spend the funds. Ethereum allows for more granularity; for example, four out of five can spend everything, three out of five can spend up to 10% per day, and two out of five can spend up to 0.5% per day. Additionally, Ethereum multisig is asynchronous – two parties can register their signatures on the blockchain at different times and the last signature will automatically send the transaction.

5. Cloud computing. The EVM technology can also be used to create a verifiable computing environment, allowing users to ask others to carry out computations and then optionally ask for proofs that computations at certain randomly selected checkpoints were done correctly. This allows for the creation of a cloud computing market where any user can participate with their desktop, laptop or specialized server, and spot-checking together with security deposits can be used to ensure that the system is trustworthy (ie. nodes cannot profitably cheat). Although such a system may not be suitable for all tasks; tasks that require a high level of inter-process communication, for example, cannot easily be done on a large cloud of nodes. Other tasks, however, are much easier to parallelize; projects like SETI@home, folding@home and genetic algorithms can easily be implemented on top of such a platform.

6. Peer-to-peer gambling. Any number of peer-to-peer gambling protocols, such as Frank Stajano and Richard Clayton’s Cyberdice, can be implemented on the Ethereum blockchain. The simplest gambling protocol is actually simply a contract for difference on the next block hash, and more advanced protocols can be built up from there, creating gambling services with near-zero fees that have no ability to cheat.

7. Prediction markets. Provided an oracle or SchellingCoin, prediction markets are also easy to implement, and prediction markets together with SchellingCoin may prove to be the first mainstream application of futarchy as a governance protocol for decentralized organizations.

8. On-chain decentralized marketplaces, using the identity and reputation system as a base.

Miscellanea And Concerns

Modified GHOST Implementation

The “Greedy Heaviest Observed Subtree” (GHOST) protocol is an innovation first introduced by Yonatan Sompolinsky and Aviv Zohar in December 2013. The motivation behind GHOST is that blockchains with fast confirmation times currently suffer from reduced security due to a high stale rate – because blocks take a certain time to propagate through the network, if miner A mines a block and then miner B happens to mine another block before miner A’s block propagates to B, miner B’s block will end up wasted and will not contribute to network security. Furthermore, there is a centralization issue: if miner A is a mining pool with 30% hashpower and B has 10% hashpower, A will have a risk of producing a stale block 70% of the time (since the other 30% of the time A produced the last block and so will get mining data immediately) whereas B will have a risk of producing a stale block 90% of the time. Thus, if the block interval is short enough for the stale rate to be high, A will be substantially more efficient simply by virtue of its size. With these two effects combined, blockchains which produce blocks quickly are very likely to lead to one mining pool having a large enough percentage of the network hashpower to have de facto control over the mining process.

As described by Sompolinsky and Zohar, GHOST solves the first issue of network security loss by including stale blocks in the calculation of which chain is the “longest”; that is to say, not just the parent and further ancestors of a block, but also the stale descendants of the block’s ancestor (in Ethereum jargon, “uncles”) are added to the calculation of which block has the largest total proof of work backing it. To solve the second issue of centralization bias, we go beyond the protocol described by Sompolinsky and Zohar, and also provide block rewards to stales: a stale block receives 87.5% of its base reward, and the nephew that includes the stale block receives the remaining 12.5%. Transaction fees, however, are not awarded to uncles.

Ethereum implements a simplified version of GHOST which only goes down seven levels. Specifically, it is defined as follows:

  • A block must specify a parent, and it must specify 0 or more uncles
  • An uncle included in block B must have the following properties:
    • It must be a direct child of the k-th generation ancestor of B, where 2 <= k <= 7.
    • It cannot be an ancestor of B
    • An uncle must be a valid block header, but does not need to be a previously verified or even valid block
    • An uncle must be different from all uncles included in previous blocks and all other uncles included in the same block (non-double-inclusion)
  • For every uncle U in block B, the miner of B gets an additional 3.125% added to its coinbase reward and the miner of U gets 93.75% of a standard coinbase reward.

This limited version of GHOST, with uncles includable only up to 7 generations, was used for two reasons. First, unlimited GHOST would include too many complications into the calculation of which uncles for a given block are valid. Second, unlimited GHOST with compensation as used in Ethereum removes the incentive for a miner to mine on the main chain and not the chain of a public attacker.

Fees

Because every transaction published into the blockchain imposes on the network the cost of needing to download and verify it, there is a need for some regulatory mechanism, typically involving transaction fees, to prevent abuse. The default approach, used in Bitcoin, is to have purely voluntary fees, relying on miners to act as the gatekeepers and set dynamic minimums. This approach has been received very favorably in the Bitcoin community particularly because it is “market-based”, allowing supply and demand between miners and transaction senders determine the price. The problem with this line of reasoning is, however, that transaction processing is not a market; although it is intuitively attractive to construe transaction processing as a service that the miner is offering to the sender, in reality every transaction that a miner includes will need to be processed by every node in the network, so the vast majority of the cost of transaction processing is borne by third parties and not the miner that is making the decision of whether or not to include it. Hence, tragedy-of-the-commons problems are very likely to occur.

However, as it turns out this flaw in the market-based mechanism, when given a particular inaccurate simplifying assumption, magically cancels itself out. The argument is as follows. Suppose that:

  1. A transaction leads to k operations, offering the reward kR to any miner that includes it where R is set by the sender and k and R are (roughly) visible to the miner beforehand.
  2. An operation has a processing cost of C to any node (ie. all nodes have equal efficiency)
  3. There are N mining nodes, each with exactly equal processing power (ie. 1/N of total)
  4. No non-mining full nodes exist.

A miner would be willing to process a transaction if the expected reward is greater than the cost. Thus, the expected reward is kR/N since the miner has a 1/N chance of processing the next block, and the processing cost for the miner is simply kC. Hence, miners will include transactions where kR/N > kC, or R > NC. Note that R is the per-operation fee provided by the sender, and is thus a lower bound on the benefit that the sender derives from the transaction, and NC is the cost to the entire network together of processing an operation. Hence, miners have the incentive to include only those transactions for which the total utilitarian benefit exceeds the cost.

However, there are several important deviations from those assumptions in reality:

  1. The miner does pay a higher cost to process the transaction than the other verifying nodes, since the extra verification time delays block propagation and thus increases the chance the block will become a stale.
  2. There do exist nonmining full nodes.
  3. The mining power distribution may end up radically inegalitarian in practice.
  4. Speculators, political enemies and crazies whose utility function includes causing harm to the network do exist, and they can cleverly set up contracts where their cost is much lower than the cost paid by other verifying nodes.

(1) provides a tendency for the miner to include fewer transactions, and (2) increases NC; hence, these two effects at least partially cancel each other out.How? (3) and (4) are the major issue; to solve them we simply institute a floating cap: no block can have more operations than BLK_LIMIT_FACTOR times the long-term exponential moving average. Specifically:

blk.oplimit = floor((blk.parent.oplimit * (EMAFACTOR - 1) + floor(parent.opcount * BLK_LIMIT_FACTOR)) / EMA_FACTOR)

BLK_LIMIT_FACTOR and EMA_FACTOR are constants that will be set to 65536 and 1.5 for the time being, but will likely be changed after further analysis.

There is another factor disincentivizing large block sizes in Bitcoin: blocks that are large will take longer to propagate, and thus have a higher probability of becoming stales. In Ethereum, highly gas-consuming blocks can also take longer to propagate both because they are physically larger and because they take longer to process the transaction state transitions to validate. This delay disincentive is a significant consideration in Bitcoin, but less so in Ethereum because of the GHOST protocol; hence, relying on regulated block limits provides a more stable baseline.

Computation And Turing-Completeness

An important note is that the Ethereum virtual machine is Turing-complete; this means that EVM code can encode any computation that can be conceivably carried out, including infinite loops. EVM code allows looping in two ways. First, there is a JUMP instruction that allows the program to jump back to a previous spot in the code, and a JUMPI instruction to do conditional jumping, allowing for statements like while x < 27: x = x * 2. Second, contracts can call other contracts, potentially allowing for looping through recursion. This naturally leads to a problem: can malicious users essentially shut miners and full nodes down by forcing them to enter into an infinite loop? The issue arises because of a problem in computer science known as the halting problem: there is no way to tell, in the general case, whether or not a given program will ever halt.

As described in the state transition section, our solution works by requiring a transaction to set a maximum number of computational steps that it is allowed to take, and if execution takes longer computation is reverted but fees are still paid. Messages work in the same way. To show the motivation behind our solution, consider the following examples:

  • An attacker creates a contract which runs an infinite loop, and then sends a transaction activating that loop to the miner. The miner will process the transaction, running the infinite loop, and wait for it to run out of gas. Even though the execution runs out of gas and stops halfway through, the transaction is still valid and the miner still claims the fee from the attacker for each computational step.
  • An attacker creates a very long infinite loop with the intent of forcing the miner to keep computing for such a long time that by the time computation finishes a few more blocks will have come out and it will not be possible for the miner to include the transaction to claim the fee. However, the attacker will be required to submit a value for STARTGASlimiting the number of computational steps that execution can take, so the miner will know ahead of time that the computation will take an excessively large number of steps.
  • An attacker sees a contract with code of some form like send(A,contract.storage[A]); contract.storage[A] = 0, and sends a transaction with just enough gas to run the first step but not the second (ie. making a withdrawal but not letting the balance go down). The contract author does not need to worry about protecting against such attacks, because if execution stops halfway through the changes they get reverted.
  • A financial contract works by taking the median of nine proprietary data feeds in order to minimize risk. An attacker takes over one of the data feeds, which is designed to be modifiable via the variable-address-call mechanism described in the section on DAOs, and converts it to run an infinite loop, thereby attempting to force any attempts to claim funds from the financial contract to run out of gas. However, the financial contract can set a gas limit on the message to prevent this problem.

The alternative to Turing-completeness is Turing-incompleteness, where JUMP and JUMPIdo not exist and only one copy of each contract is allowed to exist in the call stack at any given time. With this system, the fee system described and the uncertainties around the effectiveness of our solution might not be necessary, as the cost of executing a contract would be bounded above by its size. Additionally, Turing-incompleteness is not even that big a limitation; out of all the contract examples we have conceived internally, so far only one required a loop, and even that loop could be removed by making 26 repetitions of a one-line piece of code. Given the serious implications of Turing-completeness, and the limited benefit, why not simply have a Turing-incomplete language? In reality, however, Turing-incompleteness is far from a neat solution to the problem. To see why, consider the following contracts:

C0: call(C1); call(C1);
C1: call(C2); call(C2);
C2: call(C3); call(C3);
...
C49: call(C50); call(C50);
C50: (run one step of a program and record the change in storage)

Now, send a transaction to A. Thus, in 51 transactions, we have a contract that takes up 250computational steps. Miners could try to detect such logic bombs ahead of time by maintaining a value alongside each contract specifying the maximum number of computational steps that it can take, and calculating this for contracts calling other contracts recursively, but that would require miners to forbid contracts that create other contracts (since the creation and execution of all 26 contracts above could easily be rolled into a single contract). Another problematic point is that the address field of a message is a variable, so in general it may not even be possible to tell which other contracts a given contract will call ahead of time. Hence, all in all, we have a surprising conclusion: Turing-completeness is surprisingly easy to manage, and the lack of Turing-completeness is equally surprisingly difficult to manage unless the exact same controls are in place – but in that case why not just let the protocol be Turing-complete?

Currency And Issuance

The Ethereum network includes its own built-in currency, ether, which serves the dual purpose of providing a primary liquidity layer to allow for efficient exchange between various types of digital assets and, more importantly, of providing a mechanism for paying transaction fees. For convenience and to avoid future argument (see the current mBTC/uBTC/satoshi debate in Bitcoin), the denominations will be pre-labelled:

  • 1: wei
  • 1012: szabo
  • 1015: finney
  • 1018: ether

This should be taken as an expanded version of the concept of “dollars” and “cents” or “BTC” and “satoshi”. In the near future, we expect “ether” to be used for ordinary transactions, “finney” for microtransactions and “szabo” and “wei” for technical discussions around fees and protocol implementation; the remaining denominations may become useful later and should not be included in clients at this point.

The issuance model will be as follows:

  • Ether will be released in a currency sale at the price of 1000-2000 ether per BTC, a mechanism intended to fund the Ethereum organization and pay for development that has been used with success by other platforms such as Mastercoin and NXT. Earlier buyers will benefit from larger discounts. The BTC received from the sale will be used entirely to pay salaries and bounties to developers and invested into various for-profit and non-profit projects in the Ethereum and cryptocurrency ecosystem.
  • 0.099x the total amount sold (60102216 ETH) will be allocated to the organization to compensate early contributors and pay ETH-denominated expenses before the genesis block.
  • 0.099x the total amount sold will be maintained as a long-term reserve.
  • 0.26x the total amount sold will be allocated to miners per year forever after that point.
Group At launch After 1 year After 5 years
Currency units 1.198X 1.458X 2.498X
Purchasers 83.5% 68.6% 40.0%
Reserve spent pre-sale 8.26% 6.79% 3.96%
Reserve used post-sale 8.26% 6.79% 3.96%
Miners 0% 17.8% 52.0%

Long-Term Supply Growth Rate (percent)

SPV in bitcoin

Despite the linear currency issuance, just like with Bitcoin over time the supply growth rate nevertheless tends to zero

The two main choices in the above model are (1) the existence and size of an endowment pool, and (2) the existence of a permanently growing linear supply, as opposed to a capped supply as in Bitcoin. The justification of the endowment pool is as follows. If the endowment pool did not exist, and the linear issuance reduced to 0.217x to provide the same inflation rate, then the total quantity of ether would be 16.5% less and so each unit would be 19.8% more valuable. Hence, in the equilibrium 19.8% more ether would be purchased in the sale, so each unit would once again be exactly as valuable as before. The organization would also then have 1.198x as much BTC, which can be considered to be split into two slices: the original BTC, and the additional 0.198x. Hence, this situation is exactly equivalent to the endowment, but with one important difference: the organization holds purely BTC, and so is not incentivized to support the value of the ether unit.

The permanent linear supply growth model reduces the risk of what some see as excessive wealth concentration in Bitcoin, and gives individuals living in present and future eras a fair chance to acquire currency units, while at the same time retaining a strong incentive to obtain and hold ether because the “supply growth rate” as a percentage still tends to zero over time. We also theorize that because coins are always lost over time due to carelessness, death, etc, and coin loss can be modeled as a percentage of the total supply per year, that the total currency supply in circulation will in fact eventually stabilize at a value equal to the annual issuance divided by the loss rate (eg. at a loss rate of 1%, once the supply reaches 26X then 0.26X will be mined and 0.26X lost every year, creating an equilibrium).

Note that in the future, it is likely that Ethereum will switch to a proof-of-stake model for security, reducing the issuance requirement to somewhere between zero and 0.05X per year. In the event that the Ethereum organization loses funding or for any other reason disappears, we leave open a “social contract”: anyone has the right to create a future candidate version of Ethereum, with the only condition being that the quantity of ether must be at most equal to 60102216 * (1.198 + 0.26 * n) where n is the number of years after the genesis block. Creators are free to crowd-sell or otherwise assign some or all of the difference between the PoS-driven supply expansion and the maximum allowable supply expansion to pay for development. Candidate upgrades that do not comply with the social contract may justifiably be forked into compliant versions.

Mining Centralization

The Bitcoin mining algorithm works by having miners compute SHA256 on slightly modified versions of the block header millions of times over and over again, until eventually one node comes up with a version whose hash is less than the target (currently around 2192). However, this mining algorithm is vulnerable to two forms of centralization. First, the mining ecosystem has come to be dominated by ASICs (application-specific integrated circuits), computer chips designed for, and therefore thousands of times more efficient at, the specific task of Bitcoin mining. This means that Bitcoin mining is no longer a highly decentralized and egalitarian pursuit, requiring millions of dollars of capital to effectively participate in. Second, most Bitcoin miners do not actually perform block validation locally; instead, they rely on a centralized mining pool to provide the block headers. This problem is arguably worse: as of the time of this writing, the top three mining pools indirectly control roughly 50% of processing power in the Bitcoin network, although this is mitigated by the fact that miners can switch to other mining pools if a pool or coalition attempts a 51% attack.

The current intent at Ethereum is to use a mining algorithm where miners are required to fetch random data from the state, compute some randomly selected transactions from the last N blocks in the blockchain, and return the hash of the result. This has two important benefits. First, Ethereum contracts can include any kind of computation, so an Ethereum ASIC would essentially be an ASIC for general computation – ie. a better CPU. Second, mining requires access to the entire blockchain, forcing miners to store the entire blockchain and at least be capable of verifying every transaction. This removes the need for centralized mining pools; although mining pools can still serve the legitimate role of evening out the randomness of reward distribution, this function can be served equally well by peer-to-peer pools with no central control.

This model is untested, and there may be difficulties along the way in avoiding certain clever optimizations when using contract execution as a mining algorithm. However, one notably interesting feature of this algorithm is that it allows anyone to “poison the well”, by introducing a large number of contracts into the blockchain specifically designed to stymie certain ASICs. The economic incentives exist for ASIC manufacturers to use such a trick to attack each other. Thus, the solution that we are developing is ultimately an adaptive economic human solution rather than purely a technical one.

Scalability

One common concern about Ethereum is the issue of scalability. Like Bitcoin, Ethereum suffers from the flaw that every transaction needs to be processed by every node in the network. With Bitcoin, the size of the current blockchain rests at about 15 GB, growing by about 1 MB per hour. If the Bitcoin network were to process Visa’s 2000 transactions per second, it would grow by 1 MB per three seconds (1 GB per hour, 8 TB per year). Ethereum is likely to suffer a similar growth pattern, worsened by the fact that there will be many applications on top of the Ethereum blockchain instead of just a currency as is the case with Bitcoin, but ameliorated by the fact that Ethereum full nodes need to store just the state instead of the entire blockchain history.

The problem with such a large blockchain size is centralization risk. If the blockchain size increases to, say, 100 TB, then the likely scenario would be that only a very small number of large businesses would run full nodes, with all regular users using light SPV nodes. In such a situation, there arises the potential concern that the full nodes could band together and all agree to cheat in some profitable fashion (eg. change the block reward, give themselves BTC). Light nodes would have no way of detecting this immediately. Of course, at least one honest full node would likely exist, and after a few hours information about the fraud would trickle out through channels like Reddit, but at that point it would be too late: it would be up to the ordinary users to organize an effort to blacklist the given blocks, a massive and likely infeasible coordination problem on a similar scale as that of pulling off a successful 51% attack. In the case of Bitcoin, this is currently a problem, but there exists a blockchain modification suggested by Peter Todd which will alleviate this issue.

In the near term, Ethereum will use two additional strategies to cope with this problem. First, because of the blockchain-based mining algorithms, at least every miner will be forced to be a full node, creating a lower bound on the number of full nodes. Second and more importantly, however, we will include an intermediate state tree root in the blockchain after processing each transaction. Even if block validation is centralized, as long as one honest verifying node exists, the centralization problem can be circumvented via a verification protocol. If a miner publishes an invalid block, that block must either be badly formatted, or the state S[n] is incorrect. Since S[0] is known to be correct, there must be some first state S[i] that is incorrect where S[i-1] is correct. The verifying node would provide the index i, along with a “proof of invalidity” consisting of the subset of Patricia tree nodes needing to process APPLY(S[i-1],TX[i]) -> S[i]. Nodes would be able to use those nodes to run that part of the computation, and see that the S[i] generated does not match the S[i] provided.

Another, more sophisticated, attack would involve the malicious miners publishing incomplete blocks, so the full information does not even exist to determine whether or not blocks are valid. The solution to this is a challenge-response protocol: verification nodes issue “challenges” in the form of target transaction indices, and upon receiving a node a light node treats the block as untrusted until another node, whether the miner or another verifier, provides a subset of Patricia nodes as a proof of validity.

Conclusion

The Ethereum protocol was originally conceived as an upgraded version of a cryptocurrency, providing advanced features such as on-blockchain escrow, withdrawal limits, financial contracts, gambling markets and the like via a highly generalized programming language. The Ethereum protocol would not “support” any of the applications directly, but the existence of a Turing-complete programming language means that arbitrary contracts can theoretically be created for any transaction type or application. What is more interesting about Ethereum, however, is that the Ethereum protocol moves far beyond just currency. Protocols around decentralized file storage, decentralized computation and decentralized prediction markets, among dozens of other such concepts, have the potential to substantially increase the efficiency of the computational industry, and provide a massive boost to other peer-to-peer protocols by adding for the first time an economic layer. Finally, there is also a substantial array of applications that have nothing to do with money at all.

The concept of an arbitrary state transition function as implemented by the Ethereum protocol provides for a platform with unique potential; rather than being a closed-ended, single-purpose protocol intended for a specific array of applications in data storage, gambling or finance, Ethereum is open-ended by design, and we believe that it is extremely well-suited to serving as a foundational layer for a very large number of both financial and non-financial protocols in the years to come.

Notes and Further Reading

Notes

  1. A sophisticated reader may notice that in fact a Bitcoin address is the hash of the elliptic curve public key, and not the public key itself. However, it is in fact perfectly legitimate cryptographic terminology to refer to the pubkey hash as a public key itself. This is because Bitcoin’s cryptography can be considered to be a custom digital signature algorithm, where the public key consists of the hash of the ECC pubkey, the signature consists of the ECC pubkey concatenated with the ECC signature, and the verification algorithm involves checking the ECC pubkey in the signature against the ECC pubkey hash provided as a public key and then verifying the ECC signature against the ECC pubkey.
  2. Technically, the median of the 11 previous blocks.
  3. The Ethereum protocol should be as simple as practical, but it may be necessary to have quite a high level of complexity, for instance to scale, to internalize costs of storage, bandwidth and I/O, for security, privacy, transparency, etc. Where complexity is necessary, documentation should be as clear, concise and up-to-date as possible, so that someone completely unschooled in Ethereum can learn it and become an expert.
  4. See the Yellow Paper for the Ethereum Virtual Machine, while also there are many topics in the Ethereum wiki, such as implementations and networking protocols. Then for research and possible future implementation there is ethresear.ch. Also there is a phase 1 sharding roadmap which includes intended changes for Ethereum 2.0, to roll out over the next few years, starting from 2018.
  5. Another way of expressing this is abstraction. The latest roadmap is planning to abstract execution, allowing execution engines to not necessarily have to follow one canonical specification, but for instance it could be tailored for a specific application, as well as a shard. (This heterogeneity of execution engines is not explicitly stated in the roadmap. There is also heterogeneous sharding, which Vlad Zamfir conceptualized.)
  6. Internally, 2 and “CHARLIE” are both numbers, with the latter being in big-endian base 256 representation. Numbers can be at least 0 and at most 2256-1.

Further Reading

  1. Intrinsic value: http://bitcoinmagazine.com/8640/an-exploration-of-intrinsic-value-what-it-is-why-bitcoin-doesnt-have-it-and-why-bitcoin-does-have-it/
  2. Smart property: https://en.bitcoin.it/wiki/Smart_Property
  3. Smart contracts: https://en.bitcoin.it/wiki/Contracts
  4. B-money: http://www.weidai.com/bmoney.txt
  5. Reusable proofs of work: http://www.finney.org/~hal/rpow/
  6. Secure property titles with owner authority: http://szabo.best.vwh.net/securetitle.html
  7. Bitcoin whitepaper: http://bitcoin.org/bitcoin.pdf
  8. Namecoin: https://namecoin.org/
  9. Zooko’s triangle: http://en.wikipedia.org/wiki/Zooko’s_triangle
  10. Colored coins whitepaper: https://docs.google.com/a/buterin.com/document/d/1AnkP_cVZTCMLIzw4DvsW6M8Q2JC0lIzrTLuoWu2z1BE/edit
  11. Mastercoin whitepaper: https://github.com/mastercoin-MSC/spec
  12. Decentralized autonomous corporations, Bitcoin Magazine: http://bitcoinmagazine.com/7050/bootstrapping-a-decentralized-autonomous-corporation-part-i/
  13. Simplified payment verification: https://en.bitcoin.it/wiki/Scalability#Simplifiedpaymentverification
  14. Merkle trees: http://en.wikipedia.org/wiki/Merkle_tree
  15. Patricia trees: http://en.wikipedia.org/wiki/Patricia_tree
  16. GHOST: https://eprint.iacr.org/2013/881.pdf
  17. StorJ and Autonomous Agents, Jeff Garzik: http://garzikrants.blogspot.ca/2013/01/storj-and-bitcoin-autonomous-agents.html
  18. Mike Hearn on Smart Property at Turing Festival: http://www.youtube.com/watch?v=Pu4PAMFPo5Y
  19. Ethereum RLP: https://github.com/ethereum/wiki/wiki/%5BEnglish%5D-RLP
  20. Ethereum Merkle Patricia trees: https://github.com/ethereum/wiki/wiki/%5BEnglish%5D-Patricia-Tree
  21. Peter Todd on Merkle sum trees: http://sourceforge.net/p/bitcoin/mailman/message/31709140/