- 1.
It validates transactions occurring on the cryptocurrency network
- 2.
It adds blocks to the blockchain
- 3.
It creates new currency units
All of these activities occur in a distributed manner outside the control of any particular node on the cryptocurrency network. Blocks are added to the blockchain through distributed consensus.
In this chapter, we will examine how mining is carried out in a Bitcoin-like currency and how the distributed consensus algorithm adds blocks to the blockchain.
A caveat is in order. In this chapter, you will sometimes see the phrase “the blockchain” used. There is in actuality no single authoritative blockchain held by some supernode. There are blockchains held by various nodes, and some of these nodes may hold blockchains that differ from those held by other nodes. The blockchain is a distributed structure. We refer to the blockchains held by various network nodes as local blockchains. The blockchain is an aggregated abstraction of all local blockchains. The distributed consensus algorithm ensures that all divergent blockchains will converge to a single authoritative blockchain.
The Mining Process
Why would an entity engage in mining? After all, mining is an expensive process that consumes a lot of CPU cycles and hence electricity. This incentive comes from the transaction fees and the mining reward that a miner receives for successfully mining a block. This remuneration is substantial. Any entity that has a local blockchain can engage in mining a cryptocurrency. Let us see how mining works.
Consider an individual who initiates a transaction that will transfer some units of a cryptocurrency. After the transaction has been consummated, the transferor of the cryptocurrency broadcasts the transaction on the cryptocurrency peer-to-peer network.1 Miners (mining nodes), who have a local blockchain, are listening for transactions that are propagating across this network.
The task before a miner is to construct a prospective new block that can be added to the blockchain. The miner adds transactions to a candidate block and then mines this block.
Consider a miner who receives a transaction. The miner verifies that the transaction is valid. If it is invalid, the miner discards the transaction. Otherwise, the miner broadcasts this incoming transaction to other mining nodes that this miner has discovered.
The miner then examines the locktime of the received transaction. If the transaction is to be processed at some future time, the miner places it into a local cache of transactions that are to be processed in the future. This cache is called a memcache. If the miner decides to include this particular transaction in the candidate block that he intends to mine, he adds it to the block. Otherwise, the miner adds the transaction to the memcache.
The miner accesses the memcache as well as incoming transactions to add transactions to his candidate block. Subject to the constraint on the maximum size of a block, it is completely within the discretion of a miner as to which transactions and how many transactions it is going to include in a candidate block. Each transaction may have a transaction fee attached to it. It is within the sole discretion of the transferor of value to specify the transaction fee, if any. The miner obviously wants to maximize the transaction fees that he will receive if he successfully mines the candidate block.
Once the miner has decided that there are enough transactions in the candidate block, it adds a special transaction to the block. This transaction is called a coinbase transaction. It is the first transaction in the candidate block’s list of transactions. The coinbase transaction contains the quantum of new cryptocurrency that will be created if the miner succeeds in mining the block. This value is called the mining reward. The miner is entitled to the mining reward. Thus, the mining reward and the total transaction fees for the transactions included in the block are what the miner will receive if it successfully mines the block.
In Bitcoin and Helium, the amount of the mining reward is determined by an algorithm that depends on the number of blocks that have already been mined. Periodically the mining reward is halved as the size of the blockchain increases. The mining reward approaches zero asymptotically. Note that since a mining reward is new currency, it has no corresponding vin element.
The Bitcoin mining algorithm implies that a maximum of 21 million bitcoins can be produced. This suggests that Bitcoin is asymptotically deflationary; its value (purchasing power) must rise over time. Other cryptocurrencies can implement different currency creation algorithms. For example, the mining reward can be constant or vary in accordance with the volume of transactions per fixed unit of time. The TRON cryptocurrency has the dubious distinction of permitting the creation of an unlimited number of currency units by ostensibly distributed committee consensus. Helium follows the Bitcoin algorithm.
The miner calculates the SHA-256 hash of the header of the most recent block in its blockchain and pairs this value with the prevblockhash key of the candidate block. It next computes the merkle root of the transactions in the block and also inserts this value into the block header. Similarly, the miner puts the current Unix time, difficulty bits, initial nonce value, and version number into the block header. Recall that the genesis block has height 0, the succeeding block has height 1, and so forth.
After the miner has specified the block header, it is ready to mine the candidate block.
Mining a Block
Now that the miner has a candidate block ready for mining, it can start the actual process. The header block of a candidate block has a number called a nonce; this is a non-negative integer initialized to zero. The nonce starts the mining process and is incremented as the process proceeds.
The miner computes the SHA-256 hexadecimal digest of the block header, which includes the nonce. This hexadecimal string is converted into a (very large) number and compared with the current difficulty number specified for the network. If the computed value is less than the difficulty number, the block is said to be mined.2 If the SHA-256 hash is greater than or equal to the difficulty number, then the nonce is incremented and the SHA-256 hash is recomputed and again compared to the difficulty number. This process continues until a hashed value less than the difficulty number is obtained.
In Bitcoin, the nonce is a 32-bit integer, and it frequently happens that the computed value will fail to fall below the difficulty number as the computation runs through the entire range of 2**32 – 1 nonces. If this occurs, the nonce is again initialized at 0 and is placed in the transaction id field of the first vin list element of the coinbase transaction. This is not a concern for our Python implementation since Python can perform arithmetic on arbitrarily large numbers.
While the miner is mining its candidate block, other miners are also mining blocks. The blocks that the other miners are mining need not contain the same transactions as the candidate block of our miner. Indeed, some miners may have local blockchains that differ from the blockchains of other miners.
If a miner succeeds in mining a candidate block, it will add the new block to its local blockchain and then broadcast the mined block on the cryptocurrency network. This is an invitation for other nodes to include this block in their local blockchains.
- 1.
The correct difficulty number has been used.
- 2.
The block has a computed hash that is less than the difficulty number for the nonce in the block header.
- 3.
The block height of the new block is valid.
- 4.
The merkle root of the block transactions matches the merkle root in the block header.
- 5.
No transaction in the block is locked.
- 6.
The transactions in the block are valid.
- 7.
If a transaction transfers a coinbase value, then a required minimum number of blocks have been added to the blockchain since the block which contained this coinbase transaction.
- 8.
There is a prevblockhash match as discussed in the following.
If the mined node passes this suite of validation tests, the node can add this block to its blockchain.
Block Height Validation
- 1.
If the height is negative or zero, the block is rejected.
- 2.
If the height value is less than the height of the node’s blockchain, the block is rejected.
- 3.
If the block height is equal to the node’s blockchain height plus 2, the block is placed in an orphan list.
- 4.
If the block height is greater than the node’s blockchain height plus 2, the block is placed in an orphan list. The node’s blockchain may be outdated. The node requisitions other nodes for the latest block(s), so that it can bring its blockchain up to date, if so required.
Previous Block Validation
A node that has received and validated a new block can now add the new block. The following algorithm enumerates the steps in adding the block to a blockchain.
Definitions
Let previousblockhash be the value of the prevblockhash key in the mined block.
Let block_name[latest] be the latest block on a blockchain that is named “block_name”.
Let block_name[latest-1] be the second latest block on a blockchain that is named “block_name”.
Let hash(block_name[x]) be the SHA-256 hash of the block header of the block named block_name, where x is latest or latest-1.
If the node has one blockchain, then name it primary_blockchain.
If the node has two blockchains, name one the primary blockchain and the other one the secondary_blockchain.
Algorithm
- 1.
If the node has one blockchain and
- 2.
Else if the node has one blockchain and
previousblockhash == hash(primary_blockchain[latest -1]
then- (a)
Make a new blockchain called secondary_blockchain by removing the latest element of primary_blockchain
- (b)
Append the mined block to secondary_blockchain
- (a)
- 3.
If the node has two blockchains, then
- 4.
If none of the aforementioned is possible, then the node places the block and is added to an orphan blocks list.
Note that in the second case the node has two blockchains which are competing to become the dominant blockchain.
If a node that receives a newly mined block is a miner, it will check to see if any transactions in the mined block are also in the block that it has been mining. If this is the case, the miner will discard the block that it has been mining and return the transactions in this block to the mempool. If this is not the case, it can keep on mining its candidate block. The miner next updates its memcache by removing any transactions in the mempool that are also in the mined block that has been received.
Finally, the node broadcasts the new block that it has received on the cryptocurrency network so that other nodes can verify the block and add it to their local blockchains.
Proof of Work
Mining a block is a computationally intensive task. When a block is mined, the inclusion of the nonce and the difficulty number in the block enables other miners to verify that the miner has in fact mined the block. This verification is called proof of work. A block which fails this proof of work will not be used by other miners to extend their local blockchains.
The Difficulty Number
The difficulty number is not some fixed number. It dynamically changes over time. For example, in Bitcoin the constraint is that a block should be mined approximately every ten minutes. The purpose of the difficulty number is to ensure that on average a block is mined in this time period. In the Bitcoin ecosystem, the difficulty number is adjusted every two weeks to ensure that blocks are mined approximately every ten minutes. Helium follows this model.
In Helium, after a fixed number of blocks have been mined, the mining nodes check to determine the frequency with which blocks are being mined. If the rate is too high, the difficulty number is adjusted downward, and if the blocks are being produced too slowly, the difficulty number is increased. Each node adjusts its difficulty number. When a mining node receives a mined block, it checks to determine that the correct difficulty number has been used.
The fraction 20/1000 constrains the new difficulty number to vary by no more than 20% from its previous value.
Hash Power Computations
The Distributed Consensus Algorithm
We are now ready to discuss how nodes on a cryptocurrency network arrive at a consensus as to the dominant blockchain on the network.
Each full node in a cryptocurrency network maintains its own blockchain.4 Furthermore, such a node does not know what the blockchains of other full nodes look like. Conceivably, these local blockchains can differ from each other. The distributed consensus algorithm decides how blocks will be added to these local blockchains when different nodes in the cryptocurrency network have blockchains which differ from each other. This algorithm implements a distributed consensus that converges the nodes to the same blockchain without requiring any node to trust any other node. This distributed consensus algorithm constitutes the single most important innovation in the implementation of Bitcoin by Satoshi Nakamoto.
To be precise, the distributed consensus algorithm that we are discussing is called the proof-of-work distributed consensus algorithm to distinguish it from the proof-of-stake distributed consensus algorithm.
Before we begin our examination of this distributed consensus algorithm, note that there is no clock or timekeeper in the system and there is also no concept of the ordering of transactions on the basis of time or otherwise. A node processes blocks as they are received, and these blocks need not arrive in the order that they were produced. A node may not receive a block for any number of reasons including a temporary partition in the network.
Mining is a costly endeavor and requires the expenditure of CPU cycles (electricity) to solve a difficult mathematical problem. A miner is only willing to undertake the cost of mining if it can be compensated by an adequate mining reward and transaction fees. It is this incentive for profit which impels miners to converge to the same view of the blockchain. As we shall see, this incentive compels nodes to be honest and follow the blockchain consensus algorithm.
Consider a cryptocurrency network where all of the nodes initially have the same view of the blockchain.
Nodes in a Cryptocurrency Network
Node X Mines a Block
At this point in time, there are two views of the blockchain since some nodes have not yet appended the new block to their blockchains. These nodes are effectively one block behind.
Node Y Mines a Block
Now the block mined by Y reaches node C. Node C notices that it cannot add this block to the head of its version of the blockchain since the previous block hash value does not match. Node C also observes that this block could be added if the block before the block at the head of the blockchain was in fact at the head of the blockchain. So node C creates a secondary blockchain by removing the node at the head of the blockchain and adding the new block to this secondary blockchain. At this stage, node C has a primary blockchain and a secondary blockchain, and the heads of these blockchains are different.
All of the nodes with a star follow the same logic and create primary and secondary blockchains as node Y’s block reaches them.
Similarly , as the block mined by X reaches the nodes with the angled rectangles, these nodes accept the incoming block but have to create primary and secondary blockchains.
Now both of the blocks mined by X and Y reach the node with a rectangle, and it too creates a primary and secondary blockchain.
Node d Mines a block
Similarly, the nodes that received the block from node X first can add the new block to their secondary blockchains. These nodes add the block to their secondary blockchains and delete their primary blockchains and designate the secondary blockchain as the primary blockchain. Notice once again that the shorter blockchain is deleted.
At this stage, all of the nodes on the network have converged to the same view of the blockchain. The preceding process describes the distributed consensus algorithm which was implemented first by Satoshi Nakamoto for Bitcoin. The reason why all miners will follow the longest blockchain is because if they do not, the majority of miners will add blocks to the longest blockchain and the dissenting miners will discover that though they can mine blocks, they will not be able to persuade other miners to add these blocks to the shorter blockchain and thus they will be unable to collect mining rewards and transaction fees.
Because the longest blockchain has the most hashing power expended on it, nodes consider this blockchain to be the correct one and will work on extending it.
As we have just seen, when the two forked blockchains, the primary blockchain and the secondary blockchain, re-converge, one of these blockchains is deleted. So what happens to the transactions in the deleted blockchain? Let’s assume that the primary blockchain is deleted because the secondary blockchain is one block ahead of it. Since the secondary blockchain was created from the primary blockchain with the exclusion of the latest block on the primary blockchain, we only have to consider the transactions in this particular block. We simply put those transactions in this block which are also not in the the latest block of the secondary blockchain, back into the mempool.
Malicious Nodes
We can note three aspects of malicious nodes. A malicious node cannot create a transaction that steals someone’s cryptocurrency because it needs the private key of this individual in order to digitally sign the malicious transaction.
Secondly, a malicious mining node may decide not to process a transaction. But this is a useless action since other mining nodes that receive the transaction will include it in their candidate blocks.
Lastly, a number of malicious nodes acting in concert cannot impose their common blockchain as the dominant blockchain on the network unless they control at least 51% of the total hashing power. The reason for this is that if these nodes have at least 51% of the total hashing power, then the probability that the next block will be produced by them and added to their version of the blockchain is at least 51%. Thus, since they control the majority of the hashing power, they will be able to extend their version of the blockchain and impose it as the primary blockchain on the network.
Transaction Confirmations
When a block is mined and added to the blockchain, we say that the transactions in the block have one confirmation. When another block is added, the transactions in the previous block are said to have two confirmations. As the number of blocks added to the blockchain keeps on increasing, so do the confirmations. In Bitcoin, transactions that receive six confirmations are considered to be irreversible because of the prohibitively expensive hashing power that would be required for the network to undo a transaction in a block that is six blocks deep. A malicious node would have to control the majority of the hashing power of the network to undo these blocks and build an alternate longest blockchain.
The Double-Spend Problem
Consider the following scenario. Alice has one helium coin and she transmits it to Smith. Alice propagates the transaction on the network and a miner starts mining a block that contains this transaction. Now Alice goes somewhere else and spends the same helium again. Since no miner has yet mined a block containing Alice’s first transaction, the second transaction also propagates on the network, and some miners add it to their candidate blocks and start mining these blocks. This is the double-spend problem. Alice has apparently spent her helium twice.
The first miner successfully mines its block and broadcasts the block on the network. This block contains Alice’s first transaction. A very short time after, a second miner also successfully mines a block and propagates it on the network. This block contains Alice’s second transaction.
Consider a node that receives one of these blocks. This node will extend its blockchain with the received block and update its Chainstate database. If a short time later it receives the second block, it will reject this block because a query on the Chainstate database will show that Alice does not have the required value for the transaction. Due to the examination of the Chainstate, a node will only have one of the two blocks containing a transaction by Alice. This is true even if a node has bifurcated to a primary and secondary blockchain.
This example shows why transaction confirmations are important. For small value transactions, two confirmations are sufficient (20 minutes on the bitcoin network on average). For large value transactions, at least six confirmations should be obtained (about one hour on the bitcoin network).
Conclusion
In this chapter, we have examined how the cryptocurrency mining process works. Mining performs a very important function in distributed cryptocurrencies that are modeled similarly to Bitcoin. Mining nodes verify transactions and add blocks to local blockchains. These nodes also reach a distributed consensus on the primary or dominant blockchain without any reliance on trust, a centralized clock, or a supernode coordinator. We have studied the manner in which blockchains in the network converge to one view of the blockchain through the application of the distributed consensus algorithm. Lastly, we have examined how distributed cryptocurrencies solve the “double-spend” problem.
In the next chapter, we will write program code for a Helium mining node.
References
https://github.com/bitcoin/bitcoin
Antonopoulos, Andreas. Mastering Bitcoin, 1st ed., 2015, O’Reilly Media, Inc.