Bitcoin Transaction Pool
In this article, we learn about transaction pools in Bitcoin, this is where validated transactions awaiting mining are stored.
Table of contents.
- Introduction.
- The Mempool.
- Non-final Pool.
- Summary.
- References.
Introduction
Most nodes on the Bitcoin blockchain store a temporary list of unconfirmed transactions referred to as the memory pool / mempool / transaction pool. Bitcoin nodes use this pool to track transactions known to a network but have not been included in the blockchain. For example, a wallet uses a transaction pool to track incoming payments to the owner that have been received but not yet confirmed.
The following image serves as a reminder of the life cycle of a Bitcoin transaction from when it is initiated to completion;
We also have node implementations that maintain separate pools of orphaned transactions - transactions whose sources are missing during processing. These are stored in the orphan pool until the parent transaction arrives.
After a transaction is added to the transaction pool, the orphan pool is checked for orphans referencing the transaction's outputs(children). Matching orphans are validated and if valid, are removed from the orphan pool and included in the transaction pool. This completes the parent chain. This process is repeated recursively looking for descendants until no more can be found. This means that the arrival of a parent initiates the reconstruction of an entire chain of interdependent transactions by reuniting orphans with parents down the chain.
Transaction and orphan pools are stored in memory rather than persistent storage. In the beginning, a new node is empty, later on, it is gradually filled with new transactions received in the network.
Other implementations maintain a UTXO pool that stores all unspent outputs. Note that a UTXO pool is not similar to a UTXO set. A UTXO pool does not start out empty like a transaction or orphan pool instead has millions of entries of unspent outputs since the genesis block. This pool can be stored in local memory or an indexed database table on persistent storage such as a hard disk.
In general, a node can house different kinds of transaction pools depending on how a transaction is classified.
Transaction pools can serve two different purposes leading to two different types of transaction pools, the mempool or the non-final pool.
The Mempool.
This is a set of validated transactions known to the miner. It consists of transactions in the mempool that meet the criteria such as enough transaction fees, to be included in new blocks that they may create and transactions in the secondary mempool that don't meet the criteria.
Addition
A transaction that is accepted into this pool should not cause conflicts with existing transactions. In addition to this, the miner also needs to impose a limit on how long they can spend validating complex SV scripts. For example, if a script takes more than a second to validate, it should be dropped. The mempool is limited in terms of the number of transactions it can hold, meaning that once it is full, restrictions are imposed until it becomes empty again.
A fee is imposed and all transactions have to pay it in order to be included in the pool.
We also have a secondary mempool where transactions that don't meet a miner's criteria but may meet it are stored, The criteria is usually a set transaction fee. The secondary mempool stores transactions that could be eventually included in another miner's block, retaining them ensures fast block validation and optimal network performance.
Removal
During block creation, miners gather transactions from the mempool to construct a candidate block. Similarly, when receiving a block a validator can speed up the process of validation if the transaction IDs match the transaction IDs in the validators pool.
The expiry of transactions in this pool is also checked. Transactions older than the expiry age are removed to bring the size below the maximum allowed size of a pool. This process starts from transactions with the lowest fee going to the highest fee. Remember we said that in Bitcoin transactions are given priority depending on the transaction fee meaning that the higher the fees the higher the chances the transaction will be processed first. In this case, transactions with lower fees that are older than the expiry age are removed from the pool first. This results in a bump in the minimum transaction fee for a transaction to be accepted in a pool.
Currently, the maximum pool size is 1 GB and the expiry age is 14days/226 hours.
There are cases whereby a transaction manages to make the cut into a pool by paying high fees but the fees are not high enough for the transaction to be processed immediately. In such cases, changing the output from the lower fee transaction is used to make a child transaction with high enough fees to pay for both parent and child. This means that even transactions a miner would not consider to mine are promoted into the mempool properly.
Non-final Pool.
This pool tracks transactions that may need further revisions from the involved parties. Once finalized, they are moved into the primary pool. Note that submitting non-final transactions means that the users are committing to a payment, and if the transaction finalizes before it expires out of the pool, the payment will be executed.
The defaukt maximium size is 50MBs.
Addition
The criteria for entering beyond a transaction qualifying as non-final are similar to the one used for entering the main mempool.
Additional limitations are added for transactions that are able to enter this pool, for example, if a non-final transaction is considered for addition to this pool and it goes ahead to spend the outputs of another non-final transaction, it is rejected.
Removal
This pool performs periodic checks on all transactions, this is to check if a transaction in the pool is finalized or inactive.
A transaction is finalized if the lock time has passed. This in itself overrides all other pre-existing conditions that make a transaction non-final. It now becomes a candidate to move to the primary pool.
In this pool, the maximum expiry age is 4 weeks/28days, after which the transaction is dropped.
Summary
A transaction pool is a data structure that stores a set of transactions that have been validated and not yet mined.
In Bitcoin, transactions are received, verified, and added to the transaction pool awaiting confirmation the relayed to neighboring nodes for propagation across the entire blockchain.
While the transaction and orphan pools house unconfirmed transactions, UTXO pools contain confirmed outputs.
References
Current Mempool Size
Mastering Bitcoin, Chapter 8 Mining and Consensus.