Transaction Flow

When a transaction is received (net_processing.cpp#Ln1762), it's checked whether it should be processed.

It will be deserialized into a CTransactionRef representing a CTransaction
mark this message as received on the pfrom (a CNode - a peer node) where the message came from(net_processing.cpp#Ln1779,1786,1787).
check if message already received and processed and valid the transaction (AcceptToMemoryPool)
check state of the mem pool (txmempool.cpp#Ln610)

Block Create Flow

When a "generate" request is received, the Miner will trigger the std::unique_ptr<CBlockTemplate> BlockAssembler::CreateNewBlock(const CScript& scriptPubKeyIn, bool fMineWitnessTx) in miner.cpp to generate a Block template. We break down the this procedure into 10 steps below:

Create a clean and empty block template.
Reserve some space for coinbase Tx.
Lock the Mempool critical section.
Decide whether to include witness Txs.
Choose Txs to pack into the package which will be added into the Block. The higher feerate of Txs get the higher priority to be packed into the block.
Create coinbase Tx.
Assign the hashPrevBlock to previous block hash.
Assign the block nouce to 0.
Run TestBlockValidity() (There are 4 validate func)
Return this block template as a unique_ptr

This block template pointer will be returned to mining.cpp for further processes, such as PoW for finding the nouce value. Step 5 & 10, are the main part of creating the new block, and we'll talk about more detail in the following two paragraph.

Choose the Txs

Before we dive into the function, there are some terms we need to know.

In txmempool.h, there are two major classes, one is CTXMemPool, and another is CTXMemPoolEntry.

CTXMemPool

It stores the valid Txs in the mempool that maybe included into the next block. Major attributes are list below:

mapTx is a multi_index, sort the mempool Txs in 4 criteria
- Tx hash (TxID)
- Fee rate (with ancestor_score / with descendant_score)
- Time in the pool
- Mining score (feerate modified by any fee deltas from PrioritiseTransaction)
mapNextTx = map(COutPoint, CTransaction)
- COutPoint, a class which record the combination with TxID and index #
- （他功能是幹嘛的？？）
mapLinks is a std::map<txiter, TxLinks, CompareIteratorByHash>
- It’s a map, each txiter is map to a TxLinks which has 2 setEntries for tracking txiter’s parents and children Txs
- It is sorted by TxID (Tx hash), according to the CompareIteratorByHash
- Why we need this? Because we need to update fee rate sort every time when a new Tx arrive （目前看到, 是這樣假設)

CTxMemPoolEntry

It stores the data about the corresponding Tx in the mempool, like total number of descendants / ancestors, size of descendants / ancestors, Fees (nModFeesWithDescendants / nModFeesWithAncestors) of Tx itself and its descendants / ancestors, etc.

(目前看到, nModFeesWithDescendants / nModFeesWithAncestors這兩個值, 在初始化時就會把Tx 自身的 fee assign 進去, 接下來, 會根據新進 Mempool 的 Txs, 來更新這個值, 推測每個 Tx 進來都會更新子孫、祖先的 feerate, 不過還沒 trace 到這段 code, Tx 進 Mempool 這段, 沒看完整)

Major attributes are list below:

nCountWithDescendants
nSizeWithDescendants
nModFeesWithDescendants
nCountWithAncestors
nSizeWithAncestors
nModFeesWithAncestors
nSigOpCostWithAncestors (a counter for signature operations, such as OP_CHECKSIG)

In CTxMemPoolEntry constructor, we can see that most of the attributes are initialized by the values of Tx itself.

    CTxMemPoolEntry::CTxMemPoolEntry(const CTransactionRef& _tx, const CAmount& _nFee,
                                 int64_t _nTime, unsigned int _entryHeight,
                                 bool _spendsCoinbase, int64_t _sigOpsCost, LockPoints lp):
    tx(_tx), nFee(_nFee), nTime(_nTime), entryHeight(_entryHeight),
    spendsCoinbase(_spendsCoinbase), sigOpCost(_sigOpsCost), lockPoints(lp){

        nTxWeight = GetTransactionWeight(*tx);
        nUsageSize = RecursiveDynamicUsage(tx);

        nCountWithDescendants = 1;
        nSizeWithDescendants = GetTxSize();
        nModFeesWithDescendants = nFee;

        feeDelta = 0;

        nCountWithAncestors = 1;
        nSizeWithAncestors = GetTxSize();
        nModFeesWithAncestors = nFee;
        nSigOpCostWithAncestors = sigOpCost;
    }

In default, Miners use nModFeesWithAncestors value as the priority for ordering the Txs in the MemPool and choose the current best one with its ancestors Txs to pack them into a package, which will be included in the next mining block.

That's it! Now we are ready to step into the main packing function - addPackageTxs () .

void BlockAssembler::addPackageTxs(int &nPackagesSelected, int &nDescendantsUpdated)
{
    // mapModifiedTx will store sorted packages after they are modified
    // because some of their txs are already in the block
    indexed_modified_transaction_set mapModifiedTx;
    // Keep track of entries that failed inclusion, to avoid duplicate work
    CTxMemPool::setEntries failedTx;

    // Start by adding all descendants of previously added txs to mapModifiedTx
    // and modifying them for their already included ancestors
    UpdatePackagesForAdded(inBlock, mapModifiedTx);

    CTxMemPool::indexed_transaction_set::index<ancestor_score>::type::iterator mi = mempool.mapTx.get<ancestor_score>().begin();
    CTxMemPool::txiter iter;

    // Limit the number of attempts to add transactions to the block when it is
    // close to full; this is just a simple heuristic to finish quickly if the
    // mempool has a lot of entries.
    const int64_t MAX_CONSECUTIVE_FAILURES = 1000;
    int64_t nConsecutiveFailed = 0;

    ......
}

First thing first, addPackageTxs() has some variables initialization:

mapModifiedTx
- A multi-index container, it can store the descendant Txs which ancestor Tx has already been included in the template block
failedTx
- Store the Txs that failed to be included in the template block
mi
- An iterator for mapTx which is sorted by ancestor_score
MAX_CONSECUTIVE_FAILURES
- Define the maximum consecutive failure times for Tx block inclusion, default - 1000 times
nConsecutiveFailed
- A counter counts the consecutive failure times of Tx block inclusion

while (mi != mempool.mapTx.get<ancestor_score>().end() || !mapModifiedTx.empty())
{
    // First try to find a new transaction in mapTx to evaluate.
    if (mi != mempool.mapTx.get<ancestor_score>().end() &&
            SkipMapTxEntry(mempool.mapTx.project<0>(mi), mapModifiedTx, failedTx)) {
        ++mi;
        continue;
    }

    // Now that mi is not stale, determine which transaction to evaluate:
    // the next entry from mapTx, or the best from mapModifiedTx?
    bool fUsingModified = false;

    modtxscoreiter modit = mapModifiedTx.get<ancestor_score>().begin();
    if (mi == mempool.mapTx.get<ancestor_score>().end()) {
        // We're out of entries in mapTx; use the entry from mapModifiedTx
        iter = modit->iter;
        fUsingModified = true;
    } else {
        // Try to compare the mapTx entry to the mapModifiedTx entry
        iter = mempool.mapTx.project<0>(mi);
        if (modit != mapModifiedTx.get<ancestor_score>().end() &&
                CompareModifiedEntry()(*modit, CTxMemPoolModifiedEntry(iter))) {
            // The best entry in mapModifiedTx has higher score
            // than the one from mapTx.
            // Switch which transaction (package) to consider
            iter = modit->iter;
            fUsingModified = true;
        } else {
            // Either no entry in mapModifiedTx, or it's worse than mapTx.
            // Increment mi for the next loop iteration.
            ++mi;
        }
    }

(從這裡繼續！！！！！！！！！！！！！！！！！！！！！！！！！)

Check block validation

bool TestBlockValidity(CValidationState& state, const CChainParams& chainparams, const CBlock& block, CBlockIndex* pindexPrev, bool fCheckPOW = true, bool fCheckMerkleRoot = true);

AcceptToMemoryPoolWorker

Before any transaction is place into memory pool, it must go through a series of validations. All the validations are done in here.

CheckTransaction: performs basic validation on transaction
1. contains at least one input, CTxIn
2. contains at least one output, CTxOut
3. check the size of the transaction isn't over the limit, MAX_BLOCK_WEIGHT
4. check if any output contains negative value or value too big
5. (iffCheckDuplicateInputs, for performance reason) check for inputs duplicate
6. (if isCoinBase) check if the input scriptSig size is bigger than 1 and less than 101
7. (if not CoinBase) make sure all inputs have a prevout
Check if the transaction is a CoinBase, a CoinBase can only exists in a Block and not as a loose transaction
check witness status on the transaction and wether segregated witness is active
check if transaction in standard format, IsStandardTx
1. nVersion of transaction cannot be greater than CTransaction::MAX_STANDARD_VERSION and less than 1
2. GetTransactionWeight on transaction cannot be greater than or equal to MAX_STANDARD_TX_WEIGHT
3. for each input, scriptSig size cannot be greater than 1650 and needs to be IsPushOnly()
4. for each ouput
  1. check if scriptSig is standard, IsStandard
  2. if output type is TX_MULTISIG and -permitbaremultisig is not set to true then its not valid
  3. if IsDust it's not valid
    1. dust is calculated base on the size of the out, which means for a certain size of output there is a specified minimum amount of fees required. policy.cpp
check if the transaction can be mined in the next block, CheckFinalTx, if not reject it because dont want to keep too many transaction CheckFinalTx --> IsFinalTx
- mainly check if any input has nSequence not equal to CTxIn::SEQUENCE_FINAL. (Setting nSequence to this value for every input in a transaction disables nLockTime)
check if the has already exists in pool
collect inputs with conflict,setConflicts. If there is conflict then that means this transaction is trying to replace a previous one
1. any input with prevout already in mempool's mapNextTx
check if all the inputs' prevout is already in the mempool
1. if input's prevout is not in CoinViewCache 's HaveCoinInCache add it to coins_to_uncache
2. if prevout is not in CoinView return false
  - but if the corresponding output is in CoinViewCache , then we probably already know about the transaction. return duplicated error
calculate nValueIn the total value of this transaction
check if the Transaction can be mined in the next block, BIP68. CheckSequenceLocks
check if Transaction contains non-standard pay-to-script-hash. Main reason is to avoid denial-of-service attack of using a very expensive-to-check-upon-redemption script like: DUP CHECKSIG DROP ...repeated 100 times... OP_1 AreInputsStandard
check if Transaction contains non-standard witness in P2WSH. IsWitnessStandard
check if there is any spending from CoinBase to be checked for COINBASE_MATURITY later
use the Transaction to create a mempool entry, CtxMemPoolEntry Ln610
check if the number of sigops exceeds the limit, MAX_STANDARD_TX_SIGOPS_COST which is 80000/5 = 16,000. Eventhough, MAX_BLOCK_SIGOPS is set at 20,000
check if the total fee is less than the fee required for the transaction to enter the mempool Ln624
check if the total fee is less than the minRelayTxFee Ln629
check if fee is too high according, currently checking is turned off, by passing limit to be 0. Ln633
check if ancestors in mempool are too long with configured limits, and collect all ancestors, in CalculateMemPoolAncestors. Ln645
- it retrieves all parents of the transaction using each inputs
- check all parents if combined descendants tx size or count is over the limit and check if combined ancestors tx size or count is over the limit
check if any ancestor contains in the setConflicts, this means spending conflicting transactions
- if any ancestor is in the setConflicts (the transaction is about to replace) that means the current transaction is trying to spend outputs in the transaction that is going to be voided which is invalid
if this transaction is a replacement transaction
1. calculate whether it is economically worth the replace
  - the calculation is based on the fee of the replacement transaction and the transaction it is about to replace
2. if the number of transaction that is affected (have conflict) is greater than the limit maxDescendantsToVisit then it is rejected validation.cpp#L730
3. check if any of the input for the replacement transaction comes from unconfirmed transactions (mempool), if so reject the transaction (replacement transaction needs to have the parents of the transactions that it is replacing to be confirmed) validation.cpp#L749
4. the replacement fee must be greater than the fee it has conflict with (the transaction fee it replaces)validation.cpp#L771
5. in addition to paying more fee, the conflicting transaction must pay more relay fee as well validation.cpp#L782
check all the inputs against previous transactions (this is done last because the check is CPU intensive)
check transaction against the current mempool for invalid script validation.cpp
clear memory status validation.cpp#L848-877
add transaction to mempool

Note: Modified Fee

There is something called "modified fee". It is used to modify fee when calculating the priorities of transactions to go into the current block to be mined. You can use this prioritisetransction command to make a "delta" to the transaction fee in satoshi to affect the priority of the transaction in the local node. The modified fee is used only to calculate the priority of the transaction and does not affect the actual transaction fee paid.

Mining and Consensus(Jack&Steven)