Bitcoin source code analysis - Transaction (2) - Principles

I have been writing this article intermittently... It must have been almost three weeks? So the style before and after can be quite different. I'm really lazy... I really haven't worked hard enough recently. The project of rewriting bitcoin in python is also stuck in the network programming part (this aspect is really my weakness)

This article is full of words -_- and nothing else. This will cause a lot of pressure on the readers...

Although the title talks about principles, some of the principles have actually been explained scatteredly in previous articles, and it feels a bit repetitive to write them out. . So it's best to read the previous ones first, which may look better. . After I finish writing everything, I will probably reorganize it and write it again in a more readable form (this is a flag)

This article will explain in detail the nature of Bitcoin transactions. Similarly, we will not discuss blocks in this article, but only discuss how Tx operates in the entire Bitcoin system. The terminology used in this article is inherited from the previous article, and the details discussed in the previous article are directly used.

In the entire Bitcoin source code, the most important file is only main.cpp/.h (and script.cpp/.h), and the other parts are subsidiary functions. The main.cpp/.h file contains code that will be called "core" in subsequent versions.

We all know that there are two core concepts in the bitcoin system, Tx and Block. However, these two concepts are closely related and even difficult to separate. But when we simply talk about Tx, we will avoid some explanations related to Block below.

In the Bitcoin system, the so-called transaction should include two processes:

Generate transaction -> transaction is verified by (miner)

We can see that in this process, there should be three associated people: the transferor, the payee, and the miner(s)

Next, we will introduce the operating principles respectively, and use these two processes as an entry point to expand my understanding of Bitcoin transactions.

Transaction generation
Now we briefly describe the process of generating a transaction:

A wants to transfer 1 btc to B, then the process is as follows (this part of the code is implemented in the main.cpp CreateTransaction() function):

B first told A his Bitcoin address through some means (the address will be described in detail in other articles)

Then A generates scriptPubkey based on the address given by B and appends some other information (not necessary). This scriptPubkey is the "lock" mentioned in the previous article. Then, according to the value of the amount to be transferred, A starts to generate a transaction:

First, A checks all CWalletTx belonging to itself. In fact, it checks the global variable mapWallet:

WalletTx generally refers to the Tx that Out points to itself, and Out pointing to itself actually means that the address in the scriptPubkey script of Out is generated by the public key held by itself locally. That is to say, the addresses corresponding to all the public keys you hold. As long as the address of Out's scriptPubkey is in your own public key address library, then this Tx belongs to IsMine() (but the one located in WalletTx also includes the Tx you mentioned)

Out is randomly selected in a certain way until the sum of the values ​​contained in the selected Out is greater than or equal to the value to be transferred. In Bitcoin, we call the value that each Out can provide credit.

Then start creating Tx based on the Tx where the selected Out is located.

First, generate a TxOut based on the scriptPubkey generated from the address provided by B and the value of the transfer and fill it into vOut[0] of the newly created transaction (as mentioned above, a transaction is composed of n TxIn and n TxOut, and here Indicates that the Out representing the real transfer is filled into the 0th TxOut)

If the sum of credits composed of the selected Out is greater than the value that needs to be transferred, it means that there must be change after the transaction is completed. Since Bitcoin adopts the "UTXO" mechanism, all the selected Out should be "spent". However, the sum of the selected Out at this time is greater than the value that needs to be transferred, so there must be a difference. Here, bitcoin creates another Out, but the scriptPubkey of this newly created Out uses its own bitcoin address (this address in the 0.1 source code is to select the first Out from the selected Out to extract its own bitcoin address (Because these Outs all point to oneself, since they point to oneself, they must be Bitcoin addresses that one can control). Of course, you can also choose to create a new key generation address), and the value of this newly created Out is the sum of credits. Subtract the value to be transferred, which is the change. Then fill the newly created Out into the next TxOut list. (That is vOut[1])

Then create the TxIn list of this Tx based on all the selected Outs (see above for the reason). Here we simply use the hash of the Tx where the TxOut is located and the index (number) of the Tx where the Out is located as parameters to construct a TxIn, and add the TxIn to the TxIn list.

After constructing the TxIn List, you need to generate the scriptSig of these TxIns (that is, provide the keys to operate these Outs, see above). Here is how to complete the signature. The article about script will be described in detail later. In short, you can understand it here. Because these TxIns are composed of TxOut pointing to themselves, and the scriptPubkey of these TxOuts are composed of addresses generated by the public keys they hold, so they can provide the private keys corresponding to these public keys to generate signatures (Solver function), These signatures are the "keys" mentioned in prose (but please note that whether the signature generated here can be used or not is beyond the control of others when creating this Tx. This is the wonderful thing about Bitcoin. Maintaining these signatures is Either it is legal, or it relies on the majority of miners, which will be described in the next process).

At this time, a transaction is actually created. According to the rules of Bitcoin, the transaction fee of this Tx is actually determined by the size of the Tx (the size is largely composed of the participating TxIn and the size of the script), so Here you need to re-verify whether the sum of credits you can provide after adding transaction fees exceeds the sum of credits you can provide. If it exceeds, then you can only re-use the total cost plus transaction fees as the value of the transfer, and jump to the first step to restart the calculation. .

After a transaction is actually established, there is some finishing work, such as filling in the pre-transactions that should constitute the transaction (AddSupportingTransactions) and modifying some local storage information (CommitTransactionSpent). There is one key point in modifying the local storage information. , which is to mark that the transaction has been spent. Note that the tag here is bound to CWalletTx, and the tag is the transaction associated with the TxIn of the current newly generated transaction. Because we generally believe that a participant should only provide one address in a transaction, so for this trader, the fSpend tag of CWalletTx can represent whether this transaction has been spent on the trader's Out (that is, fSpend is targeted at this trader) and can save a lot on retrieval later.

Transaction verification
Now describe a Tx verification process. Please note that the focus of the verification of Tx is not the verification of Tx by the initiator of the transaction, but the verification of the Tx by the majority of miners, which is what we call the consensus on Tx in a narrow sense. My own verification of the Tx is of no use. Only when most people in the entire system agree with the Tx can the Tx be considered established. (To be precise, when the Tx is packaged into the Block, and after the Block is broadcast, most nodes agree with it and add it to the longest chain , then the Tx is truly established, but this article will not discuss the block first). But please note that starting from this part, during the discussion, please understand that the trader A who initiated the transaction and the miners who authenticated the transaction were verified based on the information obtained from where.

Okay, let's continue the hypothetical scenario in the previous section. A transferred 1 btc to B. After checking his past Tx, A created a new Tx based on the information he saved and believed that the information was legitimate, and passed it through Broadcast in some way (this part will be described in the network-related article). But after the miner receives this Tx, the verification can begin (the subjects we omit next are all miners):

The verification part is in the CTransaction::AcceptTransaction() function. First of all, let’s make it clear that in the Bitcoin system, no matter what is queried, it is based on the hash of this thing. That is to say, we generally think that a hash is the index ID of this object (instance, entity)

First, query the MemoryPool of your own Tx (which is the global variable mapTransaction) and local storage based on the hash of the received information to see if the Tx already exists. If it already exists, skip the authentication process.

Check the Tx pointers (conceptual pointers) held by all TxIns of the transactions to be authenticated (the conceptual pointer here is COutPoint, which is which Tx this TxIn comes from). If this pointer is already in one of its own caches (mapNextTx, use COutPoint As the key, the value is the Tx included in this TxIn (which has appeared before) (representing that there is already a Tx or the same Tx is received)). First check which one is the Tx already in the cache and the Tx currently received." Update", if updated, keep it, otherwise exit the verification

Check whether the information provided by the received TxIn of this transaction matches the existing Tx information stored locally :

If it is coinbase, skip all the following steps

Whether the Tx (TxIndex) corresponding to TxIn exists locally

Simply check whether the basic information of prevout held by TxIn and the out corresponding to Tx are the same.

Check whether the signature script provided by TxIn matches the public script of TxOut corresponding to Tx stored locally.

Check whether the corresponding Out part (vSpent[prevout.n]) of the Tx corresponding to the TxIn stored in the local TxIndex has been marked (marked means that the out has been spent. Pay attention to the spent and CWalletTx here. spend is a little different), if it has not been marked, the TxIn is legal and will be marked below. If it has been marked, it is wrong.

If the above is correct, there will be no storage here, and the storage will be placed in the block packaging section.

So far this transaction has been verified

Put the transaction into the memory pool and wait for packaging

Do some finishing work.

Next, this Tx will exist in the miner's memory pool. This miner may be obligated to help you broadcast this verified and correct transaction to more people. Other miners will also perform the same verification process locally after receiving it. and potentially broadcast to more people . Then wait for the miner to package the transaction into the block. When the block is recognized by most miners, the transaction will take effect.

Simple Analysis
In the two parts above, we introduced the generation and verification of Tx in detail. In fact, the real process is much more complicated than the above, and there are many branches of processing. . . But even so, if the above is not compared with the corresponding code, it is estimated that any beginner will be confused after reading it. I just put in bold the parts that I think are more important. Part of the focus that I have highlighted in bold is to highlight how a Tx works in a distributed environment. Even if each node does not trust each other , it can still accept transactions generated by others. In the two processes above:

The generation of Tx corresponds to the transaction initiator

The verification of Tx corresponds to the process of acknowledging the transaction.

If we follow the traditional process (online banking model), these two processes should be as follows: ps (For the concept of central minting node, please refer to the article in the Bitcoin white paper)

The transaction initiator obtains the balance in his account from the central minting node (such as a bank) (optional) -> the transaction initiator obtains the relevant information of the transaction recipient (such as the other party's bank card number) -> the transaction initiator transfers the transfer amount and The other party’s information is told to the central minting node

The central minting node obtains the initiator's information -> checks the initiator's account balance -> determines whether the transfer amount is below the balance -> destroys the amount that needs to be transferred from the initiator's account (reduces the assets of the initiator's account) -> mints it to the recipient The account accepts the transfer amount (adds the same asset to the recipient's account) -> the destruction and minting constitutes a "transaction"

In the overall process of Bitcoin transfer, these two processes are different. The perspectives of the transaction initiator and the transaction receiver are the same, but the role of verifying the transaction has changed from the original "central minting node" to "All miners and everyone who accepts blocks containing this Tx"

The transaction initiator checks its own locally stored information (Wallet) to obtain the total assets that it can use (not required) -> the transaction initiator obtains the relevant information of the transaction recipient (Bitcoin address) -> after local inspection (not required) Must) generate a transaction Tx and broadcast it

The miners receive the transaction -> use their own local storage information (all TxIndex) to verify whether the historical transaction information corresponding to the TxIn of the received information is consistent, which mainly includes two aspects

Whether the Out of Tx corresponding to TxIn is unspent ( UTXO )

Whether the pubkey_script of Tx Out corresponding to TxIn can be successfully verified by the sig_script provided by TxIn

-> If the verification is successful, it will be put into the memory pool and wait to be packaged into the block.

====

-> When packaged into a block, modify your local storage (TxIndex) -> Broadcast this block to more people

==== (Block-related parts will be explained in detail later)

-> (Other miners) If the block received is the longest chain -> Verify this block - > Save it if the verification is successful ( other people’s history ) -> Eliminate the relevant Tx in the memory pool -> Interrupt Your own packaging block process -> repackage the remaining Tx in the memory pool

The last point to mention: If a miner does not have many Tx in the memory pool, or he selectively accepts some Tx and eliminates some Tx (for example, the transaction fee given is not high enough), if he is very powerful, he can always obtain packaged blocks. If you have accounting rights, it is very likely that some transactions will never be packaged. Even if other miners package your transactions, as long as these other miners cannot grab this powerful miner, your transaction will still not be valid. , because " it is not recorded in the history record ", you cannot generate the remaining transactions based on this transaction.

Summarize
After the above analysis, it can be concluded from the comparison between the traditional method and the Bitcoin method that, at least in my opinion, the biggest difference is:

Where is the process of verifying history moved?

Because in ensuring the correctness of the transaction process, the key point is that the success of a transaction depends on whether the previous transaction is correct, such as:

In traditional transactions, it is necessary to check that the user's account balance meets the transfer amount.

In Bitcoin, it is necessary to check whether the TxIn used by Tx has been spent and whether it is controllable by the initiator of this transaction (pubkey_script, sig_script pairing)

There are quite significant differences between traditional transactions and Bitcoin:

Traditional transactions must rely on central nodes , and it is absolutely impossible to bypass the center to generate transactions, otherwise the transaction will not be recognized.

The verification history of Bitcoin transactions relies entirely on the history stored by individuals , without exception, whether it is an individual or a miner

This goes back to the questions raised in the analysis chapter:

How does a Tx work in a distributed environment? Even if each node does not trust each other , it can still accept transactions generated by others.

The answer to this question is simple:

Because nodes do not trust each other, they trust themselves . Nodes only believe in their own historical records and will only make judgments based on their own historical records. Then the question arises: Is my historical record right or wrong? (Right and wrong here are relative meanings. It should be said whether it is consistent with most people)

This issue involves the highest principle of Bitcoin, which only recognizes the chain with the longest workload as the only recognized Bitcoin chain.

Then the remaining problems will continue to be analyzed in the block part.