An Exploration of Bitcoin Transactions, the Blockchain, and Miners

What will follow is a walk-through of bitcoin’s circular lifecycle as it gets transacted from person to person. Because of the circular nature, there’s not an ideal starting point, because you must always know what came before to truly make sense of where you currently are. So we will start with some unexplained givens, and work our way around the circle from there until we finally understand what we at first had to take for granted.

We will be working our way around this circle, starting at Build, and taking some detours along the way to explain certain concepts in more detail.

What’s in a Transaction? (Build) #

Wallet vs Address #

Addresses are a surprisingly complex topic, and I won’t be covering exactly how they work here. But the distinction between a wallet and an address is important. An address is where bitcoin is sent, and a wallet has many addresses. When you view your wallet, you’re able to see all the funds sent to all the addresses available under the wallet.

I’ll interject with these asides to define terms from time to time.

Transactions are how bitcoin moves. You can move it from your wallet to a friend’s wallet, or from an exchange’s wallet to your wallet, or from one of your wallets to another one of your wallets, maybe even the same wallet. At its core, a transaction is just a list of inputs and a list of outputs. The inputs are the bitcoin that is going to be spent, and the outputs are where the bitcoin is going. When an input is used in a transaction, all of it is used. So if you have an input of 1 BTC, your transaction is going to have to do something with that entire amount. “But what if I only want to spend 0.1 BTC?” No problem, just add an output to pay yourself back the remainder.

This is the basics of what a transaction looks like. A transaction can have one or more inputs and one or more outputs. We’ll get to know why you might have more than one input when we talk about where inputs come from.

Prove Yourself (Sign) #

Now that you’ve built your transaction, it must be signed. Specifically, the inputs must be signed to be considered valid to spend. This is what keeps other people from spending your bitcoin. Only you have your private key, and therefore only you can produce a valid signature for the inputs. If you are using a software wallet, this step (and possibly even the build step) is hidden away from you. You would just have specified an amount and a destination address, and it would have built and signed the transaction behind the scenes. If you are using a hardware wallet, these steps are more manual because the computer or phone you use to build the transaction does not possess the private key to create the signature. This inconvenience is actually a security feature, and why you would use a hardware wallet. You cannot spend your bitcoin without using the hardware wallet to sign the transaction. This added requirement makes it impossible for a remote hacker to access your private keys. Someone would need physical access to your hardware wallet to access your bitcoin.

In the case of someone using a Coldcard, you would build your transaction, export it, and copy it to a Micro SD card. Then you would power up and log in to the Coldcard, insert the Micro SD, and select “sign a transaction”. It would find the unsigned transaction on the Micro SD and display the primary information about the transaction: amount being transacted, addresses it’s being paid to, and fee (we haven’t talked about this, but we will come back to it). Once you confirm that everything looks correct, you tell it to sign the transaction. It will sign it using the private key stored in the device’s secure element and write the signed transaction back to the Micro SD.

Then finally, you would take that signed transaction back to your computer and…

Telling the World (Broadcast) #

Now that you have a signed transaction, you can broadcast it on the bitcoin network. Broadcasting can happen through your own node if you’re running one and are connected to it, or through other nodes. Most wallet software (be it for computer or phone) is designed to work without the user running their own node. That means that the app developers have a node that they are operating that you can use to broadcast transactions through.

When you pass it to the first node, that node (whether it’s yours or some 3rd party’s) will validate the transaction. Once it has validated it, it will broadcast the transaction to all the nodes that it is connected to. That cycle will continue until all nodes on the network have received and independently verified the transaction. This process takes about 12 seconds for a majority of the nodes to see the transaction, but it can take upwards of a minute for every node to receive the transaction¹.

Don’t Trust, Verify (Validation) #

Each node will look at the transaction and will validate it against the consensus rules. The consensus rules are the rules that all nodes in the network must agree on to remain cooperative in the same network. A node that does not follow the consensus rules will eventually get banned by its peer nodes.

What do the consensus rules consist of? Mostly of rules to consider a block valid or not, but a subset of the rules are concerned with checking if a transaction is valid or not. The rules that apply to transactions are that the inputs and outputs are balanced (you can’t spend more than the inputs) and that the signatures provided for the inputs are valid. The signature is a cryptographic signature that is computed using some math that is out of the scope of this blog post, and out of the scope of my understanding. The critical thing to know is that the cryptographic verification mathematically proves that the private key that created the original address is the same private key that is signing the spend authorization to send that bitcoin to another address.

Confirmed vs. Valid #

Valid Transaction: a transaction that passes the consensus rules

Confirmed Transaction: a transaction that has been included in a block on the blockchain. After it has appeared in a new block, it is said to have one confirmation. Each block that gets mined after its inclusion in the blockchain is counted as an additional confirmation.

So the inputs need to match or exceed the outputs, and they all need to be signed. But we haven’t actually talked about what an input is or where it comes from.

Origin of an Input #

Every input is the output of a previous transaction. That makes sense, every input you spend had to come from someone paying you at some point. Bitcoin wallet balances may work differently than how you mentally model your USD bank account. You may conceptually think of your bank account as a balance. Your paycheck comes in; the balance goes up. You buy dinner; your balance goes down. When money comes into your bank account, you imagine it adding to the balance. And when you spend, you don’t think about where the origin of that money came from, whether it was from this paycheck or that Venmo from a friend. It just comes out of the balance, the sum of all the money you have.

That is not how bitcoin is managed. Every payment you receive is kept track of individually. Each one is called a UTXO: Unspent Transaction Output. It is precisely what the name implies, an unspent output from a transaction. Someone paid you in a prior transaction, your address was in one of the outputs of that transaction, and you have not spent it yet. So it’s a UTXO. When you build a transaction, the inputs are UTXOs. Transactions consume UTXOs as inputs and produce new UTXOs as outputs. It also may become clear now why you would have more than one input in a transaction. If you want to spend an amount greater than what you’ve ever been paid in one output, you would need to use multiple UTXOs to have enough bitcoin available in the transaction.

Ok, that’s all well and good, but how did the first input happen? Where did that come from? What a perfect segue into our next lifecycle stage!

This isn’t your Minecraft Block (Confirm) #

Before we get to mining, we should probably step back to the broadcast and verification process briefly. You broadcasted the transaction from some initial node. That node verified the transaction, according to the consensus rules. Once that node verified that the transaction was valid, it sends the transaction on to each node that it is connected to. This process repeats until every node in the bitcoin network has seen the transaction, and has validated it. If any node deems the transaction invalid, it does not pass the transaction on to other nodes. Why might one node consider a transaction valid, while another does not? That sounds like a topic for another post.

Eventually, after propagating through the network, your transaction will arrive at the node of a miner. A miner is a member of the bitcoin network that is trying to create blocks. What are blocks? They are the fundamental element of the blockchain, which is the immutable distributed ledger that contains all bitcoin transactions.

Let’s break down a couple of words in that description so that they’re clear. Immutable: the blockchain does not change; it only gets added on to over time. Distributed: there is no one authoritative source of truth for the blockchain. It is spread out over each full node on the network. Each node verifies that the chain is valid, and maintains its own copy.

The blocks contain transactions. Miners collect transactions that are being broadcast around the network put them in a block that they are trying to build. Now we arrive at the question of “what exactly is a miner doing when they mine a block?” First, we need a quick detour into the world of hash functions.

Hash Functions #

Hash functions play a central role in the mining process. They are another element brought in from cryptography and use lots of complex math that I won’t go over here. But there are a few key things to know about hash functions, specifically SHA-256, which is the function used in bitcoin mining.

Hash functions take in an arbitrary amount of data and spit out a fixed amount of data. In the case of SHA-256, it outputs 256 bits of data, regardless of the input size. They are also deterministic, which means a given input will always produce the same output. Another critical thing about cryptographic hash functions is that changes to the input should produce unpredictable changes to the output. You shouldn’t be able to look at two outputs and tell “how close” their inputs were. A piece of data with a single difference will produce a completely different looking hash output. Another key thing about hash functions is that they do not work in reverse. You cannot take an output of a hash function and run it backwards to arrive at the original input. This can be reasoned about intuitively: you can’t expect to take some arbitrarily large input and output a small hash and be able to reverse back into the full input. Information is lost through the hashing process. If someone told you to summarize Leo Tolstoy’s War and Peace into one sentence, it would not be reasonable to expect someone who had never read War and Peace to be able to read your one-sentence summary and then work backward to rewrite all of War and Peace with 100% accuracy. The same principle applies to hash functions.

I’ve embedded a little hash function playground below. You can type into the box and view the SHA-256 output. You can see how the output size is fixed, no matter your input, and that the changes in the output vary wildly even when making incremental changes to the input. You can also see that if you delete characters and go back to a previous input, that you get the same output as before.

It is also helpful to note that the output here is a hexadecimal number. It is a number, but it’s base-16, so you count up past 9 using the letters a through f. You don’t have to understand base-16, but it is important to know that these outputs can be compared numerically (greater than, less than, equal to).

(Here’s a “fun” game to play: see how long it takes you to find an input where the output starts with two zeros 00.... I promise this will be relevant, and it’s ok to give up.)

Mining a Block #

With that basic understanding of hash functions, we can start to look at what mining is. A miner takes transactions that it has validated and groups them together into a block. But the block is not yet valid. For the block to be valid, it needs to have an SHA-256 hash output value less than some amount.

You may be wondering since hash functions are deterministic, how a miner gets different hash outputs if they’re dealing with the same set of transactions. Wouldn’t they always get the same output? There is a space in the block, for something called a nonce. The nonce is a throwaway value that, by definition, doesn’t mean anything. Its only purpose is to alter the input so that you get a different hash output. So the miners “mine” by trying a nonce value, hashing the block, and checking to see if the hash output is less than the set target. An individual miner will do that process trillions of times per second. We’ll talk more about the target in a minute.

If you tried out my “two zeros” exercise above, you might have found it very difficult (maybe not, depends how lucky you were with your input). The lower your target value is, the harder it is to find a valid hash. This can be reasoned about intuitively: if you limit the valid hashes to only hashes that start with 0, you are only allowing ¹⁄₁₆th of all the possible hashes (Because it could have started with 0, 1, 2, …, 9, a, b, c, d, e, f. Starting with zero is just one out of a possible sixteen values). If you require two zeros, you’re now only accepting ¹⁄₂₅₆th of all possible hashes. When you get to 19 zeros (which is about where the target is at the time of writing), you are only accepting ~1.3e-23 of all possible hash values. If you were going to try and find a valid hash with that level of difficulty in a reasonable amount of time (say about 10 minutes), you would need to be trying about 98 exahashes/sec. That’s exa- the SI prefix for Quintillion, or a billion billion. 98,000,000,000,000,000,000. It’s a lot.

And that amount is currently how many hashes the network of bitcoin miners tries each second, to find valid blocks.

This process is often referred to as proof of work. A phrase which is referring to the fact that because the block hashes to an incredibly rare hash output, there is proof that work was done since it almost certainly would not have happened by accident.

I made a demo to show what mining looks like. It’s like the earlier demo, but here, you provide some data that will be our fake transaction data. Then you set the difficulty that you want, represented here as the number of leading zeros, and click start. The miner will iterate through nonces trying to find a matching hash output. So it’s hashing your input + the nonce value, and this is how it’s getting new hash outputs. Remember that hash outputs are deterministic, so the only way to get a new output is to have a new input. That is precisely the function that the nonce serves.

You can adjust the difficulty from one leading zero to six. One to three leading zeros can be found pretty quickly on most hardware. Four to five might take a minute to find. With six, you may be waiting for several minutes. You can stop the search by clicking the button again.

Now let’s talk more about that target that determines which hash outputs are valid and which are not. This is referred to as the difficulty, and it fluctuates over time, every 2,016 blocks, to be precise. The goal of the network is to mine a block, on average, every 10 minutes. To achieve this, every 2,016 blocks (2 weeks if each block took 10 minutes to mine) a difficulty is computed by taking the last difficulty number used, and the time it took to mine the last 2,016 blocks and moving the difficulty up or down a magnitude proportional to the difference in actual time vs. desired time. So if the previous 2,016 blocks were mined in 13 days, but the goal was 14 days, then the next 2,016 blocks will have a greater difficulty, which translates to a smaller difficulty number (because the goal is to find a hash less than or equal to the difficulty number, so making the number smaller reduces the amount of valid hash outputs, and makes finding one more difficult).

Difficulty Epoch #

A period of 2,016 blocks is referred to as the difficulty epoch.

You can view a visualization of the current difficulty epoch at diff.cryptothis.com

It also can move the other direction. If the previous difficulty epoch took 18 days instead of the desired 14 days to complete, the difficulty would drop (meaning the difficulty number goes up, increasing the amount of valid hash outputs, making finding one easier). This is how the network maintains a consistent interval on the mining process even while the miners on the network are constantly increasing in number and efficiency.

When bitcoin first started, mining was done using CPUs (central processing unit), the generalized processors you find in any computer or phone. Then people moved on to using GPUs (graphics processing unit), a more specialized processor capable of working much faster, but less flexible in the types of tasks it can do. Finally, mining moved on to where it is now, which is done by ASICs (application-specific integrated circuit). ASICs are the least adaptable of the processors, it can literally only do one thing, but it does that one thing extremely quickly. The ASICs are used to compute the SHA-256 hash at the speed of electrons, which is to say: very fast.

Because of this, and the fact that there are so many miners on the network, it is virtually impossible for someone to mine a block using their home computer. The additional hash rate provided by your computer would be so minuscule compared to the existing hash rate. The odds of your computer finding a valid hash before any of the other miners on the network are such that you would be much better off saving the electricity cost and using it to buy a lottery ticket.

I skipped over some important parts of what goes into a block. We know that transactions go into a block. We know that there’s a nonce that lets us fiddle with the hash output of the block. Another crucial thing the block contains is the hash output of the previous block. This is what makes it a blockchain. Because each block contains within it a reference to the previous block, any attempt to change the data in a prior block will render all the proceeding blocks invalid, because the next block in the chain will be referencing a hash that no longer matches the hash output of its previous block.

There are scenarios that can cause the blockchain to diverge, referred to as forking. I’ll save those details for another post. But the most important thing to know about forks is how they are resolved. Nodes will accept the chain that was the most difficult to make, which generally just translates to whichever chain is longest.

Where does bitcoin come from? #

When a miner is working on a block, the first transaction in their block is one that they create, called the coinbase transaction. It’s used to pay themselves a reward, which is how new bitcoins get introduced into circulation. The coinbase transaction is different from other transactions, it has no inputs. The output goes to the miner’s address, and its amount is for the current block reward + transaction fees from the transactions in the block.

Block Reward and Halvings #

The reward amount is part of the consensus rules. If someone mined a block and tried to pay themselves an incorrect reward amount, no nodes would accept the block as valid.

The reward was initially 50 BTC. Every 210,000 blocks (~ 4 years), the reward is halved. It halved for the first time to 25 BTC in 2012, and then again in 2016 to 12.5 BTC, and most recently in May 2020 down to the current reward (as of this writing) of 6.25 BTC. These reward halvings are referred to as a halving or more tongue-in-cheekily as a halvening.

We skipped over fees earlier. Remember that example transaction I showed you at the beginning of this post where you used a 1 BTC input, and gave 0.9 BTC back to yourself, and 0.1 BTC to a friend? Well, in actuality you would need to include some kind of fee. The fee amount is recorded into the transaction implicitly by the difference between the sum of the inputs and the sum of the outputs. So in our example transaction, the difference is 0 (1 - (0.9 + 0.1) = 0). If you needed to pay your friend 0.1 BTC, then you would receive back less than 0.9, probably something more like 0.899999, which would leave 0.000001 for a fee (or 100 sats).

Satoshis (sats) #

A bitcoin is divisible up to eight decimal places. The smallest possible amount of bitcoin is one one-hundred-millionth of a bitcoin, and it is called a sat, short for Satoshi, the pseudonymous creator of Bitcoin.

So the miner gets to pay themselves in the coinbase transaction for the block reward (which is currently 6.25 BTC) and for the sum of all the fees in the transactions. The fee amount of the transaction is what incentivizes miners to include your transaction in their block. The blocks have a limited capacity, and if more transactions are needing to be confirmed than there is space in the upcoming block, the miners will prioritize including transactions that give them the most reward.

Recap #

We covered a lot of things, so let’s run through it all briefly to solidify our mental model.

Build #

When we transact with bitcoin, we build a transaction that consists of inputs and outputs. Inputs are UTXOs (unspent transaction outputs) that belong to us; they are outputs that were paid to us in earlier transactions. They are used as inputs to this transaction, and we create new outputs using the address(es) that we are sending bitcoin to. When we add inputs to a transaction, all of its value must be “spent” in the transaction, so any amount left over that we don’t intend to spend will be paid back to us. This is called the “change output” but there’s nothing special about it, it’s just a normal output; it just happens to be paid to an address that we control.

Sign #

Once we’ve built the transaction, we can sign it. Specifically, we’re signing the inputs, communicating to the network that we are indeed the owner of the address that the bitcoin was originally paid to, and we are authorizing the spending of them now, in this transaction. If you are using a software wallet, signing will probably happen transparently and seamlessly. If you are using a hardware wallet, this will involve more ceremony, as you need to get the unsigned transaction to the hardware wallet so that it can sign it, and get the signed transaction back to your computer or phone for broadcasting.

Broadcast & Validate #

Then we broadcast the transaction to the network, using either our own node or someone else’s. That node will validate the transaction, and if it determines it to be valid, it will forward it along to all the nodes it’s connected to. Each of those nodes will do the same thing, validating the transaction before continuing to broadcast it out to their connected nodes. Eventually, our transaction will have made its way around the world to each bitcoin node on the network. Some of those nodes will be miners.

Mempool #

The mempool is the name for the set of transactions that are valid but are still awaiting being included in a block.

You can view a visualization of the mempool at mempool.space

Confirm #

Miners take transactions from the mempool and try to build a block containing them. If there are more transactions in the mempool than will fit in a block, the miners will preferentially try to include transactions that net them a higher fee. Each miner will iterate through nonce values, hashing the block, and testing to see if its output fits under the current difficulty threshold. If it finds a valid block, it broadcasts it on the network the same way a transaction is broadcast. Each node will validate the block and each transaction within it. Transactions in the new block are removed from the node’s mempool.

Additionally, every 2,016 blocks a new mining difficulty is calculated. If the mining were happening too quickly (on average faster than 10 minutes per block), it would get more difficult. If the mining were happening too slowly (on average longer than 10 minutes per block), the difficulty would drop.

Hopefully if you made it through all of that, you feel like you have a better understanding of what is going on when you transact with bitcoin. And hopefully you also feel like you understand a bit more about what mining looks like, why it’s difficult, and why you’re unlikely to make any money mining on your home computer.

If this left you with more questions, I highly recommend Andreas Antonopoulos’ book Mastering Bitcoin. It covers almost every aspect of Bitcoin in great detail. I think it’s readable for non-software developers, while still being useful to those who are software developers. Definitely check it out if you found this post interesting. And I mean, you’re reading the bottom, so you probably were, or you’re one of my poor loved ones who is proofreading this for me for the umpteenth time. ❤️