Create and sign a standard raw Bitcoin transaction.

This is a second iteration of the previous project Creating and signing a standard raw Bitcoin transaction and draws heavily on the work already done there.


This will involve:
- Creating a standard raw Bitcoin transaction.
- Signing the transaction using ECDSA cryptography.
- Broadcasting the transaction so that it can be mined, which will confirm its validity.


My working definition of a standard transaction:
- It has at least one input and at least one output.
- All input and output addresses are Pay-To-Public-Key-Hash (P2PKH).
- All input scriptSigs contain uncompressed public keys.


Note: The form of the transaction that is actually signed differs slightly for each input address.







- Goal
- Contents
- Brief Summary
- Summary
- Downloadable Assets
- Recipes For Using Various Downloadable Assets
- Notes
- Further Work
- Project Log







During this project, I developed a toolset that allows me to precisely and quickly:
1) generate a Bitcoin address from a private key,
2) construct and sign a standard Bitcoin transaction with one input and one output,
and
3) construct and sign a nonstandard Bitcoin transaction with one standard input and one nonstandard output.

Please read the Recipes For Using Various Downloadable Assets section if you would like to know how to use this toolset.

Here are the notable occurrences in this project:
- I generated two standard addresses, a "source address" and a "target" address.
- I looked up the then-current receiving address on my LocalBitcoins account. This was the "final" address. It was nonstandard.
- I transferred some bitcoin from my LocalBitcoins account to the source address.
- I constructed and signed a standard transaction that moved the available bitcoin from the source address to the target address. This transaction was then broadcast and mined.
- I constructed and signed a nonstandard transaction that moved the available bitcoin from the target address to the final address. This transaction was then broadcast and mined.

Please read the Summary section for a more thorough description of this project.











Original plan of action for this project:


- Generate an address: the "source" address.
- Generate a second address: the "target" address.
- Look up my current receiving address on my LocalBitcoins account: the "final" address. This will be a nonstandard address.
- Transfer some bitcoin from my LocalBitcoins account to the source address. I'll call this tx0. "tx" = transaction.
- Look up recent mining fees.
- Construct a standard raw Bitcoin transaction that moves the available bitcoin from the source address to the target address. Be careful to include a mining fee. This is tx1.
- Sign the transaction using the private key of the source address and ECDSA cryptography. Before signing, the transaction must be altered slightly to get the transaction-in-signable-form.
- Embed the transaction signature into the transaction.
- Broadcast the transaction.
- Confirm that the transaction has been mined (included in a block in the Bitcoin blockchain). This verifies that the transaction was valid (i.e. in the correct format, with a mathematically valid signature).
- Look up recent mining fees.
- Construct a second nonstandard raw Bitcoin transaction that moves the available bitcoin from the target address to the final address. Be careful to include a mining fee. This is tx2.
- Sign the transaction using the private key of the target address and ECDSA cryptography. Before signing, the transaction must be altered slightly to get the transaction-in-signable-form.
- Embed the transaction signature into the transaction.
- Broadcast the transaction.
- Confirm that the transaction has been mined (included in a block in the Bitcoin blockchain). This verifies that the transaction was valid (i.e. in the correct format, with a mathematically valid signature).




What I actually did during the project:


- Created a project directory:
creating_and_signing_a_standard_raw_bitcoin_transaction_2
- Within the project directory, created a work directory named "work".
- Downloaded the asset
generate_bitcoin_address_2.py
along with its dependencies:
-- bjorn_edstrom_ripemd160.py
-- ecdsa-0.10.tar.gz
-- pypy_sha256.py
- Moved these downloaded files to the work directory.
- Unzipped ecdsa-0.10.tar.gz and copied the directory "ecdsa" within it to the work directory.
- Rewrote
generate_bitcoin_address_2.py
into the two files
generate_bitcoin_address_3.py
and
bitcoin_functions.py.
- Used
generate_bitcoin_address_3.py
to generate the source address.
- Used
generate_bitcoin_address_3.py
to generate the target address.
- Looked up my current receiving address on my LocalBitcoins account, the final address.
- Estimated an amount of bitcoin that would cover the high end of expected transaction and deposit fees, with extra so as to have room to manoeuvre.
- Transferred some bitcoin from my LocalBitcoins account to the source address. This transfer is included in a Bitcoin transaction, created by LocalBitcoins, that I refer to as tx0 during this project.
- Used a web service to look up recent mining fees.
- Chose excerpts from the first iteration of this project, Creating and signing a standard raw Bitcoin transaction, that outline the format of a standard raw transaction and the data that must be gathered in order to construct it.
- Wrote a script
create_transaction.py
that can construct and sign a single-input, single-output standard transaction.
- For the needs of
create_transaction.py
I wrote a supporting file
transaction.py
and additional functions for
bitcoin_functions.py.
- Looked up some of the necessary transaction input data on a web service. Some additional work was required to process the data.
- Using
create_transaction.py,
constructed and signed a standard raw Bitcoin transaction that moves the available bitcoin from the source address to the target address. A mining fee was included, set at the low end of the then-current fee range. This was tx1.
- The script
create_transaction.py
automated these steps in the original plan, so that they were achieved in one step:
{
- Construct a standard raw Bitcoin transaction that moves the available bitcoin from the source address to the target address. Be careful to include a mining fee. This is tx1.
- Sign the transaction using the private key of the source address and ECDSA cryptography. Before signing, the transaction must be altered slightly to get the transaction-in-signable-form.
- Embed the transaction signature into the transaction.
}
- Used a web service to successfully decode this transaction, which indicated that it was formatted correctly.
- Used a web service to successfully broadcast this signed transaction. Time taken for mining + 6-block-deep confirmation = 4.5 hours. This verified that the transaction was valid (i.e. in the correct format, with a mathematically valid signature).
- Used a web service to look up recent mining fees.
- Chose an asset and excerpts from the first iteration of this project, Creating and signing a standard raw Bitcoin transaction, that outline the format of a Pay-To-Script-Hash (P2SH) transaction output and how to obtain the redeem script hash from the P2SH address.
- Wrote a script
create_nonstandard_transaction.py
that can construct and sign a single-input, single-output nonstandard transaction, where the input is standard Pay-To-Public-Key-Hash (P2PKH) and the output is P2SH.
- For the needs of
create_nonstandard_transaction.py,
I wrote a supporting file
nonstandard_transaction.py
and additional functions for
nonstandard_bitcoin_functions.py.
I also added or altered functions in
bitcoin_functions.py.
- Using
create_nonstandard_transaction.py,
constructed and signed a standard raw Bitcoin transaction that moves the available bitcoin from the target address to the final address. A mining fee was included, set at the low end of the then-current fee range. This was tx2.
- The script
create_nonstandard_transaction.py
automated these steps in the original plan, so that they were achieved in one step:
{
- Construct a second nonstandard raw Bitcoin transaction that moves the available bitcoin from the target address to the final address. Be careful to include a mining fee. This is tx2.
- Sign the transaction using the private key of the target address and ECDSA cryptography. Before signing, the transaction must be altered slightly to get the transaction-in-signable-form.
- Embed the transaction signature into the transaction.
}
- Used a web service to successfully decode this transaction, which indicated that it was formatted correctly.
- Used a web service to successfully broadcast this signed transaction. Time taken for mining + 6-block-deep confirmation ~= 6 hours. This verified that the transaction was valid (i.e. in the correct format, with a mathematically valid signature).
- Rewrote
create_transaction.py,
incorporating various improvements developed in
create_nonstandard_transaction.py.
- Rewrote
create_nonstandard_transaction.py
slightly.
- Rewrote various other code files to refactor, tidy up output, make the code cleaner, add notes (author, date, description, environments, comments, instructions), and do a complete check of the final state of the code.
- Performed final runs of these scripts:
create_transaction.py
create_nonstandard_transaction.py
and
generate_bitcoin_address_3.py,
confirming that their output matched the output produced and used earlier in the project.




Financial Information:


- I calculated an amount of bitcoin that was large enough to ensure that I would be able to pay two Bitcoin transaction fees + the LocalBitcoins deposit fee, with extra so as to have room to manoeuver. This was 0.00242 bitcoin, or about $15 at then-current prices.
- I transferred 0.00242 bitcoin in tx0.
- LocalBitcoins charged a transaction fee of 0.00005 bitcoin for tx0. This was subtracted from my LocalBitcoins wallet balance, not from the amount that was actually sent.
- For tx1, I used a transaction fee of 223 satoshis (0.00000223 bitcoin).
- For tx2, I used a transaction fee of 225 satoshis (0.00000225 bitcoin).
- When it accepted and processed tx2, LocalBitcoins charged a deposit fee of 0.00015000 bitcoin.
- Total bitcoin spent: LocalBitcoins transaction fee for tx0 + tx1 transaction fee + tx2 transaction fee + LocalBitcoins deposit fee for tx2 = 0.00005000 + 0.00000223 + 0.00000225 + 0.00015000 = 0.00020448 bitcoin.




Recipes:


The following recipes can be found in the Recipes For Using Various Downloadable Assets section.

- Recipe 1: Setup
-- Some preparation for the other recipes.

- Recipe 2: generate_bitcoin_address_3.py
-- Generating a standard Bitcoin address from a private key.

- Recipe 3: create_transaction.py
-- Constructing and signing a standard Bitcoin transaction with one input and one output.

- Recipe 4: create_nonstandard_transaction.py
-- Constructing and signing a nonstandard Bitcoin transaction with one input and one output.




Notes section:


The Notes section contains the following parts:

- Definitions and acryonyms
- Web services used during this project
- Gathering data in order to construct a new transaction using the scripts created in this project
- Pay-To-Script-Hash (P2SH) addresses
- Addresses generated and/or used in this project
- Transactions created during this project











My work computer: Aineko, a 2008 Macbook running Mac OS X 10.6.8 Snow Leopard.




My working definition of a standard transaction:
- It has at least one input and at least one output.
- All input and output addresses are Pay-To-Public-Key-Hash (P2PKH).
- All input scriptSigs contain uncompressed public keys.


My working definition of a nonstandard transaction:
- It has at least one input and at least one output.
- At least one input or output address is not a standard Pay-To-Public-Key-Hash (P2PKH) address.


I define a "standard" address to be an uncompressed single-signature Pay-To-Public-Key-Hash (P2PKH) address.



The following assets were developed during this project and were run using Python 2.7.13 on Aineko.

Description: A library of functions for handling standard Bitcoin data types.
bitcoin_functions.py [paywalled]

Description: A script that creates and signs a nonstandard Bitcoin transaction with one standard P2PKH input and one nonstandard P2SH output.
create_nonstandard_transaction.py [paywalled]

Description: A script that creates and signs a standard Bitcoin transaction with one input and one output.
create_transaction.py [paywalled]

Description: A script that generates a standard Bitcoin address from a private key.
generate_bitcoin_address_3.py [paywalled]

Description: A library of functions for handling nonstandard Bitcoin data types.
nonstandard_bitcoin_functions.py [paywalled]

Description: A library of classes for nonstandard Bitcoin transactions.
nonstandard_transaction.py [paywalled]

Description: A library of classes for standard Bitcoin transactions.
transaction.py [paywalled]


Note: The asset
generate_bitcoin_address_3.py
was based on the asset
generate_bitcoin_address_2.py [paywalled],
associated with the article Verifying a signed deed of the GPG 1.4.10 source code. Please see this previous article for details concerning this previous asset.


The following assets are associated with the article Reading and verifying a standard raw bitcoin transaction. Please read the article for information and advice on these assets.

Description: An implementation of RIPEMD-160, written by Björn Edström.
bjorn_edstrom_ripemd160.py [paywalled]

Description: A Python implementation of ECDSA cryptography, written by Peter Pearson.
ecdsa-0.10.tar.gz [paywalled]

Description: A Python implementation of SHA256.
pypy_sha256.py [paywalled]













Recipe list:


- Recipe 1: Setup
-- Preparation for the other recipes.

- Recipe 2: generate_bitcoin_address_3.py
-- Generating a standard Bitcoin address from a private key.

- Recipe 3: create_transaction.py
-- Constructing and signing a standard Bitcoin transaction with one input and one output.

- Recipe 4: create_nonstandard_transaction.py
-- Constructing and signing a nonstandard Bitcoin transaction with one input and one output.




Recipe 1: Setup


Initial condition: You must have Python 2.7.x installed on your computer. These code assets were tested under Python 2.7.13 running on Mac OS X 10.6.8 (Snow Leopard), and should run successfully on other versions of Python 2.7.


Steps:

- Create a work directory.

- Browse to the Downloadable Assets section of this article.

- Download the asset bjorn_edstrom_ripemd160.py.

- Download the asset ecdsa-0.10.tar.gz.

- Download the asset pypy_sha256.py.

- Download the asset bitcoin_functions.py.

- Move these assets into the work directory.

- Unpack the zipped tape archive file ecdsa-0.10.tar.gz. This can be done by opening a terminal, changing directory to the work directory, and running the following command:

This unpacking should produce a new directory named ecdsa-0.10 in the work directory. The directory ecdsa-0.10 should contain a directory named ecdsa. Copy the ecdsa directory into the work directory.




Recipe 2: generate_bitcoin_address_3.py


Notes:
- generate_bitcoin_address_3.py is a script that generates a standard Bitcoin address from a private key.
- All control values in this script are strings.


Steps:

- Work through Recipe 1: Setup.

- Generate a random private key that is between 1 and 32 bytes long. This requires some work (e.g. using dice) and is not covered in this recipe.

Note: Ideally a private key should be 32 bytes long. It can be raw bytes (printable ASCII characters) or hex bytes (hex characters). Hex characters must be lowercase. 1 byte is represented by 2 hex characters, so a 32-byte private key will be 64 hex characters long. I recommend hex bytes, as these can represent all possible private keys. I find that raw byte private keys are convenient during testing.

- Browse to the Downloadable Assets section of this article.

- Download the asset generate_bitcoin_address_3.py. Move it into the work directory.

- Open the file generate_bitcoin_address_3.py in a text editor. Scroll to the section of text that lies between

and


- If the private key is in raw bytes, set the variable to and set the variable to the private key.

- If the private key is in hex bytes, set the variable to and set the variable to the private key.

- Open a terminal and change directory to the work directory.

- Run the following command:


- The output should contain the Bitcoin address that corresponds to your private key.

Note: Please go to the "Addresses generated and/or used in this project" part of the Notes section to see two examples of the output of this script.




Recipe 3: create_transaction.py


Notes:
- create_transaction.py is a script that creates and signs a standard Bitcoin transaction with one input and one output. The input address and the output address must both be standard P2PKH addresses. Standard P2PKH addresses start with the character '1'.
- All control values in this script are strings.
- New transactions spend unspent-outputs-of-previous-transactions.
- The bitcoin balance of an address is the sum of the unspent outputs sent by previous transactions to this address.
- To spend bitcoin from an address, the bitcoin amount must be constructed from complete unspent outputs. Any change can be sent in a new unspent output back to the original address or another that you control. Unspent outputs cannot be broken into smaller unspent outputs prior to spending - new unspent outputs (larger or smaller) must be created by a new transaction.


Definitions:
- input address: the address that currently contains the bitcoin that you want to transfer in this transaction.
- output address: the address to which the bitcoin will be transferred by this transaction.
- previous_output_hash: big-endian 32-byte double-SHA256 hash of a previous transaction.
- txid: "transaction id", the little-endian form of the previous_output_hash.


Steps:

- Work through Recipe 1: Setup.

- Gather the following pieces of information:

-- The txid of the previous transaction that transferred the unspent output to the input address.

-- The index of this unspent output in the previous transaction (the "previous_output_index"). This index is implicit and starts at zero.

-- The private key (in hex bytes) of the input address.

-- The bitcoin amount (or satoshi amount) contained within the unspent output. Let's call this the "input amount".

- Choose the following pieces of information:

-- The output address.

-- The bitcoin amount (or satoshi amount) that you wish to transfer to the output address. Let's call this the "output amount".

-- The change address. Currently, this is the same as the output address. Any value not assigned to an output address will be assigned to the change address. The fee will then be subtracted from the value assigned to the change address.

-- The fee (in satoshi) or fee_rate (in satoshi / byte). The fee must be an integer e.g. . The fee_rate must be an integer e.g. or a float e.g. . You should look up the range of fee rates for transactions that are currently being mined.

- Generate a random value that is between 1 and 32 bytes long. This requires some work (e.g. using dice) and is not covered in this recipe.

Note: This random value will be used for creating the ECDSA signature of the transaction. Every ECDSA signature you ever make should use a different random value. Ideally a random value should be 32 bytes long. It can be raw bytes (printable ASCII characters) or hex bytes (hex characters). Hex characters must be lowercase. 1 byte is represented by 2 hex characters, so a 32-byte random value will be 64 hex characters long. I recommend hex bytes, as these can represent all possible random values. I find that raw byte random values are convenient during testing.

- Browse to the Downloadable Assets section of this article.

- Download the asset transaction.py.

- Download the asset create_transaction.py.

- Move these assets into the work directory.

- Open the file create_transaction.py in a text editor. Scroll to the section of text that lies between

and


- Set the variable to the random value.

- If the random value is in raw bytes, set the variable to .

- If the random value is in hex bytes, set the variable to .

- The variable is a dictionary that contains several other variables.

-- Set the value of the variable to be the txid.

-- Set the value of the variable to be the index of the unspent output.

-- Set the value of the variable to be the private key.

-- If the input amount is in satoshi, set the value of the variable to be the input amount.

-- If the input amount is in bitcoin, set the value of the variable to be the input amount.

Note: Currently, if both of the variables and are set, then will take precedence. You can comment out the unused variable of the pair by placing a number sign ('#') at the start of the relevant line.

- The variable is a dictionary that contains several other variables.

-- Set the value of the variable to be the output address.

-- If the output amount is in satoshi, set the value of the variable to be the output amount.

-- If the output amount is in bitcoin, set the value of the variable to be the output amount.

Note: Currently, if both of the variables and are set, will take precedence. You can comment out the unused variable of the pair by placing a number sign ('#') at the start of the relevant line.

- Set the variable to be the change address.

- If you have chosen to set the fee as an absolute number of satoshi, set the variable to be the fee amount and set the variable to be .

- If you have chosen to set the fee as a fee rate (in satoshi / byte), set the variable to be the fee rate and set the variable to be .

- Open a terminal and change directory to the work directory.

- Run the following command:


- The output should contain the signed transaction as a hex byte sequence without spaces.

Note: Please go to the "Transactions created during this project" part of the Notes section to see an example of the output of this script.




Recipe 4: create_nonstandard_transaction.py


Notes:
- create_nonstandard_transaction.py is a script that creates and signs a nonstandard Bitcoin transaction with one input and one output. The input address must be a standard P2PKH address and the output address must be a nonstandard P2SH address. P2PKH addresses start with the character '1'. P2SH addresses start with the character '3'.
- All control values in this script are strings.
- New transactions spend unspent-outputs-of-previous-transactions.
- The bitcoin balance of an address is the sum of the unspent outputs sent by previous transactions to this address.
- To spend bitcoin from an address, the bitcoin amount must be constructed from complete unspent outputs. Any change can be sent in a new unspent output back to the original address or another that you control. Unspent outputs cannot be broken into smaller unspent outputs prior to spending - new unspent outputs (larger or smaller) must be created by a new transaction.


Definitions:
- input address: the address that currently contains the bitcoin that you want to transfer in this transaction.
- output address: the address to which the bitcoin will be transferred by this transaction.
- previous_output_hash: big-endian 32-byte double-SHA256 hash of a previous transaction.
- txid: "transaction id", the little-endian form of the previous_output_hash.


Steps:

- Work through Recipe 1: Setup.

- Gather the following pieces of information:

-- The txid of the previous transaction that transferred the unspent output to the input address.

-- The index of this unspent output in the previous transaction (the "previous_output_index"). This index is implicit and starts at zero.

-- The private key (in hex bytes) of the input address.

-- The bitcoin amount (or satoshi amount) contained within the unspent output. Let's call this the "input amount".

- Choose the following pieces of information:

-- The output address.

-- The bitcoin amount (or satoshi amount) that you wish to transfer to the output address. Let's call this the "output amount".

-- The change address. Currently, this is the same as the output address. Any value not assigned to an output address will be assigned to the change address. The fee will then be subtracted from the value assigned to the change address.

-- The fee (in satoshi) or fee_rate (in satoshi / byte). The fee must be an integer e.g. . The fee_rate must be an integer e.g. or a float e.g. . You should look up the range of fee rates for transactions that are currently being mined.

- Generate a random value that is between 1 and 32 bytes long. This requires some work (e.g. using dice) and is not covered in this recipe.

Note: This random value will be used for creating the ECDSA signature of the transaction. Every ECDSA signature you ever make should use a different random value. Ideally a random value should be 32 bytes long. It can be raw bytes (printable ASCII characters) or hex bytes (hex characters). Hex characters must be lowercase. 1 byte is represented by 2 hex characters, so a 32-byte random value will be 64 hex characters long. I recommend hex bytes, as these can represent all possible random values. I find that raw byte random values are convenient during testing.

- Browse to the Downloadable Assets section of this article.

- Download the asset transaction.py.

- Download the asset nonstandard_bitcoin_functions.py

- Download the asset nonstandard_transaction.py.

- Download the asset create_nonstandard_transaction.py

- Move these assets into the work directory.

- Open the file create_nonstandard_transaction.py in a text editor. Scroll to the section of text that lies between

and


- Set the variable to the random value.

- If the random value is in raw bytes, set the variable to .

- If the random value is in hex bytes, set the variable to .

- The variable is a dictionary that contains several other variables.

-- Set the value of the variable to be the txid.

-- Set the value of the variable to be the index of the unspent output.

-- Set the value of the variable to be the private key.

-- If the input amount is in satoshi, set the value of the variable to be the input amount.

-- If the input amount is in bitcoin, set the value of the variable to be the input amount.

Note: Currently, if both of the variables and are set, then will take precedence. You can comment out the unused variable of the pair by placing a number sign ('#') at the start of the relevant line.

-- Set the value of the variable to be .

- The variable is a dictionary that contains several other variables.

-- Set the value of the variable to be the output address.

-- If the output amount is in satoshi, set the value of the variable to be the output amount.

-- If the output amount is in bitcoin, set the value of the variable to be the output amount.

Note: Currently, if both of the variables and are set, will take precedence. You can comment out the unused variable of the pair by placing a number sign ('#') at the start of the relevant line.

-- Set the value of the variable to be .

- Set the variable to be the change address.

- If you have chosen to set the fee as an absolute number of satoshi, set the variable to be the fee amount and set the variable to be .

- If you have chosen to set the fee as a fee rate (in satoshi / byte), set the variable to be the fee rate and set the variable to be .

- Open a terminal and change directory to the work directory.

- Run the following command:


- The output should contain the signed transaction as a hex byte sequence without spaces.

Note: Please go to the "Transactions created during this project" part of the Notes section to see an example of the output of this script.













Parts:
- Definitions and acryonyms
- Web services used during this project
- Gathering data in order to construct a new transaction using the scripts created in this project
- Pay-To-Script-Hash (P2SH) addresses
- Addresses generated and/or used in this project
- Transactions created during this project




Definitions and acryonyms


txid = "transaction id"

P2PKH = Pay-To-Public-Key-Hash

P2SH = Pay-To-Script-Hash


I'm going to define:
- previous_output_hash = big-endian 32-byte double-SHA256 hash of a previous transaction.
- txid = little-endian form of the previous_output_hash


My working definition of a standard transaction:
- It has at least one input and at least one output.
- All input and output addresses are Pay-To-Public-Key-Hash (P2PKH).
- All input scriptSigs contain uncompressed public keys.


My working definition of a nonstandard transaction:
- It has at least one input and at least one output.
- At least one input or output address is not a standard Pay-To-Public-Key-Hash (P2PKH) address.




Web services used during this project


I used
bitcoinfees.earn.com
to observe the then-current transaction fees and thereby choose fees for my transactions. It displays estimated delays (in blocks and in minutes) for ranges of fee rates (in satoshis / byte).

I used the search field at
live.blockcypher.com
to search for transactions by txid.

On the resulting transaction pages e.g.
live.blockcypher.com/btc/tx/e4609e0f1ca854b8b07381f32ba31adbad9713205f5a4f3f56a5a32853d47855
I was able to click Advanced Details / API Call, leading to a raw text dump of the particular transaction, e.g.
api.blockcypher.com/v1/btc/main/txs/e4609e0f1ca854b8b07381f32ba31adbad9713205f5a4f3f56a5a32853d47855?limit=50&includeHex=true

I used
live.blockcypher.com/btc/decodetx
to decode signed transactions, indicating that they were formatted correctly.

I used
live.blockcypher.com/btc/pushtx
to upload signed transactions for broadcast to the Bitcoin network.




Gathering data in order to construct a new transaction using the scripts created in this project


New transactions spend unspent-outputs-of-previous-transactions.

The scripts
- create_transaction.py
- create_nonstandard_transaction.py
can derive various other necessary values from the control values.

Please note that these scripts only support creating transactions with a single input and a single output.


create_transaction.py:

The controls are:
- [input] random_value
- [input] txid
- [input] previous_output_index
- [input] private_key_hex
- [input] bitcoin_amount (or satoshi_amount)
- [output] address
- [output] bitcoin_amount (or satoshi_amount)
- [output] change_address
- [output] fee (or fee_rate)

Notes:
- [input] random_value: This is the random entropy used for signing the new transaction with private_key_hex using ECDSA cryptography. It must be new and random for every signature ever made.
- [input] txid: This is the transaction id (txid) of an existing transaction containing an unspent output.
- [input] previous_output_index: This is the implicit index of the unspent output in the list of outputs in the existing transaction that contains it.
- [input] private_key_hex: This is the private key of the address containing the unspent output.
- [input] bitcoin_amount (or satoshi_amount): This is the bitcoin amount contained in the unspent output.
- [output] address: This is the address to which the new transaction will transfer bitcoin.
- [output] bitcoin_amount (or satoshi_amount): This is the amount to be transferred to the output address.
- [output] change_address: This is the address that will receive any unassigned input bitcoin. Currently, this is the same as the output address. The fee will be subtracted from the amount directed to the change address output.
- [output] fee (or fee_rate): Specifies the fee to be paid to a miner for including the new transaction in a new block. The fee is stored in the transaction implicitly as the difference between the total input value and the total output value.


create_nonstandard_transaction.py:

The controls are the same as those for
create_transaction.py,
with two additional ones:
- [input] input_type
- [output] output_type

Notes:
- [input] input_type: This is the nonstandard transaction input type. It can be either "p2pkh" (standard) or "p2sh" (nonstandard). Currently, only p2pkh is supported for nonstandard transaction inputs.
- [output] output_type: This is the nonstandard transaction output type. It can be either "p2pkh" (standard) or "p2sh" (nonstandard). Currently, only p2sh is supported for nonstandard transaction outputs.




Pay-To-Script-Hash (P2SH) addresses


Pay-To-Script-Hash (P2SH) addresses begin with a '3' character. Example:
The final address
328cTqexYnQRbN5Dgs12D89sYiPPvtWVbF
begins with a '3' and is therefore a P2SH address.

The reason for this is: A string of 21 bytes that starts with the '05' byte, once converted into base58check encoding (with a 4-byte checksum added by the encoding function), always starts with the base58 symbol '3'.

I examined P2SH addresses in some detail in the first iteration of this project:
Creating and signing a standard raw Bitcoin transaction

Summaries of my results can be found in this previous project, in the Notes / Discovery section, in these parts:
- P2SH multi-signature addresses - security
- P2SH multi-signature addresses - results from this project




Addresses generated and/or used in this project


Source address:




Target address:




Final address (my then-current receiving address on LocalBitcoins):






Transactions created during this project


I'll use:
- txid0 to mean the txid (transaction id) of tx0
- txid1 to mean the txid of tx1
- txid2 to mean the txid of tx2



tx0:

I transferred some bitcoin from my LocalBitcoins account to the source address. LocalBitcoins created a larger transaction, which included many other transfers, that executed this transfer.

txid0:


Link:
live.blockcypher.com/btc/tx/e4609e0f1ca854b8b07381f32ba31adbad9713205f5a4f3f56a5a32853d47855

Key details:
- block_hash:
00000000000000000022ae51b839f59396e49ffb8217bd10fa00a486360ef2e9
- block_height:
543926
- fees:
69494
- size:
1348
- confirmed:
2018-10-01T14:41:56Z

From some reading, the "Z" suffix on Unix timestamps indicates UTC time values.

UTC = Coordinated Universal Time (identical to GMT = Greenwich Mean Time).



tx1:

tx1 transferred the available bitcoin from the source address to the target address, minus a transaction fee.

txid1:


Link:
live.blockcypher.com/btc/tx/745e224ccba0a033c55ea80523f207da18b903418ac1f5d293eed62c19e0334d

Key details:
- block_hash:
0000000000000000000eecb4423435b1e52db78e8c83ae9e271978c2ef9d186e
- block_height:
544268
- size:
223 bytes
- fees:
223
- confirmed:
2018-10-03T23:25:01Z

Actual fee rate for tx1 is 223 / 223 = 1 (satoshi / byte).

Time taken for mining + 6-block-deep confirmation = 4.5 hours.

The estimated delay for a transaction with this fee rate was:
3-23 blocks (25-300 minutes) for a fee rate of 1-2 satoshis / byte.



tx2:

tx2 transferred the available bitcoin from the target address to the final address, minus a transaction fee.

txid2:


Link:
live.blockcypher.com/btc/tx/598f47dfb44da351c8576fb975bdab3b023777bb1c1d8ba727afd528ca4d7cf2

Key details:
- block_hash:
0000000000000000000d1f94b44eafbe52692ec790caf63a045d25e3080ba095
- block_height:
545334
- size:
221 bytes
- fees:
225
- confirmed:
2018-10-11T16:00:06Z

Actual fee rate for tx2 is 225 / 221 ~= 1.0181 (satoshi / byte).

Time taken for mining + 6-block-deep confirmation ~= 6 hours.

The estimated delay for a transaction with this fee rate was:
2-11 blocks (10-180 minutes) for a fee rate of 1-2 satoshis / byte.






Creation of tx1 (final run of the script - the final result has been confirmed to be identical to the original transaction that was actually broadcast):








Creation of tx2 (final run of the script - the final result has been confirmed to be identical to the original transaction that was actually broadcast):













The P2SH outputScript is 3 single-byte opcodes + a 20-byte redeemScriptHash i.e. 23 bytes. Will a nonstandard transaction with a single P2PKH input and a single P2SH output always be 221 bytes long?


In contrast, the P2PKH scriptPubKey is 5 single-byte opcodes + a 20-byte publicKeyHash i.e. 25 bytes. Will a standard transaction with a single P2PKH input and a single P2PKH output always be 223 bytes long?


Add an "Actual transaction size" output (in bytes) to
create_transaction.py
and
create_nonstandard_transaction.py,
to be displayed after the transaction has been signed and its signature has been embedded into it.
- Also a corresponding "Actual fee rate" output (in satoshi / byte).


Add a control for "scriptPubKey of unspent output" to
create_transaction.py
and
create_nonstandard_transaction.py,
which should be confirmed to be identical to the scriptPubKey derived from the corresponding private_key_hex control.
- The scriptPubKey of an unspent output should be extracted from the previous transaction that contains this unspent output.
- This is a check that ensures that any strangeness concerning the unspent output's scriptPubKey-in-the-blockchain will be caught at the start of the script, rather than when the transaction is broadcast and never mined.


Need a way to look up an output by its index in a previous transaction that is better than reading the transaction and manually counting outputs.


Construct and broadcast a standard transaction with two inputs and one output.
- test whether, within the transaction-in-signable-form, the input-not-being-used-to-sign is removed and its script_length var_int set to 0x00.


Construct and broadcast a standard transaction with one input and two outputs.


Construct and broadcast a standard transaction with two inputs and two outputs.


In transaction.py, in the class Transaction, write a validate_self() instance method, that will all the properties of a transaction prior to signing (and the properties of all its inputs and outputs). It should be called prior to signing.
- Also do this for
nonstandard_transaction.py.


In transaction.py, in the class Transaction, in the method get_signed_form(), check that the transaction has actually been signed (i.e. that every input has an associated scriptSig). This check should be performed prior to constructing and returning the signed_form.
- Also add this check to the method
get_description_of_signed_form().
- Also add both these checks to
nonstandard_transaction.py.


In create_transaction.py, validate the length domain of the random_value in the preparation phase, long before transaction signing is performed.
- Also do this for
create_nonstandard_transaction.py.


In create_transaction.py, how to handle the case where multiple transaction outputs have the change_address? Currently, the change address is more accurately a single output sent to that address. If two transaction outputs have the change address, the transaction fee can only be subtracted from one of these outputs. The fee could be subtracted from the total of all outputs sent to the change address. Alternatively, raise an error if multiple outputs have the same address.
- Same problem exists in
create_nonstandard_transaction.py.


Construct and broadcast a transaction that sends two outputs to the same address.


In bitcoin_functions.py, in the function
private_key_hex_to_public_key_hex(input)
investigate whether:
- verifying_key_bytes is always 64 bytes long.
- any padding / checking is done explicitly and carefully in the ECDSA library.


In bitcoin_functions.py, in the function
int_to_var_int(input),
expand it so that it can handle var_ints that are larger than 1 byte.


In bitcoin_functions.py, in the function
satoshi_to_bitcoin(input):
- how many satoshi can actually exist, in total? I.e. what should be the maximum permissible input value?


In bitcoin_functions.py, in the function
bitcoin_address_to_public_key_hash_hex(input)
confirm that first byte in the result is 0x00.


In bitcoin_functions.py, write a function called
validate_string_is_digits().
This can then be used to validate the fee control value (if fee is used rather than fee_rate) in
create_transaction.py.
- Also add this validation check in
create_nonstandard_transaction.py.


Construct and broadcast a nonstandard transaction with one input and two outputs, one P2PKH output and one P2SH output.


In nonstandard_bitcoin_functions.py, in the function
validate_p2sh_address(input),
are p2sh addresses all the same length? if so, run a validate_string_length() on the input.


In create_transaction.py, add a validate_txid() check. On second thought, this should actually just be a validate_hex_length() check. txids are 32-byte little-endian hashes and should be able to have any value.
- also do this for
create_nonstandard_transaction.py.


In create_transaction.py, add a validate_address() check.


In create_nonstandard_transaction.py, add a
validate_nonstandard_address()
check.
- This should check for different types of addresses and call the appropriate validation functions.


Construct and broadcast a transaction that spends two inputs from the same address.














Plan of action:
- Generate an address: the "source" address.
- Generate a second address: the "target" address.
- Look up my current receiving address on my LocalBitcoins account: the "final" address. This will be a nonstandard address.
- Transfer some bitcoin from my LocalBitcoins account to the source address. I'll call this tx0. "tx" = transaction.
- Look up recent mining fees.
- Construct a standard raw Bitcoin transaction that moves the available bitcoin from the source address to the target address. Be careful to include a mining fee. This is tx1.
- Sign the transaction using the private key of the source address and ECDSA cryptography. Before signing, the transaction must be altered slightly to get the transaction-in-signable-form.
- Embed the transaction signature into the transaction.
- Broadcast the transaction.
- Confirm that the transaction has been mined (included in a block in the Bitcoin blockchain). This verifies that the transaction was valid (i.e. in the correct format, with a mathematically valid signature).
- Look up recent mining fees.
- Construct a second nonstandard raw Bitcoin transaction that moves the available bitcoin from the target address to the final address. Be careful to include a mining fee. This is tx2.
- Sign the transaction using the private key of the target address and ECDSA cryptography. Before signing, the transaction must be altered slightly to get the transaction-in-signable-form.
- Embed the transaction signature into the transaction.
- Broadcast the transaction.
- Confirm that the transaction has been mined (included in a block in the Bitcoin blockchain). This verifies that the transaction was valid (i.e. in the correct format, with a mathematically valid signature).




My work computer: Aineko, a 2008 Macbook running Mac OS X 10.6.8 Snow Leopard.




Create a project directory:
creating_and_signing_a_standard_raw_bitcoin_transaction_2

In the project directory, create a work directory named "work".



Step 1) Generate an address: the "source" address.


Browse to the previous project Reading and verifying a standard raw bitcoin transaction, go to the Downloadable Assets section, download the following assets, and place them in the work directory.

- bjorn_edstrom_ripemd160.py
- ecdsa-0.10.tar.gz
- pypy_sha256.py


Browse to the previous project Verifying a signed deed of the GPG 1.4.10 source code, go to the Downloadable Assets section, download the following asset, and place it in the work directory.

- generate_bitcoin_address_2.py


generate_bitcoin_address_2.py
relies on the other three assets.


Open a terminal. Change directory to the work directory.






Unpack the zipped tape archive file ecdsa-0.10.tar.gz. The following command can be used to do this:






The unpacking has produced a new directory named ecdsa-0.10. ecdsa-0.10 should contain a directory named ecdsa. Copy the ecdsa directory to the work directory. The ecdsa library can now be imported from a Python script in the work directory.



Create a temporary archive directory "tmp" in the work directory.
Move:
- ecdsa-0.10
- ecdsa-0.10.tar.gz
to tmp.


Next: I'm going to rewrite
generate_bitcoin_address_2.py
into
generate_bitcoin_address_3.py
and
bitcoin_functions.py.


Use the command below to make the script
generate_bitcoin_address_3.py
executable.




[development occurs here]


Ok. It's ready. I'll use the raw bytes private key
"the_mote_in_god's_eye"
for the source address.








Source address:
- Private key (raw bytes):
"the_mote_in_god's_eye"
- Bitcoin address:
138obEZkdWaWEQ4x8ZAYw4MybHSZtX1Nam





Step 2) Generate a second address: the "target" address.


I'll use the raw bytes private key
"roadside_picnic"
for the target address.





Target address:
- Private key (raw bytes):
"roadside_picnic"
- Bitcoin address:
13xPBB175FtPbPQ84iB8KuawaVy3mHrady





Step 3) Look up my current receiving address on my LocalBitcoins account: the "final" address. This will be a nonstandard address.


Log in to my LocalBitcoins account.

My current receiving address is:
328cTqexYnQRbN5Dgs12D89sYiPPvtWVbF

This is the "final" address.

Current deposit fee for incoming transactions:
0.00015 BTC





Step 4) Transfer some bitcoin from my LocalBitcoins account to the source address. I'll call this tx0. "tx" = transaction.


How much to transfer?

Well, I need pay to two Bitcoin transaction fees + the LocalBitcoins deposit fee, with some extra so as to have room to manoeuvre.

LocalBitcoins also charges a sending fee of 0.00005 bitcoin. This will be deducted from my LocalBitcoins wallet, not from the transaction amount.


I'll look up the current transaction fee.

Browse to:
bitcoinfees.earn.com

The fees are displayed in satoshis per byte of transaction data. A satoshi is 0.00000001 bitcoin.

I'll use the term "fee rate" to indicate the cost of a transaction in satoshis / byte.

I'll use the term "fee" to indicate the cost of a transaction in satoshis.

I'll assume that I'll use a high fee rate during tx1 and tx2, in order to have slack, although I will attempt to pay a smaller one.

A fee rate of 60 satoshis / byte looks quite high.

I can roughly assume that a standard transaction with 1 input and 1 output will be about 225 bytes.

A fee will therefore be 225 bytes * 60 satoshis / byte = 225 * 60 = 13500 satoshis.

I'll do 2 transactions. 2 * 13500 = 27000 satoshis.

Add the LocalBitcoins deposit fee, which is:
0.00015 BTC = 0.00015000 BTC = 15000 satoshis

So:
27000 satoshis + 15000 satoshis = 42000 satoshis

42000 satoshis = 0.00042 bitcoin

I'll add 0.002 bitcoin extra (about $13 at current prices).

So:
0.00042 + 0.002 = 0.00242 bitcoin


In my LocalBitcoins account, transfer 0.00242 bitcoin to the source address:
138obEZkdWaWEQ4x8ZAYw4MybHSZtX1Nam


LocalBitcoins Transaction fee: 0.00005 BTC


Browse to LocalBitcoins / Wallet / Transactions.

Shows "Pending".

Wait. Refresh.

Shows txid:



Browse to:
live.blockcypher.com

Search for this txid.

It exists in the blockcypher database and has 1 confirmation.

[some time passes]

Refresh. It now has 6+ confirmations.





Step 5) Look up recent mining fees.

Browse to:
bitcoinfees.earn.com

I see that a transaction with a fee rate of 1-2 satoshis / byte will probably be mined. Estimated delay = 3-23 blocks (25-300 minutes).





Step 6) Construct a standard raw Bitcoin transaction that moves the available bitcoin from the source address to the target address. Be careful to include a mining fee. This is tx1.




Excerpts from the previous project Creating and signing a standard raw Bitcoin transaction:

This is my current working definition of the raw format of a standard Bitcoin transaction.


Standard transaction:

- version: 4 bytes (little-endian)
- input_count: (var_int)
- [for each input:]
-- previous_output_hash (aka "txid"): 32 bytes (big-endian)
-- previous_output_index: 4 bytes (little-endian)
-- script_length: (var_int)
-- scriptSig [see below]
-- sequence: 4 bytes (little-endian)
- output_count: (var_int)
- [for each output:]
-- value: 8 bytes (little-endian)
-- script_length: (var_int)
-- scriptPubKey [see below]
- block lock time: 4 bytes

scriptSig:

- PUSHDATA: 47 (approximately)
- [derived property] PUSHDATA decimal value: 71 (approximately)
- signature_data: 71 bytes (approximately)
- PUSHDATA: 41
- [derived property] PUSHDATA decimal value: 65
- public_key_data: 65 bytes ("04", 32-byte big-endian X value, 32-byte big-endian Y value)

scriptPubKey:

- OP_DUP: 76
- OP_HASH160: a9
- PUSHDATA: 14
- [derived property] PUSHDATA decimal value: 20
- public_key_hash: 20 bytes (big-endian)
- OP_EQUALVERIFY: 88
- OP_CHECKSIG: ac

[...]


- A var_int ("variable integer") value is used to store a number of variable byte length. It can be used to indicate the length in bytes of the next item, e.g. script_length, which is a number equal to the number of bytes in scriptSig. It can also be used to store a number, e.g. input_count (i.e. number of input items). A one-byte var_int has a maximum value of 0xFC (252 in decimal).

- The transaction fee is stored implicitly in a transaction as the difference between the sum of the input values and the sum of the output values.


[...]


- My working definition of a standard address is: Pay-To-Public-Key-Hash (P2PKH) where the public key was not compressed before being hashed.

- My working definition of a standard input is: an unspent output stored in a standard address.

- The inputs for a new transaction are a set of as-yet-unspent outputs from previous transactions. An input transfers value out of a previous address and an output transfers value into a new address. (It would perhaps be more accurate to say that value is "associated with" an address, rather than that the value is "contained within" it.) A transaction is a collection of inputs and outputs, formatted into a single item.

- scriptPubKey is the "destination" of the bitcoin in a transaction and contains the hash of a public key. The corresponding private key is required in order to spend this bitcoin later.

- The previous_output_hash is:
-- the hash of the previous transaction that:
--- contains the unspent output that:
---- will be used as an input in this new transaction

- The previous_output_index is the implicit index (starting at 0) of the unspent output in the list of unspent outputs in the previous block.

- The domain of the opcode PUSHDATA is 1-75 in decimal (0x01-0x4b in hex).


[...]


Gathering data in order to construct a new transaction



Transaction data that should be originally chosen when the transaction is being planned:

- [for each output:]
-- value: 8 bytes (little-endian)

Notes:
- Include at least one "change address" output. Inputs should be added as necessary to achieve the desired output values. The sum of the inputs may be greater than the sum of the desired output values, i.e. some bitcoin may be left over. Send this left-over bitcoin to the change address output. Include an additional input that is specifically for paying the as-yet-unknown transaction fee. Send the bitcoin from this input to the change address output.


Transaction data that can be known immediately:

- version: 4 bytes (little-endian) = 01 00 00 00
- [for each input:]
-- sequence: 4 bytes (little-endian) = ff ff ff ff
- block lock time: 4 bytes = 00 00 00 00
- hash type: 4 bytes (little-endian) = 01 00 00 00
- hash type: 1 byte = 01

Notes:
- I think that the version is always hardcoded as '1' for the main Bitcoin network. In 4 little-endian hex bytes, this is 01 00 00 00.
- [Hearsay] Sequence can always be set to 0xffffffff. It is a nonfunctional legacy feature that was designed to allow multiple signers to agree to update a transaction.
- [Hearsay] Block lock time can always be set to 0x00000000. It was designed to ask miners not to mine a transaction until a specified block height is reached. A block lock time of 0x00000000 specifies that the transaction can be mined at any block height after 0.
- The 4-byte hash_type is appended to the transaction before signing.
- The 1-byte hash_type is appended to the transaction signature.


Transaction data that can be derived from the destination addresses:

- output_count: (var_int)
- [for each output:]
-- script_length: (var_int)
-- scriptPubKey [see below]

Notes:
- The destination addresses must be standard addresses (i.e. single-signature Pay-To-Public-Key-Hash (P2PKH)). For other destination addresses (e.g. P2SH multi-signature), the scriptPubKey format below will be different.

scriptPubKey:

- OP_DUP: 76
- OP_HASH160: a9
- PUSHDATA: 14
- [derived property] PUSHDATA decimal value: 20
- public_key_hash: 20 bytes (big-endian)
- OP_EQUALVERIFY: 88
- OP_CHECKSIG: ac

Transaction data that can be acquired by analysing previous transactions on the blockchain:

- input_count: (var_int)
- [for each input:]
-- previous_output_hash (aka "txid"): 32 bytes (big-endian)
-- previous_output_index: 4 bytes (little-endian)
-- scriptPubKey of relevant unspent output from previous transaction

Notes:
- "scriptPubKey of relevant unspent output from previous transaction" is used when signing a transaction to make a signature for a particular input. It can be derived from the input address. However, it should also be found within the relevant unspent output in a previous transaction, and then compared against the version derived from the input address as a sanity check.


Transaction data that must be calculated after all other data has been assembled:

-- script_length: (var_int)
-- scriptSig [see below]

scriptSig:

- PUSHDATA: 46 (approximately)
- [derived property] PUSHDATA decimal value: 70 (approximately)
- signature_data: 70 bytes (approximately)
- PUSHDATA: 41
- [derived property] PUSHDATA decimal value: 65
- public_key_data: 65 bytes ("04", 32-byte big-endian X value, 32-byte big-endian Y value)

Notes:
- public_key_data in the scriptSig can be derived from the private key of the input address.
- signature_data in the scriptSig is the signature created by signing the entire transaction using the private key of one particular input address.

I'm going to define:
- previous_output_hash = big-endian 32-byte double-SHA256 hash of a previous transaction.
- txid = little-endian form of the previous_output_hash



Next: Gather data in order to construct new transaction tx1.


tx1 will transfer balance from source address to target address, minus a transaction fee.

Source address:
- Private key (raw bytes):
"the_mote_in_god's_eye"
- Bitcoin address:
138obEZkdWaWEQ4x8ZAYw4MybHSZtX1Nam
- Current balance: 0.00242 BTC

Target address:
- Private key (raw bytes):
"roadside_picnic"
- Bitcoin address:
13xPBB175FtPbPQ84iB8KuawaVy3mHrady


Transaction data that should be originally chosen when the transaction is being planned:


We have 1 output. The output value will be the single input (with a value of 0.00242 BTC = 242000 satoshis) minus the fee.

Assuming a transaction size of 225 bytes, and an initial fee rate of 1 byte / satoshi, the the transaction fee is:
225 bytes * 1 byte / satoshi = 225 satoshis.

The output value = 0.242000 - 225 satoshis = 241775 satoshis

I need to calculate the 8-byte little-endian form of 241775.


Hm. I'll write a new script:
- create_transaction.py

Its inputs will be the transaction input information as specified in the excerpt above.

Any general bitcoin-related functions needed for this script will go into
- bitcoin_functions.py


[development occurs here]


I'll also write transaction.py, which will hold classes for:
- standard transaction
- standard output
- standard input


[development occurs here]


Hm. Ok. Next, need to look at:

Transaction data that can be acquired by analysing previous transactions on the blockchain:

- input_count: (var_int)
- [for each input:]
-- previous_output_hash (aka "txid"): 32 bytes (big-endian)
-- previous_output_index: 4 bytes (little-endian)
-- scriptPubKey of relevant unspent output from previous transaction

Notes:
- "scriptPubKey of relevant unspent output from previous transaction" is used when signing a transaction to make a signature for a particular input. It can be derived from the input address. However, it should also be found within the relevant unspent output in a previous transaction, and then compared against the version derived from the input address as a sanity check.

Ok. Need to look up tx0 in a blockchain information service.

Its txid is:



This txid is the little-endian form of the previous_output_hash for this input.


Browse to:
live.blockcypher.com

Search for this txid.

Result:
live.blockcypher.com/btc/tx/e4609e0f1ca854b8b07381f32ba31adbad9713205f5a4f3f56a5a32853d47855

Search on page for source address.
138obEZkdWaWEQ4x8ZAYw4MybHSZtX1Nam

Hm. Not enough information.

Click button "API Call".

Ah. Result = a raw text dump of the transaction. Search for source address.

I find:



Hm. Well the "script" value is the scriptPubKey I need for the signable form of tx1.

I still need to know the index of this output in the transaction. This index is implicit and zero-indexed.

There are 39 outputs. Copy them into a text file. Search for the source address, then count up from there until the top output is reached.

I count 8, not including the source address's output. Starting from the top, from 0, this gives the source address' output an index of 8.


In order to get the previous_output_hash (which is big-endian), I need to reverse the byte order of the txid (which is little-endian).


- txid:






- previous_output_hash:
5578d45328a3a5563f4f5a5f201397addb1aa32bf38173b0b854a81c0f9e60e4
- previous_output_index: 8
- scriptPubKey of relevant unspent output from previous transaction:
76a914176a0e0c2b9b0e77d630712ad301b749102a304488ac

Also, input address (source address):
138obEZkdWaWEQ4x8ZAYw4MybHSZtX1Nam


Hm. I'll make a input.create() function that takes a txid as input.


[development occurs here]



Ok. Finished.



[various attempts to sign and broadcast the transaction occur here and are omitted.]









Earlier, I ran this command to make the script
create_transaction.py
executable.




Run the script.





Signed transaction:





Let's see if a blockchain information service can decode this transaction.


Browse to:
live.blockcypher.com/btc/decodetx

Paste the signed transaction into the text box named "Transaction Hex". Network is set to "Bitcoin".

Click "Decode Transaction".

Result:





Looks good. No immediate error reported.




Let's broadcast the signed transaction.


Browse to:
live.blockcypher.com/btc/pushtx


Paste the signed transaction into the text box named "Transaction Hex". Network is set to "Bitcoin".


Click "Broadcast Transaction".

Result:
Webpage loads:
live.blockcypher.com/btc/tx/745e224ccba0a033c55ea80523f207da18b903418ac1f5d293eed62c19e0334d
Details:
- Transaction Successfully Broadcst
- AMOUNT TRANSACTED: 0.00241777 BTC
- FEES: 0.00000223 BTC
- Miner Preference: LOW
- Size: 223 bytes
- Version: 1
- Relayed By: 23.20.8.168



Time: 8:11 pm


Time: 9 am (next day)


Refresh:
live.blockcypher.com/btc/tx/745e224ccba0a033c55ea80523f207da18b903418ac1f5d293eed62c19e0334d

Confirmations: 6+

Click "Advanced Details". Click "API Call".

Result:





Excellent.

Key details:
- block_hash:
0000000000000000000eecb4423435b1e52db78e8c83ae9e271978c2ef9d186e
- block_height: 544268
- confirmed: 2018-10-03 at 23:25:01 UTC
- received: 2018-10-03 at 18:55:16.947 UTC
- txid:
745e224ccba0a033c55ea80523f207da18b903418ac1f5d293eed62c19e0334d

From some reading, the "Z" suffix on Unix timestamps indicates UTC time values.

UTC = Coordinated Universal Time (identical to GMT = Greenwich Mean Time).

18:55 to 23:25 = 4.5 hours for confirmation.


Good.





I have completed steps 6-10:
- Construct a standard raw Bitcoin transaction that moves the available bitcoin from the source address to the target address. Be careful to include a mining fee. This is tx1.
- Sign the transaction using the private key of the source address and ECDSA cryptography. Before signing, the transaction must be altered slightly to get the transaction-in-signable-form.
- Embed the transaction signature into the transaction.
- Broadcast the transaction.
- Confirm that the transaction has been mined (included in a block in the Bitcoin blockchain). This verifies that the transaction was valid (i.e. in the correct format, with a mathematically valid signature).






Step 11) Look up recent mining fees.


Browse to:
bitcoinfees.earn.com

I see that a transaction with a fee rate of 1-2 satoshis / byte will probably be mined. Estimated delay = 2-11 blocks (10-180 minutes).




Steps 12-16)

- Construct a second nonstandard raw Bitcoin transaction that moves the available bitcoin from the target address to the final address. Be careful to include a mining fee. This is tx2.
- Sign the transaction using the private key of the target address and ECDSA cryptography. Before signing, the transaction must be altered slightly to get the transaction-in-signable-form.
- Embed the transaction signature into the transaction.
- Broadcast the transaction.
- Confirm that the transaction has been mined (included in a block in the Bitcoin blockchain). This verifies that the transaction was valid (i.e. in the correct format, with a mathematically valid signature).





The final address
328cTqexYnQRbN5Dgs12D89sYiPPvtWVbF
begins with a '3' and is therefore a Pay-To-Script-Hash (P2SH) address.




I examined P2SH addresses in some detail in the first iteration of this project:
Creating and signing a standard raw Bitcoin transaction

Summaries of my results can be found in this previous project, in the Notes / Discovery section, in these parts:
- P2SH multi-signature addresses - security
- P2SH multi-signature addresses - results from this project



Excerpts from this previous project:

P2SH multi-signature addresses - results from this project



To send to a P2SH multi-signature address, I don't need to worry about how to spend from it. I just need to construct an outputScript from the P2SH address and use it in the transaction output.

The P2SH outputScript is:

OP_HASH160 PUSHDATA(20) [20-byte op_hash160 hash of redeemScript] OP_EQUAL

- 20 in hex is 0x14.

The address format of P2SH addresses is:

base58-encode( [one-byte version] + [20-byte hash of redeemScript] + [4-byte checksum] )

On the main network, the version byte is 0x05.


[...]


I then constructed an outputScript from the redeemScript hash.

outputScript:

- OP_HASH160: a9
- PUSHDATA: 14
- [derived property] PUSHDATA decimal value: 20
- public_key_hash: 31 6f 8d 5c 41 d8 8d 3a 0d f1 79 fc 5a d7 65 d5 7f 7f 46 67
- OP_EQUAL: 87

I was able to construct and sign a valid transaction that transferred bitcoin from the P2PKH target address to the P2SH multi-signature LocalBitcoins receiving address. This was transaction 2.

In this previous project, I also wrote a script
get_script_hash_from_p2sh_address.py
that extracts the script hash from the P2SH address.

Browse to this previous project, go to the Downloadable Digital Assets section, and download
get_script_hash_from_p2sh_address.py


I also included an example of how to use this script. Go to the Recipes For Using Various Downloadable Assets section, and find Recipe 3.

Excerpt:

Recipe 3: get_script_hash_from_p2sh_address.py

1) Set the variable

address

to contain the P2SH Bitcoin address whose script hash you wish to extract.

2) Run the script.

aineko:work stjohnpiano$ python get_script_hash_from_p2sh_address.py

address: 36CQfj2Yt54sZttJYTb5ywuS7YGEQLfzCE
result_int2 integer value lies within possible domain of integer values for: [0x05] + [20-byte script hash]
in hex: 05316f8d5c41d88d3a0df179fc5ad765d57f7f46678b3e88cf
remove checksum: 05316f8d5c41d88d3a0df179fc5ad765d57f7f4667
remove first 0x05 byte.
script hash: 316f8d5c41d88d3a0df179fc5ad765d57f7f4667

Future: Check the validity of the checksum in the P2SH address, rather than just removing it.

Using this previous work, I will write a new script
create_nonstandard_transaction.py
that will create a transaction that sends bitcoin to a P2SH address. This will be tx2.

I will also write supporting library files:
- nonstandard_transaction.py
- nonstandard_bitcoin_functions.py

For the transaction input, I need the details of the currently-unspent output stored in (associated with) the target address.

txid:


In the data (included earlier) for this txid from
live.blockcypher.com
I can see that:
- previous_output_index = 0, because a) output indices are implicit and 0-indexed and b) there is only 1 output in this transaction.
- value = 241777 (in satoshis)

I need the private key of the address for this unspent output in order to sign tx2.

I created the address
13xPBB175FtPbPQ84iB8KuawaVy3mHrady
in this project, so I know that
private_key_hex =
726f6164736964655f7069636e6963


Details for the desired output of tx2:
- address:
328cTqexYnQRbN5Dgs12D89sYiPPvtWVbF
- satoshi_amount:
241777 (the specified fee will be subtracted from this).


I'll set a fee of 225 satoshis. This will probably be slightly above 1 satoshi / byte.



[development occurs here]



Ok. Finished.






Earlier, I ran this command to make the script
create_nonstandard_transaction.py
executable.




Run the script.







Signed transaction:





Let's see if a blockchain information service can decode this transaction.


Browse to:
live.blockcypher.com/btc/decodetx

Paste the signed transaction into the text box named "Transaction Hex". Network is set to "Bitcoin".

Click "Decode Transaction".

Result:






Looks good. No immediate error reported.




Let's broadcast the signed transaction.


Browse to:
live.blockcypher.com/btc/pushtx


Paste the signed transaction into the text box named "Transaction Hex". Network is set to "Bitcoin".


Click "Broadcast Transaction".

Result:
Webpage loads:
live.blockcypher.com/btc/tx/598f47dfb44da351c8576fb975bdab3b023777bb1c1d8ba727afd528ca4d7cf2
Details:
- Transaction Successfully Broadcst
- AMOUNT TRANSACTED: 0.00241552 BTC
- FEES: 0.00000225 BTC
- Confirmations: 0/6
- Miner Preference: LOW
- Size: 221 bytes
- Version: 1
- Relayed By: 54.198.103.112



Time: 11:19 am



Note:
- tx2's size is 221 bytes, not 223 bytes as I expected. This is because the P2SH outputScript is 3 single-byte opcodes + a 20-byte redeemScriptHash i.e. 23 bytes, rather than the 25-byte P2PKH scriptPubKey.

In the code in
create_nonstandard_transaction.py,
in the estimate-transaction-size code, I used "25" as the estimated size of the p2sh outputScript.
- Change this to "23".

Final fee rate is 225 / 221 ~= 1.0181 (satoshi / byte).


Log into LocalBitcoins.
Browse to Wallet / Receive bitcoins.

An incoming transaction is listed.
- Time: 11:07 am [Note: I checked my computer clock. It is 12 minutes fast.]
- Address:
328cTqexYnQRbN5Dgs12D89sYiPPvtWVbF
- BTC: 0.00241552
- Confirmed in about: 30 minutes



Time: 12:25 pm
Refresh transaction page.
- Confirmations: 0/6


Time: 15:26 pm
Refresh transaction page.
- Confirmations: 0/6


Time: 18:17 pm
Refresh transaction page.
- Confirmations: 6+

Click Advanced Details. Then click API Call.

Result = a new page containing a raw text dump of the transaction.
api.blockcypher.com/v1/btc/main/txs/598f47dfb44da351c8576fb975bdab3b023777bb1c1d8ba727afd528ca4d7cf2?limit=50&includeHex=true





Excellent.


Key details:
- block_hash:
0000000000000000000d1f94b44eafbe52692ec790caf63a045d25e3080ba095
- block_height: 545334
- confirmed: 2018-10-11T16:00:06 UTC
- received: 2018-10-11T10:06:29.33 UTC
- txid ("hash" in the results above):
598f47dfb44da351c8576fb975bdab3b023777bb1c1d8ba727afd528ca4d7cf2

From some reading, the "Z" suffix on Unix timestamps indicates UTC time values.

UTC = Coordinated Universal Time (identical to GMT = Greenwich Mean Time).

10:06 to 16:00 ~= 6 hours for confirmation.


Good.



Log into LocalBitcoins.
Browse to Wallet / Transactions.
- Latest received transaction:
-- Datetime: 10/11/2018 17:02
-- Received BTC: 0.00226552
-- Description: Deposit to 328cTq......
-- Deposit Fee BTC: 0.00015000




I have completed steps 12-16:
- Construct a second nonstandard raw Bitcoin transaction that moves the available bitcoin from the target address to the final address. Be careful to include a mining fee. This is tx2.
- Sign the transaction using the private key of the target address and ECDSA cryptography. Before signing, the transaction must be altered slightly to get the transaction-in-signable-form.
- Embed the transaction signature into the transaction.
- Broadcast the transaction.
- Confirm that the transaction has been mined (included in a block in the Bitcoin blockchain). This verifies that the transaction was valid (i.e. in the correct format, with a mathematically valid signature).




I'll rewrite
create_transaction.py,
incorporating various improvements developed in
create_nonstandard_transaction.py.

I'll add a temporary test condition that raises an error if the transaction is not the expected value i.e. the signed standard transaction created earlier, tx1:




[development occurs here]


I've also rewritten
create_nonstandard_transaction.py
slightly, in order to handle properly (in input_data and output_data in CONTROLS).







Ok. Let's do a final run-through.











Run the script.






Signed transaction:





Let's test that this is the same as the transaction (tx1) that I broadcast earlier.






Excellent.









I'll also do a final run-through of
create_nonstandard_transaction.py.












Run the script.






Signed transaction:





Let's test that this is the same as the transaction (tx2) that I broadcast earlier.







Excellent.





And finally, a run-through of
generate_bitcoin_address_3.py,
because I have changed it and some of the functions it relies upon.









Run the script.





Address generated here:
13xPBB175FtPbPQ84iB8KuawaVy3mHrady


Confirm that the address generated here matches the address generated earlier (the target address).






Good.




Loose end: Delete temporary archive directory "tmp" in the work directory.




That's the end of this project.

















Changes from the original text:

- I have not always preserved the format of any excerpts from webpages on other sites (e.g. not preserving the original bold/italic styles, changing the list structures, not preserving hyperlinks).

- I have not always preserved the format of any computer output (e.g. from running bash commands). Examples: Setting input lines in bold text, adding/removing newlines in order to make a sequence of commands easier to read, using hyphens for lists and sublists instead of indentation, breaking wide tables into consecutive sections.