edgecase
Drums, drums in the deep.
Author: StJohn Piano
Published: 2018-11-17
Datafeed Article 72
This article has been digitally signed by Edgecase Datafeed.
2532 words - 770 lines - 20 pages





Parts


- Description
- Assets
- Notes
- Recipe
- Example




Description


This recipe describes a method of creating and signing a nonstandard Bitcoin transaction with one standard P2PKH input and one nonstandard P2SH output.

This transaction will authorise the transfer of an "unspent output" stored at a standard P2PKH address to a nonstandard P2SH address, minus a transaction fee. This unspent output contains a particular value of bitcoin and must be transferred completely in the transaction.




Assets


Asset: A library of functions for handling standard Bitcoin data types.
bitcoin_functions.py [paywalled]

Asset: An implementation of RIPEMD-160, written by Björn Edström.
bjorn_edstrom_ripemd160.py [paywalled]

Asset: A script that creates and signs a nonstandard Bitcoin transaction with one standard P2PKH input and one nonstandard P2SH output.
create_nonstandard_transaction.py [paywalled]

Asset: A Python implementation of ECDSA cryptography, written by Peter Pearson.
ecdsa-0.10.tar.gz [paywalled]

Asset: A library of functions for handling nonstandard Bitcoin data types.
nonstandard_bitcoin_functions.py [paywalled]

Asset: A library of classes for nonstandard Bitcoin transactions.
nonstandard_transaction.py [paywalled]

Asset: A Python implementation of SHA256.
pypy_sha256.py [paywalled]

Asset: A library of classes for standard Bitcoin transactions.
transaction.py [paywalled]




Notes


My working definition of a nonstandard transaction:
- It has at least one input and at least one output.
- At least one input or output address is not a standard Pay-To-Public-Key-Hash (P2PKH) address.

The code used in this recipe only supports nonstandard transactions with one standard input and one nonstandard output. A standard input spends bitcoin from a standard address. A nonstandard output sends bitcoin to a nonstandard address.

Example of a nonstandard address:
- A Pay-To-Script-Hash (P2SH) address.

Pay-To-Script-Hash (P2SH) addresses begin with the character '3'. Example:
The address
328cTqexYnQRbN5Dgs12D89sYiPPvtWVbF
begins with a '3' and is therefore a P2SH address.

Pay-To-Public-Key-Hash (P2PKH) addresses start with the character '1'. Example:
The address
1AGygbyEFYduWkkmZbbvirgS9kuBBMLJCP
begins with a '1' and is therefore a P2PKH address.

This recipe assumes that you already have a Bitcoin address and have transferred some bitcoin to it. If not, the following article describes how to generate a Bitcoin address:
Recipe for generating a Bitcoin address

This recipe was originally published in the article Creating and signing a standard raw Bitcoin transaction: Iteration #2, in the section Recipes For Using Various Downloadable Assets, in the part "Recipe 4: create_nonstandard_transaction.py". The recipe shown here has been edited for republication.

Several points concerning Bitcoin transactions:
- New transactions spend unspent-outputs-of-previous-transactions.
- The bitcoin balance of an address is the sum of the unspent outputs sent by previous transactions to this address.
- To spend bitcoin from an address, the bitcoin amount must be constructed from complete unspent outputs. Any change can be sent in a new unspent output back to the original address or another that you control. Unspent outputs cannot be broken into smaller unspent outputs prior to spending - new unspent outputs (larger or smaller) must be created by a new transaction.




Recipe


Definitions:
- input address: the address that currently contains the bitcoin that you want to transfer in this transaction.
- output address: the address to which the bitcoin will be transferred by this transaction.
- previous_output_hash: big-endian 32-byte double-SHA256 hash of a previous transaction.
- tx: "transaction".
- txid: "transaction id", the little-endian form of the previous_output_hash.


Initial conditions:
- You must have Python 2.7.x installed on your computer. These code assets were developed / tested under Python 2.7.13 running on Mac OS X 10.6.8 (Snow Leopard), and should run successfully on other versions of Python 2.7.
- You must already have a Bitcoin address and the corresponding Bitcoin private key (ideally 32 bytes long). A 32-byte private key will be 64 hex characters long. When used as a control value in this recipe, it must be a 64-character string that consists only of hex characters (the ten digit characters "0123456789" and the lower-case alphabetical characters "abcdef").
- You will need 32 bytes of entropy (per input, but in this recipe there is only 1 input). The following article describes one way to generate this entropy: Recipe for generating entropy bytes using dice. Every ECDSA signature you ever make should use a different random value. Ideally this random value should be 32 bytes long. 1 byte is represented by 2 hex characters, so a 32-byte random value will be 64 hex characters long. When used in this recipe, the random value must be a 64-character string that consists only of hex characters (the ten digit characters "0123456789" and the lower-case alphabetical characters "abcdef").


1) Gather the following pieces of information:

- The txid of the previous transaction that transferred the unspent output to the input address.

- The index of this unspent output in the previous transaction (the "previous_output_index"). This index is implicit and starts at zero, and can be found by examining the list of outputs in the previous transaction, and looking for the particular unspent output that you wish to transfer.

- The private key (in hex bytes) of the input address. The input address must be a standard P2PKH address.

- The bitcoin amount (or satoshi amount) contained within the unspent output. Let's call this the "input amount".

- A 32-byte random value (1 per input), as described in the initial conditions. The transaction will be signed once using each input and each signature requires 32 bytes of entropy.


2) Choose the following pieces of information:

- The output address. The output address must be a nonstandard P2SH address.

- The bitcoin amount (or satoshi amount) that you wish to transfer to the output address. Let's call this the "output amount".

Note: This should be equal to the input amount (the code does not have the ability to send change). Any unassigned value will still be sent to the single output address (which in this recipe is the same as the change address).

- The change address. Any value not assigned to an output address will be assigned to the change address. The fee will be subtracted from the final value assigned to the change address.

Note: This must be the same as the output address. Since, in this recipe, the change address is the same as the only output address, the fee will always be subtracted from the value assigned to the single output address.

- The fee (in satoshi) or fee_rate (in satoshi / byte). The fee must be an integer e.g.
"225"
. The fee_rate must be an integer e.g.
"1"
or a float e.g.
"1.2"
.

Note: Before choosing a fee or fee rate, you should look up the range of fee rates for transactions that are currently being mined.


3) Create a work directory.


4) Browse to the Assets part of this article and download all the linked assets. List of assets:
- bitcoin_functions.py
- bjorn_edstrom_ripemd160.py
- create_nonstandard_transaction.py
- ecdsa-0.10.tar.gz
- nonstandard_bitcoin_functions.py
- nonstandard_transaction.py
- pypy_sha256.py
- transaction.py


5) Move these assets into the work directory.


6) Unpack the zipped tape archive file ecdsa-0.10.tar.gz. This can be done by opening a terminal, changing directory to the work directory, and running the following command:
tar -zxvf ecdsa-0.10.tar.gz

This unpacking should produce a new directory named "ecdsa-0.10" in the work directory. The directory "ecdsa-0.10" should contain a directory named "ecdsa". Copy the "ecdsa" directory into the work directory.


7) Open the file create_nonstandard_transaction.py in a text editor. Scroll to lines 41-65, which should be the section of text that lies between
##### START CONTROLS

and
##### END CONTROLS


Note: In create_nonstandard_transaction.py, all control values are strings (i.e. are contained within quotation marks).


8) Set the variable
random_value
to be the 32-byte random value.


9) Set the variable
random_value_type
to
"hex_bytes"
.


10) The variable
input_data
is a dictionary that contains several other variables.

- Set the value of the variable
input_data["txid"]
to be the txid.

- Set the value of the variable
input_data["previous_output_index"]
to be the index of the unspent output.

- Set the value of the variable
input_data["private_key_hex"]
to be the private key.

- If the input amount is in satoshi, set the value of the variable
input_data["satoshi_amount"]
to be the input amount.

- If the input amount is in bitcoin, set the value of the variable
input_data["bitcoin_amount"]
to be the input amount.

Note: Currently, if both of the variables
input_data["satoshi_amount"]
and
input_data["bitcoin_amount"]
are set, then
input_data["bitcoin_amount"]
will take precedence. You can comment out the unused variable of the pair by placing a number sign ('#') at the start of the relevant line.

- Set the value of the variable
input_data["input_type"]
to be
"p2pkh"
.


11) The variable
output_data
is a dictionary that contains several other variables.

- Set the value of the variable
output_data["address"]
to be the output address.

- If the output amount is in satoshi, set the value of the variable
output_data["satoshi_amount"]
to be the output amount.

- If the output amount is in bitcoin, set the value of the variable
output_data["bitcoin_amount"]
to be the output amount.

Note: Currently, if both of the variables
output_data["satoshi_amount"]
and
output_data["bitcoin_amount"]
are set,
output_data["bitcoin_amount"]
will take precedence. You can comment out the unused variable of the pair by placing a number sign ('#') at the start of the relevant line.

- Set the value of the variable
output_data["output_type"]
to be
"p2sh"
.


12) Set the variable
change_address
to be the change address.


13) If you have chosen to set the fee as an absolute number of satoshi, set the variable
fee
to be the fee amount and set the variable
fee_type
to be
"fee"
.


14) If you have chosen to set the fee as a fee rate (in satoshi / byte), set the variable
fee_rate
to be the fee rate and set the variable
fee_type
to be
"fee_rate"
.


15) Open a terminal and change directory to the work directory.


16) Run the following command:
python create_nonstandard_transaction.py


At the end, the output should contain the signed transaction as a hex byte sequence without spaces.




Example


I'm going to attempt to replicate tx1 from the article Using a transaction to validate a Bitcoin address


In the linked article, I can see the information gathered / chosen in order to create tx1:

Test address:
- Private key (hex bytes):
a26e15954d2dafcee70eeaaa084eab8a4c1a30b0f71a42be4d8da20123bff121
- Bitcoin address:
1AGygbyEFYduWkkmZbbvirgS9kuBBMLJCP

Previous transaction containing the unspent output that I wish to spend:
- txid:
bf7a16ef4bf8763b7bf7f02b03155c9a2f9b997d19e64a770bf3f7d3f24cc48b
- previous_output_index:
0

input amount (= value of unspent output (satoshi) sent to the input address):
78989

Output address / final address:
3QpwGejV9Dyai72HDhXZVkYBWioyuLVjti

output amount (= input amount):
78989 (satoshi)

fee rate (satoshi / byte):
1

The change address is:
3QpwGejV9Dyai72HDhXZVkYBWioyuLVjti


Random value:
258f67ebadfc5c460f0f939ba600d49c3eef672241d21f164897d19fb6fe64c9


Note: Both the private key and the random value are 32 bytes long.



Let's put these gathered / chosen values into the order described within the recipe:


Gathered:

- txid:
bf7a16ef4bf8763b7bf7f02b03155c9a2f9b997d19e64a770bf3f7d3f24cc48b

- previous_output_index:
0

- private key (hex bytes):
a26e15954d2dafcee70eeaaa084eab8a4c1a30b0f71a42be4d8da20123bff121

- input amount (satoshi):
78989

- 32-byte random value:
258f67ebadfc5c460f0f939ba600d49c3eef672241d21f164897d19fb6fe64c9


Chosen:

- output address:
3QpwGejV9Dyai72HDhXZVkYBWioyuLVjti

- output amount (satoshi):
78989

- change address:
3QpwGejV9Dyai72HDhXZVkYBWioyuLVjti

- fee_rate (satoshi / byte):
1




Follow the recipe.

Open the file create_nonstandard_transaction.py in a text editor. Scroll to lines 41-65, which is the section of text that lies between
##### START CONTROLS

and
##### END CONTROLS


As shown in the recipe, set the relevant variables to contain the gathered / chosen control values.

Result:

create_nonstandard_transaction.py

python 2.7.13
	##### START CONTROLS
	random_value = "258f67ebadfc5c460f0f939ba600d49c3eef672241d21f164897d19fb6fe64c9"
	# random_value must be between 1 and 32 bytes. If random_value_type is "raw_bytes", then random_value must be between 1 and 32 ASCII characters. If random_value_type is "hex_bytes", then random_value must be an even number of hex characters and between 2 and 64 hex characters. 
	# Note: Every ECDSA signature (one for each input in a transaction) requires new random entropy. 
	# random_value_type options: ["raw_bytes", "hex_bytes"]
	random_value_type = "hex_bytes"
	input_data = {
		"txid": "bf7a16ef4bf8763b7bf7f02b03155c9a2f9b997d19e64a770bf3f7d3f24cc48b",
		"previous_output_index": "0",
		"private_key_hex": "a26e15954d2dafcee70eeaaa084eab8a4c1a30b0f71a42be4d8da20123bff121",
		"satoshi_amount": "78989", 
		#"bitcoin_amount": "0.00241777", 
		"input_type": "p2pkh", 
		}
	output_data = {
		"address": "3QpwGejV9Dyai72HDhXZVkYBWioyuLVjti",
		"satoshi_amount": "78989",
		#"bitcoin_amount": "0.00241",
		"output_type": "p2sh", 
		}
	change_address = "3QpwGejV9Dyai72HDhXZVkYBWioyuLVjti" 
	# note: the fee will be subtracted from the amount that is being sent to the change address.
	fee = "225" # satoshi
	fee_rate = "1" # satoshi / byte
	# fee_type options: ["fee", "fee_rate"]
	fee_type = "fee_rate"
	##### END CONTROLS




Now run the script. This is a demonstration of creating a signed transaction that authorises a specified transfer of bitcoin.


aineko:work stjohnpiano$ python create_nonstandard_transaction.py


### START CREATION OF NONSTANDARD BITCOIN TRANSACTION

- Fee type: fee_rate
- Fee rate: 1.0 (satoshi / byte)
- Number of inputs (i.e. as-yet-unspent outputs): 1
- Number of outputs: 1
- Change address: 3QpwGejV9Dyai72HDhXZVkYBWioyuLVjti
- Amount to be sent to the change address: 0.00078989
- Input addresses, with total-value-to-be-sent:
-- 1AGygbyEFYduWkkmZbbvirgS9kuBBMLJCP: 0.00078989
- Output addresses, with total-value-to-be-received:
-- 3QpwGejV9Dyai72HDhXZVkYBWioyuLVjti: 0.00078989
- Total value of all inputs: 0.00078989
- Total value of all outputs: 0.00078989
- Total value of all inputs exactly matches total value of all outputs.
- Estimated transaction size: 221 bytes
- Fee rate: 1.0 (satoshi / byte)
- Calculate 221 * 1.0 and round up to nearest satoshi.
- Final fee: 221 (satoshi)
- Final fee rate (using estimated transaction size): 1.0000 (satoshi per byte)
- Fee subtracted from amount to be sent to change address.
- New amount to be sent to change address: 78768 (satoshi)

Input 0:
Input (without signature):
- previous_output_hash: 8bc44cf2d3f7f30b774ae6197d999b2f9a5c15032bf0f77b3b76f84bef167abf
- previous_output_index: 00000000
- sequence: ffffffff
- private_key_hex: a26e15954d2dafcee70eeaaa084eab8a4c1a30b0f71a42be4d8da20123bff121
- public_key_hex: 049d8a4c3947fb596c2664218f279be9ba2c0421ca006888243877cdb0c0146202a2a2af299928cc634d58976db32023facf269c1fb7069d9af25c54c07e85eb8e
- script_length_scriptPubKey: 19
- scriptPubKey: 76a91465beb01aab71c9da76f38a383c65e3693dff54cc88ac
- script_length: None
- scriptSig: None
- address: 1AGygbyEFYduWkkmZbbvirgS9kuBBMLJCP
- previous_output_index_int: 0
- txid: bf7a16ef4bf8763b7bf7f02b03155c9a2f9b997d19e64a770bf3f7d3f24cc48b

Output 0:
Output:
- value: b033010000000000
- script_length: 17
- script: a914fdca5619962026e5d80a84d3e25f5cea931ce38587
- address: 3QpwGejV9Dyai72HDhXZVkYBWioyuLVjti
- bitcoin_amount: 0.00078768
- satoshi_amount: 78768
- redeem_script_hash_hex: fdca5619962026e5d80a84d3e25f5cea931ce385

Nonstandard Transaction (unsigned form):
- version: 01000000
- input_count: 01
- Input:
-- previous_output_hash: 8bc44cf2d3f7f30b774ae6197d999b2f9a5c15032bf0f77b3b76f84bef167abf
-- previous_output_index: 00000000
-- script_length: None
-- scriptSig: None
-- sequence: ffffffff
- output_count: 01
- Output [P2SH]:
-- value: b033010000000000
-- script_length: 17
-- script: a914fdca5619962026e5d80a84d3e25f5cea931ce38587
- block_lock_time: 00000000

Nonstandard Transaction (signable form):
- version: 01000000
- input_count: 01
- Input [to be used to sign this signable form]:
-- previous_output_hash: 8bc44cf2d3f7f30b774ae6197d999b2f9a5c15032bf0f77b3b76f84bef167abf
-- previous_output_index: 00000000
-- script_length_scriptPubKey: 19
-- scriptPubKey: 76a91465beb01aab71c9da76f38a383c65e3693dff54cc88ac
-- sequence: ffffffff
- output_count: 01
- Output [P2SH]:
-- value: b033010000000000
-- script_length: 17
-- script: a914fdca5619962026e5d80a84d3e25f5cea931ce38587
- block_lock_time: 00000000
- hash_type_4_byte: 01000000

Nonstandard Transaction (signed form):
- version: 01000000
- input_count: 01
- Input:
-- previous_output_hash: 8bc44cf2d3f7f30b774ae6197d999b2f9a5c15032bf0f77b3b76f84bef167abf
-- previous_output_index: 00000000
-- script_length: 8a
-- scriptSig: 4730440220419ecae5ca010db8a2c7ec04cdebaa58a921b0102034db08d6456087c793494e022048cf345bf504905a23021b3dbd95d7a53d8786678505a3c5cd72e03ea259df230141049d8a4c3947fb596c2664218f279be9ba2c0421ca006888243877cdb0c0146202a2a2af299928cc634d58976db32023facf269c1fb7069d9af25c54c07e85eb8e
-- sequence: ffffffff
- output_count: 01
- Output [P2SH]:
-- value: b033010000000000
-- script_length: 17
-- script: a914fdca5619962026e5d80a84d3e25f5cea931ce38587
- block_lock_time: 00000000
- hash_type_4_byte: 01000000

Signed transaction:
01000000018bc44cf2d3f7f30b774ae6197d999b2f9a5c15032bf0f77b3b76f84bef167abf000000008a4730440220419ecae5ca010db8a2c7ec04cdebaa58a921b0102034db08d6456087c793494e022048cf345bf504905a23021b3dbd95d7a53d8786678505a3c5cd72e03ea259df230141049d8a4c3947fb596c2664218f279be9ba2c0421ca006888243877cdb0c0146202a2a2af299928cc634d58976db32023facf269c1fb7069d9af25c54c07e85eb8effffffff01b03301000000000017a914fdca5619962026e5d80a84d3e25f5cea931ce385870000000001000000

### END CREATION OF BITCOIN TRANSACTION




The final signed transaction is:

01000000018bc44cf2d3f7f30b774ae6197d999b2f9a5c15032bf0f77b3b76f84bef167abf000000008a4730440220419ecae5ca010db8a2c7ec04cdebaa58a921b0102034db08d6456087c793494e022048cf345bf504905a23021b3dbd95d7a53d8786678505a3c5cd72e03ea259df230141049d8a4c3947fb596c2664218f279be9ba2c0421ca006888243877cdb0c0146202a2a2af299928cc634d58976db32023facf269c1fb7069d9af25c54c07e85eb8effffffff01b03301000000000017a914fdca5619962026e5d80a84d3e25f5cea931ce385870000000001000000




I'll confirm that this is identical to the original signed tx1 that was broadcast in the previous project.


[original tx1]
aineko:work stjohnpiano$ s1="01000000018bc44cf2d3f7f30b774ae6197d999b2f9a5c15032bf0f77b3b76f84bef167abf000000008a4730440220419ecae5ca010db8a2c7ec04cdebaa58a921b0102034db08d6456087c793494e022048cf345bf504905a23021b3dbd95d7a53d8786678505a3c5cd72e03ea259df230141049d8a4c3947fb596c2664218f279be9ba2c0421ca006888243877cdb0c0146202a2a2af299928cc634d58976db32023facf269c1fb7069d9af25c54c07e85eb8effffffff01b03301000000000017a914fdca5619962026e5d80a84d3e25f5cea931ce385870000000001000000"


[newly-calculated tx1]
aineko:work stjohnpiano$ s2="01000000018bc44cf2d3f7f30b774ae6197d999b2f9a5c15032bf0f77b3b76f84bef167abf000000008a4730440220419ecae5ca010db8a2c7ec04cdebaa58a921b0102034db08d6456087c793494e022048cf345bf504905a23021b3dbd95d7a53d8786678505a3c5cd72e03ea259df230141049d8a4c3947fb596c2664218f279be9ba2c0421ca006888243877cdb0c0146202a2a2af299928cc634d58976db32023facf269c1fb7069d9af25c54c07e85eb8effffffff01b03301000000000017a914fdca5619962026e5d80a84d3e25f5cea931ce385870000000001000000"


aineko:work stjohnpiano$ [[ "$s1" = "$s2" ]] && echo equal || echo not-equal

equal



Excellent. They're identical. I've followed the recipe, using the same inputs, and have re-created the signed transaction from the previous project.