stjohn_piano 2020-10-30 yes BRIEF SUMMARY I developed a tool for extracting entropy from dice roll results. It aggregates rolls, using various group sizes, to achieve a high extraction efficiency. CONTENTS - Brief Summary - Summary - Downloadable Assets - How To Use The Tools - Project Log SUMMARY I developed a tool for extracting entropy from dice roll results. It aggregates rolls, using various group sizes, to achieve a high extraction efficiency. To produce a Bitcoin private key, i.e. 32 bytes (256 bits) of entropy, it requires an estimated 103 dice rolls, from which it will extract entropy at ~97% efficiency. Please note that the required number of rolls and the efficiency are only mathematical estimates, not guarantees. For small numbers of rolls, less aggregation is possible, so the efficiency is lower. For 1 bit, the efficiency is ~52%. For 1 byte, ~66%. For 2 bytes, ~80%. For 8 bytes, ~93%. Please see the Downloadable Assets section for the tool and a corresponding test suite, and the How To Use The Tool section for instructions and examples. DOWNLOADABLE ASSETS Assets of this article: List: - entropy_from_dice.py - test_entropy_from_dice.py Asset: A tool that extracts entropy from dice roll results. Python 2.7.12. asset entropy_from_dice.py entropy_from_dice.py deb813f85c19367e9f47ca865837f97385bfe53c3e19b76221df9e9a16d4b712 Asset: A test suite for the extraction tool. Pytest 4.6.11. asset test_entropy_from_dice.py test_entropy_from_dice.py b0f71adf835480eae6abe0bb92c9e3b875ba092afddf2f25ffeec4edbfa0ae9c HOW TO USE THE TOOLS Help: python entropy_from_dice.py --help Main: python entropy_from_dice.py -f dice_rolls.txt Note: By default, the tool looks for a file named 'dice_rolls.txt' in its local directory, so in the above command the filepath argument is optional. python entropy_from_dice.py --diceRollsString 664532 --outputType=bits --entropyAmount=2 --keepExtraEntropy Estimate: python entropy_from_dice.py -n 32 --estimate --debug Note: I prefer to increase an estimation result by about 5-10%, so that I can be fairly sure that the desired amount of entropy will be produced on the first run. Test: pytest -q Note: The test suite file needs to be in the same directory as the tool file. Examples of use: stjohn@judgement:work$ python entropy_from_dice.py -f dice_rolls.txt 089f7ffa2076aeb36d09a0d095296116cf6e3a93a1fcce0ca31e7628c43001d8 [64 hex characters = 32 bytes] stjohn@judgement:work$ echo -n '089f7ffa2076aeb36d09a0d095296116cf6e3a93a1fcce0ca31e7628c43001d8' | wc -c 64 [By default, the tool looks for a file named 'dice_rolls.txt' in its local directory.] stjohn@judgement:work$ python entropy_from_dice.py 089f7ffa2076aeb36d09a0d095296116cf6e3a93a1fcce0ca31e7628c43001d8 stjohn@judgement:work$ python entropy_from_dice.py --diceRollsString 664532 --outputType=bits --entropyAmount=2 --keepExtraEntropy 1011001101 stjohn@judgement:work$ python entropy_from_dice.py --diceRollsString 4532123123123 --outputType=bits --entropyAmount=0 --keepExtraEntropy --logLevel=info - diceRollsString: 4532123123123666 - Number of dice rolls: 13 - Extracted bits: '101000010011000001110010010000110' - Number of extracted bits: 33 - Extraction efficiency: 98.20% 101000010011000001110010010000110 stjohn@judgement:work$ python entropy_from_dice.py -n 32 --estimate 103 stjohn@judgement:work$ python entropy_from_dice.py -n 32 --estimate --debug DEBUG [263: estimateRequiredDiceRolls] Estimating required dice rolls for 256 bits of entropy: DEBUG [288: estimateRequiredDiceRolls] dice group size 53 (137 bits), 1 groups, 137 bits handled, 119 bits to go. DEBUG [288: estimateRequiredDiceRolls] dice group size 53 (137 bits), 0 groups, 0 bits handled, 119 bits to go. DEBUG [288: estimateRequiredDiceRolls] dice group size 12 (31 bits), 3 groups, 93 bits handled, 26 bits to go. DEBUG [288: estimateRequiredDiceRolls] dice group size 12 (31 bits), 0 groups, 0 bits handled, 26 bits to go. DEBUG [288: estimateRequiredDiceRolls] dice group size 7 (18 bits), 1 groups, 18 bits handled, 8 bits to go. DEBUG [288: estimateRequiredDiceRolls] dice group size 7 (18 bits), 0 groups, 0 bits handled, 8 bits to go. DEBUG [288: estimateRequiredDiceRolls] dice group size 2 (5 bits), 1 groups, 5 bits handled, 3 bits to go. DEBUG [288: estimateRequiredDiceRolls] dice group size 2 (5 bits), 0 groups, 0 bits handled, 3 bits to go. DEBUG [288: estimateRequiredDiceRolls] dice group size 1 (2 bits), 1 groups, 2 bits handled, 1 bits to go. DEBUG [279: estimateRequiredDiceRolls] No smaller group sizes to use. Add 1 more group of the current size. DEBUG [288: estimateRequiredDiceRolls] dice group size 1 (2 bits), 1 groups, 1 bits handled, 0 bits to go. INFO [126: main] Estimated dice rolls required for 256 bits of entropy (with efficiency=0.9716): 103 stjohn@judgement:work$ python entropy_from_dice.py --diceRollsString 43 --outputType=bits --entropyAmount=0 --keepExtraEntropy --debug INFO [189: processDiceRolls] diceRollsString: 43 INFO [191: processDiceRolls] Number of dice rolls: 2 DEBUG [193: processDiceRolls] Processing dice rolls: DEBUG [331: convertDiceRollsToBitsUsingLargestPossibleGroups] diceRollsString: 43 DEBUG [332: convertDiceRollsToBitsUsingLargestPossibleGroups] Converting diceRollsString to bits using groupSizeList [53, 12, 7, 2, 1] DEBUG [339: convertDiceRollsToBitsUsingLargestPossibleGroups] Can't use groupSize=53 - it's larger than the remaining number of dice rolls (2). DEBUG [339: convertDiceRollsToBitsUsingLargestPossibleGroups] Can't use groupSize=12 - it's larger than the remaining number of dice rolls (2). DEBUG [339: convertDiceRollsToBitsUsingLargestPossibleGroups] Can't use groupSize=7 - it's larger than the remaining number of dice rolls (2). DEBUG [346: convertDiceRollsToBitsUsingLargestPossibleGroups] Number of dice rolls remaining = 2. Using groupSize=2. Converting dice rolls section '43' to bits. DEBUG [387: convertDiceRollsToBits] diceRollsString: 43 DEBUG [388: convertDiceRollsToBits] len(diceRollsString)=2, groupSize=2, a=6**groupSize=6**2=36 DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=36. Result: 2**5 = 32. DEBUG [391: convertDiceRollsToBits] For a=6**groupSize=36, nextLowestIntegerPowerOf2 = 32 (=2**5) DEBUG [399: convertDiceRollsToBits] maxUsableValue = powerOf2 - 1 = 32-1 = 31 DEBUG [403: convertDiceRollsToBits] maxDiceRollValue (maxUsableValue converted back into a dice roll value): 62 DEBUG [423: convertDiceRollsToBits] - rolls section '43' ('32' in base6), value=20, bits='10100' DEBUG [424: convertDiceRollsToBits] 0 groups of 2 dice roll values ignored because they were greater than the maxDiceRollValue 62 (51 in base6, 31 in base10). DEBUG [425: convertDiceRollsToBits] Result: bits = '10100' INFO [195: processDiceRolls] Extracted bits: '10100' INFO [197: processDiceRolls] Number of extracted bits: 5 INFO [200: processDiceRolls] Extraction efficiency: 96.71% 10100 stjohn@judgement:work$ pytest test_entropy_from_dice.py -q ....................................................................... [ 78%] ................... [100%] 90 passed in 0.24 seconds PROJECT LOG In this project, I'm going to work on extracting entropy bytes from dice roll results. I'll aggregate the dice rolls to increase the efficiency of the extraction process. In the previous project article Generating_entropy_with_dice edgecase 64 Generating entropy with dice , the entropy extraction efficiency was 66%. 1 dice roll has 6 possible outcomes: 1,2,3,4,5,6. A single roll result from 1 dice is a number in base 6 (with a 1-indexed numeral system). Let's calculate the Shannon entropy of a dice. Entropy(S) = - SUM_i (p_i log_2(p_i)) For p_i = 6: Entropy(S) = - SUM_i=1_6 ((1/6) log_2(1/6)) = - log_2(1/6) = log_2(6) ~= 2.5850 So: 1 dice roll has 2.5850 bits of entropy, as a theoretical maximum. One way to think about this is that it takes ~2.5850 bits to transmit the dice roll result over a communication channel that can only handle binary data. 2 dice rolls have 2 * 2.5850 = 5.1700 bits of entropy. Also: 2 dice rolls have 6 * 6 = 36 possible results. Table of possible results: (1,1) (1,2) (1,3) (1,4) (1,5) (1,6) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (3,4) (3,5) (3,6) (4,1) (4,2) (4,3) (4,4) (4,5) (4,6) (5,1) (5,2) (5,3) (5,4) (5,5) (5,6) (6,1) (6,2) (6,3) (6,4) (6,5) (6,6) 32 is an integer power of 2. 32 = 2 ** 5 So the base 10 numbers in [0,31] can be mapped exactly to the base 2 numbers in [0, 11111]. 36 - 32 = 4. I won't attempt to extract the entropy in the last 4 possible results. I'll just recurse i.e. move on to the next dice roll value. 2^5 = 5 bits. So, in 32/36 cases of rolling 2 dice, I'll be able to extract 5 bits of entropy. In 4/36 cases: 0 bits of entropy. So, on average: 4/36 * 0 + 32/36 * 5 ~= 4.4444 bits of entropy per 2 dice rolls. This has an efficiency of 4.4444 / 5.1700 ~= 0.8600 = 86.00%. I can generalise this efficiency calculation for different numbers of aggregated dice. Note: In this approach, I've chosen to always multiply the first fraction in the equation by 0, so it can be ignored. Algorithm: let b = bits per dice roll = log_2(6) ~= 2.5850. let n = number of aggregated dice (dice group size). let a = 6^n = size of possibility space of dice group outcome. let m = greatest integer power of 2 that is less than or equal to a. let x = integer exponent of m. let efficiency = ef = ((m/a) * x) / (n * b) = (m * x) / (a * n * b) Let's test this algorithm with n=1. b = 2.5850 n = 1 a = 6^n = 6^1 = 6 m = 4 (= 2^2) x = 2 ef = (m * x) / (a * n * b) = (4 * 2) / (6 * 1 * 2.5850) = 8 / (6 * 2.5850) ~= 0.5158 ~= 51.58% Hm. It looks like my original calculation in the previous project was skewed. I simply ignored dice roll results in [5,6] and assumed that, because I was processing results in [1,2,3,4] that comprised 66% of the possibility space, I was extracting 66% of the available entropy. I extracted 1 base16 hex byte per 2 dice rolls, i.e. 4 bits per 2 dice rolls, or 2 bits per dice roll. If 2 bits of entropy was the theoretical maximum entropy available in each dice roll, then, yes, the efficiency would have 66%. However, there is an additional fractional part of the entropy per dice roll, 2.5850 - 2 = 0.5850, which I didn't take account of. So, the actual efficiency of my previous approach was 51.58%. From earlier, the efficiency of a new process that aggregated the dice properly into groups of 2 would be 86.00%, which is a significant gain. Let's test this algorithm with n=2, to confirm that the result is as expected. b = 2.5850 n = 2 a = 6^n = 6^2 = 36 m = 32 (= 2^5) x = 5 ef = (m * x) / (a * n * b) = (32 * 5) / (36 * 2 * 2.5850) = 160 / (72 * 2.5850) ~= 0.8600 = 86.00% Good. This matches the previous manually-calculated result. Let's automate this calculation, and perform it for more numbers. calc_ef.py python_2.7.12 yes def main(): results = {} b = 2.5850 for n in xrange(1,101): a = 6**n m, x = nextLowestIntegerPowerOf2(a) ef = (m * x) / (a * n * b) ef *= 100 print "{n} dice aggregated: efficiency = {ef:.2f}%".format(n=n, ef=ef) results[n] = ef max_n = max(results, key=lambda k: results[k]) print "- Greatest efficiency ({ef:.2f}%) found at {n} dice.".format(n=max_n, ef=results[max_n]) print "- Next: Sort by efficiency." for n in sorted(results, key=lambda k: results[k], reverse=True): print "{n} dice ({ef:.2f}%)".format(n=n, ef=results[n]) def nextLowestIntegerPowerOf2(n): i = 0 m = 0 # increment power of 2 until we surpass the given value. while m \< n: m = 2**i i += 1 # go back down one power. i -= 2 m = 2**i return m, i if __name__ == '__main__': main() stjohn@judgement:work$ python calc_ef.py 1 dice aggregated: efficiency = 51.58% 2 dice aggregated: efficiency = 85.97% 3 dice aggregated: efficiency = 53.49% 4 dice aggregated: efficiency = 76.41% 5 dice aggregated: efficiency = 48.91% 6 dice aggregated: efficiency = 67.92% 7 dice aggregated: efficiency = 93.15% 8 dice aggregated: efficiency = 60.38% 9 dice aggregated: efficiency = 82.29% 10 dice aggregated: efficiency = 53.67% 11 dice aggregated: efficiency = 72.86% 12 dice aggregated: efficiency = 98.59% 13 dice aggregated: efficiency = 64.59% 14 dice aggregated: efficiency = 87.23% 15 dice aggregated: efficiency = 57.29% 16 dice aggregated: efficiency = 77.27% 17 dice aggregated: efficiency = 50.85% 18 dice aggregated: efficiency = 68.50% 19 dice aggregated: efficiency = 92.17% 20 dice aggregated: efficiency = 60.76% 21 dice aggregated: efficiency = 81.69% 22 dice aggregated: efficiency = 53.91% 23 dice aggregated: efficiency = 72.44% 24 dice aggregated: efficiency = 97.26% 25 dice aggregated: efficiency = 64.26% 26 dice aggregated: efficiency = 86.24% 27 dice aggregated: efficiency = 57.02% 28 dice aggregated: efficiency = 76.50% 29 dice aggregated: efficiency = 50.61% 30 dice aggregated: efficiency = 67.87% 31 dice aggregated: efficiency = 90.99% 32 dice aggregated: efficiency = 60.23% 33 dice aggregated: efficiency = 80.72% 34 dice aggregated: efficiency = 53.46% 35 dice aggregated: efficiency = 71.63% 36 dice aggregated: efficiency = 95.95% 37 dice aggregated: efficiency = 63.58% 38 dice aggregated: efficiency = 85.15% 39 dice aggregated: efficiency = 56.44% 40 dice aggregated: efficiency = 75.57% 41 dice aggregated: efficiency = 50.11% 42 dice aggregated: efficiency = 67.08% 43 dice aggregated: efficiency = 89.79% 44 dice aggregated: efficiency = 59.55% 45 dice aggregated: efficiency = 79.70% 46 dice aggregated: efficiency = 52.87% 47 dice aggregated: efficiency = 70.75% 48 dice aggregated: efficiency = 94.66% 49 dice aggregated: efficiency = 62.82% 50 dice aggregated: efficiency = 84.04% 51 dice aggregated: efficiency = 55.78% 52 dice aggregated: efficiency = 74.61% 53 dice aggregated: efficiency = 99.79% 54 dice aggregated: efficiency = 66.25% 55 dice aggregated: efficiency = 88.59% 56 dice aggregated: efficiency = 58.83% 57 dice aggregated: efficiency = 78.66% 58 dice aggregated: efficiency = 52.24% 59 dice aggregated: efficiency = 69.85% 60 dice aggregated: efficiency = 93.39% 61 dice aggregated: efficiency = 62.03% 62 dice aggregated: efficiency = 82.93% 63 dice aggregated: efficiency = 55.09% 64 dice aggregated: efficiency = 73.64% 65 dice aggregated: efficiency = 98.43% 66 dice aggregated: efficiency = 65.40% 67 dice aggregated: efficiency = 87.41% 68 dice aggregated: efficiency = 58.08% 69 dice aggregated: efficiency = 77.63% 70 dice aggregated: efficiency = 51.59% 71 dice aggregated: efficiency = 68.94% 72 dice aggregated: efficiency = 92.13% 73 dice aggregated: efficiency = 61.23% 74 dice aggregated: efficiency = 81.82% 75 dice aggregated: efficiency = 54.39% 76 dice aggregated: efficiency = 72.67% 77 dice aggregated: efficiency = 97.10% 78 dice aggregated: efficiency = 64.55% 79 dice aggregated: efficiency = 86.24% 80 dice aggregated: efficiency = 57.33% 81 dice aggregated: efficiency = 76.60% 82 dice aggregated: efficiency = 50.93% 83 dice aggregated: efficiency = 68.04% 84 dice aggregated: efficiency = 90.89% 85 dice aggregated: efficiency = 60.43% 86 dice aggregated: efficiency = 80.73% 87 dice aggregated: efficiency = 53.68% 88 dice aggregated: efficiency = 71.71% 89 dice aggregated: efficiency = 95.79% 90 dice aggregated: efficiency = 63.70% 91 dice aggregated: efficiency = 85.08% 92 dice aggregated: efficiency = 56.58% 93 dice aggregated: efficiency = 75.58% 94 dice aggregated: efficiency = 50.27% 95 dice aggregated: efficiency = 67.14% 96 dice aggregated: efficiency = 89.67% 97 dice aggregated: efficiency = 59.64% 98 dice aggregated: efficiency = 79.65% 99 dice aggregated: efficiency = 52.98% 100 dice aggregated: efficiency = 70.76% - Greatest efficiency (99.79%) found at 53 dice. - Next: Sort by efficiency. 53 dice (99.79%) 12 dice (98.59%) 65 dice (98.43%) 24 dice (97.26%) 77 dice (97.10%) 36 dice (95.95%) 89 dice (95.79%) 48 dice (94.66%) 60 dice (93.39%) 7 dice (93.15%) 19 dice (92.17%) 72 dice (92.13%) 31 dice (90.99%) 84 dice (90.89%) 43 dice (89.79%) 96 dice (89.67%) 55 dice (88.59%) 67 dice (87.41%) 14 dice (87.23%) 26 dice (86.24%) 79 dice (86.24%) 2 dice (85.97%) 38 dice (85.15%) 91 dice (85.08%) 50 dice (84.04%) 62 dice (82.93%) 9 dice (82.29%) 74 dice (81.82%) 21 dice (81.69%) 86 dice (80.73%) 33 dice (80.72%) 45 dice (79.70%) 98 dice (79.65%) 57 dice (78.66%) 69 dice (77.63%) 16 dice (77.27%) 81 dice (76.60%) 28 dice (76.50%) 4 dice (76.41%) 93 dice (75.58%) 40 dice (75.57%) 52 dice (74.61%) 64 dice (73.64%) 11 dice (72.86%) 76 dice (72.67%) 23 dice (72.44%) 88 dice (71.71%) 35 dice (71.63%) 100 dice (70.76%) 47 dice (70.75%) 59 dice (69.85%) 71 dice (68.94%) 18 dice (68.50%) 83 dice (68.04%) 6 dice (67.92%) 30 dice (67.87%) 95 dice (67.14%) 42 dice (67.08%) 54 dice (66.25%) 66 dice (65.40%) 13 dice (64.59%) 78 dice (64.55%) 25 dice (64.26%) 90 dice (63.70%) 37 dice (63.58%) 49 dice (62.82%) 61 dice (62.03%) 73 dice (61.23%) 20 dice (60.76%) 85 dice (60.43%) 8 dice (60.38%) 32 dice (60.23%) 97 dice (59.64%) 44 dice (59.55%) 56 dice (58.83%) 68 dice (58.08%) 80 dice (57.33%) 15 dice (57.29%) 27 dice (57.02%) 92 dice (56.58%) 39 dice (56.44%) 51 dice (55.78%) 63 dice (55.09%) 75 dice (54.39%) 22 dice (53.91%) 87 dice (53.68%) 10 dice (53.67%) 3 dice (53.49%) 34 dice (53.46%) 99 dice (52.98%) 46 dice (52.87%) 58 dice (52.24%) 70 dice (51.59%) 1 dice (51.58%) 82 dice (50.93%) 17 dice (50.85%) 29 dice (50.61%) 94 dice (50.27%) 41 dice (50.11%) 5 dice (48.91%) Let's pick out the most pragmatic group sizes. We take the most efficient group size, which is 53, and then always pick a smaller size as we go further down the list of sizes sorted by efficiency. So: 53, 12, 7, 2, 1. 53 dice (99.79%) 12 dice (98.59%) 7 dice (93.15%) 2 dice (85.97%) 1 dice (51.58%) I'll write an implementation that uses groups of 2 dice first. Later, I'll extend it to 7 and 12 dice, and see how inconvenient it is in practice to work with groups of those sizes. Hm. Instead of refusing to use a final group of 11, could aggregate by 12 until we run out of groups of 12, then aggregate by 7, then by 2, then by 1. So we can use multiple group sizes to aggregate rolls, in order of decreasing size. Let's get some base conversion data. 6*6 = 36. I'll go up to 50. base_values.py python_2.7.12 yes def main(): print 'base10 | base6 | dice roll | base2' for i in xrange(51): value_base10 = i value_base2 = bin(i) # Remove initial '0b'. value_base2 = value_base2[2:] value_base6 = base10_to_base6(value_base10) # Increment each dice roll character by 1. dice_roll = ''.join([str(int(x)+1) for x in value_base6]) output = "{a:\<{b}} | {c:\<{d}} | {e:\<{f}} | {g:\<{h}}".format( a=value_base10, b=len('base10'), c=value_base6, d=len('base6'), e=dice_roll, f=len('dice_roll'), g=value_base2, h=len('value_base2') ) print output def base10_to_base6(v): # v = 'value' if v == 0: return '0' r = '' # r = 'result' while v \> 0: v, remainder = divmod(v, 6) # Add new base6 digits to the front of r. # - This is because on each iteration we're dividing by a higher power of 6. r = str(remainder) + r return r if __name__ == '__main__': main() stjohn@judgement:work$ python base_values.py base10 | base6 | dice roll | base2 0 | 0 | 1 | 0 1 | 1 | 2 | 1 2 | 2 | 3 | 10 3 | 3 | 4 | 11 4 | 4 | 5 | 100 5 | 5 | 6 | 101 6 | 10 | 21 | 110 7 | 11 | 22 | 111 8 | 12 | 23 | 1000 9 | 13 | 24 | 1001 10 | 14 | 25 | 1010 11 | 15 | 26 | 1011 12 | 20 | 31 | 1100 13 | 21 | 32 | 1101 14 | 22 | 33 | 1110 15 | 23 | 34 | 1111 16 | 24 | 35 | 10000 17 | 25 | 36 | 10001 18 | 30 | 41 | 10010 19 | 31 | 42 | 10011 20 | 32 | 43 | 10100 21 | 33 | 44 | 10101 22 | 34 | 45 | 10110 23 | 35 | 46 | 10111 24 | 40 | 51 | 11000 25 | 41 | 52 | 11001 26 | 42 | 53 | 11010 27 | 43 | 54 | 11011 28 | 44 | 55 | 11100 29 | 45 | 56 | 11101 30 | 50 | 61 | 11110 31 | 51 | 62 | 11111 32 | 52 | 63 | 100000 33 | 53 | 64 | 100001 34 | 54 | 65 | 100010 35 | 55 | 66 | 100011 36 | 100 | 211 | 100100 37 | 101 | 212 | 100101 38 | 102 | 213 | 100110 39 | 103 | 214 | 100111 40 | 104 | 215 | 101000 41 | 105 | 216 | 101001 42 | 110 | 221 | 101010 43 | 111 | 222 | 101011 44 | 112 | 223 | 101100 45 | 113 | 224 | 101101 46 | 114 | 225 | 101110 47 | 115 | 226 | 101111 48 | 120 | 231 | 110000 49 | 121 | 232 | 110001 50 | 122 | 233 | 110010 Reading through the output, I can see that the base conversions have worked properly. I can reuse the conversion code in other programs. Although: I'll need some code that converts dice rolls to the other bases. convert_dice_rolls_to_other_bases.py python_2.7.12 yes def main(): print 'dice roll | base6 | base10 | base2' for i in xrange(51): value_base10 = i value_base6 = base10_to_base6(value_base10) # Increment each dice roll character by 1. dice_roll = ''.join([str(int(x)+1) for x in value_base6]) # Now, convert from the dice roll value back into the various bases. value_base6_2 = ''.join([str(int(x)-1) for x in dice_roll]) value_base10_2 = int(value_base6_2, 6) value_base2_2 = bin(value_base10_2) # Remove initial '0b'. value_base2_2 = value_base2_2[2:] output = "{e:\<{f}} | {c:\<{d}} | {a:\<{b}} | {g:\<{h}}".format( a=value_base10_2, b=len('base10'), c=value_base6_2, d=len('base6'), e=dice_roll, f=len('dice_roll'), g=value_base2_2, h=len('value_base2') ) print output def base10_to_base6(v): # v = 'value' if v == 0: return '0' r = '' # r = 'result' while v \> 0: v, remainder = divmod(v, 6) # Add new base6 digits to the front of r. # - This is because on each iteration we're dividing by a higher power of 6. r = str(remainder) + r return r if __name__ == '__main__': main() stjohn@judgement:work$ python convert_dice_rolls_to_other_bases.py dice roll | base6 | base10 | base2 1 | 0 | 0 | 0 2 | 1 | 1 | 1 3 | 2 | 2 | 10 4 | 3 | 3 | 11 5 | 4 | 4 | 100 6 | 5 | 5 | 101 21 | 10 | 6 | 110 22 | 11 | 7 | 111 23 | 12 | 8 | 1000 24 | 13 | 9 | 1001 25 | 14 | 10 | 1010 26 | 15 | 11 | 1011 31 | 20 | 12 | 1100 32 | 21 | 13 | 1101 33 | 22 | 14 | 1110 34 | 23 | 15 | 1111 35 | 24 | 16 | 10000 36 | 25 | 17 | 10001 41 | 30 | 18 | 10010 42 | 31 | 19 | 10011 43 | 32 | 20 | 10100 44 | 33 | 21 | 10101 45 | 34 | 22 | 10110 46 | 35 | 23 | 10111 51 | 40 | 24 | 11000 52 | 41 | 25 | 11001 53 | 42 | 26 | 11010 54 | 43 | 27 | 11011 55 | 44 | 28 | 11100 56 | 45 | 29 | 11101 61 | 50 | 30 | 11110 62 | 51 | 31 | 11111 63 | 52 | 32 | 100000 64 | 53 | 33 | 100001 65 | 54 | 34 | 100010 66 | 55 | 35 | 100011 211 | 100 | 36 | 100100 212 | 101 | 37 | 100101 213 | 102 | 38 | 100110 214 | 103 | 39 | 100111 215 | 104 | 40 | 101000 216 | 105 | 41 | 101001 221 | 110 | 42 | 101010 222 | 111 | 43 | 101011 223 | 112 | 44 | 101100 224 | 113 | 45 | 101101 225 | 114 | 46 | 101110 226 | 115 | 47 | 101111 231 | 120 | 48 | 110000 232 | 121 | 49 | 110001 233 | 122 | 50 | 110010 Now, for each of the group sizes that we chose earlier, let's find out the maximum dice roll value that we'll process. (Because, recall, we ignore the values that don't fit into the [0, nextLowestPowerOf2] domain.) Group size list: 1,2,7,12,53 Also, find the number of bits (i.e. the length of one axis of the possibility space in base 2) for each of these group sizes. max_dice_roll_values.py python_2.7.12 yes def main(): for i in [1,2,7,12,53]: nDice = i maxDiceRoll = '6' * nDice # Convert this dice roll into a base10 value. max_base6 = ''.join([str(int(x)-1) for x in maxDiceRoll]) max_base10 = int(max_base6, 6) # Find the maxUsableValue and various details about it. maxUsableValue, exponent = nextLowestIntegerPowerOf2(max_base10) maxUsableValue -= 1 # 0-index the domain. max_base6 = base10_to_base6(maxUsableValue) maxUsableDiceRoll = ''.join([str(int(x)+1) for x in max_base6]) max_base2 = bin(maxUsableValue)[2:] output = '\\nNumber of dice: {}'.format(nDice) output += '\\n- maxUsableValue (base10): {}'.format(maxUsableValue) output += '\\n- maxUsableValue (base2): {}'.format(max_base2) output += '\\n- number of bits: {}'.format(exponent) output += '\\n- maxUsableValue (base6): {}'.format(max_base6) output += '\\n- maxUsableValue (dice roll): {}'.format(maxUsableDiceRoll) print output def nextLowestIntegerPowerOf2(n): i = 0 m = 0 # increment power of 2 until we surpass the given value. while m \< n: m = 2**i i += 1 # go back down one power. i -= 2 m = 2**i return m, i def base10_to_base6(v): # v = 'value' if v == 0: return '0' r = '' # r = 'result' while v \> 0: v, remainder = divmod(v, 6) # Add new base6 digits to the front of r. # - This is because on each iteration we're dividing by a higher power of 6. r = str(remainder) + r return r if __name__ == '__main__': main() stjohn@judgement:work$ python max_dice_roll_values.py Number of dice: 1 - maxUsableValue (base10): 3 - maxUsableValue (base2): 11 - number of bits: 2 - maxUsableValue (base6): 3 - maxUsableValue (dice roll): 4 Number of dice: 2 - maxUsableValue (base10): 31 - maxUsableValue (base2): 11111 - number of bits: 5 - maxUsableValue (base6): 51 - maxUsableValue (dice roll): 62 Number of dice: 7 - maxUsableValue (base10): 262143 - maxUsableValue (base2): 111111111111111111 - number of bits: 18 - maxUsableValue (base6): 5341343 - maxUsableValue (dice roll): 6452454 Number of dice: 12 - maxUsableValue (base10): 2147483647 - maxUsableValue (base2): 1111111111111111111111111111111 - number of bits: 31 - maxUsableValue (base6): 553032005531 - maxUsableValue (dice roll): 664143116642 Number of dice: 53 - maxUsableValue (base10): 174224571863520493293247799005065324265471 - maxUsableValue (base2): 11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 - number of bits: 137 - maxUsableValue (base6): 55531440222042340011252522140533220014350441204441251 - maxUsableValue (dice roll): 66642551333153451122363633251644331125461552315552362 So, combining this with the earlier efficiency calculations: - From 1 dice, we can extract 2 bits of entropy with 51.58% efficiency. - From a group of 2 dice, we can extract 5 bits of entropy with 85.97% efficiency. - From a group of 7 dice, we can extract 18 bits of entropy with 93.15% efficiency. - From a group of 12 dice, we can extract 31 bits of entropy with 98.59% efficiency. - From a group of 53 dice, we can extract 137 bits of entropy with 99.79% efficiency. Note: In all of these cases, if the dice roll result is greater than the maxUsableValue, we will extract 0 bits of entropy with 0% efficiency. [A lot of development work occurs here.] I've written a tool that performs the entropy extraction from dice roll results. It's called: entropy_from_dice.py My Python version: stjohn@judgement:work$ python --version Python 2.7.12 Work machine: - Name: Judgement - Windows 10 - Windows Subsystem for Linux (WSL): Ubuntu 16.04 - I'm working in the WSL Ubuntu terminal. The previous project article Generating_entropy_with_dice edgecase 64 Generating entropy with dice has an asset called dice_rolls.txt, which contains a set of 200 dice roll results. In the linked project's log, the entropy output extracted from these results is recorded: 15e7e195332b0aa8a684dc3be1f29dd04daa0a5434c11a8b9e6ad180df076ae47c62 I've tested that, with diceGroupSize=1, my new tool produces the same result. I've also done this for: - The project article Using_a_transaction_to_validate_a_Bitcoin_address edgecase 66 Using a transaction to validate a Bitcoin address , which has an asset called dice_rolls_2.txt. - The project article Storing_bitcoin_on_an_offline_Raspberry_Pi edgecase 75 Storing bitcoin on an offline Raspberry Pi , which has assets called: -- dice_rolls_3.txt. -- dice_rolls_4.txt. -- dice_rolls_5.txt. -- dice_rolls_6.txt. -- dice_rolls_7.txt. -- Exception: There was a mistake in this project, in which the entropy for dice_rolls_6.txt was extracted from dice_rolls_5.txt instead, so in this case my result didn't match the result in the project. Entropy values: - dice_rolls.txt: 15e7e195332b0aa8a684dc3be1f29dd04daa0a5434c11a8b9e6ad180df076ae47c62 - dice_rolls_2.txt: a26e15954d2dafcee70eeaaa084eab8a4c1a30b0f71a42be4d8da20123bff121258f67ebadfc5c460f0f939ba600d49c3eef672241d21f164897d19fb6fe64c9fd83 - dice_rolls_3.txt: c592e1dad5e9871fdeffb551b4544b0a1cf0378c6371d7a397ff5faf04c934ca30 - dice_rolls_4.txt: 91fbb8a38082b09e6ed984fda95866dfab95dc66b0cb6b5133dd87d07315364e279d9258 - dice_rolls_5.txt: 644dc86dd0cb4518d9d7f68db2d6e9455e34ceeb32c897df2616418c4c9bb04006 - dice_rolls_6.txt: [accidentally the same as the entropy for dice_rolls_5.txt] - my output: 340cd7ed66c4fee0a7570d28eafef25f60db58bcbf78a4dc60c6e617a42cd7e7eec3 - dice_rolls_7.txt: f78c5ebf0efac89d60aedb38c30906ad5b22ae522a3e29819248c21d59e8397f9b315219aa I've also written a pytest test suite for the new tool, called: test_entropy_from_dice.py I installed pytest during this project. Some details: stjohn@judgement:work$ pip --version pip 20.2.4 from /home/stjohn/.local/lib/python2.7/site-packages/pip (python 2.7) stjohn@judgement:work$ pip install pytest [...] Installing collected packages: atomicwrites, contextlib2, six, scandir, pathlib2, zipp, configparser, importlib-metadata, pluggy, pyparsing, packaging, more-itertools, py, attrs, backports.functools-lru-cache, wcwidth, funcsigs, pytest Successfully installed atomicwrites-1.4.0 attrs-20.2.0 backports.functools-lru-cache-1.6.1 configparser-4.0.2 contextlib2-0.6.0.post1 funcsigs-1.0.2 importlib-metadata-2.0.0 more-itertools-5.0.0 packaging-20.4 pathlib2-2.3.5 pluggy-0.13.1 py-1.9.0 pyparsing-2.4.7 pytest-4.6.11 scandir-1.10.0 six-1.15.0 wcwidth-0.2.5 zipp-1.2.0 Let's record some examples of using entropy_from_dice.py Note: If it uses a large groupSize, and the dice roll results value is greater than the maxUsableValue, it tries again with a smaller groupSize. Example: The diceRollString '64', when processed with groupSize=2, is greater than the maxUsableValue of '62', so the result is empty (''). Try again with groupSize=1. For groupSize=1, '6' is greater than than the maxUsableValue of '4', but '4' is converted to '11' in binary. So: We get '11' instead of nothing. stjohn@judgement:work$ python entropy_from_dice.py -h usage: entropy_from_dice.py [-h] [-f DICEROLLSFILEPATH] [-r DICEROLLSSTRING] [-n ENTROPYAMOUNT] [-g DICEGROUPSIZES [DICEGROUPSIZES ...]] [-o {bits,bytes}] [-l {debug,info,warning,error}] [-d] [-e] [-k] Extract entropy (as bits or bytes) from dice roll values. optional arguments: -h, --help show this help message and exit -f DICEROLLSFILEPATH, --diceRollsFilePath DICEROLLSFILEPATH path to file containing the dice rolls (default: 'dice_rolls.txt'). -r DICEROLLSSTRING, --diceRollsString DICEROLLSSTRING sequence of dice rolls (e.g. '451236'). This overrides --diceRollsFilePath. -n ENTROPYAMOUNT, --entropyAmount ENTROPYAMOUNT desired number of entropy bytes (default: '32'). If --outputType is set to 'bits', then this value will be treated as the desired number of entropy bits. -g DICEGROUPSIZES [DICEGROUPSIZES ...], --diceGroupSizes DICEGROUPSIZES [DICEGROUPSIZES ...] ordered list of integer sizes to use when aggregating the dice rolls into groups (default: '[53, 12, 7, 2, 1]'). This argument is used for testing. -o {bits,bytes}, --outputType {bits,bytes} Choose format of output (default: 'bytes'). -l {debug,info,warning,error}, --logLevel {debug,info,warning,error} Choose logging level (default: 'error'). -d, --debug Sets logLevel to 'debug'. This overrides --logLevel. -e, --estimate Estimate required number of dice rolls for the desired amount of entropy. -k, --keepExtraEntropy If there is extra entropy than the desired amount, don't remove it. Supplementary Notes: - Dice roll values must consist only of characters in the list "123456". - Whitespace characters (newlines, tabs, and spaces) are permitted in the dice roll results file, and are ignored during processing. - Dice rolls are aggregated into groups to more efficiently extract entropy. - Expected entropy per dice roll = log_2(6) ~= 2.5850 bits. - Bits per byte = 8. - Bits per 32 bytes = 32 * 8 = 256. stjohn@judgement:work$ python entropy_from_dice.py --diceRollsString 1 --outputType=bits --entropyAmount=2 00 [same, but in shorthand] stjohn@judgement:work$ python entropy_from_dice.py -r 1 --outputType=bits -n 2 00 stjohn@judgement:work$ python entropy_from_dice.py --outputType=bits -n 2 -r 2 01 stjohn@judgement:work$ python entropy_from_dice.py --outputType=bits -n 2 -r 3 10 stjohn@judgement:work$ python entropy_from_dice.py --outputType=bits -n 2 -r 4 11 stjohn@judgement:work$ python entropy_from_dice.py --outputType=bits -n 1 -r 5 Traceback (most recent call last): File "entropy_from_dice.py", line 534, in \ main() File "entropy_from_dice.py", line 136, in main entropy = processDiceRolls(args) File "entropy_from_dice.py", line 210, in processDiceRolls stop(report) File "entropy_from_dice.py", line 528, in stop raise Exception(msg) Exception: Problem: Desired amount of entropy = 1 bits, but only 0 bits were produced. Estimated new dice rolls required: 2 stjohn@judgement:work$ python entropy_from_dice.py --outputType=bits -n 0 -r 5 --keepExtraEntropy stjohn@judgement:work$ python entropy_from_dice.py --outputType=bits -n=0 --keepExtraEntropy -r 561234 111010000101111 stjohn@judgement:work$ python entropy_from_dice.py --outputType=bits -n=0 --keepExtraEntropy -r 561234 --diceGroupSizes 1 00011011 stjohn@judgement:work$ python entropy_from_dice.py --outputType=bits -n=0 --keepExtraEntropy -r 561234 --diceGroupSizes 2 111010000101111 stjohn@judgement:work$ python entropy_from_dice.py -n 0 -r 561234 -k --diceGroupSizes 2 --outputType=bytes e8 stjohn@judgement:work$ python entropy_from_dice.py -n 0 -r 561234 -k e8 [demonstrate use of --debug option] stjohn@judgement:work$ python entropy_from_dice.py -r 12341 -k -n 2 Traceback (most recent call last): File "entropy_from_dice.py", line 538, in \ main() File "entropy_from_dice.py", line 136, in main entropy = processDiceRolls(args) File "entropy_from_dice.py", line 240, in processDiceRolls stop(report) File "entropy_from_dice.py", line 532, in stop raise Exception(msg) Exception: Problem: Desired amount of entropy = 2 bytes, but only 1 bytes were produced. Estimated new dice rolls required: 4 stjohn@judgement:work$ python entropy_from_dice.py -r 12341 -k -n 2 --debug INFO [189: processDiceRolls] diceRollsString: 12341 INFO [191: processDiceRolls] Number of dice rolls: 5 DEBUG [193: processDiceRolls] Processing dice rolls: DEBUG [331: convertDiceRollsToBitsUsingLargestPossibleGroups] diceRollsString: 12341 DEBUG [332: convertDiceRollsToBitsUsingLargestPossibleGroups] Converting diceRollsString to bits using groupSizeList [53, 12, 7, 2, 1] DEBUG [339: convertDiceRollsToBitsUsingLargestPossibleGroups] Can't use groupSize=53 - it's larger than the remaining number of dice rolls (5). DEBUG [339: convertDiceRollsToBitsUsingLargestPossibleGroups] Can't use groupSize=12 - it's larger than the remaining number of dice rolls (5). DEBUG [339: convertDiceRollsToBitsUsingLargestPossibleGroups] Can't use groupSize=7 - it's larger than the remaining number of dice rolls (5). DEBUG [346: convertDiceRollsToBitsUsingLargestPossibleGroups] Number of dice rolls remaining = 5. Using groupSize=2. Converting dice rolls section '12' to bits. DEBUG [387: convertDiceRollsToBits] diceRollsString: 12 DEBUG [388: convertDiceRollsToBits] len(diceRollsString)=2, groupSize=2, a=6**groupSize=6**2=36 DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=36. Result: 2**5 = 32. DEBUG [391: convertDiceRollsToBits] For a=6**groupSize=36, nextLowestIntegerPowerOf2 = 32 (=2**5) DEBUG [399: convertDiceRollsToBits] maxUsableValue = powerOf2 - 1 = 32-1 = 31 DEBUG [403: convertDiceRollsToBits] maxDiceRollValue (maxUsableValue converted back into a dice roll value): 62 DEBUG [423: convertDiceRollsToBits] - rolls section '12' ('01' in base6), value=1, bits='00001' DEBUG [424: convertDiceRollsToBits] 0 groups of 2 dice roll values ignored because they were greater than the maxDiceRollValue 62 (51 in base6, 31 in base10). DEBUG [425: convertDiceRollsToBits] Result: bits = '00001' DEBUG [346: convertDiceRollsToBitsUsingLargestPossibleGroups] Number of dice rolls remaining = 3. Using groupSize=2. Converting dice rolls section '34' to bits. DEBUG [387: convertDiceRollsToBits] diceRollsString: 34 DEBUG [388: convertDiceRollsToBits] len(diceRollsString)=2, groupSize=2, a=6**groupSize=6**2=36 DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=36. Result: 2**5 = 32. DEBUG [391: convertDiceRollsToBits] For a=6**groupSize=36, nextLowestIntegerPowerOf2 = 32 (=2**5) DEBUG [399: convertDiceRollsToBits] maxUsableValue = powerOf2 - 1 = 32-1 = 31 DEBUG [403: convertDiceRollsToBits] maxDiceRollValue (maxUsableValue converted back into a dice roll value): 62 DEBUG [423: convertDiceRollsToBits] - rolls section '34' ('23' in base6), value=15, bits='01111' DEBUG [424: convertDiceRollsToBits] 0 groups of 2 dice roll values ignored because they were greater than the maxDiceRollValue 62 (51 in base6, 31 in base10). DEBUG [425: convertDiceRollsToBits] Result: bits = '01111' DEBUG [339: convertDiceRollsToBitsUsingLargestPossibleGroups] Can't use groupSize=2 - it's larger than the remaining number of dice rolls (1). DEBUG [346: convertDiceRollsToBitsUsingLargestPossibleGroups] Number of dice rolls remaining = 1. Using groupSize=1. Converting dice rolls section '1' to bits. DEBUG [387: convertDiceRollsToBits] diceRollsString: 1 DEBUG [388: convertDiceRollsToBits] len(diceRollsString)=1, groupSize=1, a=6**groupSize=6**1=6 DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=6. Result: 2**2 = 4. DEBUG [391: convertDiceRollsToBits] For a=6**groupSize=6, nextLowestIntegerPowerOf2 = 4 (=2**2) DEBUG [399: convertDiceRollsToBits] maxUsableValue = powerOf2 - 1 = 4-1 = 3 DEBUG [403: convertDiceRollsToBits] maxDiceRollValue (maxUsableValue converted back into a dice roll value): 4 DEBUG [423: convertDiceRollsToBits] - rolls section '1' ('0' in base6), value=0, bits='00' DEBUG [424: convertDiceRollsToBits] 0 groups of 1 dice roll values ignored because they were greater than the maxDiceRollValue 4 (3 in base6, 3 in base10). DEBUG [425: convertDiceRollsToBits] Result: bits = '00' INFO [195: processDiceRolls] Extracted bits: '000010111100' INFO [197: processDiceRolls] Number of extracted bits: 12 INFO [200: processDiceRolls] Extraction efficiency: 92.84% DEBUG [216: processDiceRolls] Converting extracted bits to bytes: DEBUG [221: processDiceRolls] Partial byte: 4 bits not used. INFO [229: processDiceRolls] Extracted bytes (base 16): '0b' INFO [231: processDiceRolls] Number of extracted bytes: 1 INFO [234: processDiceRolls] Extraction efficiency: 61.90% DEBUG [237: processDiceRolls] Problem: Desired amount of entropy = 2 bytes, but only 1 bytes were produced. DEBUG [263: estimateRequiredDiceRolls] Estimating required dice rolls for 4 bits of entropy: DEBUG [288: estimateRequiredDiceRolls] dice group size 53 (137 bits), 0 groups, 0 bits handled, 4 bits to go. DEBUG [288: estimateRequiredDiceRolls] dice group size 12 (31 bits), 0 groups, 0 bits handled, 4 bits to go. DEBUG [288: estimateRequiredDiceRolls] dice group size 7 (18 bits), 0 groups, 0 bits handled, 4 bits to go. DEBUG [288: estimateRequiredDiceRolls] dice group size 2 (5 bits), 0 groups, 0 bits handled, 4 bits to go. DEBUG [288: estimateRequiredDiceRolls] dice group size 1 (2 bits), 2 groups, 4 bits handled, 0 bits to go. DEBUG [241: processDiceRolls] Normalised efficiency: 0.5158 Traceback (most recent call last): File "entropy_from_dice.py", line 541, in \ main() File "entropy_from_dice.py", line 136, in main entropy = processDiceRolls(args) File "entropy_from_dice.py", line 243, in processDiceRolls stop(report) File "entropy_from_dice.py", line 535, in stop raise Exception(msg) Exception: Problem: Desired amount of entropy = 2 bytes, but only 1 bytes were produced. Estimated new dice rolls required: 4 [demonstrate use of the estimation capability] stjohn@judgement:work$ python entropy_from_dice.py -n 0 --estimate 0 stjohn@judgement:work$ python entropy_from_dice.py -n 0 --estimate --debug INFO [126: main] Estimated dice rolls required for 0 bits of entropy (with efficiency=1.0000): 0 stjohn@judgement:work$ python entropy_from_dice.py -n 1 --estimate --debug DEBUG [261: estimateRequiredDiceRolls] Estimating required dice rolls for 8 bits of entropy: DEBUG [286: estimateRequiredDiceRolls] dice group size 53 (137 bits), 0 groups, 0 bits handled, 8 bits to go. DEBUG [286: estimateRequiredDiceRolls] dice group size 12 (31 bits), 0 groups, 0 bits handled, 8 bits to go. DEBUG [286: estimateRequiredDiceRolls] dice group size 7 (18 bits), 0 groups, 0 bits handled, 8 bits to go. DEBUG [286: estimateRequiredDiceRolls] dice group size 2 (5 bits), 1 groups, 5 bits handled, 3 bits to go. DEBUG [286: estimateRequiredDiceRolls] dice group size 2 (5 bits), 0 groups, 0 bits handled, 3 bits to go. DEBUG [286: estimateRequiredDiceRolls] dice group size 1 (2 bits), 1 groups, 2 bits handled, 1 bits to go. DEBUG [277: estimateRequiredDiceRolls] No smaller group sizes to use. Add 1 more group of the current size. DEBUG [286: estimateRequiredDiceRolls] dice group size 1 (2 bits), 1 groups, 1 bits handled, 0 bits to go. INFO [126: main] Estimated dice rolls required for 8 bits of entropy (with efficiency=0.6632): 6 stjohn@judgement:work$ python entropy_from_dice.py -n 1 --estimate 6 stjohn@judgement:work$ python entropy_from_dice.py -n 1 --estimate --outputType=bits 2 stjohn@judgement:work$ python entropy_from_dice.py -n 32 --estimate 103 stjohn@judgement:work$ python entropy_from_dice.py -n 32 --estimate --diceGroupSizes 1 193 stjohn@judgement:work$ python entropy_from_dice.py -n 64 --estimate 202 stjohn@judgement:work$ python entropy_from_dice.py -n 64 --estimate --diceGroupSizes 1 385 stjohn@judgement:work$ python entropy_from_dice.py -n 64 --estimate --diceGroupSizes 2 1 232 stjohn@judgement:work$ python entropy_from_dice.py -n 64 --estimate --diceGroupSizes 5 2 1 Traceback (most recent call last): File "entropy_from_dice.py", line 541, in \ main() File "entropy_from_dice.py", line 112, in main raise ValueError(msg) ValueError: groupSize 5 is not in list of permitted dice group sizes: [53, 12, 7, 2, 1] stjohn@judgement:work$ python entropy_from_dice.py -n 64 --estimate --diceGroupSizes 7 2 1 215 stjohn@judgement:work$ python entropy_from_dice.py -n 64 --estimate --diceGroupSizes 12 7 2 1 203 stjohn@judgement:work$ python entropy_from_dice.py -n 64 --estimate --diceGroupSizes 53 12 7 2 1 202 stjohn@judgement:work$ python entropy_from_dice.py -n 20 --estimate 63 stjohn@judgement:work$ python entropy_from_dice.py -n 32 --estimate --debug DEBUG [263: estimateRequiredDiceRolls] Estimating required dice rolls for 256 bits of entropy: DEBUG [288: estimateRequiredDiceRolls] dice group size 53 (137 bits), 1 groups, 137 bits handled, 119 bits to go. DEBUG [288: estimateRequiredDiceRolls] dice group size 53 (137 bits), 0 groups, 0 bits handled, 119 bits to go. DEBUG [288: estimateRequiredDiceRolls] dice group size 12 (31 bits), 3 groups, 93 bits handled, 26 bits to go. DEBUG [288: estimateRequiredDiceRolls] dice group size 12 (31 bits), 0 groups, 0 bits handled, 26 bits to go. DEBUG [288: estimateRequiredDiceRolls] dice group size 7 (18 bits), 1 groups, 18 bits handled, 8 bits to go. DEBUG [288: estimateRequiredDiceRolls] dice group size 7 (18 bits), 0 groups, 0 bits handled, 8 bits to go. DEBUG [288: estimateRequiredDiceRolls] dice group size 2 (5 bits), 1 groups, 5 bits handled, 3 bits to go. DEBUG [288: estimateRequiredDiceRolls] dice group size 2 (5 bits), 0 groups, 0 bits handled, 3 bits to go. DEBUG [288: estimateRequiredDiceRolls] dice group size 1 (2 bits), 1 groups, 2 bits handled, 1 bits to go. DEBUG [279: estimateRequiredDiceRolls] No smaller group sizes to use. Add 1 more group of the current size. DEBUG [288: estimateRequiredDiceRolls] dice group size 1 (2 bits), 1 groups, 1 bits handled, 0 bits to go. INFO [126: main] Estimated dice rolls required for 256 bits of entropy (with efficiency=0.9716): 103 stjohn@judgement:work$ cat dice_rolls.txt 12224 32455 43512 63552 22164 14133 41133 33313 32336 61214 24161 64534 43126 65454 13563 25466 26462 61651 25156 45236 33631 13365 22211 42146 15162 61523 56355 65313 43624 32555 33364 26155 52361 11545 66244 15162 64235 63343 21244 12313 stjohn@judgement:work$ python entropy_from_dice.py -f dice_rolls.txt 089f7ffa2076aeb36d09a0d095296116cf6e3a93a1fcce0ca31e7628c43001d8 [Note: By default, the tool looks for a file named 'dice_rolls.txt' in its local directory.] stjohn@judgement:work$ python entropy_from_dice.py 089f7ffa2076aeb36d09a0d095296116cf6e3a93a1fcce0ca31e7628c43001d8 stjohn@judgement:work$ python entropy_from_dice.py -f dice_rolls.txt --keepExtraEntropy 089f7ffa2076aeb36d09a0d095296116cf6e3a93a1fcce0ca31e7628c43001d8c0a81aa21f5fe0948993141fff83cfad37ff44197315da78eb8277bce94fc90b stjohn@judgement:work$ python entropy_from_dice.py -f dice_rolls.txt -n 0 stjohn@judgement:work$ python entropy_from_dice.py -f dice_rolls.txt -n 1 08 stjohn@judgement:work$ python entropy_from_dice.py -f dice_rolls.txt -n 2 089f stjohn@judgement:work$ python entropy_from_dice.py -f dice_rolls.txt -n 5 089f7ffa20 stjohn@judgement:work$ python entropy_from_dice.py -f dice_rolls.txt --logLevel=info - diceRollsString: 12224324554351263552221641413341133333133233661214241616453443126654541356325466264626165125156452363363113365222114214615162615235635565313436243255533364261555236111545662441516264235633432124412313 - Number of dice rolls: 200 - Extracted bits: '000010001001111101111111111110100010000001110110101011101011001101101101000010011010000011010000100101010010100101100001000101101100111101101110001110101001001110100001111111001100111000001100101000110001111001110110001010001100010000110000000000011101100011000000101010000001101010100010000111110101111111100000100101001000100110010011000101000001111111111111100000111100111110101101001101111111111101000100000110010111001100010101110110100111100011101011100000100111011110111100111010010100111111001001000010110010' - Number of extracted bits: 516 - Extraction efficiency: 99.81% - Extracted bytes (base 16): '089f7ffa2076aeb36d09a0d095296116cf6e3a93a1fcce0ca31e7628c43001d8c0a81aa21f5fe0948993141fff83cfad37ff44197315da78eb8277bce94fc90b' - Number of extracted bytes: 64 - Extraction efficiency: 99.03% - Entropy truncated down from 128 bytes to 32 bytes. 089f7ffa2076aeb36d09a0d095296116cf6e3a93a1fcce0ca31e7628c43001d8 stjohn@judgement:work$ python entropy_from_dice.py -n 64 -e --logLevel=info - Estimated dice rolls required for 512 bits of entropy (with efficiency=0.9870): 202 stjohn@judgement:work$ python entropy_from_dice.py -f dice_rolls.txt --debug INFO [189: processDiceRolls] diceRollsString: 12224324554351263552221641413341133333133233661214241616453443126654541356325466264626165125156452363363113365222114214615162615235635565313436243255533364261555236111545662441516264235633432124412313 INFO [191: processDiceRolls] Number of dice rolls: 200 DEBUG [193: processDiceRolls] Processing dice rolls: DEBUG [331: convertDiceRollsToBitsUsingLargestPossibleGroups] diceRollsString: 12224324554351263552221641413341133333133233661214241616453443126654541356325466264626165125156452363363113365222114214615162615235635565313436243255533364261555236111545662441516264235633432124412313 DEBUG [332: convertDiceRollsToBitsUsingLargestPossibleGroups] Converting diceRollsString to bits using groupSizeList [53, 12, 7, 2, 1] DEBUG [346: convertDiceRollsToBitsUsingLargestPossibleGroups] Number of dice rolls remaining = 200. Using groupSize=53. Converting dice rolls section '12224324554351263552221641413341133333133233661214241' to bits. DEBUG [387: convertDiceRollsToBits] diceRollsString: 12224324554351263552221641413341133333133233661214241 DEBUG [388: convertDiceRollsToBits] len(diceRollsString)=53, groupSize=53, a=6**groupSize=6**53=174588755932389037098918153698611839369216 DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=174588755932389037098918153698611839369216. Result: 2**137 = 174224571863520493293247799005065324265472. DEBUG [391: convertDiceRollsToBits] For a=6**groupSize=174588755932389037098918153698611839369216, nextLowestIntegerPowerOf2 = 174224571863520493293247799005065324265472 (=2**137) DEBUG [399: convertDiceRollsToBits] maxUsableValue = powerOf2 - 1 = 174224571863520493293247799005065324265472-1 = 174224571863520493293247799005065324265471 DEBUG [403: convertDiceRollsToBits] maxDiceRollValue (maxUsableValue converted back into a dice roll value): 66642551333153451122363633251644331125461552315552362 DEBUG [423: convertDiceRollsToBits] - rolls section '12224324554351263552221641413341133333133233661214241' ('01113213443240152441110530302230022222022122550103130' in base6), value=5868541363145551599096222659694389505438, bits='00001000100111110111111111111010001000000111011010101110101100110110110100001001101000001101000010010101001010010110000100010110110011110' DEBUG [424: convertDiceRollsToBits] 0 groups of 53 dice roll values ignored because they were greater than the maxDiceRollValue 66642551333153451122363633251644331125461552315552362 (55531440222042340011252522140533220014350441204441251 in base6, 174224571863520493293247799005065324265471 in base10). DEBUG [425: convertDiceRollsToBits] Result: bits = '00001000100111110111111111111010001000000111011010101110101100110110110100001001101000001101000010010101001010010110000100010110110011110' DEBUG [346: convertDiceRollsToBitsUsingLargestPossibleGroups] Number of dice rolls remaining = 147. Using groupSize=53. Converting dice rolls section '61645344312665454135632546626462616512515645236336311' to bits. DEBUG [387: convertDiceRollsToBits] diceRollsString: 61645344312665454135632546626462616512515645236336311 DEBUG [388: convertDiceRollsToBits] len(diceRollsString)=53, groupSize=53, a=6**groupSize=6**53=174588755932389037098918153698611839369216 DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=174588755932389037098918153698611839369216. Result: 2**137 = 174224571863520493293247799005065324265472. DEBUG [391: convertDiceRollsToBits] For a=6**groupSize=174588755932389037098918153698611839369216, nextLowestIntegerPowerOf2 = 174224571863520493293247799005065324265472 (=2**137) DEBUG [399: convertDiceRollsToBits] maxUsableValue = powerOf2 - 1 = 174224571863520493293247799005065324265472-1 = 174224571863520493293247799005065324265471 DEBUG [403: convertDiceRollsToBits] maxDiceRollValue (maxUsableValue converted back into a dice roll value): 66642551333153451122363633251644331125461552315552362 DEBUG [423: convertDiceRollsToBits] - rolls section '61645344312665454135632546626462616512515645236336311' ('50534233201554343024521435515351505401404534125225200' in base6), value=150035688552776692208861034798034245059232, bits='11011100011101010010011101000011111110011001110000011001010001100011110011101100010100011000100001100000000000111011000110000001010100000' DEBUG [424: convertDiceRollsToBits] 0 groups of 53 dice roll values ignored because they were greater than the maxDiceRollValue 66642551333153451122363633251644331125461552315552362 (55531440222042340011252522140533220014350441204441251 in base6, 174224571863520493293247799005065324265471 in base10). DEBUG [425: convertDiceRollsToBits] Result: bits = '11011100011101010010011101000011111110011001110000011001010001100011110011101100010100011000100001100000000000111011000110000001010100000' DEBUG [346: convertDiceRollsToBitsUsingLargestPossibleGroups] Number of dice rolls remaining = 94. Using groupSize=53. Converting dice rolls section '33652221142146151626152356355653134362432555333642615' to bits. DEBUG [387: convertDiceRollsToBits] diceRollsString: 33652221142146151626152356355653134362432555333642615 DEBUG [388: convertDiceRollsToBits] len(diceRollsString)=53, groupSize=53, a=6**groupSize=6**53=174588755932389037098918153698611839369216 DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=174588755932389037098918153698611839369216. Result: 2**137 = 174224571863520493293247799005065324265472. DEBUG [391: convertDiceRollsToBits] For a=6**groupSize=174588755932389037098918153698611839369216, nextLowestIntegerPowerOf2 = 174224571863520493293247799005065324265472 (=2**137) DEBUG [399: convertDiceRollsToBits] maxUsableValue = powerOf2 - 1 = 174224571863520493293247799005065324265472-1 = 174224571863520493293247799005065324265471 DEBUG [403: convertDiceRollsToBits] maxDiceRollValue (maxUsableValue converted back into a dice roll value): 66642551333153451122363633251644331125461552315552362 DEBUG [423: convertDiceRollsToBits] - rolls section '33652221142146151626152356355653134362432555333642615' ('22541110031035040515041245244542023251321444222531504' in base6), value=72502715048689294800548021779702212459040, bits='01101010100010000111110101111111100000100101001000100110010011000101000001111111111111100000111100111110101101001101111111111101000100000' DEBUG [424: convertDiceRollsToBits] 0 groups of 53 dice roll values ignored because they were greater than the maxDiceRollValue 66642551333153451122363633251644331125461552315552362 (55531440222042340011252522140533220014350441204441251 in base6, 174224571863520493293247799005065324265471 in base10). DEBUG [425: convertDiceRollsToBits] Result: bits = '01101010100010000111110101111111100000100101001000100110010011000101000001111111111111100000111100111110101101001101111111111101000100000' DEBUG [339: convertDiceRollsToBitsUsingLargestPossibleGroups] Can't use groupSize=53 - it's larger than the remaining number of dice rolls (41). DEBUG [346: convertDiceRollsToBitsUsingLargestPossibleGroups] Number of dice rolls remaining = 41. Using groupSize=12. Converting dice rolls section '552361115456' to bits. DEBUG [387: convertDiceRollsToBits] diceRollsString: 552361115456 DEBUG [388: convertDiceRollsToBits] len(diceRollsString)=12, groupSize=12, a=6**groupSize=6**12=2176782336 DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=2176782336. Result: 2**31 = 2147483648. DEBUG [391: convertDiceRollsToBits] For a=6**groupSize=2176782336, nextLowestIntegerPowerOf2 = 2147483648 (=2**31) DEBUG [399: convertDiceRollsToBits] maxUsableValue = powerOf2 - 1 = 2147483648-1 = 2147483647 DEBUG [403: convertDiceRollsToBits] maxDiceRollValue (maxUsableValue converted back into a dice roll value): 664143116642 DEBUG [423: convertDiceRollsToBits] - rolls section '552361115456' ('441250004345' in base6), value=1707890537, bits='1100101110011000101011101101001' DEBUG [424: convertDiceRollsToBits] 0 groups of 12 dice roll values ignored because they were greater than the maxDiceRollValue 664143116642 (553032005531 in base6, 2147483647 in base10). DEBUG [425: convertDiceRollsToBits] Result: bits = '1100101110011000101011101101001' DEBUG [346: convertDiceRollsToBitsUsingLargestPossibleGroups] Number of dice rolls remaining = 29. Using groupSize=12. Converting dice rolls section '624415162642' to bits. DEBUG [387: convertDiceRollsToBits] diceRollsString: 624415162642 DEBUG [388: convertDiceRollsToBits] len(diceRollsString)=12, groupSize=12, a=6**groupSize=6**12=2176782336 DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=2176782336. Result: 2**31 = 2147483648. DEBUG [391: convertDiceRollsToBits] For a=6**groupSize=2176782336, nextLowestIntegerPowerOf2 = 2147483648 (=2**31) DEBUG [399: convertDiceRollsToBits] maxUsableValue = powerOf2 - 1 = 2147483648-1 = 2147483647 DEBUG [403: convertDiceRollsToBits] maxDiceRollValue (maxUsableValue converted back into a dice roll value): 664143116642 DEBUG [423: convertDiceRollsToBits] - rolls section '624415162642' ('513304051531' in base6), value=1909916911, bits='1110001110101110000010011101111' DEBUG [424: convertDiceRollsToBits] 0 groups of 12 dice roll values ignored because they were greater than the maxDiceRollValue 664143116642 (553032005531 in base6, 2147483647 in base10). DEBUG [425: convertDiceRollsToBits] Result: bits = '1110001110101110000010011101111' DEBUG [346: convertDiceRollsToBitsUsingLargestPossibleGroups] Number of dice rolls remaining = 17. Using groupSize=12. Converting dice rolls section '356334321244' to bits. DEBUG [387: convertDiceRollsToBits] diceRollsString: 356334321244 DEBUG [388: convertDiceRollsToBits] len(diceRollsString)=12, groupSize=12, a=6**groupSize=6**12=2176782336 DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=2176782336. Result: 2**31 = 2147483648. DEBUG [391: convertDiceRollsToBits] For a=6**groupSize=2176782336, nextLowestIntegerPowerOf2 = 2147483648 (=2**31) DEBUG [399: convertDiceRollsToBits] maxUsableValue = powerOf2 - 1 = 2147483648-1 = 2147483647 DEBUG [403: convertDiceRollsToBits] maxDiceRollValue (maxUsableValue converted back into a dice roll value): 664143116642 DEBUG [423: convertDiceRollsToBits] - rolls section '356334321244' ('245223210133' in base6), value=1021923273, bits='0111100111010010100111111001001' DEBUG [424: convertDiceRollsToBits] 0 groups of 12 dice roll values ignored because they were greater than the maxDiceRollValue 664143116642 (553032005531 in base6, 2147483647 in base10). DEBUG [425: convertDiceRollsToBits] Result: bits = '0111100111010010100111111001001' DEBUG [339: convertDiceRollsToBitsUsingLargestPossibleGroups] Can't use groupSize=12 - it's larger than the remaining number of dice rolls (5). DEBUG [339: convertDiceRollsToBitsUsingLargestPossibleGroups] Can't use groupSize=7 - it's larger than the remaining number of dice rolls (5). DEBUG [346: convertDiceRollsToBitsUsingLargestPossibleGroups] Number of dice rolls remaining = 5. Using groupSize=2. Converting dice rolls section '12' to bits. DEBUG [387: convertDiceRollsToBits] diceRollsString: 12 DEBUG [388: convertDiceRollsToBits] len(diceRollsString)=2, groupSize=2, a=6**groupSize=6**2=36 DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=36. Result: 2**5 = 32. DEBUG [391: convertDiceRollsToBits] For a=6**groupSize=36, nextLowestIntegerPowerOf2 = 32 (=2**5) DEBUG [399: convertDiceRollsToBits] maxUsableValue = powerOf2 - 1 = 32-1 = 31 DEBUG [403: convertDiceRollsToBits] maxDiceRollValue (maxUsableValue converted back into a dice roll value): 62 DEBUG [423: convertDiceRollsToBits] - rolls section '12' ('01' in base6), value=1, bits='00001' DEBUG [424: convertDiceRollsToBits] 0 groups of 2 dice roll values ignored because they were greater than the maxDiceRollValue 62 (51 in base6, 31 in base10). DEBUG [425: convertDiceRollsToBits] Result: bits = '00001' DEBUG [346: convertDiceRollsToBitsUsingLargestPossibleGroups] Number of dice rolls remaining = 3. Using groupSize=2. Converting dice rolls section '31' to bits. DEBUG [387: convertDiceRollsToBits] diceRollsString: 31 DEBUG [388: convertDiceRollsToBits] len(diceRollsString)=2, groupSize=2, a=6**groupSize=6**2=36 DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=36. Result: 2**5 = 32. DEBUG [391: convertDiceRollsToBits] For a=6**groupSize=36, nextLowestIntegerPowerOf2 = 32 (=2**5) DEBUG [399: convertDiceRollsToBits] maxUsableValue = powerOf2 - 1 = 32-1 = 31 DEBUG [403: convertDiceRollsToBits] maxDiceRollValue (maxUsableValue converted back into a dice roll value): 62 DEBUG [423: convertDiceRollsToBits] - rolls section '31' ('20' in base6), value=12, bits='01100' DEBUG [424: convertDiceRollsToBits] 0 groups of 2 dice roll values ignored because they were greater than the maxDiceRollValue 62 (51 in base6, 31 in base10). DEBUG [425: convertDiceRollsToBits] Result: bits = '01100' DEBUG [339: convertDiceRollsToBitsUsingLargestPossibleGroups] Can't use groupSize=2 - it's larger than the remaining number of dice rolls (1). DEBUG [346: convertDiceRollsToBitsUsingLargestPossibleGroups] Number of dice rolls remaining = 1. Using groupSize=1. Converting dice rolls section '3' to bits. DEBUG [387: convertDiceRollsToBits] diceRollsString: 3 DEBUG [388: convertDiceRollsToBits] len(diceRollsString)=1, groupSize=1, a=6**groupSize=6**1=6 DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=6. Result: 2**2 = 4. DEBUG [391: convertDiceRollsToBits] For a=6**groupSize=6, nextLowestIntegerPowerOf2 = 4 (=2**2) DEBUG [399: convertDiceRollsToBits] maxUsableValue = powerOf2 - 1 = 4-1 = 3 DEBUG [403: convertDiceRollsToBits] maxDiceRollValue (maxUsableValue converted back into a dice roll value): 4 DEBUG [423: convertDiceRollsToBits] - rolls section '3' ('2' in base6), value=2, bits='10' DEBUG [424: convertDiceRollsToBits] 0 groups of 1 dice roll values ignored because they were greater than the maxDiceRollValue 4 (3 in base6, 3 in base10). DEBUG [425: convertDiceRollsToBits] Result: bits = '10' INFO [195: processDiceRolls] Extracted bits: '000010001001111101111111111110100010000001110110101011101011001101101101000010011010000011010000100101010010100101100001000101101100111101101110001110101001001110100001111111001100111000001100101000110001111001110110001010001100010000110000000000011101100011000000101010000001101010100010000111110101111111100000100101001000100110010011000101000001111111111111100000111100111110101101001101111111111101000100000110010111001100010101110110100111100011101011100000100111011110111100111010010100111111001001000010110010' INFO [197: processDiceRolls] Number of extracted bits: 516 INFO [200: processDiceRolls] Extraction efficiency: 99.81% DEBUG [216: processDiceRolls] Converting extracted bits to bytes: DEBUG [221: processDiceRolls] Partial byte: 4 bits not used. INFO [229: processDiceRolls] Extracted bytes (base 16): '089f7ffa2076aeb36d09a0d095296116cf6e3a93a1fcce0ca31e7628c43001d8c0a81aa21f5fe0948993141fff83cfad37ff44197315da78eb8277bce94fc90b' INFO [231: processDiceRolls] Number of extracted bytes: 64 INFO [234: processDiceRolls] Extraction efficiency: 99.03% INFO [147: main] Entropy truncated down from 128 bytes to 32 bytes. 089f7ffa2076aeb36d09a0d095296116cf6e3a93a1fcce0ca31e7628c43001d8 And some examples of testing the tool. stjohn@judgement:work$ pytest test_entropy_from_dice.py ============================= test session starts ============================= platform linux2 -- Python 2.7.12, pytest-4.6.11, py-1.9.0, pluggy-0.13.1 rootdir: /mnt/c/Users/User/Desktop/stuff/articles/generating_entropy_from_dice_2/work collected 90 items test_entropy_from_dice.py ............................................. [ 50%] ............................................. [100%] ========================== 90 passed in 0.26 seconds ========================== stjohn@judgement:work$ pytest test_entropy_from_dice.py -q ....................................................................... [ 78%] ................... [100%] 90 passed in 0.24 seconds stjohn@judgement:work$ pytest test_entropy_from_dice.py::test_f1_one -s ============================= test session starts ============================= platform linux2 -- Python 2.7.12, pytest-4.6.11, py-1.9.0, pluggy-0.13.1 rootdir: /mnt/c/Users/User/Desktop/stuff/articles/generating_entropy_from_dice_2/work collected 1 item test_entropy_from_dice.py . ========================== 1 passed in 0.05 seconds =========================== stjohn@judgement:work$ pytest test_entropy_from_dice.py::test_f1_one -q . [100%] 1 passed in 0.05 seconds stjohn@judgement:work$ pytest -o log_cli=true --log-cli-level=DEBUG --log-format="%(levelname)s [%(lineno)s: %(funcName)s] %(message)s" test_entropy_from_dice.py::test_f1_one ============================= test session starts ============================= platform linux2 -- Python 2.7.12, pytest-4.6.11, py-1.9.0, pluggy-0.13.1 rootdir: /mnt/c/Users/User/Desktop/stuff/articles/generating_entropy_from_dice_2/work collected 1 item test_entropy_from_dice.py::test_f1_one -------------------------------- live log call -------------------------------- DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=1. Result: 2**0 = 1. PASSED [100%] ========================== 1 passed in 0.06 seconds =========================== stjohn@judgement:work$ pytest -o log_cli=true --log-cli-level=DEBUG --log-format="%(levelname)s [%(lineno)s: %(funcName)s] %(message)s" test_entropy_from_dice.py::test_f6_6262_to_bytes ============================= test session starts ============================= platform linux2 -- Python 2.7.12, pytest-4.6.11, py-1.9.0, pluggy-0.13.1 rootdir: /mnt/c/Users/User/Desktop/stuff/articles/generating_entropy_from_dice_2/work collected 1 item test_entropy_from_dice.py::test_f6_6262_to_bytes -------------------------------- live log call -------------------------------- INFO [189: processDiceRolls] diceRollsString: 6262 INFO [191: processDiceRolls] Number of dice rolls: 4 DEBUG [193: processDiceRolls] Processing dice rolls: DEBUG [331: convertDiceRollsToBitsUsingLargestPossibleGroups] diceRollsString: 6262 DEBUG [332: convertDiceRollsToBitsUsingLargestPossibleGroups] Converting diceRollsString to bits using groupSizeList [2, 1] DEBUG [346: convertDiceRollsToBitsUsingLargestPossibleGroups] Number of dice rolls remaining = 4. Using groupSize=2. Converting dice rolls section '62' to bits. DEBUG [387: convertDiceRollsToBits] diceRollsString: 62 DEBUG [388: convertDiceRollsToBits] len(diceRollsString)=2, groupSize=2, a=6**groupSize=6**2=36 DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=36. Result: 2**5 = 32. DEBUG [391: convertDiceRollsToBits] For a=6**groupSize=36, nextLowestIntegerPowerOf2 = 32 (=2**5) DEBUG [399: convertDiceRollsToBits] maxUsableValue = powerOf2 - 1 = 32-1 = 31 DEBUG [403: convertDiceRollsToBits] maxDiceRollValue (maxUsableValue converted back into a dice roll value): 62 DEBUG [423: convertDiceRollsToBits] - rolls section '62' ('51' in base6), value=31, bits='11111' DEBUG [424: convertDiceRollsToBits] 0 groups of 2 dice roll values ignored because they were greater than the maxDiceRollValue 62 (51 in base6, 31 in base10). DEBUG [425: convertDiceRollsToBits] Result: bits = '11111' DEBUG [346: convertDiceRollsToBitsUsingLargestPossibleGroups] Number of dice rolls remaining = 2. Using groupSize=2. Converting dice rolls section '62' to bits. DEBUG [387: convertDiceRollsToBits] diceRollsString: 62 DEBUG [388: convertDiceRollsToBits] len(diceRollsString)=2, groupSize=2, a=6**groupSize=6**2=36 DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=36. Result: 2**5 = 32. DEBUG [391: convertDiceRollsToBits] For a=6**groupSize=36, nextLowestIntegerPowerOf2 = 32 (=2**5) DEBUG [399: convertDiceRollsToBits] maxUsableValue = powerOf2 - 1 = 32-1 = 31 DEBUG [403: convertDiceRollsToBits] maxDiceRollValue (maxUsableValue converted back into a dice roll value): 62 DEBUG [423: convertDiceRollsToBits] - rolls section '62' ('51' in base6), value=31, bits='11111' DEBUG [424: convertDiceRollsToBits] 0 groups of 2 dice roll values ignored because they were greater than the maxDiceRollValue 62 (51 in base6, 31 in base10). DEBUG [425: convertDiceRollsToBits] Result: bits = '11111' INFO [195: processDiceRolls] Extracted bits: '1111111111' INFO [197: processDiceRolls] Number of extracted bits: 10 INFO [200: processDiceRolls] Extraction efficiency: 96.71% DEBUG [216: processDiceRolls] Converting extracted bits to bytes: DEBUG [221: processDiceRolls] Partial byte: 2 bits not used. INFO [229: processDiceRolls] Extracted bytes (base 16): 'ff' INFO [231: processDiceRolls] Number of extracted bytes: 1 INFO [234: processDiceRolls] Extraction efficiency: 77.37% PASSED [100%] ========================== 1 passed in 0.09 seconds =========================== [test recursion functionality] stjohn@judgement:work$ pytest -o log_cli=true --log-cli-level=DEBUG --log-format="%(levelname)s [%(lineno)s: %(funcName)s] %(message)s" test_entropy_from_dice.py::test_f4_recurse_64 ============================= test session starts ============================= platform linux2 -- Python 2.7.12, pytest-4.6.11, py-1.9.0, pluggy-0.13.1 rootdir: /mnt/c/Users/User/Desktop/stuff/articles/generating_entropy_from_dice_2/work collected 1 item test_entropy_from_dice.py::test_f4_recurse_64 -------------------------------- live log call -------------------------------- DEBUG [331: convertDiceRollsToBitsUsingLargestPossibleGroups] diceRollsString: 64 DEBUG [332: convertDiceRollsToBitsUsingLargestPossibleGroups] Converting diceRollsString to bits using groupSizeList [2, 1] DEBUG [346: convertDiceRollsToBitsUsingLargestPossibleGroups] Number of dice rolls remaining = 2. Using groupSize=2. Converting dice rolls section '64' to bits. DEBUG [387: convertDiceRollsToBits] diceRollsString: 64 DEBUG [388: convertDiceRollsToBits] len(diceRollsString)=2, groupSize=2, a=6**groupSize=6**2=36 DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=36. Result: 2**5 = 32. DEBUG [391: convertDiceRollsToBits] For a=6**groupSize=36, nextLowestIntegerPowerOf2 = 32 (=2**5) DEBUG [399: convertDiceRollsToBits] maxUsableValue = powerOf2 - 1 = 32-1 = 31 DEBUG [403: convertDiceRollsToBits] maxDiceRollValue (maxUsableValue converted back into a dice roll value): 62 DEBUG [415: convertDiceRollsToBits] - rolls section '64' ('53' in base6) ignored. DEBUG [424: convertDiceRollsToBits] 1 groups of 2 dice roll values ignored because they were greater than the maxDiceRollValue 62 (51 in base6, 31 in base10). DEBUG [425: convertDiceRollsToBits] Result: bits = '' DEBUG [353: convertDiceRollsToBitsUsingLargestPossibleGroups] No bits extracted. Try to process this section again with a smaller groupSize. DEBUG [331: convertDiceRollsToBitsUsingLargestPossibleGroups] diceRollsString: 64 DEBUG [332: convertDiceRollsToBitsUsingLargestPossibleGroups] Converting diceRollsString to bits using groupSizeList [1] DEBUG [346: convertDiceRollsToBitsUsingLargestPossibleGroups] Number of dice rolls remaining = 2. Using groupSize=1. Converting dice rolls section '6' to bits. DEBUG [387: convertDiceRollsToBits] diceRollsString: 6 DEBUG [388: convertDiceRollsToBits] len(diceRollsString)=1, groupSize=1, a=6**groupSize=6**1=6 DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=6. Result: 2**2 = 4. DEBUG [391: convertDiceRollsToBits] For a=6**groupSize=6, nextLowestIntegerPowerOf2 = 4 (=2**2) DEBUG [399: convertDiceRollsToBits] maxUsableValue = powerOf2 - 1 = 4-1 = 3 DEBUG [403: convertDiceRollsToBits] maxDiceRollValue (maxUsableValue converted back into a dice roll value): 4 DEBUG [415: convertDiceRollsToBits] - rolls section '6' ('5' in base6) ignored. DEBUG [424: convertDiceRollsToBits] 1 groups of 1 dice roll values ignored because they were greater than the maxDiceRollValue 4 (3 in base6, 3 in base10). DEBUG [425: convertDiceRollsToBits] Result: bits = '' DEBUG [346: convertDiceRollsToBitsUsingLargestPossibleGroups] Number of dice rolls remaining = 1. Using groupSize=1. Converting dice rolls section '4' to bits. DEBUG [387: convertDiceRollsToBits] diceRollsString: 4 DEBUG [388: convertDiceRollsToBits] len(diceRollsString)=1, groupSize=1, a=6**groupSize=6**1=6 DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=6. Result: 2**2 = 4. DEBUG [391: convertDiceRollsToBits] For a=6**groupSize=6, nextLowestIntegerPowerOf2 = 4 (=2**2) DEBUG [399: convertDiceRollsToBits] maxUsableValue = powerOf2 - 1 = 4-1 = 3 DEBUG [403: convertDiceRollsToBits] maxDiceRollValue (maxUsableValue converted back into a dice roll value): 4 DEBUG [423: convertDiceRollsToBits] - rolls section '4' ('3' in base6), value=3, bits='11' DEBUG [424: convertDiceRollsToBits] 0 groups of 1 dice roll values ignored because they were greater than the maxDiceRollValue 4 (3 in base6, 3 in base10). DEBUG [425: convertDiceRollsToBits] Result: bits = '11' PASSED [100%] ========================== 1 passed in 0.09 seconds =========================== stjohn@judgement:work$ pytest -o log_cli=true --log-cli-level=DEBUG --log-format="%(levelname)s [%(lineno)s: %(funcName)s] %(message)s" test_entropy_from_dice.py::test_f4_recurse_group_7 ============================= test session starts ============================= platform linux2 -- Python 2.7.12, pytest-4.6.11, py-1.9.0, pluggy-0.13.1 rootdir: /mnt/c/Users/User/Desktop/stuff/articles/generating_entropy_from_dice_2/work collected 1 item test_entropy_from_dice.py::test_f4_recurse_group_7 -------------------------------- live log call -------------------------------- DEBUG [331: convertDiceRollsToBitsUsingLargestPossibleGroups] diceRollsString: 6452463 DEBUG [332: convertDiceRollsToBitsUsingLargestPossibleGroups] Converting diceRollsString to bits using groupSizeList [7, 2, 1] DEBUG [346: convertDiceRollsToBitsUsingLargestPossibleGroups] Number of dice rolls remaining = 7. Using groupSize=7. Converting dice rolls section '6452463' to bits. DEBUG [387: convertDiceRollsToBits] diceRollsString: 6452463 DEBUG [388: convertDiceRollsToBits] len(diceRollsString)=7, groupSize=7, a=6**groupSize=6**7=279936 DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=279936. Result: 2**18 = 262144. DEBUG [391: convertDiceRollsToBits] For a=6**groupSize=279936, nextLowestIntegerPowerOf2 = 262144 (=2**18) DEBUG [399: convertDiceRollsToBits] maxUsableValue = powerOf2 - 1 = 262144-1 = 262143 DEBUG [403: convertDiceRollsToBits] maxDiceRollValue (maxUsableValue converted back into a dice roll value): 6452454 DEBUG [415: convertDiceRollsToBits] - rolls section '6452463' ('5341352' in base6) ignored. DEBUG [424: convertDiceRollsToBits] 1 groups of 7 dice roll values ignored because they were greater than the maxDiceRollValue 6452454 (5341343 in base6, 262143 in base10). DEBUG [425: convertDiceRollsToBits] Result: bits = '' DEBUG [353: convertDiceRollsToBitsUsingLargestPossibleGroups] No bits extracted. Try to process this section again with a smaller groupSize. DEBUG [331: convertDiceRollsToBitsUsingLargestPossibleGroups] diceRollsString: 6452463 DEBUG [332: convertDiceRollsToBitsUsingLargestPossibleGroups] Converting diceRollsString to bits using groupSizeList [2, 1] DEBUG [346: convertDiceRollsToBitsUsingLargestPossibleGroups] Number of dice rolls remaining = 7. Using groupSize=2. Converting dice rolls section '64' to bits. DEBUG [387: convertDiceRollsToBits] diceRollsString: 64 DEBUG [388: convertDiceRollsToBits] len(diceRollsString)=2, groupSize=2, a=6**groupSize=6**2=36 DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=36. Result: 2**5 = 32. DEBUG [391: convertDiceRollsToBits] For a=6**groupSize=36, nextLowestIntegerPowerOf2 = 32 (=2**5) DEBUG [399: convertDiceRollsToBits] maxUsableValue = powerOf2 - 1 = 32-1 = 31 DEBUG [403: convertDiceRollsToBits] maxDiceRollValue (maxUsableValue converted back into a dice roll value): 62 DEBUG [415: convertDiceRollsToBits] - rolls section '64' ('53' in base6) ignored. DEBUG [424: convertDiceRollsToBits] 1 groups of 2 dice roll values ignored because they were greater than the maxDiceRollValue 62 (51 in base6, 31 in base10). DEBUG [425: convertDiceRollsToBits] Result: bits = '' DEBUG [353: convertDiceRollsToBitsUsingLargestPossibleGroups] No bits extracted. Try to process this section again with a smaller groupSize. DEBUG [331: convertDiceRollsToBitsUsingLargestPossibleGroups] diceRollsString: 64 DEBUG [332: convertDiceRollsToBitsUsingLargestPossibleGroups] Converting diceRollsString to bits using groupSizeList [1] DEBUG [346: convertDiceRollsToBitsUsingLargestPossibleGroups] Number of dice rolls remaining = 2. Using groupSize=1. Converting dice rolls section '6' to bits. DEBUG [387: convertDiceRollsToBits] diceRollsString: 6 DEBUG [388: convertDiceRollsToBits] len(diceRollsString)=1, groupSize=1, a=6**groupSize=6**1=6 DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=6. Result: 2**2 = 4. DEBUG [391: convertDiceRollsToBits] For a=6**groupSize=6, nextLowestIntegerPowerOf2 = 4 (=2**2) DEBUG [399: convertDiceRollsToBits] maxUsableValue = powerOf2 - 1 = 4-1 = 3 DEBUG [403: convertDiceRollsToBits] maxDiceRollValue (maxUsableValue converted back into a dice roll value): 4 DEBUG [415: convertDiceRollsToBits] - rolls section '6' ('5' in base6) ignored. DEBUG [424: convertDiceRollsToBits] 1 groups of 1 dice roll values ignored because they were greater than the maxDiceRollValue 4 (3 in base6, 3 in base10). DEBUG [425: convertDiceRollsToBits] Result: bits = '' DEBUG [346: convertDiceRollsToBitsUsingLargestPossibleGroups] Number of dice rolls remaining = 1. Using groupSize=1. Converting dice rolls section '4' to bits. DEBUG [387: convertDiceRollsToBits] diceRollsString: 4 DEBUG [388: convertDiceRollsToBits] len(diceRollsString)=1, groupSize=1, a=6**groupSize=6**1=6 DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=6. Result: 2**2 = 4. DEBUG [391: convertDiceRollsToBits] For a=6**groupSize=6, nextLowestIntegerPowerOf2 = 4 (=2**2) DEBUG [399: convertDiceRollsToBits] maxUsableValue = powerOf2 - 1 = 4-1 = 3 DEBUG [403: convertDiceRollsToBits] maxDiceRollValue (maxUsableValue converted back into a dice roll value): 4 DEBUG [423: convertDiceRollsToBits] - rolls section '4' ('3' in base6), value=3, bits='11' DEBUG [424: convertDiceRollsToBits] 0 groups of 1 dice roll values ignored because they were greater than the maxDiceRollValue 4 (3 in base6, 3 in base10). DEBUG [425: convertDiceRollsToBits] Result: bits = '11' DEBUG [346: convertDiceRollsToBitsUsingLargestPossibleGroups] Number of dice rolls remaining = 5. Using groupSize=2. Converting dice rolls section '52' to bits. DEBUG [387: convertDiceRollsToBits] diceRollsString: 52 DEBUG [388: convertDiceRollsToBits] len(diceRollsString)=2, groupSize=2, a=6**groupSize=6**2=36 DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=36. Result: 2**5 = 32. DEBUG [391: convertDiceRollsToBits] For a=6**groupSize=36, nextLowestIntegerPowerOf2 = 32 (=2**5) DEBUG [399: convertDiceRollsToBits] maxUsableValue = powerOf2 - 1 = 32-1 = 31 DEBUG [403: convertDiceRollsToBits] maxDiceRollValue (maxUsableValue converted back into a dice roll value): 62 DEBUG [423: convertDiceRollsToBits] - rolls section '52' ('41' in base6), value=25, bits='11001' DEBUG [424: convertDiceRollsToBits] 0 groups of 2 dice roll values ignored because they were greater than the maxDiceRollValue 62 (51 in base6, 31 in base10). DEBUG [425: convertDiceRollsToBits] Result: bits = '11001' DEBUG [346: convertDiceRollsToBitsUsingLargestPossibleGroups] Number of dice rolls remaining = 3. Using groupSize=2. Converting dice rolls section '46' to bits. DEBUG [387: convertDiceRollsToBits] diceRollsString: 46 DEBUG [388: convertDiceRollsToBits] len(diceRollsString)=2, groupSize=2, a=6**groupSize=6**2=36 DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=36. Result: 2**5 = 32. DEBUG [391: convertDiceRollsToBits] For a=6**groupSize=36, nextLowestIntegerPowerOf2 = 32 (=2**5) DEBUG [399: convertDiceRollsToBits] maxUsableValue = powerOf2 - 1 = 32-1 = 31 DEBUG [403: convertDiceRollsToBits] maxDiceRollValue (maxUsableValue converted back into a dice roll value): 62 DEBUG [423: convertDiceRollsToBits] - rolls section '46' ('35' in base6), value=23, bits='10111' DEBUG [424: convertDiceRollsToBits] 0 groups of 2 dice roll values ignored because they were greater than the maxDiceRollValue 62 (51 in base6, 31 in base10). DEBUG [425: convertDiceRollsToBits] Result: bits = '10111' DEBUG [339: convertDiceRollsToBitsUsingLargestPossibleGroups] Can't use groupSize=2 - it's larger than the remaining number of dice rolls (1). DEBUG [346: convertDiceRollsToBitsUsingLargestPossibleGroups] Number of dice rolls remaining = 1. Using groupSize=1. Converting dice rolls section '3' to bits. DEBUG [387: convertDiceRollsToBits] diceRollsString: 3 DEBUG [388: convertDiceRollsToBits] len(diceRollsString)=1, groupSize=1, a=6**groupSize=6**1=6 DEBUG [486: nextLowestIntegerPowerOf2] Find greatest power of 2 that is less than or equal to n=6. Result: 2**2 = 4. DEBUG [391: convertDiceRollsToBits] For a=6**groupSize=6, nextLowestIntegerPowerOf2 = 4 (=2**2) DEBUG [399: convertDiceRollsToBits] maxUsableValue = powerOf2 - 1 = 4-1 = 3 DEBUG [403: convertDiceRollsToBits] maxDiceRollValue (maxUsableValue converted back into a dice roll value): 4 DEBUG [423: convertDiceRollsToBits] - rolls section '3' ('2' in base6), value=2, bits='10' DEBUG [424: convertDiceRollsToBits] 0 groups of 1 dice roll values ignored because they were greater than the maxDiceRollValue 4 (3 in base6, 3 in base10). DEBUG [425: convertDiceRollsToBits] Result: bits = '10' PASSED [100%] ========================== 1 passed in 0.15 seconds =========================== Great. That's the end of this project. I found these links helpful during this project: hyperlink http://crypto.stackexchange.com/questions/6175/how-to-best-obtain-bit-sequences-from-throwing-normal-dice crypto.stackexchange.com/questions/6175/how-to-best-obtain-bit-sequences-from-throwing-normal-dice hyperlink http://math.stackexchange.com/questions/2916887/shannon-entropy-of-a-fair-dice math.stackexchange.com/questions/2916887/shannon-entropy-of-a-fair-dice hyperlink http://www.math.hawaii.edu/~ramsey/Probability/TwoDice.html www.math.hawaii.edu/~ramsey/Probability/TwoDice.html