2023.04 Vol.2

Base58Check

The bitcoin ecosystem uses a neat and unique encoding scheme, Base58Check, for bitcoin addresses. Before I dive into it, I always have to start at the beginning with encodings. Like, why do we even encode things anyways?

Computers think in 1’s and 0’s, a.k.a binary or Base2. The smallest chunk of data is “bit” which is either a 1 or a 0. I know, I took it way back, but its really where the value of encodings comes from. Even as a programmer, I rarely think about these 1’s and 0’s. I am usually operating at a more abstract level like “this thing has this property” and I don’t actually know off-hand how the computer is thinking about that thing in binary.

So what does this have to do with encoding. Well, computers are good at thinking in bits, but humans sure aren’t. For example, the number 121 is represented as 1111001 in binary. Compare that to this random byte 11010111. I honestly don’t know what number that is, I just randomly typed it in. It’s really tough for humans to “eyeball” binary and have any sense of what the data means. But let’s look at these random decimal-based numbers: 21 and 132. I instantly know one is bigger by an order of magnitude than the other, one is odd and one is even, and one is my favorite number. Information dense. For whatever reason, humans naturally think of things in groups of ten. This is one of the benefits of encoding. We can take some binary data and encode it in a different base in order to make it easier on us humans. For example, 11010111 in Base16 is just D7. Base16, also known as hexadecimal, works nicely with binary because every 4 bits map to one of the 16 hexadecimal digits (0-9 and a-f). Our standard Base10 (decimal) is even more readable than Base16 since we use it everywhere, but the conversion is not as straightforward, so not as popular with the computers.

So encoding helps with human readability. Another use case is when an interface only accepts text as a parameter, but you want to send some binary data through it. The common example is sending an image over email. If you just shove the image binary data into the interface which is expecting text, it will treat it as text and weird unexpected things might happen. For example, some text symbols mean special things in the email protocol, so you might end up cutting a message off early if the binary just happens to trigger it. If you take the binary, encode it in hexadecimal, and then send it the interface will work just fine. The app on the other side just has to know to decode the hexadecimal. This magic isn’t free, binary data is usually much more efficient. Since a character is usually stored in a byte (8 bits), the encoding from binary to hex turns every 4 bits into 8. But the use-ability! Technically we could use something like Base64 which encodes more data into a character, but it eats away at the readability benefit with the 64 characters. Trade-offs.

Back to bitcoin. Bitcoin addresses are like virtual spots to send bitcoin. These spots are defined by the script used to lock the bitcoin, so the spot could be as straightforward as “the person who controls $THIS private key”. Some more complex spots are “2 of 3 people control this bitcoin” or a simple payment channel like “2 of 2 people control this bitcoin OR 1 person does after 3 days”. How are these variable length locking scripts turned into predictable length addresses? While bitcoin transaction outputs can be locked with any script, there are two dominant patterns which both involve paying to a hash: P2PKH (Pay To Public Key Hash) and P2SH (Pay To Script Hash). More info on those transaction scripts in the bitcoin manual. But this means as long as a payee knows what hash they want to get paid too, they can share just that hash to payers. Payers can now easily build (most likely with their wallet app) the correct P2PKH and P2SH transactions.

The simplicity is awesome, but if either human makes a mistake (as humans are prone to do) with that hash in the transfer then that bitcoin gets sent to a completely different, unintended, and most likely gone forever location. Taking the binary hash and encoding into something like Base64 already makes it more use-able in human-land. As mentioned above, it would now be way safer to email this thing. But the standard bitcoin encoding scheme, Base58Check, adds a few more layers of protection for us silly humans.

The first extra layer is using Base58 instead of Base64 which omits specific problematic characters: 0, O, l, I, +, /. The plus and slash are dropped just cause those are the only non-alphanumeric characters in Base64. The rest is pretty obvious just looking at them, they could easily be swapped on accident or on purpose by a bad actor. Base58 isn’t as easy for computers to encode things as Base64, but Base58Check is optimizing for best-for-humans.

Next up, Base58Check adds a few special bytes to the hash before encoding it. The hash is prefix’d with a version byte which maps to common things which are encoded with Base58Check like the previously mentioned public keys or script hashes. This makes it easy for humans to look at the encoding output and have a clue what its encoded, for example a public key hashes start with 1 and a script hashes start with 3.

Finally, a checksum is tacked on to the end of the hash to make it easy to detect any tampering or errors in the hash.

SHA256(SHA256(prefix+data))

The checksum algorithm includes the version prefix, but only the first 4 bytes of the output is tacked on the hash being encoded.

So the prefix + hash + checksum are encoded with Base58 to form a bitcoin address, a nice human-friendly version of a virtual spot to send bitcoin.

I am curious about the double SHA256 used to calculate the checksum, it sounds like it was just Satoshi being extra careful and trying to avoid some attack vectors like “length extension attacks”. Anyways, next up, gonna have to take a look at the new (since 2017…) Beck32 address encoding scheme.