Study notesWeek 5: Decentralized Storage And Content Addressing
Week 5
Decentralized Storage And Content Addressing
Decentralized systems from zero, week 5
Name data by what it contains and fetch it from anyone.
Outcomes
What You Can Do By The End
Contrast location and content addressing
Compute hashes in Python
Build and verify a Merkle tree
Explain Git as a Merkle DAG
Compare IPFS, Filecoin and Arweave
The problem
Dead Links And Location Addressing
A web address names a server and a path
It says nothing about the file's content
The owner can change, move or delete the file
Your link then shows something else, or nothing
The idea
Content Addressing
The address is a fingerprint of the bytes
Change the bytes and the address changes
Same bytes give the same address everywhere
Fetch from anyone, verify yourself
Tool
Hash Functions In Practice
SHA-256 gives 64 hex characters
Deterministic, fast and one way
Tiny change gives a totally different digest
MD5 and SHA-1 have known collision attacks
import hashlib
for text in [b'hello', b'hello!']:
print(hashlib.sha256(text).hexdigest())
Code
Content-Addressed Store
Put data in, get its hash back
Get data out by hash and verify
Identical data is stored once
import hashlib
store = {}
def put(data):
key = hashlib.sha256(data).hexdigest()
store[key] = data
return key
def get(key):
data = store[key]
assert hashlib.sha256(data).hexdigest() == key
return data
k = put(b'Hello')
Ralph Merkle
Merkle Trees
Hash each block
Hash pairs of hashes, layer by layer
One root commits to all the data
Changing any block changes the root
root
/ \
h(AB) h(CD)
/ \ / \
h(A) h(B) h(C) h(D)
Code
Merkle Root In Python
Hash the blocks
Combine pairs until one hash remains
Duplicate the last hash if the count is odd
import hashlib
h = lambda b: hashlib.sha256(b).digest()
def root(blocks):
layer = [h(b) for b in blocks]
while len(layer) > 1:
if len(layer) % 2: layer.append(layer[-1])
layer = [h(layer[i] + layer[i+1])
for i in range(0, len(layer), 2)]
return layer[0].hex()
print(root([b'A', b'B', b'C', b'D']))
Compact evidence
Merkle Proofs
Show block C plus the sibling hashes up to the root