Context
Many-As-One is a from-scratch distributed systems project in Python + gRPC / Protocol Buffers. It spans the storage, consensus and compute layers: a replicated file system with client-side caching, two replication strategies, a Raft-backed key-value store, and a fault-tolerant data-parallel ML trainer running on top of them.
The name is the idea. Replication and consensus are what make many independent machines behave as one consistent system.
Storage layer
I built two RPC layers by hand — a gRPC file service and an equivalent length-prefixed, encrypted TCP transport (pickle + Fernet) — so the caching and versioning logic stays transport-agnostic.
The file system gives close-to-open consistency through a version-validated client cache: unchanged files are served locally with zero network transfer, and writes commit atomically on close.
Replication and consensus
Two strategies, with a written rationale for choosing between them:
- A complete Raft engine: leader election, log replication with fast conflict backtracking, persistence and snapshot install.
- A leaner primary-backup cluster with failover and re-sync, covered by a 32-test suite.
I then reused the Raft engine to build a key-value store in the spirit of etcd: linearizable Put, Delete and compare-and-swap, leader-served reads with an optional ReadIndex barrier, and idempotent retries that survive a failover.
Compute layer
On top of the storage stack sits a data-parallel ML trainer — logistic regression trained across workers through a parameter server. The model is checkpointed into the Raft key-value store, so training resumes after a coordinator crash.
The bug worth keeping
I found and fixed a lost-write linearizability bug in my Raft commit path, and wrote a hermetic regression test that reproduces it. That test is the part I would point at first: consensus code is easy to write and hard to prove, and this failure only appeared under one specific interleaving.
Stack
Python · gRPC · Protocol Buffers · threading · cryptography (Fernet) · Make
Concepts: RPC, client caching and cache coherence, idempotent retries, leader election, log replication, primary-backup replication, linearizability, snapshotting, data-parallel SGD and parameter servers.