Building a Distributed Key-Value Store in C++ (Part 1)

Why build a distributed KV store?

There are few better ways to get hands-on with distributed systems and C++ than building a distributed key-value store from scratch. This journey is both a learning experience and an exploration of ideas that power modern systems like Redis, etcd, and Consul.

In this series I'll attempt to walk through each step of designing and implementing a distributed KV store, from single-node basics to a replicated, fault-tolerant system.

The big idea

A key-value store is a simple database that stores data as a collection of key-value pairs — think of it as a glorified dictionary, but with persistence, concurrency control, replication, and more.

Eventually the store should support:

• Multi-node communication
• Replication and eventual consistency
• Leader election
• Crash recovery
• Client-server networking

But we get there step by step.

Architecture overview

Step 1: start simple

The journey begins with a very basic Hello World in-memory KV store.

File structure

distribkvstore/
├── CMakeLists.txt
├── src/
│   ├── kvstore.cpp
│   ├── kvstore.hpp
│   └── main.cpp
└── tests/
    └── test_kvstore.cpp  (planned)

Basic operations

Three operations for now:

• put(key, value) — Insert or update a value
• get(key) — Retrieve a value
• del(key) — Delete a key

Example usage (main.cpp)

int main() {
  KVStore store;
  store.put("hello", "world");
 
  if (auto val = store.get("hello")) {
    std::cout << "GET hello: " << *val << "\n";
  }
 
  if (store.del("hello")) {
    std::cout << "deleted hello\n";
  }
 
  if (!store.get("hello")) {
    std::cout << "hello not found after deletion\n";
  }
 
  return 0;
}

The code compiles cleanly with CMake and serves as the foundation for everything that follows.

Roadmap

1. Phase 1 — Local store ✅ Done: KVStore class with put, get, del; command-line demo
2. Phase 2 — Persistence (next): Store data on disk using an append-only log for crash recovery and future compaction
3. Phase 3 — Networking (planned): Expose the store over TCP sockets with a lightweight protocol and a CLI client
4. Phase 4 — Multi-node architecture (planned): Run multiple instances with replication and eventual (or synchronous) consistency
5. Phase 5 — Consensus (planned): Implement Raft or Gossip for leader election, partition handling, and write safety
6. Phase 6 — Testing & resilience (final): Fuzz and chaos testing, graceful failure handling, and performance tuning

Next steps

In Part 2 I add unit tests and integrate persistence via a simple append-only log. That paves the way toward recoverability — one of the central challenges in distributed design.