It's closest to TiDB's key-value layer; a building block for more complex systems. More traditional, monolithic databases like CockroachDB (SQL) or FaunaDB (NoSQL) trade off extensibility for the benefits in performance and operations that come from very tight coupling.

In my understanding, FoundationDB's transaction management is closest to FaunaDB's; read/write sets are linearized in memory in preprocessing nodes and distributed asynchronously to the replicas rather than locked on the replica leaders like Spanner or CockroachDB. This is why FoundationDB doesn't support long-lived transactions.

It's interesting that the FoundationDB team chose to unwind their service architecture (there used to be separate transaction manager and replica processes), I assume in the interests of ease of operations.

It is not clear to me how leader election and failover works for the transaction management role. Maybe somebody from the team can clarify.

> monolithic databases like CockroachDB (SQL) or FaunaDB (NoSQL) trade off extensibility

The CockroachDB documentation says their SQL implementation is layered on top of their K/V interface: https://www.cockroachlabs.com/docs/stable/architecture/overv...

This would make it similar to TiDB/TiKV and FoundationDB.

Here's and old article from VoltDB insisting that layering SQL on pure K/V forfeits too much performance: https://www.voltdb.com/blog/2015/04/01/foundationdbs-lesson-...

Edit: And a response: https://news.ycombinator.com/item?id=15505194

I don't recall them ever having separate processes by default. You can set that up though by limiting roles.

Maybe it was the same binary. Pretty sure you had to operationally manage the roles separately and now they are integrated.

On the other hand, I don't really know.

No, it's always been possible to have as little as one fdbserver process and have a complete key/value store. Internally it is "microservices" though - it will start a "proxy", a "resolver", a "log", a "storage", etc within that one process.

Wait CockroachDB is monolithic? I though it was distributed.

I mean it in the terms of monolithic process vs. service-oriented architecture, distinct from a distributed vs. centralized operational topology.

FaunaDB and CockroachDB are implemented as monolithic processes and can break encapsulation boundaries for performance reasons. For example, FaunaDB does aggressive predicate pushdown to accelerate intersections and joins, which you cannot do if you have to conform to a key/value interface exclusively. It can also eliminate all network overhead for query data that's local to the processing node.

I understand how that terminology is confusing though...how would you explain it?

I think the use of monolithic here is that CockroachDB is higher level than FoundationDB. The project was not designed to be your do-everything-DB that you layer different systems on, but has a specific goal and is intended to be used directly.