You’ve heard of ACID transactions and BASE. Push those both aside for AIDE:
Atomic — transaction all-or-nothing, no partial application
Isolated — snapshot at OPEN, no dirty reads during transaction
Durable — committed, stays committed, will propagate everywhere eventually
Eventually-Consistent — global state converges, but not synchronously at commit time
AIDE is more than just data transactions. It covers schema and constraint changes as well. Frontier can handle database catalogs, and cluster co-ordination without locks or leaders.
Database Catalog. To update the schema or security/permission/programmable logic/constraints, we make the change in the INFORMATION_SCHEMA tables, and in the same transaction do the actual change (such as adding a column) to the table/sub. A node that can see that transaction knows the table has changed. A node that doesn’t see the transaction won’t try using the new column and won’t be affected by it.
Cluster Co-ordination. That’s what transactions are. A run time config change happens in a transaction which is seen or not by each node.
data placement, shards, node registry, peering, node health all can happen in transactions, but for the most part are actually things that can be just reported and don’t require synchronization.
Barrier Synchronization: Gemini told me “Coordinates multi-stage processes where Node A, B, and C must all finish Phase 1 before any of them can start Phase 2.” A multi stage process, so a node (or an agent working through a node) wants all the nodes to do something, and only when they’re all done it wants (but might go down) some other thing to happen. That sounds like a pipeline to me. Node A, B, and C have an output sub that they write to. A periodic process checks that they have all written and starts phase 2. The periodic process is in the system, and not tied to any one node, and can restart the processes that were on any of Nodes A, B and C if one of them crashed.
Part of what got me working on an RDBMS system in the first place was the pain of schema changes in existing RDBMSs. I’ve always thought it was ridiculous that adding a column meant essentially taking the table off-line for some time when you’re working with large tables.
The same should work for constraints and other database catalog info. Eventual consistency of atomic isolated transactions everywhere, including database schema.
AIDE transactions are Atomic, Isolated, Durable, and Eventually-Consistent. Not ACID, not BASE. We’re not dealing with Soft-State here.
AIDE is exactly what Frontier provides