Implementing a persistent store

Every DeRec SDK requires the application to supply four storage adapters and one outbound transport. This chapter is the schema-and-query level walkthrough of the four store adapters at the data-model layer — partition keys, conceptual storage shape, queries the library makes, and idempotency contracts. It is language-agnostic on purpose: the per-language wiring lives in the SDK chapters, and the shape documented here is the same across all of them.

Read first: the Rust, .Net, and TypeScript SDK chapters for the language-specific wiring, and the Concepts chapters for protocol context. Two reference implementations ship in the lib-derec repository — SQLite under bindings/sqlite/ and Postgres under bindings/postgres/. Both implement the same trait surface, both are intentionally small (a few hundred lines each), and either can be copied wholesale as a starting point for a new backend.

---

The four stores

The library splits persistence along four orthogonal axes: paired channels, the shares each channel holds, per-channel cryptographic secrets, and the application-supplied Secret snapshot. Each store is partition-keyed by secret_id so a single backing database can serve many Secrets on the same device without leakage — the library never queries across secret_ids.

The trait method names below are quoted verbatim from library/src/protocol/traits.rs; signatures are simplified to focus on the parameters and the return value (each method actually returns a typed future — SecretStoreFuture, ChannelStoreFuture, or ShareStoreFuture — wrapping the value shown).

DeRecChannelStore

• Partition key — secret_id (u64).

• Sub-key — channel_id (ChannelId).

• Conceptual shape — one entry per paired channel, opaque to the application. The library serialises the Channel (counterparty transport endpoint, role, communication info, optional replica_id, lifecycle status, creation timestamp) and hands the value over for blind persistence.

The store also owns the channel-link graph: an undirected, idempotent, transitive relation recording that two channels belong to the same Owner identity. Recovery and Discovery resolve the linked set via this store, then load the corresponding shares via DeRecShareStore::load_many.

Required operations:

• load(secret_id, channel_id) -> Option<Channel>

• save(secret_id, channel) -> () — channel id is taken from Channel::id

• remove(secret_id, channel_id) -> bool — true iff an entry was removed

• channels(secret_id) -> Vec<Channel> — every channel in the partition

• link_channel(secret_id, a, b) -> () — record an undirected, idempotent, transitive edge

• linked_channels(secret_id, channel_id) -> Vec<ChannelId> — the connected component, including channel_id itself

Concrete SQLite schema (from bindings/sqlite/migrations/0001_init.sql):

CREATE TABLE channels (
    secret_id  INTEGER NOT NULL,
    channel_id INTEGER NOT NULL,
    data       BLOB    NOT NULL,
    PRIMARY KEY (secret_id, channel_id)
);

CREATE TABLE channel_links (
    secret_id INTEGER NOT NULL,
    a         INTEGER NOT NULL,
    b         INTEGER NOT NULL,
    PRIMARY KEY (secret_id, a, b)
);

channel_links is the undirected adjacency list backing link_channel / linked_channels. Idempotency: re-saving the same (secret_id, channel_id, channel) replaces the row; re-linking an already-linked pair is a no-op.

---

DeRecShareStore

• Partition key — secret_id.

• Conceptual storage key — (secret_id, channel_id, version, replica_id). The fourth component is the part that surprises people. A naive backend that keys only on (secret_id, channel_id, version) will silently lose concurrent writes from distinct replicas; the trait contract forbids that silent coalescing.

Each entry is opaque protobuf bytes — the byte format depends on which side stored it (Owner-side stores a CommittedDeRecShare; Helper-side stores the full StoreShareRequestMessage). The store itself never decodes.

Required operations:

• load(secret_id, channel_id, versions: &[u32]) -> Vec<Share> — empty versions slice means "all versions"; missing versions are silently skipped

• load_many(secret_id, channel_ids: &[ChannelId], versions: &[u32]) -> Vec<Share> — same shape across multiple channels in one round-trip; used by broadcast queries

• load_all(secret_id, channel_ids: &[ChannelId]) -> Vec<Share> — every share for every listed channel and every version; used by Discovery

• latest_version(secret_id) -> Option<u32> — highest version across all channels in the partition, or None if empty

• save(secret_id, channel_id, share) -> ()

• remove_channel(secret_id, channel_id) -> () — drop every share under the channel; idempotent

Concrete SQLite schema (from bindings/sqlite/migrations/0001_init.sql):

CREATE TABLE shares (
    secret_id       INTEGER NOT NULL,
    channel_id      INTEGER NOT NULL,
    version         INTEGER NOT NULL,
    replica_id      INTEGER,
    share_secret_id INTEGER NOT NULL,
    bytes           BLOB    NOT NULL
);
CREATE UNIQUE INDEX shares_uniq
    ON shares (secret_id, channel_id, version, COALESCE(replica_id, -1));

Note the COALESCE(replica_id, -1) trick: SQLite forbids expressions inside PRIMARY KEY / UNIQUE constraints, so the four-tuple uniqueness contract — which must treat two Owner re-sends (both replica_id = NULL) as the same row while keeping distinct replicas' writes apart — is expressed via a separate UNIQUE INDEX with a sentinel. The ON CONFLICT clause inside the save implementation targets the same expression so the idempotent re-send path matches the index exactly.

The share_secret_id column carries the denormalised Share::secret_id field. It must match the partition key secret_id; the SQLite reference asserts on this in debug builds.

Postgres divergence (from bindings/postgres/migrations/0001_init.sql): Postgres 15+ supports UNIQUE NULLS NOT DISTINCT, so the equivalent constraint is a single UNIQUE clause inline on the table:

CREATE TABLE shares (
    secret_id       BIGINT NOT NULL,
    channel_id      BIGINT NOT NULL,
    version         BIGINT NOT NULL,
    replica_id      BIGINT,
    share_secret_id BIGINT NOT NULL,
    bytes           BYTEA  NOT NULL,
    CONSTRAINT shares_uniq
        UNIQUE NULLS NOT DISTINCT (secret_id, channel_id, version, replica_id)
);

Both schemas express the same contract; the SQLite sentinel and the Postgres NULLS NOT DISTINCT are two routes to the same behaviour.

replica_id = NULL is a real value, not a missing one. Two Owner re-sends (both replica_id = NULL) for the same (secret_id, channel_id, version) must collide and overwrite each other (idempotent re-send). An Owner write and a Replica write for the same (secret_id, channel_id, version) must both survive side-by-side. Backends without a NULLS NOT DISTINCT equivalent must use a sentinel like COALESCE(replica_id, -1).

For why the four-tuple key matters at the protocol level, see Replica conflict resolution.

---

DeRecSecretStore

• Partition key — secret_id.

• Sub-key — (channel_id, kind) where kind is SecretKind::{SharedKey, PairingSecret, PairingContact}.

• Conceptual shape — per-channel cryptographic material. SharedKey is the post-pairing symmetric channel key (sensitive — implementations should persist with keychain-grade protection). PairingSecret is the ephemeral ECIES / ML-KEM key material used during pairing, removed once the shared key is derived. PairingContact is the initiator's ContactMessage, held transiently between start and pairing completion.

The library decides which kind to store; the application implements blind CRUD.

Required operations:

• load(secret_id, channel_id, kind) -> Option<SecretValue>

• load_many(secret_id, channel_ids: &[ChannelId], kind, missing_policy: MissingPolicy) -> Vec<(ChannelId, SecretValue)> — used by broadcast paths; missing_policy is either Skip (silently drop missing channels) or Fail (return MissingEntries carrying the unmatched channel ids)

• save(secret_id, channel_id, value: SecretValue) -> () — kind is inferred from the SecretValue variant

• remove(secret_id, channel_id, kind) -> () — idempotent

Concrete SQLite schema:

CREATE TABLE secrets (
    secret_id  INTEGER NOT NULL,
    channel_id INTEGER NOT NULL,
    kind       INTEGER NOT NULL,
    data       BLOB    NOT NULL,
    PRIMARY KEY (secret_id, channel_id, kind)
);

VSS guarantee: individual Verifiable Secret Sharing shares reveal zero information about the original secret, so share storage does not require keychain-grade protection — only this store does.

---

DeRecUserSecretStore

• Partition key — secret_id.

• Conceptual shape — at most one entry per secret_id — the latest application-supplied Secret snapshot. The library reads this so the pair-completion auto-publish hook can ship the current Secret to a freshly-paired Helper or Replica Destination without an explicit re-publish from the app.

Required operations:

• load_latest(secret_id) -> Option<UserSecrets>

• save_latest(secret_id, value: UserSecrets) -> () — overwrites any prior entry; the store keeps only the latest snapshot

• remove(secret_id) -> () — idempotent

Concrete SQLite schema:

CREATE TABLE user_secrets (
    secret_id   INTEGER NOT NULL PRIMARY KEY,
    version     INTEGER NOT NULL,
    description TEXT,
    payload     BLOB    NOT NULL
);

The version column mirrors UserSecrets::version — the monotonically increasing Secret version the protocol bumps on every publish. description is the optional human-readable label forwarded to helpers in StoreShareRequest.description. payload is the encoded UserSecrets::secrets blob.

---

Common patterns and pitfalls

• All four stores are partition-keyed by secret_id. Multiple Secrets coexist in one database without leakage. The library never issues a cross-partition query.

• All operations are async-shaped. Even sync backends must return a future (Box::pin(std::future::ready(...)) in Rust, Task<T> in C#, Promise<T> in TypeScript). This is what lets the same store implementation target native async runtimes, FFI hosts, and WASM.

• The library holds each store by &mut Self. Concurrent calls into the same store instance do not happen — implementations need no internal synchronisation.

• secret_id partitioning is enforced by the queries, not by the data type. Every public method takes secret_id as its first parameter; a backend that omits it from its WHERE clauses will leak across Secrets.

• The denormalised Share::secret_id must match the partition key. The library always supplies them paired; implementations may assert on the match.

Every save must be idempotent. The library may re-send during pairing or sharing under failure (timeouts, connection drops, replays). A save that matches an existing entry on its full conceptual key MUST replace it in place; receivers MUST converge to the same end state regardless of how many times the same payload is delivered. This applies to all four stores.

For why the share store's conceptual key includes replica_id rather than stopping at (secret_id, channel_id, version), see Replica conflict resolution.

---

Reference implementations

Backend	Location	Notes
SQLite	bindings/sqlite/src/stores/	In-memory or on-disk; uses COALESCE(replica_id, -1) for the shares uniqueness.
Postgres	bindings/postgres/src/stores/	Requires Postgres 15+ for NULLS NOT DISTINCT. Smoke runs against the local docker container.

Both bindings implement the same four traits over the same schema shape. They are intentionally small — a developer building a new backend (RocksDB, IndexedDB, a remote KV service, an in-process mock for tests) can fork either and substitute the storage primitives.

The Rust SDK chapter walks through wiring these into a DeRecProtocolBuilder, the .Net chapter shows the C# interface equivalent, and the TypeScript chapter shows the JS object equivalent.