Secret

The Secret is the central protocol-level object that an Owner identity protects. It carries the user-facing data along with everything the protocol needs to operate: the roster of paired Helpers, the roster of paired Replicas, and the keys those rosters depend on. Every Secret is self-contained — a device that holds the current Secret holds the full Owner identity.

This chapter is the canonical reference for the Secret. It covers the shape, the lifecycle, and the design choice that makes the Secret self-contained.

Two views

At the primitives level, a Secret is an opaque bag of bytes. The threshold secret-sharing scheme treats it as a single value to be split into shares; the math doesn't care what's inside.

At the protocol level, a Secret is a self-contained structured object:

{
  "secret_id": "...",
  "version":   "...",
  "secret": {
    "helpers": [...],
    "secrets": [...],
    "replicas": {
      "channel_id": "...",
      "shared_key": "...",
      "members": [...]
    }
  }
}

The outer level — secret_id, version, secret — is the addressable container. The inner secret field is where the substance lives: the rosters of paired participants and the user-facing data. Helpers store opaque shares of the bag-of-bytes view; the structured view exists only on devices that hold the Owner identity.

The fields

secret_id

The protocol-level identifier of this Secret. Fixed for the lifetime of the Owner identity — it is chosen when the identity is created and never changes. The secret_id partitions every other piece of state in the protocol: paired channels, stored shares, version history. An application may manage multiple Owner identities, each with its own secret_id, but a single Owner identity always has exactly one.

version

A monotonic counter that increments every time the inner secret field changes. The (secret_id, version) pair uniquely identifies a specific snapshot of the Secret.

version is always incremented on any change to secret — there is no batching, no debounce. Every observable change is a new version.

secret.helpers

The roster of paired Helpers. Each entry carries:

• The Helper's channel_id — the protocol-level handle for the channel.

• The Helper's transports — every endpoint it advertised, in its preference order, at most one per protocol. A peer may serve several protocols (https, grpcs, …); which one to dial is the application transport's decision.

• The per-Helper Shared Key — the symmetric key that authenticates and encrypts every message on this channel.

• Any application communication_info the Helper advertised.

Because the per-Helper Shared Key lives inside the Secret, every device in the Owner identity that holds the current Secret can talk to this Helper without further pairing. This is what makes recovery transitivity work — see Recovery transitivity.

secret.replicas

The replica group as a composite, not a flat list. It carries the member roster plus the two things every member shares:

• channel_id — the one channel every member is addressed on.

• shared_key — the 32-byte replica-group Shared Key. Unlike the per-Helper Shared Key (one per Helper), this is one key shared across the whole group. A new Replica adopts it during its initial sync — see The shared replica-group key.

• members — one entry per member, each carrying its replica_id (its global stable identity, and the key it is stored under), its transports (every endpoint it advertised, in preference order), its role, and any application communication_info.

The publisher's own entry is built the same way as any other: its full endpoint list and its communication_info, filled in from the instance's current configuration on every publish. A member that changes its endpoints or communication_info is therefore described correctly by the next version it publishes.

The field is absent entirely when this secret_id has no replica setup.

The roster includes the source and the reader itself. A group whose members cannot name themselves is not reconstructible from the payload, so omitting any member — including the writer — is a defect, not an optimization.

role is absolute: exactly one member carries Source, and that is what the member is within the group, not what it is relative to whoever is reading. A device that has paired but not yet hydrated may hold a provisional role for its admitter; the arriving roster overwrites it.

Note that channel_id and shared_key are group-level rather than per-member, which is why this is a composite. A member is identified by its replica_id — never by the channel, which every member shares, and which a member can even move between during an admission handover.

secret.secrets

The user-facing data the application is protecting. Each entry is one logical secret — a wallet seed phrase, a private key, a credential, encrypted backup material, etc. The protocol does not impose a structure on what an individual entry looks like; applications choose the shape they need.

Some applications might opt to use onse UserSecret per secret to be protected (e.g. one UserSecret for Wallet Seed Phrase, one for TOPT Recovery Code, etc) while others might use a single UserSecret to put them all in a custom format.

Writer attribution

The Secret structure itself carries no writer stamp, and the recoverable payload below has none. Attribution is carried alongside each version, in two places:

• The local snapshot. UserSecrets.author_replica_id (crate::protocol::types::UserSecrets) records the replica-group member that published the version this device holds. It is set on every publish, whatever changed, and is None when no group member published it: an Owner with no replica identity, or a snapshot rebuilt by restore.

• The replica-sync payload. ReplicaSecretPayload.author_replica_id carries the publisher with the version to every other member. The enclosing message's replicaId cannot do this job: on a push it names the publisher, but on a ReplicaDiscovery catch-up it names whichever member answered.

A member compares the two when a copy arrives at the version it already holds: the same author is a re-send, a different author is a conflict. A held version with no recorded author is compared by its user secrets instead. See Conflict resolution.

Helper-bound messages carry no replica identity at all. Every member publishes to a Helper as the Owner, and a Helper holds exactly one share per version: a second write at the same version with different content is refused with VERSION_CONFLICT rather than stored alongside.

Recoverable encoding

When the Secret is protected, the structured object above is serialized into the recoverable payload — the bytes that are encrypted, VSS-split to Helpers, and reconstructed from a threshold of shares on recovery. Because a Secret protected by one DeRec implementation must be recoverable by another, this format is deliberately self-describing and implementation-independent:

secret_data = [ 1-byte version ] · payload

• The leading byte is the encoding's major version, read before decompression so a future version can change the compression as well as the schema. New payloads are written as 3. It bumps only on breaking changes (in lockstep with the protocol majors that touch the Secret), and is a property of the encoding — not the version counter above, and not the protocol version.

• In v3, payload = gzip(JSON) (RFC 1952). In the JSON, byte fields are standard-padded base64 (RFC 4648 §4), u64 fields are decimal strings, keys are snake_case, and absent or empty optional fields are omitted. The schema:

Secret      { helpers:[Helper], secrets:[UserSecret], replicas?:Replicas }
Helper      { channel_id:str<u64>, transports:[Endpoint], shared_key:base64,
              communication_info?:{str:str} }
Endpoint    { uri:str, protocol:int }
UserSecret  { id:base64, name:str, data:base64 }
Replicas    { channel_id:str<u64>, shared_key:base64, members:[ReplicaInfo] }
ReplicaInfo { replica_id:str<u64>, transports:[Endpoint], role:str,
              communication_info?:{str:str} }

• Endpoint.protocol is the TransportProtocol.Protocol discriminant, the same integer the wire type carries (0 = HTTPS, 1 = gRPC), so no reader has to infer it from the URI scheme. transports lists every endpoint the peer advertised, in its preference order.

• role is the string "Source" or "Destination".

Any conforming implementation can therefore recover a Secret by reading the version byte, gunzipping, and parsing the JSON — no DeRec-specific schema library required. The normative field-by-field schema and encoding rules live in the derec-library protocol::types::secret module documentation, which is the source of truth for the format. Golden vectors (a canonical Secret paired with its exact JSON, for both v3 and v2) ship under library/tests/golden/; gzip decoding is universal but encoding is not byte-stable, so conformance is defined on the JSON, pre-gzip.

Older encodings

The decoder accepts majors 3 and 2, and rejects anything else.

• v2 is still decoded. It is identical to v3 except that each Helper and ReplicaInfo carries a single transport_uri:str instead of transports. The decoder lifts that URI into a one-entry transports list, deriving the protocol from its scheme. A URI that is not a valid endpoint — for example, one whose scheme is not https://, http://, grpcs:// or grpc:// — decodes with no endpoints rather than a guessed one; restore then skips that peer and reports it as PeerNotRestored with reason NoTransports. Nothing is ever written as v2.

v1 is rejected outright, not upgraded. In v1 the roster listed only the destination peers; the source was named separately by a bare owner_replica_id and had no roster entry of its own, so nothing in the payload carries its endpoint. Upgrading would mean inventing one, so the decoder refuses rather than silently producing a degraded Secret.

Any payload written by 0.0.1-alpha.9 or earlier must be re-published before it can be recovered. Those releases stamp major byte 1, and the current decoder returns an unsupported-version error for it rather than falling back. Re-publish at the current version before upgrading, while a release that can still read the old payload is available.

This gzip-JSON encoding applies to the helper-recoverable payload only. The internal replica-sync message that carries the full Secret to a paired Replica (ReplicaSecretPayload) stays protobuf, like every other transport message — see Messages. Both encode the same structured Secret; an SDK hands your application the same typed object either way.

How the Secret evolves

Every observable change to the inner secret field is a new version. The triggers, in practice:

• User secrets change. An entry in secret.secrets is added, removed, or rotated.

• The Helper roster changes. A Helper is paired or unpaired, or an existing Helper's addressing changes.

• The Replica roster changes. A Replica is paired or unpaired.

Every increment is a full new snapshot, not a diff. The protocol does not transmit per-field changes — when version increments, the new state of every field is published in full to every paired Replica, and a fresh share goes to every paired Helper. See Secret synchronization for the wire mechanics.

Worked example

The table below traces a single Secret from "just created" through pairing two Replicas, three Helpers, and a third Replica. Threshold t = 3.

State notation per device: v=N (h=H, s=S, r=R) — version N of the Secret, with H paired Helpers, S user secrets, and R paired Replicas (the originating device counts itself in r). The annotation shares list means the share map in the snapshot is populated; absence means it isn't yet.

Step	Action	latest_version after	Replica A	Replica B	Replica C	Helpers
0	new()	None ("v0")	—	—	—	—
1	pair replica A → verify_fingerprint	1	✓ v=1 (h=0, s=0, r=1)	—	—	—
2	ProtectSecret([s1])	2	✓ v=2 (h=0, s=1, r=1)	—	—	—
3	pair replica B → verify_fingerprint	3	✓ v=3 (h=0, s=1, r=2)	✓ v=3 — bootstrap	—	—
4	pair helper #1 → auto-publish	4	✓ v=4 (h=1, s=1, r=2)	✓ v=4	—	— (1<3)
5	pair helper #2 → auto-publish	5	✓ v=5 (h=2, s=1, r=2)	✓ v=5	—	— (2<3)
6	ProtectSecret([s1, s2])	6	✓ v=6 (h=2, s=2, r=2)	✓ v=6	—	— (2<3)
7	pair helper #3 → auto-publish	7	✓ v=7 (h=3, s=2, r=2, shares list)	✓ v=7 (same)	—	✓ all 3 Helpers get VSS shares
8	pair replica C → verify_fingerprint	8	✓ v=8 (h=3, s=2, r=3, shares list)	✓ v=8 (same)	✓ v=8 — full bootstrap (h=3, s=2, r=3, shares list)	✓ all 3 Helpers get fresh VSS shares at v=8

Worth highlighting:

• Every step bumps version by exactly one. It does not matter whether the trigger was a user-secret change, a roster change, or a Helper join — the version increments once per change and the new snapshot is what gets distributed.

• The Replica roster includes the device itself. Step 1 starts at r=1, not r=0 — the originating device is in secret.replicas from the beginning.

• Joining as a Replica is a bootstrap. Steps 3 and 8 show the new device receiving the full Secret at the current version directly. There is no per-version replay to catch up.

• Below the threshold, no Helper shares are distributed. Steps 4–6 add Helpers and a user secret without producing any wire traffic to Helpers (annotated 1<3 / 2<3). The Helpers are paired but holding nothing.

• The first time the threshold is met, every paired Helper receives a share. Step 7 is the inflection point — pairing the third Helper takes the roster from 2 to 3, the threshold is reached, and all three Helpers receive a VSS share at v=7 in one round (not just the newly-paired one).

• Subsequent changes refresh every Helper. Step 8 adds a Replica; all three Helpers get fresh shares stamped with the new version. The threshold-recovery path stays consistent with the current Secret without any extra application-level coordination.

Self-containment

The Secret is self-contained: it carries every key, every roster entry, every configuration value that the Owner identity needs in order to drive the protocol. There is no extra state held outside the Secret that a device would also need in order to act as the Owner identity.

This is the design choice that makes the replica feature work. A device that holds the current Secret holds everything:

• The plaintext user data, in secret.secrets.

• The per-Helper Shared Keys, in secret.helpers. Any of them is enough to talk to the corresponding Helper.

• The replica-group Shared Key and channel, in secret.replicas. Enough to talk to the rest of the replica group.

The cost of this design is documented in Recovery transitivity: a compromised device that holds the Secret is equivalent to a compromised Owner. There is no privilege separation between devices in the Owner identity — they all hold the same thing.

Three fields are the material worth protecting

UserSecret::data, HelperInfo::shared_key, and Replicas::shared_key are the plaintext user data and the symmetric keys. Everything else in the Secret is structure — ids, URIs, roles, versions.

The Rust types carry that distinction into their Debug output. prost::Message requires Debug as a supertrait, and a derived one would print all three verbatim, so a single tracing::debug!(?secret) or {:?} on an error path would put plaintext secrets and symmetric keys into a log file. Since 0.0.2 those three fields print as <redacted, N bytes> instead; structural fields stay visible, so the output is still worth reading. Key-bearing types elsewhere in the crate (SecretValue, PairingKeyMaterial) sidestep the problem differently — they derive no Debug at all.

This protects Rust Debug output, and nothing else. It is not a general redaction facility, and it does not travel across the binding boundary. The TypeScript, React Native, Go, and .NET SDKs hand you ordinary objects and structs — console.log(event.secret) on a recovered Secret prints the user data and every per-Helper shared key in full.

Serialization is deliberately unaffected on every binding, Rust included: the Secret has to round-trip intact for replica sync and recovery to work at all. So anything that serializes — a crash reporter attaching state, an analytics payload, a structured log that encodes rather than formats, an error-tracking SDK snapshotting local variables — still captures the real bytes.

Treat the redaction as a guard against one specific accident, not as permission to log Secrets.

What's not in the Secret

The protocol leaves a few things to the implementation, deliberately:

• Where the Secret is stored locally. Applications decide between in-memory, on-disk, encrypted-at-rest with a device key, etc.

• How the Secret is serialised at rest. The wire format is part of the protocol; the at-rest format is the application's choice.

• Cached state about in-flight protocol flows. Pending sharing rounds, outstanding recovery sessions, accumulators — these are runtime state owned by the protocol implementation and are not part of the Secret.

These boundaries keep the Secret as the durable, replicable snapshot of "what the Owner identity is", while the runtime layer manages the volatile state.