Primitives and Orchestrator

The reference DeRec library exposes the protocol at two altitudes. DeRecProtocol is the recommended path — it manages the state machine, wire encoding, encryption, validation, and persistence so the application can focus on its own concerns and react to typed events. The lower altitude — the primitives API — is a set of plain stateless functions, one per flow component, that lets the application drive every byte itself.

Both speak the same wire protocol. An application built on either tier interoperates with any conforming DeRec peer regardless of which tier the peer chose.

This chapter explains the trade-off so applications can make the right call. The default should be DeRecProtocol. Primitives are the right answer only when something about the host environment prevents DeRecProtocol from fitting.

DeRecProtocol — the orchestrator

DeRecProtocol is an event-driven object that fully manages the protocol on the application's behalf. It runs the protocol state machine but keeps none of that state in memory — everything it needs between messages is persisted through the storage abstractions below, so the object itself is effectively stateless. The application supplies a small, well-defined set of inputs:

• Storage — implementations of the protocol's storage abstractions for channels, shares, the Secret, and the orchestrator's in-flight protocol state (outstanding verification challenges, in-progress recovery, pending unpairs, the active sharing round). The application picks the backend (in-memory, SQLite, Postgres, the application's own database, etc.).

• Transport — a callback that delivers outbound bytes to a peer, and an input that feeds inbound bytes back into the orchestrator. The callback receives every endpoint the peer advertised, in the peer's preference order; it dials whichever it chooses, stops at the first success, and fails only if every endpoint failed. The library opens no socket and does not rank endpoints. Each SDK ships adapters that do the bookkeeping, so the application implements a single-endpoint send: SendOne (ISendOne in .NET) wrapped in SequentialFailover or SingleEndpointTransport in Rust, .NET, and Go; sequentialFailover(dialer) / singleEndpointTransport(dialer) in TypeScript and React Native.

• Configuration — secret_id, replica_id (when the application uses replicas), the recovery threshold, and the local node's own transport endpoints: a non-empty list in preference order with at most one entry per protocol (https://, http://, grpcs://, grpc://), set with with_own_transports / withOwnTransports / WithOwnTransports / Go Config.OwnTransports.

Everything else is handled automatically:

• Wire encoding and decoding of every protocol message.

• Channel encryption under the per-channel Shared Key.

• Validation of inbound messages against the protocol's rules.

• The state machine — pairing state, in-flight requests, response binding, role gating.

• Persistence of channel records, shares, the Secret, and all in-flight protocol state via the storage abstractions — the orchestrator holds nothing in memory between calls.

• Event emission — the application reacts to typed events (PairingCompleted, ShareStored, SecretRecovered, ReplicaSecretReceived, ActionRequired, …) instead of parsing wire bytes.

The API surface for typical use is small: call start to initiate a flow, call process when bytes arrive on the transport, and call accept / reject on any ActionRequired events that need an application-level policy decision. That is the whole loop.

Primitives — plain stateless functions

The primitives layer exposes the building blocks the orchestrator composes. Each flow has its own module of stateless produce / process helpers. They have no state of their own — every call is a pure function over its arguments.

When an application drives the primitives directly, it takes on every responsibility the orchestrator would otherwise handle:

• Encoding outbound messages and decoding inbound ones.

• Encrypting outbound and decrypting inbound under the per-channel Shared Key.

• Validating inbound messages and rejecting malformed or replayed ones.

• Tracking in-flight state — outstanding requests, accumulators, timeouts, role gating.

• Persisting and reloading channel records, shares, and the Secret.

• Binding inbound responses back to the requests that triggered them.

The primitives expose no opinion on any of the above. They are deliberately minimal so they can be embedded in environments where the orchestrator's assumptions do not hold.

Driving the primitives directly means taking on the binding gates that defend against replay. Verification responses, recovery accumulators, and pending unpair acknowledgements all rely on the orchestrator retaining outstanding-request state — which DeRecProtocol persists through its state store — so it can validate inbound responses against the specific request they answer. An integration that drives the primitives directly MUST implement equivalent retention, or its inbound-response handling is open to replay attacks. The orchestrator does this for you.

It also means owning most of the transport scheme decision. Endpoint checking has two stages: structural validation (non-empty, length cap, no control characters, scheme consistent with the declared protocol) always applies; the scheme policy — whether plaintext http:// or grpc:// may be used, and for whose endpoint — is a TransportPolicy, built from the unsafe_connection setting.

Two pairing primitives apply policy themselves: pairing::response::produce takes a TransportPolicy and filters the requester's advertised endpoints through it (TransportPolicy::new(false) is the production posture), and the pairing primitives check the parameter_range each side advertises, refusing a pairing whose ranges do not overlap. The other primitives that return a peer endpoint — the sharing, recovery, discovery, verification, and unpairing request extractors — validate structure only; the caller should run TransportPolicy::check_peer on what they return.

Every SDK consumer goes through the orchestrator and is unaffected. See Transport endpoints and plaintext connections.

When to use which

The orchestrator is the right answer for nearly every application — phones, desktops, server-side services, browser extensions, and stateless or load-balanced back ends alike. Because it keeps no in-memory session state of its own — every in-flight flow is persisted through the state store — it fits both long-lived processes and per-request hosts. It reduces the integration to wiring storage and transport, then handling typed events.

Primitives are the right answer only when something specific prevents the orchestrator from fitting. Examples:

• Restricted environments. Embedded targets, FFI-only consumers, or runtimes that cannot host the orchestrator's storage abstractions may need to drive the wire directly.

• Custom message-queue architectures. A host whose existing actor or message-bus system already owns per-flow state may want the primitives to slot in alongside that infrastructure rather than run a second state machine on top of it.

• Wire-level control. Integration work that needs to inspect or manipulate raw wire bytes — protocol research, interop testing, or non-Rust hosts driving the protocol over a thin shim.

Statelessness by itself is not on this list. Because the orchestrator externalises its in-flight state to the state store, a serverless or load-balanced deployment can back that store with a shared database and keep the orchestrator — pointing every store at shared storage is enough. If none of the constraints above apply, the choice is DeRecProtocol.

Same wire, either way

Both tiers emit and accept the same wire format. A DeRecProtocol instance and a primitives-driven integration can pair with each other, exchange shares, and recover a Secret without either side being aware of the other's altitude.

This is what makes the choice safe to revisit per-deployment. An application may start on DeRecProtocol and migrate hot paths to primitives later if a constraint emerges, or lift a primitives-driven prototype onto the orchestrator once the production host can sustain it. Conformance is at the wire layer, not at the API layer.

For the practical walkthrough — picking a storage backend, wiring the transport, an end-to-end example on each tier — see Getting Started — Choose your integration path.