Go

This section guides Go developers through installing, understanding, and integrating the DeRec protocol using the official github.com/derecalliance/lib-derec/packages/go module.

---

Go SDK Overview

The DeRec Go SDK provides a managed surface over the same DeRecProtocol orchestrator the Rust SDK exposes. It binds the Rust derec-library core through ebitengine/purego — a pure-Go FFI layer, so no cgo and no C compiler are required to build or consume the module. The compiled Rust core is embedded per-platform and loaded lazily on first use; there is no external native dependency to install and no explicit initialization step.

The Go API operates on serialized []byte wire messages plus a small set of typed Go structs (decoded events, store domain types). No protocol logic lives in Go — framing, encryption, validation, and reconstruction all happen in the Rust core. Errors surface as *derec.Error, carrying a Category and Code; Process wraps it in a *protocol.ProcessError (see Error Handling).

The SDK allows applications to:

• Distribute Secret shares across trusted Helpers using a threshold secret-sharing scheme

• Verify Helpers still hold shares

• Recover the Secret from a threshold of Helpers

• Optionally extend the Owner identity across multiple devices via the replica feature, with automatic Secret synchronization between them

Two things differ from the Rust / TypeScript / .NET bindings. There is no fluent builder — you configure a protocol.Config struct and call protocol.New(...). And events are not typed subclasses: Process / Start return a flat []protocol.Event, each discriminated by its Type field against the protocol.EventType* constants. Both are called out where they first appear below.

---

Installation

go get github.com/derecalliance/lib-derec/packages/go@v0.0.8

These chapters document lib-derec 0.0.8.

The module requires Go 1.23+. It lives in a subdirectory of the lib-derec repository, so its release tags take the form packages/go/v0.0.8; go get ...@v0.0.8 resolves them for you. Every lib-derec package ships at the same version.

The module is pre-1.0. Breaking changes between releases are expected — pin an exact version and treat upgrades as a deliberate review step.

Supported platforms

The compiled Rust core ships embedded for four OS/arch combinations. The correct binary is selected at compile time by Go build tags:

OS	amd64	arm64
macOS	✅	✅
Linux	✅	✅

Windows is not supported. The module embeds native libraries for macOS and Linux only — there is no windows build tag, so a Windows target fails to compile. If you need Windows, build the Rust core for x86_64-pc-windows-msvc yourself and add a matching //go:embed build-tagged file under internal/native/; this is not wired into the release build today.

---

Quick Start

Implement the five store interfaces plus a transport, then construct and drive the orchestrator. The example below is the minimum scaffold; the flow chapters show how each protocol round-trip flows through Process(...).

import (
	"fmt"

	"google.golang.org/protobuf/proto"

	"github.com/derecalliance/lib-derec/packages/go/derecpb"
	"github.com/derecalliance/lib-derec/packages/go/protocol"
)

// 1. Application-supplied persistence + transport. The module ships no
//    default implementation — your app owns storage and network I/O.
//    Each is your implementation of the corresponding protocol interface.
var (
	channels    protocol.ChannelStore    = NewMyChannelStore()
	shares      protocol.ShareStore      = NewMyShareStore()
	secrets     protocol.SecretStore     = NewMySecretStore()
	userSecrets protocol.UserSecretStore = NewMyUserSecretStore()
	state       protocol.StateStore      = NewMyStateStore()
	transport   protocol.Transport       = protocol.SequentialFailover{Dialer: NewMyDialer()}
)

// 2. Construct the orchestrator. One DeRecProtocol instance manages one Secret.
//    There is no builder — configuration is a Config struct.
p, err := protocol.New(channels, shares, secrets, userSecrets, state, transport, protocol.Config{
	SecretID: 0xC0FFEE,
	OwnTransports: []protocol.TransportProtocolParam{
		{URI: "https://owner.example.com", Protocol: int32(derecpb.Protocol_HTTPS)},
	},
	Threshold: proto.Uint32(3), // nil means the default (3)
})
if err != nil {
	panic(err)
}
defer p.Close() // the instance owns a native handle; always Close it.

// 3. Drive an inbound envelope through the orchestrator. Process returns every
//    Event it produced while handling the envelope.
events, err := p.Process(inboundEnvelope)
if err != nil {
	panic(err) // see "When process fails" below for the real handling
}
for _, ev := range events {
	switch ev.Type {
	case protocol.EventTypePairingCompleted:
		// ev.ChannelID is the (rekeyed) long-term channel id.
		fmt.Printf("paired channel %d\n", ev.ChannelID)
	case protocol.EventTypeActionRequired:
		// App consents; the library resumes the flow and returns its own events.
		if _, err := p.Accept(ev.Action); err != nil {
			panic(err)
		}
	case protocol.EventTypeShareStored:
		fmt.Printf("stored share v%d for %d\n", *ev.Version, ev.ChannelID)
	case protocol.EventTypeSecretRecovered:
		// ev.Secret is the typed *Secret (already decoded). Show it to the user,
		// then call p.Restore(*ev.Secret, version) — see recovery.md.
		installRecoveredSecret(ev.Secret)
	}
}

protocol.New requires all six arguments — the five stores (ChannelStore, ShareStore, SecretStore, UserSecretStore, StateStore) and the Transport — and returns an error if any is nil. There is no compile-time typestate guarantee like the Rust builder; a missing dependency surfaces as a New error at construction time.

Config.OwnTransports is required. It lists every endpoint this application serves, in preference order, with at most one entry per protocol (derecpb.Protocol_HTTPS or derecpb.Protocol_GRPC). An empty list fails New with a *derec.Error whose Code is derec.CodeInvalidInput; two entries of the same protocol are refused too. List only endpoints you actually serve: delivery is push-only, so an advertised endpoint nobody listens on makes pairing succeed and replies vanish.

Optional Config fields:

Field	Type	Default (zero value / nil)
Threshold	*uint32	nil = 3. Every non-nil value is forwarded; a value below 2, including 0, fails New. Build one inline with proto.Uint32(n).
CommunicationInfo	map[string]string	empty
Timeouts	*protocol.Timeouts	library defaults — see Timeouts
UnsafeConnection	*bool	nil = production posture — see below
AutoRespondOnFailure	bool	false
UnpairAck	protocol.UnpairAck	UnpairAckRequired
AutoReplyTo	bool	false
AutoAccept	protocol.AutoAcceptPolicy	every flow off — see Auto-accept
ReplicaID	*uint64	unset; required for replica flows — obtain once with derec.GenerateReplicaID()
ParameterRange	*protocol.ParameterRange	nil = no constraints advertised; see Pairing

UnsafeConnection: &allow with allow == true accepts plaintext http:// and grpc:// endpoints to any host, including endpoints a peer supplies. Unset or false, plaintext is accepted only for an endpoint this device configured for itself that names loopback (localhost, 127.0.0.1, ::1). Development only. See Transport endpoints and plaintext connections.

Serialize calls on one instance. The DeRecProtocol type is documented as not safe for concurrent use. The native handle does serialize overlapping calls behind a mutex, but Close must never race an in-flight call, and the Go wrapper's own state is not synchronized — so drive an instance from a single owning goroutine, or hold a mutex around every call. Tick counts: it mutates the same round state Process does. Two distinct instances bound to the same SecretID and the same stores always need an application lock. See Concurrency and scheduling.

Timeouts

Config.Timeouts configures four independent budgets. Every value is in whole seconds. A nil field means "use the library default"; every non-nil value, including 0, is forwarded as given and the library decides what it means.

p, err := protocol.New(channels, shares, secrets, userSecrets, state, transport, protocol.Config{
	SecretID: 0xC0FFEE,
	OwnTransports: []protocol.TransportProtocolParam{
		{URI: "https://owner.example.com", Protocol: int32(derecpb.Protocol_HTTPS)},
	},
	Timeouts: &protocol.Timeouts{
		InboundMessageSecs: proto.Uint64(300), // replay window
		SharingRoundSecs:   proto.Uint64(60),  // wait on a silent peer
		UnpairAckSecs:      proto.Uint64(60),  // wait on an unpair ack
		ExpiredChannels: &protocol.RemoveExpiredChannelsPolicy{
			Enabled:       true,
			TimeoutInSecs: 300, // fingerprint window
		},
	},
})

Field	Type	Default
InboundMessageSecs	*uint64	300
SharingRoundSecs	*uint64	60
UnpairAckSecs	*uint64	60
ExpiredChannels	*protocol.RemoveExpiredChannelsPolicy	{Enabled: true, TimeoutInSecs: 300}

InboundMessageSecs is a security boundary — it bounds how stale a message may be and still be accepted, so it has to tolerate transport latency and clock skew. A message older than it is dropped and reported as an EventTypeMessageIgnored event with Reason == protocol.IgnoreReasonExpired. The other three are liveness budgets. ExpiredChannels governs the automatic sweep of channels still awaiting fingerprint confirmation; set Enabled: false to leave that sweep to the application, which calls p.RemoveExpiredChannels(olderThanSecs) itself (see Channel management). See Concurrency and scheduling.

Tick()

Timeouts advance only inside Process and Tick. A publish whose peers all go quiet has nothing left to close it: the round stays open and no SharingComplete is ever emitted.

ticker := time.NewTicker(15 * time.Second) // shorter than SharingRoundSecs
for range ticker.C {
	events, err := p.Tick()
	if err != nil {
		return err
	}
	for _, ev := range events {
		handle(ev)
	}
}

Tick is idempotent and safe on an idle protocol — with nothing in flight it touches no state and returns no events. Hold the same lock you hold for Process. See Concurrency and scheduling.

Auto-accept

Config.AutoAccept is a protocol.AutoAcceptPolicy with one bool per inbound request kind: Pairing, PrePair, StoreShare, VerifyShare, Discovery, GetShare, Unpair, UpdateChannelInfo. The zero value turns every flow off, so every request surfaces as EventTypeActionRequired. A flow set to true is accepted inside Process, which emits EventTypeAutoAccepted (carrying ActionKind) in place of ActionRequired, followed by the events a manual Accept would have produced. Go has no All() constructor; set each field explicitly. Read the per-flow caveats before enabling any of them — StoreShare removes the helper's only admission-control point. See Channel management and AutoAcceptPolicy.

---

Protocol Flows

The module exposes every DeRec flow through a single p.Start(flowKind, params) entry point plus the inbound Process / Accept / Reject loop. Each FlowKind takes its own params struct; passing a mismatched one is a returned error.

FlowKind	Params	Chapter
FlowKindPairing	PairingParams{Kind, Contact, PeerCommunicationInfo}	Pairing
FlowKindDiscovery	DiscoveryParams{Target}	Recovery
FlowKindProtectSecret	ProtectSecretParams{Secrets, Description}	Share Distribution
FlowKindVerifyShares	VerifySharesParams{SecretID, Version, Target}	Verification
FlowKindRecoverSecret	RecoverSecretParams{SecretID, Version}	Recovery
FlowKindUnpair	UnpairParams{ChannelID, Memo}	Channel management
FlowKindUpdateChannelInfo	UpdateChannelInfoParams{Target, CommunicationInfo, OwnTransports}	Channel management
FlowKindReplicaDiscovery	ReplicaDiscoveryParams{} or nil	Channel management
FlowKindUnpairReplica	UnpairReplicaParams{ReplicaID, Memo}	Channel management

FlowKindReplicaDiscovery and FlowKindUnpairReplica are replica-only and require Config.ReplicaID:

// Ask the group whether this device is behind, and catch up if it is.
events, err := p.Start(protocol.FlowKindReplicaDiscovery, protocol.ReplicaDiscoveryParams{})

Flow	Purpose
Pairing	Establish secure communication between two participants — Owner ↔ Helper, or Replica pairings
Share Distribution	Split the Secret and distribute shares to Helpers; sync the full Secret to every paired Replica
Verification	Verify Helpers still hold shares
Recovery	Recover shares and reconstruct the Secret
Channel management	Unpair, UpdateChannelInfo, ReplicaDiscovery, UnpairReplica, rejecting requests, auto-accept, timeouts

Each flow follows the same shape: the initiator calls p.Start(flowKind, params); the library produces outbound envelopes through your Transport.Send; the responder feeds inbound envelopes to p.Process and answers any EventTypeActionRequired event with p.Accept(ev.Action) or p.Reject(ev.Action, status, memo). Fan-out flows (FlowKindDiscovery, FlowKindVerifyShares, FlowKindUpdateChannelInfo) take a protocol.Target, built with protocol.TargetAll(), protocol.TargetOne(channelID), or protocol.TargetMany(channelIDs...). TargetAll() means every Paired helper channel; a Pending or unknown id named in TargetOne / TargetMany is dropped, not refused.

The full flow catalog lives in Protocol Specification / Flows.

---

Transport

The SDK opens no sockets. Your protocol.Transport delivers every outbound envelope:

type Transport interface {
	Send(endpoints []protocol.Endpoint, message []byte) error
}

endpoints is every address the peer advertised, in the peer's order, already filtered by the transport policy; it is never empty, and every entry addresses the same peer. The library does not rank them. Deliver once: stop at the first endpoint that accepts the message, and return an error only when none did. protocol.Endpoint carries URI string and Protocol int32 (derecpb.Protocol_HTTPS = 0, derecpb.Protocol_GRPC = 1). Recognised schemes are exactly https://, http://, grpcs:// and grpc://.

Prefer the adapters to writing Send by hand. Implement protocol.SendOne — delivery to one endpoint — and wrap it:

type httpDialer struct{ client *http.Client }

func (d httpDialer) SendOne(endpoint protocol.Endpoint, message []byte) error {
	resp, err := d.client.Post(endpoint.URI, "application/octet-stream", bytes.NewReader(message))
	if err != nil {
		return err
	}
	defer resp.Body.Close()
	if resp.StatusCode/100 != 2 {
		return fmt.Errorf("deliver to %s: %s", endpoint.URI, resp.Status)
	}
	return nil
}

// Tries each endpoint in the peer's order, stops at the first success. The default.
var transport protocol.Transport = protocol.SequentialFailover{Dialer: httpDialer{client: http.DefaultClient}}

// Uses the first endpoint only — makes declining failover visible to a reviewer.
var single protocol.Transport = protocol.SingleEndpointTransport{Dialer: httpDialer{client: http.DefaultClient}}

Taking endpoints[0] inside your own Send compiles and passes every test, and silently gives up the failover the list exists to provide.

Config.AutoReplyTo: true makes every outbound request carry replyToTransports set to the current OwnTransports, so the peer answers there for that exchange without persisting it. Every *Started event and every ActionRequired carries a TraceID (uint64) that the peer echoes on its response; envelope.ReadTraceID(bytes) from github.com/derecalliance/lib-derec/packages/go/primitives/envelope reads it off a raw envelope. A peer behind NAT that must answer on the request's own connection uses that trace id to pick its reply; see "Serving DeRec over request/response transports" in the Rust library README.

---

Message Model

The SDK operates on serialized protocol envelopes rather than exposing protobuf types directly on the orchestrator API.

Secret format. The recoverable Secret — the bytes Helpers store and recovery reconstructs — is [ version byte ] · payload; the version byte is currently 3, the decoder also accepts 2, and 1 is rejected. The payload is gzip-compressed JSON (RFC 1952), with byte fields as standard-padded base64 (RFC 4648 §4) and u64 fields as decimal strings; v3 records each roster entry's endpoints as transports: [{uri, protocol}] and role as a string. This is internal to the SDK — you always pass and receive a typed Secret — and only matters for cross-app interop or primitive-layer use. See Concepts → Secret for the full schema. The replica-sync message and every other wire message stay protobuf.

• Inbound and outbound envelopes are plain []byte on both sides of the orchestrator

• These bytes represent:

  • A serialized DeRecMessage envelope, or

  • A plain serialized ContactMessage (the out-of-band bootstrap, see Pairing)

• For encrypted protocol exchanges, the DeRecMessage envelope contains an encrypted inner protobuf message — the library performs the encryption, decryption, and framing for you

Applications are responsible for:

• Delivering outbound []byte envelopes — the library passes them to Transport.Send(endpoints, message) — to the counterparty

• Receiving inbound []byte envelopes and passing them back through DeRecProtocol.Process(...)

Every u64 identifier is a typed uint64 — on events (ChannelID, PairingChannelID, SecretID, TraceID, FromReplicaID, the other replica ids, Synced / Behind), on the recovered Secret (Helper.ChannelID, Replicas.ChannelID, Replica.ReplicaID), and in flow params. Print them with %d. Event.Kind, Event.SenderKind and PairingParams.Kind are protocol.SenderKind; Replica.Role is protocol.ReplicaRole (ReplicaRoleSource / ReplicaRoleDestination); every event Status is derecpb.StatusEnum. Option-typed Rust fields are Go pointers (Version, ReplicaID, AuthorReplicaID, ShareSize, …) — check Type before dereferencing.

The wire schema ships with the module: package github.com/derecalliance/lib-derec/packages/go/derecpb/schema embeds FileDescriptorSet (the full import closure) and Proto (an embed.FS of the flat .proto sources). See Schema distribution.

derec.CurrentProtocolVersion() returns the derec.ProtocolVersion{Major, Minor} the embedded core speaks — 0.0 at this release. Record it in audit logs; it is distinct from the library version. See Versioning.

---

Error Handling

All library-side errors surface as *derec.Error:

import (
	"errors"

	"github.com/derecalliance/lib-derec/packages/go/derec"
)

_, err := p.Start(protocol.FlowKindVerifyShares, params)
if err != nil {
	var e *derec.Error
	if errors.As(err, &e) {
		// e.Category / e.Code  — int32 derec.Category* / derec.Code* constants
		// e.CategoryName() / e.CodeName() — the library's stable names, e.g. "role_mismatch"
		// e.PeerStatus / e.PeerMemo — populated when Code == derec.CodeNonOKStatus
		// e.Expected  / e.Got       — populated when Code == derec.CodeVersionMismatch
		// e.ConflictingChannelIDs   — populated when Code == derec.CodeRestoreConflict
		switch e.Code {
		case derec.CodeRoleMismatch:
			// A flow was started against a channel where this node holds the wrong role.
		case derec.CodeReplicaIDNotConfigured:
			// A replica-mode flow was attempted without a ReplicaID in Config.
		default:
			log.Printf("derec error: %v", e)
		}
	}
}

derec.Error.Category and derec.Error.Code are int32 constants that mirror the core's category and code values one-for-one; CategoryName() / CodeName() return the same lower_snake names every SDK reports ("unknown" for a value the library does not define), and Error() includes both. Branch on Code, never parse Message. Binding-side failures use category derec.CategoryFFI. The same (Category, Code) pair carries the same meaning across every language binding — see the cross-binding error catalog.

When process fails

Process returns a *protocol.ProcessError when the core rejects a message:

// package protocol
type ProcessError struct {
	ChannelID *uint64      // channel the message arrived on; nil when the bytes were not a decodable envelope
	Events    []Event      // events produced before the failure
	Err       *derec.Error // the underlying error
}

Before handling the message, Process settles any expired sharing-round and unpair deadlines and saves the result. If the message then fails, those events travel on the error and are never reported again — handle Events exactly as a successful call's events, then handle the error:

events, err := p.Process(envelope)
var perr *protocol.ProcessError
switch {
case err == nil:
	handle(events)
case errors.As(err, &perr):
	handle(perr.Events)
	report(perr.ChannelID, perr.Err)
default:
	return err // closed instance or event-decode failure
}

ProcessError.Unwrap() returns the *derec.Error, so errors.As(err, &derecErr) still works on a Process error. A direct type assertion err.(*derec.Error) no longer matches it. See When process fails.

---

API Reference

The module is documented with standard godoc; browse it on pkg.go.dev. The published module is the source of truth for every method, Config field, and Event field documented in this section.

*protocol.DeRecProtocol methods:

Method	Purpose
protocol.New(channels, shares, secrets, userSecrets, state, transport, cfg)	Construct an instance
Close()	Free the native handle; idempotent
SecretID() uint64	The secret this instance manages, as passed in Config.SecretID
CreateContact(channelID *uint64, mode ContactMode, nonce *uint64)	Produce an out-of-band contact — see Pairing
Start(flowKind, params)	Start a flow
Process(message)	Handle an inbound envelope
Accept(action) / Reject(action, status derecpb.StatusEnum, memo string)	Answer an ActionRequired
Tick()	Advance timeouts without an inbound message
Restore(secret, version)	Commit a recovered Secret — see Recovery
GetFingerprint(channelID) / VerifyFingerprint(channelID, fingerprint)	Out-of-band channel confirmation
RemoveExpiredChannels(olderThanSecs uint64) ([]uint64, error)	Sweep Pending channels by hand
SetOwnTransports([]TransportProtocolParam) / SetCommunicationInfo(map[string]string)	Change what this node advertises — see Channel management

Package derec also exports derec.GenerateReplicaID() uint64 (never 0) and derec.CurrentProtocolVersion().

The module also ships a primitives layer (primitives/pairing, sharing, verification, discovery, recovery, unpairing, envelope) for applications that drive the wire messages themselves; the chapters in this section use the orchestrator.

---

SDK Architecture

Like the .NET binding, the Go SDK wraps the same C ABI the Rust core exports — but it does so through purego rather than cgo, so the toolchain needs no C compiler. The compiled derec-library core is embedded in the module and loaded at runtime; every binding is a thin translation over the same core.