Skip to main content

Data architecture

The fluxrig data architecture supports integrity in mission-critical environments. Every signal in the rig is self-describing, bit-perfect, and audit-ready.

The signal: fluxMsg​

At the center of the ecosystem is the fluxMsg, a structured representation of a transactional signal. It acts as an envelope carrying the raw payload and its associated operational context.

Deterministic binary serialization​

We use Deterministic CBOR (RFC 8949) as the project wire language. This provides the foundation for institutional data-plane trust:

  1. Bit-Perfect Stability: The same logical message results in an identical byte-for-byte signature, enabling reliable cryptographic verification.
  2. Type Fidelity: Field types are preserved throughout the signal path, eliminating ambiguity in financial and industrial calculations.
  3. Payload Efficiency: Native support for binary blobs allows the transport of legacy payloads (e.g., ISO8583 bitmaps) without encoding overhead.

The fluxMsg structure​

For the full technical specification of the fluxMsg fields, system flags, and wire-format requirements, see the Data Model Reference.

Technical identity model​

To ensure data sovereignty and end-to-end auditability across thousands of distributed nodes, we implement a dual-track identity model.

flux_id (transactional tracer)​

Every signal entering the rig receives a flux_id, a 128-bit, time-ordered unique identifier (UUID v7).

  • Standard: RFC 9562 (UUID v7).
  • Property: Chronologically sortable (millisecond precision), globally unique, and optimized for native indexing in storage engines.
  • Role: The primary key for telemetry joins, distributed traces, and audit archives.

entity_id (persistent component identity)​

Components that require persistent identity (Racks, Gears, Scenarios) receive an entity_id.

  • Structure: A 128-bit UUID v7 that embeds the EntityType and a machine_id Hint for stateless traceability.
  • Role: Provides immutable, human-interpretable identities for infrastructure components across their entire lifecycle.

Schema governance​

We strictly separate the Tactical Structure of a signal from its Topological Path.

fluxSpec (logical schema)​

Defined via the Spec Definition Language (SDL), these schemas define the structure of external protocols (e.g., "ISO8583 Dialects").

  • Immutability: Specs are stored in a Content-Addressable Store (CAS) using SHA-256 hashing. Any modification generates a new hash, preventing silent failures in the distributed data-plane.
  • Validation: Every node in the fleet must re-verify the spec hash before executing logic on a new version.

Scenarios (topological blueprint)​

The Scenario is the declarative blueprint of a pipeline. It defines the signal flow between Gears. The Mixer pushes it as an immutable, signed artifact to the edge.

Sovereign persistence strategy​

Every Rack keeps its logs in a local write-ahead log. They therefore survive a connectivity failure. The Rack ships them when the bus returns.

  1. Log WAL: The Rack records every log line it produces to a local Write-Ahead Log (WAL) using binary storage, capped at store.wal_max_size_mb. Business messages are not written to it: they travel over the bus.
  2. Non-Intrusive Signal Tap: The Rack taps observability telemetry (Metrics/Traces) from the main execution path. It uploads this data asynchronously, so observability never adds latency to the business hot-path.
  3. Data Residency Sovereignty: Detailed payloads (e.g., raw financial messages) can remain exclusively in the local edge vault. Only high-level metadata then reaches the central Mixer.

NOTE

Privacy roadmap. Edge Tokenization and Deterministic Field Masking are currently in the future roadmap. They scrub sensitive fields natively within the Gear runtime. In the current release, manage masking with custom Gear logic. Use the Bento Gear as an alternative.

Technical references​