Technical stack

fluxrig is built on a modern, high-performance stack designed for reliability, extensibility, and long-term maintainability.

Stand on the Shoulders of Giants: fluxrig is proudly built upon the open source ecosystem. We orchestrate best-in-class technologies to deliver a unified enterprise platform, giving full credit and gratitude to the communities maintaining these foundational projects.

Mixer & Rack (OSS)

• Language: Go (golang) 1.25+

  • Stability policy: We adhere to the level n version policy (bleeding edge). We track the latest stable Go release.
  • Rationale: To support modern integrations (e.g., bento, Wasm) and leverage cutting-edge ergonomics (iterators), we accept the trade-off of being on the latest release.
  • Deployment targets: CGO_ENABLED=0 creates statically linked binaries. Distributed as native Linux executables or <15MB scratch Docker containers.

• Orchestration: Temporal.io (Go SDK)

  WARNING

  • Wasm sandbox: [Roadmap] Secure, polyglot execution of custom business logic in Rust, Go, or TypeScript.

• ISO8583 library: Moov-io/iso8583 (apache 2.0) - core parsing engine (used in both native core and Wasm gears).

• API architecture: cbor (data plane) + Rest (control plane).

  • Interface definition: Go structs (serialized via cbor).
  • Control plane: Rest API (served by Mixer).
  • Internal transport: cbor messages over NATS (snake protocol).

• Messaging & streaming:

  • Transport: NATS JetStream (Synadia) - chosen for high-performance, distributed persistence (snake protocol).
    • Streams: See wire protocols for the detailed subject topology (fluxrig.control, fluxrig.data).
  • Library: Watermill (mit) - Go library for building event-driven applications.
    • Role: Standardizes the "publisher/subscriber" interface. Current: NATS (durable). Planned: gochannel (RAM) for "turbo wire".
    • Benefits: Middleware (OTel, pivot tracing), mockability, and router pattern.

• CLI framework: Cobra (apache 2.0) - standard library for building powerful CLI applications.

  • Key commands: Run, inspect config, inspect logs, logs, data query

• Configuration: Koanf (mit) - lightweight, extensible configuration management (toml/yaml/env) replacing viper.

• Logging:

  • Library: Log/slog (Go stdlib) - structured, high-performance logging.
  • Rotation: Lumberjack (mit).
  • Environment overrides:
    • FLUXRIG_TRACE=1 - Override all log levels to trace (useful for debugging).
    • FLUXRIG_DISABLE_TELEMETRY=1 - Disable telemetry shipping.

• Gear runtime: Wazero (apache 2.0)

  WARNING

  [Roadmap]: Zero dependency WebAssembly runtime for Go. Enables secure, platform-independent gear execution.

Integration & ecosystem

• Universal I/O engine: Bento (mit)
  • Fork: We explicitly use the WarpStream Labs fork (mit) to ensure permissive licensing, avoiding the Benthos/Redpanda (BSL) restrictions.
  • Role: Provides 100+ native connectors (Kafka, s3, amqp, etc.) and the bloblang mapping language.
  • Integration: Wrapped as a native Gear (pkg/gears/native/bento).

Data & analytics

• Operational db:

  • Standard tier: DuckDB (embedded). high-performance OLAP + oltp for local state.
  • Enterprise: Distributed sql (external). supports ha, replication, and concurrency for Mixer state.

• Analytics dw: DuckDB (mit) + parquet (apache 2.0) - high-performance OLAP on local storage.

• Visualization:

  • Oss: Grafana - industry standard for observability dashboards.
  • Enterprise: Signoz (open source apm) - evaluation candidate (full-stack apm & tracing ui).

• Secure store:

  • OSS: AES-256-GCM (software encryption).
  • Enterprise: HashiCorp Vault or AWS KMS.

Web management UI (enterprise)

• Framework: Reflex (apache 2.0) - pure Python web framework that compiles to react/next.js.
• Ui library: Tailwindcss (mit) + radix ui (mit).
• Visual editor: React flow (mit) - node-based graph editor for designing scenarios.
• Code editor: Monaco editor (mit) - embedded for editing specs and scripts.

AI & analytics (local) — future / research

• Inference engine: ONNX Runtime / llama.cpp - for running AI models locally within the container.
• Models:
  • Scikit-Learn (BSD) (Isolation Forest) for Anomaly Detection.
  • Quantized LLMs (e.g., Llama-3) (Community License) for the Natural Language Assistant.

Observability, tracing & ID generation

fluxrig uses a simplified observability strategy based on OpenTelemetry and the DuckLake pattern:

Tier	Backend(s)	Scale	Phase
Embedded	DuckDB + Parquet (S3/Local)	< 100M events/day	[Roadmap]
Standard	Arc + OpenSearch	Team deployments	[Roadmap]
Enterprise	ClickHouse + SigNoz	Petabyte scale	[Roadmap]

Category	Tool/Library	Type	Purpose
Telemetry SDK	OpenTelemetry Go (Apache 2.0)	Library	Metrics, Logs, & Distributed Tracing.
Query Engine	DuckDB (MIT)	OLAP	Stateless SQL engine for Metrics & Traces (Embedded).
Query Lang	PRQL (Apache 2.0)	Language	Time-series optimized query language (compiles to SQL).
Logs	OpenSearch (Apache 2.0)	Search	Evaluation Candidate for Log Search.
Analytics	ClickHouse (Apache 2.0)	Database	Evaluation Candidate for Petabyte-scale OLAP (Enterprise).
ID Generation	Sonyflake (Apache 2.0)	Library	Distributed k-sortable 64-bit IDs (pkg/idgen)
OTLP Export	OTLP	Standard	Vendor-neutral export to any OTel-compatible backend.

Ref: See Observability Architecture for detailed tier documentation.

Testing, security & compliance

Security First: fluxrig development follows strict security standards (PCI-DSS, OWASP, SSF) from the very beginning. All architectural decisions, dependency choices, and coding practices are audited to ensure compliance.

• Testing frameworks:

  • Robot framework (Apache 2.0): Keyword-driven testing for E2E and Acceptance testing.
  • Python (PSF): Scripting language for advanced test scenarios and the fluxrig SDK.
  • Testcontainers (MIT): Ephemeral Docker containers for integration testing.

• Static analysis (SAST):

  • Gosec (Apache 2.0): Go security checker (SQLi, credentials, etc.).
  • Govulncheck (BSD): Go vulnerability database checker.
  • Bandit (Apache 2.0): Python security linter.

• Container & secret security:

  • Trivy (Apache 2.0): Comprehensive container and filesystem vulnerability scanner.
  • Trufflehog (AGPL - External CLI): Scans git history for high-entropy secrets.

• Code coverage:

  • Go cover: Built-in Go test -coverprofile for tracking unit test coverage.
  • Standards:
    • Core Libraries (pkg/*): > 70% required.
    • Overall project: > 60% required.
  • Coverage.py: Code coverage measurement for Python scripts.

Utility libraries

fluxrig also utilizes these key utility libraries:

• Gin: High-performance http web framework.
• Google/uuid: Robust uuid generation.
• Gopkg.in/yaml.v3: Strict yaml marshaling/unmarshaling.

Documentation & build

The documentation site and PDF generation are powered by:

• Docusaurus: Modern static site generator (Web version).
• MkDocs: Static site generator (PDF Core version).
• Material for MkDocs: Theming and UX for the PDF pipeline.
• Mike: Versioning management for MkDocs (git-based).
• Docusaurus OpenAPI: Modern OpenAPI reference renderer (Scalar/Redocusaurus).
• Mkdocs-with-pdf: PDF generation plugin (using weasyprint).
• Mermaid: Diagramming and visualization (rendered via mmdc CLI).
• Python: Pre-processing scripts for validation and layout.