Architecture note

Ferro — models execute inside your infrastructure

Ferro is the runtime under Quorum. It loads a small language model and a local evidence index inside your own network, answers from them, and writes the decision record to your own database. No prompt, document or inference call crosses your boundary, so there is no subprocessor in the decision path to disclose or review.

What is actually running

Three processes and one store, all inside the boundary. Nothing calls out, and there is no vendor-side control plane to lose access to.

Small language model

A 7–14B instruction-tuned model, quantised to 4-bit, loaded once and held resident. It answers from retrieved evidence and is never asked to recall facts from its weights — the evidence index is the source, the model is the reader.

Local evidence index

A hybrid index — BM25 plus dense vectors — over your policy documents, ledger entries and prior cases. Built locally, stored locally, rebuilt incrementally as sources change. Documents never leave the host that indexes them.

Decision-record writer

Writes the immutable record — sources, weights, confidence, model build, reviewer, timestamp — into your database. It is a separate process from the model deliberately: a record must still be written when inference fails.

Model store

Version-pinned weights on local disk, checksummed at load. A build is either the one the record names or the runtime refuses to serve the request.

Reference configuration and its latency envelope

One node per environment. The numbers below are the reference configuration, measured, not projected — size against them and adjust for your own concurrency.

GPU

One accelerator, 48 GB VRAM. Holds the quantised model and its KV cache with headroom for eight concurrent decision sessions.

Host

128 GB system memory, 16 physical cores, local NVMe for the evidence index. The index is latency-critical and must not sit on network storage.

Median time to first token

240 ms at the reference configuration, measured at the eighth concurrent session. There is no round trip to a hosted API, so latency is a hardware question you control rather than a vendor status page you read.

CPU-only

Supported at roughly a fifth of the throughput. That suits batch review of a day of decisions and does not suit interactive use — the reviewer feels every second of it.

Deployment topologies

Three, and the difference between them is failure behaviour, not features. The decision record is identical in all three.

Single node

One host, one model, one index. The common case for a first decision surface. Inference is unavailable while the host is down; queued decisions resume, and no record is lost because the writer is a separate process.

HA pair

Two nodes, active-active, with the evidence index replicated and the record writer pointed at your existing highly available database. Loss of one node halves throughput and changes nothing else.

Air-gapped

No egress at all, including for updates. Model builds and index updates arrive as signed bundles on removable media and are verified against a published checksum before load. This is the topology the architecture was designed around; the other two are relaxations of it.

What version-pinning actually guarantees

That an audit two years from now reproduces the same output from the same inputs. Every decision record names the exact model build, the index generation and the weighting configuration that produced it. Replay a record against that triple and you get the same recommendation and the same confidence figure.

Guaranteed

Same model build, same index generation, same configuration, same inputs — same recommendation, same confidence, same reversal conditions. Sampling is off in the decision path; a decision is not a place for temperature.

Not guaranteed

Reproduction across a hardware change. Different accelerators reduce in different orders, and floating-point addition is not associative, so a replay on unlike hardware can differ in the last decimal of a weight. The record stores the hardware class for that reason. Say it plainly rather than discover it in an audit.

Where Ferro deliberately does less

A small local model is not a large hosted one, and the honest version of this page says where that costs you something.

General reasoning

A frontier hosted model reasons better on open-ended questions. Ferro is not asked open-ended questions: it reads retrieved evidence and states what it supports. Where your decision genuinely needs open-ended reasoning, Quorum can call a hosted model instead — and then you have a subprocessor to disclose.

Breadth of world knowledge

The model is not the knowledge source and is not consulted as one. If a fact is not in your evidence index, the correct output is that the evidence does not support a recommendation. That is a feature here and a limitation elsewhere.

Long-context ingestion

Effective context is smaller than a hosted frontier model. Retrieval does the work instead, which is why the index is hybrid rather than vector-only. Very long single documents are chunked with their structure preserved, and the citation points at the chunk, not the file.

Zero-touch operation

It is software in your building. Someone patches it, watches its disk, and rotates its logs. A hosted API is genuinely less work; it is also the thing you were trying not to have.