Architecture

Covered in this page

This page covers how the Ootle network is put together:

The four node roles and which of them vote.
How the state space is sharded, and what a shard group is.
A worked example of how 256 shards land in committees.
How committees are formed and how validators are assigned to them.
How epochs rotate membership.
The peer-to-peer transport and message routing.

Node roles

Only one role produces finality. The rest read, index, or serve.

Node	Votes?	Responsibility
Validator node	Yes	Participates in HotStuff consensus and executes transactions for its shard group.
Indexer	No	Indexes blocks, transactions, substates, and events; serves queries over REST/JSON-RPC.
Wallet daemon	No	Custodies keys, constructs and signs transactions, and submits them via the indexer.
Base-layer node	No	An upstream Minotari node. Ootle reads validator registrations and epoch data from it.

A validator node runs three things at once: the consensus protocol, the execution engine (TVM), and a connection to the base layer through the epoch oracle (crates/epoch_oracles/src/base_layer/oracle.rs). The oracle is the seam between the two layers — it is what turns “who registered on the base layer” into “who may vote in this epoch.”

Sharding

Ootle shards the state space, not the contracts. Every possible substate has a deterministic address in a 256-bit space, and the network divides that space into fixed slices. That one choice has three consequences which shape everything below:

A component lives in exactly one shard. Its address is derived from its identity, not assigned on demand, so a transaction touching only that component is routed to exactly one committee.
Templates are never sharded. A published template is a global substate in shard 0, so every committee can resolve it. Sharding the code would mean replicating it per shard for no benefit.
A transaction that spans shards pays a coordination cost. It must be sequenced by each involved committee and bound together with pledges — the mechanism in Consensus.

Preshards and shards

The address space is carved into a configurable number of preshards. There are 256 of them:

pub enum NumPreshards {
    P1 = 1, P2 = 2, P4 = 4, P8 = 8, P16 = 16,
    P32 = 32, P64 = 64, P128 = 128, P256 = 256,
}
impl NumPreshards {
    pub const MAX: Self = Self::P256;
    pub const fn current() -> Self { Self::P256 }
}

A shard is one preshard. The number of shards is the number of preshards — 256 — and shards are numbered 1 to 256 (all_shards_iter yields 1..=as_u32()). Shard numbering is 1-indexed.

Shard 0 is reserved as the global shard (crates/common_types/src/shard.rs): it holds global substates. The clearest member of that set is a published template, which is a global resource available to every committee. A substate is global when SubstateId::is_global() is true, which is exactly the case for templates (crates/engine_types/src/substate.rs).

Shard groups

Committees do not each own a single shard; they own a contiguous run of them — a shard group (crates/common_types/src/shard_group.rs), modelled as an inclusive range start..=end_inclusive. A shard group promises start <= end_inclusive, and a decoded value that violates the ordering is a decode error rather than an invalid group.

The 256 shards are distributed across the active committees as evenly as possible:

pub fn all_shard_groups_iter(&self, num_committees: u32) -> impl Iterator<Item = ShardGroup> {
    let num_shards = self.as_u32();
    let num_shards_per_committee = num_shards / num_committees;
    let mut remainder = num_shards % num_committees;
    // ... spreads the remainder across the leading groups, 1-indexed
}

The number of committees scales with how many validators are registered:

fn calculate_num_committees(num_vns: u64, committee_size: NonZeroU32) -> u32 {
    // Number of committees is proportional to the number of validators available.
    cmp::min(cmp::max(1, num_vns / u64::from(committee_size.get())), u64::from(u32::MAX)) as u32
}

From an object key to a committee

Four words get used loosely here, and they are worth keeping apart: a preshard and a shard are the same thing (one of the 256 slices of the address space); a shard group is a contiguous run of shards; and a committee is the set of validators that administer one group. A substate belongs to exactly one shard, a shard to exactly one group, and a validator to exactly one committee.

A worked example

Take committee_size_per_shard_group = 40 — the value on the live networks — and 160 registered validators:

num_committees   = 160 / 40 = 4
shards_per_group = 256 / 4  = 64        (no remainder)

Four committees, each owning a contiguous block of 64 shards: 1–64, 65–128, 129–192, 193–256. When the count does not divide evenly, the remainder is spread across the leading groups — one extra shard to each of the first remainder groups, in order. Three committees give $256 = 3 \times 85 + 1$ , so the split is 86 + 85 + 85; six give $256 = 6 \times 42 + 4$ , so 43 + 43 + 43 + 43 + 42 + 42.

Two consequences are easy to miss:

A validator serves exactly one shard group. It cannot propose for, or vote on, state outside its group’s address range. Consensus is therefore not one protocol running once; it is one instance per committee, running in parallel over disjoint state. That is why the Consensus page always speaks of a committee, in the singular.
Templates belong to no committee. They are global substates in shard 0, which is exactly what lets every committee resolve a template in order to execute it. Shard 0 sits outside the 1–256 groups because it is reserved — and that reservation is why shard numbering starts at 1.

Committees

A committee is the set of validators responsible for one shard group. It is the unit of consensus: HotStuff runs among the members of a committee, and a quorum of that committee finalizes its blocks.

The committee size is a per-network constant:

Network	`committee_size_per_shard_group`
Mainnet	40
Esmeralda (testnet)	40
Testnet	40
Devnet	7 (`DEFAULT_DEVNET_COMMITTEE_SIZE`)

Source: crates/consensus/src/consensus_constants.rs. Note that this is a per-shard-group constant, and the number of groups is derived from the validator count — so these are not the “committee sizes” of a single global committee.

How a validator lands in a shard group

Membership is deterministic, not random, but on a real network the shard key is computed on the base layer and delivered with the registration — Ootle does not derive it:

ValidatorNodeChange::Add {
    // ...
    shard_key: {
        let hash = Hash32::try_from(add.shard_key.as_slice())?;
        SubstateAddress::from_hash_and_version(hash, 0)
    },
}

Minotari derives that value from the validator node’s public key and a base-layer block hash, and re-rolls it periodically, so a validator’s shard key — and the shard group it falls into — carries base-layer entropy and can change between registrations. A validator’s claim key (also called a registration key) is the credential it proves on the base layer when it registers; rotating it is what re-clusters that validator.

Local swarms and tests have no base layer to draw from, so the configured epoch oracle stands in for it: it derives a shard key from the config file, hashing the validator’s own registration fields into the low bytes of its shard group’s address range (crates/epoch_oracles/src/configured/config.rs).

Epochs

Time on Ootle is measured in epochs. An epoch is a fixed number of base-layer blocks, so the epoch clock is really the base layer’s clock — Ootle reads the base-layer epoch length through the oracle (crates/epoch_oracles/src/base_layer/oracle.rs) rather than defining its own.

At each epoch boundary the committee composition is recomputed from the current set of registered validators. Members whose registration has lapsed or who have been evicted drop out; newly registered validators join. Because a validator’s shard key can change — the base layer re-rolls it periodically, and in the dev config oracle it is a function of the registration epoch — a validator can move to a different shard group over time.

The number of epochs a transaction may remain valid for is bounded separately by max_transaction_validity_epochs in crates/consensus/src/consensus_constants.rs — a transaction carries the last epoch it may be sequenced in, and past that it can never land.

The peer-to-peer layer

Ootle nodes talk over libp2p (networking/swarm/src/behaviour.rs). The transport stack is:

TCP with Noise for encryption and authentication (with a network-specific prologue), and yamux for stream multiplexing.
QUIC as an alternative transport.
A relay client so nodes behind NAT can still be reached.

Messages ride on gossip topics. Both topics are deliberately single and network-wide:

Topic	Carries
`transactions`	Submitted transactions, delivered to every validator.
`consensus`	Consensus messages — proposals, votes, certificates — delivered to every validator.

They are defined in crates/p2p/src/gossip.rs and applications/tari_validator_node/src/p2p/services/consensus_gossip/service.rs respectively.

Scoping the topics per shard group would tear the gossipsub mesh down every time the network reshards: validators unsubscribe and resubscribe at the epoch boundary, and each new topic’s mesh has to be rebuilt from scratch, leaving the network poorly connected until it re-forms — exactly when a fresh epoch’s traffic starts flowing. One network-wide topic survives resharding untouched, and it keeps the whole validator set in a single well-connected mesh, which matters because shard groups have to communicate across shard boundaries anyway. It also means indexers (applications/tari_indexer/src/transaction_gossip/service.rs), which subscribe to the same transaction topic, see every transaction without tracking shard assignments.

Network-wide propagation is not the same as network-wide work: every validator receives a gossiped transaction, and each one checks whether it holds any of the inputs (is_involved, mempool/service.rs:298) before passing it to consensus. Nodes outside the involved committees drop it. The router can carry prefixed topics (for example transactions-0-15), dispatching on the segment before the first delimiter and falling back to the whole topic when there is none (networking/core/src/message_mode.rs); Ootle’s own topics carry no such prefix.

A transaction whose inputs span shard groups therefore already reaches every committee by the time it is gossiped — see cross-shard transaction for how those committees then coordinate.

Summary

Four node roles; only validators vote.
State is sharded into 256 preshards (shards 1–256), with shard 0 reserved for global substates such as templates.
Shards are grouped into contiguous shard groups, one per committee. A validator serves exactly one group, so consensus runs once per committee over disjoint state.
Committee size is a constant (40 on live networks, 7 on devnet); the number of committees scales with the validator count, and a non-dividing remainder is spread across the leading groups.
Membership is deterministic — no VRF, no sortition — and the shard key itself is computed on the base layer (a dev-only config oracle stands in for local swarms).
Epochs rotate membership, driven by the base layer’s block height.
Nodes communicate over libp2p with network-wide gossip topics.