Topology: YAML topology files and ordered gateway tiers#679
Open
thiell wants to merge 4 commits into
Open
Conversation
Turn TopologyParser into a thin facade over an internal ConfigParser and an intermediate route list, so that a second configuration format can be added. No behavior change. Signed-off-by: Stephane Thiell <stephane@thiell.com>
Add a YAML topology file syntax, selected by .yaml/.yml file extension. Routes are a list of gateways/targets objects and the same gateways may appear in several routes. topology.yaml is preferred over topology.conf at each config location. INI stays fully supported. PyYAML is imported lazily, as for YAML group files. Signed-off-by: Stephane Thiell <stephane@thiell.com>
In a YAML topology file, the gateways of a route may be a list of entries with nodes, weight and tier attributes. A tier is used only when all gateways in lower tiers are unreachable. Within a tier, the load is shared using weighted least-connections. Resolved tiers ride the propagation tree sent to gateways; older trees without tiers are still accepted. Signed-off-by: Stephane Thiell <stephane@thiell.com>
Document the topology.yaml syntax, ordered gateway tiers and discovery precedence in config.rst and clush.rst. INI topology.conf remains documented as fully supported. Signed-off-by: Stephane Thiell <stephane@thiell.com>
thiell
force-pushed
the
feature/gh374-topology-yaml
branch
from
July 19, 2026 15:27
feacb1d to
63b60a9
Compare
This was referenced Jul 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the design discussed in #374 (see the design comment). Closes #374.
YAML topology files
topology.yamlfile syntax: routes as a list ofgateways/targetsobjects. Selected by file extension (.yaml/.yml), so--topologyworks with both syntaxes; INItopology.confremains fully supported.gatewaysmay appear in several routes: one gateway pool can reach several target groups, with symmetric failback — the core need from the PR Tree: support optional route weights in topology.conf (#374) #678 review.topology.yamlis preferred overtopology.confwhen both exist at the same config location.TopologyParserbecomes a thin facade over per-format backends; graph building and validation are unchanged and format-agnostic. The INI code path is untouched and still covered by the existing tests.install_requiresalready pulls PyYAML for pip installs, so this mostly matters for distro packages.Ordered gateway tiers
gatewaysof a route may be a list of entries with descriptive attributes, in the style of modern YAML tools such as lnetctl:nodes(node set, required),weight(load share within a tier, default 1) andtier(failover rank, default 1). A tier is used only when all gateways in lower tiers are unreachable:gateways: [gw1, gw2]) is a load-shared pool, equivalent togateways: gw[1-2]— failover is always explicit withtier, and the entry form keeps the schema extensible for future per-gateway attributes.weight(integers >= 1). Weight 0 is rejected: a standby gateway is a higher tier, so there is only one mechanism.TopologyTree; trees from older versions are still accepted, and the wire protocol is unchanged.Docs and tests
config.rst/clush.rstdocument the YAML syntax, quoting rules (@groupvalues must be quoted), precedence and tiers;conf/topology.yaml.exampleadded.tests/TreeRouterTest.pycovers gateway selection: tier order, failover, mutual failback, weighted distribution, pool equivalence of plain lists, multi-level trees, pickled-tree round-trip, pre-1.11 tree compatibility. YAML parser tests added toTreeTopologyTest, including INI/YAML tree equivalence.Supersedes PR #678 (now a draft): the INI
[weights]section is not kept, and its weighted least-connections selection is reused as the within-tier policy.