feat(speculation): generator contract and bestfirst impl

submitqueue/extension/speculation/generator/BUILD.bazel

@@ -0,0 +1,9 @@
+load("@rules_go//go:def.bzl", "go_library")
+
+go_library(
+    name = "go_default_library",
+    srcs = ["generator.go"],
+    importpath = "github.com/uber/submitqueue/submitqueue/extension/speculation/generator",
+    visibility = ["//visibility:public"],
+    deps = ["//submitqueue/entity:go_default_library"],
+)

submitqueue/extension/speculation/generator/README.md

@@ -0,0 +1,19 @@
+# generator
+
+The `generator` package is a piece the `standard` `Speculator` is built from: a `Generator` produces the queue's candidate paths as one stream, best first, across all heads. It is **not** controller-facing — the speculate controller only knows the `Speculator` contract, and a different `Speculator` need not split its work this way. So there is no `Config` or `Factory` here; a `Generator` is chosen when the `standard` `Speculator` is constructed.
+
+`Open` starts the stream over the queue's live batches and their path sets and returns a `PathIterator`. The caller pulls one candidate at a time and the generator does only the work that answer needs. Candidates descend in score, never repeat, and never contradict a known fact. A candidate's score means something only for the current run; scores are never stored.
+
+## `bestfirst`
+
+A head waiting on unfinished dependencies has one path per combination of guesses — 2ⁿ of them — while callers want the best few. `bestfirst` hands them out in score order without building the rest.
+
+Dependencies that already finished are not guesses: landed means included, failed or cancelled means excluded. They stay in the path but drop out of the search. Each unfinished one is a two-way guess whose chance of landing comes from an injected `scorer.Scorer`, remembered per dependency so one shared by several heads is scored once; a dependency that is not a live batch is treated as a coin flip. A path's score is its guesses' chances multiplied together. Heads with more unfinished dependencies than the depth bound are skipped until some resolve, and paths that already finished are passed over — so a failed path drops out and the next-best takes its place.
+
+**The best path is not "bet everything lands."** Each guess takes the *likelier* outcome, so a dependency that will probably fail is excluded. Every other path flips some of those guesses, and each flip costs a known factor. One consequence worth knowing: flipping two cheap guesses can beat flipping one expensive guess, and the ordering handles that correctly.
+
+**Laziness at two levels.** Across heads, a heap holds each head's current offer — a real, already-built path, never an estimate — so the top of the heap is the best path in the queue. Within a head the same shape repeats: it keeps a small heap, hands out its best, and builds only the one or two paths that come after it. Pulling *k* candidates builds about *k* paths, whatever the size of the set behind them.
+
+`bestfirst` discards nothing: pull long enough and every path within the depth bound comes out. It does no conflict relaxation (`dropped` bets) — that is the `Speculator`'s call, not the generator's.
+
+The ordering trick behind all this — how the walk reaches every path exactly once, in score order, without tracking what it has already seen — is documented with a worked example on `expand` in `bestfirst.go`. The behavior is pinned by tests in `bestfirst_test.go`.

submitqueue/extension/speculation/generator/bestfirst/BUILD.bazel

@@ -0,0 +1,26 @@
+load("@rules_go//go:def.bzl", "go_library", "go_test")
+
+go_library(
+    name = "go_default_library",
+    srcs = ["bestfirst.go"],
+    importpath = "github.com/uber/submitqueue/submitqueue/extension/speculation/generator/bestfirst",
+    visibility = ["//visibility:public"],
+    deps = [
+        "//submitqueue/entity:go_default_library",
+        "//submitqueue/extension/scorer:go_default_library",
+        "//submitqueue/extension/speculation/generator:go_default_library",
+    ],
+)
+
+go_test(
+    name = "go_default_test",
+    srcs = ["bestfirst_test.go"],
+    embed = [":go_default_library"],
+    deps = [
+        "//submitqueue/entity:go_default_library",
+        "//submitqueue/extension/scorer:go_default_library",
+        "//submitqueue/extension/speculation/generator:go_default_library",
+        "@com_github_stretchr_testify//assert:go_default_library",
+        "@com_github_stretchr_testify//require:go_default_library",
+    ],
+)