feat: capability extensions — evaluatePermissions, MCP verdict tools, pagination & rerank docs

.changeset/evaluate-permissions.md

@@ -0,0 +1,5 @@
+---
+"@refkit/core": minor
+---
+
+Export evaluatePermissions/PermissionKey/EvaluateOptions — programmable strict-deny gate; evaluateUse intents are now presets over it (behavior unchanged).

.changeset/mcp-verdict-tools.md

@@ -0,0 +1,14 @@
+---
+"@refkit/mcp": minor
+---
+
+New stateless `evaluate_use` and `build_attribution` MCP tools — evaluate a
+license against an intended use, or build an attribution credit line, without a
+search round-trip. Zero-config `defaultProviders` now reads unified
+`REFKIT_<PROVIDER>_KEY` env names first (`REFKIT_UNSPLASH_KEY`,
+`REFKIT_PEXELS_KEY`, `REFKIT_PIXABAY_KEY`, `REFKIT_FLICKR_KEY`,
+`REFKIT_SMITHSONIAN_KEY`, `REFKIT_BRAVE_KEY`, `REFKIT_FREESOUND_KEY`,
+`REFKIT_JAMENDO_CLIENT_ID`, `REFKIT_EUROPEANA_KEY`), falling back to the legacy
+names (`UNSPLASH_KEY`, `PEXELS_KEY`, `PIXABAY_KEY`, `FLICKR_KEY`, `SI_KEY`,
+`BRAVE_TOKEN`, `FREESOUND_TOKEN`, `JAMENDO_CLIENT_ID`, `EUROPEANA_KEY`), which
+are still honored.

README.md

@@ -108,6 +108,18 @@ console.log(meta.providers)
 console.log(meta.warnings)
 ```
 
+`controls.page` is a **provider-local** cursor: each provider paginates its own result stream independently, and refkit does not track a unified offset across sources. Because pages are fused with Reciprocal Rank Fusion per request, page N+1 is not guaranteed to be disjoint from page N — results can overlap or shift relative to the previous page. For a "load more" UI, dedupe across pages by `canonicalUrl` rather than assuming stable, non-overlapping windows:
+
+```ts
+import { canonicalizeUrl } from '@refkit/core'
+
+const seen = new Set(prev.map((r) => canonicalizeUrl(r.canonicalUrl)))
+const nextPage = await refkit.search({ query, modalities: ['image'], controls: { page: 2 } })
+const newOnly = nextPage.filter((r) => !seen.has(canonicalizeUrl(r.canonicalUrl)))
+```
+
+(core's own merge/dedup normalizes URLs the same way, so this recipe stays consistent with what refkit dedupes internally.)
+
 ## Ranking & rerank
 
 By default, results are fused across sources with **Reciprocal Rank Fusion** — cross-source-orderable, but not query-aware. For sharper relevance, pass a **reranker**:
@@ -229,7 +241,7 @@ Agents can use refkit in two ways:
 npx -y @refkit/mcp
 ```
 
-It boots with the keyless sources (Met, Art Institute, Wikimedia, Openverse, Project Gutenberg, PoetryDB, Rijksmuseum, Poly Haven, ambientCG, Internet Archive) and auto-adds any BYOK source whose key is in the environment (`UNSPLASH_KEY`, `PEXELS_KEY`, `BRAVE_TOKEN`, …). Pass `intent` to annotate each result with a use-verdict (may I use this, is attribution required); `gateFor` to return only allowed results. Or wire your own providers/keys via `serveStdio(createRefkit({ … }))` — see [`@refkit/mcp`](https://www.npmjs.com/package/@refkit/mcp).
+It boots with the keyless sources (Met, Art Institute, Wikimedia, Openverse, Project Gutenberg, PoetryDB, Rijksmuseum, Poly Haven, ambientCG, Internet Archive) and auto-adds any BYOK source whose key is in the environment (`REFKIT_UNSPLASH_KEY`, `REFKIT_PEXELS_KEY`, `REFKIT_BRAVE_KEY`, … — legacy names like `UNSPLASH_KEY`, `PEXELS_KEY`, `BRAVE_TOKEN` still work as fallbacks). Pass `intent` to annotate each result with a use-verdict (may I use this, is attribution required); `gateFor` to return only allowed results. Beyond search, `evaluate_use` and `build_attribution` expose the same license-verdict and attribution logic as standalone stateless tools, for when an agent already has a license id and just needs a verdict or a credit line. Or wire your own providers/keys via `serveStdio(createRefkit({ … }))` — see [`@refkit/mcp`](https://www.npmjs.com/package/@refkit/mcp).
 
 ## Not legal advice
 

docs/examples/semantic-rerank.md

@@ -0,0 +1,98 @@
+# Cookbook: BYO-embedding semantic reranker
+
+refkit ships no embedding model — [`lexicalReranker`](../../README.md#ranking--rerank) (term-coverage +
+resolution + license weighting) remains the zero-dep default. This recipe wires a
+`Reranker` to a host-provided embeddings endpoint for query-aware semantic ranking.
+
+Imports come only from `@refkit/core`; the embeddings call is the one intentional
+seam to your own backend.
+
+```ts
+import type { Reranker, RerankInput, Reference } from '@refkit/core'
+
+/** Host-provided endpoint: POST { input: string[] } -> { embeddings: number[][] },
+ *  one embedding per input string, same order. Swap in your own provider. */
+async function fetchEmbeddings(
+  texts: string[],
+  signal?: AbortSignal,
+): Promise<number[][]> {
+  const res = await fetch('https://your-embeddings-host.example/v1/embed', {
+    method: 'POST',
+    headers: { 'content-type': 'application/json' },
+    body: JSON.stringify({ input: texts }),
+    signal,
+  })
+  if (!res.ok) throw new Error(`embeddings request failed: ${res.status}`)
+  const { embeddings } = (await res.json()) as { embeddings: number[][] }
+  return embeddings
+}
+
+function dot(a: number[], b: number[]): number {
+  let sum = 0
+  for (let i = 0; i < a.length; i++) sum += a[i] * b[i]
+  return sum
+}
+
+function norm(a: number[]): number {
+  return Math.sqrt(dot(a, a))
+}
+
+/** Cosine similarity in [-1, 1]; 0 when either vector is zero-length. */
+function cosineSimilarity(a: number[], b: number[]): number {
+  const denom = norm(a) * norm(b)
+  return denom === 0 ? 0 : dot(a, b) / denom
+}
+
+function refText(ref: Reference): string {
+  return `${ref.title ?? ''} ${ref.text?.excerpt ?? ''}`.trim()
+}
+
+/** Semantic reranker: embeds the query and each ref's title+excerpt, scores by
+ *  cosine similarity, sorts descending, and rewrites `relevance` to a normalized
+ *  0..1 score. Preserves every `referenceSchema` invariant — refs are copied,
+ *  never mutated, and none are dropped or fabricated. */
+export function semanticReranker(): Reranker {
+  return async ({ query, refs, signal }: RerankInput): Promise<Reference[]> => {
+    if (refs.length === 0) return []
+
+    const vectors = await fetchEmbeddings([query, ...refs.map(refText)], signal)
+    const expected = 1 + refs.length
+    if (vectors.length !== expected) {
+      throw new Error(`embeddings count mismatch: got ${vectors.length}, expected ${expected}`)
+    }
+    const [queryVec, ...refVecs] = vectors
+
+    const scored = refs.map((ref, i) => ({
+      ref,
+      score: cosineSimilarity(queryVec, refVecs[i]),
+    }))
+    scored.sort((a, b) => b.score - a.score)
+
+    // Cosine similarity is [-1, 1]; normalize to referenceSchema's required 0..1.
+    return scored.map(({ ref, score }) => ({
+      ...ref, // copy — never mutate the input ref in place
+      relevance: (score + 1) / 2,
+    }))
+  }
+}
+```
+
+Usage:
+
+```ts
+import { createRefkit } from '@refkit/core'
+import { semanticReranker } from './semantic-reranker'
+
+const refkit = createRefkit({ providers: [/* ... */] })
+
+const refs = await refkit.search({
+  query: 'cyberpunk alley at night',
+  modalities: ['image'],
+  rerank: semanticReranker(),
+})
+```
+
+**Invariants a custom `Reranker` must preserve** (see `Reranker` in `packages/core/src/rerank.ts`):
+copy each `Reference` rather than mutating it in place, keep `relevance` within `0..1`,
+and return a reorder/subset of the input — no dropped required fields, no duplicated or
+fabricated refs.

docs/superpowers/plans/2026-07-03-capability-extensions.md

@@ -0,0 +1,68 @@
+# Wave 4 — Capability Extensions Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: superpowers:subagent-driven-development or executing-plans, task-by-task.
+
+**Goal:** (4a) export a programmable low-level `evaluatePermissions` with `evaluateUse` refactored into behavior-identical presets; (4b) document pagination semantics; (4c) stateless `evaluate_use` + `build_attribution` MCP tools and unified `REFKIT_<PROVIDER>_KEY` env names with legacy fallbacks; (4d) a runnable semantic-rerank cookbook. Branch: `m13t/wave4-capability-extensions` (off main).
+
+**Hard constraint for 4a:** ALL existing evaluate-use tests pass UNCHANGED — the refactor is proven behavior-identical by the existing suite (reason strings included).
+
+---
+
+### Task W4.1: core — `evaluatePermissions` (4a)
+
+**Files:** modify `packages/core/src/evaluate-use.ts`, `packages/core/src/index.ts`; test `packages/core/src/__tests__/evaluate-use.test.ts` (ADD only).
+
+Design (implementer adapts internals so existing tests stay green verbatim):
+
+```ts
+/** The tri-state permission axes of LicenseFacts. */
+export type PermissionKey = 'commercialUse' | 'derivatives' | 'redistribution'
+
+export interface EvaluateOptions {
+  /** Treat editorial-only sources as denied (commercial-flavored uses). Default false. */
+  denyEditorialOnly?: boolean
+  /** Enforce attributionRequired as allowed-with-attribution. Default true; presets
+   *  disable it for internal-moodboard (note-only). */
+  enforceAttribution?: boolean
+}
+
+/** Low-level, programmable strict-deny gate: unknown license → needs-review;
+ *  jurisdiction mismatch → needs-review; each required permission false → denied,
+ *  'unknown' → needs-review; else allowed(-with-attribution). evaluateUse's four
+ *  intents are presets over this. */
+export function evaluatePermissions(
+  r: RightsRecord,
+  required: readonly PermissionKey[],
+  ctx?: { userJurisdiction?: string },
+  opts?: EvaluateOptions,
+): Verdict
+```
+
+`evaluateUse` becomes: map intent → (required, opts) preset and delegate; keep the moodboard "attribution required by license but not enforced" reason note and every existing reason string byte-identical (existing tests are the oracle). Export `evaluatePermissions`, `PermissionKey`, `EvaluateOptions` from core index. New tests (ADD): custom permission set `['derivatives']` on CC-BY-ND → denied naming CC-BY-ND; empty required on CC-BY with enforceAttribution true → allowed-with-attribution; unknown license → needs-review regardless of required; parity spot-check `evaluatePermissions(rec, ['commercialUse'], undefined, {denyEditorialOnly:true})` equals `evaluateUse(rec,'commercial-product')` for a few licenses. Changeset: `.changeset/evaluate-permissions.md` — `"@refkit/core": minor` — "Export evaluatePermissions/PermissionKey/EvaluateOptions — programmable strict-deny gate; evaluateUse intents are now presets over it (behavior unchanged)." Verify: full repo green (304/22). Commit `feat(core): programmable evaluatePermissions; evaluateUse intents become presets`.
+
+### Task W4.2: MCP — stateless verdict tools + env unification (4c)
+
+**Files:** modify `packages/mcp/src/index.ts`, `packages/mcp/src/cli.ts`, `packages/mcp/src/__tests__/mcp.test.ts`, `packages/mcp/README.md` (if it documents env vars — check), changeset.
+
+1. Register two tools in `createRefkitMcpServer` (stateless — inputs carry the rights fields; no session cache):
+   - `evaluate_use`: input { license: z.enum(LICENSE_IDS from core), licenseVersion?, author?, title?, canonicalUrl: string, intent: z.enum(INTENTS), editorialOnly?: boolean, jurisdiction?: string, userJurisdiction?: string }. Build a RightsRecord (rehostPolicy 'cache-allowed', raw { sourceTerms: '', sourceUrl: canonicalUrl } — verdict doesn't read them), call core `evaluateUse(rights, intent, { userJurisdiction })`; output { decision, reasons, confidence, disclaimer, attribution? (text when verdict allows-with-attribution — via buildAttribution) }. Description: mirrors search_references' "not legal advice" framing.
+   - `build_attribution`: input { license, licenseVersion?, author?, title?, canonicalUrl } → core buildAttribution output { required, text?, html? }.
+2. cli.ts `defaultProviders`: unified names first with legacy fallback — `REFKIT_UNSPLASH_KEY ?? UNSPLASH_KEY`, `REFKIT_PEXELS_KEY ?? PEXELS_KEY`, `REFKIT_PIXABAY_KEY ?? PIXABAY_KEY`, `REFKIT_FLICKR_KEY ?? FLICKR_KEY`, `REFKIT_SMITHSONIAN_KEY ?? SI_KEY`, `REFKIT_BRAVE_KEY ?? BRAVE_TOKEN`, `REFKIT_FREESOUND_KEY ?? FREESOUND_TOKEN`, `REFKIT_JAMENDO_CLIENT_ID ?? JAMENDO_CLIENT_ID`, `REFKIT_EUROPEANA_KEY ?? EUROPEANA_KEY`. Comment documents the convention; `defaultProviders` is already unit-tested — extend its tests to cover a REFKIT_-name and a legacy-name both working.
+3. Tests: tool-level tests for both new tools (CC-BY → allowed-with-attribution + credit line; unknown → needs-review; CC0 → attribution not required). Follow mcp.test.ts's existing server-invocation pattern.
+4. Changeset `"@refkit/mcp": minor`: new tools + env aliases (legacy names still honored).
+5. Root README MCP section: mention the two new tools in one sentence; keep the env example working.
+
+Verify: full repo green. Commit `feat(mcp): stateless evaluate_use + build_attribution tools; REFKIT_* env aliases`.
+
+### Task W4.3: docs — pagination semantics + rerank cookbook (4b, 4d)
+
+**Files:** modify `packages/core/src/provider.ts` (SearchControls.page JSDoc), root `README.md` (Search controls section), create `docs/examples/semantic-rerank.md`.
+
+1. `SearchControls.page` JSDoc: `/** Provider-local page cursor: each provider paginates its own result stream; after RRF merging, page N+1 may overlap or shift relative to page N. For UI "load more", dedupe across pages by canonicalUrl (see README). */` README Search controls section: 3-sentence paragraph stating the same + a dedupe recipe snippet (`const seen = new Set(prev.map(r => r.canonicalUrl))` filter).
+2. `docs/examples/semantic-rerank.md`: runnable BYO-embedding `Reranker` — fetch embeddings from a host-provided endpoint for `[query, ...refs.map(title+excerpt)]`, cosine similarity, sort desc, rewrite relevance to normalized score, note on preserving referenceSchema invariants (0..1, no mutation — copy refs), and a one-line pointer that lexicalReranker remains the zero-dep default. TypeScript, imports from '@refkit/core' only, ~60 lines, self-contained.
+
+No changeset (docs only). Verify repo green. Commit `docs: pagination semantics + semantic-rerank cookbook`.
+
+---
+
+**Wave finish (coordinator):** final whole-wave review → push → PR to main. Changesets: core minor + mcp minor land in Tasks 1–2.

packages/core/src/__tests__/evaluate-use.test.ts

@@ -1,5 +1,5 @@
 import { describe, expect, it } from 'vitest'
-import { evaluateUse, NOT_LEGAL_ADVICE } from '../evaluate-use'
+import { evaluatePermissions, evaluateUse, NOT_LEGAL_ADVICE } from '../evaluate-use'
 import type { RightsRecord } from '../rights'
 import { LICENSE_FACTS } from '../license'
 import type { LicenseId } from '../license'
@@ -102,3 +102,46 @@ describe('evaluateUse — strict-deny', () => {
     expect(evaluateUse(rec('CC-BY-ND'), 'redistribution').decision).toBe('allowed-with-attribution')
   })
 })
+
+describe('evaluatePermissions — programmable strict-deny gate', () => {
+  it('custom permission set: derivatives-only on CC-BY-ND → denied, naming CC-BY-ND', () => {
+    const v = evaluatePermissions(rec('CC-BY-ND'), ['derivatives'])
+    expect(v.decision).toBe('denied')
+    expect(v.reasons.join(' ')).toContain('CC-BY-ND')
+  })
+
+  it('empty required set on CC-BY with enforceAttribution (default true) → allowed-with-attribution', () => {
+    const v = evaluatePermissions(rec('CC-BY'), [])
+    expect(v.decision).toBe('allowed-with-attribution')
+  })
+
+  it('unknown license → needs-review regardless of required permissions', () => {
+    const v = evaluatePermissions(rec('unknown'), ['commercialUse', 'derivatives', 'redistribution'])
+    expect(v.decision).toBe('needs-review')
+    expect(v.confidence).toBe('low')
+  })
+
+  it('preset parity: evaluatePermissions(rec, [commercialUse], undefined, {denyEditorialOnly:true, label}) equals evaluateUse(rec, commercial-product)', () => {
+    const licenses: LicenseId[] = ['CC0-1.0', 'CC-BY', 'CC-BY-NC', 'CC-BY-ND', 'proprietary', 'unknown']
+    for (const license of licenses) {
+      const viaPermissions = evaluatePermissions(rec(license), ['commercialUse'], undefined, { denyEditorialOnly: true, label: 'commercial-product' })
+      const viaUse = evaluateUse(rec(license), 'commercial-product')
+      expect(viaPermissions.decision).toBe(viaUse.decision)
+      expect(viaPermissions.reasons).toEqual(viaUse.reasons)
+    }
+  })
+
+  it('evaluateUse success reasons name the intent (historical wording)', () => {
+    expect(evaluateUse(rec('CC0-1.0'), 'commercial-product').reasons).toContain('permitted for commercial-product under CC0-1.0')
+    expect(evaluateUse(rec('CC0-1.0'), 'internal-moodboard').reasons).toContain('permitted for internal-moodboard under CC0-1.0')
+  })
+
+  it('standalone evaluatePermissions defaults the label to the permission set', () => {
+    expect(evaluatePermissions(rec('CC0-1.0'), ['commercialUse']).reasons).toContain('permitted for commercialUse under CC0-1.0')
+  })
+
+  it('lenient-attribution note interpolates the custom label', () => {
+    const v = evaluatePermissions(rec('CC-BY'), [], undefined, { enforceAttribution: false, label: 'archival-review' })
+    expect(v.reasons.some(r => r.includes('not enforced for archival-review use'))).toBe(true)
+  })
+})