From 6308a513aee7e046abc8ae87899fc90ac6cdd20b Mon Sep 17 00:00:00 2001 From: Kris Zyp Date: Mon, 13 Jul 2026 11:24:41 -0600 Subject: [PATCH 1/9] docs(security): document the secrets store, operations, and deploy_component registryAuth Adds v5 reference documentation for the encrypted secrets store (#1550/#1554/ #1582) and the deploy_component private-registry auth path (#1717), none of which was previously documented. - reference/security/secrets.md (new): store model, custody (Pro) vs OSS core, the two delivery tiers (process.env global vs scoped grants), the config.yaml `env:` declaration block, the `import { secrets } from 'harper'` accessor, full client-side `enc:v1:` envelope format + Node reference client, and the threat model. Added to the Security sidebar. - reference/operations-api/operations.md: new Secrets section (set_secret, grant_secret, revoke_secret, list_secrets, delete_secret, get_secrets_public_key) plus a registryAuth subsection under deploy_component. - reference/components/javascript-environment.md: add `secrets` to the Harper API globals list. - reference/environment-variables/overview.md: cross-link to the secrets store as the production alternative to a committed .env. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../components/javascript-environment.md | 12 + reference/environment-variables/overview.md | 4 + reference/operations-api/operations.md | 108 ++++++++ reference/security/secrets.md | 233 ++++++++++++++++++ sidebarsReference.ts | 5 + 5 files changed, 362 insertions(+) create mode 100644 reference/security/secrets.md diff --git a/reference/components/javascript-environment.md b/reference/components/javascript-environment.md index b9aecda3e..f25a4100b 100644 --- a/reference/components/javascript-environment.md +++ b/reference/components/javascript-environment.md @@ -133,6 +133,18 @@ See [Logging API](../logging/api.md) for full reference. A `Map` of all resources registered on this Harper server, keyed by their URL path. Each entry contains the resource class, path, and routing metadata. Use this to look up or enumerate registered resources programmatically. +### `secrets` + +A read-only, name→value map of the scoped secrets granted to the current component (plus any `process.env`-resolved values it declared). Read secrets from the encrypted [secrets store](../security/secrets.md) without them landing in `process.env`: + +```javascript +import { secrets } from 'harper'; + +const { STRIPE_KEY } = secrets; +``` + +A module-top-level destructure is the recommended idiom — it binds correctly under every component loader. See [Secrets](../security/secrets.md) for full reference. + ### `config` The configuration object for the current application, as provided by the component's configuration (e.g. `config.yaml`). Defaults to an empty object if no configuration is provided. diff --git a/reference/environment-variables/overview.md b/reference/environment-variables/overview.md index d30d004b6..eb299e181 100644 --- a/reference/environment-variables/overview.md +++ b/reference/environment-variables/overview.md @@ -13,6 +13,10 @@ Harper supports loading environment variables in Harper applications `process.en If you are looking for information on how to configure your Harper installation using environment variables, see [Configuration](../configuration/overview.md) section for more information. ::: +:::tip +For production credentials, prefer the encrypted, replicated [secrets store](../security/secrets.md) over a committed `.env` file. Secrets are stored as ciphertext, delivered to components via `process.env` or a per-component `secrets` accessor, and never appear in operation logs or replication payloads. The same `enc:v1:` envelope format also lets you encrypt individual `.env` values at rest. +::: + ## Configuration | Option | Type | Required | Description | diff --git a/reference/operations-api/operations.md b/reference/operations-api/operations.md index e958e8817..3114d1750 100644 --- a/reference/operations-api/operations.md +++ b/reference/operations-api/operations.md @@ -606,6 +606,29 @@ Additional parameters: - `urlPath` — override the HTTP URL path the component is mounted at (e.g. `"/api/v2"`) - `install_allow_scripts` — set to `true` to allow npm pre/post install scripts (disabled by default) +- `registryAuth` — credentials for installing a component from a private npm registry (see below) + +#### Private-registry authentication (`registryAuth`) + +When a component is installed from a private npm registry, `registryAuth` supplies the credential. It is an array of entries, each naming a `registry` and providing the credential exactly one of two ways: + +| Field | Description | +| ---------- | --------------------------------------------------------------------------------------------- | +| `registry` | The registry URL or host the credential applies to. **Required.** | +| `token` | A literal auth token, or | +| `secret` | The name of an [`hdb_secret`](../security/secrets.md) row to resolve the token from. | +| `scope` | Optional npm `@scope` (e.g. `"@my-org"`) the entry applies to; omit to set the default registry. | + +A provided **`token`** is not treated as ephemeral: Harper ingests it into the encrypted [secrets store](../security/secrets.md) and references it everywhere, so package-reference deploys keep working through rollback, reboot, and new peers joining — without re-supplying the token. The token is encrypted at rest, stripped from the operation before replication and from the operations log, and only ever crosses the cluster as ciphertext. Using a **`secret`** reference names an existing store row directly. Ingesting a token requires custody on the deploying node; on OSS core without custody, a literal token falls back to a transient, this-node-only credential (not persisted or replicated). + +```json +{ + "operation": "deploy_component", + "project": "my-app", + "package": "npm:@my-org/my-app@1.2.3", + "registryAuth": [{ "registry": "https://registry.my-org.com", "token": "npm_...", "scope": "@my-org" }] +} +``` The response includes a `deployment_id` that can be used to query the deployment record: @@ -724,6 +747,91 @@ Adds an SSH key (must be ed25519) for authenticating deployments from private re --- +## Secrets + +Operations for managing the encrypted [secrets store](../security/secrets.md) (`system.hdb_secret`). All secret operations are **`super_user` only**. Values are never returned or logged by any of these operations. + +Detailed documentation: [Secrets](../security/secrets.md) + +| Operation | Description | Role Required | +| ------------------------ | ------------------------------------------------------------- | ------------- | +| `set_secret` | Creates or updates a secret and chooses its delivery tier | super_user | +| `grant_secret` | Adds a component to a scoped secret's grants (idempotent) | super_user | +| `revoke_secret` | Removes a component from a scoped secret's grants (idempotent) | super_user | +| `list_secrets` | Lists secret metadata — never envelopes or values | super_user | +| `delete_secret` | Deletes a secret row | super_user | +| `get_secrets_public_key` | Returns the cluster public key for client-side encryption | super_user | + +### `set_secret` + +Creates or updates a secret. Supply exactly one of `value` (plaintext, encrypted on ingest — requires custody on this node) or `envelope` (an `enc:v1:` ciphertext produced client-side against `get_secrets_public_key`). The delivery tier is `processEnv: true` **or** `grants` — the two are mutually exclusive. On update, tier and metadata default to the stored row, so a value rotation preserves the tier without re-specifying it. + +| Parameter | Type | Description | +| ------------ | ---------- | -------------------------------------------------------------------------------- | +| `name` | string | Secret name (word characters, dots, dashes). **Required.** | +| `value` | string | Plaintext value; encrypted immediately, then discarded. Requires custody. | +| `envelope` | string | `enc:v1:` ciphertext (alternative to `value`). | +| `processEnv` | boolean | `true` delivers the secret via `process.env` (global tier). | +| `grants` | string[] | Components allowed to read the secret via the `secrets` accessor (scoped tier). | +| `metadata` | object | Optional free-form label object (not a payload store). | + +```json +{ + "operation": "set_secret", + "name": "STRIPE_KEY", + "value": "sk_live_...", + "grants": ["payments-service"] +} +``` + +Response: + +```json +{ "name": "STRIPE_KEY", "kid": "", "created": true } +``` + +### `grant_secret` / `revoke_secret` + +Add or remove a component from a scoped secret's grants list. Both are idempotent. A `processEnv` (global) secret cannot be granted — convert it with `set_secret` `processEnv: false` first. + +```json +{ "operation": "grant_secret", "name": "STRIPE_KEY", "component": "payments-service" } +``` + +Response includes the updated `grants` array and a `changed` flag (`false` when the call was a no-op). + +### `list_secrets` + +Returns metadata for every secret — **never** envelopes or values. Each entry includes `name`, `kid`, `grants`, `processEnv`, `metadata`, `unverified`, `updated_by`, timestamps, and `kid_matches_custody` (so a stale row on a cloned/rekeyed node is immediately visible). The response also carries the node's `custody_fingerprint` (`null` when no custody is held). + +```json +{ "operation": "list_secrets" } +``` + +### `delete_secret` + +Removes a secret row by `name`. Not cryptographic erasure — audit/transaction logs and backups retain the encrypted envelope. + +```json +{ "operation": "delete_secret", "name": "STRIPE_KEY" } +``` + +### `get_secrets_public_key` + +Returns the cluster secrets public key for client-side envelope encryption. Requires custody on the node. + +```json +{ "operation": "get_secrets_public_key" } +``` + +Response: + +```json +{ "public_key": "-----BEGIN PUBLIC KEY-----\n...", "fingerprint": "" } +``` + +--- + ## Replication & Clustering Operations for configuring and managing Harper cluster replication. diff --git a/reference/security/secrets.md b/reference/security/secrets.md new file mode 100644 index 000000000..a8a4ef9e2 --- /dev/null +++ b/reference/security/secrets.md @@ -0,0 +1,233 @@ +--- +id: secrets +title: Secrets +--- + +Harper provides an encrypted, replicated **secrets store** for supplying credentials and other sensitive configuration to your components without hardcoding them or leaving them in plaintext on disk. Secrets are held in the `system.hdb_secret` system table as ciphertext, replicate across the cluster like any other system table, and are delivered to components in one of two tiers: as `process.env` variables, or through a per-component `secrets` accessor. + +This is the production-grade alternative to committing a `.env` file. For the simpler file-based approach (including encrypted `.env` values that share this store's wire format), see [Environment Variables](../environment-variables/overview.md). + +## Concepts + +### The store + +Every secret is a named row in `system.hdb_secret`. Only ciphertext is ever stored — a plaintext value submitted to `set_secret` is encrypted immediately and the plaintext is discarded. Values are never returned by any read operation, never written to the operations log, and never travel in the replication payload as plaintext; rows reach peers as encrypted envelopes through normal system-table replication. + +Secret names may contain word characters, dots, and dashes (e.g. `STRIPE_KEY`, `deploy.my-app.registry`). + +### Custody (Harper Pro) + +The private key that decrypts secrets — the **custody** — is held by a Harper Pro component. Open-source core ships no custody: + +- **With custody** (Pro): the node can encrypt on ingest (`set_secret` with a plaintext `value`), serve `get_secrets_public_key`, and decrypt secrets for delivery to components. +- **Without custody** (OSS core, or a node that does not hold the key): the node can still **store and replicate** client-encrypted envelopes, but it cannot decrypt them. A plaintext `set_secret` fails; a scoped secret a component requests is reported as `custody-unavailable`. + +A single cluster-shared keypair is distributed to every node the same way the JWT keypair is (node clone/join), so a secret encrypted once by a client can be decrypted on any node that holds custody. + +### Delivery tiers + +Each secret is delivered exactly one of two ways, decided by the row itself. The two tiers are mutually exclusive — a `process.env` secret is already global, so scoping it would be meaningless, and `set_secret` rejects a row that is both. + +| Tier | Row flag | Reaches components as | In `process.env`? | Inherited by child processes? | +| ---- | -------- | --------------------- | ----------------- | ----------------------------- | +| **Global** | `processEnv: true` | `process.env.NAME` | Yes | Yes | +| **Scoped** | `grants: [...]` | `secrets.NAME` (granted components only) | No | No | + +**Global tier** secrets are materialized into the real `process.env` at startup, before components load — exactly like `.env` values, with the same semantics (a pre-existing real environment variable always wins over the store). There is no isolation between components on this tier. + +**Scoped tier** secrets never land in `process.env`. They are exposed only through the per-component accessor, so they are not inherited by child processes and are invisible to environment dumps. The `grants` list on the row is the authority for which components can see the secret; a row with neither `processEnv` nor any grant is **inert** — visible to no one — until it is granted. + +:::note +JS-level scoping is a slowdown layer, not a hard security boundary — components share a process, so OS-level isolation (containers/uids) remains the real boundary between untrusted code. Scoping keeps a secret out of `process.env` and out of ungranted components' reach; it does not sandbox a component that is determined to reach into the process. +::: + +## Managing secrets (Operations API) + +All secret operations are **`super_user` only** and are documented in the [Operations Reference](../operations-api/operations.md#secrets). The core operations: + +| Operation | Purpose | +| --------- | ------- | +| `set_secret` | Create or update a secret (and choose its tier) | +| `grant_secret` / `revoke_secret` | Add or remove a component from a scoped secret's grants | +| `list_secrets` | List secret metadata (never values) | +| `delete_secret` | Remove a secret row | +| `get_secrets_public_key` | Fetch the cluster public key for client-side encryption | + +### Create a global (process.env) secret + +With custody on the node, submit the plaintext `value` and Harper encrypts it on ingest: + +```json +{ + "operation": "set_secret", + "name": "DATABASE_URL", + "value": "postgres://user:pass@host/db", + "processEnv": true +} +``` + +Every component now sees `process.env.DATABASE_URL`. + +### Create a scoped secret and grant it + +```json +{ + "operation": "set_secret", + "name": "STRIPE_KEY", + "value": "sk_live_...", + "grants": ["payments-service"] +} +``` + +`grants` may be set at creation (above) or managed separately: + +```json +{ "operation": "grant_secret", "name": "STRIPE_KEY", "component": "payments-service" } +``` + +```json +{ "operation": "revoke_secret", "name": "STRIPE_KEY", "component": "payments-service" } +``` + +Granting a not-yet-deployed component is legal — grants may precede the deploy. + +## Consuming secrets in a component + +### Declare what you expect + +A component declares the environment it needs in an `env:` block in its `config.yaml`. A declaration is a **request, not a grant** — it cannot widen access to a scoped secret; the `grants` list on the row is the only authority for that. + +```yaml +# config.yaml +env: + NODE_ENV: production # string value = an inline literal written to process.env + DATABASE_URL: { required: true, description: Primary database } # satisfied from the store or env + SENTRY_DSN: { required: false } # optional +``` + +- A **string** value is an inline literal, written directly into `process.env` (a convenience for non-secret config; do not put real secrets here — they land in `config.yaml`). +- An **object** value is a declaration satisfied from `process.env` (a global-tier secret, a literal, or a real environment variable) — it does not by itself expose a scoped secret. + +A `required: true` declaration that cannot be satisfied stops **that component** from loading (the instance keeps running). The failure reason is one of: + +- `missing` — no matching row and no environment variable. +- `ungranted` — a scoped row exists but this component is not in its grants. +- `custody-unavailable` — a row exists but cannot be decrypted on this node (no custody). + +### Read scoped secrets with the `secrets` accessor + +Scoped secrets are read through the process-wide `secrets` export — a read-only, name→value map of the secrets granted to the current component (plus any `process.env`-resolved values it declared): + +```js +import { secrets } from 'harper'; + +const stripe = new Stripe(secrets.STRIPE_KEY); +``` + +The recommended idiom is a **module-top-level destructure**, which binds correctly in every component-loading mode: + +```js +import { secrets } from 'harper'; + +const { STRIPE_KEY, WEBHOOK_SECRET } = secrets; +``` + +The `secrets` object is frozen and enumerable (`Object.keys(secrets)`, spread). Global-tier secrets are read from `process.env` as usual and do not need the accessor. + +:::note +Under the VM/compartment component loaders the `harper` module is per-scope, so `import { secrets } from 'harper'` binds to the loading component exactly. Under the native loader the `harper` package is a process-wide singleton, so `secrets` binds to the current component via the component-load context; accessing it **outside** a component-load context fails loudly rather than guessing which component is asking. The module-top-level destructure above is exact in all modes. +::: + +### Healing after changes + +Materialization happens once per load cycle — there is no live re-materialization. A secret that was granted late, rotated, or whose custody came up after boot heals on the next **restart or component reload**, when the store is re-read. + +## Client-side encryption (encrypt before it leaves the client) + +For the strongest posture, a client can encrypt a secret value **before** sending it, so plaintext never reaches the operations API at all — not even on the node that stores it. This is also how a node **without custody** can accept a secret: it stores the opaque envelope verbatim. + +The flow: + +1. **Fetch the public key** with `get_secrets_public_key` (requires custody somewhere to answer). Cache it by `fingerprint`: + + ```json + { "operation": "get_secrets_public_key" } + ``` + + ```json + { "public_key": "-----BEGIN PUBLIC KEY-----\n...", "fingerprint": "" } + ``` + +2. **Encrypt the value** into an `enc:v1:` envelope (see below), using the `fingerprint` as the `kid`. + +3. **Submit the envelope** instead of a plaintext `value`: + + ```json + { + "operation": "set_secret", + "name": "STRIPE_KEY", + "envelope": "enc:v1:", + "grants": ["payments-service"] + } + ``` + +The server derives `kid` from the sealed envelope (a separate client-supplied `kid` field is never trusted) and, if it holds custody, verifies the `kid` matches this cluster's key; a mismatch is rejected with the current fingerprint. Without custody the row is accepted but flagged `unverified: true` in `list_secrets` so a bad envelope is visible rather than silently failing at consumption time. + +### Envelope format + +An encrypted value is the literal prefix `enc:v1:` followed by the base64url encoding of a JSON envelope. Hybrid encryption is used — AES-256-GCM encrypts the value, RSA-OAEP (SHA-256) wraps the AES key — because the RSA key is too small to directly encrypt multi-line secrets such as PEM keys: + +``` +enc:v1: +``` + +```jsonc +{ + "kid": "", // which key encrypted this + "k": "", + "iv": "", + "ct": "", + "tag": "" +} +``` + +`kid` lets multiple keypairs coexist during rotation: the decryptor selects the matching private key, or rejects the value if it holds no key for that `kid`. + +The same `enc:v1:` envelope format is used for encrypted `.env` values, so a value encrypted once can be used with either the secrets store or `loadEnv`. + +### Reference client (Node.js) + +```js +import { randomBytes, publicEncrypt, createCipheriv, constants } from 'node:crypto'; + +function encryptSecret(plaintext, publicKeyPem, kid) { + const aesKey = randomBytes(32); + const iv = randomBytes(12); + const cipher = createCipheriv('aes-256-gcm', aesKey, iv); + const ct = Buffer.concat([cipher.update(plaintext, 'utf8'), cipher.final()]); + const tag = cipher.getAuthTag(); + const k = publicEncrypt({ key: publicKeyPem, padding: constants.RSA_PKCS1_OAEP_PADDING, oaepHash: 'sha256' }, aesKey); + const envelope = { + kid, // the `fingerprint` from get_secrets_public_key + k: k.toString('base64'), + iv: iv.toString('base64'), + ct: ct.toString('base64'), + tag: tag.toString('base64'), + }; + return 'enc:v1:' + Buffer.from(JSON.stringify(envelope)).toString('base64url'); +} +``` + +## Private-registry credentials + +`deploy_component` accepts a `registryAuth` array so a component installed from a private npm registry can authenticate. A provided token is ingested into the secrets store (as a reference, encrypted) rather than travelling in the operation body or persisting as a plaintext `.npmrc`, so package-reference deploys survive rollback, reboot, and new peers joining. See [`deploy_component`](../operations-api/operations.md#deploy_component). + +## Threat model + +**Protects against:** theft of on-disk config/`.env` files, the editor/operations read surface, secrets appearing in operations logs and replication payloads, and an operator observing traffic at the TLS-terminating layer. Client-side encryption additionally keeps plaintext off the operations API entirely. + +**Does not protect against:** a fully compromised running node — a node that holds custody necessarily holds the private key and the decrypted values in memory. This is defense-in-depth for accidental and at-rest exposure, not protection from a compromised host. OS-level isolation remains the boundary between untrusted components. + +:::note +`delete_secret` removes the row but is **not** cryptographic erasure — audit logs, transaction logs, and backups retain the encrypted envelope. +::: diff --git a/sidebarsReference.ts b/sidebarsReference.ts index 639578008..d58b1b170 100644 --- a/sidebarsReference.ts +++ b/sidebarsReference.ts @@ -325,6 +325,11 @@ const sidebars: SidebarsConfig = { id: 'security/impersonation', label: 'Impersonation', }, + { + type: 'doc', + id: 'security/secrets', + label: 'Secrets', + }, ], }, { From d0f1579a040bdbf395462c74134c39779b0841aa Mon Sep 17 00:00:00 2001 From: Dawson Toth Date: Tue, 14 Jul 2026 14:49:19 -0400 Subject: [PATCH 2/9] docs(security): secrets change subscription and live scoped accessor (#584) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * style(operations-api): apply prettier table alignment Pre-existing format:check violation on the kris/secrets-docs branch; the repo-wide prettier gate must be clean for CI. Co-Authored-By: Claude Opus 4.8 * docs(security): document secrets change subscription and live scoped accessor Documents harper#1787 (live secret change detection) on the new secrets reference page: - secrets.subscribe(name) — async iterable yielding the current value then every change; shown as read-now + fire-and-forget background subscribe so module load isn't blocked (the practical hot-swap-on-rotation pattern). - Live scoped-tier accessor: a fresh secrets.NAME read reflects the latest value; a destructure is now called out as a point-in-time copy. - Reworked 'Healing after changes' into a per-tier 'How each tier sees a change' table (scoped = live; global/process.env = reload-only) and corrected the 'frozen' accessor description to the read-only live view, with subscribe as a non-enumerable member. Ships in v5.2.0 alongside the rest of the secrets store, so no differential VersionBadge (the page is uniformly 5.2.0 surface). Co-Authored-By: Claude Opus 4.8 * docs(security): dedupe the subscribe example's startup rebuild Address review on #584: secrets.subscribe() replays the current value first, so guard on key change to skip the redundant client rebuild on startup, and note that semantic inline. Co-Authored-By: Claude Opus 4.8 * docs(security): guard the subscribe example against unhandled rejections Mirror the studio review fix (HarperFast/studio#1501): the fire-and-forget subscribe loop is copy-paste code, so wrap it in try/catch and console.error the failure so a stream error can't become an unhandled promise rejection. Co-Authored-By: Claude Opus 4.8 * docs(security): lead with the live accessor; subscription is the exception Per feedback: reading secrets.NAME (undestructured) returns the live value on every read, so most components just use the secret where they need it — e.g. per request inside a resource — and get the current value with no subscription. Only a destructure takes a snapshot. Reframe subscribe() as the case for long-lived objects built from a secret (client/pool/signer) that must be rebuilt on rotation, and make the loader note's per-loader boundary explicit (vm/compartment read live anywhere; native resolves only during load). Co-Authored-By: Claude Opus 4.8 --------- Co-authored-by: Claude Opus 4.8 --- reference/operations-api/operations.md | 36 +++++----- reference/security/secrets.md | 97 +++++++++++++++++++++----- 2 files changed, 99 insertions(+), 34 deletions(-) diff --git a/reference/operations-api/operations.md b/reference/operations-api/operations.md index 3114d1750..c0dabf015 100644 --- a/reference/operations-api/operations.md +++ b/reference/operations-api/operations.md @@ -612,11 +612,11 @@ Additional parameters: When a component is installed from a private npm registry, `registryAuth` supplies the credential. It is an array of entries, each naming a `registry` and providing the credential exactly one of two ways: -| Field | Description | -| ---------- | --------------------------------------------------------------------------------------------- | -| `registry` | The registry URL or host the credential applies to. **Required.** | -| `token` | A literal auth token, or | -| `secret` | The name of an [`hdb_secret`](../security/secrets.md) row to resolve the token from. | +| Field | Description | +| ---------- | ------------------------------------------------------------------------------------------------ | +| `registry` | The registry URL or host the credential applies to. **Required.** | +| `token` | A literal auth token, or | +| `secret` | The name of an [`hdb_secret`](../security/secrets.md) row to resolve the token from. | | `scope` | Optional npm `@scope` (e.g. `"@my-org"`) the entry applies to; omit to set the default registry. | A provided **`token`** is not treated as ephemeral: Harper ingests it into the encrypted [secrets store](../security/secrets.md) and references it everywhere, so package-reference deploys keep working through rollback, reboot, and new peers joining — without re-supplying the token. The token is encrypted at rest, stripped from the operation before replication and from the operations log, and only ever crosses the cluster as ciphertext. Using a **`secret`** reference names an existing store row directly. Ingesting a token requires custody on the deploying node; on OSS core without custody, a literal token falls back to a transient, this-node-only credential (not persisted or replicated). @@ -754,26 +754,26 @@ Operations for managing the encrypted [secrets store](../security/secrets.md) (` Detailed documentation: [Secrets](../security/secrets.md) | Operation | Description | Role Required | -| ------------------------ | ------------------------------------------------------------- | ------------- | -| `set_secret` | Creates or updates a secret and chooses its delivery tier | super_user | +| ------------------------ | -------------------------------------------------------------- | ------------- | +| `set_secret` | Creates or updates a secret and chooses its delivery tier | super_user | | `grant_secret` | Adds a component to a scoped secret's grants (idempotent) | super_user | | `revoke_secret` | Removes a component from a scoped secret's grants (idempotent) | super_user | -| `list_secrets` | Lists secret metadata — never envelopes or values | super_user | -| `delete_secret` | Deletes a secret row | super_user | -| `get_secrets_public_key` | Returns the cluster public key for client-side encryption | super_user | +| `list_secrets` | Lists secret metadata — never envelopes or values | super_user | +| `delete_secret` | Deletes a secret row | super_user | +| `get_secrets_public_key` | Returns the cluster public key for client-side encryption | super_user | ### `set_secret` Creates or updates a secret. Supply exactly one of `value` (plaintext, encrypted on ingest — requires custody on this node) or `envelope` (an `enc:v1:` ciphertext produced client-side against `get_secrets_public_key`). The delivery tier is `processEnv: true` **or** `grants` — the two are mutually exclusive. On update, tier and metadata default to the stored row, so a value rotation preserves the tier without re-specifying it. -| Parameter | Type | Description | -| ------------ | ---------- | -------------------------------------------------------------------------------- | -| `name` | string | Secret name (word characters, dots, dashes). **Required.** | -| `value` | string | Plaintext value; encrypted immediately, then discarded. Requires custody. | -| `envelope` | string | `enc:v1:` ciphertext (alternative to `value`). | -| `processEnv` | boolean | `true` delivers the secret via `process.env` (global tier). | -| `grants` | string[] | Components allowed to read the secret via the `secrets` accessor (scoped tier). | -| `metadata` | object | Optional free-form label object (not a payload store). | +| Parameter | Type | Description | +| ------------ | -------- | ------------------------------------------------------------------------------- | +| `name` | string | Secret name (word characters, dots, dashes). **Required.** | +| `value` | string | Plaintext value; encrypted immediately, then discarded. Requires custody. | +| `envelope` | string | `enc:v1:` ciphertext (alternative to `value`). | +| `processEnv` | boolean | `true` delivers the secret via `process.env` (global tier). | +| `grants` | string[] | Components allowed to read the secret via the `secrets` accessor (scoped tier). | +| `metadata` | object | Optional free-form label object (not a payload store). | ```json { diff --git a/reference/security/secrets.md b/reference/security/secrets.md index a8a4ef9e2..6eeeb7012 100644 --- a/reference/security/secrets.md +++ b/reference/security/secrets.md @@ -28,10 +28,10 @@ A single cluster-shared keypair is distributed to every node the same way the JW Each secret is delivered exactly one of two ways, decided by the row itself. The two tiers are mutually exclusive — a `process.env` secret is already global, so scoping it would be meaningless, and `set_secret` rejects a row that is both. -| Tier | Row flag | Reaches components as | In `process.env`? | Inherited by child processes? | -| ---- | -------- | --------------------- | ----------------- | ----------------------------- | -| **Global** | `processEnv: true` | `process.env.NAME` | Yes | Yes | -| **Scoped** | `grants: [...]` | `secrets.NAME` (granted components only) | No | No | +| Tier | Row flag | Reaches components as | In `process.env`? | Inherited by child processes? | +| ---------- | ------------------ | ---------------------------------------- | ----------------- | ----------------------------- | +| **Global** | `processEnv: true` | `process.env.NAME` | Yes | Yes | +| **Scoped** | `grants: [...]` | `secrets.NAME` (granted components only) | No | No | **Global tier** secrets are materialized into the real `process.env` at startup, before components load — exactly like `.env` values, with the same semantics (a pre-existing real environment variable always wins over the store). There is no isolation between components on this tier. @@ -45,13 +45,13 @@ JS-level scoping is a slowdown layer, not a hard security boundary — component All secret operations are **`super_user` only** and are documented in the [Operations Reference](../operations-api/operations.md#secrets). The core operations: -| Operation | Purpose | -| --------- | ------- | -| `set_secret` | Create or update a secret (and choose its tier) | +| Operation | Purpose | +| -------------------------------- | ------------------------------------------------------- | +| `set_secret` | Create or update a secret (and choose its tier) | | `grant_secret` / `revoke_secret` | Add or remove a component from a scoped secret's grants | -| `list_secrets` | List secret metadata (never values) | -| `delete_secret` | Remove a secret row | -| `get_secrets_public_key` | Fetch the cluster public key for client-side encryption | +| `list_secrets` | List secret metadata (never values) | +| `delete_secret` | Remove a secret row | +| `get_secrets_public_key` | Fetch the cluster public key for client-side encryption | ### Create a global (process.env) secret @@ -124,7 +124,23 @@ import { secrets } from 'harper'; const stripe = new Stripe(secrets.STRIPE_KEY); ``` -The recommended idiom is a **module-top-level destructure**, which binds correctly in every component-loading mode: +**The accessor is live.** On the scoped tier a property read (`secrets.STRIPE_KEY`) always returns the latest stored value — a rotation is reflected on the next read, no reload required. So most components need nothing more than the accessor: read the secret **where you use it**, and every request sees the current value. Reading inside a resource handler is the common case: + +```js +import { Resource, secrets } from 'harper'; + +export class Report extends Resource { + async get() { + // Read at call time, so each request uses the current token — even after a rotation. + const response = await fetch('https://api.example.com/report', { + headers: { authorization: `Bearer ${secrets.API_TOKEN}` }, + }); + return response.json(); + } +} +``` + +The one thing that is **not** live is a **destructure**: `const { STRIPE_KEY } = secrets` — or any assignment that copies the value into your own variable — takes a **point-in-time snapshot** and won't observe a later rotation. That's fine for a value read once and used as-is, and it's the idiom to use when the read must happen at module load (see the loader note below): ```js import { secrets } from 'harper'; @@ -132,15 +148,64 @@ import { secrets } from 'harper'; const { STRIPE_KEY, WEBHOOK_SECRET } = secrets; ``` -The `secrets` object is frozen and enumerable (`Object.keys(secrets)`, spread). Global-tier secrets are read from `process.env` as usual and do not need the accessor. +The `secrets` object is a read-only, live view of the secrets granted to the component; its secret names are enumerable (`Object.keys(secrets)`, spread), while the `subscribe` method below is a non-enumerable member so it never leaks into a spread of values. Global-tier secrets are read from `process.env` as usual and do not need the accessor. :::note -Under the VM/compartment component loaders the `harper` module is per-scope, so `import { secrets } from 'harper'` binds to the loading component exactly. Under the native loader the `harper` package is a process-wide singleton, so `secrets` binds to the current component via the component-load context; accessing it **outside** a component-load context fails loudly rather than guessing which component is asking. The module-top-level destructure above is exact in all modes. +Under the VM/compartment component loaders (the default) the `harper` module is per-scope, so `import { secrets } from 'harper'` is a view bound to your component — you can read `secrets.NAME` live from anywhere in it, including inside request handlers. Under the native loader the `harper` package is a process-wide singleton that resolves the calling component from the component-load context, so a read **outside** component load fails loudly rather than guessing which component is asking; there, read at module top level during load (the destructure above — a snapshot) or use `secrets.subscribe()` to track changes. The module-top-level destructure is exact in all modes. ::: -### Healing after changes +### React to rotations with `secrets.subscribe()` + +Reading `secrets.NAME` per use (above) already picks up rotations, so reach for a subscription only when you build something **from** a secret and keep it around — an API client, a connection pool, a cached signer — and need to rebuild that object when the secret changes. + +`secrets.subscribe(name)` returns an async iterable that yields the secret's **current value immediately**, then a new value on **every change** — a grant, a rotation (`set_secret`), a revoke, or a delete. It lets a component hot-swap a rotated secret without a restart. + +The iterator stays open for the life of the component, so a top-level `for await` would block module load forever. Instead, use the current value right away, then subscribe in a **background task** so module load is never blocked and the rest of the component keeps initializing: + +```js +import { secrets } from 'harper'; + +// Ready to serve immediately with the current value... +let currentKey = secrets.STRIPE_KEY; +let stripe = new Stripe(currentKey); + +// ...then hot-swap on every rotation, without blocking module load. +(async () => { + try { + for await (const key of secrets.subscribe('STRIPE_KEY')) { + if (key === currentKey) continue; // the first yield is the value we already have + currentKey = key; + stripe = new Stripe(currentKey); + } + } catch (error) { + // Log and keep serving the last client; it stays valid until the next reload. + console.error('secret subscription ended for STRIPE_KEY:', error); + } +})(); +``` + +The first value `subscribe` yields is the secret's **current** value — the same one read synchronously above — so the guard skips a redundant rebuild on startup and only rebuilds the client when the key actually rotates. + +Authority is re-evaluated on **every** event through the same rules as the read accessor: + +- A **revoke** or **delete** yields `undefined` (no plaintext is retained), while the stream stays **open**. +- A later **re-grant** or **re-add** resumes delivery on the same iterator — no restart, no re-subscribe. + +`subscribe` is a reserved name: it can never be a secret's name, and never resolves to a secret value. + +### How each tier sees a change + +The two tiers observe changes differently: + +| Read path | Scoped tier (`grants`) | Global tier (`processEnv`) | +| --------------------------- | ---------------------------------------------------- | ------------------------------------------------- | +| `secrets.NAME` accessor | **Live** — reflects the latest value on a fresh read | Current value at load; reload-only | +| `secrets.subscribe('NAME')` | **Live** — streams every change | Current value only; reload-only | +| `process.env.NAME` | n/a | Reload-only — never re-mutated under running code | + +**Scoped-tier** consumers see grants, rotations, revokes, and late-arriving custody take effect live, through the accessor or a subscription. -Materialization happens once per load cycle — there is no live re-materialization. A secret that was granted late, rotated, or whose custody came up after boot heals on the next **restart or component reload**, when the store is re-read. +**Global-tier** secrets are materialized into `process.env` once at startup, before components load, and are deliberately **not** re-materialized under running code: child processes inherit `process.env`, and the "a pre-existing real environment variable always wins" precedence must hold. A change to a global secret — or custody that only came up after boot — heals on the next **restart or component reload**, when the store is re-read. ## Client-side encryption (encrypt before it leaves the client) @@ -187,7 +252,7 @@ enc:v1: "k": "", "iv": "", "ct": "", - "tag": "" + "tag": "", } ``` From 2768b55a5343aedc774b3c33b5e64c1bf53bf182 Mon Sep 17 00:00:00 2001 From: Kris Zyp Date: Thu, 16 Jul 2026 08:33:52 -0600 Subject: [PATCH 3/9] =?UTF-8?q?docs(security):=20rename=20registryAuth=20?= =?UTF-8?q?=E2=86=92=20credentials;=20document=20git-host=20deploy=20auth?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit deploy_component's registryAuth field was renamed to a general-purpose `credentials` array (harper#1797) that now carries both npm-registry (`registry`) and git-host (`host`) credential entries; the git-over-HTTPS path (harper#1792) serves the token to git from memory via a credential helper. Update the operations reference and the secrets page accordingly, including the derived secret-name conventions and a rename note. Co-Authored-By: Claude Opus 4.8 (1M context) --- reference/operations-api/operations.md | 48 ++++++++++++++++++++------ reference/security/secrets.md | 6 ++-- 2 files changed, 40 insertions(+), 14 deletions(-) diff --git a/reference/operations-api/operations.md b/reference/operations-api/operations.md index c0dabf015..4ec6e14d8 100644 --- a/reference/operations-api/operations.md +++ b/reference/operations-api/operations.md @@ -606,30 +606,56 @@ Additional parameters: - `urlPath` — override the HTTP URL path the component is mounted at (e.g. `"/api/v2"`) - `install_allow_scripts` — set to `true` to allow npm pre/post install scripts (disabled by default) -- `registryAuth` — credentials for installing a component from a private npm registry (see below) +- `credentials` — credentials for installing a component from a private npm registry or private git repository (see below) -#### Private-registry authentication (`registryAuth`) +#### Deploy credentials (`credentials`) -When a component is installed from a private npm registry, `registryAuth` supplies the credential. It is an array of entries, each naming a `registry` and providing the credential exactly one of two ways: +When a component is installed from a private source, `credentials` supplies the authentication. It is an array of entries; each entry is one of two kinds, identified by its key: -| Field | Description | -| ---------- | ------------------------------------------------------------------------------------------------ | -| `registry` | The registry URL or host the credential applies to. **Required.** | -| `token` | A literal auth token, or | -| `secret` | The name of an [`hdb_secret`](../security/secrets.md) row to resolve the token from. | -| `scope` | Optional npm `@scope` (e.g. `"@my-org"`) the entry applies to; omit to set the default registry. | +- **npm registry auth** — an entry with a `registry` key, applied to a private npm registry. +- **git host auth** — an entry with a `host` key, applied to a private git repository fetched by reference (e.g. `package: "github:my-org/my-app#semver:v1.2.3"`). -A provided **`token`** is not treated as ephemeral: Harper ingests it into the encrypted [secrets store](../security/secrets.md) and references it everywhere, so package-reference deploys keep working through rollback, reboot, and new peers joining — without re-supplying the token. The token is encrypted at rest, stripped from the operation before replication and from the operations log, and only ever crosses the cluster as ciphertext. Using a **`secret`** reference names an existing store row directly. Ingesting a token requires custody on the deploying node; on OSS core without custody, a literal token falls back to a transient, this-node-only credential (not persisted or replicated). +An entry provides its credential exactly one of two ways — a literal `token`, or a `secret` reference: + +| Field | Kind | Description | +| ---------- | -------- | ------------------------------------------------------------------------------------------------ | +| `registry` | npm | The registry URL or host the credential applies to. **Required** for an npm entry. | +| `scope` | npm | Optional npm `@scope` (e.g. `"@my-org"`) the entry applies to; omit to set the default registry. | +| `host` | git | The bare git host the credential applies to (e.g. `"github.com"`). **Required** for a git entry. | +| `username` | git | Optional git HTTPS username. Defaults to `x-access-token` (GitHub); GitLab uses `oauth2`, Bitbucket `x-token-auth`. | +| `token` | both | A literal auth token, **or** | +| `secret` | both | The name of an [`hdb_secret`](../security/secrets.md) row to resolve the token from. | + +A provided **`token`** is not treated as ephemeral: Harper ingests it into the encrypted [secrets store](../security/secrets.md) and references it everywhere, so package-reference deploys keep working through rollback, reboot, and new peers joining — without re-supplying the token. The token is encrypted at rest, stripped from the operation before replication and from the operations log, and only ever crosses the cluster as ciphertext. A git-host token is additionally served to git **from memory** (via a credential helper) — it is never written to a file or into a URL. Using a **`secret`** reference names an existing store row directly. Ingesting a token requires custody on the deploying node; on OSS core without custody, a literal token falls back to a transient, this-node-only credential (not persisted or replicated). + +Ingested tokens are stored under a derived name granted to the component — `deploy..` for a registry entry, `deploy..git.` for a git entry — so re-deploying with a rotated token idempotently updates the same row. + +Private npm registry: ```json { "operation": "deploy_component", "project": "my-app", "package": "npm:@my-org/my-app@1.2.3", - "registryAuth": [{ "registry": "https://registry.my-org.com", "token": "npm_...", "scope": "@my-org" }] + "credentials": [{ "registry": "https://registry.my-org.com", "scope": "@my-org", "token": "npm_..." }] +} +``` + +Private git repository (token resolved from an existing secret): + +```json +{ + "operation": "deploy_component", + "project": "my-app", + "package": "github:my-org/my-app#semver:v1.2.3", + "credentials": [{ "host": "github.com", "secret": "deploy.my-app.git.github_com" }] } ``` +:::note +`credentials` replaces the earlier `registryAuth` field (renamed while the feature was in alpha, before it grew to carry git-host credentials). `registryAuth` is now rejected with an error directing you to `credentials`. +::: + The response includes a `deployment_id` that can be used to query the deployment record: ```json diff --git a/reference/security/secrets.md b/reference/security/secrets.md index 6eeeb7012..10162ed8a 100644 --- a/reference/security/secrets.md +++ b/reference/security/secrets.md @@ -13,7 +13,7 @@ This is the production-grade alternative to committing a `.env` file. For the si Every secret is a named row in `system.hdb_secret`. Only ciphertext is ever stored — a plaintext value submitted to `set_secret` is encrypted immediately and the plaintext is discarded. Values are never returned by any read operation, never written to the operations log, and never travel in the replication payload as plaintext; rows reach peers as encrypted envelopes through normal system-table replication. -Secret names may contain word characters, dots, and dashes (e.g. `STRIPE_KEY`, `deploy.my-app.registry`). +Secret names may contain word characters, dots, and dashes (e.g. `STRIPE_KEY`, `deploy.my-app.git.github_com`). ### Custody (Harper Pro) @@ -283,9 +283,9 @@ function encryptSecret(plaintext, publicKeyPem, kid) { } ``` -## Private-registry credentials +## Private-source deploy credentials -`deploy_component` accepts a `registryAuth` array so a component installed from a private npm registry can authenticate. A provided token is ingested into the secrets store (as a reference, encrypted) rather than travelling in the operation body or persisting as a plaintext `.npmrc`, so package-reference deploys survive rollback, reboot, and new peers joining. See [`deploy_component`](../operations-api/operations.md#deploy_component). +`deploy_component` accepts a `credentials` array so a component installed from a private **npm registry** or private **git repository** can authenticate. A provided token is ingested into the secrets store (as a reference, encrypted) rather than travelling in the operation body, persisting as a plaintext `.npmrc`, or being written to disk for git — so package-reference deploys survive rollback, reboot, and new peers joining. Ingested tokens are stored under a derived name (`deploy..` or `deploy..git.`) granted to the component. See [`deploy_component`](../operations-api/operations.md#deploy_component). ## Threat model From 94e774bda960369737253ed6ff313462639eb4bd Mon Sep 17 00:00:00 2001 From: Kris Zyp Date: Thu, 16 Jul 2026 08:38:33 -0600 Subject: [PATCH 4/9] docs(security): mention Studio UI for secrets; address review nits - Note that Harper Studio offers a graphical interface for managing secrets (Dawson's suggestion) on both the Secrets page and the operations section. - Clarify that a string env value is stored plaintext in config.yaml (VCS). - Add a list_secrets response example. Co-Authored-By: Claude Opus 4.8 (1M context) --- reference/operations-api/operations.md | 26 ++++++++++++++++++++++++++ reference/security/secrets.md | 4 +++- 2 files changed, 29 insertions(+), 1 deletion(-) diff --git a/reference/operations-api/operations.md b/reference/operations-api/operations.md index 4ec6e14d8..b09fb8506 100644 --- a/reference/operations-api/operations.md +++ b/reference/operations-api/operations.md @@ -779,6 +779,10 @@ Operations for managing the encrypted [secrets store](../security/secrets.md) (` Detailed documentation: [Secrets](../security/secrets.md) +:::tip +Prefer a UI? [Harper Studio](../studio/overview.md) provides a graphical interface for creating, granting, and rotating secrets — it drives these operations for you, so you don't have to hand-craft the request bodies below. +::: + | Operation | Description | Role Required | | ------------------------ | -------------------------------------------------------------- | ------------- | | `set_secret` | Creates or updates a secret and chooses its delivery tier | super_user | @@ -834,6 +838,28 @@ Returns metadata for every secret — **never** envelopes or values. Each entry { "operation": "list_secrets" } ``` +Response: + +```json +{ + "secrets": [ + { + "name": "STRIPE_KEY", + "kid": "a1b2c3d4...", + "grants": ["payments-service"], + "processEnv": false, + "metadata": {}, + "unverified": false, + "updated_by": "admin", + "__createdtime__": 1700000000000, + "__updatedtime__": 1700000000000, + "kid_matches_custody": true + } + ], + "custody_fingerprint": "a1b2c3d4..." +} +``` + ### `delete_secret` Removes a secret row by `name`. Not cryptographic erasure — audit/transaction logs and backups retain the encrypted envelope. diff --git a/reference/security/secrets.md b/reference/security/secrets.md index 10162ed8a..2e2a0a842 100644 --- a/reference/security/secrets.md +++ b/reference/security/secrets.md @@ -43,6 +43,8 @@ JS-level scoping is a slowdown layer, not a hard security boundary — component ## Managing secrets (Operations API) +Secrets can be managed graphically in [Harper Studio](../studio/overview.md), which provides a friendly interface for creating, granting, and rotating secrets without hand-crafting operation requests. The operations below are the underlying API that Studio (and your own tooling) drives. + All secret operations are **`super_user` only** and are documented in the [Operations Reference](../operations-api/operations.md#secrets). The core operations: | Operation | Purpose | @@ -105,7 +107,7 @@ env: SENTRY_DSN: { required: false } # optional ``` -- A **string** value is an inline literal, written directly into `process.env` (a convenience for non-secret config; do not put real secrets here — they land in `config.yaml`). +- A **string** value is an inline literal, written directly into `process.env` (a convenience for non-secret config; do not put real secrets here — they are stored in plaintext in `config.yaml`, which is typically committed to version control). - An **object** value is a declaration satisfied from `process.env` (a global-tier secret, a literal, or a real environment variable) — it does not by itself expose a scoped secret. A `required: true` declaration that cannot be satisfied stops **that component** from loading (the instance keeps running). The failure reason is one of: From fbad5fe6a43ce624da55fa9734fbf038781206b9 Mon Sep 17 00:00:00 2001 From: Kris Zyp Date: Fri, 17 Jul 2026 09:49:55 -0600 Subject: [PATCH 5/9] docs(components): fix install_allow_scripts field name (install_allowInstallScripts never existed) Co-Authored-By: Claude Fable 5 --- reference/components/applications.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/reference/components/applications.md b/reference/components/applications.md index 6bff5d156..dfc70cd33 100644 --- a/reference/components/applications.md +++ b/reference/components/applications.md @@ -213,7 +213,7 @@ Creates a new component project in the component root directory using a template - `template` _(optional)_ — Git URL of a template repository. Defaults to `https://github.com/HarperFast/application-template` - `install_command` _(optional)_ — Install command. Defaults to `npm install` - `install_timeout` _(optional)_ — Install timeout in milliseconds. Defaults to `300000` (5 minutes) -- `install_allowInstallScripts` _(optional)_ — Allow install scripts to run. Defaults to `false`, which causes `--ignore-scripts` to be passed to the install command (this is ignored with `install_command`). +- `install_allow_scripts` _(optional)_ — Allow install scripts to run. Defaults to `false`, which causes `--ignore-scripts` to be passed to the install command (this is ignored with `install_command`). - `replicated` _(optional)_ — Replicate to all cluster nodes ```json @@ -235,7 +235,7 @@ Deploys a component using a package reference or a base64-encoded `.tar` payload - `replicated` _(optional)_ — Replicate to all cluster nodes - `install_command` _(optional)_ — Install command override - `install_timeout` _(optional)_ — Install timeout override in milliseconds -- `install_allowInstallScripts` _(optional)_ — Allow install scripts to run. Defaults to `false`, which causes `--ignore-scripts` to be passed to the install command (this is ignored with `install_command`). +- `install_allow_scripts` _(optional)_ — Allow install scripts to run. Defaults to `false`, which causes `--ignore-scripts` to be passed to the install command (this is ignored with `install_command`). ```json { From a7cf04ecc14ff3812761f02d6602c6d4b66a54a7 Mon Sep 17 00:00:00 2001 From: Kris Zyp Date: Fri, 17 Jul 2026 09:49:55 -0600 Subject: [PATCH 6/9] =?UTF-8?q?docs(learn):=20CI/CD=20deployment=20guide?= =?UTF-8?q?=20=E2=80=94=20git-tag=20vs=20registry,=20deploy=20credentials,?= =?UTF-8?q?=20GitHub=20Actions?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 --- learn/developers/deploying-from-ci.mdx | 217 +++++++++++++++++++++++++ sidebarsLearn.ts | 5 + 2 files changed, 222 insertions(+) create mode 100644 learn/developers/deploying-from-ci.mdx diff --git a/learn/developers/deploying-from-ci.mdx b/learn/developers/deploying-from-ci.mdx new file mode 100644 index 000000000..ac1da5fe5 --- /dev/null +++ b/learn/developers/deploying-from-ci.mdx @@ -0,0 +1,217 @@ +--- +title: Deploying from a CI/CD Pipeline +--- + +Once an application is past the prototype stage, you want deployments to happen from your CI/CD pipeline, not from a laptop: a tag or release triggers a workflow, the workflow validates the code, and the pipeline tells your Harper cluster to deploy it. Harper's [`deploy_component`](/reference/v5/operations-api/operations#deploy_component) operation is a single HTTP call, so any CI system can drive it — this guide uses GitHub Actions and GitHub's hosting (repositories and the GitHub Packages npm registry), but the patterns translate directly to GitLab, Bitbucket, or any private npm registry. + +There are two ways to deliver an application to Harper, and choosing the right one is most of the battle: + +- **Deploy from a git tag** — Harper installs your application directly from a tagged commit in your repository. Best for applications that **run from source**. +- **Deploy from a package registry** — CI builds and publishes an artifact to a private npm registry; Harper installs that artifact by version. Best for applications that **need a build step**. + +Both work with private sources: `deploy_component` accepts a [`credentials`](/reference/v5/operations-api/operations#deploy-credentials-credentials) array carrying auth for a private git host or a private npm registry, backed by Harper's encrypted [secrets store](/reference/v5/security/secrets). + +## What You Will Learn + +- How to choose between git-tag deploys and registry-artifact deploys +- How to provision a deploy credential once and reference it from CI, so your pipeline never handles the GitHub token +- A GitHub Actions workflow that deploys a tagged release straight from a private repository +- A GitHub Actions workflow that builds, publishes to GitHub Packages, and deploys the published artifact +- How to deploy across a cluster and verify the deployment record + +## Prerequisites + +- A Harper Fabric cluster (or a self-managed Harper 5.2+ cluster with the Pro secrets component providing [custody](/reference/v5/security/secrets)) and a `super_user` credential for its Operations API +- A GitHub repository containing a working Harper application ([Create your First Application](../getting-started/create-your-first-application.mdx) if you don't have one) +- Familiarity with GitHub Actions basics (workflows, repository secrets) + +## Choosing a delivery model + +**Deploy from a git tag when your application runs from source.** Most Harper applications do: a `config.yaml`, a `schema.graphql`, resource classes in JavaScript (or TypeScript that Harper runs directly via type stripping), and npm dependencies that install cleanly. For these, the repository *is* the artifact. A git-tag deploy has no publish step to maintain — you tag a release, CI calls `deploy_component` with a reference like `github:my-org/my-app#semver:v1.2.3`, and every node installs exactly that commit. Semver-tagged references also keep the deployed version auditable: the deployment record and the component config both name the tag. + +**Deploy from a registry when your application needs a build.** If CI must produce output that isn't in the repository — a bundling step, compiled assets, code generation, native compilation — publish the *built* package to a registry and deploy that. This is better than making Harper build it for two reasons: + +1. **The artifact you tested is the artifact you run.** CI builds once, tests the result, publishes it. Every Harper node installs that identical, immutable artifact instead of re-running your build independently. +2. **No build scripts run on your database nodes.** When Harper installs a credentialed git reference, npm's pack step runs with `--ignore-scripts` by default, precisely so repository code can't run during install with the deploy credential in reach. A repository that *requires* a `prepare`/build script to be runnable forces you to set `install_allow_scripts: true`, which allows that script — and your dependencies' install scripts — to execute on the node during deployment. A registry artifact needs none of that: it's already built. + +A second constraint pushes the same direction: a git-reference deploy authenticates only the top-level repository. If your application has **private git-hosted dependencies** (a `package.json` dependency pointing at another private repo), their installation is not credentialed and will fail. Private *registry* dependencies work fine — registry credentials apply to the whole install. If your dependency graph reaches into private git repos, publish those dependencies (or the whole app) to a private registry instead. + +| Your application... | Deploy from | +| ------------------------------------------------------ | ---------------- | +| Runs from source (schema, resources, npm dependencies) | Git tag | +| Needs a build step (bundling, codegen, compilation) | Private registry | +| Has private git-hosted dependencies | Private registry | + +## Provision the deploy credential once + +Both paths need Harper to authenticate to GitHub — and it's worth understanding *when*. The credential is not only used at the moment you deploy: Harper persists it (encrypted, in the [secrets store](/reference/v5/security/secrets)) with the component's configuration so that later installs — a new node joining the cluster, a rollback, a re-install after restore — can fetch the package again without you re-supplying auth. + +That has one practical consequence: **use a durable token, not your workflow's ephemeral `GITHUB_TOKEN`**. The automatic `GITHUB_TOKEN` in a GitHub Actions run expires when the job ends. It's fine for things that happen *inside* the job (like publishing a package), but if you hand it to Harper as the deploy credential, any install that happens after the workflow finishes fails authentication. Instead, create a token whose lifetime matches how long nodes may need to fetch this component: + +- **For a private repository (git-tag deploys):** a GitHub [fine-grained personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) with **Contents: read-only** permission, scoped to just the application repository. An organization can attach these to a machine account so they aren't tied to an individual. +- **For GitHub Packages (registry deploys):** a token with the **`read:packages`** scope. + +Store it in Harper once, granted to your component, using [`set_secret`](/reference/v5/operations-api/operations#set_secret): + +```json +{ + "operation": "set_secret", + "name": "gh-deploy-token", + "value": "github_pat_...", + "grants": ["my-app"] +} +``` + +From then on, CI references it by name — `{ "host": "github.com", "secret": "gh-deploy-token" }` — and the GitHub token never appears in your pipeline at all. The only secret your CI system holds is the Harper Operations API credential. This is the recommended setup: it separates the credential's lifecycle (rotate it with another `set_secret` call whenever you like — the deploy operation doesn't change) from the pipeline's. + +:::note Literal tokens +You can also pass the token inline: `{ "host": "github.com", "token": "github_pat_..." }`. Harper ingests it into the secrets store automatically (under a derived name like `deploy.my-app.git.github_com`) and the token is stripped from the operation before it is logged or replicated, so this is safe — re-deploying with a new token rotates the stored one. It just means your CI system holds the GitHub token too. Prefer the `secret` reference. +::: + +## Path A: deploy a tagged release from a private repository + +The flow: push a semver tag (`v1.2.3`) → the workflow calls `deploy_component` with a `#semver:` git reference → every node clones that tag and installs. + +Add two repository secrets to GitHub — `HARPER_OPS_URL` (your cluster's Operations API endpoint, e.g. `https://my-cluster.example.com:9925`) and `HARPER_OPS_AUTH` (a base64 `user:password` Basic credential or a bearer token for a `super_user`). Then: + +```yaml +# .github/workflows/deploy.yml +name: Deploy to Harper +on: + push: + tags: ['v*'] + +jobs: + deploy: + runs-on: ubuntu-latest + steps: + - name: Deploy tag to Harper + run: | + curl --fail-with-body -s "$HARPER_OPS_URL" \ + -H "Content-Type: application/json" \ + -H "Authorization: Basic $HARPER_OPS_AUTH" \ + -d @- <`) — useful if you want deploys keyed to commits rather than tags. +- **`replicated: true`** runs the deploy across the cluster; each node fetches and installs the package itself, resolving the credential from the (replicated) secrets store. +- **`restart: "rolling"`** restarts nodes sequentially so the cluster keeps serving throughout. Use `restart: true` for a single-node or dev instance. +- The token is served to git **from memory** on each node — it is never written into a URL, a lockfile, or a file on disk. + +If this repository needs a build step to be runnable, stop — don't reach for `install_allow_scripts`. Use Path B. + +## Path B: build in CI, publish to GitHub Packages, deploy the artifact + +The flow: push a tag → CI builds and tests → `npm publish` to GitHub Packages → `deploy_component` installs the published version from the registry. + +Your `package.json` needs a scoped name and a `publishConfig` pointing at GitHub Packages: + +```json +{ + "name": "@my-org/my-app", + "version": "1.2.3", + "publishConfig": { "registry": "https://npm.pkg.github.com" }, + "files": ["config.yaml", "schema.graphql", "dist/"] +} +``` + +Use `files` deliberately — it defines exactly what ships in the artifact. Then: + +```yaml +# .github/workflows/deploy.yml +name: Build, publish, and deploy to Harper +on: + push: + tags: ['v*'] + +jobs: + deploy: + runs-on: ubuntu-latest + permissions: + contents: read + packages: write + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-node@v4 + with: + node-version: 22 + registry-url: https://npm.pkg.github.com + - run: npm ci + - run: npm run build + - run: npm test + - name: Publish to GitHub Packages + run: npm publish + env: + NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + - name: Deploy to Harper + run: | + VERSION="${GITHUB_REF_NAME#v}" + curl --fail-with-body -s "$HARPER_OPS_URL" \ + -H "Content-Type: application/json" \ + -H "Authorization: Basic $HARPER_OPS_AUTH" \ + -d @- <" +} +``` + +A useful pattern is to have the workflow poll [`get_deployment`](/reference/v5/operations-api/operations#get_deployment) until the deployment reaches `success` (or fail the job if it reports `failed`), so a broken deploy fails the pipeline rather than being discovered later. [`list_deployments`](/reference/v5/operations-api/operations#list_deployments) gives you the history — handy for answering "what exactly is deployed right now, and when did it change?" + +## Operational notes + +- **Rollback is just another deploy.** Both paths deploy immutable versions, so rolling back is re-running the deploy with the previous tag or version. This is a big part of why pinned references (`#semver:v1.2.3`, `@my-org/my-app@1.2.3`) beat branch references (`#main`) in a pipeline: `#main` moves, so you can neither audit nor re-deploy "what was running yesterday." +- **Rotating the deploy credential** is a `set_secret` call with the same name and new value — no change to the pipeline or the component. +- **Git-host credentials require git ≥ 2.31** on the Harper nodes, and are not supported on Windows nodes (the deploy fails with a clear error rather than degrading security). Registry credentials have no such constraints. Fabric instances satisfy both. +- **Self-managed OSS core without the Pro secrets component:** `secret` references can't be resolved (no decryption custody), and a literal `token` is used transiently for that node's install only — it is not persisted for later installs. The durable-credential patterns in this guide assume custody (Fabric provides it). + +## Additional Resources + +- [`deploy_component` and deploy credentials reference](/reference/v5/operations-api/operations#deploy_component) +- [Secrets store reference](/reference/v5/security/secrets) — the encrypted store behind `set_secret` and `secret` references +- [Deployment records](/reference/v5/operations-api/operations#deployment-operations) — `list_deployments`, `get_deployment` +- [Applications reference](/reference/v5/components/applications) — component structure and the full set of component operations +- [GitHub: fine-grained personal access tokens](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) +- [GitHub Packages: npm registry](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-npm-registry) diff --git a/sidebarsLearn.ts b/sidebarsLearn.ts index 23b906d75..d266f53a1 100644 --- a/sidebarsLearn.ts +++ b/sidebarsLearn.ts @@ -61,6 +61,11 @@ const sidebarsLearn: SidebarsConfig = { id: 'developers/write-through-caching', label: 'Write-Through Caching', }, + { + type: 'doc', + id: 'developers/deploying-from-ci', + label: 'Deploying from a CI/CD Pipeline', + }, ], }, { From 3754e6b3f98fae17981ed74f1263d3b0ba0e101f Mon Sep 17 00:00:00 2001 From: Kris Zyp Date: Fri, 17 Jul 2026 10:00:12 -0600 Subject: [PATCH 7/9] docs(learn): recommend inline token as the primary credential flow; mention Studio UI for set_secret Co-Authored-By: Claude Fable 5 --- learn/developers/deploying-from-ci.mdx | 28 ++++++++++++++------------ 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/learn/developers/deploying-from-ci.mdx b/learn/developers/deploying-from-ci.mdx index ac1da5fe5..e61872657 100644 --- a/learn/developers/deploying-from-ci.mdx +++ b/learn/developers/deploying-from-ci.mdx @@ -14,7 +14,7 @@ Both work with private sources: `deploy_component` accepts a [`credentials`](/re ## What You Will Learn - How to choose between git-tag deploys and registry-artifact deploys -- How to provision a deploy credential once and reference it from CI, so your pipeline never handles the GitHub token +- How to supply a deploy credential from CI — and why it must be a durable token, not the workflow's ephemeral `GITHUB_TOKEN` - A GitHub Actions workflow that deploys a tagged release straight from a private repository - A GitHub Actions workflow that builds, publishes to GitHub Packages, and deploys the published artifact - How to deploy across a cluster and verify the deployment record @@ -42,7 +42,7 @@ A second constraint pushes the same direction: a git-reference deploy authentica | Needs a build step (bundling, codegen, compilation) | Private registry | | Has private git-hosted dependencies | Private registry | -## Provision the deploy credential once +## Create a durable deploy token Both paths need Harper to authenticate to GitHub — and it's worth understanding *when*. The credential is not only used at the moment you deploy: Harper persists it (encrypted, in the [secrets store](/reference/v5/security/secrets)) with the component's configuration so that later installs — a new node joining the cluster, a rollback, a re-install after restore — can fetch the package again without you re-supplying auth. @@ -51,7 +51,11 @@ That has one practical consequence: **use a durable token, not your workflow's e - **For a private repository (git-tag deploys):** a GitHub [fine-grained personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) with **Contents: read-only** permission, scoped to just the application repository. An organization can attach these to a machine account so they aren't tied to an individual. - **For GitHub Packages (registry deploys):** a token with the **`read:packages`** scope. -Store it in Harper once, granted to your component, using [`set_secret`](/reference/v5/operations-api/operations#set_secret): +Then hand it to Harper in one of two ways: + +**Pass it inline with the deploy (simplest — used in the workflows below).** Store the token as a repository secret in GitHub Actions and pass it in the `credentials` entry: `{ "host": "github.com", "token": "github_pat_..." }`. There is no separate provisioning step: Harper ingests the token into the encrypted secrets store on the way in (under a derived name like `deploy.my-app.git.github_com`, granted to the component), strips it from the operation before anything is logged or replicated, and reuses the stored copy for every later install. Re-deploying with a new token value rotates the stored one, so rotation is just updating the GitHub Actions secret. + +**Or provision it in Harper once, and reference it by name.** Create a secret with [`set_secret`](/reference/v5/operations-api/operations#set_secret) — or in [Harper Studio](/reference/v5/studio/overview), which provides a UI for creating, granting, and rotating secrets — granted to your component: ```json { @@ -62,17 +66,13 @@ Store it in Harper once, granted to your component, using [`set_secret`](/refere } ``` -From then on, CI references it by name — `{ "host": "github.com", "secret": "gh-deploy-token" }` — and the GitHub token never appears in your pipeline at all. The only secret your CI system holds is the Harper Operations API credential. This is the recommended setup: it separates the credential's lifecycle (rotate it with another `set_secret` call whenever you like — the deploy operation doesn't change) from the pipeline's. - -:::note Literal tokens -You can also pass the token inline: `{ "host": "github.com", "token": "github_pat_..." }`. Harper ingests it into the secrets store automatically (under a derived name like `deploy.my-app.git.github_com`) and the token is stripped from the operation before it is logged or replicated, so this is safe — re-deploying with a new token rotates the stored one. It just means your CI system holds the GitHub token too. Prefer the `secret` reference. -::: +CI then references it by name — `{ "host": "github.com", "secret": "gh-deploy-token" }` — and the GitHub token never appears in your pipeline at all; the only secret CI holds is the Harper Operations API credential. Choose this when you want the credential's lifecycle managed in Harper (or shared across pipelines) rather than living in each CI system's secrets. ## Path A: deploy a tagged release from a private repository The flow: push a semver tag (`v1.2.3`) → the workflow calls `deploy_component` with a `#semver:` git reference → every node clones that tag and installs. -Add two repository secrets to GitHub — `HARPER_OPS_URL` (your cluster's Operations API endpoint, e.g. `https://my-cluster.example.com:9925`) and `HARPER_OPS_AUTH` (a base64 `user:password` Basic credential or a bearer token for a `super_user`). Then: +Add three repository secrets to GitHub — `HARPER_OPS_URL` (your cluster's Operations API endpoint, e.g. `https://my-cluster.example.com:9925`), `HARPER_OPS_AUTH` (a base64 `user:password` Basic credential or a bearer token for a `super_user`), and `GH_DEPLOY_TOKEN` (the durable PAT from the previous section). Then: ```yaml # .github/workflows/deploy.yml @@ -95,7 +95,7 @@ jobs: "operation": "deploy_component", "project": "my-app", "package": "github:my-org/my-app#semver:${GITHUB_REF_NAME}", - "credentials": [{ "host": "github.com", "secret": "gh-deploy-token" }], + "credentials": [{ "host": "github.com", "token": "${GH_DEPLOY_TOKEN}" }], "replicated": true, "restart": "rolling" } @@ -103,6 +103,7 @@ jobs: env: HARPER_OPS_URL: ${{ secrets.HARPER_OPS_URL }} HARPER_OPS_AUTH: ${{ secrets.HARPER_OPS_AUTH }} + GH_DEPLOY_TOKEN: ${{ secrets.GH_DEPLOY_TOKEN }} ``` A few details worth noting: @@ -169,7 +170,7 @@ jobs: "project": "my-app", "package": "@my-org/my-app@${VERSION}", "credentials": [ - { "registry": "https://npm.pkg.github.com", "scope": "@my-org", "secret": "gh-packages-token" } + { "registry": "https://npm.pkg.github.com", "scope": "@my-org", "token": "${GH_PACKAGES_TOKEN}" } ], "replicated": true, "restart": "rolling" @@ -178,12 +179,13 @@ jobs: env: HARPER_OPS_URL: ${{ secrets.HARPER_OPS_URL }} HARPER_OPS_AUTH: ${{ secrets.HARPER_OPS_AUTH }} + GH_PACKAGES_TOKEN: ${{ secrets.GH_PACKAGES_TOKEN }} ``` Note the two tokens play different roles, matching their lifetimes: - **Publishing** uses the workflow's ephemeral `GITHUB_TOKEN` (with `packages: write`). Publishing happens entirely inside the job, so an ephemeral token is exactly right. -- **Installing** uses `gh-packages-token`, the durable `read:packages` secret you provisioned in Harper — because installs can happen long after this workflow ends. +- **Installing** uses `GH_PACKAGES_TOKEN`, the durable `read:packages` token — because installs can happen long after this workflow ends, and Harper stores this one for them. The `scope` field maps the `@my-org` scope to GitHub Packages on the installing node, so scoped dependencies from the same registry resolve too. @@ -203,7 +205,7 @@ A useful pattern is to have the workflow poll [`get_deployment`](/reference/v5/o ## Operational notes - **Rollback is just another deploy.** Both paths deploy immutable versions, so rolling back is re-running the deploy with the previous tag or version. This is a big part of why pinned references (`#semver:v1.2.3`, `@my-org/my-app@1.2.3`) beat branch references (`#main`) in a pipeline: `#main` moves, so you can neither audit nor re-deploy "what was running yesterday." -- **Rotating the deploy credential** is a `set_secret` call with the same name and new value — no change to the pipeline or the component. +- **Rotating the deploy credential**: with inline tokens, update the CI secret — the next deploy overwrites the stored copy. With a named secret, it's a `set_secret` call (or an edit in Harper Studio) with the same name and a new value; the pipeline doesn't change. - **Git-host credentials require git ≥ 2.31** on the Harper nodes, and are not supported on Windows nodes (the deploy fails with a clear error rather than degrading security). Registry credentials have no such constraints. Fabric instances satisfy both. - **Self-managed OSS core without the Pro secrets component:** `secret` references can't be resolved (no decryption custody), and a literal `token` is used transiently for that node's install only — it is not persisted for later installs. The durable-credential patterns in this guide assume custody (Fabric provides it). From bd847dc7c8e2ef040994a70201a1ce060472d544 Mon Sep 17 00:00:00 2001 From: Kris Zyp Date: Sat, 18 Jul 2026 11:43:31 -0600 Subject: [PATCH 8/9] docs: fix prettier formatting in deploy-from-CI guide and operations reference CI's Format Check was failing on markdown table alignment and italic-marker style in these two files. Co-Authored-By: Claude Sonnet 5 --- learn/developers/deploying-from-ci.mdx | 16 ++++++++-------- reference/operations-api/operations.md | 16 ++++++++-------- 2 files changed, 16 insertions(+), 16 deletions(-) diff --git a/learn/developers/deploying-from-ci.mdx b/learn/developers/deploying-from-ci.mdx index e61872657..053c856f2 100644 --- a/learn/developers/deploying-from-ci.mdx +++ b/learn/developers/deploying-from-ci.mdx @@ -27,26 +27,26 @@ Both work with private sources: `deploy_component` accepts a [`credentials`](/re ## Choosing a delivery model -**Deploy from a git tag when your application runs from source.** Most Harper applications do: a `config.yaml`, a `schema.graphql`, resource classes in JavaScript (or TypeScript that Harper runs directly via type stripping), and npm dependencies that install cleanly. For these, the repository *is* the artifact. A git-tag deploy has no publish step to maintain — you tag a release, CI calls `deploy_component` with a reference like `github:my-org/my-app#semver:v1.2.3`, and every node installs exactly that commit. Semver-tagged references also keep the deployed version auditable: the deployment record and the component config both name the tag. +**Deploy from a git tag when your application runs from source.** Most Harper applications do: a `config.yaml`, a `schema.graphql`, resource classes in JavaScript (or TypeScript that Harper runs directly via type stripping), and npm dependencies that install cleanly. For these, the repository _is_ the artifact. A git-tag deploy has no publish step to maintain — you tag a release, CI calls `deploy_component` with a reference like `github:my-org/my-app#semver:v1.2.3`, and every node installs exactly that commit. Semver-tagged references also keep the deployed version auditable: the deployment record and the component config both name the tag. -**Deploy from a registry when your application needs a build.** If CI must produce output that isn't in the repository — a bundling step, compiled assets, code generation, native compilation — publish the *built* package to a registry and deploy that. This is better than making Harper build it for two reasons: +**Deploy from a registry when your application needs a build.** If CI must produce output that isn't in the repository — a bundling step, compiled assets, code generation, native compilation — publish the _built_ package to a registry and deploy that. This is better than making Harper build it for two reasons: 1. **The artifact you tested is the artifact you run.** CI builds once, tests the result, publishes it. Every Harper node installs that identical, immutable artifact instead of re-running your build independently. -2. **No build scripts run on your database nodes.** When Harper installs a credentialed git reference, npm's pack step runs with `--ignore-scripts` by default, precisely so repository code can't run during install with the deploy credential in reach. A repository that *requires* a `prepare`/build script to be runnable forces you to set `install_allow_scripts: true`, which allows that script — and your dependencies' install scripts — to execute on the node during deployment. A registry artifact needs none of that: it's already built. +2. **No build scripts run on your database nodes.** When Harper installs a credentialed git reference, npm's pack step runs with `--ignore-scripts` by default, precisely so repository code can't run during install with the deploy credential in reach. A repository that _requires_ a `prepare`/build script to be runnable forces you to set `install_allow_scripts: true`, which allows that script — and your dependencies' install scripts — to execute on the node during deployment. A registry artifact needs none of that: it's already built. -A second constraint pushes the same direction: a git-reference deploy authenticates only the top-level repository. If your application has **private git-hosted dependencies** (a `package.json` dependency pointing at another private repo), their installation is not credentialed and will fail. Private *registry* dependencies work fine — registry credentials apply to the whole install. If your dependency graph reaches into private git repos, publish those dependencies (or the whole app) to a private registry instead. +A second constraint pushes the same direction: a git-reference deploy authenticates only the top-level repository. If your application has **private git-hosted dependencies** (a `package.json` dependency pointing at another private repo), their installation is not credentialed and will fail. Private _registry_ dependencies work fine — registry credentials apply to the whole install. If your dependency graph reaches into private git repos, publish those dependencies (or the whole app) to a private registry instead. | Your application... | Deploy from | | ------------------------------------------------------ | ---------------- | | Runs from source (schema, resources, npm dependencies) | Git tag | -| Needs a build step (bundling, codegen, compilation) | Private registry | -| Has private git-hosted dependencies | Private registry | +| Needs a build step (bundling, codegen, compilation) | Private registry | +| Has private git-hosted dependencies | Private registry | ## Create a durable deploy token -Both paths need Harper to authenticate to GitHub — and it's worth understanding *when*. The credential is not only used at the moment you deploy: Harper persists it (encrypted, in the [secrets store](/reference/v5/security/secrets)) with the component's configuration so that later installs — a new node joining the cluster, a rollback, a re-install after restore — can fetch the package again without you re-supplying auth. +Both paths need Harper to authenticate to GitHub — and it's worth understanding _when_. The credential is not only used at the moment you deploy: Harper persists it (encrypted, in the [secrets store](/reference/v5/security/secrets)) with the component's configuration so that later installs — a new node joining the cluster, a rollback, a re-install after restore — can fetch the package again without you re-supplying auth. -That has one practical consequence: **use a durable token, not your workflow's ephemeral `GITHUB_TOKEN`**. The automatic `GITHUB_TOKEN` in a GitHub Actions run expires when the job ends. It's fine for things that happen *inside* the job (like publishing a package), but if you hand it to Harper as the deploy credential, any install that happens after the workflow finishes fails authentication. Instead, create a token whose lifetime matches how long nodes may need to fetch this component: +That has one practical consequence: **use a durable token, not your workflow's ephemeral `GITHUB_TOKEN`**. The automatic `GITHUB_TOKEN` in a GitHub Actions run expires when the job ends. It's fine for things that happen _inside_ the job (like publishing a package), but if you hand it to Harper as the deploy credential, any install that happens after the workflow finishes fails authentication. Instead, create a token whose lifetime matches how long nodes may need to fetch this component: - **For a private repository (git-tag deploys):** a GitHub [fine-grained personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) with **Contents: read-only** permission, scoped to just the application repository. An organization can attach these to a machine account so they aren't tied to an individual. - **For GitHub Packages (registry deploys):** a token with the **`read:packages`** scope. diff --git a/reference/operations-api/operations.md b/reference/operations-api/operations.md index b09fb8506..034f6676b 100644 --- a/reference/operations-api/operations.md +++ b/reference/operations-api/operations.md @@ -617,14 +617,14 @@ When a component is installed from a private source, `credentials` supplies the An entry provides its credential exactly one of two ways — a literal `token`, or a `secret` reference: -| Field | Kind | Description | -| ---------- | -------- | ------------------------------------------------------------------------------------------------ | -| `registry` | npm | The registry URL or host the credential applies to. **Required** for an npm entry. | -| `scope` | npm | Optional npm `@scope` (e.g. `"@my-org"`) the entry applies to; omit to set the default registry. | -| `host` | git | The bare git host the credential applies to (e.g. `"github.com"`). **Required** for a git entry. | -| `username` | git | Optional git HTTPS username. Defaults to `x-access-token` (GitHub); GitLab uses `oauth2`, Bitbucket `x-token-auth`. | -| `token` | both | A literal auth token, **or** | -| `secret` | both | The name of an [`hdb_secret`](../security/secrets.md) row to resolve the token from. | +| Field | Kind | Description | +| ---------- | ---- | ------------------------------------------------------------------------------------------------------------------- | +| `registry` | npm | The registry URL or host the credential applies to. **Required** for an npm entry. | +| `scope` | npm | Optional npm `@scope` (e.g. `"@my-org"`) the entry applies to; omit to set the default registry. | +| `host` | git | The bare git host the credential applies to (e.g. `"github.com"`). **Required** for a git entry. | +| `username` | git | Optional git HTTPS username. Defaults to `x-access-token` (GitHub); GitLab uses `oauth2`, Bitbucket `x-token-auth`. | +| `token` | both | A literal auth token, **or** | +| `secret` | both | The name of an [`hdb_secret`](../security/secrets.md) row to resolve the token from. | A provided **`token`** is not treated as ephemeral: Harper ingests it into the encrypted [secrets store](../security/secrets.md) and references it everywhere, so package-reference deploys keep working through rollback, reboot, and new peers joining — without re-supplying the token. The token is encrypted at rest, stripped from the operation before replication and from the operations log, and only ever crosses the cluster as ciphertext. A git-host token is additionally served to git **from memory** (via a credential helper) — it is never written to a file or into a URL. Using a **`secret`** reference names an existing store row directly. Ingesting a token requires custody on the deploying node; on OSS core without custody, a literal token falls back to a transient, this-node-only credential (not persisted or replicated). From d73e79a0c7f61d32b2673699df435b295c04ac58 Mon Sep 17 00:00:00 2001 From: Kris Zyp Date: Mon, 20 Jul 2026 20:19:39 -0600 Subject: [PATCH 9/9] =?UTF-8?q?docs(learn):=20fix=20CI=20deploy=20guide=20?= =?UTF-8?q?review=20nits=20=E2=80=94=20auth=20header,=20secret=20name=20ty?= =?UTF-8?q?po?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - HARPER_OPS_AUTH now documented (and used in the curl examples) as the full Authorization header value, so a Bearer token works as well as Basic — the hardcoded "Authorization: Basic $HARPER_OPS_AUTH" would break for bearer-token users. - Fix derived secret name example: deploy.my-app.git.github_com -> deploy.my-app.git.github.com (deriveGitSecretName preserves dots). Addresses review feedback from gemini-code-assist and cb1kenobi on PR #594. The heredoc-EOF-indentation concern raised alongside the auth header comments does not apply here: verified against a YAML parser that the block-scalar's common-indentation stripping puts EOF at column 0 in the actual generated script, since it shares curl's base indent. Co-Authored-By: Claude Sonnet 5 --- learn/developers/deploying-from-ci.mdx | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/learn/developers/deploying-from-ci.mdx b/learn/developers/deploying-from-ci.mdx index 053c856f2..611852bd3 100644 --- a/learn/developers/deploying-from-ci.mdx +++ b/learn/developers/deploying-from-ci.mdx @@ -53,7 +53,7 @@ That has one practical consequence: **use a durable token, not your workflow's e Then hand it to Harper in one of two ways: -**Pass it inline with the deploy (simplest — used in the workflows below).** Store the token as a repository secret in GitHub Actions and pass it in the `credentials` entry: `{ "host": "github.com", "token": "github_pat_..." }`. There is no separate provisioning step: Harper ingests the token into the encrypted secrets store on the way in (under a derived name like `deploy.my-app.git.github_com`, granted to the component), strips it from the operation before anything is logged or replicated, and reuses the stored copy for every later install. Re-deploying with a new token value rotates the stored one, so rotation is just updating the GitHub Actions secret. +**Pass it inline with the deploy (simplest — used in the workflows below).** Store the token as a repository secret in GitHub Actions and pass it in the `credentials` entry: `{ "host": "github.com", "token": "github_pat_..." }`. There is no separate provisioning step: Harper ingests the token into the encrypted secrets store on the way in (under a derived name like `deploy.my-app.git.github.com`, granted to the component), strips it from the operation before anything is logged or replicated, and reuses the stored copy for every later install. Re-deploying with a new token value rotates the stored one, so rotation is just updating the GitHub Actions secret. **Or provision it in Harper once, and reference it by name.** Create a secret with [`set_secret`](/reference/v5/operations-api/operations#set_secret) — or in [Harper Studio](/reference/v5/studio/overview), which provides a UI for creating, granting, and rotating secrets — granted to your component: @@ -72,7 +72,7 @@ CI then references it by name — `{ "host": "github.com", "secret": "gh-deploy- The flow: push a semver tag (`v1.2.3`) → the workflow calls `deploy_component` with a `#semver:` git reference → every node clones that tag and installs. -Add three repository secrets to GitHub — `HARPER_OPS_URL` (your cluster's Operations API endpoint, e.g. `https://my-cluster.example.com:9925`), `HARPER_OPS_AUTH` (a base64 `user:password` Basic credential or a bearer token for a `super_user`), and `GH_DEPLOY_TOKEN` (the durable PAT from the previous section). Then: +Add three repository secrets to GitHub — `HARPER_OPS_URL` (your cluster's Operations API endpoint, e.g. `https://my-cluster.example.com:9925`), `HARPER_OPS_AUTH` (the full `Authorization` header value for a `super_user` — `Basic ` or `Bearer `), and `GH_DEPLOY_TOKEN` (the durable PAT from the previous section). Then: ```yaml # .github/workflows/deploy.yml @@ -89,7 +89,7 @@ jobs: run: | curl --fail-with-body -s "$HARPER_OPS_URL" \ -H "Content-Type: application/json" \ - -H "Authorization: Basic $HARPER_OPS_AUTH" \ + -H "Authorization: $HARPER_OPS_AUTH" \ -d @- <