Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -237,4 +237,18 @@ For significant releases, write an `ANNOUNCEMENT.md` for the GitHub Discussions
> [!IMPORTANT]
> Critical information users must know.
```
4. After copying content to GitHub Discussions, delete `ANNOUNCEMENT.md` from the repo
4. After copying content to GitHub Discussions, delete `ANNOUNCEMENT.md` from the repo

## Agent skills

### Issue tracker

Issues are tracked in **Linear** (team: Product Engineering, key `CIP-`), not GitHub Issues. GitHub holds code and PRs only. See `docs/agents/issue-tracker.md`.

### Triage labels

**Defaults** β€” the five canonical roles, each label string equal to its name. Only `wontfix` exists in Linear today; the rest are created on demand. See `docs/agents/triage-labels.md`.

### Domain docs

**Multi-context** β€” the root `CONTEXT-MAP.md` points at per-package `CONTEXT.md` files under `packages/*/`. See `docs/agents/domain.md`.
87 changes: 87 additions & 0 deletions CONTEXT-MAP.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# Context Map

CipherStash Proxy is a Cargo workspace. Each context below owns a distinct domain
vocabulary. Read the `CONTEXT.md` for the context you're working in before exploring
its code.

Per-context `CONTEXT.md` files are created lazily by `/domain-modeling` as terms get
resolved β€” a missing one is expected, not a gap to fill upfront.

| Context | Path | Domain |
|---|---|---|
| Proxy | [`packages/cipherstash-proxy/`](./packages/cipherstash-proxy/CONTEXT.md) | PostgreSQL wire protocol, connection and message handling, client authentication, TLS, ZeroKMS key management, encrypt/decrypt of column values |
| EQL Mapper | [`packages/eql-mapper/`](./packages/eql-mapper/CONTEXT.md) | SQL parsing, type inference over statements, schema analysis, transformation rules that rewrite plaintext SQL into EQL v2 operations |
| Integration | `packages/cipherstash-proxy-integration/` | End-to-end test harness β€” container fixtures, encrypted-scenario coverage across the proxy and mapper together |
| Showcase | `packages/showcase/` | Healthcare example data model demonstrating EQL v2 encryption with realistic relationships |

`packages/eql-mapper-macros/` is proc-macro support for EQL Mapper, not a context of its
own β€” treat it as part of the EQL Mapper context.

## Relationships

- **Proxy β†’ EQL Mapper**: Proxy loads the database schema and hands it over with each
column marked native or encrypted; EQL Mapper returns a type-checked, rewritten
statement. Proxy then reads the per-node EQL term shapes back out to decide how to
encrypt each value.
- **Identity across the seam**: EQL Mapper's `TableColumn` and Proxy's `Identifier` are
the same `table.column` pair under two names. That pair is the only key joining a typed
AST node to its encryption config.
- **Capability across the seam β€” currently broken.** Proxy marks every encrypted column
with *all* `EqlTrait`s (`packages/cipherstash-proxy/src/proxy/schema/manager.rs:146`)
because it derives them from the PostgreSQL column type alone. The column encrypt
config, which knows the SEM terms actually configured, is loaded by a separate manager
that never meets the schema loader. EQL Mapper's bound checking is therefore
unreachable in production: a query needing `Ord` on a column with no ordering SEM term
type-checks cleanly and fails later. Read `EqlTraits` on a column as *intended*
capability, not observed.

## Shared vocabulary

Terms defined once for the whole system live here rather than in any one context.

- **EQL v2** β€” Encrypt Query Language; the SQL-level encoding that makes encrypted
values searchable.
- **`eql_v2_encrypted`** β€” the PostgreSQL column type holding an encrypted value.
- **ZeroKMS** β€” CipherStash's key management service, which the proxy calls to encrypt
and decrypt.
- **Keyset** β€” the ZeroKMS key collection a workspace encrypts against.
- **SEM (searchable encrypted metadata)** β€” the encrypted material stored alongside a
value that makes some operation on it possible without decryption. A **SEM term** is
one such piece of metadata; ORE, unique, match and SteVec are SEM **types**.
- **EqlTrait** β€” a *capability* an encrypted column has: equality, ordering, token
match, JSON traversal, containment. Several SEM types can satisfy the same trait, so
the relationship is many-to-one: SEM terms are the storage, traits are what the storage
buys you.

Say **capability/trait** when you mean what a query may do, and **SEM term** when you
mean what is written to the column. Do not call either an **index** β€” that word is
reserved for a PostgreSQL index.

> The EQL config JSON and `cipherstash_client`'s `ColumnConfig` still spell SEM terms
> `indexes` / `IndexType`. That spelling is a wire and storage format shared with EQL
> and customer-authored config, so it is not ours to rename unilaterally β€” prefer SEM
> in prose and new code, and read `indexes` in those payloads as "SEM terms".

## Cross-context term collisions

Terms that mean different things depending on where you are. Resolve them against your
own context.

- **Type** β€” a PostgreSQL wire-protocol type in Proxy; an inferred SQL expression type
in EQL Mapper.
- **Index** β€” means a PostgreSQL index in *both* contexts (EQL Mapper's only use of the
word is GIN indexes). Where it appears meaning searchable encryption β€” the `indexes`
key in EQL config, `IndexType`, "Unknown Index Term" β€” read it as **SEM term** and see
the shared vocabulary above.
- **Column** β€” an encrypted column's runtime config in Proxy; a schema column or a
projection column in EQL Mapper. Also `DataColumn` (a wire value) and
`RowDescriptionField` (a result descriptor) in Proxy.
- **Statement** β€” a type-analysed statement in Proxy; the parsed AST or a
`TypeCheckedStatement` in EQL Mapper. Four senses are live in Proxy alone.
- **Term** β€” the *shape* of an encrypted payload (`EqlTerm`) in EQL Mapper; a piece of
searchable encrypted metadata in Proxy and EQL config. Qualify as *EQL term* or
*SEM term*.
- **Projection** β€” a type in EQL Mapper's lattice; in Proxy, a positional list where a
missing entry means "native, do not encrypt".
- **Session** β€” banned in Proxy (see its `CONTEXT.md`); PostgreSQL owns the word, and the
code has used it for both a connection and a single statement.
61 changes: 61 additions & 0 deletions docs/agents/domain.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Domain Docs

How the engineering skills should consume this repo's domain documentation when
exploring the codebase.

This is a **multi-context** repo β€” a Cargo workspace whose packages carry distinct
domain vocabularies.

## Before exploring, read these

- **`CONTEXT-MAP.md`** at the repo root β€” it points at one `CONTEXT.md` per context.
Read each one relevant to the topic.
- **`docs/adr/`** β€” system-wide architectural decisions.
- **`packages/<name>/docs/adr/`** β€” context-scoped decisions for that package.

If any of these files don't exist, **proceed silently**. Don't flag their absence;
don't suggest creating them upfront. The `/domain-modeling` skill (reached via
`/grill-with-docs` and `/improve-codebase-architecture`) creates them lazily when terms
or decisions actually get resolved.

Note that `ARCHITECTURE.md` at the repo root already describes system structure. It is
not a glossary β€” read it for orientation, but domain *terms* belong in `CONTEXT.md`.

## File structure

This repo is a Cargo workspace, so contexts live under `packages/`, not `src/`:

```text
/
β”œβ”€β”€ CONTEXT-MAP.md
β”œβ”€β”€ docs/adr/ ← system-wide decisions
└── packages/
β”œβ”€β”€ cipherstash-proxy/
β”‚ β”œβ”€β”€ CONTEXT.md
β”‚ └── docs/adr/ ← context-specific decisions
└── eql-mapper/
β”œβ”€β”€ CONTEXT.md
└── docs/adr/
```

## Use the glossary's vocabulary

When your output names a domain concept (in an issue title, a refactor proposal, a
hypothesis, a test name), use the term as defined in the relevant `CONTEXT.md`. Don't
drift to synonyms the glossary explicitly avoids.

If the concept you need isn't in the glossary yet, that's a signal β€” either you're
inventing language the project doesn't use (reconsider) or there's a real gap (note it
for `/domain-modeling`).

Watch for terms that mean different things in different contexts. "Type", for example,
means a PostgreSQL wire type in `cipherstash-proxy` and an inferred SQL type in
`eql-mapper`. That divergence is exactly why this repo is multi-context β€” resolve the
term against the context you're working in, not the other one.

## Flag ADR conflicts

If your output contradicts an existing ADR, surface it explicitly rather than silently
overriding:

> _Contradicts ADR-000N (its title) β€” but worth reopening because…_
73 changes: 73 additions & 0 deletions docs/agents/issue-tracker.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# Issue Tracker

Issues for this repo live in **Linear**, not GitHub Issues.

- **Workspace:** CipherStash
- **Team:** Product Engineering
- **Issue key prefix:** `CIP-` (e.g. `CIP-3233`)

GitHub (`cipherstash/proxy`) holds code, branches, and pull requests. It is not the
issue tracker. Do not run `gh issue create` for this repo.

## How to read and write issues

Two access paths exist. **Prefer the CLI for writes.**

### `linear-cli` (preferred)

The `linear` command (`@schpet/linear-cli`, or `npx @schpet/linear-cli`) β€” see the
`linear-cli` skill for full usage.

```bash
linear issue list
linear issue view CIP-3233
linear issue create --title "..." --description-file /path/to/desc.md
linear issue update CIP-3233 --status "In Progress"
```

Always pass markdown via `--description-file` / `--body-file` rather than inline
arguments β€” inline content produces literal `\n` sequences in the Linear UI.

### Linear MCP tools (fallback, and for querying)

`mcp__claude_ai_Linear__*` β€” good for search and read-heavy work
(`list_issues`, `get_issue`, `save_issue`). Note these are
interactively authenticated and **may be unavailable in headless or cron runs**, so
anything scripted should prefer the CLI.

## Statuses

The Product Engineering workflow:

| Status | Type |
|---|---|
| `Backlog` | backlog |
| `Todo` | unstarted |
| `In Progress` | started |
| `In Review` | started |
| `Done` | completed |
| `Canceled` | canceled |
| `Duplicate` | duplicate |

## How the skills should use this

- **`to-tickets`** β€” create one Linear issue per tracer-bullet ticket in team
Product Engineering, starting in `Todo`. Express blocking edges as **native Linear
blocked-by relations**, not as prose in the description and not as files under
`.scratch/`. Any issue whose blockers are `Done` is grabbable.
- **`implement`** β€” take an issue by its `CIP-####` key. Move it to `In Progress` when
work starts, and `In Review` when the PR opens. Leave the move to `Done` to the
human merging.
- **`to-spec`** β€” link the resulting spec back to the originating issue as a comment or
attachment on the Linear issue.
- **`code-review`** β€” the "Spec" axis reads the originating `CIP-####` issue for intent.

## Linking PRs to issues

Reference the issue key in the PR title or body (e.g. `CIP-3233`) so Linear's GitHub
integration attaches the PR to the issue automatically.

## PRs as a request surface

**Off.** External pull requests are not treated as incoming triage requests for this
repo. Flip this flag if that changes.
23 changes: 23 additions & 0 deletions docs/agents/triage-labels.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Triage Labels

The skills speak in terms of five canonical triage roles. This file maps those roles to
the actual label strings used in this repo's issue tracker (Linear, team Product
Engineering β€” see [issue-tracker.md](./issue-tracker.md)).

| Label in mattpocock/skills | Label in our tracker | Meaning |
| -------------------------- | -------------------- | ---------------------------------------- |
| `needs-triage` | `needs-triage` | Maintainer needs to evaluate this issue |
| `needs-info` | `needs-info` | Waiting on reporter for more information |
| `ready-for-agent` | `ready-for-agent` | Fully specified, ready for an AFK agent |
| `ready-for-human` | `ready-for-human` | Requires human implementation |
| `wontfix` | `wontfix` | Will not be actioned |

When a skill mentions a role (e.g. "apply the AFK-ready triage label"), use the
corresponding label string from this table.

## Creating the labels

Only `wontfix` currently exists on the Product Engineering team. The other four are
created lazily β€” the first time `triage` needs one, create it on the team rather than
failing. These are workspace-level Linear labels, unparented; do not nest them under the
existing `Product` / `R&D` / `Issue type` groups.
115 changes: 115 additions & 0 deletions packages/cipherstash-proxy/CONTEXT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
# Proxy

Speaks the PostgreSQL wire protocol to client applications, hands their SQL to EQL Mapper
for type checking and rewriting, encrypts and decrypts column values through ZeroKMS, and
forwards everything to the real database. It owns everything about *connections* and
*ciphertext*; it owns nothing about type inference.

## Language

### Connections and statements

**Connection**:
One client application's TCP connection to Proxy, from startup to termination. All
per-client state hangs off it.
_Avoid_: session β€” PostgreSQL already owns that word for something else, and this
codebase has used it for both a connection and a single statement.

**Context**:
The mutable state belonging to one connection β€” its prepared statements, portals,
pending describes and executes, and keyset.

**Frontend** / **Backend**:
The two halves of the proxy pipeline by direction of travel: Frontend carries client to
database, Backend carries database to client. These are *not* the PostgreSQL protocol's
frontend/backend roles, and Backend is not the database.

**Statement**:
A type-analysed SQL statement β€” its param columns, projection columns, literal columns
and PostgreSQL param type OIDs. The parsed AST and EQL Mapper's `TypeCheckedStatement`
are different things; do not call either of them a Statement here.

**Portal**:
A prepared statement bound to parameter values, ready to execute. Either `Encrypted`,
carrying the analysed statement, or `Passthrough` when nothing in it is encrypted.

**Statement metrics scope**:
The measurement window around a single statement β€” opened at Parse or Query, closed when
the statement completes. Many of these occur per connection.
_Avoid_: session (`start_session`, `SessionId` and the
`..._statements_session_duration_seconds` metric all use this sense and are misnamed).

**Passthrough**:
Traffic forwarded without encryption or rewriting. Three independent conditions produce
it and they are not interchangeable: no encrypt config is loaded, mapping is disabled by
configuration, or `SET CIPHERSTASH.UNSAFE_DISABLE_MAPPING` was issued on this connection.
Name the condition rather than saying "passthrough" alone.

### Encryption

**Column**:
An encrypted column as this crate sees it β€” its `Identifier`, its column config, its
resolved PostgreSQL type, and the EQL term shape it takes in the current statement.
_Avoid_: confusing with EQL Mapper's schema/projection columns, or with `DataColumn`
(a value on the wire) and `RowDescriptionField` (a result descriptor).

**Identifier**:
The `table.column` pair that keys the encrypt config. EQL Mapper calls the same thing a
`TableColumn`.

**Encrypt credentials**:
The CipherStash client credentials Proxy authenticates to ZeroKMS with β€” client id,
client key, default keyset. Read from `[encrypt]` / `CS_ENCRYPT__*`.
_Avoid_: `EncryptConfig`, which currently names this *and* an unrelated struct.

**Column encrypt config**:
The per-column encryption configuration β€” which columns are encrypted and with which SEM
terms β€” read from the EQL config table in the *database* at runtime, and reloaded when
DDL is observed.
_Avoid_: `EncryptConfig`; also "encrypt schema", despite `ReloadCommand::EncryptSchema`
naming the reload of this thing.

**Client id**:
Ambiguous β€” always qualify. The **connection number** is the incrementing counter stamped
on log lines to correlate one connection's traffic. The **CipherStash client id** is the
credential UUID from `CS_ENCRYPT__CLIENT_ID`. They share a name and a field spelling and
are otherwise unrelated.

**Keyset**:
The ZeroKMS collection of keys a workspace encrypts against. Addressed by UUID or by
name, and selectable per connection via `SET CIPHERSTASH.KEYSET_ID` / `KEYSET_NAME`.

**Scoped cipher**:
A cipher bound to exactly one keyset. Cached per keyset, so switching keyset switches
cipher.

**Plaintext**:
A value in its typed, pre-encryption form. The boundary type between PostgreSQL wire
values and ZeroKMS.

**EQL ciphertext**:
An encrypted value in the shape stored in the database. What encryption produces and
decryption consumes.

**Store** / **Query** operation:
The two directions of encryption. Storing writes a value with all its SEM terms; querying
produces only the term needed to match. The statement's EQL term shape decides which.

### Control plane

**`SET CIPHERSTASH.*`**:
Proxy's in-band control API, intercepted rather than forwarded β€” `KEYSET_ID`,
`KEYSET_NAME`, `UNSAFE_DISABLE_MAPPING`. These are the supported spellings; log messages
that print `CIPHERSTASH.DISABLE_MAPPING` are wrong.

**Reload**:
Re-reading state from the database after observed DDL. Two independent things reload: the
database schema, and the column encrypt config.

## Note on `session`

This glossary bans `session` because the code uses it for two different things and
PostgreSQL uses it for a third. The identifiers `SessionId`, `start_session`,
`finish_session` and the Prometheus metric name all still carry it. Read them as
*statement metrics scope*; the doc comments on `Frontend` and `Backend` that say "session
context" mean *connection*.
Loading
Loading