Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
482 changes: 163 additions & 319 deletions Cargo.lock

Large diffs are not rendered by default.

21 changes: 7 additions & 14 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
[workspace]
resolver = "2"
members = ["packages/*"]
# Vendored crate is consumed only via [patch.crates-io] below, not as a member.
exclude = ["vendor/stack-auth"]

[workspace.package]
version = "2.2.4"
Expand Down Expand Up @@ -45,9 +43,9 @@ debug = true

[workspace.dependencies]
sqltk = { version = "0.10.0" }
cipherstash-client = { version = "=0.34.1-alpha.4" }
cipherstash-config = { version = "=0.34.1-alpha.4" }
cts-common = { version = "=0.34.1-alpha.4" }
cipherstash-client = { version = "=0.42.0" }
cipherstash-config = { version = "=0.42.0" }
cts-common = { version = "=0.42.0" }

thiserror = "2.0.9"
tokio = { version = "1.44.2", features = ["full"] }
Expand All @@ -59,12 +57,7 @@ tracing-subscriber = { version = "^0.3.20", features = [
"std",
] }

# HOTFIX (CIP-3159): backport the stack-auth token-refresh CancelGuard fix onto
# the 0.34.1-alpha.4 source that cipherstash-client 0.34.1-alpha.4 pins. Without
# this, a cancelled get_token() future could strand `refresh_in_progress = true`,
# wedging all later refreshes and causing ZeroKMS "Request not authorized" exactly
# ~15 min (token TTL) after startup. The patch keeps version 0.34.1-alpha.4 so it
# satisfies cipherstash-client's exact pin while replacing the registry source.
# Remove once Proxy moves to a cipherstash-client built against stack-auth >= 0.36.0.
[patch.crates-io]
stack-auth = { path = "vendor/stack-auth" }
# The CIP-3159 stack-auth hotfix patch was removed here: cipherstash-client 0.42.0
# requires stack-auth ^0.42.0, which carries the CancelGuard token-refresh fix
# upstream (landed in stack-auth 0.36.0). The vendored copy under vendor/stack-auth
# is no longer referenced and has been removed.
67 changes: 67 additions & 0 deletions docs/errors.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,12 @@
- [Column configuration mismatch](#encrypt-column-config-mismatch)
- [Missing encrypt configuration](#encrypt-missing-encrypt-configuration)
- [Unexpected SET keyset](#encrypt-unexpected-set-keyset)
- [Encrypted jsonb column configured for ORE ordering](#encrypt-ste-vec-ore-mode-unsupported)

- Decrypt errors:
- [Column could not be deserialised](#encrypt-column-could-not-be-deserialised)
- [Encrypted jsonb value has no root entry](#encrypt-ste-vec-missing-root-entry)
- [Encrypted jsonb entry has an invalid selector](#encrypt-ste-vec-selector-invalid)

- Configuration errors:
- [Missing or invalid TLS configuration](#config-missing-or-invalid-tls)
Expand Down Expand Up @@ -621,6 +624,29 @@ Cannot SET CIPHERSTASH.KEYSET if a default keyset has been configured.
<!-- ---------------------------------------------------------------------------------------------------- -->


## Encrypted jsonb column configured for ORE ordering <a id='encrypt-ste-vec-ore-mode-unsupported'></a>

An encrypted `jsonb` (SteVec) column is configured for Standard-mode ORE ordering, which EQL v3 does not support.

EQL v3 orders encrypted `jsonb` entries by the CLLW-OPE (`op`) term and has no representation for CLLW-ORE (`oc`). A column carried over from an earlier configuration that used ORE ordering therefore cannot be encrypted under EQL v3.


### Error message

```
An encrypted jsonb column is configured for ORE ordering, which EQL v3 does not support.
```


### How to fix

1. Reconfigure the column to use a supported ordering mode.
2. Re-encrypt the column's data under the new configuration.


<!-- ---------------------------------------------------------------------------------------------------- -->


# Decrypt errors


Expand Down Expand Up @@ -657,6 +683,47 @@ If the error persists, please contact CipherStash [support](https://cipherstash.
<!-- ---------------------------------------------------------------------------------------------------- -->


## Encrypted jsonb value has no root entry <a id='encrypt-ste-vec-missing-root-entry'></a>

An encrypted `jsonb` (SteVec) value has an empty `sv` array and cannot be decrypted.

The first entry of a SteVec document (`sv[0]`) is its decryption root. A document with no entries has nothing to decrypt, which indicates the stored value has been truncated or altered by another process.


### Error message

```
Encrypted jsonb value has no root entry and cannot be decrypted.
```


### How to fix

1. Check that the data in the encrypted column has not been modified outside CipherStash Proxy.
2. If the error persists, please contact CipherStash [support](https://cipherstash.com/support).


<!-- ---------------------------------------------------------------------------------------------------- -->


## Encrypted jsonb entry has an invalid selector <a id='encrypt-ste-vec-selector-invalid'></a>

An encrypted `jsonb` (SteVec) entry has a selector that is not exactly 16 hex-encoded bytes.

A SteVec entry's selector is the source of both AEAD bindings (nonce and AAD), so it must be exactly 16 hex-encoded bytes. A selector of any other length indicates the stored value has been altered by another process.


### Error message

```
Encrypted jsonb entry has an invalid selector '{selector}'.
```


### How to fix

1. Check that the data in the encrypted column has not been modified outside CipherStash Proxy.
2. If the error persists, please contact CipherStash [support](https://cipherstash.com/support).


<!-- ---------------------------------------------------------------------------------------------------- -->
Expand Down
2 changes: 1 addition & 1 deletion mise.local.example.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ CS_CLIENT_KEY = "client-key"
CS_CLIENT_ID = "client-id"

# The release of EQL that the proxy tests will use and releases will be built with
CS_EQL_VERSION = "eql-2.3.0-pre.3"
CS_EQL_VERSION = "eql-3.0.2"

# TLS variables are required for providing TLS to Proxy's clients.
# CS_TLS__TYPE can be either "Path" or "Pem" (case-sensitive).
Expand Down
2 changes: 1 addition & 1 deletion mise.toml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ CS_PROXY__HOST = "host.docker.internal"
# Misc
DOCKER_CLI_HINTS = "false" # Please don't show us What's Next.

CS_EQL_VERSION = "eql-2.3.0-pre.3"
CS_EQL_VERSION = "eql-3.0.2"


[tools]
Expand Down
41 changes: 38 additions & 3 deletions packages/cipherstash-proxy/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,23 @@ pub enum EncryptError {
#[error("InvalidIndexTerm")]
InvalidIndexTerm,

/// EQL v3 orders encrypted jsonb entries by the CLLW-OPE (`op`) term and has
/// no representation for CLLW-ORE (`oc`), so a column configured for
/// Standard-mode ste_vec cannot be encrypted. The column has to be
/// reconfigured and its data re-encrypted.
#[error("An encrypted jsonb column is configured for ORE ordering, which EQL v3 does not support. For help visit {}#encrypt-ste-vec-ore-mode-unsupported", ERROR_DOC_BASE_URL)]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Entry needs to be added to errors.md

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done β€” added an Encrypted jsonb column configured for ORE ordering section under Encrypt errors with the encrypt-ste-vec-ore-mode-unsupported anchor (it was referenced here but the anchor didn't exist, so this was a dangling link).

SteVecOreModeUnsupported,

/// `sv[0]` is the decryption root of a SteVec document, so an empty `sv`
/// array leaves nothing to decrypt.
#[error("Encrypted jsonb value has no root entry and cannot be decrypted. For help visit {}#encrypt-ste-vec-missing-root-entry", ERROR_DOC_BASE_URL)]
SteVecMissingRootEntry,

/// A SteVec entry's selector is the source of both AEAD bindings (nonce and
/// AAD), so it must be exactly 16 hex-encoded bytes.
#[error("Encrypted jsonb entry has an invalid selector '{selector}'. For help visit {}#encrypt-ste-vec-selector-invalid", ERROR_DOC_BASE_URL)]
SteVecSelectorInvalid { selector: String },

#[error(
"KeysetId `{id}` could not be parsed using `SET CIPHERSTASH.KEYSET_ID`. KeysetId should be a valid UUID. For help visit {}#encrypt-keyset-id-could-not-be-parsed",
ERROR_DOC_BASE_URL
Expand Down Expand Up @@ -340,10 +357,28 @@ impl From<cipherstash_client::eql::EqlError> for EncryptError {
cipherstash_client::eql::EqlError::ColumnConfigurationMismatch { table, column } => {
Self::ColumnConfigurationMismatch { table, column }
}
cipherstash_client::eql::EqlError::CouldNotDecryptDataForKeyset { keyset_id } => {
Self::CouldNotDecryptDataForKeyset { keyset_id }
}
// cipherstash-client 0.42.0 added a `#[source]` zerokms::Error here
// so callers can walk the chain to the underlying RetrieveKeyError.
// Proxy's variant carries only the keyset id, so the chain stops at
// this boundary β€” worth threading through if a keyset decrypt
// failure ever needs diagnosing from Proxy's logs alone.
cipherstash_client::eql::EqlError::CouldNotDecryptDataForKeyset {
keyset_id, ..
} => Self::CouldNotDecryptDataForKeyset { keyset_id },
cipherstash_client::eql::EqlError::InvalidIndexTerm => Self::InvalidIndexTerm,

// EQL v3 orders jsonb entries by the byte-comparable CLLW-OPE `op`
// term and has no CLLW-ORE (`oc`) representation, so a SteVec column
// still configured in Standard mode cannot be written at all.
cipherstash_client::eql::EqlError::UnsupportedSteVecOreInV3 => {
Self::SteVecOreModeUnsupported
}

cipherstash_client::eql::EqlError::UnsupportedV3QueryTerm => Self::InvalidIndexTerm,

// Only reachable by asking the client for a v2 payload, which Proxy
// never does β€” v3 is the only envelope it speaks.
cipherstash_client::eql::EqlError::UnsupportedSteVecInV2 => Self::InvalidIndexTerm,
cipherstash_client::eql::EqlError::MissingCiphertext(identifier) => {
Self::ColumnCouldNotBeDeserialised {
table: identifier.table,
Expand Down
11 changes: 9 additions & 2 deletions packages/cipherstash-proxy/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,20 @@ pub use crate::config::{DatabaseConfig, ServerConfig, TandemConfig, TlsConfig};
pub use crate::log::init;
pub use crate::proxy::Proxy;
pub use cipherstash_client::encryption::Plaintext;
pub use cipherstash_client::eql::{EqlCiphertext, Identifier};
// EQL v3 is the only wire envelope Proxy speaks. The v2 types
// (`EqlCiphertext`, `EqlOutput`, `EqlQueryPayload`) are deliberately not
// re-exported β€” v2 support is retired, so anything still reaching for them
// should fail to resolve rather than silently keep writing v2 payloads.
pub use cipherstash_client::eql::{
EqlCiphertextV3 as EqlCiphertext, EqlOutputV3 as EqlOutput,
EqlQueryPayloadV3 as EqlQueryPayload, Identifier,
};

use std::mem;

pub const VERSION: &str = env!("CARGO_PKG_VERSION");

pub const EQL_SCHEMA_VERSION: u16 = 2;
pub const EQL_SCHEMA_VERSION: u16 = 3;

pub const SIZE_U8: usize = mem::size_of::<u8>();
pub const SIZE_I16: usize = mem::size_of::<i16>();
Expand Down
10 changes: 5 additions & 5 deletions packages/cipherstash-proxy/src/postgresql/backend.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ use crate::prometheus::{
ROWS_PASSTHROUGH_TOTAL, ROWS_TOTAL, SERVER_BYTES_RECEIVED_TOTAL,
};
use crate::proxy::EncryptionService;
use crate::EqlCiphertext;
use bytes::BytesMut;
use cipherstash_client::eql::EqlCiphertext;
use metrics::{counter, histogram};
use std::time::Instant;
use tokio::io::AsyncRead;
Expand Down Expand Up @@ -538,7 +538,7 @@ where
for (col, ct) in projection_columns.iter().zip(ciphertexts) {
match (col, ct) {
(Some(col), Some(ct)) => {
if col.identifier != ct.identifier {
if &col.identifier != ct.identifier() {
return Err(EncryptError::ColumnConfigurationMismatch {
table: col.identifier.table.to_owned(),
column: col.identifier.column.to_owned(),
Expand All @@ -553,8 +553,8 @@ where
// ciphertext with no column configuration is bad
(None, Some(ct)) => {
return Err(EncryptError::ColumnConfigurationMismatch {
table: ct.identifier.table.to_owned(),
column: ct.identifier.column.to_owned(),
table: ct.identifier().table.to_owned(),
column: ct.identifier().column.to_owned(),
}
.into());
}
Expand Down Expand Up @@ -749,7 +749,7 @@ mod tests {
_keyset_id: Option<KeysetIdentifier>,
_plaintexts: Vec<Option<cipherstash_client::encryption::Plaintext>>,
_columns: &[Option<Column>],
) -> Result<Vec<Option<crate::EqlCiphertext>>, Error> {
) -> Result<Vec<Option<crate::EqlOutput>>, Error> {
Ok(vec![])
}

Expand Down
4 changes: 2 additions & 2 deletions packages/cipherstash-proxy/src/postgresql/context/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -752,7 +752,7 @@ where
&self,
plaintexts: Vec<Option<cipherstash_client::encryption::Plaintext>>,
columns: &[Option<Column>],
) -> Result<Vec<Option<crate::EqlCiphertext>>, Error> {
) -> Result<Vec<Option<crate::EqlOutput>>, Error> {
let keyset_id = self.keyset_identifier();

self.encryption
Expand Down Expand Up @@ -1077,7 +1077,7 @@ mod tests {
_keyset_id: Option<KeysetIdentifier>,
_plaintexts: Vec<Option<cipherstash_client::encryption::Plaintext>>,
_columns: &[Option<Column>],
) -> Result<Vec<Option<crate::EqlCiphertext>>, Error> {
) -> Result<Vec<Option<crate::EqlOutput>>, Error> {
Ok(vec![])
}

Expand Down
10 changes: 5 additions & 5 deletions packages/cipherstash-proxy/src/postgresql/frontend.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ use crate::prometheus::{
STATEMENTS_PASSTHROUGH_TOTAL, STATEMENTS_UNMAPPABLE_TOTAL,
};
use crate::proxy::EncryptionService;
use crate::EqlCiphertext;
use crate::EqlOutput;
use bytes::BytesMut;
use cipherstash_client::encryption::Plaintext;
use eql_mapper::{self, EqlMapperError, EqlTerm, TypeCheckedStatement};
Expand Down Expand Up @@ -582,13 +582,13 @@ where
/// # Returns
///
/// Vector of encrypted values corresponding to each literal, with `None` for
/// literals that don't require encryption and `Some(EqlCiphertext)` for encrypted values.
/// literals that don't require encryption and `Some(EqlOutput)` for encrypted values.
async fn encrypt_literals(
&mut self,
session_id: SessionId,
typed_statement: &TypeCheckedStatement<'_>,
literal_columns: &Vec<Option<Column>>,
) -> Result<Vec<Option<EqlCiphertext>>, Error> {
) -> Result<Vec<Option<EqlOutput>>, Error> {
let literal_values = typed_statement.literal_values();
if literal_values.is_empty() {
debug!(target: MAPPER,
Expand Down Expand Up @@ -643,7 +643,7 @@ where
async fn transform_statement(
&mut self,
typed_statement: &TypeCheckedStatement<'_>,
encrypted_literals: &Vec<Option<EqlCiphertext>>,
encrypted_literals: &Vec<Option<EqlOutput>>,
) -> Result<Option<ast::Statement>, Error> {
// Convert literals to ast Expr
let mut encrypted_expressions = vec![];
Expand Down Expand Up @@ -1042,7 +1042,7 @@ where
session_id: Option<SessionId>,
bind: &Bind,
statement: &Statement,
) -> Result<Vec<Option<crate::EqlCiphertext>>, Error> {
) -> Result<Vec<Option<crate::EqlOutput>>, Error> {
let plaintexts =
bind.to_plaintext(&statement.param_columns, &statement.postgres_param_types)?;

Expand Down
4 changes: 2 additions & 2 deletions packages/cipherstash-proxy/src/postgresql/messages/bind.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ use crate::postgresql::context::column::Column;
use crate::postgresql::data::bind_param_from_sql;
use crate::postgresql::format_code::FormatCode;
use crate::postgresql::protocol::BytesMutReadString;
use crate::EqlOutput;
use crate::{SIZE_I16, SIZE_I32};
use bytes::{Buf, BufMut, BytesMut};
use cipherstash_client::encryption::Plaintext;
use cipherstash_client::eql::EqlCiphertext;
use postgres_types::Type;
use std::fmt::{self, Display, Formatter};
use std::io::Cursor;
Expand Down Expand Up @@ -81,7 +81,7 @@ impl Bind {
Ok(plaintexts)
}

pub fn rewrite(&mut self, encrypted: Vec<Option<EqlCiphertext>>) -> Result<(), Error> {
pub fn rewrite(&mut self, encrypted: Vec<Option<EqlOutput>>) -> Result<(), Error> {
for (idx, ct) in encrypted.iter().enumerate() {
if let Some(ct) = ct {
let json = serde_json::to_value(ct)?;
Expand Down
Loading
Loading